I spent 13 years designing electronics of this exact nature: three phase induction motor reduced voltage soft starters and variable frequency AC drives. I spent the last few of those years as a VFD applications engineer helping customers select and configure this equipment for various loads and industries around the country as well.
You will not be able to build something that is cheap and safe. The voltages and currents involved are well beyond the safety margin of a hobbyiest, especially someone who is openly avoiding buying commercial units in order to save money. Don't do it!
While the theory behind AC motor control is very straightforward, the detail level work (heat sink sizing, snubbering, gate drive requirements, de-sat protection, motor overload calculations, bus capacitor protection, etc.) can be quite tricky to get down, especially with heavy duty cycling and regenerative power modes which a carnival ride will CERTAINLY be generating. I strongly caution you against trying to build something of this nature unless you have significant experience not only in microcontrollers and embedded systems design but also significant experience in power electronics and three phase circuitry. People get hurt and killed building this stuff.
My first question for you is whether speed control is really required, or if you only require a soft start up and slow down. Do you vary the speed of the motor once it is started? If not, you may be able to get away with a MUCH cheaper reduced voltage soft starter. These units act like three phase light dimmers; they only adjust the applied voltage to the motor. You will not have a lot of torque at low speeds, but with the right design of motor (NEMA class D) you can achieve exactly what you're after with a fraction of the cost and maintenance.
If you really do need to vary the full-load speed of the motor then you are more or less stuck using a variable frequency drive. As you are aware these are expensive and if you buy cheap you are likely to replace them sooner due to your high surge current (they call this "constant torque") application. What I would definitely recommend doing if this is the case would be to contact various manufacturers (Allen-Bradley, Cutler-Hammer, SAF drives, Benshaw, Yaskawa, etc.) and ask for reconditioned units. Ask for a drive capable of delivering 150% rated current for 30s (this is usually known as heavy duty) or size the drive 30-50% larger than your nominal current rating. You will also likely be running off of generator power which is notorious for being undersized and prone to brownouts and surges as the load requirements change with the state of the equipment being run. Drives don't like that (voltage sags cause current spikes as the motor starts slipping and surges can cause you to overvoltage the bus capacitors) and have a tendency to either fault out or blow up.
I am all about the little guy building something and saving a buck, but this is not the type of project to do this on. If you really want to build a three phase AC drive, start with a little 10HP 480V motor with a hand brake on a test bench. You have all the potential for experiencing the pants-filling sensation of an H-bridge failure or a bus capacitor explosion two feet from your head but without the potential lawsuits and loss of life (except perhaps your own).
It's absolutely common to derate three-phase inverters for single-phase source operation. The front end (diode bridge) is under more stress because all the current now only comes from 2/3 of the diodes. The DC bus capacitor also takes more of a beating due to higher ripple current since you're missing a phase that helps keep the bus charged. When I worked in this industry we usually derated the drive 33%, possibly more if it was for a constant torque application.
There are usually line reactors or a DC choke as well, as you've mentioned, but I don't think I've ever heard of rewiring or trying to use the now-unused third phase coil for anything. Even if you were to try to put it in series with one of the other phases you would introduce imbalances in how the inverter is taking power from the line.
I guess the bigger question is what you're trying to achieve by using this unused third coil? I don't think anything useful can come from it, and in fact if you're going to go to all the trouble to rewind it for marginal gain you might just be better off using a DC link choke instead of the line reactor.
Best Answer
If you have a single-phase facility, you can't just plug in a piece of 3-phase equipment. You will need to find a way to adapt it for single-phase power. That will probably not be particularly easy and might not be possible at all. It could be done with no loss in efficiency and performance, but is more likely to be less efficient and not perform as well.
1A. If it can be adapted, that can be done safely.
2A. The efficiency reduction will probably be in the range of 0 to 15%.
3A. It is not a matter of adjustment, it a matter of changing the design in some way.
4A. The current increase will probably be in the range of 0 to 15%.
It is relatively easy to use single-phase equipment in a 3-phase facility. Three-phase facilities usually have some single-phase equipment connected. The only concern is that you should try to keep the overall usage balanced among the phases.
1B. Safety is not an issue.
2B The single phase equipment may be less efficient than equivalent 3-phase equipment by as much as 15%.
3B No adjustments will be required.
4B The current increase will be somewhat comparable to the efficiency decrease.
The basic ground rule and method for determining cost effectiveness is to list all the anticipated equipment required and get prices and specifications from the suppliers. Also get information from the utility that will supply the power and from installers.
Edit: You can not really make a good decision with out learning more specifics. The more specific information you put int your planning, the more likely you can make a good decision.