"The upper part of the figure shows a simple inverter configuration in which a bidirectional switch is used to alternate the voltage supply between the two halves of the primary of the transformer. The secondary side of the transformer outputs a voltage square waveform whose amplitude is given by the transformation ratio of the transformer and whose frequency is given by the switching time of the switch.
The lower part of the figure shows an inverter in which the transformer is used also to command the two BJT's. When the upper BJT is in saturation, the transformer primary is subjected to a DC voltage. This voltage is mirrored into the two auxiliary windings which drives the bases of the BJT's. While approaching magnetic saturation, the voltage in these two auxiliary windings falls down and so the upper BJT turns off. Since in the transformer core the H field is not zero, a current must feed the transformer. This causes a rapid voltage inversion across all the transformer windings, which turns on the lower BJT. This cycle repeats indefinitely and causes a square voltage waveform on the secondary output of the transformer." (Source: http://en.wikipedia.org/wiki/File:Inverter_ckt_01cjc.png)
This answers provides some links to relevant literature: What is the use of transformers with 3 pairs of wires?
The power output of a self-oscillating resonant converter like this depends of the saturation characteristics of the ferrite core. Therefore, such circuits are used in special applications like compact fluorescent lamps where precise trimming of the components is worth the effort because every tiny amount of cost counts. If you desire to have a known frequency imposed by an external oscillator, you may want to use something like an external timer IC (555 or the like). Also, resonant converters alone are no controlled devices (except if you like to call the core saturation a means of control). If precise control is required (like in LCD display backlights), the driving current for the resonant converter is often provided by a controlled stage similar to a step-down-converter. This answer provides some literature links on this topic: Electronic Drivers for Fluorescent Lamps: How is the DC-to-AC Conversion done?
The main division is between BJTs and FETs, with the big difference being the former are controlled with current and the latter with voltage.
If you're building small quantities of something and aren't very familiar with the various choices and how you can use the characteristics to advantage, it's probably simpler to stick mosly with MOSFETs. They tend to be more expensive than equivalent BJTs, but are conceptually easier to work with for beginners. If you get "logic level" MOSFETS, then it becomes particularly simple to drive them. You can drive a N channel low side switch directly from a microcontroller pin. IRLML2502 is a great little FET for this as long as you aren't exceeding 20V.
Once you get familiar with simple FETs, it's worth it to get used to how bipolars work too. Being different, they have the own advantages and disadvantages. Having to drive them with current may seem like a hassle, but can be a advantage too. They basically look like a diode accross the B-E junction, so this never goes very high in voltage. That means you can switch 100s of Volts or more from low voltage logic circuits. Since the B-E voltage is fixed at first approximation, it allows for topologies like emitter followers. You can use a FET in source follower configuration, but generally the characteristics aren't as good.
Another important difference is in full on switching behaviour. BJTs look like a fixed voltage source, usually 200mV or so at full saturation to as high as a Volt in high current cases. MOSFETs look more like a low resistance. This allows lower voltage accross the switch in most cases, which is one reason you see FETs in power switching applications so much. However, at high currents the fixed voltage of a BJT is lower than the current times the Rdson of the FET. This is especially true when the transistor has to be able to handle high voltages. BJT have generally better characteristics at high voltages, hence the existance of IGBTs. A IGBT is really a FET used to turn on a BJT, which then does the heavy lifting.
There are many many more things that could be said. I've listed only a few to get things started. The real answer would be a whole book, which I don't have time for.
Best Answer
What you are asking about is oscillation. It's a very broad subject and spans everything from mechanical oscillators (like a "grandfather's clock's pendulum and escapement mechanisms tied to its gearing to the clock face) to crystal oscillators to simple relaxation oscillators (both flyback and astable), which also have a mechanical equivalent. A comprehensive view of the entire topic would occupy many books.
But we can pick on exactly the case you mention -- the so-called "Joule thief" circuit found in many different incarnations. The simplest form is something like this:
simulate this circuit – Schematic created using CircuitLab
The left side is closer to how you'd build it. You fold a wire in half and then thread it through a toroid core, building a "counter-wound auto-transformer" of sorts. It will have three contacts, which include both original ends of the wire plus a third contact where you folded the wire before making the transformer. Also, if you follow the usual instructions for making this transformer, the inductance of \$L_1\$ equals the inductance of \$L_2\$.
The right side is closer to a schematic representation designed to understand how the circuit works. Note that all I've done is some modest re-arrangement. It's still the exact same circuit as on the left. Nothing has changed. But it is easier to use the right side when explaining how it works.
Note the dots. This is important for understanding how it works.
When the battery is first attached, the currents all start out at zero. Since there is no current just yet, the voltage drop across \$R_1\$ is also zero. So initially, the battery voltage, less the \$V_\text{BE}\$ junction voltage, appears across \$L_2\$. But while \$L_2\$ does momentarily resist a too-rapid change in current, it does allow change to occur. Within a very, very short time the battery voltage, less the \$V_\text{BE}\$ junction voltage, appears across \$R_1\$ and this supplies some base current to \$Q_1\$, turning \$Q_1\$ on.
Once \$Q_1\$ is on, its collector pulls down hard on \$L_1\$, turning the LED off and causing the full battery voltage (less a small \$V_{_{\text{CE}_\text{SAT}}}\$ for \$Q_1\$) to appear across \$L_1\$. This battery voltage across \$L_1\$ causes the collector current (and the current in \$L_1\$) to rise rapidly but at a controlled rate. So the current ramps upward in \$L_1\$ and in the collector of \$Q_1\$.
If you ignored \$L_2\$, the base current will be something like \$I_{_\text{B}}=\frac{V_{_\text{BAT}}-V_{_\text{BE}}}{R_1}\$. But, because \$Q_1\$ has turned on, there is now almost the full battery voltage across \$L_1\$. The transformer behavior causes the same voltage to appear across \$L_2\$. And here, the dots become important. The more positive end of \$L_1\$ is where the dot is at. So the more positive end of \$L_2\$ will also be where its dot is at. So that point is more positive than the battery voltage. This is very important to its function for a variety of reasons: (1) it boosts the battery voltage providing still more base drive current; and, (2) it adds "positive feedback" that reinforces the on state of \$Q_1\$. So the actual current in \$R_1\$ will be more like \$I_{_\text{B}}=\frac{2\cdot V_{_\text{BAT}}-V_{_\text{BE}}-V_{_{\text{CE}_\text{SAT}}}}{R_1}\$. And that fact will keep \$Q_1\$ on for a somewhat longer time.
Eventually, one of two things happens. Either the transformer's toroid core saturates, leading to an extremely rapid change in \$L_1\$'s current and quickly exhausting the \$\beta\$ current gain of \$Q_1\$, or else the \$\beta\$ current gain of \$Q_1\$ is exhausted before the toroid core saturates. Either way, \$Q_1\$'s \$\beta\$ current gain is exhausted and \$Q_1\$ (even with its enhanced base current) can no longer support the ever-increasing current that \$L_1\$ "wants" when a fixed-voltage is applied across it.
At this point, \$Q_1\$ goes out of saturation and goes into active mode. It does this by relaxing its grip on its collector, allowing the collector to float. \$L_1\$, however, won't have any of this. It was quite happy before increasing its current rapidly and it already now has a high current in it which it demands will continue. Just the same, \$Q_1\$ is done with this and allows the voltage at its collector to rise back upwards. That drops the voltage across \$L_1\$ a little, but even with a smaller voltage across \$L_1\$ it only means a smaller increase in \$L_1\$'s current. But increase it still means. But \$Q_1\$ can't increase. It just can't. So the collector voltage goes still higher and higher, trying to stop the increase. But \$L_1\$ doesn't care. The only way the current in \$L_1\$ can decline is if the voltage across \$L_1\$ flips over and changes sign. Which is exactly what happens. The voltage at the collector of \$Q_1\$ rapidly flips and becomes higher than the battery voltage, so that the sign of the voltage across \$L_1\$ can change.
Now, \$L_1\$ still has all that current in it which has to go somewhere. Guess what? There's that handy LED over there! That looks like a good place to dump that current. So the voltage rises at the collector of \$Q_1\$ until the LED turns on. Now, this is a white LED and it probably needs something like \$3.5\:\text{V}\$ to operate. Well, \$L_1\$ has no trouble helping out there. It immediately modifies the voltage at the collector such that the LED can in fact turn on and accept the inductor's current.
But this also means that the voltage across \$L_2\$ flips over, as well! Remember, this is a transformer. \$L_2\$ was, previously, adding voltage to the battery voltage to help increase the base current. But now, because \$L_1\$ reacted so quickly to reverse its voltage in order to dump current into the LED, it also reverses the voltage across \$L_2\$, too. (It can't help doing that.) Now, this means that \$L_2\$ subtracts from the battery voltage and basically turns \$Q_1\$ completely off.
There's a moment we missed, here. That's just at the place where the collector voltage is rising up, but the voltage across \$L_1\$ hasn't quite reversed itself, just yet. As the collector "lets up" and floats upward, there is a diminishing voltage across \$L_1\$. This diminished voltage across \$L_1\$ yields a similarly diminished voltage across \$L_2\$ (transformer action.) That leads to a lower base drive current in \$Q_1\$. Which means that \$Q_1\$, which was able to handle more collector current beforehand, can handle just that much less collector current. Which means the collector has to rise still further as \$Q_1\$ approaches being turned off. \$L_1\$ is very unhappy with change in \$Q_1\$, too, and reacts. If the current in \$L_1\$ can't increase, and can't even stay the same, there is only one response possible -- the magnetic field must start to collapse. The moment this takes place, the voltage across \$L_1\$ reverses itself, the collector voltage rises above the battery voltage, the voltage in \$L_2\$ also reverses itself and greatly reduces the base current towards zero, and this whole process rapidly feeds on itself. Very quickly \$Q_1\$ is turned completely off.
Now that \$L_1\$'s magnetic field is collapsing, it's current can decline as it drives current into the LED. Eventually, the magnetic field energy has completely collapsed to zero and no more current is possible. At this point the voltage across \$L_1\$ returns to zero, the voltage across \$L_2\$ also returns to zero, and now \$R_1\$ can supply a starting base current needed to turn \$Q_1\$ back on, which then places a voltage across \$L_1\$, leading to a supporting voltage across \$L_2\$ that increases the base current, again, and the cycle repeats itself another time.
This whole process takes time as it stores increasing energy in \$L_1\$. However, eventually, the BJT cannot continue to support those increases in the magnetic field and then the magnetic field must collapse. This collapse is used to turn the BJT off and drive current into the LED. When the stored energy in the magnetic field is exhausted, the process repeats.
So one of the keys is the temporary storage of energy "somewhere." This can be done by temporarily storing energy in magnetic fields (inductors), temporarily storing energy in electric fields (capacitors), or both. You can slosh the energy back and forth between magnetic and electric fields, too (tank circuit.) But you need a place to temporarily store energy. That's one of the keys. With that key, plus a way of providing sufficient positive feedback to keep things from finding a "quiescent point" in some halfway-place, gives you an oscillator. The trick, as always, is working out good ways to achieve both in a simple circuit.