Electronic – Why is the transistor switched on even though the collector is not connected

The base-emitter junction (BE) of a NPN transistor behaves like a diode:

So you basically have this equivalent circuit:

As you can see, your LED is in series with the resistor and the transistor's diode (the BE junction). Some current (depending on the value of your resistor) is passing through this circuit, making the LED light.

One way to avoid this is, as The Photon said, using a MOSFET. Another way is using the NPN transistor in the common-emitter configuration:

From this comment of yours:

Actually, I read the specifications on the transistor and the maximum base input is 5V

I can understand why you think you should drive the transistor's base from 4.5V (< 5V). But you can drive the base with a higher voltage as long as you have a resistor in series. The BE junction, acting as a diode, will have a fixed voltage drop (typically 0.7V) across it. The rest of the voltage will go across the resistor which also limits the current through the BE junction.

You really should post a schematic instead of all the hand waving. However, there is one obvious flaw in your description. One side of the relay could should be connected to the collector of the transistor as you say, but the other side must be to the power supply, not ground.

When the transistor is off, the collector is effectively not connected or left floating. When the transistor is turned on, it will pull the collector low. Since that end of the coil will be low, the other end must be high for there to be voltage accross the coil so that the relay can turn on.

Here is how you should hook things up:

Don't leave off D1. It is there to catch the inductive kickback from the coil when it is suddenly shut off. Without it, the coil will generate high voltage and fry the transistor.