Electronic – Six LEDs using one 3 V button cell

It is highly likely that your red LEDs are of lower luminous efficiency than your white LEDs.

Also, driving an LED incorrectly can destroy it or reduce it's output from then on. Such damage can occur almost instantaneously. Red LEDs have lower operating voltages than White LEDs so are more likely to be damaged by the application of a constant voltage that is too high and not current limited.

The efficiency of LEDs varies widely. It can be expressed in lumens/Watt = l/W.

The best white and the best red LEDs have similar enough l/W ratings that both should appear about equally bright when driven with the same power input. White LEDs typically have operating voltages = Vf in the 3.0 - 3.6V range and red LEDs have Vf = 2.0 to 2.5 Volts. So a red LED would need about 50% more current to achieve the same power as a white LED as Power = V x I.

You state the LED output range as 500 - 20,000 mCd. That's a 40:1 brightness range.

Note that candela are a measure both of energy out and area of illumination. Less area = more brightness for the same light energy.

If the LEDs are to be directly powered by the batteries, the blue, white, and possibly the green LEDs (depending on which type of green LED it is) will not work at all on 3 Volts: They require a forward voltage of 3+ Volts, 3.4 Volts as specified in the question.

Also, once the voltage starts dropping due to depletion, even the marginally lit ones (e.g. the green, may be the orange) will no longer work.

Therefore ideally only red LEDs should be used: Red LEDs typically have the lowest operating voltage amongst visible colors. Alternatively, as noted, 3 cells would be needed in series.

Now about power:

LEDs can be operated typically at half the nominal current or less. So 20 LEDs operated in parallel at around 10 mA each will require 200 mA current, well within the capabilities of typical AA Alkaline cells.

For instance, the Energizer E91 Alkaline AA battery is rated for around 3 to 4 hours at 200 mA current. Put three of those in series, add in an appropriate current limiting resistor for each LED, and it will all work.