I am wondering if an 18.5V, 3.5A charger be able to charger a 18V battery without any bad side effects? The old charger (not functioning anymore) was 24V 0.2A. I know the charger must have a higher voltage rating than the battery to be able to completely charge it.
Charge 18V battery with charger rated at only 18.5V
Probably not. The voltage spec written on the side of a battery pack is usually nominal voltage during discharging, not maximum voltage during charging. The wall adapter input to the battery charger needs to be the maximum battery voltage during charging, plus however much drops across the battery charger voltage regulator, plus the drop in protection circuitry & wiring resistance etc., plus enough that if the regulator is operating at the low end of the spec variation range (e.g. +/- 5%) it's still high enough.
If the original wall adapter was 24V, you probably need to find another 24V adapter. It's possible the charger will work with a somewhat lower (or higher) voltage, but as the only information we have is what was specified by the manufacturer, we have to take their word for it.