Electronic – Best Option for Charging a High Voltage Capacitor

100VDC is in the domain of Switch Mode power supplies, which, for example, phone chargers are. You can get a ringing-core SMPS down to a cubic centimetre or two, especially if the system doesn't require high degrees of isolation between high and low voltage sides.

I would look into buying or building a small ringing-core SMPS, or using those cheap crappy fake Apple mini cubes from China for 5V and then going to 3.3V if you are lazy. Those things are dangerous and should not be used in the mains, but if you open them up and make sure no shorts can happen by unfolding the innards or at least glueing some stuff up, you have a tiny 5V supply from 80V to 300V DC.

I have a semi-serious hobby project with a couple of friends where we need to charge a pulse operated capacitor rated to around 4kV with 1500nF capacitance.

You didn't say exactly what voltage you wanted to charge the capacitor to, but generally speaking you shouldn't go all the way up to the rating if you want the device to be reliable.

For this I need a high voltage supply

You can get a high voltage module from distributors like Digikey or Mouser.

https://www.digikey.com/product-detail/en/xp-power/G30/1470-4014-ND/6802063

https://www.xppower.com/portals/0/pdfs/SF_G_Series.pdf

If I were to use an off the shelf high voltage source of around 10kV maximum voltage and 20W power, how will the system react when the source is connected, possibly through appropriate resistor?

Someone suggested me that the namely excessive 10kV would immediately break the capacitor as the voltage drop is all seen through the insulation inside the capacitor.

If you just put a resistor in series with the capacitor then the capacitor will likely break. If you put the capacitor at the output of a voltage divider then you can lower the voltage enough so that it wont break.

Will an excess voltage (measured at the source) break the capacitor before actually charging it?

No. The capacitor will break once it charges up past its rating. Therefore there must be something to limit the input voltage to the capacitor. For this a voltage divider is a good option.

If not, how does a typical HV source react to the situation? If connected straight to a capacitor (with effectively zero resistance), will the source just see the connection as about short circuit and promptly break or blow a fuse? Or will it just slowly start to ramp the voltage up, until the external capacitor is at equal voltage to source target (or until the capacitor breaks)

That question is impossible to answer in general without knowing the ratings of the components inside the power supply, and what fuses (if any) are used. But I would venture to say that the output will probably just ramp up. Nearly all DC power supplies include some capacitance at their output. You would just be putting a little more in parallel with whatever is already there.