Some cap vendors make their own parts. Some buy caps from a smaller manufacture with the reels re-branded at the fab. Watch out. I got into a mlcc failure investigation in 2002 and started inspecting caps on a reel under a microscope. 3/10 came off the reel cracked. A crack will sooner or later lead to a short. Cracks are not obvious even under a microscope. They may present as a subtle colour shift if the crack is below the surface layer. Some cracks may be sufficient to present a short immediately. Not all. The vendor's manufacturer in this case eventually identified a hopper where the caps were getting cracked.
MLCC are very sensitive to mechanical stress. Especially bigger than 1210 sizes. I found a big power bypass cap adjacent to a heavy mechanical connector once. The nearest mounting hole was 2" away! They were cracking at a rate of 5/10 during install of the unit. A fraction of those were catching fire. The fire would continue to burn until it melted the copper, breaking the power connection.
Another effect of a crack is a reduction of the max working voltage of the cap. It might be spec'd for 200 V. But once cracked it may break down at 40 V. Cracked caps burst into flames in my lab when tested - even below their rated voltage.
Another way to warm up caps is exceeding their max ac current. It's easy to think of caps as zero power dissipation devices. Especially the higher Q mlcc. But they're not. Calculate the power dissipated in caps and do not exceed power/ac current limits. Shows up in power circuits and converters commonly.
The rectified AC waveform catches the peaks. The input 9VAC is RMS (Root-Mean-Square average) equivalent -- the actual amplitude of the sinewave is about 40% higher than the RMS average (square root of 2 is 1.414). So on your picture the 9V equivalent is about 70% of the way between 0V and the peaks.
The numbers don't work out exactly to the ideal square-root-of-two crest factor, because there is some voltage drop across the two diodes that are on, and also because there is some variation in the line voltage.
The reason RMS is used to describe AC voltages, is because the amount of power (heat) delivered to the resistive load is the same as it would be for a 9V DC 1A source.
Edit: explaining the observed difference in load voltage measurement for different load capacitor condition, explaining why DMM gives wrong measurement for full-wave rectified waveform...
Voltage is not actually being boosted in this circuit. When the capacitor is removed, the full-wave rectified signal doesn't sustain the peak voltages. As Ignacio Vazquez-Abrams mentions, the DMM may not be measuring the waveform correctly, especially in the case where there was no capacitor -- assuming you measured with the DMM's DC Voltage setting, without the capacitor the full-wave rectified waveform would be confusing the measurement. The 9V DC measurement reported by the DMM matches the rated 9V AC RMS equivalent, so maybe the DMM was somehow measuring the RMS value. Then when you added a capacitor, the waveform peaks were sustained long enough for the DMM to start measuring accurately. Sadly, it is possible for measurement equipment to "lie" to us under some conditions. Happens to the best of us sometimes.
The DMM is just a electronic machine, it's not a magic box that always gives the right voltage measurement. Most DMM's use a measurement technique called dual-slope integration, where a capacitor is first rapidly charged up to the voltage being sampled, and then the sampling capacitor is discharged through a constant-current source. The DMM counts how long it takes to discharge the capacitor back to zero. The value of that counter is what the DMM displays. Calibration depends on the current source, the comparator offset voltage, and the quality of the sampling capacitor. This technique is cheap to implement and it works great, as long as the input signal doesn't change very quickly. But when connected to that full-wave rectified signal, the sampling capacitor doesn't stay at the peak voltage. It's indeterminate where the sample interval begins and ends, so it's hard to know exactly how many counts the DMM might report. So if the C is omitted, then it's not really a DC circuit, so the DMM DC measurement isn't valid.
You also asked about using different value of C. The bridge rectifier is not regulated, its output voltage can vary with different load impedance. Changing the load capacitance C affects the capacitor's reactance Xc, which also affects load impedance.
$$
Xc = \frac{1}{2 * pi * frequency * C}
$$
A lower load impedance, just like load resistance, will draw more current at a given voltage. But unlike resistance, the current and voltage waveforms may be out of phase. So it's possible to have voltage across a capacitor even with zero current, and it's possible to have current through an inductor even with zero voltage (under some conditions).
Putting a reactance in parallel with a resistance is a bit more complicated than putting resistors in parallel, because the voltage and current waveforms are in-phase for the resistor but are 90 degrees out-of-phase for the capacitor. In AC circuit analysis, we use complex numbers and phasor notation (yes, that really is a thing) to model these AC circuit elements. If you think of impedance as a vector, with the length of the vector acting similar to resistance in Ohm's law, and reactance acting at right angles to resistance, then putting the resistor and capacitor in parallel gives the total load impedance Z. Although it's possible to go deeper into the maths, there's another important point worth mentioning:
This circuit isn't regulated. If you want to get 12V DC output, you can't just select a capacitor value and expect this to always give 12V DC output regardless of how much load current is drawn. This circuit is a good building block to start with, but the full-wave rectified output voltage will vary with changes in the utility line-in voltage as well as the load current. If you really want it to be regulated, add a regulator circuit such as 78M05 (or 78M12 if you really need 12V). In that case you'll need the full-wave bridge to provide a bit more then 12V so that the regulator has some headroom to work with (but not too much, because the linear regulator works by wasting the unwanted energy.)
AC circuits theory can be kind of mind-bending at first, because there are all these surprising mathematical things like imaginary numbers and Euler's law, that turn out to actually work in real life. The comment about how the capacitor "evens out the peaks" is... kind of true... but it's a major oversimplification. As you've discovered, a qualitative statement like that doesn't help you determine how much capacitance you need to achieve your design goal of making a 12V DC power supply.
I'm not going to be able to fully explain AC circuits theory here, but here are at least some interesting breadcrumbs:
See https://en.wikipedia.org/wiki/Electrical_reactance
See https://en.wikipedia.org/wiki/Electrical_impedance
See https://en.wikipedia.org/wiki/Phasor
Best Answer
At page 7, on the equivalent circuit of the RA08H1317M, the output is already AC coupled. Same thing on page 1 on the block diagram.
If you need to AC couple outside of the component anyway, then you have to find the power dispated by the capacitor due to ESR:
\$P_{cap}=R_{ESR} \cdot I^2=\frac{R_{ESR}}{R_{50Ω}} \cdot P_{RF}<\frac{0.1Ω}{50Ω} \cdot 10W=20mW\$
This is close to the typical power rating for size 0603. Size 0805 and 1206 should have no problem if you asume an ESR below 100mΩ. You should probably use a NPO dielectric.