You didn't give the all-important information about what frequencies you are using for the communication. You can't know if a filter is right if you don't know what frequencies you want to separate.
However, the main point is to attenuate the power line frequency by a large factor. Here is one simple passive way to do that:
This is a simplistic filter which is basically 4 high pass filters strung together. I'm assuming the communication frequency is high compared to the power frequency, like 100 kHz or more. Each high pass filter stage has a rolloff of 1.6 kHz. 60 Hz is far enough below that so you can make the approximation it will be attenuated by 1.6 kHz / 60 Hz = 27. These filter stages will interact some to give more overall attenuation, but even ignoring that you get at least the single filter stage attenuation to the power of the number of filter stages, or about 495 k in this case. That means the 120 V 60 Hz power signal will be down to less than 240 µV out of this filter. 100 kHz, which we're assuming is your lowest frequency of interest, is 63 times the high pass filter rollof frequency, so won't be effected much by this filter.
The signal directly out of this filter is not ready for running into a chip like a opamp or microcontroller A/D yet. The power line has all kinds of nasty noise on it, with some transients that can be a few 100 V. This filter will pass such short term spikes, which would blow up a opamp or microcontroller. There needs to be some sort of clamping function, keeping in mind that those component have to be able to handle 100 V or more for a short time.
The voltage rating of the filter components needs to be carefully considered too. C1 will take most of the 60 Hz voltage, and certainly needs to be rated for power line operation. Even after just the first stage, the 60 Hz voltage will be low enough to not be a problem for ordinary resistors. However, the resistors need to be able to handle the temporary high voltage spikes that won't be attenuated by the filter. Either get resistors rated for a few 100 Volts, or implement the resistors shown in the schematic by 3 or 4 ordinary equal resistors in series.
You integrate the PSD (power spectral density) of the signal over various widths centered on the center of the channel until you get 95% of the total.
This can be done analytically for some signals, but must be determined empirically for others.
It's an interesting experiment to take the FFT of a signal to get it's PSD, truncate that spectrum to various bandwidths, and then perform the inverse FFT to see how the original waveform gets distorted — and for digital signals, what happens to the eye diagram. It can give you some real insight into what happens with real-world communications channels.