It's really hard to tell from your description. A good description of the noise and when it happens would be a good start.
You already said that the noise happens when you play a note. What happens if the volume is turned down? Either using the control on the electronic piano or by using the pot in your circuit. If the noise goes away then you can assume that the problem is either with the signal level on the input, or that you're asking the amp to put out more than the rated 3.5 watts.
If I were a betting man, I would guess this is the case. You're using a 3.5 watt amp to power a 15 watt speaker. Normally for a 15 watt speaker you would be using a 15 to 30 watt amp. So your amp is undersized.
Adding a cap like what @Manmanguruman suggests isn't a bad idea, but I would hope that there is already some caps inside the amp module. Still, you would use the largest cap that is somewhat practical. Start with about 470 uF and higher.
The pot should have a log taper, not a linear taper. Changing the pot will make the volume control more useful-- but will do nothing for your noise problem.
It is also possible that you have a signal loading problem with the piano. Basically, the piano might not be able to handle having an amp plugged into it. I would hope the piano was designed better, but I have seen some products do some stupid things. Turning the volume down with your linear/log pot should help this issue.
Other than that, the only thing I can think of would be a component failure. Something like the speaker, amp, battery, or piano actually being broken in some way.
C12 should not connect arbitrarily to a local ground but should connect directly to the microphone ground on J2: -
That's my 1st observation - when you have MICN and MICP pins they need to route side by side to minimize pick-up also. The big problem with where C12 is grounded is that between its ground point (junction of C7 and C5) there could exist several hundred micro-volts of high frequency ground noise due to currents in the ground plane. This is why that chip provides a mic-negative pin - it uses a differential amplifier to amplify the mic signal directly.
There may be other issues like C5 also not connecting directly to the mic ground on J2 also. C7 and C6 also must ground right at the microphone negative point too. Also R4 causes the MICP lead to be very long and clearly, the longer the microphone tracks are, the more they'll pick-up.
Best Answer
The core problem that you're experiencing is probably that the SID is not a low-noise device. When I had a C64 way back in the day we would turn up the volume to listen to the noise-- as a debugging aid. We could tell, roughly, if the CPU crashed by the noise generated from the SID!
That being said, there are some things that you can do to minimize the noise. I have no idea if any of these will reduce the noise enough in your case, but it's a start. Here goes, in no particular order:
There you go!