As this is only your second time working with electronics, I'll try to keep my terminology simple.
It's hard to see exactly how you have wired up the sensor and LED, but I can take a guess. (If I'm wrong, then everything below probably makes no sense).
The sensor is connected between + power and the Arduino input, while the LED is connected between - power and the Arduino input.
When you press the sensor, electrical current can flow from the + side of the power the Arduino sensor pin, charging it up and giving it a high voltage. Current also flows through the LED, causing it to light up.
Now, what happens When you release the sensor? The electrical charge inside the Arduino sensor pin which was giving it a high voltage, will now flow as current through the LED to - power, bringing the voltage down, so that the Arduino sees you've let go.
But what happens if you don't have an LED in there? The electrical charge in the Arduino sensor pin has nowhere to go, and so it just stays there, and the voltage doesn't change.
The reason the Arduino's sensor pin behaves like this is because it behaves like a tiny capacitor. It can store a small amount of electrical charge, and thus 'remember' the voltage that was placed on them by the sensor.
So, how can you fix it? You'll need to have somewhere for this charge to flow. If not an LED, then a resistor should do. Any value between 1k and 1000k will probably work fine.
Yes, putting C1 and C2 in parallel to R1 and R2 provides good RF rejection on the gain resistor lines.
I built the circuit and did some "poor man's EMC testing" by attaching a long cable to the sensor and wrapping it around my mobile phone, then calling the mailbox. The result of the modified circuit were very good, showing only a small increase in noise, while an alternative configuration showed readings all over the place.
Best Answer
You will have to use the ADC to measure the Piezo Sensor, the diodes and op-amps are there to protect the ADC, some piezo elements can produce a swing of upto 90V with a simple knock on the table. The Diodes clamp the voltage into the operable range of the ADC. The op-amp is to amplify the signal, if you see the characteristic load curve from the datasheet you can see that it is not a linear relation and for lower application of loads the sensor give output in the range of a few millivolts.