You have two problems right now that are separate from each other.
1) Button bounce
2) Improper button wiring
First, you need to introduce a delay into your interrupt function so that it won't keep registering each rise and fall of the bounce as a separate button press. Use "delayMicroseconds()" because apparently you can't use regular "delay()" inside interrupts. Id suggest a 20,000 microsecond delay.
Second, you need to put a pull down resistor onto the digital pin that you are sampling.
The problem you are having is that the pin is in input mode and thus at high impedance (very high resistance) and thus has a tenancy to "float". If you put a voltage to it, it charges up like a capacitor and since the pin itself has very high resistance to ground, it takes a while for the charge to dissipate to the point where the Arduino will register a LOW again. A 10k resistor from the pin to ground will force the "capacitor" to discharge quickly after current from the button has stopped flowing.
Finally, the Arduino is pretty quick at it's read/write operations (I'm a mechanical engineer, ALL electronics happen "quickly" in comparison to mechanical systems, so it's relative I guess). Here's a link to some test results.
TL;DR verson:
What these values tell me, is that we can do about:
10 analog 10-bit readings per millisecond with analogRead()
128 pwm settings per millisecond with analogWrite()
220 pin reads per millisecond with digitalRead()
224 pin writes per millisecond with digitalWrite()
1056 pin reads per millisecond with direct port reads
1059 pin writes per millisecond with direct port writes
Before we bury this question I'd like to come back to my forgotten child.
Short answer: The dragon was faulty, an exchanged unit worked just fine. Thereafter I somehow totally forgot to close this question. I'm sorry.
Best Answer
Brazilian greetings!
First of all thanks Joby for your example. Secondly, his example has just a minor error. The number 0x20 is not correct. It should be 0x04. Also, just as a suggestion, I would not use hexadecimal numbers like 0xFB, 0x20, or 0x04 in the code. I would suggest using the PIN port definitions found in the io.h and other ones referenced by header file. I have rewritten Joby's example below, with some comments for the beginners.
/*
comments for beginners
comment #1: (1 << PD2) generates the binary 00000100. The operation "~" flips all the digits, i.e., the binary now is 11111011. Finally the &= applies the logic "AND" between DDRD and 11111011 and the result is placed again in DDRD memory. Note: What the operator "AND" does is for each bit in the DDRD memory, it compares with the binary number above. If the bit in DDRD is 0 and the bit in the binary at the same bite position is 1, then the resulting bit is 0, if the DDRD is 1 and the bit in the binary is 1, the resulting bit is 1, and if the bit in the DDRD is 1 or 0 and the bit in the binary is 0 then the resulting bit is always 0. In summary, the command DDRD &= ~(1 << PD2) changes only the bit PD2 to zero and leave the other ones (zeros or ones) untouched. It seems a little bit complicated, but after you get used to it, it is the best way of changing a bit in a bite without changing the other bits.
comment #2: (1 << PD2) generates the binary 00000100. Using the same logic "AND" described in comment #1, the command "PIND & 0000100" checks only if the PIND2 (our input pin where the push button is connected to) is set to high or not. All the other pins will be FALSE since the binary bits are set to 0, and since the binary bit #2 is set to 1, the IF statement will be TRUE only if the PD2 input is set to high or FALSE if the PD2 input is set to low.
comment #3: Following the logic explained in comment #1, this command sets output pin PINB2 in port PORTB to high voltage. If your LED is correct connected to this pin port with a resistor of ~300 ohms, and that resistor is connected to the ground, the LED should turn on.
comment #4: The LED should turn off for the same reasons explained in the previous comments.
Final considerations:
a) To avoid voltage oscillation in the input pin PD2 when the push button is not pressed (open circuit), I strong recommend to place a pull-down resistor (1 kOhm or higher), so that the LED does not light up accidentally due to this random voltage oscillation.
b) A disclaim note: The ideas described here are to be used as educational only and they should NOT be used in any real system before consulting an expert in electronics.
*/