I see you're using unsigned integers. By definition, in C (I don't know about C++), unsigned arithmetic does not overflow ... so, at least for C, your point is moot :)
With signed integers, once there has been overflow, undefined behaviour (UB) has occurred and your program can do anything (for example: render tests inconclusive).
#include <limits.h>
int a = <something>;
int x = <something>;
a += x; /* UB */
if (a < 0) { /* Unreliable test */
/* ... */
}
To create a conforming program, you need to test for overflow before generating said overflow. The method can be used with unsigned integers too:
// For addition
#include <limits.h>
int a = <something>;
int x = <something>;
if ((x > 0) && (a > INT_MAX - x)) /* `a + x` would overflow */;
if ((x < 0) && (a < INT_MIN - x)) /* `a + x` would underflow */;
// For subtraction
#include <limits.h>
int a = <something>;
int x = <something>;
if ((x < 0) && (a > INT_MAX + x)) /* `a - x` would overflow */;
if ((x > 0) && (a < INT_MIN + x)) /* `a - x` would underflow */;
// For multiplication
#include <limits.h>
int a = <something>;
int x = <something>;
// There may be a need to check for -1 for two's complement machines.
// If one number is -1 and another is INT_MIN, multiplying them we get abs(INT_MIN) which is 1 higher than INT_MAX
if ((a == -1) && (x == INT_MIN)) /* `a * x` can overflow */
if ((x == -1) && (a == INT_MIN)) /* `a * x` (or `a / x`) can overflow */
// general case
if (a > INT_MAX / x) /* `a * x` would overflow */;
if ((a < INT_MIN / x)) /* `a * x` would underflow */;
For division (except for the INT_MIN
and -1
special case), there isn't any possibility of going over INT_MIN
or INT_MAX
.
Best Answer
That's a good problem. In order to solve that problem you will also have to disable ASLR otherwise the address of g() will be unpredictable.
Disable ASLR:
Disable canaries:
After canaries and ASLR are disabled it should be a straight forward attack like the ones described in Smashing the Stack for Fun and Profit
Here is a list of security features used in ubuntu: https://wiki.ubuntu.com/Security/Features You don't have to worry about NX bits, the address of g() will always be in a executable region of memory because it is within the TEXT memory segment. NX bits only come into play if you are trying to execute shellcode on the stack or heap, which is not required for this assignment.
Now go and clobber that EIP!