C++ – How to measure performance indepently of the machine used

cperformanceprofiler

I had a routine that was performing good. However, I had to make a change to it. The change improved the routine's precision but hurt the performance.

The routine is lots of math calculations and is probably CPU bond (I still have to do more rigorous testing on this, but I'm 99% sure). It is written in C++ (compiler is Borland C++ 6).

I want to measure the performance of the routine now, first I thought about measuring the execution time, but that is a kind of flawed approach in my opinion since there could be much more things going on.

I ran into this topic then: Techniques to measure application performance – Stack Overflow. I liked the idea of measuring through MFlops.

My boss suggested to try to use some kind of measurement by cpu clock cycles, so the tests would be machine-independent, however, I think this approach falls into the MFlops testing.

In my opinion measuring both things (time of execution and MFlops) is the way to go, but I would like to hear from the stackoverflow experts what do you guys think.

What is the way to go to measure performance of a routine that is known as CPU bond?

Best Answer

CPU clock cycles don't mean that much either, if your application is memory-bound. On a faster CPU, you'll just spend more CPU cycles waiting on the same cache miss. (Mathematical apps are probably not I/O bound).

Another problem is that the number of clock cycles for a certain instruction sequence will still vary across architectures (and that even includes between Intel Core1 / Core2). So, as an absolute measure of performance, clock cycles on one CPU is hardly an improvement.

I would argue they're in fact worse as a measure. Unlike time, users don't care about cycles. This matters especially with modern multi-core CPUs. An "inefficient" algorithms using twice the number of cycles and 3 cores will finish in 67% of the time. Users will probably like that.

Related Solutions

C++ – What are the differences between a pointer variable and a reference variable in C++

A pointer can be re-assigned:

int x = 5;
int y = 6;
int *p;
p = &x;
p = &y;
*p = 10;
assert(x == 5);
assert(y == 10);

A reference cannot be re-bound, and must be bound at initialization:

int x = 5;
int y = 6;
int &q; // error
int &r = x;

A pointer variable has its own identity: a distinct, visible memory address that can be taken with the unary & operator and a certain amount of space that can be measured with the sizeof operator. Using those operators on a reference returns a value corresponding to whatever the reference is bound to; the reference’s own address and size are invisible. Since the reference assumes the identity of the original variable in this way, it is convenient to think of a reference as another name for the same variable.
```
int x = 0;
int &r = x;
int *p = &x;
int *p2 = &r;

assert(p == p2); // &x == &r
assert(&p != &p2);
```

You can have arbitrarily nested pointers to pointers offering extra levels of indirection. References only offer one level of indirection.

int x = 0;
int y = 0;
int *p = &x;
int *q = &y;
int **pp = &p;

**pp = 2;
pp = &q; // *pp is now q
**pp = 4;

assert(y == 4);
assert(x == 2);

A pointer can be assigned nullptr, whereas a reference must be bound to an existing object. If you try hard enough, you can bind a reference to nullptr, but this is undefined and will not behave consistently.

/* the code below is undefined; your compiler may optimise it
 * differently, emit warnings, or outright refuse to compile it */

int &r = *static_cast<int *>(nullptr);

// prints "null" under GCC 10
std::cout
    << (&r != nullptr
        ? "not null" : "null")
    << std::endl;

bool f(int &r) { return &r != nullptr; }

// prints "not null" under GCC 10
std::cout
    << (f(*static_cast<int *>(nullptr))
        ? "not null" : "null")
    << std::endl;

You can, however, have a reference to a pointer whose value is nullptr.

Pointers can iterate over an array; you can use ++ to go to the next item that a pointer is pointing to, and + 4 to go to the 5th element. This is no matter what size the object is that the pointer points to.
A pointer needs to be dereferenced with * to access the memory location it points to, whereas a reference can be used directly. A pointer to a class/struct uses -> to access its members whereas a reference uses a ..
References cannot be put into an array, whereas pointers can be (Mentioned by user @litb)
Const references can be bound to temporaries. Pointers cannot (not without some indirection):
```
const int &x = int(12); // legal C++
int *y = &int(12); // illegal to take the address of a temporary.
```
This makes const & more convenient to use in argument lists and so forth.

C++ – How to iterate over the words of a string

I use this to split string by a delimiter. The first puts the results in a pre-constructed vector, the second returns a new vector.

#include <string>
#include <sstream>
#include <vector>
#include <iterator>

template <typename Out>
void split(const std::string &s, char delim, Out result) {
    std::istringstream iss(s);
    std::string item;
    while (std::getline(iss, item, delim)) {
        *result++ = item;
    }
}

std::vector<std::string> split(const std::string &s, char delim) {
    std::vector<std::string> elems;
    split(s, delim, std::back_inserter(elems));
    return elems;
}

Note that this solution does not skip empty tokens, so the following will find 4 items, one of which is empty:

std::vector<std::string> x = split("one:two::three", ':');

Best Answer

Related Solutions

C++ – What are the differences between a pointer variable and a reference variable in C++

C++ – How to iterate over the words of a string

Related Topic