Implicit barrier at the end of #pragma for

openmpopenmpiparallel-processing

Friends, I am trying to learn the openMP paradigm.
I used the following code to understand the #omp for pragma.

int main(void){
int tid;
int i;

omp_set_num_threads(5);
#pragma omp parallel \
    private(tid)
{
    tid=omp_get_thread_num();
    printf("tid=%d started ...\n", tid);
    fflush(stdout);

    #pragma omp for
    for(i=1; i<=20; i++){
        printf("t%d - i%d \n",
                omp_get_thread_num(), i);
        fflush(stdout);
    }

    printf("tid=%d work done ...\n", tid);
}

return 0;

}

In the above code, there is an implicit barrier at the end of #pragma omp parallel, meaning all the threads 0,1,2,3,4 must reach there before going to the next statement.

So, to check this barrier, I enclosed this "pragma for" in a condition if(tid!=0), meaning all threads except thread 0 i.e 1,2,3,4 should complete their work in the loop and wait for thread0 indefinitely. But, to my surprise this is not happening. Every thread is doing its iteration and completing successfully. i.e t1 completes iterations 5,6,7,8 —- t2 does 9,10,11,12 —- t3 does 13,14,15,16 and t4 does 17,18,19,20. Please note : iteration 1,2,3,4 were never completed.

To dig deeper, instead of tid!=0, I enclosed the same #pragma for in tid!=1 meaning instead of thread0, thread1 bypasses the barrier. To my surprise, the program now hangs and all threads wait for the thread1.

Can somebody please tell me the explanation for such unexpected behavior. Final code that hanged : 

int main(void){
int tid;
int i;

omp_set_num_threads(5);
#pragma omp parallel \
    private(tid)
{
    tid=omp_get_thread_num();
    printf("tid=%d started ...\n", tid);
    fflush(stdout);

    if(tid!=1){
        /* worksharing */
        #pragma omp for
        for(i=1; i<=20; i++){
            printf("t%d - i%d \n", 
                omp_get_thread_num(), i);
            fflush(stdout);
        }
    }else{
        printf("t1 reached here. \n");
    }

    printf("tid=%d work done ...\n", tid);
}

return 0;

}

I tried setting i shared or private, but it did not change the behavior of the program.

Best Answer

The problem here is that the behaviour is undefined by the standard. From Section 2.5, line 21 of the OpenMP 3.1 specification (but the text has stayed the same more or less since the beginning):

• Each worksharing region must be encountered by all threads in a team or by none at all.

Where omp for is a worksharing construct. So yes, I too would normally expect a hang with your code, but the compiler is entitled to assume that what you're doing never happens, and so the end result -- it sometimes hangs but sometimes doesn't, depending on the details on which threads you hold up -- maybe isn't that surprising.

Related Topic