So, I've been playing with a unicorn deployment on my personal server. There are a variety of reasons behind this – one of them being that I didn't particularly want to have to compile my own version of nginx in order to get phusion passenger. (I like having things package-managed.)
Regardless, unicorn is exhibiting some strange behavior. When you send Unocorn the USR2 signal, it should essentially hot-swap itself by copying it's pid file from unicorn.pid to unicorn.pid.oldbin. I've configured my unicorn.rb file so that if it finds said unicorn.pid.oldbin when it starts – it shoots the old process a QUIT – just like I'm supposed to.
However, this doesn't appear to be working because none of the pid's are changing. So, either Unicorn is inheriting the same PID – or something is amiss.
Below is an strace of the master Unicorn process. I'm not spectacular at reading system calls – but I don't see anything jumping out at me here. Could someone take a look at it and tell me what I'm missing? Thanks!
Process 32177 attached - interrupt to quit
select(6, [5], NULL, NULL, {20, 476644}) = ? ERESTARTNOHAND (To be restarted)
--- SIGUSR2 (User defined signal 2) @ 0 (0) ---
sigreturn() = ? (mask now [])
fcntl64(6, F_GETFL) = 0x1 (flags O_WRONLY)
fcntl64(6, F_SETFL, O_WRONLY|O_NONBLOCK) = 0
write(6, ".", 1) = 1
select(6, [5], NULL, NULL, {10, 814939}) = 1 (in [5], left {10, 814936})
fcntl64(5, F_GETFL) = 0 (flags O_RDONLY)
fcntl64(5, F_SETFL, O_RDONLY|O_NONBLOCK) = 0
read(5, ".", 11) = 1
waitpid(-1, 0xbf987ee4, WNOHANG) = 0
open("/www/web/tmp/unicorn.pid.oldbin", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
open("/www/web/tmp/unicorn.pid", O_RDONLY|O_LARGEFILE) = 4
fstat64(4, {st_mode=S_IFREG|0644, st_size=6, ...}) = 0
fstat64(4, {st_mode=S_IFREG|0644, st_size=6, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb777c000
_llseek(4, 0, [0], SEEK_CUR) = 0
read(4, "32177\n", 4096) = 6
read(4, "", 4096) = 0
close(4) = 0
munmap(0xb777c000, 4096) = 0
unlink("/www/web/tmp/unicorn.pid") = 0
open("/www/web/tmp/0.0132033250636285.32177", O_RDWR|O_CREAT|O_EXCL|O_LARGEFILE, 0644) = 4
fcntl64(4, F_GETFL) = 0x8002 (flags O_RDWR|O_LARGEFILE)
fstat64(4, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb777c000
_llseek(4, 0, [0], SEEK_CUR) = 0
write(4, "32177\n", 6) = 6
rename("/www/web/tmp/0.0132033250636285.32177", "/www/web/tmp/unicorn.pid.oldbin") = 0
close(4) = 0
munmap(0xb777c000, 4096) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xb74939d8) = 32184
waitpid(-1, 0xbf987ee4, WNOHANG) = 0
gettimeofday({1312941623, 777508}, NULL) = 0
gettimeofday({1312941623, 777684}, NULL) = 0
select(6, [5], NULL, NULL, {26, 0}) = ? ERESTARTNOHAND (To be restarted)
--- SIGCHLD (Child exited) @ 0 (0) ---
sigreturn() = ? (mask now [])
fcntl64(6, F_GETFL) = 0x801 (flags O_WRONLY|O_NONBLOCK)
write(6, ".", 1) = 1
select(6, [5], NULL, NULL, {25, 595766}) = 1 (in [5], left {25, 595763})
fcntl64(5, F_GETFL) = 0x800 (flags O_RDONLY|O_NONBLOCK)
read(5, ".", 11) = 1
waitpid(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 1}], WNOHANG) = 32184
gettimeofday({1312941624, 183506}, NULL) = 0
write(2, "reaped #<Process::Status: pid=32"..., 57) = 57
open("/www/web/tmp/unicorn.pid", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
open("/www/web/tmp/unicorn.pid.oldbin", O_RDONLY|O_LARGEFILE) = 4
fstat64(4, {st_mode=S_IFREG|0644, st_size=6, ...}) = 0
fstat64(4, {st_mode=S_IFREG|0644, st_size=6, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb777c000
_llseek(4, 0, [0], SEEK_CUR) = 0
read(4, "32177\n", 4096) = 6
read(4, "", 4096) = 0
close(4) = 0
munmap(0xb777c000, 4096) = 0
unlink("/www/web/tmp/unicorn.pid.oldbin") = 0
open("/www/web/tmp/0.712879319799392.32177", O_RDWR|O_CREAT|O_EXCL|O_LARGEFILE, 0644) = 4
fcntl64(4, F_GETFL) = 0x8002 (flags O_RDWR|O_LARGEFILE)
fstat64(4, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb777c000
_llseek(4, 0, [0], SEEK_CUR) = 0
write(4, "32177\n", 6) = 6
rename("/www/web/tmp/0.712879319799392.32177", "/www/web/tmp/unicorn.pid") = 0
close(4) = 0
munmap(0xb777c000, 4096) = 0
waitpid(-1, 0xbf987ee4, WNOHANG) = 0
gettimeofday({1312941624, 186901}, NULL) = 0
gettimeofday({1312941624, 187058}, NULL) = 0
select(6, [5], NULL, NULL, {25, 0}^C <unfinished ...>
Process 32177 detached
Best Answer
I found the solution to this problem.
It turns out that there were messages that I missed in my logfile that were indicating that the startup process of the new child failed. The reason for this failure is that I did not declare unicorn in my Gemfile.
For some reason, it seems that it is possible to start unicorn from the command line without unicorn existing in the Gemfile, but it isn't possible for Unicorn to start itself without being mentioned in the Gemfile.
I'd love to hear an explanation for this from someone who knows why this is, but as it stands my problem has been solved. I am able to successfully cycle unicorn.