Linux – WSGI cannot access a file, but permissions are correct

I am debugging a problem where MoinMoin on CentOS is throwing a permissions error, but I can't track down where the problematic file / directory is.

I ran strace -vp <pid> on the apache pid; when I have the problem I see this:

epoll_wait(10, {{EPOLLIN, {u32=3487534344, u64=140367313734920}}}, 2, 10000) = 1
accept4(6, {sa_family=AF_INET6, sin6_port=htons(52621), inet_pton
    (AF_INET6, "::ffff:105.193.30.91", &sin6_addr), sin6_flowinfo=0, 
    sin6_scope_id=0}, [28], SOCK_CLOEXEC) = 11
## Later on...
read(7, 0x7fffa658ad7f, 1)              = -1 EAGAIN (Resource temporarily 
    unavailable)

However, since apache is already running, I see no corresponding open() on the file referred to as 7; thus I see the permissions problem, but I still don't know which file is the problem.

I know I could try to catch all the file opens when I respawn apache, but I'm hoping there is a way to map file 7 to a real filename… is there a way to do this?

EDIT 1:

Using @lain's guidance, I ran lsof | grep 266474069, but the results are unclear…

[root@lnxlmf moin]# ls -la /proc/9707/fd/7
lr-x------. 1 root root 64 Aug 28 15:39 /proc/9707/fd/7 -> pipe:[266474069]
[root@lnxlmf moin]#

[root@lnxlmf moin]# lsof | grep 266474069
httpd      9703      root    7r     FIFO                0,8          0t0  266474069 pipe
httpd      9703      root    8w     FIFO                0,8          0t0  266474069 pipe
httpd      9705    apache    7r     FIFO                0,8          0t0  266474069 pipe
httpd      9705    apache    8w     FIFO                0,8          0t0  266474069 pipe
httpd      9706    apache    7r     FIFO                0,8          0t0  266474069 pipe
httpd      9706    apache    8w     FIFO                0,8          0t0  266474069 pipe
httpd      9707    apache    7r     FIFO                0,8          0t0  266474069 pipe
httpd      9707    apache    8w     FIFO                0,8          0t0  266474069 pipe
httpd      9733    apache    7r     FIFO                0,8          0t0  266474069 pipe
httpd      9733    apache    8w     FIFO                0,8          0t0  266474069 pipe
[root@lnxlmf moin]#

I see that this is a FIFO pipe, but what does this mean for my system configuration? How can I trace the root cause for the EAGAIN issue?

EDIT 2:

Thanks to @Alan Curry, running strace -fp <pid_of_wsgi_proc> seems to get me a bit farther…

[pid  9731] stat("/opt/moin/share/moin/wikiconfig.py", {st_mode=S_IFREG|0770, st_size=6463, ...}) = 0
[pid  9731] stat("/opt/moin/share/moin/data/pages", {st_mode=S_IFDIR|0770, st_size=4096, ...}) = 0
[pid  9731] access("/opt/moin/share/moin/data/pages", R_OK|W_OK|X_OK) = -1 EACCES (Permission denied)

However, both apache and wsgi run as the apache user…

[root@lnxlmf moin]# ps auxw | grep -E "apache|wsgi"
apache   10187  0.0  0.1 373488  5884 ?        Sl   17:18   0:00 moin_http_wsgi
apache   10188  0.0  0.1 373488  5884 ?        Sl   17:18   0:00 moin_https_wsgi
apache   10189  0.0  0.1 185096  5824 ?        S    17:18   0:00 /usr/sbin/httpd
root     10243  0.0  0.0 103240   848 pts/1    S+   17:21   0:00 grep -E apache|wsgi
[root@lnxlmf moin]#

Yet, when I run the following commands and restart apache, I still can't fix the problem…

[root@lnxlmf moin]# pwd
/opt/moin/share/moin
[root@lnxlmf moin]# chown -R apache:apache data/
[root@lnxlmf moin]# sudo chmod -R ug+rwx data/
[root@lnxlmf moin]# sudo chmod -R o-rwx data/

I am using this in my wiki http config:

<VirtualHost *:443>
  ServerName netwiki.foo.com
  DocumentRoot /opt/moin/share/moin
  WSGIScriptAlias / /opt/moin/share/moin/server/moin.wsgi
  WSGIDaemonProcess moin_https display-name=moin_https_wsgi \
      user=apache group=apache \
      processes=1 threads=10 maximum-requests=1000 umask=0007
  WSGIProcessGroup moin_https
  WSGIApplicationGroup %{GLOBAL}

  # Generate with...
  # openssl req -new -x509 -days 365 -nodes -out netwiki.pem -keyout netwiki.key
  SSLEngine on
  SSLCertificateFile /etc/httpd/ssl/netwiki.pem
  SSLCertificateKeyFile /etc/httpd/ssl/netwiki.key
</VirtualHost>