Ubuntu – PHP-FPM Workers messing up php.ini loading

I installed PHP 5.3.8 from source on Ubuntu Natty. I also enabled FPM during configuration.

However, I've been having issues installing PHP extensions. When I add the extension to my php.ini file (e.g. extension=apc.so) and restart PHP (i.e. /etc/init.d/php-fpm restart), I get an error like the one below:

Starting php-fpm PHP Warning:  Module 'apc' already loaded in Unknown on line 0
<br />
<b>Warning</b>:  Module 'apc' already loaded in <b>Unknown</b> on line <b>0</b><br />
PHP Warning:  Module 'geoip' already loaded in Unknown on line 0
<br />
<b>Warning</b>:  Module 'geoip' already loaded in <b>Unknown</b> on line <b>0</b><br />
PHP Warning:  Module 'imagick' already loaded in Unknown on line 0
<br />
<b>Warning</b>:  Module 'imagick' already loaded in <b>Unknown</b> on line <b>0</b><br />
PHP Warning:  Module 'memcache' already loaded in Unknown on line 0
<br />

My PHP-FPM conf is has the setting pm.start_servers = 5, so I have about 5 PHP-FPM workers active. Are they all trying to load the php.ini file when I restart PHP (hence the message that the module has already been loaded)?! If they are, is there a way I can prevent that to stop the errors as it the errors become fatal when I load Zend Loader?

Thanks in advance.