I have a PHP based CMS which makes requests to service provider (a java based ECM(enterprise content management) currently located in the same server as my PHP) and inserts the response to mysql.
What initially starts at a good rate degrades slowly to 1/10th of the original data fetch and insert speed which i check using the count of the records. I'm having troubles figuring out the reason for the degradation and need to find if the cURL request is delayed at the ECM end where response is generated (as XML) or when the response is returned back to my PHP end. There's no change in the server load w.r.t the database (mysql) and I have my own reasons to believe it couldn't a database optimization issue.
What I'd actually like to do is know if there's someway to track what requests are being made, when they reach the ECM when the response is given and when it reaches the PHP again (a sum total of each cURL call's activity). I checked my apache access log if they are logged as GET and POST requests but couldn't find them. The only option in code level is to set a verbose though i'm not sure on how useful the result would be. Hence wish to know possible options for the same.
Best Answer
You can take a packet capture of the process (with tcpdump or wireshark), later analyse it to see how much time it takes every http transaction. Bear in mind that you can also take packet dumps for the 127.0.0.1 interface, since it may be needed if the requests are being done to localhost.
In order to see the curl requests you can provide a specific user agent to the curl request and later on look for requests from that user agent in the apache logs.