Flush Magento Cache - This clears the cache (var/cache) of any items that Magento knows it has created.
Flush Cache Storage - Clears everything in var/cache, regardless of how those files in there were created.
So, if you want to be safe you're clearing everything, you may choose "Flush Cache Storage" which will essentially clear out var/cache.
For the compiler, I would recommend flushing the Magento cache after enabling compilation and running the compilation process. This ensures that the cache is cleared of any non-compiled data.
When disabling compilation, I would disable it first, then flush the Magento cache afterwards. This again ensures the cache is clear of any compiled data.
Unless you are testing things a lot, I would always recommend leaving the caches on. Compilation can be hit or miss in terms of performance. I've seen it make things faster, and many times seen compilation make things slower and cause issues with 3rd party extensions. I'd recommend getting a baseline for a category page load time (using Firebug / developer tools) with compilation off, then again with compilation on, and see if there is a large difference.
You'd likely be better off using things like an opcode cache in PHP, proper MySQL query caching, combining css/js files, using gzip compression, using a Full Page Cache extension, and proper settings for browser caching of files.
What I believe you will be seeing is what I have seen when I initially set up a single Redis instance for both cache and session. I was seeing a random 2.5 second timeout when accessing the session data and found a couple of great articles from Colin Mollenhour and Fabrizio Branca on these sorts of issues.
TL;DR - use different Redis instances for cache, session and FPC.
This is for a couple of reasons:
- Redis is single threaded, if you have multiple databases being accessed by multiple processes you are more likely to have contention and waits for access to the data. See Colin's "Re: Using Redis as a Cache Backend in Magento" for more information on this point
- You can create different configurations for each instance, allowing you to persist the sessions to disk, without persisting the cache data to disk. See Fabrizio's Redis Optimization post for more information on this.
edit: spelling mistake
Best Answer
I would say to check for all the crons which could re-enable all the caches in late-evening/night.
Check linux cron
crontab -lCheck magento's cron
select job_type, scheduled_at, executed_at from cron_schedule