I have an MVC 5 ASP.net app that is working perfectly. A colleague of mine has made a project that reads in data from files and churns some of it to output some calculations for me to display in the web application. I call his methods and I get the data back and display on screen.
As part of his project he has implemented a kind of caching system, it's not using any of the caching tools in. NET, it's simply holding some of the files required for calculating the numbers in memory as these take a long time to load in and are definitely the bottle neck.
So I need to do two things
1) Have one instance of the class within the project so it makes use of the in memory files.
2) Setup a worker process that runs at 7am everyday to load those files into memo.
For number one I have instantiated my colleague's class as a static class and this seems to work some of the time but it's unreliable for some reason I'm guessing due to app pool timeouts, I'm also worried about the effectiveness of this and whether or not I will have issues with multiple users sending concurrent requests. Is there a better way to achieve this? I've looked at various .NET caching libraries so I could get my colleague to implement a proper cache but for now I'm assuming I can't ask him. I'm a little confused as to why in MVC every time a user sends a new request a new controller is instantiated hence instantiating my colleague's class again and losing the in-memory files?
For number 2 I've read about webbackgrounder a library for ASP.net that can run background tasks. All I need to do is whilst the app is running on IIS then run a method everyday at 7am.
Update:
I can't save the output because I need to calculate these vlaues on the fly, there's about 10 different inputs into the calculation that can change to be any number of different values.
I can indeed put the input files into a database but that doesn't solve my problem, it wouldn't be any faster than loading them in from files, which takes too long as it is, I want to hold these items in memory hence the cache approach.
Best Answer
Deploy your colleague's library as a cached microservice
Assuming you can't change your friend's library at all, you don't want to use a database, and you want to maintain a reasonable level of process isolation, here's the right way to do it:
Write a bit of glue to expose your colleague's library as a RESTful microservice. If you've never done this before, start here. It shouldn't take more than 100 lines of code to wrap one function call.
Use the uniform interface to manage cache and its duration.
Set up ASP.NET output caching. You can do it programmatically or use the OutputCacheAttribute, whichever is easier for you. Set the cache to an absolute expiration of 7:01 am.
Set IIS to enable output caching per these instructions.
Set the application pool for this service to recycle at 6:59 am.
Modify your web site to utilize the new service.
If your colleague's code takes a long time to run, the microservice may not be responsive when handling its very first request. To deal with this, you need to "warm up" the service. Set up warmup using IIS auto-start; details can be found here.
With this approach, your colleague's code is completely isolated from yours, and the caching logic simply leverages standard features in IIS. With the caching and recycle scheme set up properly, you should have seamless transitions when the program is recycled, since IIS will continue to serve requests in flight from the old cache while building up the new cache. We set the cache expiration to exactly 7:01 am (as opposed to letting the app pool recycle take care of it) just in case there is some other network node looking at your cache headers, e.g. a proxy server that might cache the response if it isn't encrypted.