Javascript – Display large amount of data to client through pagination

It seems what you need is a wrapper for all the parameters that define a page (say, pageNumber, pageSize, sortType, totalCount, etc.) and use this DataRequest object as the key for your caching mechanism. From this point you have a number of options to handle the cache:

• Implement some sort of timeout mechanism to refresh the cache (based on how often the data changes).
• Have a listener that checks database changes and updates the cache based the above parameters.
• If the changes are done by the same process, you can always mark the cache as outdated with every change and check this flag when a page is requested.

The first two might involve a scheduler mechanism to trigger on some interval or based on an event. The last one might be the simpler if you have a single data access point.

Lastly, as @DanPichelman mentioned, it can quickly become an overly complicated algorithm that outweighs the benefits, so be sure the gain in performance justify the complexity of the algorithm.

The DB backend is "insecure". It is possible to tamper the data (many persons have access to the machine, perhaps it's "in the cloud") [...]

What I want to be able to do is make very difficult for third parties that have read/write access to the DB machine [...]

Your goal shouldn't be to make it very difficult for third parties to modify the data from your database. Your goal should be to prevent anyone but the administrators to access both the server and the database.

What you are doing right now is like giving your bank account number and password to complete strangers and then ask how you make it difficult for those strangers to take your money. Well, you might think of not sharing the account information in the first place.

It's also unclear what “in the cloud” has to do with security. Most cloud providers such as Amazon or Microsoft do a great job of hosting virtual machines in a very secure manner.

Therefore:

• Get rid of those “third parties” who can SSH your server with su access. This is the wrong way of hosting web applications. Don't do that. And if you do, please, inform your users on the website that all personal information they give you can and will be shared with complete strangers.

• Use TLS. There are no reasons not to use one, especially since free services exist.

• Don't trust client-side. No, you can't sign JavaScript, and in general, on client-side, you can't protect your users neither from themselves, nor from any crap they install on their machines. Viruses would be able to inject themselves into a browser and modify JavaScript; if you do a mechanism which checks if JavaScript is tampered, they would be able to tamper the checking mechanism itself.

• While you can encrypt information on client-side, there are few good reasons limited to few cases where you would actually do it. The reason is that, once again, client machine shouldn't be trusted, because it's usually difficult for a user to secure his device. The server, on the other hand, can be secured very easily, and with TLS, man in the middle attacks could effectively be prevented.

Thus, ensure both your servers and the communication between the servers and the clients are secure.

A few notes:

Sadly to make the software compatible with cell phones I will need to keep the iterations of PBKDF2 low [...] (perhaps I could use Scrypt instead of PBKDF2... It should be more resistant to GPU attacks)

This is why you need to compute the hash on the server. If you reduce the number of iterations, you're making it easier for the attacker to bruteforce the hashes.

Searching for a different algorithm is a wrong solution here. Remember that an attacker will have powerful CPUs and GPUs. Your goal is to slow an attacker, and you do that by increasing the iterations.

After this I could add a two step verification (using the algorithm used by Google Authenticator, for example). The key of this would be shared between user and server.

Which means that your “third parties” will have full access to those keys. In this context, two-step verification makes things much worse: it gives you a sense of security, while providing absolutely no benefit.