Amazon-web-services – Amazon SimpleDB vs Amazon DynamoDB

The existing SimpleDB API does not lend itself naturally to being a distributed counter. But it certainly can be done.

Working strictly within SimpleDB there are 2 ways to make it work. An easy method that requires something like a cron job to clean up. Or a much more complex technique that cleans as it goes.

The Easy Way

The easy way is to make a different item for each "hit". With a single attribute which is the key. Pump the domain(s) with counts quickly and easily. When you need to fetch the count (presumable much less often) you have to issue a query

SELECT count(*) FROM domain WHERE key='myKey'

Of course this will cause your domain(s) to grow unbounded and the queries will take longer and longer to execute over time. The solution is a summary record where you roll up all the counts collected so far for each key. It's just an item with attributes for the key {summary='myKey'} and a "Last-Updated" timestamp with granularity down to the millisecond. This also requires that you add the "timestamp" attribute to your "hit" items. The summary records don't need to be in the same domain. In fact, depending on your setup, they might best be kept in a separate domain. Either way you can use the key as the itemName and use GetAttributes instead of doing a SELECT.

Now getting the count is a two step process. You have to pull the summary record and also query for 'Timestamp' strictly greater than whatever the 'Last-Updated' time is in your summary record and add the two counts together.

SELECT count(*) FROM domain WHERE key='myKey' AND timestamp > '...'

You will also need a way to update your summary record periodically. You can do this on a schedule (every hour) or dynamically based on some other criteria (for example do it during regular processing whenever the query returns more than one page). Just make sure that when you update your summary record you base it on a time that is far enough in the past that you are past the eventual consistency window. 1 minute is more than safe.

This solution works in the face of concurrent updates because even if many summary records are written at the same time, they are all correct and whichever one wins will still be correct because the count and the 'Last-Updated' attribute will be consistent with each other.

This also works well across multiple domains even if you keep your summary records with the hit records, you can pull the summary records from all your domains simultaneously and then issue your queries to all domains in parallel. The reason to do this is if you need higher throughput for a key than what you can get from one domain.

This works well with caching. If your cache fails you have an authoritative backup.

The time will come where someone wants to go back and edit / remove / add a record that has an old 'Timestamp' value. You will have to update your summary record (for that domain) at that time or your counts will be off until you recompute that summary.

This will give you a count that is in sync with the data currently viewable within the consistency window. This won't give you a count that is accurate up to the millisecond.

The Hard Way

The other way way is to do the normal read - increment - store mechanism but also write a composite value that includes a version number along with your value. Where the version number you use is 1 greater than the version number of the value you are updating.

get(key) returns the attribute value="Ver015 Count089"

Here you retrieve a count of 89 that was stored as version 15. When you do an update you write a value like this:

put(key, value="Ver016 Count090")

The previous value is not removed and you end up with an audit trail of updates that are reminiscent of lamport clocks.

This requires you to do a few extra things.

1. the ability to identify and resolve conflicts whenever you do a GET
2. a simple version number isn't going to work you'll want to include a timestamp with resolution down to at least the millisecond and maybe a process ID as well.
3. in practice you'll want your value to include the current version number and the version number of the value your update is based on to more easily resolve conflicts.
4. you can't keep an infinite audit trail in one item so you'll need to issue delete's for older values as you go.

What you get with this technique is like a tree of divergent updates. you'll have one value and then all of a sudden multiple updates will occur and you will have a bunch of updates based off the same old value none of which know about each other.

When I say resolve conflicts at GET time I mean that if you read an item and the value looks like this:

      11 --- 12
     /
10 --- 11
     \
       11

You have to to be able to figure that the real value is 14. Which you can do if you include for each new value the version of the value(s) you are updating.

It shouldn't be rocket science

If all you want is a simple counter: this is way over-kill. It shouldn't be rocket science to make a simple counter. Which is why SimpleDB may not be the best choice for making simple counters.

That isn't the only way but most of those things will need to be done if you implement an SimpleDB solution in lieu of actually having a lock.

Don't get me wrong, I actually like this method precisely because there is no lock and the bound on the number of processes that can use this counter simultaneously is around 100. (because of the limit on the number of attributes in an item) And you can get beyond 100 with some changes.

Note

But if all these implementation details were hidden from you and you just had to call increment(key), it wouldn't be complex at all. With SimpleDB the client library is the key to making the complex things simple. But currently there are no publicly available libraries that implement this functionality (to my knowledge).

The problem is having wrong mod on the file.

Easily solved by executing -

chmod 400 mykey.pem

Taken from Amazon's instructions -

Your key file must not be publicly viewable for SSH to work. Use this command if needed: chmod 400 mykey.pem

400 protects it by making it read only and only for the owner.