Where and what are the resources?
REST is all about addressing resources in a stateless, discoverable manner. It does not have to be implemented over HTTP, nor does it have to rely on JSON or XML, although it is strongly recommended that a hypermedia data format is used (see the HATEOAS principle) since links and ids are desirable.
So, the question becomes: How does one think about synchronization in terms of resources?
What is bi-directional sync?**
Bi-directional sync is the process of updating the resources present on a graph of nodes so that, at the end of the process, all nodes have updated their resources in accordance with the rules governing those resources. Typically, this is understood to be that all nodes would have the latest version of the resources as present within the graph. In the simplest case the graph consists of two nodes: local and remote. Local initiates the sync.
So the key resource that needs to be addressed is a transaction log and, therefore, a sync process might look like this for the "items" collection under HTTP:
Step 1 - Local retrieves the transaction log
Local: GET /remotehost/items/transactions?earliest=2000-01-01T12:34:56.789Z
Remote: 200 OK with body containing transaction log containing fields similar to this.
itemId
- a UUID to provide a shared primary key
updatedAt
- timestamp to provide a co-ordinated point when the data was last updated (assuming that a revision history is not required)
fingerprint
- a SHA1 hash of the contents of the data for rapid comparison if updateAt
is a few seconds out
itemURI
- a full URI to the item to allow retrieval later
Step 2 - Local compares the remote transaction log with its own
This is the application of the business rules of how to sync. Typically, the itemId
will identify the local resource, then compare the fingerprint. If there is a difference then a comparison of updatedAt
is made. If these are too close to call then a decision will need to be made to pull based on the other node (perhaps it is more important), or to push to the other node (this node is more important). If the remote resource is not present locally then a push entry is made (this contains the actual data for insert/update). Any local resources not present in the remote transaction log are assumed to be unchanged.
The pull requests are made against the remote node so that the data exists locally using the itemURI
. They are not applied locally until later.
Step 3 - Push local sync transaction log to remote
Local: PUT /remotehost/items/transactions
with body containing the local sync transaction log.
The remote node might process this synchronously (if it's small and quick) or asynchronously (think 202 ACCEPTED) if it's likely to incur a lot of overhead. Assuming a synchronous operation, then the outcome will be either 200 OK or 409 CONFLICT depending on the success or failure. In the case of a 409 CONFLICT, then the process has to be started again since there has been an optimistic locking failure at the remote node (someone changed the data during the sync). The remote updates are processed under their own application transaction.
Step 4 - Update locally
The data pulled in Step 2 is applied locally under an application transaction.
While the above is not perfect (there are several situations where local and remote may get into trouble and having remote pull data from local is probably more efficient than stuffing it into a big PUT) it does demonstrate how REST can be used during a bi-directional synchronization process.
I've previously worked in an environment where we've had SSO and local developer environments.
The key problem that needs to be worked on with SSO and developer environments is that the domain cookie needs to be able to be retrieved when hitting the local dev environment.
Admittingly, part of this has to do with how we set up the environment (and it was many years ago that I was involved with this). It was a polyglot environment with a mixture of static html, old perl cgis, a weblogic server for Java EE, an iis server for some apps that needed to run with asp, and something that engineering ran. The way that SSO communicated this information was that a reverse proxy stuck into the http headers the authenticated user name and all associated access they had. This way, no matter who got it, they could look at the headers and continue from there.
First off, DNS was set up so that each developer had a host name in the 'dev.company.com' domain - thus 'sxu.dev.company.com' (for you) and 'jsmith.dev.comapny.com' for John Smith. All of these names (CNAMES) pointed to a single common dev reverse proxy (the reverse proxy had very little on it) that then looked at the virtual host name that was coming through and then forwarded out the request to the appropriate developer's machine. Note that the interactions with the SSO code was completely contained within the reverse proxy so that no additional libraries needed to be installed anywhere else.
The interaction with infrastructure was simply to add another cname to common.dev.company.com whenever we had a new hire, and then we would add their name to a file that was run through some m4 macros to generate the proper apache http.conf file for the reverse proxy (that did take some work the first time we did it).
With this idea in place, you may wish to consider setting up a reverse proxy that acts to just forward everything to the appropriate developer's machine over http (no matter what type of connection it received). https://sxu.ci.company.com/xyz
goes to the reverse proxy, which handles the https, and forwards it to your dev box at http://10.1.2.3/xyz
. The gotcha with this approach that you have to watch for is that if you have any absolute paths in the code, or the dev's server tries to be aware of where its installed and the protocol being used to generate a link that is the same format, things might go a bit wonky (thats a technical term).
This avoids the problem of setting up https locally (only one server has is running https) or modifying the sso server to go to a non-https url. You've got different setup in a different place.
I am unsure if your local dev boxes respond to access other than localhost (its a valid configuration), if so, those will need to be modified because the request is coming from the proxy in this case rather than the local browser.
I do want to point out the side benefit that it will mean that other devs can hit your environment (you want someone to reproduce a bug while you're watching the log files that you don't know how to repo - have them hit your server).
The second option is a not-non-standard approach (I do recall several production SSO systems being configured to allow a limited set of appUrl type parameters - in part so that the external person could use SSO and go to a specific page once logged in. Adding a mapping of 'dev->localhost:8000' would allow for this, however you will need to reconsider how to address the problem of localhost doesn't get company.com cookies. You will likely need to modify your local boxes to identify as localhost.company.com for that to work.
Best Answer
Option 1 is definitely the best route in my experience. I sometimes use Option 2, typically in scenarios such as hooking my local application into a remote database (that usually also has another instance of the application hooked up to it, do this with extreme caution).
Advantages of running all applications locally for development:
Running everything locally does incur a setup cost (depending on how complex your runtime dependencies are, this can be anywhere between a few minutes to several days of work), but even so, the overhead is a one-time effort.
To address this, you could write a startup script that launches both services together, or use something like
docker
anddocker-compose
for running your apps locally. Using docker-compose, if you register dependencies between docker containers in thedocker-compose.yml
file, the API will launch when you start the front-end app. This kind of application orchestration is exactly whatdocker-compose
is made for. Also on the plus side, with Docker you only have to solve the environment setup problem once, and it can be highly repeatable for other developers.The downside of Docker is that it can be tricky (or impossible) to hook an IDE debugger into the container depending on your runtime. In those cases it may be best to set up your local development environment natively on the machine and use the startup script approach for orchestration of services.