When maintaining an API that is used by third parties it is inevitable that you will need to make changes. The level of complexity will depend on the type of change that is occuring. These are the main scenarios that come up:
- New Functionality Added to Existing API
- Old Functionality Deprecated From API
- Existing Functionality In API Changing In Some Way
New Functionality Adding To Existing API
This is the easiest scenario to support. Adding new methods to an API should not require any changes to existing clients. This is safe to deploy for the clients that need the new functionality as it has no updates for any existing client.
Old Functionality Deprecated From API
In this scenario you must communicate to existing consumers of your API that the functionality will not be supported long term. Until you either drop support for the old functionality (or until all clients have upgraded to the new functionality) you must keep the old and new API functionality at the same time. If it's a library that's provided most languages have a way to mark old methods as obsolete/deprecated. If it's a third party service of some kind it's usually best to have different endpoints for the old/new functionality.
Existing Functionality In API Changing In Some Way
This scenario will depend on the type of change. If input parameters no longer need to be used then you can just update the service/library to ignore the now extra data. In a library it would be to have the overloaded method internally call the new method that requires fewer parameters. In a hosted service you have the endpoint ignore extra data and the it can service both types of clients and run the same logic.
If the existing functionality needs to add new required elements then you must have two endpoints/methods for your service/library. Until clients update you need to support both versions.
Other Thoughts
Rather, the API is likely to extend the private objects we are using for our base API, but then we run into the same problem because added properties would also be available in the public API when they are not supposed to be.
Do not expose internal private objects through your library/service. Create your own types and map the the internal implementation. This will allow you make internal changes and minimize the amount of updating the external clients need to do.
The problem is more that it seems like many or most changes require breaking the public API if the objects aren't more separated, but I don't see a good way to do that without duplicating code.
The API whether it is a service or library needs to be stable at the integration point with the clients. The more time you take to identify what the inputs and outputs should be and keep them as separate entities will save you a lot of headaches down the road. Make the API contract its own separate entity and map to the classes that provide the real work. The time saved when internal implementations change should more that offset the extra time it took to define the extra interface.
Don't view this step as "duplicating code". While similar, they are separate entities that are worth the while to create. While external API changes almost always require a corresponding change to internal implementation, internal implementation changes shouldn't always change the external API.
Example
Suppose you are providing a payment processing solution. You are using PaymentProviderA to do credit card transactions. Later on you get a better rate through PaymentProviderB's payment processor. If your API exposed Credit Card / Bill Address fields of your type instead of PaymentProviderA's representation then the API change is 0 as the interface remained the same (hopefully anyway, if PaymentProviderB requires data that was not required by PaymentProviderA then you need to chose to either support both or keep the worse rate with PaymentProviderA).
I can't speak for Twitter and Github, but I can speak for Stack Overflow.
I wouldn't worry about "scale" as an abstract possible idea. I would worry about your present and provable needs from the next 6-12 months. Scaling requires much work, measurement and tweaking which can't really be anticipated. For example, Stack Overflow needs only a few parts to be split into services. Most of the application is happily monolithic.
One thing that Github certainly does and Twitter does not is rendering on the server, as well as on the client. We also do this. The reason is performance: no one likes to wait for a page to load and then wait for the content to load. It has been shown that even adding 0.1s to a page load time negatively impacts engagement in a substantial way.
As a consequence of this, I would not worry too much about the URL structure for the API. I would worry instead to build a shared framework that can be used on both the front end and the back end to access data, and then exposed as pages or APIs independently.
Best Answer
You should have a versioning strategy because that is key to independent evolvability, but it should be tied to Content-Type, not URLs or anything else.
Even in a closed-house setting, you should still strive to make all components of a system independently evolvable (isolated, modularised) — especially a distributed system like a client/server-based one. This both allows different teams to work on each at their own pace, and allows for different release cadences.
Why not in URLs?
URIs identify abstract things which cannot be versioned, like a user "Andy", or an invoice. A representation of that thing will have a particular serialisation, which can be versioned,
application/andys-api-v1+json.Your API (as with any website) is defined by three things. These are the only things that you need to document if your API is RESTful:
If a v1 client obtains a link to
/users/andyfrom a previous request, it can forward that to a v2 client, which can then make a request to the same URL to get data about the same Thing, but in a language (content-type) it can speak,application/andys-api-v2+json.The v1 and v2 clients might be different parts of the same program, in the midst of a development cycle. The key is that the clients both continue working throughout.