404 means "the resource you asked for doesn't exist." It's up to the server to decide when that response is appropriate. Google has apparently interpreted it to mean "your API request was malformed." Other REST APIs might also interpret it to mean "your request was well-formed, but you are asking for something which does not exist." This is why you need to read the API docs.
The "server," in turn, is (as your RFC quote indicates) anything that responds to HTTP requests appropriately. If you build a server out of a gigantic web application framework, that's your business. The only requirement is that the client gets the semantically-correct response in any given situation. It will often be the case that the web application is better suited to make that decision than (say) Apache's out-of-the-box behavior, but you can and should set this up in whatever way makes the most sense for your situation.
Interesting question.
Basically, we can reduce this down to the right way to classify things in terms analogous to OSI layers. HTTP is commonly defined as an Application Level protocol, and HTTP is indeed a generic Client/Server protocol.
However, in practice, the server is almost always a relaying device, and the client is a web browser, responsible for interpreting and rendering content: The server just passes things on to an arbitrary application, and that applications sends back arbitrary scripts which the browser is responsible for executing. The HTTP interaction itself--the request/response forms, status codes, and so on--is mostly an affair of how to request, serve, and render arbitrary content as efficiently as possible, without getting in the way. Many of the status codes and headers are indeed designed for these purposes.
The problem with trying to piggyback the HTTP protocol for handling application-specific flows, is that you're left with one of two options: 1) You must make your request/response logic a subset of the HTTP rules; or 2) You must reuse certain rules, and then the separation of concerns tends to get fuzzy. This can look nice and clean at first, but I think it's one of those design decisions you end up regretting as your project evolves.
Therefore, I would say it is better to be explicit about the separation of protocols. Let the HTTP server and the web browser do their own thing, and let the app do its own thing. The app needs to be able to make requests, and it needs the responses--and its logic as to how to request, how to interpret the responses, can be more (or less) complex than the HTTP perspective.
The other benefit of this approach, which is worth mentioning, is that applications should, generally speaking, not be dependent upon an underlying transport protocol (from a logical point of view). HTTP itself has changed in the past, and now we have HTTP 2 kicking in, following SPDY. If you view your app as no more than an HTTP functionality plugin, you might get stuck there when new infrastructures take over.
Best Answer
Regarding PUT, but applies to POST as well. The HTTP specification section 9 is a little empty on rules or even advice (SHOULD) when it comes to the scenario that you are describing. The line relevant to your question is most closely covered by:
I do not think I would lose any sleep over it, but I would ask, what do you gain by adding the chunk of JSON into the response? You've just bulked out (OK, bulked might be overkill!) the response repeating less accurately what the status code should already have told you. If your PUT resulted in a new object return 201 (as is the intention of PUT), if it updated an object return 204.
Incidentally, API aside, instead of 200, if you don't return anything use 204.
Assuming that you are developing a set of RESTful interfaces, there is no standard per se, so whatever you do, document it well,, provide examples and everything will be alright.