I was actually once involved in working on a web app, which we eventually almost ended up marketing as a standard "desktop application". For some reason, Marketing got it into their heads that one of our major clients wanted the product to "appear to be a GUI application", so we created a little Windows app that just hosts the IE ActiveX control, and points to our web app (hiding the fact that it's actually a browser). So effectively, to an untrained eye it looked like the product was a standard GUI app.
Granted, this isn't exactly what you're asking (we were still pointing at a web app, and not hosting the whole thing locally) - but it's close enough. It would have been trivial - minus some of the remote web services it used - to set this up to just have the whole thing sitting on the local machine.
Here was the biggest problem though: look and feel. Especially feel.
People expect certain behaviours from rich GUI apps (drag and drop, native-feeling windows and dialogs, etc). It is extremely hard to get a genuinely native look and feel from a web frontend. There is simply a different user flow in what is expected when you open up a web app in a browser (eg. Gmail), as opposed to when you use a rich GUI application (eg. Outlook). In my experience, trying to equate the two is asking for trouble. If you put out a "GUI app" which acts like a web app, you're likely to be flooded with usability and LAF complaints.
TL;DR - Web apps and GUI apps have different looks and feels, and a different user culture to some extent. While it's technically possible to do something like this, from my experience, I wouldn't go there (again). At best you're likely to end up with a horrendous mix of client-side scripting that will be more difficult to learn, use and maintain than doing the whole thing as a normal GUI app in the first place. And people WILL complain about things "not quite feeling right" for a native GUI app. It's tempting to think that they won't - but they will.
We went for Java (Swing) plus some native parts via JNI. While the commercial demand for multiplatformness may be marginal today, the situation may be different in 5 years, and the app's (a scientific measurement app) life cycle is going to be more like 10+ years (its C++ predecessor, still used today, has source files dated 1991). As you wrote, Java beats .NET hands down in non-Windows environments, and if we ever need to switch away from Windows, it's just a matter of re-compiling some native parts, maybe fine-tuning the GUI looks, and checking that everything works.
If you're sure you'll be Windows only, and your app is going to live just few years, then .NET may be preferred - it looks and behaves more like a native app because it is. But as a long term investment I trust Java more. Swing may look a little less than perfect, the start-up time may be longer, everything is a bit sub-optimal because of the multiplatform abstraction layer, but at least it "just works".
Best Answer
Deckard had a pretty good response. I'd like to add that it's worth considering how this project will be maintained down the road. Consider how many of your coworkers (or developers in general) are familiar with Swing et al vs. how many are familiar with RCP. My organization had some headaches when we wanted to dust off an old RCP application. It turns out that most of the people involved with designing the project had either left the company or moved into non-developmental roles. Our in-house expertise was more in Swing, so this made things more difficult than they needed to be. We also had a desire to integrate parts of it with a separate, Swing-based application, and again the fact that the old app was an RCP application made things difficult.
I'm not saying you shouldn't create an RCP app, but these are some things you might want to consider first.