I'm shocked - and indeed appalled - at the number of answers here saying "don't update unless you have to". I've done that, and whilst it's easier in the short term, it burns like hell in the long run. More frequent, smaller updates are much, much easier to manage than occasional big ones, and you get the benefit of new features, bug fixes, and so on sooner.
I don't buy this idea that library changes are somehow more difficult to test than code changes. It's just the same - you're making a change to the codebase, and you need to validate it before you commit, and more deeply before you release. But you must already have processes to do this, since you're making code changes!
If you're working in iterations, of two to four weeks length, i would suggest making updating libraries a once per iteration task, to be done as soon as possible after the start, when things are a little more relaxed than just before an iteration deadline, and the project has more capacity to absorb change. Get someone (or a pair if you do pair programming) to sit down, look at which libraries have been updated, and try bringing each one in and running a rebuild and test. Budget half a day to a day for it each iteration, perhaps. If things work, check in the changes (i'm assuming you keep libraries in source control, as we do; i'm not sure how you'd propagate the change in a controlled way if not). This will obviously be a lot easier if you have automated tests than if testing is entirely manual.
Now, the question is what you do if an update breaks things - do you spend time fixing it, or leave it out? I'd suggest leaning towards the latter; if it can be fixed in an hour, do it, but if an update is going to take significant work to integrate, then raise it as its own development task, to be estimated, prioritised, and scheduled just like any other. The chances are that unless it brings in some very crucial fix or improvement, the priority will be low, and you'll never get round to it. But you never know, by the time the next iterationly update day rolls round, the problem might have fixed itself; even if not, at least now you know that there's a roadblock on the update path, and it won't catch you by surprise.
If you're not doing iterations of that length, i would set up some kind of standalone schedule for updates - no longer than monthly. Is there some other project rhythm you could tie it to, like a monthly status review, or an architecture board meeting? Payday? Pizza night? Full moon? Whatever, you need to find something a lot shorter than a traditional release cycle, because trying to update everything in one go every 6-18 months is going to be painful and demoralising.
Needless to say, if you do stabilisation branches before releases, you wouldn't apply this policy to them. There, you'd only update libraries to get critical fixes.
If you only use a small subset of the third party API, it makes sense to write a wrapper - this helps with encapsulation and information hiding, ensuring you don't expose a possibly huge API to your own code. It can also help with making sure that any functionality you don't want to use is "hidden".
Another good reason for a wrapper is if you expect to change the third party library. If this is a piece of infrastructure you know you will not change, do not write a wrapper for it.
Best Answer
If nuget is not available, the cleanest way may be to provide an export script which pulls the source code of the needed library from their external repository (of the library vendor) into your working directory. This will work even if the SCC system of the external libs is different from Git (as long as it has a command-line interface). EDIT: this will work also when the source code is not available in an SCC system at all, just as ftp or http download, for example.
You can integrate that script into your build process, of course. When using Visual Studio, as you mentioned it, you can write either a classic Makefile (using NMake), or use MSBuild to call that script when the external lib is not in your working dir. Perhaps the most simple approach is to add that script to a prebuild event, the details depend much on how the rest of your build process is organized.