I would assign a couple of low priority bugs the first day, that way no one is screaming if they aren't done right away giving the new developer some time to get familiar with the code base.
The most critical thing to do is to have a code review of all of his work the first couple of weeks. You don't want to find out that the guy is going in the wrong direction or not following company coding standards months into things. It is better to make sure he knows what is expected from the start, and code reviews ensures this. Of course I think code reviews are good for all employees (We review 100% of our code before deployment), but they are critical for new employees and should be done in person where you can answer questions and refer them to documentation they may not have seen yet if need be.
What you don't want is a new guy coming in and using a different style from the rest of you. People often try to keep using the code style of their previous job even when it conflicts with the code style used at the new place which can create confusion and annoyance on the part of the other developers.
One thing I have noticed even with experienced developers is that some of them are not as good as they seemed to be in the interview, code review will help you find this out fast, so you can fix it. It will also encourage them to actually get something done, I have seen new employees who are not code reviewed drag out a project without showing what they were doing to anybody and then leave a week before the deadline they knew they were not going to hit because they were in over their heads and had not actually completed any part of the project. Better to check early and often with new people until you are really sure that they are working out.
Also, it is normal for the new guy to be appalled at the state of your legacy project. It's not designed the way he thinks it should have been. Expect this, hear him out and don't automatically dismiss everything he says. In particular, this person appears to have more experience than you or the other developers, he may see things you hadn't considered. However, as a manager, you have to balance the proposed changes against the current workload and deadlines. You all may want to invest some time in learning how to refactor existing code and invest some hours in your time estimates to do that especially if the new guy has some valid concerns. You probably can't support a total re-write (many people who come in new think we should start over and do it better), but you can create a refactoring plan to fix the worst of the problems if there are any that he brings up.
If you have some time where he is not expected to be fully contributing (and fully accounting for his time by client), it might also be a time when he can start on some of those refactoring things that you have wanted to do but haven't had time to do. Sometimes, it is a good thing to use the new person training period to address some things that aren't in the project plan. They can learn the code base and if what they want to do doesn't work, you haven't affected the existing schedules because you hadn't factored them into the existing schedule yet. And if it does work, you might have a big win making future maintenance easier or security better or whatever the problem is.
I do not think it's good that they make a contest out of finding the most bugs. While it is true that their job is to find bugs, their job is not "find the most bugs". Their goal isn't to find the most, their goal is to help improve the quality of the software. Rewarding them for finding more bugs is about the same as rewarding a programmer for writing the most lines of code, rather than the highest quality code.
Turning it into a game gives them an incentive to focus on finding many shallow bugs, rather than finding the most critical bugs. As you mention in your edit, this is exactly what is happening in your organization.
One could argue that any bug they find is fair game, and that all bugs need to be discovered. However, given that your team likely has limited resources, would you rather have a tester focus several hours or days probing deeply into your system trying to find really big bugs, or spend several hours or days skipping through the app looking for typographical errors and small errors in the alignment of objects on a page?
If the company really wants to make a game out of it, give the developers the power to add points to a bug. "stupid bugs" get negative points, hard to find bugs with well written reports get multiple points. This then moves the incentive from "find the most" to "be the best at doing your job". However, this isn't recommended either, because a programmer and QA analyst could work together to artificially pad their numbers.
Bottom line: don't make a game out of finding bugs. Find ways in your organization to reward good work and leave it at that. Gamification rewards people for reaching a goal. You don't want a QA analyst to have the goal of "find the most bugs", you want their goal to be "improve the quality of the software". Those two goals are not the same.
Best Answer
The answer to this question depends heavily on the business goals, as well as the client.
Enterprise:
If you are doing business with an enterprise level client who is well established in the marketplace, they are less flexible and cannot adapt to changes as quickly. Therefore, stability is an absolute requirement in most cases. There are exceptions for research and development and entering new verticals. Faster finishes first in some cases.
These types of clients generally understand that good software takes time to develop, and will work with you to try and meet the goals.
Startups:
For a new startup, the rules are drastically different. As a startup, you need to know right away if the product that you are building will indeed fulfill a need as your marketing research predicted. For a startup, getting a prototype out in the market as fast as possible can garner lots of valuable feedback about the direction the product should go.
It can also establish you as the market leader, helping you gain valuable market share in a new vertical before it becomes saturated with competition.
Since startups are small, flexible, and can rapidly adapt to change, this model works out best for them.
In summary, there are other factors to consider, but the main idea is that every project is different and will have different quality and time to market goals. It's up to executive leadership to determine an effective business strategy that includes thorough analysis of the opportunity costs of choosing one method over another.