Troubleshooting MSBuild failures

I am trying to convert my shop from using VSS to TFS.

Our current codebase consists of a large number of Visual Studio 2008 solutions and I've noticed some of them seem to cause MSBuild to fail, but I cannot figure out why. The actual solutions themselves and the projects they hold build correctly, but the overall build still fails.

The ErrorsAndWarnings.txt and Release.txt file show no errors, just warnings (various little things in the code we need to clean up)

The BuildLog.txt file is, of course, enormous and has tons and tons of difficult-to-comprehend text in it. As best I can tell though, when it gets to one of these SLN files that give it problems, MSBuild just flat out crashes and dies.

Task "MSBuild"
  Global Properties:
    Configuration=Release
    Platform=Any CPU
    OutDir=C:\TFS\REPOSITORY\Full\Binaries\Release\
    PublishDir=C:\TFS\REPOSITORY\Full\Binaries\Release\
    SkipInvalidConfigurations=true
    RunCodeAnalysis=false
    VCBuildOverride=C:\TFS\REPOSITORY\Full\Sources\Code\Solution\Solution.sln.Release.vsprops
    VCBuildAdditionalLibPaths=
    VCBuildAdditionalOptions=
    VCBuildToolPath=
    VCBuildUseEnvironment=
    TeamBuildConstants=_TEAM_BUILD_
    TargetsNotLogged=GetTargetPath;GetNativeManifest;GetCopyToOutputDirectoryItems

C:\Program Files\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1008,5): 
error MSB4018: The "MSBuild" task failed unexpectedly.
error MSB4018: System.NullReferenceException: Object reference not set to an instance of an object.
error MSB4018:    at Microsoft.Build.BuildEngine.SolutionWrapperProject.AssignDependencyLevel(ProjectInSolution project, SolutionParser solution, Dictionary`2 projectsByDependencyLevel)
error MSB4018:    at Microsoft.Build.BuildEngine.SolutionWrapperProject.AssignDependencyLevel(ProjectInSolution project, SolutionParser solution, Dictionary`2 projectsByDependencyLevel)
error MSB4018:    at Microsoft.Build.BuildEngine.SolutionWrapperProject.AssignDependencyLevels(SolutionParser solution, Dictionary`2 projectsByDependencyLevel)
error MSB4018:    at Microsoft.Build.BuildEngine.SolutionWrapperProject.CreateSolutionProject(SolutionParser solution, Project msbuildProject, BuildEventContext projectBuildEventContext, String wrapperProjectToolsVersion, Engine parentEngine, String solutionProjectCache)
error MSB4018:    at Microsoft.Build.BuildEngine.SolutionWrapperProject.Generate(SolutionParser solution, Project msbuildProject, String toolsVersionOverride, BuildEventContext projectBuildEventContext)
error MSB4018:    at Microsoft.Build.BuildEngine.Project.Load(String projectFileName, BuildEventContext buildEventContext, ProjectLoadSettings projectLoadSettings)
error MSB4018:    at Microsoft.Build.BuildEngine.Engine.GetMatchingProject(Project existingProject, String projectFullPath, BuildPropertyGroup globalPropertiesToUse, String toolsVersion, String[] targetNames, BuildEventContext buildEventContext, Boolean toolsVersionPeekedFromProjectFile)
error MSB4018:    at Microsoft.Build.BuildEngine.Engine.BuildProjectFileInternal(BuildRequest buildRequest)
error MSB4018:    at Microsoft.Build.BuildEngine.Engine.EngineBuildLoop(BuildRequest terminatingBuildRequest)
error MSB4018:    at Microsoft.Build.BuildEngine.TaskExecutionModule.BuildProjectFile(Int32 handleId, String[] projectFileNames, String[] targetNames, IDictionary[] globalPropertiesPerProject, IDictionary[] targetOutputsPerProject, EngineLoggingServices loggingServices, String[] toolsVersions, Boolean useResultsCache, Boolean unloadProjectsOnCompletion, BuildEventContext taskContext)
error MSB4018:    at Microsoft.Build.BuildEngine.EngineProxy.BuildProjectFilesInParallel(String[] projectFileNames, String[] targetNames, IDictionary[] globalProperties, IDictionary[] targetOutputsPerProject, String[] toolsVersions, Boolean useResultsCache, Boolean unloadProjectsOnCompletion)
error MSB4018:    at Microsoft.Build.Tasks.MSBuild.ExecuteTargets(ITaskItem[] projects, Hashtable propertiesTable, ArrayList targetLists, Boolean stopOnFirstFailure, Boolean rebaseOutputs, IBuildEngine2 buildEngine, TaskLoggingHelper log, ArrayList targetOutputs, Boolean useResultsCache, Boolean unloadProjectsOnCompletion, String toolsVersion)
error MSB4018:    at Microsoft.Build.Tasks.MSBuild.BuildProjectsInParallel(Hashtable propertiesTable, ArrayList targetLists, Boolean success, Boolean[] skipProjects)
error MSB4018:    at Microsoft.Build.Tasks.MSBuild.Execute()
error MSB4018:    at Microsoft.Build.BuildEngine.TaskEngine.ExecuteInstantiatedTask(EngineProxy engineProxy, ItemBucket bucket, TaskExecutionMode howToExecuteTask, ITask task, Boolean& taskResult)
Done building target "CoreCompileSolution" in project "TFSBuild.proj" -- FAILED.
Done Building Project "C:\TFS\REPOSITORY\Full\BuildType\TFSBuild.proj" (CompileSolution target(s)) -- FAILED.
Done executing task "MSBuild" -- FAILED.
Done building target "CoreCompileConfiguration" in project "TFSBuild.proj" -- FAILED.
Done Building Project "C:\TFS\REPOSITORY\Full\BuildType\TFSBuild.proj" (CompileConfiguration target(s)) -- FAILED.
Done executing task "MSBuild" -- FAILED.
Done building target "CoreCompile" in project "TFSBuild.proj" -- FAILED.
Done Building Project "C:\TFS\REPOSITORY\Full\BuildType\TFSBuild.proj" (CoreCompile target(s)) -- FAILED.
Done executing task "MSBuild" -- FAILED.
Done building target "CallCompile" in project "TFSBuild.proj" -- FAILED.

I know pasting incredibly verbose error messages in Stack Overflow is lame but the core thing to take from the above is MSBuild is crashing and I can't figure out why. It gives the usual "Object reference not set…" error, which is pretty much universally useless when it's not your code or program you're troubleshooting.

I see that it's referencing TeamFoundation.Build.targets. I look up that file that file but it doesn't really jump out at me what the problem could be. (the line after the comment is the (1008,5) referenced above)

<!-- Build using MSBuild task -->
<MSBuild BuildInParallel="$(BuildSolutionsInParallel)"
         Projects="$(Solution)"
         Properties="Configuration=$(Configuration);Platform=$(Platform);$(OutDirOption);$(PublishDirOption);SkipInvalidConfigurations=$(SkipInvalidConfigurations);$(FxCopDirOption);$(ReferencePathOption);$(CodeAnalysisOption);
                         VCBuildOverride=$(VsPropsFile);VCBuildAdditionalLibPaths=$(VCBuildAdditionalLibPaths);VCBuildAdditionalOptions=$(VCBuildAdditionalOptions);VCBuildToolPath=$(VCBuildToolPath);VCBuildUseEnvironment=$(VCBuildUseEnvironment);
                         TeamBuildConstants=$(TeamBuildConstants);TargetsNotLogged=$(TargetsNotLogged);$(CustomPropertiesForBuild);$(CustomProperties)"
         Targets="$(Targets)"
         StopOnFirstFailure="$(StopOnFirstFailure)">
  <Output TaskParameter="TargetOutputs" ItemName="CompilationOutputs" />
</MSBuild>

At first I thought this was telling me that one of the items that need to be substituted in the node was missing, but the other solutions which don't have issues have similar/identical setups.

I notice some of the items in the call stack of the MSBuild deal with targets and the issue seems to stem from the TeamFoundation.Build.targets file but I'm stuck as to what to do next to troubleshoot it.

Has anyone run into this before?