Here's the deal: We have to tell both NuGet and TFS to ignore the packages, because NuGet is trying to do source-control related stuff that it absolutely shouldn't be doing (bad form, Microsoft!). So you have to do two things.
First, add a file named .tfignore to the solution folder (note the lack of s after the tf). It's contents should be as follows:
packages
That tells TFS to ignore your packages folder. Now, you would think that this would also ignore the repositories.config file. But it won't. Why? Who knows, the ways of Microsoft are strange and mysterious. Actually, I think it's part of the NuGet stuff I outline below, but if that ever gets fixed in the future and you want to keep the repositories.config file instead of letting VS regenerate it, you should be able to use this:
packages
!packages
epositories.config
OK, so now thanks to our .tfignore file, TFS is ignoring your packages. Everything is fine, right? WRONG, because NuGet is mucking around with your source control and adding the packages to your pending changes. So now let's tell NuGet to cut it out already.
Create a folder called .nuget in the root of your solution folder.1 Now, create a file called NuGet.config, and put it in this new folder2. It's contents should look like this:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<solution>
<add key="disableSourceControlIntegration" value="true" />
</solution>
</configuration>
And now your packages should stay out of source control. Just remember to add the NuGet.configand .tfignore files to source control so they never get lost.
EDIT: If you're having issues, you may want to delete your packages folder, check in that change, and then go through the steps above.
ALSO EDIT: It looks like this won't happen with newer versions of Nuget. So maybe if you switch to VS/TFS 2017 this issue will clear up without jumping through the above hoops.
1. Add the folder using Source Control Explorer; right-click the solution->Add folder->.nuget
2. When I figured this out using VS 2013, I found the NuGet.config had to go in the .nuget folder. Even if you already have a NuGet.config file in the root of your solution folder (because, say, your company has an internal nuget feed). However, some in the comments have indicated that it works fine in the solution root in VS 2015. Personally, I switched to using TFS in git mode, so I can't test. Additionally, if you do have a custom feed, ensure that you have both the custom feed and nuget.org as keys in the Nuget.config file, or sometimes TFS will randomly decide it can't restore the packages.
https://stackoverflow.com/questions/24143925/get-tfs-to-ignore-my-packages-folder#