git-clean - Remove untracked files from the working tree

Synopsis

git clean [-d] [-f] [-i] [-n] [-q] [-e <pattern>] [-x | -X] [--] <path>…​

Description

Cleans the working tree by recursively removing files that are not under version control, starting from the current directory.

Normally, only files unknown to Git are removed, but if the -x option is specified, ignored files are also removed. This can, for example, be useful to remove all build products.

If any optional <path>... arguments are given, only those paths are affected.

Step 1 is to show what will be deleted by using the -n option:

# Print out the list of files and directories which will be removed (dry run)
git clean -n -d

Clean Step - beware: this will delete files:

# Delete the files from the repository
git clean -f

• To remove directories, run git clean -f -d or git clean -fd
• To remove ignored files, run git clean -f -X or git clean -fX
• To remove ignored and non-ignored files, run git clean -f -x or git clean -fx

Note the case difference on the X for the two latter commands.

If clean.requireForce is set to "true" (the default) in your configuration, one needs to specify -f otherwise nothing will actually happen.

Again see the git-clean docs for more information.

Options

-f, --force

If the Git configuration variable clean.requireForce is not set to false, git clean will refuse to run unless given -f, -n or -i.

-x

Don’t use the standard ignore rules read from .gitignore (per directory) and $GIT_DIR/info/exclude, but do still use the ignore rules given with -e options. This allows removing all untracked files, including build products. This can be used (possibly in conjunction with git reset) to create a pristine working directory to test a clean build.

-X

Remove only files ignored by Git. This may be useful to rebuild everything from scratch, but keep manually created files.

-n, --dry-run

Don’t actually remove anything, just show what would be done.

-d

Remove untracked directories in addition to untracked files. If an untracked directory is managed by a different Git repository, it is not removed by default. Use -f option twice if you really want to remove such a directory.

The Easy Way™

It turns out that this is such a common and useful practice that the overlords of Git made it really easy, but you have to have a newer version of Git (>= 1.7.11 May 2012). See the appendix for how to install the latest Git. Also, there's a real-world example in the walkthrough below.

0. Prepare the old repo

    cd <big-repo>
    git subtree split -P <name-of-folder> -b <name-of-new-branch>
   

Note: <name-of-folder> must NOT contain leading or trailing characters. For instance, the folder named subproject MUST be passed as subproject, NOT ./subproject/

Note for Windows users: When your folder depth is > 1, <name-of-folder> must have *nix style folder separator (/). For instance, the folder named path1\path2\subproject MUST be passed as path1/path2/subproject

1. Create the new repo

    mkdir ~/<new-repo> && cd ~/<new-repo>
    git init
    git pull </path/to/big-repo> <name-of-new-branch>
   

2. Link the new repo to GitHub or wherever

    git remote add origin <git@github.com:user/new-repo.git>
    git push -u origin master
   

3. Cleanup inside <big-repo>, if desired

    git rm -rf <name-of-folder>
   

Note: This leaves all the historical references in the repository. See the Appendix below if you're actually concerned about having committed a password or you need to decreasing the file size of your .git folder.

Walkthrough

These are the same steps as above, but following my exact steps for my repository instead of using <meta-named-things>.

Here's a project I have for implementing JavaScript browser modules in node:

tree ~/node-browser-compat

node-browser-compat
├── ArrayBuffer
├── Audio
├── Blob
├── FormData
├── atob
├── btoa
├── location
└── navigator

I want to split out a single folder, btoa, into a separate Git repository

cd ~/node-browser-compat/
git subtree split -P btoa -b btoa-only

I now have a new branch, btoa-only, that only has commits for btoa and I want to create a new repository.

mkdir ~/btoa/ && cd ~/btoa/
git init
git pull ~/node-browser-compat btoa-only

Next, I create a new repo on GitHub or Bitbucket, or whatever and add it as the origin

git remote add origin git@github.com:node-browser-compat/btoa.git
git push -u origin master

Happy day!

Note: If you created a repo with a README.md, .gitignore and LICENSE, you will need to pull first:

git pull origin master
git push origin master

Lastly, I'll want to remove the folder from the bigger repo

git rm -rf btoa

Appendix

Latest Git on macOS

To get the latest version of Git using Homebrew:

brew install git

Latest Git on Ubuntu

sudo apt-get update
sudo apt-get install git
git --version

If that doesn't work (you have a very old version of Ubuntu), try

sudo add-apt-repository ppa:git-core/ppa
sudo apt-get update
sudo apt-get install git

If that still doesn't work, try

sudo chmod +x /usr/share/doc/git/contrib/subtree/git-subtree.sh
sudo ln -s \
/usr/share/doc/git/contrib/subtree/git-subtree.sh \
/usr/lib/git-core/git-subtree

Thanks to rui.araujo from the comments.

Clearing your history

By default removing files from Git doesn't actually remove them, it just commits that they aren't there anymore. If you want to actually remove the historical references (i.e. you committed a password), you need to do this:

git filter-branch --prune-empty --tree-filter 'rm -rf <name-of-folder>' HEAD

After that, you can check that your file or folder no longer shows up in the Git history at all

git log -- <name-of-folder> # should show nothing

However, you can't "push" deletes to GitHub and the like. If you try, you'll get an error and you'll have to git pull before you can git push - and then you're back to having everything in your history.

So if you want to delete history from the "origin" - meaning to delete it from GitHub, Bitbucket, etc - you'll need to delete the repo and re-push a pruned copy of the repo. But wait - there's more! - if you're really concerned about getting rid of a password or something like that you'll need to prune the backup (see below).

Making .git smaller

The aforementioned delete history command still leaves behind a bunch of backup files - because Git is all too kind in helping you to not ruin your repo by accident. It will eventually delete orphaned files over the days and months, but it leaves them there for a while in case you realize that you accidentally deleted something you didn't want to.

So if you really want to empty the trash to reduce the clone size of a repo immediately you have to do all of this really weird stuff:

rm -rf .git/refs/original/ && \
git reflog expire --all && \
git gc --aggressive --prune=now

git reflog expire --all --expire-unreachable=0
git repack -A -d
git prune

That said, I'd recommend not performing these steps unless you know that you need to - just in case you did prune the wrong subdirectory, y'know? The backup files shouldn't get cloned when you push the repo, they'll just be in your local copy.

Credit

• http://psionides.eu/2010/02/04/sharing-code-between-projects-with-git-subtree/
• Remove a directory permanently from git
• http://blogs.atlassian.com/2013/05/alternatives-to-git-submodule-git-subtree/
• How to remove unreferenced blobs from my git repo