GitGroundworkGit
14 / 18 written chapters

G14 · Part 3 · Rescue

Danger zone

Leaked secrets, force-push accidents, commands that eat work, and how to come back from each one.

9 min read2 diagrams6 sections
06On this page
  1. You committed a secret
  2. Force-push accidents
  3. The commands that destroy work
  4. A large file in history
  5. A bad merge landed on main
  6. Big conflicts in a rebase
Fresher Mid Senior

Almost everything in Git can be undone, because commits are rarely deleted straight away. They become unreachable, and the reflog remembers them for weeks. The dangerous cases are the exceptions: work that was never committed, history that other people already copied, and data that should never have been public. This chapter is about those.

In one minute

Three rules keep you out of trouble. Commit or stash before any command that rewrites files, because Git cannot bring back edits it never saw. Never use plain git push --force; use --force-with-lease, and only on your own branch. And if you ever push a password or key, tell someone and replace the key first. Deleting the file afterwards does not remove it from history.

You committed a secret

An API key, a database password, a private key. The instinct is to delete the file and commit again. That fixes nothing: the secret is still in the earlier commit, and anyone who clones gets the full history. If the repository is public, assume the key was copied within minutes. Bots scan public pushes for credential patterns continuously.

exposure window key pushed 1. rotate 2. audit 3. purge 4. clean up scraped within minutes if public revoke the old key, issue a new one provider logs: was the old key used? filter-repo, then force-push all refs re-clone, host support, forks Copies a rewrite never reaches forks · teammates' clones · CI caches · pull request refs · anyone who already scraped it
Only step 1 closes the exposure. Steps 3 and 4 stop the secret spreading further, but they can't pull back a copy someone already has.

1. Rotate first

Revoke the credential at the provider and issue a new one, before any Git work. Once the old key is dead, what's left in history is a curiosity rather than a live threat. Then check the provider's access logs for use of the old key while it was exposed. If it was used, this is now a security incident, not a Git problem.

2. Purge the history

git filter-repo is the tool the Git project recommends. git filter-branch is slow, easy to get wrong, and its own manual now warns against it. BFG Repo-Cleaner is an older, simpler alternative.

git clone --mirror https://github.com/org/app.git app-backup.git
git clone https://github.com/org/app.git app-clean
cd app-clean

git filter-repo --invert-paths --path config/.env

echo 'sk_live_51Hx9TQ==>REMOVED' > replacements.txt
git filter-repo --replace-text replacements.txt

git remote add origin https://github.com/org/app.git
git push origin --force --all
git push origin --force --tags

--invert-paths --path removes a file from every commit. --replace-text rewrites a string wherever it appears, which is right when the secret was pasted into source code. filter-repo refuses to run on anything but a fresh clone, and it removes the origin remote afterwards so you can't push by accident. Both are deliberate safety features. BFG does the same with bfg --delete-files .env or bfg --replace-text passwords.txt, but it leaves your latest commit alone by default, so remove the secret from HEAD in a normal commit first.

3. Why the rewrite is not enough

  • Every existing clone still has it. Teammates must re-clone, or they may push the old history straight back.
  • Forks are separate repositories. Your force-push does not touch them.
  • The host keeps references you can't overwrite. On GitHub, pull requests keep their commits under refs/pull/, and old commits stay reachable by hash until garbage collection. Ask the host's support to remove cached views and PR refs.
  • CI caches, artifacts, logs and package registries may hold copies too.
Stop the next one

Turn on your host's secret scanning and push protection, which reject a push containing a known credential format. Add a pre-commit scanner such as gitleaks. Put .env in .gitignore before the first commit and commit a .env.example instead. To check existing history: git log -p -S "AKIA" --all finds every commit that added or removed that string.

For seniors: running a history rewrite for a team

The Git commands are the easy part. Announce a freeze and wait until nobody has unpushed work on the affected branches. Take a mirror backup. Rewrite in one pass and check the result with git log --all -S "the secret" returning nothing. Temporarily lift the protection that blocks force-pushes, push every branch and tag, and put the protection back. Then have every developer and CI system make a fresh clone rather than pull, because a single old clone that pushes can reintroduce the whole old history. Open pull requests are based on the old commits and usually have to be recreated. filter-repo writes a commit-map file of old to new IDs inside .git/filter-repo/; keep it, since tickets, deploy logs and release notes will keep quoting the old hashes.

Force-push accidents

You rebased your local main, ran git push --force, and your teammate's two commits from this morning vanished from the remote. They are not gone. They are unreachable on the server, and they still exist in every clone that fetched them.

BEFORE A B C D origin/main, teammate's work AFTER git push --force A B C D E origin/main unreachable, still in reflogs rescue git reflog show origin/main git branch rescue <D's hash> merge or rebase E onto it, then push normally
A force-push moves the remote branch. It does not delete C and D anywhere they were already fetched, and the server usually keeps them until garbage collection.

Recovering

  1. Find the old tip. The output of the force-push itself shows it: + 4e1a9c2...7b3f0d8 main -> main (forced update). The first hash is where the remote was. If the terminal is gone, your teammate runs git reflog show origin/main or just looks at their own main. On GitHub, the branch's activity view lists force-pushes with the before and after hashes.
  2. Pin it. git branch rescue 4e1a9c2. Now it can't be garbage-collected.
  3. Combine and push. Rebase or merge your work onto rescue, check the result, and push without force. Or, if your push was simply wrong, git push --force-with-lease origin rescue:main puts the remote back exactly as it was.

prevention

git push --force-with-lease
git push --force-with-lease --force-if-includes
git config --global alias.please "push --force-with-lease --force-if-includes"

--force-with-lease refuses to push if the remote branch has moved since you last fetched it. It has a gap: a background git fetch by your editor updates origin/main, and the lease passes even though you never looked at the new commits. --force-if-includes (Git 2.30 and later) closes that gap by also checking that the remote tip is somewhere in your local branch's reflog. On shared branches, turn on branch protection so force-pushes are rejected outright.

The commands that destroy work

CommandWhat it destroysHow to recoverSafer habit
git reset --hardUncommitted changes in tracked files, and it moves the branchCommits: git reflog, then git reset --hard HEAD@{1}. Staged but uncommitted files: git fsck --lost-found finds their blobs. Unstaged edits: gone.git stash first, or reset --keep
git clean -fdxUntracked and ignored files: .env, local config, build cachesNone from Git. Only your editor's local history or a system backup.git clean -nd dry run first, or -i
git restore . or git checkout -- .Every unstaged editNone from Git. Staged versions survive in the index.Restore named files, not .
git push --forceOther people's commits on the remote branchOld tip from push output, a teammate's reflog, or the host's activity log--force-with-lease --force-if-includes
git branch -DThe branch name, and its own reflogGit prints (was 1a2b3c4). Or search git reflog for when HEAD was on it.git branch -d, which refuses if unmerged
git stash drop, git stash clearStash entriesdrop prints the hash. After clear, git fsck --unreachable | grep commit and look for "WIP on".git stash pop only when sure
A rebase that went wrongThe old shape of the branchgit reset --hard ORIG_HEAD right after, or the entry before the rebase in git refloggit branch backup before a big rebase
git filter-repoAll old commit hashesOnly from a backup clonegit clone --mirror first
git reflog expire --expire=now --all then git gc --prune=nowEverything unreachable, for goodNoneOnly run it when you mean to erase, as after a secret purge

The pattern: Git protects commits very well, through the reflog and delayed garbage collection. It protects uncommitted work hardly at all. When in doubt, commit to a scratch branch. A throwaway commit costs nothing and is always recoverable.

A large file in history

You committed a 300 MB dataset, or node_modules. GitHub rejects any file over 100 MB, so the push fails with this exceeds GitHub's file size limit. Deleting the file in a new commit doesn't help: the push still carries the old commit. The fix depends on how far it went.

Where it isFix
In the last commit, not pushedgit rm --cached data.csv, add it to .gitignore, git commit --amend
A few commits back, not pushedgit rebase -i, mark that commit edit, remove the file, git commit --amend, git rebase --continue
Deep in local historygit filter-repo --path data.csv --invert-paths or --strip-blobs-bigger-than 50M
Pushed, and it should be in Gitgit lfs migrate import --include="*.psd" --everything, then a coordinated force-push
Pushed and shared, just bloatRemove it going forward. Rewrite only if clone size really hurts, since everyone must re-clone.

A bad merge landed on main

A feature branch was merged and production broke. main is shared and protected, so rewriting it is off the table. Revert the merge commit.

git log --merges --oneline -5
git revert -m 1 9f3e2a1
git push

A merge commit has two parents, and -m 1 tells Git which one is the mainline to keep: parent 1 is the branch you were on when you merged, normally main. The new commit undoes everything the feature branch brought in.

The trap when you merge it again

After the revert, Git still considers the feature's commits merged. Fix the bug on the branch and merge again, and only the fix arrives. The original changes stay reverted. First revert the revert, git revert <hash of the revert>, then merge the fix. Or give the feature fresh commits that Git has never seen: on the feature branch, git rebase --no-ff <commit it started from> replays every commit with a new ID, and the next merge brings all of it.

If the bad merge only reached main seconds ago, nobody has pulled it, and the branch isn't protected, git reset --hard HEAD~1 plus git push --force-with-lease is cleaner. Otherwise, revert. It's the one fix that never surprises a teammate.

Big conflicts in a rebase

When a rebase throws conflicts in twenty files, commit after commit, stop grinding. git rebase --abort puts everything back. A merge resolves each conflicting region once rather than once per replayed commit. If you really need the rebase, turn on git config --global rerere.enabled true first. Git then records each resolution and replays it automatically the next time the same conflict appears.