Removing Sensitive Data

Safely remove passwords, credentials, and other sensitive information from Git history

Accidentally committing sensitive data like passwords, API keys, or credentials is a common mistake. git-filter-repo provides powerful tools to completely remove this data from your repository history.

After removing sensitive data from history, you must:

• Rotate any exposed credentials immediately
• Force push the cleaned repository
• Ask all team members to re-clone (not pull) the repository
• Consider the data compromised if it was ever pushed to a public repository

Removing Specific Files

To remove a file containing sensitive data from all of history:

1. Clone a fresh copy

   git clone https://github.com/example/repo.git
   cd repo
   
2. Remove the file

   git filter-repo --invert-paths --path config/secrets.yml
   

   The --invert-paths flag removes the specified path instead of keeping it.

3. Verify removal

   # This should return nothing
   git log --all --full-history -- config/secrets.yml
   
4. Force push

   git remote add origin https://github.com/example/repo.git
   git push --force --all
   git push --force --tags
   

   Force pushing rewrites history. Coordinate with your team before doing this.

Removing Multiple Files

For removing many files at once, create a file listing all paths to remove:

1. Create the paths file

   Create ../files-to-delete.txt with one path per line:

   config/database.yml
   secrets.env
   credentials/api-keys.json
   .env
   
2. Remove all listed files

   git filter-repo --invert-paths --paths-from-file ../files-to-delete.txt
   

Removing Files by Pattern

Remove all files matching a specific pattern:

# Remove all .env files
git filter-repo --invert-paths --path-glob '*.env'

# Remove all files in any credentials/ directory
git filter-repo --invert-paths --path-glob '**/credentials/*'

Replace Text in Files

For removing sensitive strings embedded in files (like API keys in code):

1. Create a replacements file

   Create ../replacements.txt with the format FIND==>REPLACE:

   sk_live_abc123def456ghi789jkl==>REDACTED_API_KEY
   password123==>REDACTED_PASSWORD
   

   You can also use regex patterns:

   regex:sk_live_[a-zA-Z0-9]+==>[REDACTED_STRIPE_KEY]
   
2. Apply replacements

   git filter-repo --replace-text ../replacements.txt
   

Advanced: Using Callbacks

For complex removal scenarios, use Python callbacks:

Remove specific blob by hash

If you know the hash of a sensitive blob:

git filter-repo --blob-callback '
    if blob.original_id == b"f4ede2e944868b9a08401dafeb2b944c7166fd0a":
        blob.data = b"REDACTED"
'

Remove files with specific patterns in name

git filter-repo --filename-callback '
    if b"secret" in filename.lower() or b"password" in filename.lower():
        return None
    return filename
'

Removing a Directory

To remove an entire directory from history:

git filter-repo --path node_modules/secrets/ --invert-paths

Complete Cleanup Workflow

1. Create a fresh clone

   git clone https://github.com/example/repo.git repo-cleanup
   cd repo-cleanup
   
2. Remove sensitive data

   git filter-repo --invert-paths --path-glob '*.env' --path config/secrets.yml
   
3. Verify the cleanup

   # Check repository size
   git count-objects -vH
   
   # Verify files are gone
   git log --all --full-history -- config/secrets.yml
   
4. Replace the remote repository

   git remote add origin https://github.com/example/repo.git
   git push --force --all
   git push --force --tags
   
5. Rotate credentials

   Change all exposed passwords, API keys, and credentials immediately.

6. Notify team members

   Instruct all contributors to delete their local clones and re-clone:

   # Team members should run:
   rm -rf repo
   git clone https://github.com/example/repo.git
   

   Do not use git pull - it will bring back the old history!

Post-Cleanup Checklist

• All sensitive credentials have been rotated
• Force push completed successfully
• All team members notified to re-clone
• GitHub/GitLab support contacted (if needed for server-side cleanup)
• Repository size reduced as expected
• CI/CD secrets updated
• Documentation updated with new credentials (in secure storage)