Local GIT Repo Scanning
A. Local Repository Scanning with Gitleaks – Single Repository
This method is intended for users who want to scan a single Git repository locally.
Step One:
-
Ensure Git is installed on your system.
-
Download and Install Gitleaks, preferably on a Mac or Linux Server. For installation instructions, visit the official repository: https://github.com/gitleaks/gitleaks
-
Run any of the following commands below based on your preference:
# MacOS
brew install gitleaks
# Docker (DockerHub)
docker pull zricethezav/gitleaks:latest
docker run -v ${path_to_host_folder_to_scan}:/path zricethezav/gitleaks:latest [COMMAND] [OPTIONS] [SOURCE_PATH]
# Docker (ghcr.io)
docker pull ghcr.io/gitleaks/gitleaks:latest
docker run -v ${path_to_host_folder_to_scan}:/path ghcr.io/gitleaks/gitleaks:latest [COMMAND] [OPTIONS] [SOURCE_PATH]
#Ubuntu Linux
To install Gitleaks: “sudo apt install gitleaks”.
-
To verify Gitleaks installation, run this command “command-v gitleaks”
Step Two:
-
Clone the repo you want to scan - git clone <repository_url>. For example: git clone git@gitlab.com:account_name/repo_name.git
2. For the purpose of this documentation, the following are example of secret keys present in the repository as show below:
3. To scan the clone repository, run this command: gitleaks detect –s <cloned_repository>. For example, gitleaks detect –s repo_name
4. To generate a scan report, run this: gitleaks detect –s repo_name scanreport.json
-
Delete the secret key/API
B. Local Repository Scanning with Gitleaks – Multiple Repository
This method is for users who want to scan multiple repositories locally before committing changes. This is to ensure sensitive information is detected and remediated before committing any changes to the remote repository.
Step One:
- Make sure you have Git Installed.
2. Download and Install Gitleaks, preferably on a Mac or Linux Server. You can find more information here: https://github.com/gitleaks/gitleaks
You can run any of the command below based on your preference:
# MacOS
brew install gitleaks
# Docker (DockerHub)
docker pull zricethezav/gitleaks:latest
docker run -v ${path_to_host_folder_to_scan}:/path zricethezav/gitleaks:latest [COMMAND] [OPTIONS] [SOURCE_PATH]
# Docker (ghcr.io)
docker pull ghcr.io/gitleaks/gitleaks:latest
docker run -v ${path_to_host_folder_to_scan}:/path ghcr.io/gitleaks/gitleaks:latest [COMMAND] [OPTIONS] [SOURCE_PATH]
#Ubuntu Linux
To install Gitleaks: “sudo apt install gitleaks”.
Step Two:
-
Create a .txt file, for example, a repo.txt that will contain a list of repos to be scanned. Replace the (acount_name) and the (repo_name)
For example:
SSH
git@gitlab.com:account_name/repo_name1.git
git@gitlab.com:account_name/repo_name2.git
git@gitlab.com:account_name/repo_name3.git
Or
HTTPS
NB: If you are using SSH as a mode of authentication, ensure you have your SSH key available on your system in ~/.ssh/directory, and the SSH key should be added to your SSH agent for successful authentication.
Furthermore, if you are using HTTPS, ensure you have your personal access token, which will be used to clone your private repos.
-
Once you have the repo.txt file containing the list of repositories you would like to scan, then create a bash script, for example, repo_scanning.sh and paste this content into the repo_scanning.sh file:
#!/bin/bash
#Check if Gitleaks is installed
if ! command -v gitleaks &> /dev/null; then
echo "Gitleaks not found. Please install Gitleaks with the README documentation before running this script."
exit 1
fi
# Read the file containing the list of repository URLs
read -p "Enter the file path with the repository URLs: " repo_list_file
# Check if the file exists
if [[ ! -f "$repo_list_file" ]]; then
echo "File not found: $repo_list_file"
exit 1
fi
# Create the 'clone_repos' directory if it doesn't exist
mkdir -p ./cloned_repository
#iterate through each repository URL in the file
while IFS= read -r repo_url; do
# Skip empty lines
[[ -z "$repo_url" ]] && continue
# Generate a unique directory name for cloning
repository_name=$(basename "$repo_url" .git)
clone_directory="./cloned_repository/$repostiory_name"
echo "Cloning repository: $repo_url"
git clone "$repo_url" "$clone_directory"
if [ $? -ne 0]; then
echo "Failed to clone repository: $repo_url. Skipping. "
continue
fi
# Run Gitleaks on the cloned repository
echo "Running Gitleaks on $repository_name..."
gitleaks detect -s "$cloned_directory" -r "$cloned_directory/gitleaks_report.json"
if [ $? -eq 0]; then
echo "Gitleaks scan completed for $repository_name. Report saved to $clone_directory/gitleaks_report.json"
else
echo "Gitleaks scan failed for $repository_name. Report saved to $clone_directory/gitleaks_report.json"
fi
done < "$repo_list_file"
-
To run the script provided, run chmod +X repo_scanning.sh
-
Once the script is executable, enter ./repo_scanning.sh and it will prompt you to paste the repository URLs, such as ./repo.txt where the lists of repositories are deposited.
C. CICD GIT REPO SCANNING WITH GITLAB ULTIMATE
To scan your CI/CD Git repositories on GitLab Ultimate, you'll integrate security scanners into your CI/CD pipelines by adding scanner templates to the `.gitlab- ci.yml ` file.
Step one:
-
Open the `.gitlab-ci.yml` file and add the following to the bottom:
include:
- template: Jobs/Secret-Detection.gitlab-ci.yml
-
Ensure your `.gitlab-ci.yml ` pipeline file includes a test stage.
-
In your repository, navigate to build and select pipeline editor to validate the configuration.
-
The secret scanning generates a `gl-secret-detection-report.json` containing the secret scanning results.
-
In GitLab, Secret scanning is enabled at the project level, so for each project, navigate to the secure tab.
-
Under Security Configuration, enable secret push protection and pipeline secret Detection.
-
The pipeline secret detection will trigger a merge request to add the secret detection to you ` .gitlab-ci.yml` file.
Reference link for Secret Management