Skip to content

Excavator is a lightweight pure Golang git leak scanning tool based on SAP's credential digger (without machine learning models) which attempts to improve on performance by introducing parallel computation.

License

Notifications You must be signed in to change notification settings

ichbinfrog/excavator

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

61 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Excavator

Go Report Card License

Excavator is a lightweight pure Golang leak scanning tool which attempts to improve on performance by parallelising commit iteration.

CLI usage

Download a binary here.

# For scanning git repository (local or remote)
# Rules can be downloaded at resources/rules.yaml
excavator git <source> [flags]

# Dor scanning local directory
excavator fs <path> [flags]

Flags

  • -h , --help : display help
  • -c , --concurrent <int> : number of concurrent executions (defaults to 1), any integer given below 0 is considered as a single routine execution
  • -p , --path <string> : temporary local path to store the git repository (only applies to remote repository) (default .)
  • -r , --rules <string> : location of the rule declaration (defaults to resources/rules.yaml embedded in the binary)
  • -f , --format <string> : format of output result (default html) (currently supports yaml, html)

Global Flags

  • -v , -vv, -vvv : set verbosity levels

Scanning a repository without backend

excavator scan {repository}

Include in code

import (
  "github.com/ichbinfrog/excavator/pkg/scan"
)

func main() {
  c := &scan.GitScanner{}

  // Directory in which to store the cloned repository
  directory := ...
  
  // URL / local path of git repository
  // for private repositories the url can be set as
  // https://user:pass@host/repo.git
  repo := ...
  
  // path to rule file
  rule := ...

  // Number of concurrent go routines 
  concurrent := ...

  // Whether or not to show progress bar
  progressBar := ...

  // Output interface
  // Can be either
  //  - &YamlReport{}
  //  - &HTMLReport{}
  report := ...
  c.New(repo, directory, rule, report, progressBar)
}

Declaring rules

# rules.yaml
#
apiVersion: v1
rules:
  - # regex of rule
    definition: EAACEdEose0cBA[0-9A-Za-z]+
    # category of rule
    category: token
    # description (optional)
    description: facebook access token rule

# list of regexes of file to ignore
black_list:
  - '.*sample.*'

# list of parsers
# parsers are rules that require additional context for analysing
# for potential leaks with more precision
#
# currently supports "env" and "dockerfile" 
parsers:
  - type: "env" 
    extensions:
      - ".env" 

    # the parser uses theses values to check if the key in the <key> = <value>
    # form contains potential leaks 
    keys:               
      - "pass"
      - "host"
      - "proxy"
      - "key"

  - type: "dockerfile"
    extensions:
      - "Dockerfile"
    # keys defaults to 
    # ["pass", "host", "proxy", "key"] if not defined

# Whether or not to explore files that are in archives
# e.g. tar, gzip, zip, rar...
compressed: True

About

Excavator is a lightweight pure Golang git leak scanning tool based on SAP's credential digger (without machine learning models) which attempts to improve on performance by introducing parallel computation.

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published