lc / gau

getallurls (gau) getallurls (gau) fetches known URLs from AlienVault's Open Threat Exchange, the Wayback Machine, Common Crawl, and URLScan for any given domain. Inspired by Tomnomnom's waybackurls. Resources • Usage • Installation • ohmyzsh note Usage: Examples: To display the help for the tool use the flag: | Flag | Description | Example | |------|-------------|---------| | | list of extensions to skip | gau --blacklist ttf,woff,svg,png| | | Use alternate configuration file (default or ) | gau --config $HOME/.config/gau.toml| | | list of status codes to filter | gau --fc 404,302 | | | fetch urls from date (format: YYYYMM) | gau --from 202101 | | | list of mime-types to filter | gau --ft text/plain| | | remove different parameters of the same endpoint | gau --fp| | | output as json | gau --json | | | list of status codes to match | gau --mc 200,500 | | | list of mime-types to match |gau --mt text/html,application/json| | | filename to write results to | gau --o out.txt | | | list of providers to use (wayback,commoncrawl,otx,urlscan) | gau --providers wayback| | | http proxy to use (socks5:// or http:// | gau --proxy http://proxy.example.com:8080 | | | retries for HTTP client | gau --retries 10 | | | timeout (in seconds) for HTTP client | gau --timeout 60 | | | include subdomains of target domain | gau example.com --subs | | | number of workers to spawn | gau example.com --threads | | | fetch urls to date (format: YYYYMM) | gau example.com --to 202101 | | | show verbose output | gau --verbose example.com | | | show gau version | gau --version| Configuration Files gau automatically looks for a configuration file at or . You can point to a different configuration file using the flag. **If the configuration file is not found, gau will still run with a default configuration, but will output a message to stderr**. You can specify options and they will be used for every subsequent run of gau. Any options provided via command line flags will override options set in the configuration file. An example configuration file can be found here Installation: From source: From github : From binary: You can download the pre-built binaries from the releases page and then move them into your $PATH. From Docker: You can run gau via docker like so: You can also build a docker image with the following command and then run it Bear in mind that piping command (echo "example.com" | gau) will not work with the docker container ohmyzsh note: ohmyzsh's git plugin has an alias which maps to the command. This is problematic, causing a binary conflict between this tool "gau" and the zsh plugin alias "gau" ( ). There is currently a few workarounds which can be found in this Github issue. Useful? Donate to CommonCrawl Donate to the InternetArchive