subfinder/README.md

284 lines
10 KiB
Markdown
Raw Normal View History

2021-02-20 17:47:03 +00:00
<h1 align="center">
<img src="static/subfinder-logo.png" alt="subfinder" width="200px"></a>
2019-12-05 13:06:56 +00:00
<br>
</h1>
2021-02-20 17:47:03 +00:00
<h4 align="center">Fast passive subdomain enumeration tool.</h4>
2020-08-11 21:27:45 +00:00
2018-06-26 13:38:49 +00:00
2021-02-20 17:47:03 +00:00
<p align="center">
<a href="https://goreportcard.com/report/github.com/projectdiscovery/subfinder"><img src="https://goreportcard.com/badge/github.com/projectdiscovery/subfinder"></a>
<a href="https://github.com/projectdiscovery/subfinder/issues"><img src="https://img.shields.io/badge/contributions-welcome-brightgreen.svg?style=flat"></a>
<a href="https://github.com/projectdiscovery/subfinder/releases"><img src="https://img.shields.io/github/release/projectdiscovery/subfinder"></a>
2021-02-21 07:59:31 +00:00
<a href="https://twitter.com/pdiscoveryio"><img src="https://img.shields.io/twitter/follow/pdiscoveryio.svg?logo=twitter"></a>
2021-02-20 17:47:03 +00:00
<a href="https://discord.gg/projectdiscovery"><img src="https://img.shields.io/discord/695645237418131507.svg?logo=discord"></a>
</p>
<p align="center">
<a href="#features">Features</a>
<a href="#installation">Install</a>
<a href="#running-subfinder">Usage</a>
<a href="#post-installation-instructions">API Setup</a>
2021-04-17 12:09:01 +00:00
<a href="#subfinder-go-library">Library</a>
2021-02-20 17:47:03 +00:00
<a href="https://discord.gg/projectdiscovery">Join Discord</a>
</p>
2019-12-05 13:06:56 +00:00
2021-02-20 17:47:03 +00:00
---
2018-06-26 13:38:49 +00:00
2021-02-20 17:47:03 +00:00
Subfinder is a subdomain discovery tool that discovers valid subdomains for websites by using passive online sources. It has a simple modular architecture and is optimized for speed. subfinder is built for doing one thing only - passive subdomain enumeration, and it does that very well.
2019-12-05 13:06:56 +00:00
2021-02-20 17:47:03 +00:00
We have designed subfinder to comply with all passive sources licenses, and usage restrictions, as well as maintained a consistently passive model to make it useful to both penetration testers and bug bounty hunters alike.
2020-07-12 11:35:25 +00:00
2018-06-26 13:38:49 +00:00
2021-02-20 17:47:03 +00:00
# Features
2019-12-05 13:06:56 +00:00
<h1 align="left">
<img src="static/subfinder-run.png" alt="subfinder" width="700px"></a>
<br>
</h1>
2021-02-20 17:48:52 +00:00
- Fast and powerful resolution and wildcard elimination module
2021-02-20 17:47:03 +00:00
- **Curated** passive sources to maximize results
2019-12-05 11:19:11 +00:00
- Multiple Output formats supported (Json, File, Stdout)
2019-12-05 11:33:37 +00:00
- Optimized for speed, very fast and **lightweight** on resources
2021-02-20 17:47:03 +00:00
- **STDIN/OUT** support for integrating in workflows
2018-06-26 11:33:16 +00:00
2020-05-22 21:42:11 +00:00
2018-04-24 13:14:52 +00:00
# Usage
2018-04-14 07:17:42 +00:00
2020-08-18 01:05:46 +00:00
```sh
2019-12-05 11:19:11 +00:00
subfinder -h
2018-04-24 13:14:52 +00:00
```
This will display help for the tool. Here are all the switches it supports.
2022-03-06 18:48:13 +00:00
```yaml
Flags:
INPUT:
2022-03-06 18:48:13 +00:00
-d, -domain string[] domains to find subdomains for
-dL, -list string file containing list of domains for subdomain discovery
SOURCE:
2022-03-06 18:56:03 +00:00
-s, -sources string[] sources to use for discovery (-s crtsh,github)
2022-03-06 18:48:13 +00:00
-recursive use only recursive sources
-all Use all sources (slow) for enumeration
-es, -exclude-sources string[] sources to exclude from enumeration (-es archiveis,zoomeye)
RATE-LIMIT:
2022-03-06 18:48:13 +00:00
-rl, -rate-limit int maximum number of http requests to send per second
-t int number of concurrent goroutines for resolving (-active only) (default 10)
OUTPUT:
2022-03-06 18:48:13 +00:00
-o, -output string file to write output to
-oJ, -json write output in JSONL(ines) format
-oD, -output-dir string directory to write output (-dL only)
-cs, -collect-sources include all sources in the output (-json only)
-oI, -ip include host IP in output (-active only)
CONFIGURATION:
2022-03-06 18:48:13 +00:00
-config string flag config file (default "$HOME/.config/subfinder/config.yaml")
-pc, -provider-config string provider config file (default "$HOME/.config/subfinder/provider-config.yaml")
-r string[] comma separated list of resolvers to use
-rL, -rlist string file containing list of resolvers to use
-nW, -active display active subdomains only
-proxy string http proxy to use with subfinder
DEBUG:
2022-03-06 18:56:03 +00:00
-ls list all available sources
2022-03-06 18:48:13 +00:00
-silent show only subdomains in output
-version show version of subfinder
-v show verbose output
2022-03-06 18:56:03 +00:00
-nc, -no-color disable color in output
OPTIMIZATION:
2022-03-06 18:48:13 +00:00
-timeout int seconds to wait before timing out (default 30)
-max-time int minutes to wait for enumeration results (default 10)
```
2021-02-20 17:47:03 +00:00
# Installation
2021-09-25 16:57:32 +00:00
Subfinder requires **go1.17** to install successfully. Run the following command to get the repo -
2019-12-06 08:28:07 +00:00
2020-08-18 01:05:46 +00:00
```sh
2021-09-25 16:57:32 +00:00
go install -v github.com/projectdiscovery/subfinder/v2/cmd/subfinder@latest
2019-12-06 08:28:07 +00:00
```
2018-04-24 13:14:52 +00:00
## Post Installation Instructions
2018-04-23 00:09:34 +00:00
2019-12-05 11:19:11 +00:00
Subfinder will work after using the installation instructions however to configure Subfinder to work with certain services, you will need to have setup API keys. The following services do not work without an API key:
2018-04-23 00:09:34 +00:00
2022-03-06 18:48:13 +00:00
[Binaryedge](https://binaryedge.io), [C99](https://api.c99.nl/), [Certspotter](https://sslmate.com/certspotter/api/), [Chinaz](http://my.chinaz.com/ChinazAPI/DataCenter/MyDataApi), [Censys](https://censys.io), [Chaos](https://chaos.projectdiscovery.io), [DnsDB](https://api.dnsdb.info), [Fofa](https://fofa.so/static_pages/api_help), [Github](https://github.com), [Intelx](https://intelx.io), [Passivetotal](http://passivetotal.org), [Robtex](https://www.robtex.com/api/), [SecurityTrails](http://securitytrails.com), [Shodan](https://shodan.io), [Spyse](https://spyse.com), [Threatbook](https://x.threatbook.cn/en), [Virustotal](https://www.virustotal.com), [Zoomeye](https://www.zoomeye.org)
2019-12-05 11:19:11 +00:00
2022-03-06 15:17:40 +00:00
Theses values are stored in the `$HOME/.config/subfinder/provider-config.yaml` file which will be created when you run the tool for the first time. The configuration file uses the YAML format. Multiple API keys can be specified for each of these services from which one of them will be used for enumeration.
2019-12-05 11:19:11 +00:00
For sources that require multiple keys, namely `Censys`, `Passivetotal`, they can be added by separating them via a colon (:).
2022-03-06 18:48:13 +00:00
An example provider config file -
2019-12-05 11:19:11 +00:00
```yaml
binaryedge:
- 0bf8919b-aab9-42e4-9574-d3b639324597
- ac244e2f-b635-4581-878a-33f4e79a2c13
censys:
- ac244e2f-b635-4581-878a-33f4e79a2c13:dd510d6e-1b6e-4655-83f6-f347b363def9
certspotter: []
2020-08-10 15:59:15 +00:00
passivetotal:
2020-07-14 12:08:12 +00:00
- sample-email@user.com:sample_password
2019-12-05 11:19:11 +00:00
securitytrails: []
2020-07-14 12:08:12 +00:00
shodan:
- AAAAClP1bJJSRMEYJazgwhJKrggRwKA
2020-07-14 17:12:47 +00:00
github:
- ghp_lkyJGU3jv1xmwk4SDXavrLDJ4dl2pSJMzj4X
- ghp_gkUuhkIYdQPj13ifH4KA3cXRn8JD2lqir2d4
2018-04-23 00:09:34 +00:00
```
2018-06-26 13:38:49 +00:00
# Running Subfinder
To run the tool on a target, just use the following command.
2019-12-05 11:33:37 +00:00
2022-03-06 18:48:13 +00:00
```console
2021-09-25 18:29:22 +00:00
subfinder -d hackerone.com
__ _____ __
_______ __/ /_ / __(_)___ ____/ /__ _____
/ ___/ / / / __ \/ /_/ / __ \/ __ / _ \/ ___/
(__ ) /_/ / /_/ / __/ / / / / /_/ / __/ /
/____/\__,_/_.___/_/ /_/_/ /_/\__,_/\___/_/ v2.4.9
projectdiscovery.io
Use with caution. You are responsible for your actions
Developers assume no liability and are not responsible for any misuse or damage.
By using subfinder, you also agree to the terms of the APIs used.
[INF] Enumerating subdomains for hackerone.com
www.hackerone.com
support.hackerone.com
links.hackerone.com
api.hackerone.com
o1.email.hackerone.com
go.hackerone.com
3d.hackerone.com
resources.hackerone.com
a.ns.hackerone.com
b.ns.hackerone.com
mta-sts.hackerone.com
docs.hackerone.com
mta-sts.forwarding.hackerone.com
gslink.hackerone.com
hackerone.com
info.hackerone.com
mta-sts.managed.hackerone.com
events.hackerone.com
[INF] Found 18 subdomains for hackerone.com in 3 seconds 672 milliseconds
```
2020-07-12 11:35:25 +00:00
The subdomains discovered can be piped to other tools too. For example, you can pipe the subdomains discovered by subfinder to httpx [httpx](https://github.com/projectdiscovery/httpx) which will then find running http servers on the host.
2019-12-05 11:33:37 +00:00
2022-03-06 18:48:13 +00:00
```console
2021-09-25 16:57:32 +00:00
echo hackerone.com | subfinder -silent | httpx -silent
2019-12-05 11:47:41 +00:00
2019-12-05 11:33:37 +00:00
http://hackerone.com
http://www.hackerone.com
http://docs.hackerone.com
http://api.hackerone.com
https://docs.hackerone.com
http://mta-sts.managed.hackerone.com
```
2018-07-06 04:05:46 +00:00
2021-02-20 17:47:03 +00:00
<table>
<tr>
<td>
## Subfinder with docker
2020-07-12 11:35:25 +00:00
2021-09-25 18:29:22 +00:00
Pull the latest tagged [subfinder](https://hub.docker.com/r/projectdiscovery/subfinder) docker image:
2020-07-12 11:35:25 +00:00
2020-08-18 01:05:46 +00:00
```sh
2021-09-25 17:55:25 +00:00
docker pull projectdiscovery/subfinder:latest
2020-07-12 11:35:25 +00:00
```
2021-09-25 18:29:22 +00:00
Running subfinder using docker image:
2020-07-12 11:35:25 +00:00
2020-08-18 01:05:46 +00:00
```sh
2021-09-25 18:29:22 +00:00
docker -t projectdiscovery/subfinder:latest -d hackerone.com
2020-07-12 11:35:25 +00:00
```
2021-09-25 18:29:22 +00:00
Running subfinder using docker image with local config file:
2020-07-12 11:43:48 +00:00
2020-08-18 01:05:46 +00:00
```sh
2021-09-25 18:29:22 +00:00
docker run -v $HOME/.config/subfinder:/root/.config/subfinder -t projectdiscovery/subfinder -d hackerone.com
2020-07-12 11:35:25 +00:00
```
2021-02-20 17:47:03 +00:00
</td>
</tr>
</table>
2021-04-17 11:53:58 +00:00
<table>
<tr>
<td>
## Subfinder Go library
Usage example:
2021-09-25 16:57:32 +00:00
```go
2021-04-17 11:53:58 +00:00
package main
import (
"bytes"
"context"
"fmt"
"io"
"log"
"github.com/projectdiscovery/subfinder/v2/pkg/passive"
"github.com/projectdiscovery/subfinder/v2/pkg/resolve"
"github.com/projectdiscovery/subfinder/v2/pkg/runner"
)
func main() {
runnerInstance, err := runner.NewRunner(&runner.Options{
Threads: 10, // Thread controls the number of threads to use for active enumerations
Timeout: 30, // Timeout is the seconds to wait for sources to respond
MaxEnumerationTime: 10, // MaxEnumerationTime is the maximum amount of time in mins to wait for enumeration
Resolvers: resolve.DefaultResolvers, // Use the default list of resolvers by marshaling it to the config
Sources: passive.DefaultSources, // Use the default list of passive sources
AllSources: passive.DefaultAllSources, // Use the default list of all passive sources
Recursive: passive.DefaultRecursiveSources, // Use the default list of recursive sources
Providers: &runner.Providers{}, // Use empty api keys for all providers
})
2021-04-17 11:53:58 +00:00
buf := bytes.Buffer{}
err = runnerInstance.EnumerateSingleDomain(context.Background(), "projectdiscovery.io", []io.Writer{&buf})
if err != nil {
log.Fatal(err)
}
data, err := io.ReadAll(&buf)
2021-04-17 11:53:58 +00:00
if err != nil {
log.Fatal(err)
}
fmt.Printf("%s", data)
}
```
</td>
</tr>
</table>
2021-02-06 08:12:47 +00:00
### Resources
- [Recon with Me !!!](https://dhiyaneshgeek.github.io/bug/bounty/2020/02/06/recon-with-me/)
2018-07-06 04:05:46 +00:00
# License
2019-12-05 11:33:37 +00:00
subfinder is made with 🖤 by the [projectdiscovery](https://projectdiscovery.io) team. Community contributions have made the project what it is. See the **[Thanks.md](https://github.com/projectdiscovery/subfinder/blob/master/THANKS.md)** file for more details.
2019-12-05 13:23:02 +00:00
2020-05-16 12:12:48 +00:00
Read the disclaimer for usage at [DISCLAIMER.md](https://github.com/projectdiscovery/subfinder/blob/master/DISCLAIMER.md) and [contact us](mailto:contact@projectdiscovery.io) for any API removal.