Commit Graph

598 Commits (6534deedc6cb29ce7910c21f8bd982cd3f0832c4)

Author SHA1 Message Date
Shubham Rasal 6b142d794a
Issue 2254 uncover integration (#2786)
* nuclei -uq 'vuln:CVE-2021-26855' -t cves/2021/

- `nuclei -uq 'vuln:CVE-2021-26855' -t cves/2021/`

* Add automatic template execution using metadata

- Query uncover after the template is loaded.
- Add the received hosts to the input provider from uncover
- Make NormalizeStoreInputValue() function public to add hosts from the
  runner after uncover hosts received.

* run go mod tidy

* Remove unnecessary comments

* Resolve the requested changes

- move uncover code to protocols/common/uncover package
- Use uncover delay to create uncover rate limiter
- Use single ratelimiter object and remove not required ratelimiters
- Create Set() method for input provider interface
- Rename normalizeStoreInputValue to Set() method

* Solved the uncover running twice.

- flag StringSliceVarP adds the default value twice in the variable
- Check if provider keys exists or not
- Add uncover help block to english readme.md

* Add uncover field functionality

- ./nuclei -uq 'vuln:CVE-2021-26855' -t dns -duc -uf host
- ./nuclei -uq 'vuln:CVE-2021-26855' -t dns -duc -uf ip:port

* Update error messages and solve nuclei hang for wrong uncover engine

- Get uncover engine values from uncover package

* Resolve merge conflicts

* misc option update

* Update logging for templates

- remove duplicate env log printing
- Log message for template queries

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-11-16 11:12:39 +05:30
vrenzolaverace 2403c50c36
Add targets number in output banner (#2798)
* Add targets number in output banner (#2797)

* Add targets number in output banner (#2797)
2022-11-12 18:05:49 +05:30
Mzack9999 1fbbca66f9
Adding support to scan all v4/v6 IPs (#2709)
* Adding support to scan all v4/v6 IPs

* adding tests

* metainput prototype

* using new signature

* fixing nil pointer

* adding request context with metadata

* removing log instruction

* fixing merge conflicts

* adding clone helpers

* attempting to fix ipv6 square parenthesis wrap

* fixing dialed ip info

* fixing syntax

* fixing output ip selection

* adding integration tests

* disabling test due to gh ipv6 issue

* using ipv4 only due to GH limited networking

* extending metainput marshaling

* fixing hmap key

* adding test for httpx integration

* fixing lint error

* reworking marshaling/id-calculation

* adding ip version validation

* improving handling non url targets

* fixing condition check
2022-11-09 18:48:56 +05:30
Jaideep Khandelwal a36d42a993
Round off the duration to a minute (#2807) 2022-11-08 16:43:55 +05:30
vrenzolaverace 2aaf2a2158
Use utils helpers libraries (#2809) (#2810)
* Use utils helpers libraries (#2809)

* Use utils helpers libraries (#2809)
2022-11-07 01:54:23 +05:30
Shubham Rasal 721c4964d7
Issue 2613 custom template GitHub (#2630)
* Add custom template download/update support from github

- Accept the -gtr flag to accept the list of custom template
  repos(public/private)
- Accept the -gt flag for github token. It internally sets os.Env
  variable
- Update the flags from
   - -update to -nuclei-update for nuclei self update
   - -ut to -tup for template-update
   - -ud to -tud for custom template location
- Add github.go file which has code related to download and update
  custom templates repos.

* Reslove golint and test case error

* Take default template from community directory

- No need to give explicit community directory path.
- Update the integration test to support the change in path

* Update functional test script update template flag

* Update the path from community to nuclei-template

- Revert the code changes that were made to add community directory

* remove the comment

* Update the interactsh server url for testing

* Update race condition command

* update race condition cmd to download the templates

* Debug integration test failure

* update integration test to update templates

* Refactor downloadCustomTemplate function.

- Remove the log prining instead send the message.

* Add test case for custom template repo download

* move the download repo for loop into diff function

* refactor updateTemplate function.

* Create struct for github repos.

- Create customtemplate struct for repo.
- Add functions to customtemplate

* update readme.md file

* Refactor the downloadCustomTemplate function

- create const variables for github & community as template type
- Update gologger to INF
- Validate templateUpdate to accept only github & community value.
- Validate tempalteUpdate require githubTemplateRepo

* Resolve requested changes

* go mod update

* misc option update

* test update

* Revert back update-template flag to boolean.

- to update community templates
  `nuclei -ut`
- to update custom templates
  `nuclei -ut -gtr ehsandeep/mobile-nuclei-templates`

* Update readme to update flag documentation

* Update go.mod

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-11-03 20:27:18 +05:30
Ice3man b9472cf7e1
Added fuzzing support for query params + var dump feature (#2679)
* Added fuzzing support for query params + var dump feature

* Added query-fuzz integration test

* Fixed payloads + added keys-regex fuzz parameter

* Fixed interactsh not working + misc

* Fixed evaluation + added global variables/dsl support to payloads

* Misc fixes related to variables evaluations

* Added http variables support to fuzz

* misc

* Misc

* Added testing playground + misc renaming

* Added support for path and raw request to fuzzing

* Fixed fuzz integration test

* Fixed variable unresolved issue

* Add multiple parameter support with same name

* Added parameter value as 'value' dsl variable for parts

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2022-11-01 20:28:50 +05:30
Ice3man 74bd89dec6
Added templates-checksum + custom input support (#2744)
* Added templates-checksum + custom input support

* Fixed template name not passed for private
2022-10-29 04:21:43 +02:00
Jaideep Khandelwal 4cfde111f4
Feature 18 cloud flags (#2708)
* Add cloud flags for nuclei.

* Add flag to get output for a particular scan ID

* Add some comments to the function.

* Get timestamp and id for scan list

* Fix linting errors

* Check if type is enumeration.

* Do not show deleted scans.

* Do not use filter_result, create client once and use it everywhere with
runner.

* Fix the output of scan list to be better

* Format the nuclei scan output list.

* Remove unused constant

* misc option update

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-10-22 04:06:52 +05:30
Ice3man 363ffb75db
Added probing for URL + input based on protocol (#2614)
* Added workflow names based condition

* Added conditional filtering to workflow executor

* Replaced names with single name stringslice

* Added probing for URL + input based on protocol

* Remove debug comments

* Fixed typo

* Fixed failing tests

* Fixed workflow matcher condition + tests

* Fixed workflow item name

* Switch to if-else

* Fixed review comment strict

* Increase bulk size

* Added default port for SSL protocol + misc changes

* Fixed failing tests

* Fixed misc changes to executer

* Fixed failing self-contained and offlinehttp tests

* Fixed atomic increment operation

* misc update

* Fixed failing builds

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-10-20 17:23:00 +05:30
LuitelSamikshya 944d24a252 mics changes 2022-10-13 09:30:25 -05:00
LuitelSamikshya cb0da81a14 ratelimit library 2022-10-12 22:04:37 -05:00
Ice3man 09ceb29ba3 Fixed build error on 32bit arch 2022-10-08 01:55:18 +05:30
Ice3man 6c93d99745 Fixed failing lint tests 2022-10-08 00:10:32 +05:30
Ice3man 4f37c0b1f5 Added pagination support to result retrieval 2022-09-29 04:52:03 +05:30
Mzack9999 18f14b631c
Adding same host redirect support (#2655)
* simplifying test syntax

* adding same host redirect + refactoring redirect handling

* adding missing file

* adding support for template syntax

* adding integration test

* updating options

* fixing issue on same host redirect
2022-09-29 04:11:28 +05:30
Ice3man fc27fc94a5
Added default config generation for reporting options (#2605) 2022-09-27 02:40:34 +05:30
Sami 512eab7b21
tl flag improvements (#2596)
* tl flag improvements

* tl flag enhancement with additional filters

* added ExcluedTags filter

* tl flas to list template paths

* using stdout

* misc update

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-09-21 03:03:57 +05:30
Mzack9999 99c14f4c9c
implementation of rate limiter with bucket refill and unrestricted token burst (#2536)
* implementation of rate limiter with interval burst

* fixing import path

* fixing syntax

* adding tests

* fixing lint errors

* adding support for context

* moving rate limiter earlier to avoid hitting timeout
2022-09-19 17:09:28 +05:30
mzack 37c016a5f0 fixing lint errors 2022-09-19 08:38:52 +02:00
Ice3man 04b47b0309
Added custom json-unmarshaller + misc updates (#2556)
* Added custom json-unmarshaller + misc updates

* Added support for nuclei-cloud based scan execution

* Removed unnecessary files

* Misc

* Changes as per review comments

* misc option update

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-09-19 01:13:59 +05:30
forgedhallpass fc0763641f
New dsl functions (#2545)
* Update GO version to 1.18

* Removed redundant entry from the .gitignore file

* Added new DSL functions

to_unix_time(input string, optionalLayout string) int64
hex_to_dec(input string) float64
oct_to_dec(input string|number) float64
bin_to_dec(intput string|number) float64

* Notify if debug is enabled when a proxy cannot be validated

* Documentation: Go version requirement updated to 1.18

* test fix: Timezone agnostic date expectation in the assertion

* code review: extracted the default date-time layouts into a global variable
2022-09-07 00:44:29 +05:30
Ice3man 466176e9e8
Merge pull request #2500 from projectdiscovery/goflags-update
Updated goflags to latest + misc
2022-08-30 11:52:25 +05:30
Sandeep Singh e329428684
fix: make headless template validation optional (#2507) 2022-08-29 03:00:58 +05:30
Ice3man 8892250583 Updated goflags to latest + misc 2022-08-27 19:35:17 +05:30
Mzack9999 30054d1fb6
Adding advanced template filtering (#2374)
* Adding advanced template filtering

* fixing bug in slice

* refactoring tests

* adding test cases

* increasing error verbosity

* fixing quoted fields with spaces

* adding more test cases

* fixing merge error

* fixing lint errors

* switching to []string

* updating tag filter tests

* updating functional tests

* fixing functional test cases

* updating syntax
2022-08-25 16:52:08 +05:30
Sami 0aac36a44b
added custom config flag (#2399)
* added custom config flag

* config.yaml file in custom  directory

* lint error fix

* few updates and error checks

* fix lint error

* copy config.yaml file if the dest folder does not exist

* lint error check

* added integration test

* improved test cases

* lint error fix
2022-08-25 16:10:07 +05:30
Ice3man 7b7936b7a5
Added show-actions flag to display headless actions (#2456)
* Added show-actions flag to display headless actions

* misc update

* readme update

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-08-25 10:43:32 +05:30
Ice3man 77c81834b2
Fixed loader templateConfig nil pointer crash (#2486) 2022-08-24 19:32:56 +05:30
Ice3man 67d5769cd9
Added initial catalog interface implementation (#2318)
* Added initial catalog interface implementation

* Added OpenFile to Catalog + disk catalog implementation

* Fixed merge issues

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-08-10 11:05:58 -07:00
Mzack9999 b942ddc6ad
Fixing map race condition (#2340) 2022-07-26 18:30:15 +05:30
Mike Rheinheimer 9efba05e0c
expose hosterrorscache.Cache as an interface (#2291)
* expose hosterrorscache as an interface, change signature to capture the error reason

* use the hosterrorscache.CacheInterface as struct field so users of Nuclei embedded can provide their own cache implementation

Co-authored-by: Mike Rheinheimer <mrheinheimer@atlassian.com>
2022-07-19 02:05:53 +05:30
invist db727db006
Optionally disable templates syntax strict check (#2266)
* nuclei::templates|define strict option (default)

* renaming flag and internal variable

* misc flag update

Co-authored-by: c-f <you@example.com>
Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-07-13 17:00:11 +05:30
Sami 6c2fdd3387
Issue 2227 ntv flag run new templates added in specific version (#2275)
* ntv flag to run templates added in specified version

* added missing arguments

* misc update

* added functional test and err check

* updated the min version

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-07-13 16:49:06 +05:30
Sajad be73fde0db
use go-homedir instead of standard os.userhomedir (#2262)
* use go-homedir instead of standard os.userhomedir

* set r.templatesConfig before write attempt to avoid panic
2022-07-13 13:33:13 +05:30
Mzack9999 476773ff8c
Replacing hasstdin with helper library (#2191)
* Replacing hasstdin with helper library

* adding timeout reader on stdin

* adding large input read timeout

* reducing stdin timeout + nostdin flag

* go mod update

* readme update

* go mod tidy

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-06-30 17:51:54 +05:30
Ice3man 7875b06fc8
Added exclude-matchers support for template & matchers (#2218)
* Added exclude-matchers support for template & matchers

* Fixed panics due to typo

* Added support for only template ID + misc cleanup
2022-06-24 23:09:27 +05:30
Mzack9999 0b351e83f3
Add self diagnostic functionality (#2178)
* Adding Self-Diagnostic

* adding comment
2022-06-22 00:40:10 +05:30
Ice3man 3648c47e35
Fixed template validation edge cases (#2051) 2022-05-25 11:26:05 +05:30
Mzack9999 184f3fdc30
Fixing concurrent read/write map (#1989) 2022-05-17 14:38:48 +05:30
Mzack9999 a534b9f06c
Fixing directory walk error check on windows (#1951)
* Fixing directory walk error check on windows

* moving check to helper package

* replacing godirwalk with standard library
2022-05-08 12:22:21 +05:30
LuitelSamikshya 3ccbfe4626 disable redirects flag 2022-04-27 11:19:44 -05:00
sandeep 1b953655d1 misc updates 2022-04-19 16:14:49 +05:30
LuitelSamikshya 5eae7159c7 nil value check 2022-04-18 17:21:33 -05:00
Sami ce79a8dc57
Template folder exists changes (#1825) 2022-04-11 11:59:22 +05:30
Sandeep Singh c5712c98f4
Fixed the default behavior of srd flag (#1817)
* fix: fixed automatic storing of debug response

* small code refactor

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2022-04-06 16:49:36 +05:30
Mzack9999 0bab297328
improving default template path logic (#1806) 2022-04-05 04:03:49 +05:30
Sami 301307bb77
Issue 1705 save responses on disk (#1727)
* save response on disk

* lint error check

* store raw request/response

* lint error fix

* file path

* mock test fix

* readme update

* .txt extension

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-04-02 00:59:02 +05:30
Sandeep Singh 8ee4f9f797
Merge branch 'dev' into proxy-internal-requests 2022-03-23 23:12:51 +05:30
mzack e9125f8fca Fix for nil unexpected return value in workflow execution 2022-03-21 11:33:10 +01:00