Commit Graph

1767 Commits (6534deedc6cb29ce7910c21f8bd982cd3f0832c4)

Author SHA1 Message Date
Shubham Rasal 6b142d794a
Issue 2254 uncover integration (#2786)
* nuclei -uq 'vuln:CVE-2021-26855' -t cves/2021/

- `nuclei -uq 'vuln:CVE-2021-26855' -t cves/2021/`

* Add automatic template execution using metadata

- Query uncover after the template is loaded.
- Add the received hosts to the input provider from uncover
- Make NormalizeStoreInputValue() function public to add hosts from the
  runner after uncover hosts received.

* run go mod tidy

* Remove unnecessary comments

* Resolve the requested changes

- move uncover code to protocols/common/uncover package
- Use uncover delay to create uncover rate limiter
- Use single ratelimiter object and remove not required ratelimiters
- Create Set() method for input provider interface
- Rename normalizeStoreInputValue to Set() method

* Solved the uncover running twice.

- flag StringSliceVarP adds the default value twice in the variable
- Check if provider keys exists or not
- Add uncover help block to english readme.md

* Add uncover field functionality

- ./nuclei -uq 'vuln:CVE-2021-26855' -t dns -duc -uf host
- ./nuclei -uq 'vuln:CVE-2021-26855' -t dns -duc -uf ip:port

* Update error messages and solve nuclei hang for wrong uncover engine

- Get uncover engine values from uncover package

* Resolve merge conflicts

* misc option update

* Update logging for templates

- remove duplicate env log printing
- Log message for template queries

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-11-16 11:12:39 +05:30
forgedhallpass 0295ca19bc
Add `split` DSL function (#2838)
* Add support for showing overloaded DSL method signatures

* Add `split` DSL function #2837

* fixing lint warnings

* replacing faulty regex with strings methods

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
2022-11-14 06:08:12 +05:30
Mzack9999 1fbbca66f9
Adding support to scan all v4/v6 IPs (#2709)
* Adding support to scan all v4/v6 IPs

* adding tests

* metainput prototype

* using new signature

* fixing nil pointer

* adding request context with metadata

* removing log instruction

* fixing merge conflicts

* adding clone helpers

* attempting to fix ipv6 square parenthesis wrap

* fixing dialed ip info

* fixing syntax

* fixing output ip selection

* adding integration tests

* disabling test due to gh ipv6 issue

* using ipv4 only due to GH limited networking

* extending metainput marshaling

* fixing hmap key

* adding test for httpx integration

* fixing lint error

* reworking marshaling/id-calculation

* adding ip version validation

* improving handling non url targets

* fixing condition check
2022-11-09 18:48:56 +05:30
Mzack9999 840028fa93
adding host with optional port type logic (#2815)
* adding host with optional port type logic

* adding comment support in test cases

* updating test cases with multiple input scenarios

* readding port condition
2022-11-08 19:49:45 +05:30
Mzack9999 6ac669eb43
Replacing rdap with fixed fork (#2819)
* Replacing rdap with pd fixed fork

* mod tidy

* updating rdap commit ref

* reworking rdap client pool

* removing unused code
2022-11-08 17:27:18 +05:30
vrenzolaverace 2aaf2a2158
Use utils helpers libraries (#2809) (#2810)
* Use utils helpers libraries (#2809)

* Use utils helpers libraries (#2809)
2022-11-07 01:54:23 +05:30
Shubham Rasal 721c4964d7
Issue 2613 custom template GitHub (#2630)
* Add custom template download/update support from github

- Accept the -gtr flag to accept the list of custom template
  repos(public/private)
- Accept the -gt flag for github token. It internally sets os.Env
  variable
- Update the flags from
   - -update to -nuclei-update for nuclei self update
   - -ut to -tup for template-update
   - -ud to -tud for custom template location
- Add github.go file which has code related to download and update
  custom templates repos.

* Reslove golint and test case error

* Take default template from community directory

- No need to give explicit community directory path.
- Update the integration test to support the change in path

* Update functional test script update template flag

* Update the path from community to nuclei-template

- Revert the code changes that were made to add community directory

* remove the comment

* Update the interactsh server url for testing

* Update race condition command

* update race condition cmd to download the templates

* Debug integration test failure

* update integration test to update templates

* Refactor downloadCustomTemplate function.

- Remove the log prining instead send the message.

* Add test case for custom template repo download

* move the download repo for loop into diff function

* refactor updateTemplate function.

* Create struct for github repos.

- Create customtemplate struct for repo.
- Add functions to customtemplate

* update readme.md file

* Refactor the downloadCustomTemplate function

- create const variables for github & community as template type
- Update gologger to INF
- Validate templateUpdate to accept only github & community value.
- Validate tempalteUpdate require githubTemplateRepo

* Resolve requested changes

* go mod update

* misc option update

* test update

* Revert back update-template flag to boolean.

- to update community templates
  `nuclei -ut`
- to update custom templates
  `nuclei -ut -gtr ehsandeep/mobile-nuclei-templates`

* Update readme to update flag documentation

* Update go.mod

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-11-03 20:27:18 +05:30
Ice3man e1b0564c0a
Fixed nested expression replacement using fasttemplate (#2790) 2022-11-03 18:31:04 +05:30
GitHub Action 411e39c40b Auto Generate Syntax Docs + JSONSchema [Tue Nov 1 15:01:38 UTC 2022] 🤖 2022-11-01 15:01:38 +00:00
Ice3man b9472cf7e1
Added fuzzing support for query params + var dump feature (#2679)
* Added fuzzing support for query params + var dump feature

* Added query-fuzz integration test

* Fixed payloads + added keys-regex fuzz parameter

* Fixed interactsh not working + misc

* Fixed evaluation + added global variables/dsl support to payloads

* Misc fixes related to variables evaluations

* Added http variables support to fuzz

* misc

* Misc

* Added testing playground + misc renaming

* Added support for path and raw request to fuzzing

* Fixed fuzz integration test

* Fixed variable unresolved issue

* Add multiple parameter support with same name

* Added parameter value as 'value' dsl variable for parts

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2022-11-01 20:28:50 +05:30
Ice3man cd0517c887
Fixed invalid fields in request (#2788) 2022-11-01 16:10:41 +05:30
Sajad e163a301b9
add dns protocol ratelimit (#2763)
* add dns protocol ratelimit

* move dns ratelimit to just before sending the request
2022-10-29 04:29:13 +02:00
Mzack9999 cc0c20053a
Improving unsafe uri path (#2722)
* Improving unsafe uri path

* fixing raw path output
2022-10-27 20:09:38 +02:00
mzack fe798c07eb fixing function syntax 2022-10-25 21:01:42 +02:00
dependabot[bot] 1783207803
chore(deps): bump github.com/projectdiscovery/stringsutil from 0.0.1 to 0.0.2 in /v2 (#2751)
* chore(deps): bump github.com/projectdiscovery/stringsutil in /v2

Bumps [github.com/projectdiscovery/stringsutil](https://github.com/projectdiscovery/stringsutil) from 0.0.1 to 0.0.2.
- [Release notes](https://github.com/projectdiscovery/stringsutil/releases)
- [Commits](https://github.com/projectdiscovery/stringsutil/compare/v0.0.1...v0.0.2)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/stringsutil
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* updating function name

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2022-10-25 09:32:35 +02:00
Sajad 928f082109
set content_length as len(body) if response ContentLength is -1 (#2407)
* set content_length as len(body) if response ContentLength is -1

* move content-length calculation to utils

* adding basic tests

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2022-10-24 16:37:09 +02:00
Jaideep Khandelwal 4cfde111f4
Feature 18 cloud flags (#2708)
* Add cloud flags for nuclei.

* Add flag to get output for a particular scan ID

* Add some comments to the function.

* Get timestamp and id for scan list

* Fix linting errors

* Check if type is enumeration.

* Do not show deleted scans.

* Do not use filter_result, create client once and use it everywhere with
runner.

* Fix the output of scan list to be better

* Format the nuclei scan output list.

* Remove unused constant

* misc option update

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-10-22 04:06:52 +05:30
Ice3man 363ffb75db
Added probing for URL + input based on protocol (#2614)
* Added workflow names based condition

* Added conditional filtering to workflow executor

* Replaced names with single name stringslice

* Added probing for URL + input based on protocol

* Remove debug comments

* Fixed typo

* Fixed failing tests

* Fixed workflow matcher condition + tests

* Fixed workflow item name

* Switch to if-else

* Fixed review comment strict

* Increase bulk size

* Added default port for SSL protocol + misc changes

* Fixed failing tests

* Fixed misc changes to executer

* Fixed failing self-contained and offlinehttp tests

* Fixed atomic increment operation

* misc update

* Fixed failing builds

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-10-20 17:23:00 +05:30
Sajad 0149e94f90
add missing template filters to workflow loader (#2741) 2022-10-20 17:14:41 +05:30
Ice3man d956f08cb9
Added attack-type option to override template attack-type (#2724)
* Added attack-type option to override template attack-type

* Added docs + integration tests
2022-10-19 03:51:45 +05:30
Mzack9999 9493dfdb20
Adding automatic request condition detection (#2707)
* Adding automatic request condition detection

* adding missing checks on part

* test update as per latest change

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-10-15 15:19:04 +05:30
Shubham Rasal 897dbe8e4c
Add AS input support (#2712)
* Add AS input support

- Integrate mapcidr asn function to handle ASN number input support
- Check if input is ASN number or not. If yes then query for cidrs
- issue #2706

* Remove \r coz of failing test cases in windows os

* Replace newline char for windows

* remove extra line

* rename goldenfile dir to tests

* fixing folder name

Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
2022-10-14 00:25:00 +05:30
LuitelSamikshya 944d24a252 mics changes 2022-10-13 09:30:25 -05:00
LuitelSamikshya cb0da81a14 ratelimit library 2022-10-12 22:04:37 -05:00
Bertold Kolics 1af96fc679
Issue 2460: extended unit tests for replacer (#2691) 2022-10-10 22:00:43 +05:30
mzack 70cecf83fb Adding custom cancel function 2022-10-10 08:10:07 +02:00
Ice3man 09ceb29ba3 Fixed build error on 32bit arch 2022-10-08 01:55:18 +05:30
Ice3man 6c93d99745 Fixed failing lint tests 2022-10-08 00:10:32 +05:30
sandeep 299755b1a3 go mod + version update 2022-10-07 23:17:08 +05:30
Ice3man 9944f5e94e
Added response truncation support with flags (#2688)
* Added response truncation support with flags

* Fixed failing tests for no size
2022-10-07 20:10:00 +05:30
Shubham Rasal 3ebd1f689b
cidr input support #2665 (#2675)
* Add CIDR input support

- Add expandCIDRInputValue function which accepts the cidr,
  and stores the IPs into hmap. It uses mapcidr to get the expanded IPs
- Add test case to test expandCIDRInputValue and isCIDR function
- Update dsl_test.go which had typo. coz of failing test

* Resolve the requested changes
2022-10-07 14:28:02 +05:30
Ice3man 1fd83c03bc
Improved dsl test by removing coloring + fix datatime tests (#2654)
* Improved dsl test by removing coloring + fix datatime tests

* Fixed tests for dsl coloring
2022-10-06 03:18:05 +05:30
Mzack9999 33ed5e7c93
Merge pull request #2671 from bertold/bk/randomip-unittest
Unit test for GetRandomIPWithCidr and fixes
2022-10-04 12:29:41 +02:00
Bertold Kolics 087be32ae0 Added unit test for GetRandomIPWithCidr
Fixed an issue with handling non-zero based networks
Fixed an issue handling network addresses with single IPs
2022-10-03 18:30:55 -05:00
Ice3man e875da208e
Merge pull request #2653 from projectdiscovery/elasticsearch-host-support
Added elasticsearch host reporting field support
2022-10-03 16:47:18 +05:30
Mzack9999 781e4e6105
Shared Execution Context Prototype (#2576)
* renaming var

* Introducing shared execution context prototype

* fixing field name

* adding shared values propagation

* adding shared context lock

* add slice values normalization

* adding integration tests

* adding metadata support for dns

* adding multi-protocol context sharing test

* removing debug test files

* moving contextargs around

* adding comments

* refactoring code

- getter/setter for complex types
- using pointers to avoid heap allocations
2022-10-03 15:42:20 +05:30
Mzack9999 0b1ff2bfa4
Forcing conns to be gc-ed by default disabling keep-alive (#2642)
* Forcing conns to be gc-ed with keep-alive

* removing redundant code

keep-alive are disabled by default

* fixing merge conflict

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2022-09-30 04:25:56 +05:30
GitHub Action 935dd68355 Auto Generate Syntax Docs + JSONSchema [Wed Sep 28 22:44:26 UTC 2022] 🤖 2022-09-28 22:44:26 +00:00
Mzack9999 18f14b631c
Adding same host redirect support (#2655)
* simplifying test syntax

* adding same host redirect + refactoring redirect handling

* adding missing file

* adding support for template syntax

* adding integration test

* updating options

* fixing issue on same host redirect
2022-09-29 04:11:28 +05:30
Mzack9999 3d2374ab73
Merge pull request #2637 from projectdiscovery/dependabot/go_modules/v2/dev/github.com/go-rod/rod-0.111.0
chore(deps): bump github.com/go-rod/rod from 0.110.0 to 0.111.0 in /v2
2022-09-28 13:41:51 +02:00
mzack 2cf0005eaa updating syntax 2022-09-28 01:51:47 +02:00
Ice3man 6d96f75df7 Added elasticsearch host reporting field support 2022-09-28 01:25:51 +05:30
aprp 57b0d78dcf
add unit tests for extract.go (#2635) 2022-09-28 01:24:00 +05:30
Myung-jong Kim 9eea441b0e
Add `sort(list)`, `sort(string)`, `uniq(list)`, `uniq(string)` helper functions (#2372)
* Add feature in join() to sort a single string slice

Signed-off-by: Myung-jong Kim <mjkim610@gmail.com>

* Add sort helper function and related tests

Signed-off-by: Myung-jong Kim <mjkim610@gmail.com>

* Add uniq helper function and related tests

Signed-off-by: mjkim610 <mjkim610@gmail.com>

Signed-off-by: Myung-jong Kim <mjkim610@gmail.com>
Signed-off-by: mjkim610 <mjkim610@gmail.com>
2022-09-27 02:59:13 +05:30
Ice3man fc27fc94a5
Added default config generation for reporting options (#2605) 2022-09-27 02:40:34 +05:30
GitHub Action 3ad1fec1d2 Auto Generate Syntax Docs + JSONSchema [Tue Sep 20 21:37:56 UTC 2022] 🤖 2022-09-20 21:37:56 +00:00
Mzack9999 99c14f4c9c
implementation of rate limiter with bucket refill and unrestricted token burst (#2536)
* implementation of rate limiter with interval burst

* fixing import path

* fixing syntax

* adding tests

* fixing lint errors

* adding support for context

* moving rate limiter earlier to avoid hitting timeout
2022-09-19 17:09:28 +05:30
Ice3man 03c7e89d9e
Added workflow names based condition (#2594)
* Added workflow names based condition

* Added conditional filtering to workflow executor

* Replaced names with single name stringslice

* Fixed workflow matcher condition + tests

* Fixed workflow item name
2022-09-19 16:49:30 +05:30
mzack 37c016a5f0 fixing lint errors 2022-09-19 08:38:52 +02:00
Ice3man 04b47b0309
Added custom json-unmarshaller + misc updates (#2556)
* Added custom json-unmarshaller + misc updates

* Added support for nuclei-cloud based scan execution

* Removed unnecessary files

* Misc

* Changes as per review comments

* misc option update

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-09-19 01:13:59 +05:30