Commit Graph

471 Commits (64adad131ff8a1bda9087349e50134995c2e6258)

Author SHA1 Message Date
Pj Metz 6c79602927
Readme sj flag fix (#3579)
* fixed -sj flag description for readme.md and readme_ID.md

* misc update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-04-20 19:08:02 +05:30
sandeep 871e7016fc version update 2023-04-20 03:39:33 +05:30
Keith Chason dcb003211c
Gitlab Custom Templates (#3570)
* Configuration options for GitLab template pulls

* GitLab client creation

* GitLab hooks and property renames

* Fix filesystem writing and update environment variables

* Fix type error in formatted error message

* Migrate directory config to new nucleiconfig file

* refactor + add custom templates to tm

* typo fix + only show installed ct with -tv

* add default gitlab url if not given

* fix template valid failure

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-04-20 03:12:52 +05:30
Mzack9999 978d0bcc23
Replacing goos with osutils (#3571)
* Replacing goos with osutils

* pleasing his majesty the linter
2023-04-19 23:00:15 +05:30
Tarun Koyalwar bf08913cd0
update logic + config management refactor (#3567)
* adds template manager

* refactor: checkpoint

* centrailized config & template download logic

* refactor removed unused code

* use global template directory

* update related bug fixes

* bug fix create cfg dir if missing

* fix lint error

* bug fix skip writing template dir in callback

* misc update

* remove unused code

* use strings.equalfold for comparison

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-04-19 21:58:48 +05:30
Mzack9999 6f4b1ae48a
Replacing ccache with generic gcache (#3523)
* Replacing ccache with generic gcache

* fixing lint issues

* removing unecessary hashing + using errorutils

* making test more tolerant

* removing dead code + refactor

* removing redundant code

* removing race

* maint

* moving code

* adding more iterations

* note + typo

* temporary fixing stop-at-first-match with interact

* wrapping internal map with mux

* sort before running integration test

* fix deadlock in requestShouldStopAtFirstMatch

* add timeout to integration_test workflow

* attempting to remove outer lock

* adds interactsh protocol tests in integration_test

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-04-16 23:19:35 +05:30
Dogan Can Bakir bca0c53a41
Enable --nc option if OS is windows (#3539)
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2023-04-13 18:11:04 +05:30
sduc 7029741338
[DRAFT] Annotate CVE: add configurations metadata (#3486)
* improve annotate to use new nvd apis

* annotate

* improvements

* fix mod files

* fetch EPSS and only write CPE

* lint fixes

---------

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-04-12 16:19:49 +05:30
Shubham Rasal 45cc676f96
Evaluate payload variables (#3503)
* Evaluate payload variables

* Add variables evaluation

* Extend variables test

- to check evaluation of global variables in variables
- to check evaluation of golbal variables in payload

* Add default and cli variables to websocket, whois and dns proto

- use url.Parse with urlutil.Parse
2023-04-12 01:50:58 +05:30
Keith Chason 3476f4d1d6
JSONL(ine) Export (#3504) (#3505)
* Add initial hooks for JSONL export

* Add newline character after each result

* fix integration test (#3506)

* fix integration test

* fix interactsh fatal error

* fix default report-config.yaml

---------

Co-authored-by: Tarun Koyalwar <45962551+tarunKoyalwar@users.noreply.github.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-04-08 17:44:41 +05:30
Tarun Koyalwar a185348194
fix integration test (#3506)
* fix integration test

* fix interactsh fatal error
2023-04-04 10:09:52 +08:00
Keith Chason 4d96025bec
JSON Export Handling Updates (#3466)
* Switch -json to -jsonl

* Add JSON output file

* Update docs for EN and ID

* Fix linting issue with error wrap

* Add -j flag

* Fix call for short flag

* Correct typo "Ciper" to "Cipher" (#3468)

* migrate dsl helper functions to dsl repo (#3461)

* migrate dsl pkg code to dsl repo

* fix lint error

* upgrade dsl dependency

* upgrade deps

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>

* chore(deps): bump github.com/projectdiscovery/httpx in /v2 (#3469)

Bumps [github.com/projectdiscovery/httpx](https://github.com/projectdiscovery/httpx) from 1.2.7 to 1.2.9.
- [Release notes](https://github.com/projectdiscovery/httpx/releases)
- [Changelog](https://github.com/projectdiscovery/httpx/blob/main/.goreleaser.yml)
- [Commits](https://github.com/projectdiscovery/httpx/compare/v1.2.7...v1.2.9)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/httpx
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/weppos/publicsuffix-go in /v2 (#3472)

Bumps [github.com/weppos/publicsuffix-go](https://github.com/weppos/publicsuffix-go) from 0.20.0 to 0.30.0.
- [Release notes](https://github.com/weppos/publicsuffix-go/releases)
- [Changelog](https://github.com/weppos/publicsuffix-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/weppos/publicsuffix-go/compare/v0.20.0...v0.30.0)

---
updated-dependencies:
- dependency-name: github.com/weppos/publicsuffix-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/projectdiscovery/wappalyzergo in /v2 (#3473)

Bumps [github.com/projectdiscovery/wappalyzergo](https://github.com/projectdiscovery/wappalyzergo) from 0.0.81 to 0.0.88.
- [Release notes](https://github.com/projectdiscovery/wappalyzergo/releases)
- [Commits](https://github.com/projectdiscovery/wappalyzergo/compare/v0.0.81...v0.0.88)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/wappalyzergo
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/projectdiscovery/hmap in /v2 (#3470)

Bumps [github.com/projectdiscovery/hmap](https://github.com/projectdiscovery/hmap) from 0.0.10 to 0.0.11.
- [Release notes](https://github.com/projectdiscovery/hmap/releases)
- [Commits](https://github.com/projectdiscovery/hmap/compare/v0.0.10...v0.0.11)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/hmap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* debug catalog path

* use paths instead of filepath for aws path

* deps update (#3477)

* deps update

* fixing gologger via callback

* Moved `json-export` flag to the other exporters

* Switch "json[-_]exporter to jsonexporter"

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Ramana Reddy <90540245+RamanaReddy0M@users.noreply.github.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: shubhamrasal <shubhamdharmarasal@gmail.com>
2023-03-31 15:29:29 +05:30
Mzack9999 d4acc471c5 better fmt 2023-03-20 00:48:39 +01:00
Mzack9999 50e178fbe8 Merge branch 'dev' into pr/3406 2023-03-20 00:24:33 +01:00
Mzack9999 1b7585476e simplify execution logic 2023-03-17 17:31:28 +01:00
Mzack9999 6a0db1c234 restoring tests 2023-03-17 16:56:14 +01:00
Mzack9999 c115768b8e investigating tests 2023-03-17 15:39:12 +01:00
Mzack9999 c182434130
json templates support (load with flags, run & validate ) (#3424)
* extending template identification logic

* removing test code

* local debug

* json template loading support using flags

* blacklist meta json files

* minor changes

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-03-16 13:33:59 +05:30
Ramana Reddy c9634fae72
Issue 3350 matcher condition or not work (#3397)
* fix or condition match even interactsh includes as matcher-part (#3350)

* add integration test

* add new template to integration test

* matcher-condtion: test case for both conditions

* fix lint errors

* upgrade dependencies

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-03-15 20:45:44 +05:30
Mzack9999 0bf8fc027d
Fixing nil pointer reference + use map helper (#3421)
* Fixing nil pointer reference + use map helper

* bump tlsx version to v1.0.6

* increase interactsh polling in integration_test

* fix nil pointer dereference in integration_test

* fix lint error

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-03-14 21:27:48 +05:30
Austin Traver 0d90a555f6
adds `-track-error` option to add custom errors to max-host-error watchlist (#3399)
* Allow user to specify for "context deadline exceeded" errors to count toward the max host error count

* Convert flag to a string slice `--track-error`

* Minimize diff

* Add documentation for `-track-error`

* adds unit test & minor improvements

* update flag description

---------

Co-authored-by: Austin Traver <austin_traver@intuit.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-03-14 13:59:42 +05:30
Jordan Potti 6ab4bf25f9 Support for Jira custom fields 2023-03-10 15:55:54 -07:00
Shubham Rasal 572c8eb780
Issue 2987 fuzz options (#3355)
* Add override fuzzing type and mode flags

* Update english readme

* Fix failing tests

* Add the integration tests

- validate the command line overriding type and mode for fuzzing
2023-03-06 16:56:38 +05:30
sandeep c91588830e win os test fix 2023-03-06 01:57:55 +05:30
Tarun Koyalwar d9e953acfa
fix file input in custom vars for self contained http template (#3385)
* fix file input in variables(-V)

* fix lint error

* fix nuclei-ignore file failures
2023-03-04 04:57:27 +05:30
xm1k3 bab15e122b http.DefaultClient replaced with retryablehttp 2023-03-02 21:21:04 +01:00
xm1k3 886fdcf0a9 started move to retryablehttp 2023-03-02 14:54:01 +01:00
Mzack9999 d80cbef51d
Merge pull request #3333 from CodFrm/main
fix some json deserialization issues
2023-02-28 09:02:04 +01:00
Mzack9999 62af038617
Add template sign/verify functionality (#3029)
* add template sign/verify functionality

* fixing syntax
2023-02-26 03:54:46 +05:30
Tarun Koyalwar 21b03a2e8a
bug fix in url path and adds integration tests (#3331)
* fix unsafe edgecases+ adds integration test

* bug fixes and more url testcases

* upgrade cfssl

* fix template id in integration test
2023-02-20 22:26:04 +05:30
王一之 10693efa05 add integration tests 2023-02-17 14:23:28 +08:00
xm1k3 cc2f796d2f
Helper function for IP Formats (#3286)
* implemented ip_format helper function

* added tests on ip_format() helper

* fixes on logic

* fixes related to mapcidr docs

* better error value

* fixes + unit test to check index 11

* added call for integration tests

* fixes on dsl-functions number

---------

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2023-02-17 01:47:40 +05:30
Mzack9999 d608ffaeb2
clear after stop (#3312)
* clear after stop

* fixing data races

* adding atomic cache

* fixing lint errors

* fixing imports
2023-02-13 16:46:41 +05:30
Mzack9999 1fbd2c069a
adding json schema for extractor (#3240)
* adding json schema for extractor

* adding argument validation
2023-02-07 14:01:34 +05:30
Tarun Koyalwar 628628893c
fix make http request inconsistencies (#3243)
* fix make http request inconsistencies

* remove parameters from http vars

* fix trailingslash unit test

* naming conventions: best practices

* fix publish docs action

* remove branch ref from push
2023-02-01 17:23:28 +05:30
Tarun Koyalwar 0b2a3e296a
fix url encoding issues and inconsistencies (#3211)
* fix url encoding issues

* complete requested changes and improvements

* fix missing issue-tracker-config.yaml

* fuzz: deepcopy and use urlutil.Params
2023-01-24 22:04:52 +05:30
Mzack9999 6c56a20544
Adding support for nmhe (#3219)
* adding support for nmhe

* updating docs
2023-01-22 15:08:50 +05:30
Jaideep Khandelwal a1642be911
Enable/Disable a reporting source (#3183)
* Enable/Disable a reporting source

* misc options update

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-01-15 22:21:16 +05:30
xm1k3 63f1b9c2e5
Add timestamp for debug logs (#3110)
* replaced and tested logs on debug

* timestamp on debug logs

* removed debug.timestamp and used instead new implementation on gologger

* renamed global function to SetTimestamp(bool)

* used logger

* used DefaultLogger

* updated gologger ref

github.com/projectdiscovery/gologger@1c357ca

* bumping gologger

* resolved problem related to failing test

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2023-01-10 21:01:16 +05:30
Mzack9999 d956275e98
Adding content length edge cases (#3147)
* adding content length edge cases

* fixing CL behavior

* suppressing -1 error

* fixing path
2023-01-07 18:06:44 +05:30
Mzack9999 8beb6b06f9
Extending advanced filtering (#3146)
* adding more metadata to advanced filtering

* adding functional test cases

* converting metadata to lowercase

* misc update

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-01-05 21:59:58 +05:30
Mzack9999 a96f764959
extending request/response hijacking with native calls (#3091)
* extending request/response hijacking with native calls

* fixing tests
2023-01-05 17:26:18 +05:30
Sandeep Singh c273cbc8cb
Merge pull request #3134 from projectdiscovery/issue-3131-unused-code
Removing unused code in integration tests
2023-01-04 11:59:12 +05:30
Tarun Koyalwar e66ed30cec
fix missing trailing slash (#3127)
* raw: fix missing trailing slash

* adds rawpath integration test

* rename trailing slash test
2023-01-03 23:45:34 +05:30
Mzack9999 463f4dd8c8 removing unused code in integration tests 2023-01-03 08:24:33 +01:00
Taufik Mulyana 7093180b6d
Add support to query DNS TLSA record (#3054)
* Add support to query DNS TLSA record

* fix build test

* fix ci-lint

* set expected to 0

* test domain update

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-12-28 21:12:02 +05:30
Mzack9999 31ee08ecf0
Initialize flags with nil slice (#3089) 2022-12-28 02:29:51 +05:30
Tarun Koyalwar ff17d12ced
adds scan strategy (#3075) 2022-12-28 02:18:00 +05:30
Mzack9999 34976029d3
removing most go routine leaks (#3073)
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-12-24 19:22:14 +05:30
Sandeep Singh 96646c8f53
cloud templates targets sync (#2959)
* Add s3 bucket template provider

- Refactor the custom github template code
- add interface for template provider

* Validate if aws creds are passed if bucket flag

- refactor s3 provider struct to take client
- add function which returns the aws s3 client
- update error messages

* Add aws s3 bucket flags documentation in README.md

- Rename the github_test.go to customTemplate_test.go

* go mod update

* Move template provider code to pkg/external/customtemplates dir

* Added initial data_source sync to cloud

* Misc

* Add pagination to scan output and scan list (#2858)

* Add pagination to scan output and scan list

* Use time based parameters instead of page numbers

* Fix linting errors

* Do not check limits at client, check at server

* Remove unused constant

* Misc update

* Removed unnecessary flags

* Misc

* Misc

* Misc endpoint additions

* Added more routes

* Typo fix

* Misc fixes

* Misc

* Misc fixes to cloud target logic + use int for IDs

* Misc

* Misc fixes

* Misc

* Misc fixes

* readme update

* Add JSON output support for list-scan option (#2876)

* Add JSON output support for list-scan option

* Fix typo in cloud JSON output description

* Following changes

- Update status(finished, running) to be lower-case by default
- Convert status to upper-case in DisplayScanList()

* Update status to be lower-case by default

* Remove additional json flag, instead use existing

* Merge conflict

* Accomodate comment changes and restructure code

Co-authored-by: Jaideep K <jaideep@one2n.in>

* Use integer IDs for scan tasks

* Added get-templates-targets endpoint + JSON + validation

* Added target count list

* misc option / description updates

* Added changes as per code review

* duplicate options + typo updates

* Added tablewriter for tabular data writing by default

* Fixed list scan endpoint

* Review changes

* workflow fix

* Added cloud tags etc based filtering (#3070)

* Added omitempty for filtering request

* go mod tidy

* misc format update

Co-authored-by: shubhamrasal <shubhamdharmarasal@gmail.com>
Co-authored-by: Ice3man <nizamulrana@gmail.com>
Co-authored-by: Jaideep Khandelwal <jdk2588@gmail.com>
Co-authored-by: Siddharth Shashikar <60960197+shashikarsiddharth@users.noreply.github.com>
Co-authored-by: Jaideep K <jaideep@one2n.in>
2022-12-21 22:48:43 +05:30