Commit Graph

148 Commits (a942c05d22def0f224a45c5a172eb279ae61caea)

Author SHA1 Message Date
Tarun Koyalwar dc44105baf
nuclei v3 : misc updates (#4247)
* use parsed options while signing

* update project layout to v3

* fix .gitignore

* remove example template

* misc updates

* bump tlsx version

* hide template sig warning with env

* js: retain value while using log

* fix nil pointer derefernce

* misc doc update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-10-17 17:44:13 +05:30
sandeep 32de68d3a5 Merge branch 'v3-beta', remote-tracking branch 'origin' into dev 2023-10-16 15:00:00 +05:30
Tarun Koyalwar c35162c8ef
nuclei v3 bug fixes (#4176)
* store and generate signer keys

* fix trailing newline in code_response

* fix formatting and update error string

* fix integration test

* fix rsaSigned code integration test

* bug fixes , docs and more

* bump go -> 1.21

* use 'response' as default part in code templates

* disable sourcemaps for all js runtimes

* disable eval function

* rewrite file validation in sandbox mode

* sandbox file read improvements + minor refactor

* refactor sign and verify logic

* fix panic and missing id in code protocol

* disable re-signing code protocol templates

* fix code resigning in tests

* allow -lfa in test for signing templates

* start index from 1 in flow and multiproto

* remove testfiles

* add python in integration test

* update code protocol docs

* add python engine in template

* rework template signer

* fix integration test and more

* reworked template signer

* fix lint error

* display signature stats

* update docs

* add user fragment to signature

* use md5 to generate fragment

* update docs with code re-sign

* misc updates

* public crt update

* remove workflow info statement

* fix printing issues

* refactor preprocessor logic

* remove debug statement

* fix failing example test

* go mod tidy

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2023-10-13 13:17:27 +05:30
Keith Chason 77a03ee90f
Automated Docker Hub Documentation (#4148)
* Push README to Docker Hub

* env updates

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-09-16 14:25:36 +05:30
Tarun Koyalwar eec907a370 resolve merge conflicts 2023-09-13 20:28:48 +05:30
Mzack9999 643e9034cf
add weekly performance test with 1k targets (#3861)
* add weekly performance test with 1k targets

* enabling gh action on PR for test

* testing big list

* updating action

* lowering targets to 250

* lowering targets amount

* workflow update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-09-06 23:41:34 +05:30
Tarun Koyalwar 2d317884b5
SDK: abstracted and minimal nuclei v3 sdk (#4104)
* new sdk progress

* nuclei v3 new sdk/library

* fix TestActionGetResource broken link

* fix clistats + clustering and more

* fix lint error

* fix missing ticker

* update advanced library usage example

* fix integration tests

* misc update

* add utm_source and fix lint error

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-09-02 14:34:05 +05:30
sandeep 1ade5a0824 version update 2023-08-18 01:21:43 +05:30
Josh Soref 4c1c5301b9
Spelling (#4008)
* spelling: addresses

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: asynchronous

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: basic

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: brute force

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: constant

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: disables

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: engine

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: every time

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: execution

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: false positives

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: from

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: further

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: github

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: gitlab

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: highlight

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: hygiene

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: ignore

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: input

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: item

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: itself

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: latestxxx

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: navigation

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: negative

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: nonexistent

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: occurred

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: override

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: overrides

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: payload

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: performed

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: respective

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: retrieve

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: scanlist

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: separated

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: separator

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: severity

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: source

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: strategy

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: string

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: templates

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: terminal

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: timeout

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: trailing slash

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: trailing

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: websocket

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

---------

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2023-08-02 00:03:43 +05:30
sandeep d40ecc5b62 workflow clean up 2023-07-25 22:56:34 +05:30
sandeep 4005b8bf3f dep workflow update 2023-07-25 22:55:21 +05:30
sandeep eaa6209a77 misc update 2023-07-12 05:27:57 +05:30
Dogan Can Bakir 880adc4405
Automate merging of dependabot PRs to dep branch (#3857)
* add auto sync and merge workflows

* misc update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-07-12 05:24:55 +05:30
sandeep 238ed0e9aa added release template 2023-06-27 01:04:33 +05:30
Mzack9999 c9d0942bc1
Extend headless contextargs (#3850)
* extend headless contextargs

* using darwin-latest

* grouping page options

* temp commenting code out

* fixing test

* adding more checks

* more checks

* fixing first navigation metadata

* adding integration test

* proto update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-06-26 22:55:51 +05:30
dependabot[bot] 9a1c3d0358
chore(deps): bump golangci/golangci-lint-action from 3.5.0 to 3.6.0 (#3837)
Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 3.5.0 to 3.6.0.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases)
- [Commits](https://github.com/golangci/golangci-lint-action/compare/v3.5.0...v3.6.0)

---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-19 13:14:05 +05:30
dependabot[bot] 724863d7bc
chore(deps): bump golangci/golangci-lint-action from 3.4.0 to 3.5.0 (#3777)
Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 3.4.0 to 3.5.0.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases)
- [Commits](https://github.com/golangci/golangci-lint-action/compare/v3.4.0...v3.5.0)

---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-06 03:48:22 +05:30
sandeep 20bc5fc607 version update 2023-05-21 02:21:07 +05:30
Mzack9999 dfd4d5b855
Adding interact keepalive to reduce server-side id pruning (#3680)
* adding interact keepalive + improving init logic

* dep update

* go version update

* readme update

* version bump

* fixing invalid format

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-05-21 01:56:13 +05:30
Sandeep Singh 7ac9f05e53
updated interactsh version (#3621)
* updated interactsh version

* workflow update
2023-04-29 14:55:58 +05:30
Mzack9999 6f4b1ae48a
Replacing ccache with generic gcache (#3523)
* Replacing ccache with generic gcache

* fixing lint issues

* removing unecessary hashing + using errorutils

* making test more tolerant

* removing dead code + refactor

* removing redundant code

* removing race

* maint

* moving code

* adding more iterations

* note + typo

* temporary fixing stop-at-first-match with interact

* wrapping internal map with mux

* sort before running integration test

* fix deadlock in requestShouldStopAtFirstMatch

* add timeout to integration_test workflow

* attempting to remove outer lock

* adds interactsh protocol tests in integration_test

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-04-16 23:19:35 +05:30
dependabot[bot] a2e4b4197d
chore(deps): bump actions/setup-go from 3 to 4
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 3 to 4.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-20 05:57:09 +00:00
sandeep e1b1417a9b doc workflow update 2023-03-16 23:48:06 +05:30
Mzack9999 b3b2ec9baa
Merge pull request #3351 from projectdiscovery/issue-3081-retry-gh
Add retry script for build, test and integration commands
2023-03-06 09:53:11 +01:00
sandeep 0f80575c44 Update version number and add paths to pull request. 2023-03-06 01:03:30 +05:30
Tarun Koyalwar 3e53087617
fix missing port in matched ssl templates (#3380)
* add openssl support + fix missing port

* fix failing tests

* go mod update

* workflow update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-03-04 04:47:41 +05:30
shubhamrasal d32c9c7f56 Revert "Resolve the requested changes"
This reverts commit 4ad64eb357.
2023-03-01 19:06:30 +05:30
shubhamrasal 4ad64eb357 Resolve the requested changes
- Add the logic to retry the test case that is failed.
- Accpet the number of how many times we want to retry
- Run the last retry in debug
2023-02-24 13:42:19 +05:30
shubhamrasal ed46e9560e Revert "Add retry script for build, test and integration commands"
This reverts commit 63747a5c54.
2023-02-24 06:15:16 +05:30
shubhamrasal 311ea5bd01 Update integration tests wd 2023-02-21 20:00:48 +05:30
shubhamrasal 63747a5c54 Add retry script for build, test and integration commands 2023-02-21 19:50:52 +05:30
Tarun Koyalwar a62e57f6c0
fix publish docs (#3296)
* add branch ref while checkout

* Auto Generate Syntax Docs + JSONSchema [Thu Feb  9 13:36:50 UTC 2023] 🤖

---------

Co-authored-by: GitHub Action <action@github.com>
2023-02-09 19:14:43 +05:30
Sandeep Singh 10dd4d7540
added branch ref (#3278) 2023-02-07 04:50:23 +05:30
dependabot[bot] 5bd0f9bf69
chore(deps): bump docker/build-push-action from 3 to 4 (#3266)
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 3 to 4.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-07 04:49:09 +05:30
dependabot[bot] ed5ab5118a
chore(deps): bump golangci/golangci-lint-action from 3.3.1 to 3.4.0
Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 3.3.1 to 3.4.0.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases)
- [Commits](https://github.com/golangci/golangci-lint-action/compare/v3.3.1...v3.4.0)

---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-30 05:00:58 +00:00
Mzack9999 0b3992cdb8
moving examples to go file (#3187) 2023-01-12 14:57:32 +05:30
Sandeep Singh a6a26ec026
Adding GH Stale bot (#3137)
* Adding GH Stale bot

* Added onlyLabels to work with specific labels to start with
2023-01-03 23:54:42 +05:30
Sandeep Singh f69b42b8da
Added release announcement to discord server (#3120) 2023-01-02 16:47:27 +05:30
xm1k3 34120fbecc
#3046 persistent failed item status and #2065 failed items reporting error once (#3047)
* added logs for debug

* fixes

* removed logs

* using cache item

* implemented multiple tests

* fixed some unit tests

* implemented test for skipping

* added multiple tests together

* added mark failed

* fix on tests

* better test implementation + concurrent

* fix: fixes on concurrent tests

* removed parallel and 1 unit test

DOCS: by default the command go test runs in parallel tests for different packages, and default is the number of CPUs available (see go help build)

* fixes on go routine

* increasing parallelism of once.Do

* bumping go to 1.19 for atomic types support

* removing redundant check + fixing test concurrency on create

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
2023-01-02 13:52:06 +05:30
Sandeep Singh 42b6a7df38
Publish Docs workflow fix (#3104) 2022-12-29 19:59:36 +05:30
Tarun Koyalwar aee0870617
scanallip handle edge cases (#3080)
* bug fix:remove port during dns resolution

* scanallip fix edge cases

* add scanallips testcases

* workflow fix

* removing pull cmd

* Auto Generate Syntax Docs + JSONSchema [Sat Dec 24 13:29:21 UTC 2022] 🤖

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: GitHub Action <action@github.com>
2022-12-24 19:03:23 +05:30
Sandeep Singh 96646c8f53
cloud templates targets sync (#2959)
* Add s3 bucket template provider

- Refactor the custom github template code
- add interface for template provider

* Validate if aws creds are passed if bucket flag

- refactor s3 provider struct to take client
- add function which returns the aws s3 client
- update error messages

* Add aws s3 bucket flags documentation in README.md

- Rename the github_test.go to customTemplate_test.go

* go mod update

* Move template provider code to pkg/external/customtemplates dir

* Added initial data_source sync to cloud

* Misc

* Add pagination to scan output and scan list (#2858)

* Add pagination to scan output and scan list

* Use time based parameters instead of page numbers

* Fix linting errors

* Do not check limits at client, check at server

* Remove unused constant

* Misc update

* Removed unnecessary flags

* Misc

* Misc

* Misc endpoint additions

* Added more routes

* Typo fix

* Misc fixes

* Misc

* Misc fixes to cloud target logic + use int for IDs

* Misc

* Misc fixes

* Misc

* Misc fixes

* readme update

* Add JSON output support for list-scan option (#2876)

* Add JSON output support for list-scan option

* Fix typo in cloud JSON output description

* Following changes

- Update status(finished, running) to be lower-case by default
- Convert status to upper-case in DisplayScanList()

* Update status to be lower-case by default

* Remove additional json flag, instead use existing

* Merge conflict

* Accomodate comment changes and restructure code

Co-authored-by: Jaideep K <jaideep@one2n.in>

* Use integer IDs for scan tasks

* Added get-templates-targets endpoint + JSON + validation

* Added target count list

* misc option / description updates

* Added changes as per code review

* duplicate options + typo updates

* Added tablewriter for tabular data writing by default

* Fixed list scan endpoint

* Review changes

* workflow fix

* Added cloud tags etc based filtering (#3070)

* Added omitempty for filtering request

* go mod tidy

* misc format update

Co-authored-by: shubhamrasal <shubhamdharmarasal@gmail.com>
Co-authored-by: Ice3man <nizamulrana@gmail.com>
Co-authored-by: Jaideep Khandelwal <jdk2588@gmail.com>
Co-authored-by: Siddharth Shashikar <60960197+shashikarsiddharth@users.noreply.github.com>
Co-authored-by: Jaideep K <jaideep@one2n.in>
2022-12-21 22:48:43 +05:30
Tarun Koyalwar 63150a8ee4
add stop-at-first-match for headless protocol (#3034)
* headless: adds stop-at-first-match

* GH workflow fix

* misc fix

* headless use spm option

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-12-19 19:52:17 +05:30
dependabot[bot] e44c464c58
chore(deps): bump goreleaser/goreleaser-action from 3 to 4
Bumps [goreleaser/goreleaser-action](https://github.com/goreleaser/goreleaser-action) from 3 to 4.
- [Release notes](https://github.com/goreleaser/goreleaser-action/releases)
- [Commits](https://github.com/goreleaser/goreleaser-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: goreleaser/goreleaser-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-12-19 05:00:59 +00:00
Sandeep Singh 0e35133ad5
public docs workflow fix (#3016) 2022-12-10 20:54:31 +05:30
sandeep 2230e0be49 misc workflow update 2022-12-05 23:26:25 +05:30
Sandeep Singh 5c8e715fc6
workflows improvements (#2963) 2022-12-05 18:22:55 +05:30
sandeep 68a1091056 workflow fix 2022-12-04 23:24:59 +05:30
Sandeep Singh a4e45ed6c5
workflow fix to test with current code (#2944) 2022-12-02 04:35:02 +05:30
Shubham Rasal d5a09e733a
Issue 2772 s3 provider support (#2825)
* Add s3 bucket template provider

- Refactor the custom github template code
- add interface for template provider

* Validate if aws creds are passed if bucket flag

- refactor s3 provider struct to take client
- add function which returns the aws s3 client
- update error messages

* Add aws s3 bucket flags documentation in README.md

- Rename the github_test.go to customTemplate_test.go

* go mod update

* Move template provider code to pkg/external/customtemplates dir

* Remove github and aws update variables from flag

* Rename CustomTemplateProvider to Provider

* Update integration and function command in makefile

* Update github test case, accept token

* readme update

* go mod tidy

* Update build-test.yml

* handle empty dir in s3

* Add requested changes

- download/update s3 and github only when `-ut` is passed
- only print the missing env variable for s3
- add the custom templates path in
  ~/.config/nuclei/.template-config.json

* print custom paths only if exists in config file

* misc update

* tag update

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2022-12-02 03:57:00 +05:30