Commit Graph

128 Commits (64adad131ff8a1bda9087349e50134995c2e6258)

Author SHA1 Message Date
Mzack9999 6f4b1ae48a
Replacing ccache with generic gcache (#3523)
* Replacing ccache with generic gcache

* fixing lint issues

* removing unecessary hashing + using errorutils

* making test more tolerant

* removing dead code + refactor

* removing redundant code

* removing race

* maint

* moving code

* adding more iterations

* note + typo

* temporary fixing stop-at-first-match with interact

* wrapping internal map with mux

* sort before running integration test

* fix deadlock in requestShouldStopAtFirstMatch

* add timeout to integration_test workflow

* attempting to remove outer lock

* adds interactsh protocol tests in integration_test

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-04-16 23:19:35 +05:30
dependabot[bot] a2e4b4197d
chore(deps): bump actions/setup-go from 3 to 4
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 3 to 4.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-20 05:57:09 +00:00
sandeep e1b1417a9b doc workflow update 2023-03-16 23:48:06 +05:30
Mzack9999 b3b2ec9baa
Merge pull request #3351 from projectdiscovery/issue-3081-retry-gh
Add retry script for build, test and integration commands
2023-03-06 09:53:11 +01:00
sandeep 0f80575c44 Update version number and add paths to pull request. 2023-03-06 01:03:30 +05:30
Tarun Koyalwar 3e53087617
fix missing port in matched ssl templates (#3380)
* add openssl support + fix missing port

* fix failing tests

* go mod update

* workflow update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-03-04 04:47:41 +05:30
shubhamrasal d32c9c7f56 Revert "Resolve the requested changes"
This reverts commit 4ad64eb357.
2023-03-01 19:06:30 +05:30
shubhamrasal 4ad64eb357 Resolve the requested changes
- Add the logic to retry the test case that is failed.
- Accpet the number of how many times we want to retry
- Run the last retry in debug
2023-02-24 13:42:19 +05:30
shubhamrasal ed46e9560e Revert "Add retry script for build, test and integration commands"
This reverts commit 63747a5c54.
2023-02-24 06:15:16 +05:30
shubhamrasal 311ea5bd01 Update integration tests wd 2023-02-21 20:00:48 +05:30
shubhamrasal 63747a5c54 Add retry script for build, test and integration commands 2023-02-21 19:50:52 +05:30
Tarun Koyalwar a62e57f6c0
fix publish docs (#3296)
* add branch ref while checkout

* Auto Generate Syntax Docs + JSONSchema [Thu Feb  9 13:36:50 UTC 2023] 🤖

---------

Co-authored-by: GitHub Action <action@github.com>
2023-02-09 19:14:43 +05:30
Sandeep Singh 10dd4d7540
added branch ref (#3278) 2023-02-07 04:50:23 +05:30
dependabot[bot] 5bd0f9bf69
chore(deps): bump docker/build-push-action from 3 to 4 (#3266)
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 3 to 4.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-07 04:49:09 +05:30
dependabot[bot] ed5ab5118a
chore(deps): bump golangci/golangci-lint-action from 3.3.1 to 3.4.0
Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 3.3.1 to 3.4.0.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases)
- [Commits](https://github.com/golangci/golangci-lint-action/compare/v3.3.1...v3.4.0)

---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-30 05:00:58 +00:00
Mzack9999 0b3992cdb8
moving examples to go file (#3187) 2023-01-12 14:57:32 +05:30
Sandeep Singh a6a26ec026
Adding GH Stale bot (#3137)
* Adding GH Stale bot

* Added onlyLabels to work with specific labels to start with
2023-01-03 23:54:42 +05:30
Sandeep Singh f69b42b8da
Added release announcement to discord server (#3120) 2023-01-02 16:47:27 +05:30
xm1k3 34120fbecc
#3046 persistent failed item status and #2065 failed items reporting error once (#3047)
* added logs for debug

* fixes

* removed logs

* using cache item

* implemented multiple tests

* fixed some unit tests

* implemented test for skipping

* added multiple tests together

* added mark failed

* fix on tests

* better test implementation + concurrent

* fix: fixes on concurrent tests

* removed parallel and 1 unit test

DOCS: by default the command go test runs in parallel tests for different packages, and default is the number of CPUs available (see go help build)

* fixes on go routine

* increasing parallelism of once.Do

* bumping go to 1.19 for atomic types support

* removing redundant check + fixing test concurrency on create

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
2023-01-02 13:52:06 +05:30
Sandeep Singh 42b6a7df38
Publish Docs workflow fix (#3104) 2022-12-29 19:59:36 +05:30
Tarun Koyalwar aee0870617
scanallip handle edge cases (#3080)
* bug fix:remove port during dns resolution

* scanallip fix edge cases

* add scanallips testcases

* workflow fix

* removing pull cmd

* Auto Generate Syntax Docs + JSONSchema [Sat Dec 24 13:29:21 UTC 2022] 🤖

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: GitHub Action <action@github.com>
2022-12-24 19:03:23 +05:30
Sandeep Singh 96646c8f53
cloud templates targets sync (#2959)
* Add s3 bucket template provider

- Refactor the custom github template code
- add interface for template provider

* Validate if aws creds are passed if bucket flag

- refactor s3 provider struct to take client
- add function which returns the aws s3 client
- update error messages

* Add aws s3 bucket flags documentation in README.md

- Rename the github_test.go to customTemplate_test.go

* go mod update

* Move template provider code to pkg/external/customtemplates dir

* Added initial data_source sync to cloud

* Misc

* Add pagination to scan output and scan list (#2858)

* Add pagination to scan output and scan list

* Use time based parameters instead of page numbers

* Fix linting errors

* Do not check limits at client, check at server

* Remove unused constant

* Misc update

* Removed unnecessary flags

* Misc

* Misc

* Misc endpoint additions

* Added more routes

* Typo fix

* Misc fixes

* Misc

* Misc fixes to cloud target logic + use int for IDs

* Misc

* Misc fixes

* Misc

* Misc fixes

* readme update

* Add JSON output support for list-scan option (#2876)

* Add JSON output support for list-scan option

* Fix typo in cloud JSON output description

* Following changes

- Update status(finished, running) to be lower-case by default
- Convert status to upper-case in DisplayScanList()

* Update status to be lower-case by default

* Remove additional json flag, instead use existing

* Merge conflict

* Accomodate comment changes and restructure code

Co-authored-by: Jaideep K <jaideep@one2n.in>

* Use integer IDs for scan tasks

* Added get-templates-targets endpoint + JSON + validation

* Added target count list

* misc option / description updates

* Added changes as per code review

* duplicate options + typo updates

* Added tablewriter for tabular data writing by default

* Fixed list scan endpoint

* Review changes

* workflow fix

* Added cloud tags etc based filtering (#3070)

* Added omitempty for filtering request

* go mod tidy

* misc format update

Co-authored-by: shubhamrasal <shubhamdharmarasal@gmail.com>
Co-authored-by: Ice3man <nizamulrana@gmail.com>
Co-authored-by: Jaideep Khandelwal <jdk2588@gmail.com>
Co-authored-by: Siddharth Shashikar <60960197+shashikarsiddharth@users.noreply.github.com>
Co-authored-by: Jaideep K <jaideep@one2n.in>
2022-12-21 22:48:43 +05:30
Tarun Koyalwar 63150a8ee4
add stop-at-first-match for headless protocol (#3034)
* headless: adds stop-at-first-match

* GH workflow fix

* misc fix

* headless use spm option

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-12-19 19:52:17 +05:30
dependabot[bot] e44c464c58
chore(deps): bump goreleaser/goreleaser-action from 3 to 4
Bumps [goreleaser/goreleaser-action](https://github.com/goreleaser/goreleaser-action) from 3 to 4.
- [Release notes](https://github.com/goreleaser/goreleaser-action/releases)
- [Commits](https://github.com/goreleaser/goreleaser-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: goreleaser/goreleaser-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-12-19 05:00:59 +00:00
Sandeep Singh 0e35133ad5
public docs workflow fix (#3016) 2022-12-10 20:54:31 +05:30
sandeep 2230e0be49 misc workflow update 2022-12-05 23:26:25 +05:30
Sandeep Singh 5c8e715fc6
workflows improvements (#2963) 2022-12-05 18:22:55 +05:30
sandeep 68a1091056 workflow fix 2022-12-04 23:24:59 +05:30
Sandeep Singh a4e45ed6c5
workflow fix to test with current code (#2944) 2022-12-02 04:35:02 +05:30
Shubham Rasal d5a09e733a
Issue 2772 s3 provider support (#2825)
* Add s3 bucket template provider

- Refactor the custom github template code
- add interface for template provider

* Validate if aws creds are passed if bucket flag

- refactor s3 provider struct to take client
- add function which returns the aws s3 client
- update error messages

* Add aws s3 bucket flags documentation in README.md

- Rename the github_test.go to customTemplate_test.go

* go mod update

* Move template provider code to pkg/external/customtemplates dir

* Remove github and aws update variables from flag

* Rename CustomTemplateProvider to Provider

* Update integration and function command in makefile

* Update github test case, accept token

* readme update

* go mod tidy

* Update build-test.yml

* handle empty dir in s3

* Add requested changes

- download/update s3 and github only when `-ut` is passed
- only print the missing env variable for s3
- add the custom templates path in
  ~/.config/nuclei/.template-config.json

* print custom paths only if exists in config file

* misc update

* tag update

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2022-12-02 03:57:00 +05:30
Sandeep Singh 2b7b7649a8
build workflow update (#2911) 2022-11-29 04:20:20 +05:30
Mzack9999 2586d9b4a7 removing subset 2022-11-24 22:12:44 +01:00
Mzack9999 440bfe030a Merge branch 'dev' into issue-2832-race-condition-test 2022-11-24 22:10:42 +01:00
xm1k3 6d76fcd4e7 fix oh wrong path 2022-11-23 14:40:00 +01:00
Sandeep Singh ff9596f618
workflow updates (#2883)
* workflow updates

* added go mod download in build test
2022-11-23 17:13:52 +05:30
xm1k3 632c68a996 added multiple targets on gh action 2022-11-23 10:37:30 +01:00
xm1k3 ef6b7d9dc9 added tags to race condition 2022-11-23 10:11:46 +01:00
dependabot[bot] 2117fe2e1f
chore(deps): bump golangci/golangci-lint-action from 3.3.0 to 3.3.1 (#2844)
Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 3.3.0 to 3.3.1.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases)
- [Commits](https://github.com/golangci/golangci-lint-action/compare/v3.3.0...v3.3.1)

---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-22 22:14:57 +05:30
Shubham Rasal 721c4964d7
Issue 2613 custom template GitHub (#2630)
* Add custom template download/update support from github

- Accept the -gtr flag to accept the list of custom template
  repos(public/private)
- Accept the -gt flag for github token. It internally sets os.Env
  variable
- Update the flags from
   - -update to -nuclei-update for nuclei self update
   - -ut to -tup for template-update
   - -ud to -tud for custom template location
- Add github.go file which has code related to download and update
  custom templates repos.

* Reslove golint and test case error

* Take default template from community directory

- No need to give explicit community directory path.
- Update the integration test to support the change in path

* Update functional test script update template flag

* Update the path from community to nuclei-template

- Revert the code changes that were made to add community directory

* remove the comment

* Update the interactsh server url for testing

* Update race condition command

* update race condition cmd to download the templates

* Debug integration test failure

* update integration test to update templates

* Refactor downloadCustomTemplate function.

- Remove the log prining instead send the message.

* Add test case for custom template repo download

* move the download repo for loop into diff function

* refactor updateTemplate function.

* Create struct for github repos.

- Create customtemplate struct for repo.
- Add functions to customtemplate

* update readme.md file

* Refactor the downloadCustomTemplate function

- create const variables for github & community as template type
- Update gologger to INF
- Validate templateUpdate to accept only github & community value.
- Validate tempalteUpdate require githubTemplateRepo

* Resolve requested changes

* go mod update

* misc option update

* test update

* Revert back update-template flag to boolean.

- to update community templates
  `nuclei -ut`
- to update custom templates
  `nuclei -ut -gtr ehsandeep/mobile-nuclei-templates`

* Update readme to update flag documentation

* Update go.mod

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-11-03 20:27:18 +05:30
dependabot[bot] 855f7ad8a4
chore(deps): bump golangci/golangci-lint-action from 3.2.0 to 3.3.0
Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 3.2.0 to 3.3.0.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases)
- [Commits](https://github.com/golangci/golangci-lint-action/compare/v3.2.0...v3.3.0)

---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-10-24 05:05:09 +00:00
Ice3man e193e7c87e
Added tlsx integration to nuclei (#2522)
* Added tlsx integration to nuclei

* tls tests fix

* Added helper functions + upgrade tlsx to fix

* go mod update

* workflow fix to race test on windows

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-09-01 23:56:55 +05:30
sandeep de00155bb6 Added release announce automation 2022-08-13 01:30:08 -07:00
Sandeep Singh 1bf885e97b
Increases race condition coverage (#2293) 2022-07-19 01:10:10 +05:30
dependabot[bot] 0ce7f361ce
chore(deps): bump goreleaser/goreleaser-action from 2 to 3 (#2033)
Bumps [goreleaser/goreleaser-action](https://github.com/goreleaser/goreleaser-action) from 2 to 3.
- [Release notes](https://github.com/goreleaser/goreleaser-action/releases)
- [Commits](https://github.com/goreleaser/goreleaser-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: goreleaser/goreleaser-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-25 11:27:30 +05:30
dependabot[bot] d67e75d3cd
chore(deps): bump golangci/golangci-lint-action from 3.1.0 to 3.2.0 (#2000)
Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 3.1.0 to 3.2.0.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases)
- [Commits](https://github.com/golangci/golangci-lint-action/compare/v3.1.0...v3.2.0)

---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-16 12:35:49 +05:30
dependabot[bot] 2eb845262b
chore(deps): bump docker/setup-qemu-action from 1 to 2 (#1961)
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 1 to 2.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](https://github.com/docker/setup-qemu-action/compare/v1...v2)

---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
2022-05-10 17:49:26 +05:30
dependabot[bot] 1683851b46
chore(deps): bump docker/build-push-action from 2 to 3 (#1959)
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 2 to 3.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
2022-05-10 17:49:08 +05:30
dependabot[bot] 59fc6963e0
chore(deps): bump docker/setup-buildx-action from 1 to 2 (#1960)
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 1 to 2.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](https://github.com/docker/setup-buildx-action/compare/v1...v2)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
2022-05-10 17:48:44 +05:30
dependabot[bot] f8e462e3ab
chore(deps): bump docker/login-action from 1 to 2
Bumps [docker/login-action](https://github.com/docker/login-action) from 1 to 2.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v1...v2)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-05-09 05:05:50 +00:00
dependabot[bot] 10a4bdb5eb
chore(deps): bump github/codeql-action from 1 to 2
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 1 to 2.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/v1...v2)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-05-02 05:05:59 +00:00