Commit Graph

4073 Commits (4a6a0185f5f91f9f12cccb2387b302ca3ddecee7)

Author SHA1 Message Date
Tarun Koyalwar 4a6a0185f5
Feat template update improvements (#3675)
* path modification of official templates

* fix deprecated paths counter

* add reset flag to nuclei

* bug fix: deprecated path counter

* ignore meta files

* purge empty dirs

* fix lint error
2023-05-12 05:17:19 +05:30
Sandeep Singh 1f9a065713
Merge pull request #3640 from projectdiscovery/dev
nuclei v2.9.3 (bugfix release)
2023-05-04 01:56:05 +05:30
sandeep 79a1c99da7 version update 2023-05-04 01:47:41 +05:30
Tarun Koyalwar 37aaa5ebaa
add support for resolving old template paths (#3635)
* add support for resolving old template paths

* skip resolving if new path is specified

* add debug statement

* show error if fallback failed

* remove debug statement

* remove fallback errors

* print warning for deprecated paths

* add warnings for  deprecated paths/protocol names

* misc update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-05-04 01:43:41 +05:30
sandeep 3f8a2f63b7 retryabledns dep update 2023-05-03 21:00:28 +05:30
Tarun Koyalwar 9b3a55d6c0
fix panic while parsing tlsx response (#3641) 2023-05-03 20:47:08 +05:30
lu4nx 027880af34
update rod to v0.112.9 #3552 (#3637)
* update rod to v0.112.9

* removed unused reflection

---------

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-05-03 14:40:58 +05:30
Shubham Rasal 449afc0c5c
Issue 3564 var override (#3599)
* Check if the variables are override by other means

- you can override the template variable value using command line flags

* Update lazy eval logic

- previously, we were checking any function/expression in variable
- now, update the logic, lazy eval only if variable contains any
  protocol variable(global)

* add integration tests

* Add test to check the dsl function working in variable

* gather all generate variables logic in utils

* go mod update

* Refactor the generate variables function

* go mod update+ fix typo

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-05-02 23:49:56 +05:30
Shubham Rasal f640187709
Expose DNS fields for matchers and extractors (#3613)
* Extend dns extractor to dns answer records

* add test template

* Ignore error for dns variables are not found

* Add all the records of answer section

* Fixed the wrong typecasting
2023-05-02 17:13:11 +05:30
Sandeep Singh d55ad995d0
tlsx dep update (#3633)
* tlsx dep update

* upgrde httpx => 1.3.0

* Fix check for OS made in MustDisableSandbox() (#3631)

Signed-off-by: iamargus95 <kamathsuraj95@gmail.com>

* Fix wrong template loading in dev branch (#3629)

* Templates wrong loading

* Add tests to cover following scenarios

- check optional fields only if template loaded
- it should return warning only if template is loaded

* enable color in windows (#3634)

* enable color in windows

* fixed win workflow

* typo update

---------

Signed-off-by: iamargus95 <kamathsuraj95@gmail.com>
Co-authored-by: Ramana Reddy <ramanaredy.manda@gmail.com>
Co-authored-by: Suraj Kamath <kamathsuraj95@gmail.com>
Co-authored-by: Shubham Rasal <shubham@projectdiscovery.io>
2023-05-02 16:22:41 +05:30
Sandeep Singh 55968931a5
enable color in windows (#3634)
* enable color in windows

* fixed win workflow

* typo update
2023-05-02 16:00:39 +05:30
Shubham Rasal 6ebb8e98f4
Fix wrong template loading in dev branch (#3629)
* Templates wrong loading

* Add tests to cover following scenarios

- check optional fields only if template loaded
- it should return warning only if template is loaded
2023-05-02 15:12:55 +05:30
Suraj Kamath 537814bae8
Fix check for OS made in MustDisableSandbox() (#3631)
Signed-off-by: iamargus95 <kamathsuraj95@gmail.com>
2023-05-02 15:04:24 +05:30
dependabot[bot] bdf476fbd4
chore(deps): bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /v2 (#3626)
Bumps [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) from 1.11.61 to 1.11.64.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/feature/s3/manager/v1.11.61...feature/s3/manager/v1.11.64)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-01 12:19:42 +05:30
dependabot[bot] e2125c2c62
chore(deps): bump github.com/miekg/dns from 1.1.53 to 1.1.54 in /v2 (#3625)
Bumps [github.com/miekg/dns](https://github.com/miekg/dns) from 1.1.53 to 1.1.54.
- [Release notes](https://github.com/miekg/dns/releases)
- [Changelog](https://github.com/miekg/dns/blob/master/Makefile.release)
- [Commits](https://github.com/miekg/dns/compare/v1.1.53...v1.1.54)

---
updated-dependencies:
- dependency-name: github.com/miekg/dns
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-01 12:19:31 +05:30
dependabot[bot] fc16f1beda
chore(deps): bump github.com/xanzy/go-gitlab in /v2 (#3624)
Bumps [github.com/xanzy/go-gitlab](https://github.com/xanzy/go-gitlab) from 0.82.0 to 0.83.0.
- [Release notes](https://github.com/xanzy/go-gitlab/releases)
- [Changelog](https://github.com/xanzy/go-gitlab/blob/master/releases_test.go)
- [Commits](https://github.com/xanzy/go-gitlab/compare/v0.82.0...v0.83.0)

---
updated-dependencies:
- dependency-name: github.com/xanzy/go-gitlab
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-01 12:19:20 +05:30
Tarun Koyalwar 7f5e4e2336
aws signer: fix missing x-content-sha256 header (#3601)
* fix missing x-content-sha256 header

* fix variable priority in self-contained templates

* remove debug statement

* adds generic raw request parser for self-contained req

* more integration tests

* bug fix: 10x faster race requests

* fix failing integration test
2023-05-01 12:15:35 +05:30
Sandeep Singh 7ac9f05e53
updated interactsh version (#3621)
* updated interactsh version

* workflow update
2023-04-29 14:55:58 +05:30
Sandeep Singh 76f996dee2
tlsx dep update (#3620) 2023-04-28 15:36:27 +05:30
Dogan Can Bakir 4e0ccb38be
Check severity att while validating (#3540)
* Make severity attribute required

* Update test err msg

* minor

* Do not strict check serverity

* Fix failing test

* Don't print warning in workflow loader

- workflow loader that contains tags load all the template and parse it
- i.e it iw printing warning recursively, ignore as the templates
  already getting valiated

* Fix error typo

* Resolve comments

- split the function into two diff

---------

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: shubhamrasal <shubhamdharmarasal@gmail.com>
2023-04-27 15:27:30 +05:30
Suraj Kamath 78f1ade6c6
Update -tc flag to filter fields within the classification section (#3606)
* Add fields from Classification section in a template to the -tc flag expression evaluation

Signed-off-by: iamargus95 <kamathsuraj95@gmail.com>

* Add tests for filtering Classification section using -tc flag

Signed-off-by: iamargus95 <kamathsuraj95@gmail.com>

* Fix hyphenated Metadata keys beings added to parameters

Signed-off-by: iamargus95 <kamathsuraj95@gmail.com>

* Add tests to the fix for hyphenated fields encountered in Metadata section

Signed-off-by: iamargus95 <kamathsuraj95@gmail.com>

---------

Signed-off-by: iamargus95 <kamathsuraj95@gmail.com>
2023-04-27 00:15:35 +05:30
GitHub Action fb2580c401 Auto Generate Syntax Docs + JSONSchema [Wed Apr 26 07:06:02 UTC 2023] 🤖 2023-04-26 07:06:02 +00:00
Tarun Koyalwar 4e6ef4490e
duplicated params in self contained requests (#3608)
* fix duplicated params in self-contained+ export extracted values to file

* add integration tests + fix percentage overflow in pb

* fix integration test template id

* integration test: validate if file exists
2023-04-26 12:35:07 +05:30
Mzack9999 ea5f8a0638
Additional nil check on interactsh client (#3590) 2023-04-25 23:49:23 +05:30
dependabot[bot] 97e7081c69
chore(deps): bump github.com/projectdiscovery/networkpolicy in /v2 (#3594)
Bumps [github.com/projectdiscovery/networkpolicy](https://github.com/projectdiscovery/networkpolicy) from 0.0.4 to 0.0.5.
- [Release notes](https://github.com/projectdiscovery/networkpolicy/releases)
- [Commits](https://github.com/projectdiscovery/networkpolicy/compare/v0.0.4...v0.0.5)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/networkpolicy
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-25 15:22:21 +05:30
dependabot[bot] c8b3551035
chore(deps): bump github.com/projectdiscovery/retryabledns in /v2 (#3593)
Bumps [github.com/projectdiscovery/retryabledns](https://github.com/projectdiscovery/retryabledns) from 1.0.21 to 1.0.22.
- [Release notes](https://github.com/projectdiscovery/retryabledns/releases)
- [Commits](https://github.com/projectdiscovery/retryabledns/compare/v1.0.21...v1.0.22)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/retryabledns
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-25 15:14:17 +05:30
dependabot[bot] 04f5784471
chore(deps): bump github.com/aws/aws-sdk-go-v2/config in /v2 (#3592)
Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.18.20 to 1.18.21.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.18.20...config/v1.18.21)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-25 15:14:01 +05:30
dependabot[bot] d3d0c68315
chore(deps): bump github.com/klauspost/compress in /v2 (#3591)
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress) from 1.16.4 to 1.16.5.
- [Release notes](https://github.com/klauspost/compress/releases)
- [Changelog](https://github.com/klauspost/compress/blob/master/.goreleaser.yml)
- [Commits](https://github.com/klauspost/compress/compare/v1.16.4...v1.16.5)

---
updated-dependencies:
- dependency-name: github.com/klauspost/compress
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-25 15:13:47 +05:30
Mzack9999 34d692d9eb
Merge pull request #3589 from projectdiscovery/maint-headless-process
Removing redundant code with utils
2023-04-24 16:53:26 +02:00
Mzack9999 bb9ed7bdf0 updating go modules 2023-04-23 21:45:47 +02:00
Mzack9999 64adad131f Removing redundant code with utils 2023-04-23 21:37:25 +02:00
Pj Metz 6c79602927
Readme sj flag fix (#3579)
* fixed -sj flag description for readme.md and readme_ID.md

* misc update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-04-20 19:08:02 +05:30
sandeep fe234cf32d version update 2023-04-20 04:44:54 +05:30
Sandeep Singh e3ce33afda
Merge pull request #3573 from projectdiscovery/dev
nuclei v2.9.2
2023-04-20 04:16:46 +05:30
sandeep 871e7016fc version update 2023-04-20 03:39:33 +05:30
Keith Chason dcb003211c
Gitlab Custom Templates (#3570)
* Configuration options for GitLab template pulls

* GitLab client creation

* GitLab hooks and property renames

* Fix filesystem writing and update environment variables

* Fix type error in formatted error message

* Migrate directory config to new nucleiconfig file

* refactor + add custom templates to tm

* typo fix + only show installed ct with -tv

* add default gitlab url if not given

* fix template valid failure

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-04-20 03:12:52 +05:30
Keith Chason b211d6fa44
Duplicate error message, variable collision fix, method doc references (#3568) 2023-04-20 02:22:34 +05:30
Mzack9999 978d0bcc23
Replacing goos with osutils (#3571)
* Replacing goos with osutils

* pleasing his majesty the linter
2023-04-19 23:00:15 +05:30
Tarun Koyalwar bf08913cd0
update logic + config management refactor (#3567)
* adds template manager

* refactor: checkpoint

* centrailized config & template download logic

* refactor removed unused code

* use global template directory

* update related bug fixes

* bug fix create cfg dir if missing

* fix lint error

* bug fix skip writing template dir in callback

* misc update

* remove unused code

* use strings.equalfold for comparison

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-04-19 21:58:48 +05:30
Shubham Rasal d0f22f8b73
Issue 3488 http race (#3533)
* debug

* Add body after the request creation

* fix race_count template hangs

* remove printf

* update if condition
2023-04-19 01:57:53 +05:30
dependabot[bot] 2d769f1b4a
chore(deps): bump github.com/aws/aws-sdk-go-v2/credentials in /v2 (#3559)
Bumps [github.com/aws/aws-sdk-go-v2/credentials](https://github.com/aws/aws-sdk-go-v2) from 1.13.19 to 1.13.20.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/credentials/v1.13.19...credentials/v1.13.20)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/credentials
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-17 13:50:22 +05:30
dependabot[bot] 3d52036178
chore(deps): bump github.com/shirou/gopsutil/v3 in /v2 (#3558)
Bumps [github.com/shirou/gopsutil/v3](https://github.com/shirou/gopsutil) from 3.22.12 to 3.23.3.
- [Release notes](https://github.com/shirou/gopsutil/releases)
- [Commits](https://github.com/shirou/gopsutil/compare/v3.22.12...v3.23.3)

---
updated-dependencies:
- dependency-name: github.com/shirou/gopsutil/v3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-17 13:50:08 +05:30
dependabot[bot] 97daad2035
chore(deps): bump github.com/klauspost/compress in /v2 (#3557)
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress) from 1.16.0 to 1.16.4.
- [Release notes](https://github.com/klauspost/compress/releases)
- [Changelog](https://github.com/klauspost/compress/blob/master/.goreleaser.yml)
- [Commits](https://github.com/klauspost/compress/compare/v1.16.0...v1.16.4)

---
updated-dependencies:
- dependency-name: github.com/klauspost/compress
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-17 13:49:52 +05:30
dependabot[bot] 93639f5882
chore(deps): bump github.com/xanzy/go-gitlab in /v2 (#3555)
Bumps [github.com/xanzy/go-gitlab](https://github.com/xanzy/go-gitlab) from 0.81.0 to 0.82.0.
- [Release notes](https://github.com/xanzy/go-gitlab/releases)
- [Changelog](https://github.com/xanzy/go-gitlab/blob/master/releases_test.go)
- [Commits](https://github.com/xanzy/go-gitlab/compare/v0.81.0...v0.82.0)

---
updated-dependencies:
- dependency-name: github.com/xanzy/go-gitlab
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-17 13:49:22 +05:30
dependabot[bot] fdc4b45f3f
chore(deps): bump github.com/rs/xid from 1.4.0 to 1.5.0 in /v2 (#3556)
Bumps [github.com/rs/xid](https://github.com/rs/xid) from 1.4.0 to 1.5.0.
- [Release notes](https://github.com/rs/xid/releases)
- [Commits](https://github.com/rs/xid/compare/v1.4.0...v1.5.0)

---
updated-dependencies:
- dependency-name: github.com/rs/xid
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-17 13:48:49 +05:30
Keith Chason 786ee982ed
Azure Blob Storage Custom Template Provider (#3542)
* Initial library creation for downloading blobs from Azure

* Environment variable hooks for Azure config values

* Missing configuration checks for Azure blob connection

* Package dependencies for Azure client library

* Make output directory if it doesn't exist, add log for downloaded templates

* Add example formats for Azure properties

* Add path to the output after download

* Only download .yaml files instead of also .yml .json
2023-04-17 13:48:06 +05:30
Mzack9999 6f4b1ae48a
Replacing ccache with generic gcache (#3523)
* Replacing ccache with generic gcache

* fixing lint issues

* removing unecessary hashing + using errorutils

* making test more tolerant

* removing dead code + refactor

* removing redundant code

* removing race

* maint

* moving code

* adding more iterations

* note + typo

* temporary fixing stop-at-first-match with interact

* wrapping internal map with mux

* sort before running integration test

* fix deadlock in requestShouldStopAtFirstMatch

* add timeout to integration_test workflow

* attempting to remove outer lock

* adds interactsh protocol tests in integration_test

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-04-16 23:19:35 +05:30
Sandeep Singh 307085ef4c
tlsx dep update (#3554) 2023-04-16 22:23:51 +05:30
Dogan Can Bakir bca0c53a41
Enable --nc option if OS is windows (#3539)
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2023-04-13 18:11:04 +05:30
Keith Chason 8ffbebb5ea
Correct spelling in "access" and add arg types (#3536) 2023-04-13 18:06:54 +05:30