Commit Graph

501 Commits (22c7422eb8536ced0a007f84e80dd81900e573cc)

Author SHA1 Message Date
Mzack9999 66f0dc735c
Adding jarm helper via dsl (#3906)
* Adding jarm helper via dsl

* adding test

* removing debug file

* fixing tests

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2023-07-14 21:24:12 +05:30
Mzack9999 e5154d362a
fixing payload load (#3927)
* fixing payload load

* Added tests for load payloads edge-case + fixed error

* Added separate flags for network and file sandbox

* Fixed tests for payload loader

* Fixed integration tests locally

* readme update

---------

Co-authored-by: Ice3man <nizamulrana@gmail.com>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-07-14 19:39:32 +05:30
Sandeep Singh 9a44deb9e1
added epss-percentile field under classification (#3911)
* added epss-percentile field under classification

* lint fix
2023-07-09 01:19:56 +05:30
sandeep 28f6c37974 misc fix 2023-07-07 23:35:32 +05:30
Tarun Koyalwar 5e284879d3
adds default ztls fallback support for tls errors (#3909)
* bump fastdialer

* make -ztls flag as no-op and deprecate

* bump fastdialer version from main branch

* dep update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-07-07 05:39:14 +05:30
Mzack9999 3dca03163c
Automatic target merge in network templates (#3904)
* skip visited actual addressess

* removed test

* adding disable clustering support
2023-07-06 21:33:52 +05:30
Mzack9999 305ac6a143
Bugfix 3871 workflow concurrency (#3903)
* recursion waitgroup fix

* cleanup
2023-07-06 17:38:24 +05:30
Keith Chason b3ccb9a6e5
Exclude Raw Request Payloads (#3710)
* Add command docs and CLI hook

* Add configurable exclusion from reports

* Register the CLI argument with exporter configuration

* Switch to inverted logic with JSONRequest flag

* Switch variable name for the -include-rr/-irr flag

* Remove flags from README

* Update call for -irr and -or

* convert -irr to no-op

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-07-05 02:07:56 +05:30
Ramana Reddy 6707bc777a
fix showing multiple failure matches per template on -ms set (#3770)
* fix showing multiple failure matchers per template
add integration test

* exclude AS134029 from unit test

* Add flag for match status per request

* chore(deps): bump golangci/golangci-lint-action from 3.4.0 to 3.5.0 (#3777)

Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 3.4.0 to 3.5.0.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases)
- [Commits](https://github.com/golangci/golangci-lint-action/compare/v3.4.0...v3.5.0)

---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/xanzy/go-gitlab in /v2 (#3778)

Bumps [github.com/xanzy/go-gitlab](https://github.com/xanzy/go-gitlab) from 0.83.0 to 0.84.0.
- [Changelog](https://github.com/xanzy/go-gitlab/blob/master/releases_test.go)
- [Commits](https://github.com/xanzy/go-gitlab/compare/v0.83.0...v0.84.0)

---
updated-dependencies:
- dependency-name: github.com/xanzy/go-gitlab
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/spf13/cast from 1.5.0 to 1.5.1 in /v2 (#3780)

Bumps [github.com/spf13/cast](https://github.com/spf13/cast) from 1.5.0 to 1.5.1.
- [Release notes](https://github.com/spf13/cast/releases)
- [Commits](https://github.com/spf13/cast/compare/v1.5.0...v1.5.1)

---
updated-dependencies:
- dependency-name: github.com/spf13/cast
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* enable no-httpx when passive scan is launched (#3789)

* chore(deps): bump github.com/projectdiscovery/fastdialer from 0.0.26 to 0.0.28 in /v2 (#3779)

* chore(deps): bump github.com/projectdiscovery/fastdialer in /v2

Bumps [github.com/projectdiscovery/fastdialer](https://github.com/projectdiscovery/fastdialer) from 0.0.26 to 0.0.28.
- [Release notes](https://github.com/projectdiscovery/fastdialer/releases)
- [Commits](https://github.com/projectdiscovery/fastdialer/compare/v0.0.26...v0.0.28)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/fastdialer
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump retryabledns to 0.28

* Update the retryabledns

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: shubhamrasal <shubhamdharmarasal@gmail.com>

* deprecatedProtocolNameTemplates concurrent map writes (#3785)

* deprecatedProtocolNameTemplates

* use syncLock

* fix lint error

* change version in deprecated warning msg

* comment asnmap expand unit test

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
Co-authored-by: Tarun Koyalwar <45962551+tarunKoyalwar@users.noreply.github.com>

* Issue 3339 headless fuzz (#3790)

* Basic headless fuzzing

* Remove debug statements

* Add integration tests

* Update template

* Fix recognize payload value in matcher

* Update tempalte

* use req.SetURL()

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>

* Auto Generate Syntax Docs + JSONSchema [Fri Jun  9 00:23:32 UTC 2023] 🤖

* Add headless header and status matchers (#3794)

* add headless header and status matchers

* rename headers as header

* add integration test for header+status

* fix typo

* chore(deps): bump golang from 1.20.4-alpine to 1.20.5-alpine (#3809)

Bumps golang from 1.20.4-alpine to 1.20.5-alpine.

---
updated-dependencies:
- dependency-name: golang
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/go-playground/validator/v10 in /v2 (#3810)

Bumps [github.com/go-playground/validator/v10](https://github.com/go-playground/validator) from 10.11.2 to 10.14.1.
- [Release notes](https://github.com/go-playground/validator/releases)
- [Commits](https://github.com/go-playground/validator/compare/v10.11.2...v10.14.1)

---
updated-dependencies:
- dependency-name: github.com/go-playground/validator/v10
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/projectdiscovery/rawhttp in /v2 (#3811)

Bumps [github.com/projectdiscovery/rawhttp](https://github.com/projectdiscovery/rawhttp) from 0.1.11 to 0.1.13.
- [Release notes](https://github.com/projectdiscovery/rawhttp/releases)
- [Commits](https://github.com/projectdiscovery/rawhttp/compare/v0.1.11...v0.1.13)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/rawhttp
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/go-git/go-git/v5 from 5.6.1 to 5.7.0 in /v2 (#3812)

Bumps [github.com/go-git/go-git/v5](https://github.com/go-git/go-git) from 5.6.1 to 5.7.0.
- [Release notes](https://github.com/go-git/go-git/releases)
- [Commits](https://github.com/go-git/go-git/compare/v5.6.1...v5.7.0)

---
updated-dependencies:
- dependency-name: github.com/go-git/go-git/v5
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/projectdiscovery/hmap in /v2 (#3781)

Bumps [github.com/projectdiscovery/hmap](https://github.com/projectdiscovery/hmap) from 0.0.11 to 0.0.13.
- [Release notes](https://github.com/projectdiscovery/hmap/releases)
- [Commits](https://github.com/projectdiscovery/hmap/compare/v0.0.11...v0.0.13)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/hmap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Using safe dereferencing

* adding comment

* fixing and condition

* fixing test id

* adding integration test

* update goflags dependency

* update goflags dependency

* bump goflags v0.1.9 => v0.1.10

* handle failure matcher flags logic at executor itself

* add integration test to matcher status per request

* Adding random tls impersonate (#3844)

* adding random tls impersonate

* dep update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>

* Use templateman enhance api to populate CVE info (#3788)

* use templateman enhance api to populate cve info

* rename cve-annotate => tmc
add additional flags to format, lint and enhance template using templateman apis

* minior changes

* remove duplicate code

* misc update

* Add validate and error log option

* print if updated

* print format and enhance only if updated

* make max-request optional

* fix reference unmarshal error

* fix removing self-contained tag

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>

* fix matcher status with network protocol

* fix test

* remove -msr flag

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dogan Can Bakir <65292895+dogancanbakir@users.noreply.github.com>
Co-authored-by: shubhamrasal <shubhamdharmarasal@gmail.com>
Co-authored-by: 三米前有蕉皮 <kali-team@qq.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
Co-authored-by: Tarun Koyalwar <45962551+tarunKoyalwar@users.noreply.github.com>
Co-authored-by: Shubham Rasal <shubham@projectdiscovery.io>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2023-06-30 23:32:00 +05:30
Mzack9999 c9d0942bc1
Extend headless contextargs (#3850)
* extend headless contextargs

* using darwin-latest

* grouping page options

* temp commenting code out

* fixing test

* adding more checks

* more checks

* fixing first navigation metadata

* adding integration test

* proto update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-06-26 22:55:51 +05:30
Mzack9999 fa199ed3b3
Improving clientpool with client certificates (#3851)
* Improving clientpool with client certificates

* adding test case

* Revert "Merge branch 'dev' into issue-3800-client-cert"

This reverts commit 7f057d742f4b9bda8e83b2052e29617b86b6776d, reversing
changes made to 7297cebcf8bb0f88961b644fc2ac7c040df8ffd9.

* Revert "Revert "Merge branch 'dev' into issue-3800-client-cert""

This reverts commit 2053a248a0cdc2002e0b4b4faa3472cf11c29760.

* go fmt

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-06-26 17:45:12 +05:30
Ramana Reddy 936256460b
Use templateman enhance api to populate CVE info (#3788)
* use templateman enhance api to populate cve info

* rename cve-annotate => tmc
add additional flags to format, lint and enhance template using templateman apis

* minior changes

* remove duplicate code

* misc update

* Add validate and error log option

* print if updated

* print format and enhance only if updated

* make max-request optional

* fix reference unmarshal error

* fix removing self-contained tag

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2023-06-21 18:49:40 +05:30
Mzack9999 2a32ed9cba
Adding random tls impersonate (#3844)
* adding random tls impersonate

* dep update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-06-21 17:17:18 +05:30
Ramana Reddy cddae989f3
Add template option to disable merging target url path with raw request path (#3799)
* add template option to disable merging target url path with raw request path

* rename disable-merge-path -> disable-path-automerge
add integration test
2023-06-19 20:22:17 +05:30
Ramana Reddy fed9b36b1a update goflags dependency 2023-06-15 18:18:09 +05:30
Mzack9999 200faa107e adding integration test 2023-06-13 17:27:26 +02:00
Dogan Can Bakir a4ca2021cd
Add headless header and status matchers (#3794)
* add headless header and status matchers

* rename headers as header

* add integration test for header+status

* fix typo
2023-06-09 15:03:03 +05:30
Shubham Rasal a34b94e62f
Issue 3339 headless fuzz (#3790)
* Basic headless fuzzing

* Remove debug statements

* Add integration tests

* Update template

* Fix recognize payload value in matcher

* Update tempalte

* use req.SetURL()

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-06-09 05:50:44 +05:30
Keith Chason 4d6080f3bc
"Executer" to "Executor" (#3760)
* Fix spelling of "executer" to "executor"

* minor change: use defer file.Close()

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-06-01 02:28:10 +05:30
sandeep 74ab1428be Merge branch 'dev' 2023-05-28 16:58:08 +05:30
Mzack9999 0d2d510689
Adding support for constants (#3692)
* adding support for constants

* fixing typo

* adding integration test

* fixing lint issues

* fixing template syntax
2023-05-25 22:02:35 +05:30
Tarun Koyalwar 4a6a0185f5
Feat template update improvements (#3675)
* path modification of official templates

* fix deprecated paths counter

* add reset flag to nuclei

* bug fix: deprecated path counter

* ignore meta files

* purge empty dirs

* fix lint error
2023-05-12 05:17:19 +05:30
Shubham Rasal 9c2fa8f9c4
Add payload in dns protocol (#3632)
* add execute function in dns

* Add payload in dns protocol

* Add integration test to cover dns payload

- also check command line overriding a payload variable

* Update matchedAt and remove trailing dot

* Consider payload data for request count

- Update verbose output to print question
- Update dns requests Requests function to consider payload data

* update gitignore

* bump nuclei version to v2.9.4-dev

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-05-11 03:26:29 +05:30
Ramana Reddy 59376180b1
Add utility to write max-requests to templates (#3607)
* Add utility to write max-requests to templates

* fix lint error

* fix max-request update edge case

* fix convert max-request: 1 => max-request: 1

* WIP, most of the code is commented

* Refactor the find and replace  logic

* Skip if template has the max-requests, do not overwrite

- return errors
- add warnings

* Fix the wrong index calculation

- Refactor the getInfoBlock function to not compile regex everytime

* Update -tc flag to filter fields within the classification section (#3606)

* Add fields from Classification section in a template to the -tc flag expression evaluation

Signed-off-by: iamargus95 <kamathsuraj95@gmail.com>

* Add tests for filtering Classification section using -tc flag

Signed-off-by: iamargus95 <kamathsuraj95@gmail.com>

* Fix hyphenated Metadata keys beings added to parameters

Signed-off-by: iamargus95 <kamathsuraj95@gmail.com>

* Add tests to the fix for hyphenated fields encountered in Metadata section

Signed-off-by: iamargus95 <kamathsuraj95@gmail.com>

---------

Signed-off-by: iamargus95 <kamathsuraj95@gmail.com>

* Check severity att while validating (#3540)

* Make severity attribute required

* Update test err msg

* minor

* Do not strict check serverity

* Fix failing test

* Don't print warning in workflow loader

- workflow loader that contains tags load all the template and parse it
- i.e it iw printing warning recursively, ignore as the templates
  already getting valiated

* Fix error typo

* Resolve comments

- split the function into two diff

---------

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: shubhamrasal <shubhamdharmarasal@gmail.com>

* tlsx dep update (#3620)

* updated interactsh version (#3621)

* updated interactsh version

* workflow update

* aws signer: fix missing x-content-sha256 header (#3601)

* fix missing x-content-sha256 header

* fix variable priority in self-contained templates

* remove debug statement

* adds generic raw request parser for self-contained req

* more integration tests

* bug fix: 10x faster race requests

* fix failing integration test

* chore(deps): bump github.com/xanzy/go-gitlab in /v2 (#3624)

Bumps [github.com/xanzy/go-gitlab](https://github.com/xanzy/go-gitlab) from 0.82.0 to 0.83.0.
- [Release notes](https://github.com/xanzy/go-gitlab/releases)
- [Changelog](https://github.com/xanzy/go-gitlab/blob/master/releases_test.go)
- [Commits](https://github.com/xanzy/go-gitlab/compare/v0.82.0...v0.83.0)

---
updated-dependencies:
- dependency-name: github.com/xanzy/go-gitlab
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/miekg/dns from 1.1.53 to 1.1.54 in /v2 (#3625)

Bumps [github.com/miekg/dns](https://github.com/miekg/dns) from 1.1.53 to 1.1.54.
- [Release notes](https://github.com/miekg/dns/releases)
- [Changelog](https://github.com/miekg/dns/blob/master/Makefile.release)
- [Commits](https://github.com/miekg/dns/compare/v1.1.53...v1.1.54)

---
updated-dependencies:
- dependency-name: github.com/miekg/dns
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /v2 (#3626)

Bumps [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) from 1.11.61 to 1.11.64.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/feature/s3/manager/v1.11.61...feature/s3/manager/v1.11.64)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Fix check for OS made in MustDisableSandbox() (#3631)

Signed-off-by: iamargus95 <kamathsuraj95@gmail.com>

* Fix wrong template loading in dev branch (#3629)

* Templates wrong loading

* Add tests to cover following scenarios

- check optional fields only if template loaded
- it should return warning only if template is loaded

* enable color in windows (#3634)

* enable color in windows

* fixed win workflow

* typo update

* tlsx dep update (#3633)

* tlsx dep update

* upgrde httpx => 1.3.0

* Fix check for OS made in MustDisableSandbox() (#3631)

Signed-off-by: iamargus95 <kamathsuraj95@gmail.com>

* Fix wrong template loading in dev branch (#3629)

* Templates wrong loading

* Add tests to cover following scenarios

- check optional fields only if template loaded
- it should return warning only if template is loaded

* enable color in windows (#3634)

* enable color in windows

* fixed win workflow

* typo update

---------

Signed-off-by: iamargus95 <kamathsuraj95@gmail.com>
Co-authored-by: Ramana Reddy <ramanaredy.manda@gmail.com>
Co-authored-by: Suraj Kamath <kamathsuraj95@gmail.com>
Co-authored-by: Shubham Rasal <shubham@projectdiscovery.io>

* Expose DNS fields for matchers and extractors (#3613)

* Extend dns extractor to dns answer records

* add test template

* Ignore error for dns variables are not found

* Add all the records of answer section

* Fixed the wrong typecasting

* Issue 3564 var override (#3599)

* Check if the variables are override by other means

- you can override the template variable value using command line flags

* Update lazy eval logic

- previously, we were checking any function/expression in variable
- now, update the logic, lazy eval only if variable contains any
  protocol variable(global)

* add integration tests

* Add test to check the dsl function working in variable

* gather all generate variables logic in utils

* go mod update

* Refactor the generate variables function

* go mod update+ fix typo

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>

* update rod to v0.112.9 #3552 (#3637)

* update rod to v0.112.9

* removed unused reflection

---------

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>

* change max-requests label to max-request

---------

Signed-off-by: iamargus95 <kamathsuraj95@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: shubhamrasal <shubhamdharmarasal@gmail.com>
Co-authored-by: Suraj Kamath <kamathsuraj95@gmail.com>
Co-authored-by: Dogan Can Bakir <65292895+dogancanbakir@users.noreply.github.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: Tarun Koyalwar <45962551+tarunKoyalwar@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Shubham Rasal <shubham@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
Co-authored-by: lu4nx <lx@shellcodes.org>
2023-05-09 23:29:25 +05:30
Tarun Koyalwar c62dc01f9f
uncover logic refactor to v0.0.4 (#3663)
* uncover logic refactor to v0.0.4

* remove deprecated import: stringsutil
2023-05-09 03:57:56 +05:30
Shubham Rasal 449afc0c5c
Issue 3564 var override (#3599)
* Check if the variables are override by other means

- you can override the template variable value using command line flags

* Update lazy eval logic

- previously, we were checking any function/expression in variable
- now, update the logic, lazy eval only if variable contains any
  protocol variable(global)

* add integration tests

* Add test to check the dsl function working in variable

* gather all generate variables logic in utils

* go mod update

* Refactor the generate variables function

* go mod update+ fix typo

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-05-02 23:49:56 +05:30
Shubham Rasal f640187709
Expose DNS fields for matchers and extractors (#3613)
* Extend dns extractor to dns answer records

* add test template

* Ignore error for dns variables are not found

* Add all the records of answer section

* Fixed the wrong typecasting
2023-05-02 17:13:11 +05:30
Sandeep Singh 55968931a5
enable color in windows (#3634)
* enable color in windows

* fixed win workflow

* typo update
2023-05-02 16:00:39 +05:30
Tarun Koyalwar 7f5e4e2336
aws signer: fix missing x-content-sha256 header (#3601)
* fix missing x-content-sha256 header

* fix variable priority in self-contained templates

* remove debug statement

* adds generic raw request parser for self-contained req

* more integration tests

* bug fix: 10x faster race requests

* fix failing integration test
2023-05-01 12:15:35 +05:30
Tarun Koyalwar 4e6ef4490e
duplicated params in self contained requests (#3608)
* fix duplicated params in self-contained+ export extracted values to file

* add integration tests + fix percentage overflow in pb

* fix integration test template id

* integration test: validate if file exists
2023-04-26 12:35:07 +05:30
Pj Metz 6c79602927
Readme sj flag fix (#3579)
* fixed -sj flag description for readme.md and readme_ID.md

* misc update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-04-20 19:08:02 +05:30
sandeep 871e7016fc version update 2023-04-20 03:39:33 +05:30
Keith Chason dcb003211c
Gitlab Custom Templates (#3570)
* Configuration options for GitLab template pulls

* GitLab client creation

* GitLab hooks and property renames

* Fix filesystem writing and update environment variables

* Fix type error in formatted error message

* Migrate directory config to new nucleiconfig file

* refactor + add custom templates to tm

* typo fix + only show installed ct with -tv

* add default gitlab url if not given

* fix template valid failure

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-04-20 03:12:52 +05:30
Mzack9999 978d0bcc23
Replacing goos with osutils (#3571)
* Replacing goos with osutils

* pleasing his majesty the linter
2023-04-19 23:00:15 +05:30
Tarun Koyalwar bf08913cd0
update logic + config management refactor (#3567)
* adds template manager

* refactor: checkpoint

* centrailized config & template download logic

* refactor removed unused code

* use global template directory

* update related bug fixes

* bug fix create cfg dir if missing

* fix lint error

* bug fix skip writing template dir in callback

* misc update

* remove unused code

* use strings.equalfold for comparison

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-04-19 21:58:48 +05:30
Mzack9999 6f4b1ae48a
Replacing ccache with generic gcache (#3523)
* Replacing ccache with generic gcache

* fixing lint issues

* removing unecessary hashing + using errorutils

* making test more tolerant

* removing dead code + refactor

* removing redundant code

* removing race

* maint

* moving code

* adding more iterations

* note + typo

* temporary fixing stop-at-first-match with interact

* wrapping internal map with mux

* sort before running integration test

* fix deadlock in requestShouldStopAtFirstMatch

* add timeout to integration_test workflow

* attempting to remove outer lock

* adds interactsh protocol tests in integration_test

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-04-16 23:19:35 +05:30
Dogan Can Bakir bca0c53a41
Enable --nc option if OS is windows (#3539)
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2023-04-13 18:11:04 +05:30
sduc 7029741338
[DRAFT] Annotate CVE: add configurations metadata (#3486)
* improve annotate to use new nvd apis

* annotate

* improvements

* fix mod files

* fetch EPSS and only write CPE

* lint fixes

---------

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-04-12 16:19:49 +05:30
Shubham Rasal 45cc676f96
Evaluate payload variables (#3503)
* Evaluate payload variables

* Add variables evaluation

* Extend variables test

- to check evaluation of global variables in variables
- to check evaluation of golbal variables in payload

* Add default and cli variables to websocket, whois and dns proto

- use url.Parse with urlutil.Parse
2023-04-12 01:50:58 +05:30
Keith Chason 3476f4d1d6
JSONL(ine) Export (#3504) (#3505)
* Add initial hooks for JSONL export

* Add newline character after each result

* fix integration test (#3506)

* fix integration test

* fix interactsh fatal error

* fix default report-config.yaml

---------

Co-authored-by: Tarun Koyalwar <45962551+tarunKoyalwar@users.noreply.github.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-04-08 17:44:41 +05:30
Tarun Koyalwar a185348194
fix integration test (#3506)
* fix integration test

* fix interactsh fatal error
2023-04-04 10:09:52 +08:00
Keith Chason 4d96025bec
JSON Export Handling Updates (#3466)
* Switch -json to -jsonl

* Add JSON output file

* Update docs for EN and ID

* Fix linting issue with error wrap

* Add -j flag

* Fix call for short flag

* Correct typo "Ciper" to "Cipher" (#3468)

* migrate dsl helper functions to dsl repo (#3461)

* migrate dsl pkg code to dsl repo

* fix lint error

* upgrade dsl dependency

* upgrade deps

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>

* chore(deps): bump github.com/projectdiscovery/httpx in /v2 (#3469)

Bumps [github.com/projectdiscovery/httpx](https://github.com/projectdiscovery/httpx) from 1.2.7 to 1.2.9.
- [Release notes](https://github.com/projectdiscovery/httpx/releases)
- [Changelog](https://github.com/projectdiscovery/httpx/blob/main/.goreleaser.yml)
- [Commits](https://github.com/projectdiscovery/httpx/compare/v1.2.7...v1.2.9)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/httpx
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/weppos/publicsuffix-go in /v2 (#3472)

Bumps [github.com/weppos/publicsuffix-go](https://github.com/weppos/publicsuffix-go) from 0.20.0 to 0.30.0.
- [Release notes](https://github.com/weppos/publicsuffix-go/releases)
- [Changelog](https://github.com/weppos/publicsuffix-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/weppos/publicsuffix-go/compare/v0.20.0...v0.30.0)

---
updated-dependencies:
- dependency-name: github.com/weppos/publicsuffix-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/projectdiscovery/wappalyzergo in /v2 (#3473)

Bumps [github.com/projectdiscovery/wappalyzergo](https://github.com/projectdiscovery/wappalyzergo) from 0.0.81 to 0.0.88.
- [Release notes](https://github.com/projectdiscovery/wappalyzergo/releases)
- [Commits](https://github.com/projectdiscovery/wappalyzergo/compare/v0.0.81...v0.0.88)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/wappalyzergo
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/projectdiscovery/hmap in /v2 (#3470)

Bumps [github.com/projectdiscovery/hmap](https://github.com/projectdiscovery/hmap) from 0.0.10 to 0.0.11.
- [Release notes](https://github.com/projectdiscovery/hmap/releases)
- [Commits](https://github.com/projectdiscovery/hmap/compare/v0.0.10...v0.0.11)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/hmap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* debug catalog path

* use paths instead of filepath for aws path

* deps update (#3477)

* deps update

* fixing gologger via callback

* Moved `json-export` flag to the other exporters

* Switch "json[-_]exporter to jsonexporter"

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Ramana Reddy <90540245+RamanaReddy0M@users.noreply.github.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: shubhamrasal <shubhamdharmarasal@gmail.com>
2023-03-31 15:29:29 +05:30
Mzack9999 d4acc471c5 better fmt 2023-03-20 00:48:39 +01:00
Mzack9999 50e178fbe8 Merge branch 'dev' into pr/3406 2023-03-20 00:24:33 +01:00
Mzack9999 1b7585476e simplify execution logic 2023-03-17 17:31:28 +01:00
Mzack9999 6a0db1c234 restoring tests 2023-03-17 16:56:14 +01:00
Mzack9999 c115768b8e investigating tests 2023-03-17 15:39:12 +01:00
Mzack9999 c182434130
json templates support (load with flags, run & validate ) (#3424)
* extending template identification logic

* removing test code

* local debug

* json template loading support using flags

* blacklist meta json files

* minor changes

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-03-16 13:33:59 +05:30
Ramana Reddy c9634fae72
Issue 3350 matcher condition or not work (#3397)
* fix or condition match even interactsh includes as matcher-part (#3350)

* add integration test

* add new template to integration test

* matcher-condtion: test case for both conditions

* fix lint errors

* upgrade dependencies

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-03-15 20:45:44 +05:30
Mzack9999 0bf8fc027d
Fixing nil pointer reference + use map helper (#3421)
* Fixing nil pointer reference + use map helper

* bump tlsx version to v1.0.6

* increase interactsh polling in integration_test

* fix nil pointer dereference in integration_test

* fix lint error

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-03-14 21:27:48 +05:30