* add default get method
* remove residual payload logic from old implementation
* fuzz: clone current state of component
* fuzz: bug fix stacking of payloads in multiple mode
* improve stdout template loading stats
* stdout: force display warnings if no templates are loaded
* update flags in README.md
* quote non-ascii chars in extractor output
* aws request signature can only be used in signed & verified tmpls
* deprecate request signature
* remove logic related to deprecated fuzzing input
* update test to use ordered params
* fix interactsh-url lazy eval: #4946
* output: skip unnecessary updates when unescaping
* updates as per requested changes
* feat: move fuzz package to root directory
* feat: added support for input providers like openapi,postman,etc
* feat: integration of new fuzzing logic in engine
* bugfix: use and instead of or
* fixed lint errors
* go mod tidy
* add new reqresp type + bump utils
* custom http request parser
* use new struct type RequestResponse
* introduce unified input/target provider
* abstract input formats via new inputprovider
* completed input provider refactor
* remove duplicated code
* add sdk method to load targets
* rename component url->path
* add new yaml format + remove duplicated code
* use gopkg.in/yaml.v3 for parsing
* update .gitignore
* refactor/move + docs fuzzing in http protocol
* fuzz: header + query integration test using fuzzplayground
* fix integration test runner in windows
* feat add support for filter in http fuzz
* rewrite header/query integration test with filter
* add replace regex rule
* support kv fuzzing + misc updates
* add path fuzzing example + misc improvements
* fix matchedURL + skip httpx on multi formats
* cookie fuzz integration test
* add json body + params body tests
* feat add multipart/form-data fuzzing support
* add all fuzz body integration test
* misc bug fixes + minor refactor
* add multipart form + body form unit tests
* only run fuzzing templates if -fuzz flag is given
* refactor/move fuzz playground server to pkg
* fix integration test + refactor
* add auth types and strategies
* add file auth provider
* start implementing auth logic in http
* add logic in http protocol
* static auth implemented for http
* default :80,:443 normalization
* feat: dynamic auth init
* feat: dynamic auth using templates
* validate targets count in openapi+swagger
* inputformats: add support to accept variables
* fix workflow integration test
* update lazy cred fetch logic
* fix unit test
* drop postman support
* domain related normalization
* update secrets.yaml file format + misc updates
* add auth prefetch option
* remove old secret files
* add fuzzing+auth related sdk options
* fix/support multiple mode in kv header fuzzing
* rename 'headers' -> 'header' in fuzzing rules
* fix deadlock due to merge conflict resolution
* misc update
* add bool type in parsed value
* add openapi validation+override+ new flags
* misc updates
* remove optional path parameters when unavailable
* fix swagger.yaml file
* misc updates
* update print msg
* multiple openapi validation enchancements + appMode
* add optional params in required_openapi_vars.yaml file
* improve warning/verbose msgs in format
* fix skip-format-validation not working
* use 'params/parameter' instead of 'variable' in openapi
* add retry support for falky tests
* fix nuclei loading ignored templates (#4849)
* fix tag include logic
* fix unit test
* remove quoting in extractor output
* remove quote in debug code command
* feat: issue tracker URLs in JSON + misc fixes (#4855)
* feat: issue tracker URLs in JSON + misc fixes
* misc changes
* feat: status update support for issues
* feat: report metadata generation hook support
* feat: added CLI summary of tickets created
* misc changes
* introduce `disable-unsigned-templates` flag (#4820)
* introduce `disable-unsigned-templates` flag
* minor
* skip instead of exit
* remove duplicate imports
* use stats package + misc enhancements
* force display warning + adjust skipped stats in unsigned count
* include unsigned skipped templates without -dut flag
---------
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
* Purge cache on global callback set (#4840)
* purge cache on global callback set
* lint
* purging cache
* purge cache in runner after loading templates
* include internal cache from parsers + add global cache register/purge via config
* remove disable cache purge option
---------
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
* misc update
* add application/octet-stream support
* openapi: support path specific params
* misc option + readme update
---------
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
Co-authored-by: Tarun Koyalwar <45962551+tarunKoyalwar@users.noreply.github.com>
Co-authored-by: Dogan Can Bakir <65292895+dogancanbakir@users.noreply.github.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
* handle 1 more edgecase
* add integration test for this edgecase
* fix multi-http-var-sharing with integration test
* add -payload-concurrency (-pc) flag
* fix missing internal:true login in multiprotocol engine
* fix/handle absolute invalid url parsing
* support -pc & -jc in go sdk
* fix missing variables in code protocol operators
* add payload count parallelhttp check
* feat http response memory optimization + reuse buffers
* update nuclei version
* feat: reuse js vm's and compile to programs
* fix failing http integration test
* remove dead code + add -jsc
* feat reuse js vms in pool with concurrency
* update comments as per review
* bug fix+ update interactsh test to look for dns interaction
* try enabling all interactsh integration tests
---------
Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
* log warnings + use scanContext in flow
* refactor flow to use scanContext + log all events
* feat: internal matcher
* fix integration test
* bug fix extractor: merge dynamic values, fix missing extractors in file
* flow: fix 'No Results Found' if last statement output is false
* fix unit test
* fix race-condition & oow in extracted file output
* add mutex for file.Write + set finalizer for os.File
* fix integration test
* disable extractor save to file in lib mode(configurable)
* use sync.Once for init
* disable out of bound image write in headless
* misc updates
* fix headless screenshot test
* fix extractor save to file integration test
* remove 'to' feature in extractors
* fix net read
* only return N bytes if extra available
* use ConnReadN from readerutil
* add integration test
* print unsigned warning in stderr
* fix js protocol in flow #4318
* fix integration test: url encoding issue
* fix network protocol issue + integration tests
* multiple improvements to integration test
* replace all conn.Read() from tests
* disable network-basic.yaml in windows
* disable code protocol in win CI
* fix bitwise login ps1-snippet.yaml
* hide previous matcher events in flow
* remove dead code+ update integration tests
---------
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
* add randstr preprocessor to defaults
* fix indexing in http + preprocessor integration test
* add multi-request integration test
* skip test if asnmap is down
* store and generate signer keys
* fix trailing newline in code_response
* fix formatting and update error string
* fix integration test
* fix rsaSigned code integration test
* bug fixes , docs and more
* bump go -> 1.21
* use 'response' as default part in code templates
* disable sourcemaps for all js runtimes
* disable eval function
* rewrite file validation in sandbox mode
* sandbox file read improvements + minor refactor
* refactor sign and verify logic
* fix panic and missing id in code protocol
* disable re-signing code protocol templates
* fix code resigning in tests
* allow -lfa in test for signing templates
* start index from 1 in flow and multiproto
* remove testfiles
* add python in integration test
* update code protocol docs
* add python engine in template
* rework template signer
* fix integration test and more
* reworked template signer
* fix lint error
* display signature stats
* update docs
* add user fragment to signature
* use md5 to generate fragment
* update docs with code re-sign
* misc updates
* public crt update
* remove workflow info statement
* fix printing issues
* refactor preprocessor logic
* remove debug statement
* fix failing example test
* go mod tidy
---------
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
* use std config directories
* use NUCLEI_CONFIG_DIR env for config dir
* add template sources option in sdk
* add cloud.projectdiscovery.io to trusted domain
* fix failing test
* Added initial API docs
* dark mode fixes!
* gitlab tracker duplicate check
* integration test
* added In to search to restrict to title match
* added example GitLab yaml
---------
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
* rebase js-layer PR from @ice3man543
* package restructuring
* working
* fix duplicated event & matcher status
* fix lint error
* fix response field
* add new functions
* multiple minor improvements
* fix incorrect stats in js protocol
* sort output metadata in cli
* remove temp files
* remove dead code
* add unit and integration test
* fix lint error
* add jsdoclint using llm
* fix error in test
* add js lint using llm
* generate docs of libs
* llm lint
* remove duplicated docs
* update generated docs
* update prompt in doclint
* update docs
* temp disable version check test
* fix unit test and add retry
* fix panic in it
* update and move jsdocs
* updated jsdocs
* update docs
* update container platform in test
* dir restructure and adding docs
* add api_reference and remove markdown docs
* fix imports
* add javascript design and contribution docs
* add js protocol documentation
* update integration test and docs
* update doc ext mdx->md
* minor update to docs
* new integration test and more
* move go libs and add docs
* gen new net docs and more
* final docs update
* add new devtool
* use fastdialer
* fix build fail
* use fastdialer + network sandbox support
* add reserved keyword 'Port'
* update Port to new syntax
* misc update
* always enable templatectx in js protocol
* move docs to 'js-proto-docs' repo
* remove scrapefuncs binary
---------
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
* add flow logic
* progress
* working POC
* fix string slice normalization issue in variables
* update
* fix nil panic
* remove poll()
* load file with sandbox and more
* fix failing integration tests
* JS: log: print in vardump format
* fix missing id in protocols
* fix proto prefix in template context
* flow: add unit tests
* conditional flow support using flow
* fix proto callbacks + more unit tests
* adds integration test
* conditional flow: check if req has any matchers
* fix lint error
* deprecate iterate-all+ missing multi-proto implementation
* fix ip input in raw request
* JS: feat dedupe object+ more builtin funcs
* feat: hide protocol result using hide
* feat: async execution
* complete async execution support
* fix condition-flow without any matchers
* refactor: template executer package (tmplexec)
* flow executor working
* fix data race in templateCtx
* templateCtx redesign
* fix failing unit test
* add multiprotocol support to deprecated syntax
* fix race condition in utils & tlsx
* add documentation in flow package
* remove regions.txt file
* fix minor issue with self contained templates
* fix typos of copilot
* dep + misc update
* fix reqID: use req.Type instead of template.Type
---------
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
* use -lfa and -lna in headless
* fix lna in headless
* misc update
* fix nil pointer dereference in test
* fix lint & unit test
* use urlutil
* headless protocol scheme improvements
* add unit and integration tests
* run unit test from binary
---------
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
* Improving clientpool with client certificates
* adding test case
* Revert "Merge branch 'dev' into issue-3800-client-cert"
This reverts commit 7f057d742f4b9bda8e83b2052e29617b86b6776d, reversing
changes made to 7297cebcf8bb0f88961b644fc2ac7c040df8ffd9.
* Revert "Revert "Merge branch 'dev' into issue-3800-client-cert""
This reverts commit 2053a248a0cdc2002e0b4b4faa3472cf11c29760.
* go fmt
---------
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
* add execute function in dns
* Add payload in dns protocol
* Add integration test to cover dns payload
- also check command line overriding a payload variable
* Update matchedAt and remove trailing dot
* Consider payload data for request count
- Update verbose output to print question
- Update dns requests Requests function to consider payload data
* update gitignore
* bump nuclei version to v2.9.4-dev
---------
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
* Check if the variables are override by other means
- you can override the template variable value using command line flags
* Update lazy eval logic
- previously, we were checking any function/expression in variable
- now, update the logic, lazy eval only if variable contains any
protocol variable(global)
* add integration tests
* Add test to check the dsl function working in variable
* gather all generate variables logic in utils
* go mod update
* Refactor the generate variables function
* go mod update+ fix typo
---------
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
* Extend dns extractor to dns answer records
* add test template
* Ignore error for dns variables are not found
* Add all the records of answer section
* Fixed the wrong typecasting
* Templates wrong loading
* Add tests to cover following scenarios
- check optional fields only if template loaded
- it should return warning only if template is loaded
Tarun Koyalwar
Ice3man
Dogan Can Bakir
sandeep
Stefan Kahn
Daniil Morozov
Tarun Koyalwar
Sandeep Singh
Ramana Reddy
Josh Soref
Mzack9999
Shubham Rasal