* log warnings + use scanContext in flow
* refactor flow to use scanContext + log all events
* feat: internal matcher
* fix integration test
* bug fix extractor: merge dynamic values, fix missing extractors in file
* flow: fix 'No Results Found' if last statement output is false
* fix unit test
* fix race-condition & oow in extracted file output
* add mutex for file.Write + set finalizer for os.File
* fix integration test
* disable extractor save to file in lib mode(configurable)
* use sync.Once for init
* disable out of bound image write in headless
* misc updates
* fix headless screenshot test
* fix extractor save to file integration test
* remove 'to' feature in extractors
* fix net read
* only return N bytes if extra available
* use ConnReadN from readerutil
* add integration test
* print unsigned warning in stderr
* fix js protocol in flow #4318
* fix integration test: url encoding issue
* fix network protocol issue + integration tests
* multiple improvements to integration test
* replace all conn.Read() from tests
* disable network-basic.yaml in windows
* disable code protocol in win CI
* fix bitwise login ps1-snippet.yaml
* hide previous matcher events in flow
* remove dead code+ update integration tests
---------
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
* add randstr preprocessor to defaults
* fix indexing in http + preprocessor integration test
* add multi-request integration test
* skip test if asnmap is down
* store and generate signer keys
* fix trailing newline in code_response
* fix formatting and update error string
* fix integration test
* fix rsaSigned code integration test
* bug fixes , docs and more
* bump go -> 1.21
* use 'response' as default part in code templates
* disable sourcemaps for all js runtimes
* disable eval function
* rewrite file validation in sandbox mode
* sandbox file read improvements + minor refactor
* refactor sign and verify logic
* fix panic and missing id in code protocol
* disable re-signing code protocol templates
* fix code resigning in tests
* allow -lfa in test for signing templates
* start index from 1 in flow and multiproto
* remove testfiles
* add python in integration test
* update code protocol docs
* add python engine in template
* rework template signer
* fix integration test and more
* reworked template signer
* fix lint error
* display signature stats
* update docs
* add user fragment to signature
* use md5 to generate fragment
* update docs with code re-sign
* misc updates
* public crt update
* remove workflow info statement
* fix printing issues
* refactor preprocessor logic
* remove debug statement
* fix failing example test
* go mod tidy
---------
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
* use std config directories
* use NUCLEI_CONFIG_DIR env for config dir
* add template sources option in sdk
* add cloud.projectdiscovery.io to trusted domain
* fix failing test
* Added initial API docs
* dark mode fixes!
* gitlab tracker duplicate check
* integration test
* added In to search to restrict to title match
* added example GitLab yaml
---------
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
* rebase js-layer PR from @ice3man543
* package restructuring
* working
* fix duplicated event & matcher status
* fix lint error
* fix response field
* add new functions
* multiple minor improvements
* fix incorrect stats in js protocol
* sort output metadata in cli
* remove temp files
* remove dead code
* add unit and integration test
* fix lint error
* add jsdoclint using llm
* fix error in test
* add js lint using llm
* generate docs of libs
* llm lint
* remove duplicated docs
* update generated docs
* update prompt in doclint
* update docs
* temp disable version check test
* fix unit test and add retry
* fix panic in it
* update and move jsdocs
* updated jsdocs
* update docs
* update container platform in test
* dir restructure and adding docs
* add api_reference and remove markdown docs
* fix imports
* add javascript design and contribution docs
* add js protocol documentation
* update integration test and docs
* update doc ext mdx->md
* minor update to docs
* new integration test and more
* move go libs and add docs
* gen new net docs and more
* final docs update
* add new devtool
* use fastdialer
* fix build fail
* use fastdialer + network sandbox support
* add reserved keyword 'Port'
* update Port to new syntax
* misc update
* always enable templatectx in js protocol
* move docs to 'js-proto-docs' repo
* remove scrapefuncs binary
---------
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
* add flow logic
* progress
* working POC
* fix string slice normalization issue in variables
* update
* fix nil panic
* remove poll()
* load file with sandbox and more
* fix failing integration tests
* JS: log: print in vardump format
* fix missing id in protocols
* fix proto prefix in template context
* flow: add unit tests
* conditional flow support using flow
* fix proto callbacks + more unit tests
* adds integration test
* conditional flow: check if req has any matchers
* fix lint error
* deprecate iterate-all+ missing multi-proto implementation
* fix ip input in raw request
* JS: feat dedupe object+ more builtin funcs
* feat: hide protocol result using hide
* feat: async execution
* complete async execution support
* fix condition-flow without any matchers
* refactor: template executer package (tmplexec)
* flow executor working
* fix data race in templateCtx
* templateCtx redesign
* fix failing unit test
* add multiprotocol support to deprecated syntax
* fix race condition in utils & tlsx
* add documentation in flow package
* remove regions.txt file
* fix minor issue with self contained templates
* fix typos of copilot
* dep + misc update
* fix reqID: use req.Type instead of template.Type
---------
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
* use -lfa and -lna in headless
* fix lna in headless
* misc update
* fix nil pointer dereference in test
* fix lint & unit test
* use urlutil
* headless protocol scheme improvements
* add unit and integration tests
* run unit test from binary
---------
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
* Improving clientpool with client certificates
* adding test case
* Revert "Merge branch 'dev' into issue-3800-client-cert"
This reverts commit 7f057d742f4b9bda8e83b2052e29617b86b6776d, reversing
changes made to 7297cebcf8bb0f88961b644fc2ac7c040df8ffd9.
* Revert "Revert "Merge branch 'dev' into issue-3800-client-cert""
This reverts commit 2053a248a0cdc2002e0b4b4faa3472cf11c29760.
* go fmt
---------
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
* add execute function in dns
* Add payload in dns protocol
* Add integration test to cover dns payload
- also check command line overriding a payload variable
* Update matchedAt and remove trailing dot
* Consider payload data for request count
- Update verbose output to print question
- Update dns requests Requests function to consider payload data
* update gitignore
* bump nuclei version to v2.9.4-dev
---------
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
* Check if the variables are override by other means
- you can override the template variable value using command line flags
* Update lazy eval logic
- previously, we were checking any function/expression in variable
- now, update the logic, lazy eval only if variable contains any
protocol variable(global)
* add integration tests
* Add test to check the dsl function working in variable
* gather all generate variables logic in utils
* go mod update
* Refactor the generate variables function
* go mod update+ fix typo
---------
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
* Extend dns extractor to dns answer records
* add test template
* Ignore error for dns variables are not found
* Add all the records of answer section
* Fixed the wrong typecasting
* Templates wrong loading
* Add tests to cover following scenarios
- check optional fields only if template loaded
- it should return warning only if template is loaded
* Evaluate payload variables
* Add variables evaluation
* Extend variables test
- to check evaluation of global variables in variables
- to check evaluation of golbal variables in payload
* Add default and cli variables to websocket, whois and dns proto
- use url.Parse with urlutil.Parse
* fix or condition match even interactsh includes as matcher-part (#3350)
* add integration test
* add new template to integration test
* matcher-condtion: test case for both conditions
* fix lint errors
* upgrade dependencies
---------
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
Tarun Koyalwar
Dogan Can Bakir
sandeep
Stefan Kahn
Daniil Morozov
Tarun Koyalwar
Sandeep Singh
Ramana Reddy
Josh Soref
Mzack9999
Shubham Rasal