Commit Graph

1401 Commits (ed600871ae96dd96a96f908329799e4c98cdc47e)

Author SHA1 Message Date
mzack 1670bf874b fixing lint errors 2021-12-30 13:04:08 +01:00
mzack 7251a2ef60 implementing requested changes 2021-12-30 12:59:42 +01:00
Sajad Parra 8799379e50 update hash function 2021-12-29 18:07:48 +05:30
mzack d0040e440f enabling fallback in tests 2021-12-29 11:16:54 +01:00
mzack d252c3d6ea Adding support for automatic http fallback retry for interactsh 2021-12-29 11:06:33 +01:00
mzack 0a6b84639b adding missing comments 2021-12-29 09:51:50 +01:00
mzack 5d699cdde0 Adding support for full navigation history to headless matchers 2021-12-29 09:48:46 +01:00
Sajad Parra 5115c8b58c remove redundant contains function 2021-12-29 12:33:54 +05:30
Sajad Parra e2b39e403c use ccache to store matchedTemplates with template-id+host hash as key 2021-12-29 12:28:18 +05:30
Sandeep Singh 8fbe451d54
Merge branch 'dev' into feature-ldap 2021-12-25 12:49:44 +05:30
mzack 1f878192ec Merge branch 'dev' into issue-614-scan-resume 2021-12-23 13:15:47 +01:00
Sajad Parra 0edb4274b1 add matchedTemplates to support template wise stop at first match 2021-12-22 21:42:21 +05:30
Sandeep Singh 6ce846971b
Merge pull request #1416 from projectdiscovery/issue-1379-fix-templates-repetitions-with-project-file
Improving projectfile http request matching
2021-12-22 11:20:47 +05:30
Sajad Parra e6aabffc22 add missing custom rdap server assign if present in the template 2021-12-21 19:02:09 +05:30
mzack d1944c76b9 Improving projectfile http request matching 2021-12-21 10:54:25 +01:00
Sajad Parra c0f9c1da70 interactsh stopAtFirstMatch intergration test 2021-12-21 15:24:16 +05:30
Sajad Parra 911045ae9a add stop at first match for interactsh matchers 2021-12-21 14:20:03 +05:30
GitHub Action fdc29180a0 Auto Generate Syntax Docs + JSONSchema [Sun Dec 19 19:37:37 UTC 2021] 🤖 2021-12-19 19:37:37 +00:00
Sandeep Singh 2dc877a99f
Merge pull request #1247 from projectdiscovery/research-aws-signing
Adding support for aws request signing
2021-12-20 01:06:13 +05:30
sandeep 33e98386c9 dev version update 2021-12-19 18:16:06 +05:30
sandeep 6cde5843b5 go mod updates 2021-12-19 18:05:09 +05:30
mzack e59da29371 improving error/args handling 2021-12-18 20:06:51 +01:00
sandeep 2f00e27bd1 dev version update 2021-12-18 19:58:59 +05:30
sandeep 1297c883a6 version update 2021-12-18 19:43:52 +05:30
Sandeep Singh 6a408fd6a8
feature: exposed interaction ip information to matchers/extractors (#1395) 2021-12-18 19:32:11 +05:30
mzack 714f0c82a9 adding missing return error 2021-12-16 23:41:18 +01:00
mzack 63b194c10a refactoring stop-resume logic with better tracking 2021-12-16 22:32:03 +01:00
GitHub Action 4c70e4976b Auto Generate Syntax Docs + JSONSchema [Thu Dec 16 11:39:23 UTC 2021] 🤖 2021-12-16 11:39:23 +00:00
Sajad 5200bcd94f
Whois Protocol Support (using rdap) (#1354)
* init rdap

* add an integration test, option to supply RDAP server to execute the request on

* add rdap protocolMappings

* add debug info, add IP, ASN query type support

* rename rdap to whois, Host to Query in template

* rename pending rdap to whois

* remove port from whois varaiables

* set Host variable even if input is not a parsable url
2021-12-16 17:08:02 +05:30
LuitelSamikshya d152986485
Change filepath to path (#1382)
* filepath to path

* change to revert

* reverted change

* update: disabling cached nuclei temporarily

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2021-12-16 16:36:20 +05:30
GitHub Action 8b20f384a7 Auto Generate Syntax Docs + JSONSchema [Thu Dec 16 10:52:36 UTC 2021] 🤖 2021-12-16 10:52:36 +00:00
Mzack9999 07e7d0795b
Extending deny list to support filenames and folders (#1260)
* Extending deny list to support filenames and folders

* fixing field name

* adding missing edge case with relative path + filename

* handling root path + relative path

* Improving matchers to handle all deny cases
2021-12-16 16:21:06 +05:30
Mzack9999 1fbbce4e41
Adding support for implicit validation during marshal/unmarshal (#1329) 2021-12-16 15:39:38 +05:30
Mike Brown 71040577ce add debug logging for interactsh feature-ldap 2021-12-15 22:52:55 -08:00
forgedhallpass 85e0b96d51
bug: fixed couple of bugs in the DSL functions (#1372)
* feat: Improve DSL function UX #1295

Sort the output signatures

* feat: Improve DSL function UX #1295

Sort the output signatures.
Lint: simplified the sorting.

* bug: fixed couple of bugs in the DSL functions

Input number parameters are stored as float64 types, hence the type conversion should happen accordingly. Affected functions:
* rand_int
* wait_for
* unix_time
* rand_text_numeric

Added tests for all functions.
Related: #1261

* bug: fixed couple of bugs in the DSL functions

Handle cases when the optional input character set is an empty string. Affected methods:
* rand_char
* rand_base

* bug: fixed couple of bugs in the DSL functions

Change rand_char to return a one character string, instead of the character code

* refactor: Minor integration test changes to show the actual and expected result numbers

* test: Added integration test for all existing DSL functions

* test: Added integration test for all existing DSL functions

Fixing linter issues.

* feat: Add "repeat" DSL function

* test: Add "repeat" DSL function
2021-12-15 19:33:57 +05:30
forgedhallpass 974cbfb35e
feat: Re-run failed integration tests in debug mode (#1367)
* If the tests are executed through GitHub actions and there are failed integration tests, they will be re-executed with verbose output to help figuring out the underlying issues.
* Added some grouping to make reading the logs easier

Ticket: #1365
2021-12-14 18:13:53 +02:00
Sandeep Singh fbb80610a8
Interactsh response missing in result fix (#1370)
Co-authored-by: Ice3man543 <nizamulrana@gmail.com>
2021-12-13 23:23:22 +05:30
sandeep d6a3e30233 dev version update 2021-12-13 19:04:22 +05:30
sandeep aae06e61bd fix: updating default interactsh server to use 2021-12-13 18:42:01 +05:30
mzack e279e0437d moving resume file to config folder 2021-12-10 12:45:29 +01:00
sandeep 28b197d79f dev version update 2021-12-09 15:23:24 +05:30
forgedhallpass 10cf1053f8
feat: Improve DSL function UX #1295 (#1351)
* feat: Improve DSL function UX #1295

Sort the output signatures

* feat: Improve DSL function UX #1295

Sort the output signatures.
Lint: simplified the sorting.
2021-12-09 14:02:01 +05:30
mzack abb78658c6 adding default region 2021-12-09 08:50:54 +01:00
Sandeep Singh 17ad60d528
Merge pull request #1296 from projectdiscovery/dsl_signatures
feat: Improve DSL function UX
2021-12-09 12:48:18 +05:30
mzack e244768f32 Fixing typo in variable name while validating template payloads 2021-12-08 19:27:47 +01:00
forgedhallpass 2d5784d992 Merge branch 'dev' into dsl_signatures 2021-12-07 18:22:10 +02:00
forgedhallpass d05c4dc122 refactor: renamed misleading receiver names 2021-12-07 18:01:34 +02:00
forgedhallpass 79aed22d46 feat: Improve DSL function UX #1295
Added support for letting people know if:
* the DSL expression does not return a boolean value
* an invalid custom function signature was provided and then display all available function signatures
* an invalid function was provided and then display the correct signature

Unified the DSL function names to use snake case. The old signatures are also kept for backward compatibility.
2021-12-07 17:34:36 +02:00
mzack 197971e6fa Merge branch 'dev' into research-aws-signing 2021-12-07 09:58:55 +01:00
Mzack9999 72ec69e37d using payload relative path by default 2021-12-06 21:05:45 +01:00
mzack 9ab674adbc improving cross-os filepath handling 2021-12-06 16:04:39 +01:00
Mzack9999 a72425289b Improving path handling on windows 2021-12-06 11:38:22 +01:00
mzack c47196bcc8 Merge branch 'dev' into research-aws-signing 2021-12-06 08:17:03 +01:00
Sandeep Singh df55f7a2eb
Disabling no-sandbox in headless engine (#1135)
* Disabling no-sandbox in headless engine

* limiting disabling sandbox to bare minimum

* adding warnings related to linux os and root user requirement

Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
2021-12-05 20:14:16 +05:30
sandeep fb3b4fc510 version update 2021-12-05 19:45:44 +05:30
Mzack9999 3c88afac0c
Fixing payloads path during validation (#1320)
* Fixing payloads path during validation

* Added GH Action for public template parsing / validation

* tracking payload errors as syntax warnings

* improving path parsing + introducing hard failure for runtime errors on validation

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2021-12-05 19:41:14 +05:30
GitHub Action 0ca39bb6d2 Auto Generate Syntax Docs + JSONSchema [Fri Dec 3 03:46:08 UTC 2021] 🤖 2021-12-03 03:46:08 +00:00
Sandeep Singh eac6ea1bfe
Merge pull request #1284 from projectdiscovery/dns-optional
Added default fields for DNS requests
2021-12-03 09:14:49 +05:30
GitHub Action 5eb17833e4 Auto Generate Syntax Docs + JSONSchema [Thu Dec 2 19:06:54 UTC 2021] 🤖 2021-12-02 19:06:54 +00:00
mzack c9b9725474 moving sign routine to helper function 2021-12-02 15:57:52 +01:00
GitHub Action 46a2a4440d Auto Generate Syntax Docs + JSONSchema [Thu Dec 2 12:32:20 UTC 2021] 🤖 2021-12-02 12:32:20 +00:00
Mzack9999 40baa302b2
Merge branch 'dev' into research-aws-signing 2021-12-02 12:56:37 +01:00
Sajad Parra d0deef72f5 Merge branch 'dev' into stop-at-first-match 2021-12-02 17:26:23 +05:30
Sandeep Singh a0da7452e3
Merge pull request #1315 from projectdiscovery/validate-flag
validate flag updates
2021-12-02 17:23:34 +05:30
Sajad Parra 18f57130ca Merge branch 'dev' into stop-at-first-match 2021-12-02 17:03:02 +05:30
Sajad Parra 7a96a7c931 Merge branch 'dev' into dsl-variable-support 2021-12-02 16:50:50 +05:30
GitHub Action 9a0ce60a4f Auto Generate Syntax Docs + JSONSchema [Thu Dec 2 10:59:29 UTC 2021] 🤖 2021-12-02 10:59:29 +00:00
Sandeep Singh 3b68c29f48
Merge pull request #1288 from projectdiscovery/dynamic-value-reuse-http
Reusing dynamically extracted values as iterators in http request
2021-12-02 16:27:59 +05:30
mzack c807438ecd fixing memory allocation for boolean pointer 2021-12-02 11:19:37 +01:00
LuitelSamikshya 6d5146e540 validate flag updates 2021-12-01 10:35:18 -06:00
GitHub Action a9f6f04394 Auto Generate Syntax Docs + JSONSchema [Tue Nov 30 17:17:46 UTC 2021] 🤖 2021-11-30 17:17:46 +00:00
Ice3man543 c17aba8646 Misc fixes to part definitions 2021-11-30 22:41:02 +05:30
Sajad Parra 5b99921d75 add unit and integration tests for dsl variable #555 2021-11-30 20:20:43 +05:30
Sajad Parra 10e5595980 Merge branch 'dev' into dsl-variable-support 2021-11-30 18:41:18 +05:30
Sajad Parra de236379a1 remove stop-at-first-match info log #1188 2021-11-30 17:22:39 +05:30
Ice3man543 72a387c40a Merge from dev 2021-11-30 16:55:09 +05:30
Ice3man543 847e9d7c44 Misc changes to fix testa and lint 2021-11-30 16:26:29 +05:30
Ice3man543 eb7c704d36 Merging from dev 2021-11-30 16:23:39 +05:30
Sandeep Singh be66166b9d
Merge pull request #1282 from projectdiscovery/network-vars
Added additional variables for network + simplified logic
2021-11-30 12:10:26 +05:30
sandeep 684f332599 fix: {{Hostname}} to {{Host}} in test 2021-11-30 12:04:47 +05:30
Ice3man543 65a1858f94 fix #1303: speed regression due to missing goroutine in executer
Fixed speed regression introduced in dev due to missing go() statement when executing template input
2021-11-30 00:17:52 +05:30
mzack 2140ffa382 Adding stop-resume support 2021-11-29 14:38:45 +01:00
Sajad Parra a7ecadf189 add template level stop-at-first-match to workflows 2021-11-29 18:06:25 +05:30
Sajad Parra f60e93aae1 add template level stop at first match option 2021-11-29 16:01:06 +05:30
mzack a631262d91 Merge branch 'dev' into research-aws-signing 2021-11-29 09:36:12 +01:00
Sajad Parra d97aee01c3 Merge branch 'dev' of https://github.com/projectdiscovery/nuclei into dsl-variable-support 2021-11-29 13:36:09 +05:30
Sajad Parra 5deb454a81 merge Hostname variable to payloads in network request 2021-11-29 13:26:03 +05:30
GitHub Action 345a5a3de5 Auto Generate Syntax Docs + JSONSchema [Mon Nov 29 06:49:56 UTC 2021] 🤖 2021-11-29 06:49:56 +00:00
Sandeep Singh 95f31090dc
Merge pull request #1292 from projectdiscovery/improvements
Small refactorings and linter driven improvements
2021-11-29 12:18:28 +05:30
sandeep 5a1c7a62b2 missing goimports 2021-11-28 04:55:18 +05:30
Sandeep Singh 670f0d6775
Merge branch 'dev' into network-vars 2021-11-28 04:42:08 +05:30
LuitelSamikshya b3805999f3
Unsafe flag with base template (#1279)
* unsafe flag with base template
2021-11-28 04:40:27 +05:30
forgedhallpass c61ec5f673 feat: Improve DSL function UX #1295 2021-11-26 17:14:25 +02:00
Ice3man dfe284664c
Fixed a crash with http module (#1285) 2021-11-26 18:51:02 +05:30
mzack a3319930c0 code refactor + request dump fix
Fixes indirectly #844
2021-11-26 13:49:12 +01:00
Ice3man543 c720354be2 Added part definition information to docs + misc 2021-11-26 16:23:54 +05:30
forgedhallpass 5eca474c5e refactor(CR): better way of writing switch statement 2021-11-26 11:38:09 +02:00
forgedhallpass 7e22d70ded refactor/documentation: typos and grammatical errors 2021-11-25 18:54:16 +02:00
forgedhallpass ec6889931d refactor: linter driven fixes
* x = x + ""  => x += ""
* pre-allocating slice with known size
* added t.Helper() methods in test helpers
* complex if-else conditions replaced by switches
* errors should be checked using error.Is() instead of ==
* function parameter should start with lower case letter
* removed unnecessary type definition
* variable/label naming convention: camelCase instead of snake_case
2021-11-25 17:57:22 +02:00
forgedhallpass fdaa8e4539 refactor: removed redundant type conversion 2021-11-25 17:18:54 +02:00
forgedhallpass 47340f06b0 refactor: uniformly sorted imports 2021-11-25 17:09:20 +02:00
forgedhallpass 3fd1f57b96 refactor: godoc and comment uniformization
Adding space after // and before the godoc/comment
2021-11-25 17:03:56 +02:00
forgedhallpass c8d009654c fix: Receiver mismatch warnings 2021-11-25 16:57:43 +02:00
forgedhallpass bebe5cbcfd refactor: Remove redundant character escapes from regexes 2021-11-25 16:24:37 +02:00
forgedhallpass fdd22ab668 refactor: Wrap errors using %w in fmt.Errorf
see:
 * https://github.com/xxpxxxxp/intellij-plugin-golangci-lint/blob/master/explanation/goerr113.md
 * https://go.dev/blog/go1.13-errors#wrapping-errors-with-w
2021-11-25 16:24:37 +02:00
Ice3man543 caaa5c6594 Automatically generate docs for enum values 2021-11-25 19:34:01 +05:30
forgedhallpass 4bccb6cf8a fix: File and directory creation permission changes
Directories: 0755 (5 - group and other can read cd into the directory and read it's content)
Files: 0644 (4 - group and other can only read the created files)
Tests files: 0777
2021-11-25 14:37:57 +02:00
Sajad Parra 7e9272776d add variable support to dsl, remove dynamicValues from request struct 2021-11-25 16:19:42 +05:30
Ice3man543 f6072e9a95 Updating name of variable 2021-11-24 22:44:43 +05:30
Ice3man543 393babe3c3 Made iterate of values controllable with iterate-all-values flag in http 2021-11-24 22:40:17 +05:30
Ice3man543 273233bb81 Fixed a bug with mergemanymaps + misc lint fixes 2021-11-24 21:56:55 +05:30
Ice3man543 ef5b476c6d Reusing dynamically extracted values as iterators in http request
This PR adds the support in http module to iterate over the dynamically extracted data from extractors
and use it in other requests. This allows nuclei to follow links on pages, do operations with
multiple versions of the same extracted value, etc.
2021-11-24 21:08:08 +05:30
Ice3man543 a908a1515a Fixed example docs 2021-11-23 13:17:19 +05:30
Ice3man543 44e304179a Fixed a linter error 2021-11-23 13:15:24 +05:30
Ice3man543 831114bdbd Added default fields for DNS requests 2021-11-23 13:09:22 +05:30
Ice3man 1851e37aef
Merge pull request #1273 from projectdiscovery/fixed-missing-ip
fix #1220: no ip returned for dialers
2021-11-23 10:49:51 +05:30
Ice3man543 6c819d7917 Added additional variables for network + simplified logic 2021-11-23 10:44:31 +05:30
Ice3man 1581c96e4e
Added matched-status flag + template-path and url to output (#1272)
* Added matched-status flag + template-path and url to output
2021-11-22 17:53:25 +05:30
Ice3man543 ba67f8c8fe fix #1220: no ip returned for dialers 2021-11-22 09:51:13 +05:30
Mzack9999 ca9676f52e
Adding support for clustering within workflow (#1255)
* Adding support for clustering within workflow
2021-11-22 04:49:53 +05:30
Ice3man543 bdf1c2010b Fixed jsonschema for matchertype missing String() 2021-11-20 19:50:18 +05:30
Ice3man543 66dacccfb4 Fixed matcher type integer issue 2021-11-20 19:47:05 +05:30
Sajad f74ff3fc49
Tag based struct validation (#1256)
* Added tag based struct validation
2021-11-20 13:25:27 +05:30
mzack ed55de71d9 fixing internal normalized value 2021-11-20 02:26:16 +01:00
GitHub Action b8246ab8e5 Auto Generate Syntax Docs + JSONSchema [Fri Nov 19 10:55:27 UTC 2021] 🤖 2021-11-19 10:55:27 +00:00
LuitelSamikshya 4b8ec29d8d
struct to enums changes for Matchers (#1246)
* struct to enums changes for Matchers
2021-11-19 16:24:09 +05:30
forgedhallpass 09eba6c0cf
feat: In case of binary data, show a hexadecimal view as well #1080 (#1266)
Highlight the longest occurrences when there are matches that are substrings of each other
2021-11-19 15:45:43 +05:30
mzack 71d4e5318a . 2021-11-18 22:28:10 +01:00
mzack 359eb7c3d2 making iota zero based 2021-11-18 21:58:32 +01:00
mzack bdffa373cf solving linting issues 2021-11-18 21:54:24 +01:00
mzack b42f0d32ea Merge branch 'dev' into research-aws-signing 2021-11-18 21:50:49 +01:00
mzack ddb07ebb75 adding support for aws file credential 2021-11-18 21:44:58 +01:00
GitHub Action 5a0f483cd3 Auto Generate Syntax Docs + JSONSchema [Thu Nov 18 20:12:46 UTC 2021] 🤖 2021-11-18 20:12:46 +00:00
LuitelSamikshya 2856e7e247
Extractor + Headless Actions structures to enums (#1217)
* enum support for extractor
2021-11-19 01:41:10 +05:30
GitHub Action ccb588f383 Auto Generate Syntax Docs + JSONSchema [Thu Nov 18 13:53:42 UTC 2021] 🤖 2021-11-18 13:53:42 +00:00
Mzack9999 7f5f791e23
Adding dns trace support in dns templates (#1236)
* Adding dns trace support in dns templates + minor refactoring
2021-11-18 19:22:11 +05:30
LuitelSamikshya d078b72381
enum changes for http method (#1234)
* enum changes for http method
2021-11-18 19:20:21 +05:30
GitHub Action 25189c3ae7 Auto Generate Syntax Docs + JSONSchema [Thu Nov 18 13:50:07 UTC 2021] 🤖 2021-11-18 13:50:07 +00:00
LuitelSamikshya 64a93a4570
enum changes for network type (#1233)
* enum changes for network type
2021-11-18 19:18:47 +05:30
LuitelSamikshya 4ff90c1373
struct to enum changes for Dns Type (#1245)
* struct to enum changes for Dns Type
2021-11-18 19:17:34 +05:30
mzack 38ff8f75b1 adding signature syntax validation 2021-11-17 01:28:35 +01:00
mzack 0379afc748 Merge branch 'dev' into research-aws-signing 2021-11-17 00:03:00 +01:00
Ice3man eaa0d37065
feat #1092: Validate binary values + precompile them as well (#1213)
* Added validation for binary values + precompile them

* Changed name of the binary matcher field

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2021-11-17 02:04:27 +05:30
sandeep f73b20e415 Merge branch 'dev' of https://github.com/projectdiscovery/nuclei into research-aws-signing 2021-11-17 02:02:07 +05:30
Ice3man 50a816fce9
Fixed a crash with uninitialized interactsh client (#1251)
* Fixed a crash with uninitialized interactsh client
2021-11-16 20:02:39 +05:30
Ice3man543 cc33366acb fixed a conditon with workflow type switch 2021-11-14 15:30:28 +05:30
mzack eb61c519c3 converting to post-processor prototype 2021-11-13 03:17:05 +01:00
mzack e517797cfa moving aws signing logic to helper library 2021-11-13 02:13:48 +01:00
mzack 34889d50f8 correcting variable name 2021-11-12 19:58:12 +01:00
mzack 882a156b63 Merge branch 'dev' into research-aws-signing 2021-11-12 19:55:35 +01:00