Commit Graph

1722 Commits (231ac895522af2034093910ee1b2827b15a84575)

Author SHA1 Message Date
GitHub Action 3ad1fec1d2 Auto Generate Syntax Docs + JSONSchema [Tue Sep 20 21:37:56 UTC 2022] 🤖 2022-09-20 21:37:56 +00:00
Mzack9999 99c14f4c9c
implementation of rate limiter with bucket refill and unrestricted token burst (#2536)
* implementation of rate limiter with interval burst

* fixing import path

* fixing syntax

* adding tests

* fixing lint errors

* adding support for context

* moving rate limiter earlier to avoid hitting timeout
2022-09-19 17:09:28 +05:30
Ice3man 03c7e89d9e
Added workflow names based condition (#2594)
* Added workflow names based condition

* Added conditional filtering to workflow executor

* Replaced names with single name stringslice

* Fixed workflow matcher condition + tests

* Fixed workflow item name
2022-09-19 16:49:30 +05:30
mzack 37c016a5f0 fixing lint errors 2022-09-19 08:38:52 +02:00
Ice3man 04b47b0309
Added custom json-unmarshaller + misc updates (#2556)
* Added custom json-unmarshaller + misc updates

* Added support for nuclei-cloud based scan execution

* Removed unnecessary files

* Misc

* Changes as per review comments

* misc option update

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-09-19 01:13:59 +05:30
Ice3man 7e209dad01
Added multi-protocol requests support for templates (#2611) 2022-09-16 23:10:39 +05:30
Sajad 5377ee3f36
add proxy support at dialer level (#2549)
* add proxy support at dailer level

* add forward dialer to proxy
2022-09-16 21:36:17 +05:30
Sami f3eb0daa39
additional dsl functions (#2550)
* additional dsl functions

* avoid conversion at each iteration
2022-09-08 14:25:34 +05:30
Mzack9999 7ce03bcc5b
Optional use of local chrome for headless tests via tags (#2568) 2022-09-07 16:09:22 +05:30
forgedhallpass fc0763641f
New dsl functions (#2545)
* Update GO version to 1.18

* Removed redundant entry from the .gitignore file

* Added new DSL functions

to_unix_time(input string, optionalLayout string) int64
hex_to_dec(input string) float64
oct_to_dec(input string|number) float64
bin_to_dec(intput string|number) float64

* Notify if debug is enabled when a proxy cannot be validated

* Documentation: Go version requirement updated to 1.18

* test fix: Timezone agnostic date expectation in the assertion

* code review: extracted the default date-time layouts into a global variable
2022-09-07 00:44:29 +05:30
sandeep 567a8c60a2 dev version update 2022-09-02 12:52:07 +05:30
Ice3man e193e7c87e
Added tlsx integration to nuclei (#2522)
* Added tlsx integration to nuclei

* tls tests fix

* Added helper functions + upgrade tlsx to fix

* go mod update

* workflow fix to race test on windows

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-09-01 23:56:55 +05:30
James Turner 42a0732d68
Add sha512 support (#2517) 2022-08-31 12:36:02 +05:30
Ice3man 466176e9e8
Merge pull request #2500 from projectdiscovery/goflags-update
Updated goflags to latest + misc
2022-08-30 11:52:25 +05:30
M. Ángel Jimeno 62a4e0aa52
Return wrapped errors for DSL compilation problems (#2492)
This allows the DSL help information to be printed when in debug mode.

Fixes #2481
2022-08-29 13:41:32 +05:30
Myung-jong Kim 01fbb3050d
Added option to list DSL function (#2497)
* Add lds flag

* misc flag update

* readme update

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-08-28 16:37:21 +05:30
Ice3man 8892250583 Updated goflags to latest + misc 2022-08-27 19:35:17 +05:30
sandeep efdc57c7b2 version update 2022-08-26 14:18:32 +05:30
Sajad 011da1388d
add option to specify network interface (#2384)
* add option to specify network interface

* add source-ip flag

* fix typo

* fix err return

* readme update

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2022-08-25 17:42:35 +05:30
Mzack9999 30054d1fb6
Adding advanced template filtering (#2374)
* Adding advanced template filtering

* fixing bug in slice

* refactoring tests

* adding test cases

* increasing error verbosity

* fixing quoted fields with spaces

* adding more test cases

* fixing merge error

* fixing lint errors

* switching to []string

* updating tag filter tests

* updating functional tests

* fixing functional test cases

* updating syntax
2022-08-25 16:52:08 +05:30
Sami 0aac36a44b
added custom config flag (#2399)
* added custom config flag

* config.yaml file in custom  directory

* lint error fix

* few updates and error checks

* fix lint error

* copy config.yaml file if the dest folder does not exist

* lint error check

* added integration test

* improved test cases

* lint error fix
2022-08-25 16:10:07 +05:30
51pwn 606c361b2a
Add `substr` and `aes_cbc` DSL functions (#2361)
* 1、add DSL substr for #2304 By @hktalent
substr('xxtestxxx',2)。   testxxx
substr('xxtestxxx',2,-2)  testx
substr('xxtestxxx',2,6)   test

2、add DSL aes_cbc for #2243 By @hktalent
aes_cbc("key111key111key111key111", "dataxxxxxxdataxxxxxxdataxxxxxxdataxxxxxxdataxxxxxx")

3、fixed An error occurs when running nuclei with multiple instances #2301 By @hktalent

* refactoring helpers

* removing unwanted mutex

* commenting out test

* removing aes_cbc test due to random iv

Co-authored-by: 51pwn <51pwn@51pwn.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2022-08-25 15:50:08 +05:30
Ice3man 0be596efb4
Added variable debug support with debug mode (#2442)
* Added variable debug support with debug mode

* Added changes as per review comments

* Fixed debug request condition
2022-08-25 15:37:03 +05:30
Ice3man 7b7936b7a5
Added show-actions flag to display headless actions (#2456)
* Added show-actions flag to display headless actions

* misc update

* readme update

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-08-25 10:43:32 +05:30
M. Ángel Jimeno ecb3f21076
http: prevent HTTP 'connection' header from being added twice (#2480)
* http: prevent HTTP 'connection' header from being added twice

* misc fix

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-08-25 00:25:02 +05:30
Ice3man 8165db2633
Fixed fatal panic in http header map read (#2488) 2022-08-24 23:29:22 +05:30
Ice3man 8f313629b8
Memory usage optimizations (#2350)
* Replaced strings.Replaced with fasttemplate reducing allocations

Custom template parsing logic was replaced with fasttemplate package for reducing
allocations in the replacer.Replace hotpath leading to allocation reduction which
accounted for 30% of total nuclei allocations.

$ go test -bench=. -benchmem
goos: darwin
goarch: arm64
pkg: github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/replacer
BenchmarkReplacer-8               837232              1422 ns/op            2112 B/op         31 allocs/op
BenchmarkReplacerNew-8           3672765               320.3 ns/op            48 B/op          4 allocs/op

* Fixed tests failing

* Use pre-compiled map of DSL expressions

* Reworked expression parsing logic to reduce memory allocations

$ go test -bench=. -benchmem
goos: darwin
goarch: arm64
pkg: github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/expressions
BenchmarkEvaluate-8        31560             37769 ns/op           31731 B/op        265 allocs/op
BenchmarkEvaluateNew-8       109144              9621 ns/op            6253 B/op        116 allocs/op
2022-08-23 13:16:41 +05:30
Ice3man e7cffad312
Fixed request annotation based timeout bugs + tests + misc (#2476) 2022-08-23 12:45:55 +05:30
Dani Goland 8670c8b20d
Modified "xpath" extractor to support XML XPath in addition to HTML XPath (#2471)
* Modified "xpath" extractor to support XML XPath in addition to HTML XPath

* Updated function docs
2022-08-22 15:27:32 +05:30
xixijun 2ae7e58c83
Fix socks5 proxy not working on tor proxy (#2455)
* fix: socks5 proxy not working on tor proxy

* fix: socks5 proxy not working on tor proxy

* minor refactoring

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2022-08-22 15:18:45 +05:30
Sami d14c00fc6f
added validation for headless templates (#2423)
* added validation for headless templates

* minor update in log msg
2022-08-17 17:10:27 +04:00
Ice3man 9e531727a7
Fixed a bug with numerical regex in unresolved var detection (#2431) 2022-08-17 03:59:51 +04:00
sandeep 3193bf8f94 version update 2022-08-13 01:26:43 -07:00
Ice3man 67d5769cd9
Added initial catalog interface implementation (#2318)
* Added initial catalog interface implementation

* Added OpenFile to Catalog + disk catalog implementation

* Fixed merge issues

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-08-10 11:05:58 -07:00
Ice3man 4dc98a1d95
Added support for blank Request-URI which specifies no slash suffix (#2414) 2022-08-10 10:15:09 -07:00
Sajad c4ba2b4edb
Additional helpers (#2359)
* add starts_with, ends_with helper functions

* add test cases, update dsl signatures

* change split_starts_with to line_starts_with
2022-08-06 23:16:03 +05:30
sandeep c815f53e67 interactsh test domain update 2022-08-04 19:37:33 +05:30
forgedhallpass d24736f655 fix typo in the headless `setmethod` function #2365 2022-07-29 14:38:07 +03:00
sandeep 125588046e version update 2022-07-28 17:05:29 +05:30
Mzack9999 b942ddc6ad
Fixing map race condition (#2340) 2022-07-26 18:30:15 +05:30
Sami 4da4ca5a16
missing ip in json (#2310)
* missing ip in json

* using GetDNSData in place of GetDialedIP

* updated go mod

* bumping rawhttp test version

Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
2022-07-26 17:08:53 +05:30
sandeep 3052d8a7f6 Merge branch 'dev' of https://github.com/projectdiscovery/nuclei into dev 2022-07-24 15:00:44 +05:30
sandeep ee9f8b7651 version update 2022-07-24 15:00:36 +05:30
Sajad 005b92217f
return on parse template error unconditionally (#2327) 2022-07-24 14:56:06 +05:30
Ice3man 7d7314e3f3
Added global variables support to SSL protocol (#2325) 2022-07-22 01:35:21 +05:30
Ice3man 2873e6ebc8
Added timeout context cancellation to http requests (#2319) 2022-07-21 21:29:34 +05:30
Mike Rheinheimer 9efba05e0c
expose hosterrorscache.Cache as an interface (#2291)
* expose hosterrorscache as an interface, change signature to capture the error reason

* use the hosterrorscache.CacheInterface as struct field so users of Nuclei embedded can provide their own cache implementation

Co-authored-by: Mike Rheinheimer <mrheinheimer@atlassian.com>
2022-07-19 02:05:53 +05:30
Ice3man 07d5beb73a
Fixed race condition with raw http options (#2306) 2022-07-19 01:08:30 +05:30
Mzack9999 cf1039f49c
Adding prototype of request flow override annotations (#2161)
* Adding prototype of request flow override annotations

* reworking iteration engine

* adding directory to .gitignore
2022-07-18 14:16:03 +05:30
sandeep 4ae458df98 readme update 2022-07-18 13:11:28 +05:30