7e209dad01
Added multi-protocol requests support for templates ( #2611 )
2022-09-16 23:10:39 +05:30
5377ee3f36
add proxy support at dialer level ( #2549 )
...
* add proxy support at dailer level
* add forward dialer to proxy
2022-09-16 21:36:17 +05:30
f3eb0daa39
additional dsl functions ( #2550 )
...
* additional dsl functions
* avoid conversion at each iteration
2022-09-08 14:25:34 +05:30
7ce03bcc5b
Optional use of local chrome for headless tests via tags ( #2568 )
2022-09-07 16:09:22 +05:30
fc0763641f
New dsl functions ( #2545 )
...
* Update GO version to 1.18
* Removed redundant entry from the .gitignore file
* Added new DSL functions
to_unix_time(input string, optionalLayout string) int64
hex_to_dec(input string) float64
oct_to_dec(input string|number) float64
bin_to_dec(intput string|number) float64
* Notify if debug is enabled when a proxy cannot be validated
* Documentation: Go version requirement updated to 1.18
* test fix: Timezone agnostic date expectation in the assertion
* code review: extracted the default date-time layouts into a global variable
2022-09-07 00:44:29 +05:30
567a8c60a2
dev version update
2022-09-02 12:52:07 +05:30
e193e7c87e
Added tlsx integration to nuclei ( #2522 )
...
* Added tlsx integration to nuclei
* tls tests fix
* Added helper functions + upgrade tlsx to fix
* go mod update
* workflow fix to race test on windows
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-09-01 23:56:55 +05:30
42a0732d68
Add sha512 support ( #2517 )
2022-08-31 12:36:02 +05:30
466176e9e8
Merge pull request #2500 from projectdiscovery/goflags-update
...
Updated goflags to latest + misc
2022-08-30 11:52:25 +05:30
62a4e0aa52
Return wrapped errors for DSL compilation problems ( #2492 )
...
This allows the DSL help information to be printed when in debug mode.
Fixes #2481
2022-08-29 13:41:32 +05:30
01fbb3050d
Added option to list DSL function ( #2497 )
...
* Add lds flag
* misc flag update
* readme update
Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-08-28 16:37:21 +05:30
8892250583
Updated goflags to latest + misc
2022-08-27 19:35:17 +05:30
efdc57c7b2
version update
2022-08-26 14:18:32 +05:30
011da1388d
add option to specify network interface ( #2384 )
...
* add option to specify network interface
* add source-ip flag
* fix typo
* fix err return
* readme update
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2022-08-25 17:42:35 +05:30
30054d1fb6
Adding advanced template filtering ( #2374 )
...
* Adding advanced template filtering
* fixing bug in slice
* refactoring tests
* adding test cases
* increasing error verbosity
* fixing quoted fields with spaces
* adding more test cases
* fixing merge error
* fixing lint errors
* switching to []string
* updating tag filter tests
* updating functional tests
* fixing functional test cases
* updating syntax
2022-08-25 16:52:08 +05:30
0aac36a44b
added custom config flag ( #2399 )
...
* added custom config flag
* config.yaml file in custom directory
* lint error fix
* few updates and error checks
* fix lint error
* copy config.yaml file if the dest folder does not exist
* lint error check
* added integration test
* improved test cases
* lint error fix
2022-08-25 16:10:07 +05:30
606c361b2a
Add `substr` and `aes_cbc` DSL functions ( #2361 )
...
* 1、add DSL substr for #2304 By @hktalent
substr('xxtestxxx',2)。 testxxx
substr('xxtestxxx',2,-2) testx
substr('xxtestxxx',2,6) test
2、add DSL aes_cbc for #2243 By @hktalent
aes_cbc("key111key111key111key111", "dataxxxxxxdataxxxxxxdataxxxxxxdataxxxxxxdataxxxxxx")
3、fixed An error occurs when running nuclei with multiple instances #2301 By @hktalent
* refactoring helpers
* removing unwanted mutex
* commenting out test
* removing aes_cbc test due to random iv
Co-authored-by: 51pwn <51pwn@51pwn.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2022-08-25 15:50:08 +05:30
0be596efb4
Added variable debug support with debug mode ( #2442 )
...
* Added variable debug support with debug mode
* Added changes as per review comments
* Fixed debug request condition
2022-08-25 15:37:03 +05:30
7b7936b7a5
Added show-actions flag to display headless actions ( #2456 )
...
* Added show-actions flag to display headless actions
* misc update
* readme update
Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-08-25 10:43:32 +05:30
ecb3f21076
http: prevent HTTP 'connection' header from being added twice ( #2480 )
...
* http: prevent HTTP 'connection' header from being added twice
* misc fix
Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-08-25 00:25:02 +05:30
8165db2633
Fixed fatal panic in http header map read ( #2488 )
2022-08-24 23:29:22 +05:30
8f313629b8
Memory usage optimizations ( #2350 )
...
* Replaced strings.Replaced with fasttemplate reducing allocations
Custom template parsing logic was replaced with fasttemplate package for reducing
allocations in the replacer.Replace hotpath leading to allocation reduction which
accounted for 30% of total nuclei allocations.
$ go test -bench=. -benchmem
goos: darwin
goarch: arm64
pkg: github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/replacer
BenchmarkReplacer-8 837232 1422 ns/op 2112 B/op 31 allocs/op
BenchmarkReplacerNew-8 3672765 320.3 ns/op 48 B/op 4 allocs/op
* Fixed tests failing
* Use pre-compiled map of DSL expressions
* Reworked expression parsing logic to reduce memory allocations
$ go test -bench=. -benchmem
goos: darwin
goarch: arm64
pkg: github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/expressions
BenchmarkEvaluate-8 31560 37769 ns/op 31731 B/op 265 allocs/op
BenchmarkEvaluateNew-8 109144 9621 ns/op 6253 B/op 116 allocs/op
2022-08-23 13:16:41 +05:30
e7cffad312
Fixed request annotation based timeout bugs + tests + misc ( #2476 )
2022-08-23 12:45:55 +05:30
8670c8b20d
Modified "xpath" extractor to support XML XPath in addition to HTML XPath ( #2471 )
...
* Modified "xpath" extractor to support XML XPath in addition to HTML XPath
* Updated function docs
2022-08-22 15:27:32 +05:30
2ae7e58c83
Fix socks5 proxy not working on tor proxy ( #2455 )
...
* fix: socks5 proxy not working on tor proxy
* fix: socks5 proxy not working on tor proxy
* minor refactoring
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2022-08-22 15:18:45 +05:30
d14c00fc6f
added validation for headless templates ( #2423 )
...
* added validation for headless templates
* minor update in log msg
2022-08-17 17:10:27 +04:00
9e531727a7
Fixed a bug with numerical regex in unresolved var detection ( #2431 )
2022-08-17 03:59:51 +04:00
3193bf8f94
version update
2022-08-13 01:26:43 -07:00
67d5769cd9
Added initial catalog interface implementation ( #2318 )
...
* Added initial catalog interface implementation
* Added OpenFile to Catalog + disk catalog implementation
* Fixed merge issues
Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-08-10 11:05:58 -07:00
4dc98a1d95
Added support for blank Request-URI which specifies no slash suffix ( #2414 )
2022-08-10 10:15:09 -07:00
c4ba2b4edb
Additional helpers ( #2359 )
...
* add starts_with, ends_with helper functions
* add test cases, update dsl signatures
* change split_starts_with to line_starts_with
2022-08-06 23:16:03 +05:30
c815f53e67
interactsh test domain update
2022-08-04 19:37:33 +05:30
d24736f655
fix typo in the headless `setmethod` function #2365
2022-07-29 14:38:07 +03:00
125588046e
version update
2022-07-28 17:05:29 +05:30
b942ddc6ad
Fixing map race condition ( #2340 )
2022-07-26 18:30:15 +05:30
4da4ca5a16
missing ip in json ( #2310 )
...
* missing ip in json
* using GetDNSData in place of GetDialedIP
* updated go mod
* bumping rawhttp test version
Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
2022-07-26 17:08:53 +05:30
3052d8a7f6
Merge branch 'dev' of https://github.com/projectdiscovery/nuclei into dev
2022-07-24 15:00:44 +05:30
ee9f8b7651
version update
2022-07-24 15:00:36 +05:30
005b92217f
return on parse template error unconditionally ( #2327 )
2022-07-24 14:56:06 +05:30
7d7314e3f3
Added global variables support to SSL protocol ( #2325 )
2022-07-22 01:35:21 +05:30
2873e6ebc8
Added timeout context cancellation to http requests ( #2319 )
2022-07-21 21:29:34 +05:30
9efba05e0c
expose hosterrorscache.Cache as an interface ( #2291 )
...
* expose hosterrorscache as an interface, change signature to capture the error reason
* use the hosterrorscache.CacheInterface as struct field so users of Nuclei embedded can provide their own cache implementation
Co-authored-by: Mike Rheinheimer <mrheinheimer@atlassian.com>
2022-07-19 02:05:53 +05:30
07d5beb73a
Fixed race condition with raw http options ( #2306 )
2022-07-19 01:08:30 +05:30
cf1039f49c
Adding prototype of request flow override annotations ( #2161 )
...
* Adding prototype of request flow override annotations
* reworking iteration engine
* adding directory to .gitignore
2022-07-18 14:16:03 +05:30
4ae458df98
readme update
2022-07-18 13:11:28 +05:30
85ca247d26
chore(deps): bump github.com/go-rod/rod from 0.107.3 to 0.108.1 in /v2 ( #2272 )
...
* chore(deps): bump github.com/go-rod/rod from 0.107.3 to 0.108.1 in /v2
Bumps [github.com/go-rod/rod](https://github.com/go-rod/rod ) from 0.107.3 to 0.108.1.
- [Release notes](https://github.com/go-rod/rod/releases )
- [Commits](https://github.com/go-rod/rod/compare/v0.107.3...v0.108.1 )
---
updated-dependencies:
- dependency-name: github.com/go-rod/rod
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* increasing page timeout
* further increasing timeout
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
2022-07-14 03:14:13 +05:30
db727db006
Optionally disable templates syntax strict check ( #2266 )
...
* nuclei::templates|define strict option (default)
* renaming flag and internal variable
* misc flag update
Co-authored-by: c-f <you@example.com>
Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-07-13 17:00:11 +05:30
6c2fdd3387
Issue 2227 ntv flag run new templates added in specific version ( #2275 )
...
* ntv flag to run templates added in specified version
* added missing arguments
* misc update
* added functional test and err check
* updated the min version
Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-07-13 16:49:06 +05:30
be73fde0db
use go-homedir instead of standard os.userhomedir ( #2262 )
...
* use go-homedir instead of standard os.userhomedir
* set r.templatesConfig before write attempt to avoid panic
2022-07-13 13:33:13 +05:30
1c332bb85b
Improving RFC request/response passive parsing ( #2192 )
...
* Improving RFC request/response passive parsing
* adding test
2022-07-11 22:43:10 +05:30
3c945f6ae9
Adding stricter check on offline templates list ( #2213 )
2022-07-11 22:38:07 +05:30
5b3c2861c2
Added interact-url placeholder support to variables in http requests ( #2237 )
...
* Added interact-url placeholder support to variables in http requests
* Fixed variable errors
* Fixed issue with interactsh in req
2022-07-11 22:18:13 +05:30
73a0043f2d
fix: socks5 proxy not working on https target ( #2228 )
...
* fix: socks5 proxy not working on https target
* small name refactor
Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
2022-07-01 15:31:00 +05:30
476773ff8c
Replacing hasstdin with helper library ( #2191 )
...
* Replacing hasstdin with helper library
* adding timeout reader on stdin
* adding large input read timeout
* reducing stdin timeout + nostdin flag
* go mod update
* readme update
* go mod tidy
Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-06-30 17:51:54 +05:30
3838d06aeb
Adding prototype for unexpected fields validation in matchers ( #2171 )
...
* Adding unexpected fields validation
* using expected fields
* adding missing expected field
* using reflect
2022-06-30 16:50:54 +05:30
f3de611b49
Added enhancements for http variables support ( #2223 )
2022-06-28 20:20:18 +05:30
8040b66370
Added http request timeout support with annotations ( #2233 )
...
* Added http request timeout support with annotations
* Added nolint statements for lostcontext
* misc
* misc
2022-06-27 18:36:46 +05:30
ffe6ab04b3
Added include-templates force-loading for templates ( #2232 )
...
* Added include-templates force-loading for templates
* Fixed loader case with include-templates
* Added integration test for excluded-template in loader
2022-06-27 18:09:29 +05:30
7875b06fc8
Added exclude-matchers support for template & matchers ( #2218 )
...
* Added exclude-matchers support for template & matchers
* Fixed panics due to typo
* Added support for only template ID + misc cleanup
2022-06-24 23:09:27 +05:30
9073b753ca
Added aes_gcm implementation for DSL function ( #2196 )
...
* Added aes_gcm implementation for DSL function
* Added integration test for dsl-functions.yaml
2022-06-23 16:16:24 +05:30
7f28c048a6
version update
2022-06-22 00:45:08 +05:30
0b351e83f3
Add self diagnostic functionality ( #2178 )
...
* Adding Self-Diagnostic
* adding comment
2022-06-22 00:40:10 +05:30
1047047790
chore(deps): bump github.com/go-rod/rod from 0.106.8 to 0.107.1 in /v2 ( #2114 )
...
* chore(deps): bump github.com/go-rod/rod from 0.106.8 to 0.107.1 in /v2
Bumps [github.com/go-rod/rod](https://github.com/go-rod/rod ) from 0.106.8 to 0.107.1.
- [Release notes](https://github.com/go-rod/rod/releases )
- [Commits](https://github.com/go-rod/rod/compare/v0.106.8...v0.107.1 )
---
updated-dependencies:
- dependency-name: github.com/go-rod/rod
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* fixing function call
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
2022-06-22 00:31:08 +05:30
112762f024
Adding http request validation at compile time ( #2193 )
...
* Adding http request validation at compile time
* misc update
Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-06-22 00:26:06 +05:30
50d21c0464
Fixed a issue with random invalid matches in DSL ( #2195 )
2022-06-21 21:58:43 +05:30
af4854f90d
output: fix WriteStoreDebugData file permissions ( #2187 )
...
Fixes #2180
2022-06-20 17:10:22 +05:30
644c951449
use formedURL instead reqURL in http request dump message ( #2174 )
2022-06-16 17:24:21 +05:30
b715a601e8
Adding nil operator check on clustering ( #2173 )
...
* add nil operator check
* move nil pointer check
2022-06-16 14:41:05 +05:30
a4cdba0691
Improving literals detection in expression engine ( #2148 )
...
* Improving literals detection in expression engine
* fixing lint errors
* re-add accidentally deleted test
2022-06-13 13:55:06 +05:30
e7591ec8b3
use request numbering as per template definition in req-condition ( #2135 )
...
* use original request number instead of current iteration in request-condition
* add previousEvent tracking back for request condition
Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-06-11 14:29:05 +05:30
e575f53be7
Empty string test can be improved ( #2115 )
...
It is not recommended to use len for empty string test.
A string can be tested for its emptiness either by treating it as a slice and calculating the length of the slice, or by treating it as a string and directly comparing the value. While both produce identical code when compiled, it makes more sense to treat a string as itself, than a slice, for the sake of comparison of values.
Examples
Bad practice
len(s) == 0
Recommended
s == ""
The recommended practice is considered more idiomatic in Go.
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2022-06-11 14:11:43 +05:30
5281d342c0
Merge pull request #2138 from mikerott/dev
...
protect against multiple unnecessary Init calls
2022-06-10 18:12:31 +05:30
22a8d72c65
protect against multiple unnecessary Init calls
2022-06-09 17:18:49 -05:00
9fd9892f49
test: extra DSL function test cases
2022-06-08 20:58:46 +03:00
04e7e053de
refactor: replace date, time, time_format, time_to_string DSL functions to date_time
2022-06-08 20:41:32 +03:00
ef20e0711b
refactor: replace date, time, time_format, time_to_string DSL functions to date_time
2022-06-08 20:33:55 +03:00
a10d58c6d2
refactor: rename concat_ws DSL function to join
2022-06-08 16:31:33 +03:00
145bdaabe5
refactor: extract duplicated hashing logic
2022-06-08 16:11:15 +03:00
6d8908c352
refactor: replace hmac_sha1 and hmac_sha256 DSL functions with hmac
2022-06-08 16:00:14 +03:00
9d37bd6c0c
Add two new DSL helper functions
...
hmac_sha1 and concat_ws (with seperator) this are helpful in
signing API requests.
2022-06-07 18:26:22 +03:00
f3514e9b92
Merge branch 'dev' into new_dsl_functions
2022-06-07 17:26:10 +03:00
7170cc2828
dsl matcher separate ignorable(No parameter) errors from others ( #2127 )
2022-06-07 18:23:07 +05:30
cc37382519
Adding Client TLS1.0 ( #2091 )
...
* Adding Client TLS1.0
* bumping fastdialer version
2022-06-04 17:45:16 +05:30
b79817e0a9
change dsl evaluate warning messages to error ( #2096 )
...
* change dsl evaluate warning messages to error
* add template-id to dsl match error logs
2022-06-03 13:41:36 +05:30
fa369b728e
ssl protocol with ms flag crash ( #2101 )
...
* crash with ssl protocol when used with ms flag fix
* added missing template info in case of failure
2022-06-03 13:32:45 +05:30
900addc43f
version update
2022-06-03 03:02:11 +05:30
aebd32b198
Add decimal to hexadecimal helper function ( #2076 )
...
* Add decimal to hexadecimal auxiliary functions
* Fixed unit test
* Modify the helper function name and check the unit test.
* dsl function update
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2022-06-03 02:17:35 +05:30
2b631e8e95
Add optional line number for file templates by default ( #1966 )
...
* Add optional line number for file templates by default
* updating docs
* misc flag update
Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-06-02 17:06:50 +05:30
4bcb94e4df
Increasing timeout and fixed error msg ( #2092 )
...
* Increasing timeout and fixed error msg
* increasing timeout
2022-06-02 00:00:18 +05:30
9f600d2829
keep the original extactor var for multiple results while preparing data for matchers ( #1948 )
2022-05-31 02:46:51 +05:30
be5f1a7623
Added redirected matched URL + stop-at-first-match for redirect chains ( #2050 )
...
* Added redirected matched URL + stop-at-first-match for redirect chains
* Pleasing go-linter
2022-05-30 15:19:09 +05:30
8723a1fd70
Added header as DSL part value ( #2052 )
...
* Added header as DSL part value
* Fixed failing DSL part test cases
2022-05-30 15:16:27 +05:30
34ed4e531a
Added hang monitor for goroutine dumping ( #1949 )
...
* Added hang monitor for goroutine dumping
* misc
* Made hang monitor optional with flag
* Added stack comparison for monitoring + misc
* Removed debug statements
* misc update
Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-05-30 14:41:24 +05:30
dd3b0a3cfc
Increasing buffer size on AND condition ( #2030 )
...
* Increasing buffer size on AND condition
* adding integration tests
* fixing tests data path
2022-05-30 13:45:28 +05:30
f53f360c74
Fixed websocket status-101 response reading for http ( #2081 )
2022-05-27 22:36:42 +05:30
16a05d0aa2
Adding CLI SNI support to unsafe http ( #2077 )
...
* Adding CLI SNI support to unsafe http
* adding http unsafe sni test
2022-05-27 21:53:07 +05:30
02eaf91e6a
Adding variables support for headless templates ( #2064 )
2022-05-27 21:31:56 +05:30
3648c47e35
Fixed template validation edge cases ( #2051 )
2022-05-25 11:26:05 +05:30
096f34e4e3
refactor: DSL date/time functions
2022-05-24 13:38:26 +03:00
346db4cf15
refactor: timetostring DSL to time_to_string
2022-05-24 13:11:55 +03:00
Ice3man
Sajad
Sami
Mzack9999
forgedhallpass
sandeep
James Turner
M. Ángel Jimeno
Myung-jong Kim
sandeep
51pwn
Dani Goland
xixijun
Mike Rheinheimer
dependabot[bot]
invist
anykno
tanimdiucse123
Mike Rheinheimer
M4rtin Hsu