Commit Graph

1324 Commits (9b7a105766789c1da3f3f32352158ed251264d24)

Author SHA1 Message Date
bauthard 2dfd245f54
Merge pull request #213 from Marmelatze/json-requests-dns
add json-requests for DNS
2020-08-13 17:41:42 +05:30
bauthard e22213f208
Merge pull request #219 from manuelbua/normalize-template-load-msg
Centralize template loaded info message, add output coloring
2020-08-10 11:55:06 +05:30
Manuel Bua 06eca0b2ea Fix minor issue where a dangling comma can cause any severity to match 2020-08-07 21:27:58 +02:00
Manuel Bua d752de530b Centralize template loaded info message, add output coloring 2020-08-07 14:30:57 +02:00
Florian Pfitzer 7a1aa770e9 add json-requests for DNS 2020-08-06 06:12:57 +02:00
bauthard 234d6245db
Merge pull request #208 from Marmelatze/fix-dns-result
fix no result found problem with and condition
2020-08-06 00:37:39 +05:30
Manuel Bua f339214f65 Merge branch 'master' into fix-180-support-template-exclusions 2020-08-04 21:43:06 +02:00
Florian Pfitzer 51412f679b fix no result found problem with and condition 2020-08-04 15:09:30 +02:00
Manuel Bua 65d89d6d87 Use "warning" log-level, prevent too many informative messages 2020-08-02 20:06:45 +02:00
Manuel Bua 136d1064f9 Merge branch 'master' into fix-191-filter-by-severity 2020-08-02 18:58:37 +02:00
Manuel Bua 801d738670 Merge branch 'master' into fix-180-support-template-exclusions 2020-08-02 18:58:30 +02:00
Manuel Bua da22532f81 Add support for severity filtering 2020-08-02 18:33:55 +02:00
Manuel Bua 4a355f0fc0 Add support for template exclusions 2020-08-02 15:48:10 +02:00
Manuel Bua c6df7fbd43 Refactor template resolution logic for reusability 2020-08-02 13:49:16 +02:00
Mzack9999 a0b11d4662 go modules update 2020-08-02 12:36:17 +02:00
bauthard a7fbf5c9e0
version update 2020-08-02 13:44:34 +05:30
Manuel Bua 87dc8bc650 Remove debug code 2020-08-01 23:01:28 +02:00
Manuel Bua a3c3875e38 Better wording in comments 2020-08-01 22:07:08 +02:00
Manuel Bua 63cfd354b9 Use global pipes for capturing, experimental unified rendering 2020-08-01 21:44:14 +02:00
Manuel Bua 29576f9ced Constrain rendering to a specific refresh rate 2020-08-01 15:07:04 +02:00
Manuel Bua 7088c9532e Merge branch 'master' into experimental-progressbar-live-results 2020-07-31 23:37:45 +02:00
Manuel Bua 02238c491b Use manual progressbar rendering to allow syncing with other output
Refactor into an interface to return a no-op impl, avoid the need to
conditionally invoke any of the methods.
2020-07-31 23:36:56 +02:00
bauthard 2685fdd1b9
Merge pull request #193 from projectdiscovery/bugfix-update-go-modules
updating gologger module
2020-07-31 22:29:26 +05:30
bauthard ad0630da19
Merge pull request #192 from projectdiscovery/169-bugfix
fixing relative path issue
2020-07-31 22:29:00 +05:30
Mzack9999 5d0e1b693e no-pbar to pbar 2020-07-31 18:46:23 +02:00
Mzack9999 97f4175bc4 updating gologger module 2020-07-31 18:16:55 +02:00
Manuel Bua 058c4ffbbf Experimental live results output with active progressbar
See 333809f3d6 for an early implementation of this.
2020-07-31 17:35:05 +02:00
Mzack9999 f4486ef89a fixing relative path issue 2020-07-31 17:13:51 +02:00
Manuel Bua 2ebc0cf83e Merge branch 'master' into fix-175-results-highlight 2020-07-29 21:57:44 +02:00
Mzack9999 5cc3d9fe14 corrected payload issue 2020-07-29 21:20:39 +02:00
Mzack9999 7edad4bb33 adding -no-pbar option 2020-07-29 20:07:13 +02:00
Mzack9999 d5d745e9b0 adding missing check to pointer 2020-07-29 19:50:54 +02:00
Manuel Bua 7d22f6cfb4 Merge branch 'master' into fix-175-results-highlight 2020-07-29 01:24:14 +02:00
Manuel Bua f8e52ced4e Add initial coloring support in output results 2020-07-29 00:43:05 +02:00
Mzack9999 db11acd7b2 version bump :) 2020-07-28 20:41:12 +02:00
Mzack9999 43a96b958a making progress bar respect silent flag 2020-07-28 20:39:07 +02:00
Manuel Bua 9d2539f856 Makes stdio capturing global
This dramatically lower chances of reaching max open files limit as it
works with two os.Pipe only, but it may be sub-optimal in some cases
such as with the `-debug` switch because there are no guarded writes
anymore when using `fmt.Fprintf` directly, such as when dumping request
or responses.
2020-07-27 21:35:24 +02:00
Manuel Bua 24c53628ea Capture additional DNS logging 2020-07-27 21:15:38 +02:00
Manuel Bua ba75e35a4b Merge branch 'master' into enumeration-progressbar 2020-07-27 20:45:47 +02:00
Manuel Bua 6209b25d9a Ensure an empty progressbar is aborted 2020-07-27 20:39:13 +02:00
Manuel Bua 53b280cf32 Ensure workflows can be run alone as well 2020-07-27 20:38:48 +02:00
Mzack9999 a7c1f43b41 adding speed to workflows 2020-07-27 19:36:40 +02:00
Manuel Bua 8718d52546 Ensure input URLs and requests are present before tracking any progress 2020-07-27 18:47:23 +02:00
Manuel Bua 3916983bdb Merge branch 'master' into enumeration-progressbar 2020-07-27 00:25:36 +02:00
Manuel Bua 06cffee6aa Add support for workflows 2020-07-27 00:00:34 +02:00
Mzack9999 f45e01eda6 multi-matcher correction 2020-07-26 22:07:03 +02:00
Mzack9999 c659d432b3 hacky silent output within workflow 2020-07-26 21:30:03 +02:00
Mzack9999 573088450f adding missing input flag 2020-07-26 21:17:42 +02:00
Mzack9999 8c437fbd40 adding workflow's same directory path search for template variables 2020-07-26 20:14:05 +02:00
Manuel Bua 660c8420c1 Remove unused code 2020-07-26 16:57:19 +02:00
Manuel Bua baa1715c44 Better input sanitization, skip empty lines 2020-07-26 16:56:53 +02:00
Manuel Bua 4d8131c8d8 Add support for DNS requests 2020-07-26 16:36:01 +02:00
Manuel Bua ea6229300d Merge branch 'master' into enumeration-progressbar 2020-07-26 15:46:21 +02:00
Manuel Bua 3cc79c2c49 Properly close file after use 2020-07-26 15:35:26 +02:00
Manuel Bua 32e20b13e6 Use single progressbar, tweak styling 2020-07-26 15:31:31 +02:00
Manuel Bua b33a2b6d16 Update outdated comment 2020-07-25 23:22:09 +02:00
Manuel Bua a9560336f4 Refactoring and unused code removal 2020-07-25 23:13:58 +02:00
Manuel Bua 48cf65b563 Automatically de-duplicate supplied user input 2020-07-25 23:02:24 +02:00
Mzack9999 5eb92abb80
Merge pull request #174 from projectdiscovery/bugfix-engine-rework
engine parallelism fix
2020-07-25 22:48:48 +02:00
Manuel Bua c967997a5a Merge branch 'bugfix-engine-rework' into enumeration-progressbar-next 2020-07-25 22:47:32 +02:00
Mzack9999 731241c90e fsm in-memory deduplication 2020-07-25 22:25:21 +02:00
Manuel Bua 24341bf34c Merge branch 'bugfix-engine-rework' into enumeration-progressbar-next 2020-07-25 22:18:28 +02:00
Manuel Bua 65d7246b2f Use custom formatter for percentage, ensure fixed string size is used 2020-07-25 22:15:45 +02:00
Mzack9999 3a8ee75b93 wip - output 2020-07-25 21:44:43 +02:00
Mzack9999 4cc9ad9100 wip - output 2020-07-25 21:15:28 +02:00
Manuel Bua d1c8527f1f Merge branch 'bugfix-engine-rework' into enumeration-progressbar-next 2020-07-25 21:05:47 +02:00
Manuel Bua d8e69cacf2 Pad bar name with spaces (approx. size due to escape codes) 2020-07-25 21:03:38 +02:00
Mzack9999 c05cf9f8c6 adding missing result bool 2020-07-25 20:45:38 +02:00
Manuel Bua bb24061628 Merge branch 'bugfix-engine-rework' into enumeration-progressbar-next 2020-07-25 20:33:01 +02:00
Manuel Bua 436e7223d8 Update output coloring, set a maximum length for template names 2020-07-25 20:29:12 +02:00
Mzack9999 5fa309cb82 2nd race bugfix 2020-07-25 20:11:46 +02:00
Manuel Bua dad381e66e Cleanup pluralization 2020-07-25 19:58:17 +02:00
Manuel Bua 88e683c0b5 Always use a global progress bar, protect input file reading 2020-07-25 15:16:41 +02:00
Manuel Bua 3bfdd0c381 Increment progress instead of altering totals, wrap more std streams 2020-07-25 00:46:18 +02:00
Manuel Bua 8aecbeb821 Experimental support for new parallelism 2020-07-24 22:30:15 +02:00
Wyatt Dahlenburg 6362fad61f Updated runner to ensure relative path was respected in configuration file when using wildcards 2020-07-24 15:29:08 -05:00
Manuel Bua c7d234fe5a Merge branch 'bugfix-engine-rework' into enumeration-progressbar-next 2020-07-24 20:29:04 +02:00
Mzack9999 ab2f96c9e9 more experimental parallelism 2020-07-24 18:12:16 +02:00
Mzack9999 ffdaffd9a7 wip - code refactor 2020-07-24 13:37:01 +02:00
Mzack9999 9f1ac66032 wip - new engine parallelism fix 2020-07-23 23:28:34 +02:00
Manuel Bua 362fee27a4 Initial update to latest updates 2020-07-23 20:19:19 +02:00
Wyatt Dahlenburg 959711edb0 Support template wildcarding 2020-07-23 13:06:58 -05:00
Manuel Bua 04875a55dd Merge branch 'master' into enumeration-progressbar-next
# Conflicts:
#	v2/internal/runner/runner.go
#	v2/pkg/executor/executer_http.go
2020-07-22 22:28:26 +02:00
Florian Pfitzer b3fb45d381 json-request option for request/response output in JSON matches 2020-07-22 11:17:58 +02:00
Mzack9999 eee452a8cd progress tracker fix 2020-07-21 20:51:58 +02:00
Mzack9999 b4aabf4b34 fixed return point 2020-07-20 21:23:04 +02:00
Mzack9999 5f0edcd8aa experimental new syntax 2020-07-20 01:37:07 +02:00
Mzack9999 148c0dec89 updated go modules 2020-07-20 00:50:25 +02:00
Mzack9999 c2433fdb34 merged master 2020-07-20 00:47:04 +02:00
Mzack9999 53363ec5fc fixed port rerouting in raw requests 2020-07-20 00:41:31 +02:00
Mzack9999 2299a0163a some code refactor 2020-07-19 19:25:05 +02:00
Manuel Bua 838f844837 Better naming 2020-07-19 14:24:43 +02:00
Manuel Bua 0983e8b9fa Ensure path is resolved before using it 2020-07-19 14:04:49 +02:00
Mzack9999 fc355c79ca improved dynamic bindings 2020-07-19 03:14:19 +02:00
Mzack9999 57a496203f code refactor 2020-07-18 21:42:23 +02:00
Manuel Bua 97069afb3c Merge branch 'master' into fix-147-support-multiple-input-templates 2020-07-18 21:27:39 +02:00
Manuel Bua c161a385b0 Do not report per-template empty results 2020-07-18 20:32:00 +02:00
Manuel Bua 0833e21a17 Avoid adding empty array, ensure warning is shown to the user. 2020-07-18 18:55:52 +02:00
Manuel Bua 29cd509809 Merge branch 'master' into fix-147-support-multiple-input-templates 2020-07-17 18:21:40 +02:00
Mzack9999 4b95842723 b 2020-07-17 16:57:53 +02:00
Mzack9999 c195bdb2f8 allow non canonical header name 2020-07-17 16:04:13 +02:00
Mzack9999 c346087926 limited support for same template extraction+reuse 2020-07-17 15:07:34 +02:00
Mzack9999 66713434f8 added in-workflow cookie reuse 2020-07-16 16:32:42 +02:00
Mzack9999 a256a56993 adding in-template cookie reuse 2020-07-16 16:16:26 +02:00
Mzack9999 a4ac439790 typo 2020-07-16 14:27:52 +02:00
Mzack9999 a75ea4644b adding external headers support within workflow 2020-07-16 14:22:59 +02:00
Mzack9999 a23031a2e6 using native go for cookies 2020-07-16 12:58:56 +02:00
Mzack9999 037aa356a6 small correction 2020-07-16 12:47:38 +02:00
Mzack9999 ef8146cfd5 typo global correction 2020-07-16 10:57:28 +02:00
Mzack9999 075509f91f finalized + corrected typo 2020-07-16 10:32:00 +02:00
Mzack9999 98dad65652 prototyping 2020-07-15 00:47:01 +02:00
Mzack9999 34224c537b
Merge pull request #140 from ankh2054/master
Add description to Template files
2020-07-14 23:18:38 +02:00
Mzack9999 457cfe91ba
Merge pull request #142 from projectdiscovery/feature-workflow-improvement
improved workflow logic
2020-07-14 23:16:15 +02:00
bauthard 73832708fd
Update banner.go 2020-07-14 12:49:48 +05:30
Mzack9999 eb03174a94 workflow now returns last matched payloads value 2020-07-14 03:54:57 +02:00
Mzack9999 f10ebeb070 removed useless assignment 2020-07-14 03:28:18 +02:00
Mzack9999 120637b31f adding array return type 2020-07-14 03:26:21 +02:00
Manuel Bua f376a7d9cd Move no-results logic as Workflows will not return any 2020-07-14 00:10:08 +02:00
Manuel Bua 9d7303549b Refactor enumeration driver and streamline input processing 2020-07-14 00:04:19 +02:00
Manuel Bua 54ada7735d Add generic multiStringFlag option flag, update usage string 2020-07-14 00:01:46 +02:00
Manuel Bua e9d27edf88 Fix #148
Include template name in error string when walking a template list.
2020-07-13 19:55:16 +02:00
bauthard 661c813fdc
Update banner.go 2020-07-13 15:55:41 +05:30
Mzack9999 6fa236e205 expanded variable type support 2020-07-13 03:30:07 +02:00
Manuel Bua 987e9ccf38 Merge branch 'master' into enumeration-progressbar 2020-07-12 19:55:36 +02:00
Manuel Bua d0174c047c Respect color output user flag, remove unused Progress WaitGroup 2020-07-12 18:09:29 +02:00
Ice3man d72b57e857
Merge pull request #129 from dwisiswant0/improves-raw-requests
Improvements HTTP RAW Requests
2020-07-12 06:39:43 -07:00
Manuel Bua e877d613e4 More clear comment wording 2020-07-11 23:52:45 +02:00
Manuel Bua de82c1617e Remove unused code 2020-07-11 23:50:59 +02:00
Manuel Bua 97901f36b4 Refactor code, add meaningful comments 2020-07-11 22:57:44 +02:00
Mzack9999 2dee3507c0 adding script external variables support 2020-07-11 03:20:27 +02:00
Manuel Bua 002daadf46 Try avoid using time.After when reading captured output. 2020-07-10 23:42:49 +02:00
Mzack9999 44e726515c Closes #144 2020-07-10 22:29:49 +02:00
Manuel Bua 2fe3d354c3 Group url/host-bound log messages to avoid segmentation in the output.
This is the simple and memory-friendly way. The alternative would be to
build a map[host]strings.Builder to allow for seamless per-host output
construction and output, but memory usage would probably be higher even
with a pool of builders.
2020-07-10 22:14:39 +02:00
Mzack9999 b4bf6cf4ac improved workflow logic 2020-07-10 09:04:38 +02:00
Manuel Bua e5949c8eba Precompute total number of HTTP requests when using multiple templates 2020-07-09 21:21:37 +02:00
Manuel Bua 81eed093d2 Minor refactoring, rename mutex 2020-07-09 21:20:00 +02:00
Manuel Bua cbb8b11784 Use atomic operations to alter and track the progress 2020-07-09 21:18:06 +02:00
Manuel Bua 0ff138a477 Do not use mpb.Progress for logging
This will cause sync issues with very fast output and will defeat the
purpose of logging.

Instead, buffer both stdout/stderr and show their output at the end.
2020-07-09 20:57:24 +02:00
Manuel Bua 6c43aab488 Try use multiple mpb.Progress for distinct stdout/stderr output 2020-07-08 23:13:53 +02:00
Manuel Bua 333809f3d6 Safer stdio capturing 2020-07-08 23:13:05 +02:00
Charles Holtzkampf f2f0a1726b
Update workflows.go 2020-07-08 20:01:21 +01:00
Charles Holtzkampf 73068f5fb2
Update output_http.go 2020-07-08 19:56:18 +01:00
Charles Holtzkampf 9476b72b2e
Update output_dns.go 2020-07-08 19:55:38 +01:00
Charles Holtzkampf 2ab7b0b8f8
Update http_utils.go 2020-07-08 19:54:36 +01:00
Charles Holtzkampf 5b60336883
Add description field to template.yaml.
To describe the nature of the test being performed by template.
2020-07-08 19:21:08 +01:00
Manuel Bua 96f1f29328 Merge branch 'master' into enumeration-progressbar 2020-07-08 17:33:44 +02:00
Ice3man543 d6fb6bda51 Fixed update template issue 2020-07-08 17:25:29 +05:30
Manuel Bua df501136a6 Abort single request 2020-07-07 23:11:33 +02:00
Manuel Bua 32db612f62 Merge branch 'master' into enumeration-progressbar 2020-07-07 22:51:40 +02:00
Manuel Bua bd274cf1d6 Remove unused code 2020-07-07 22:42:34 +02:00
Manuel Bua 9e1b39549f Minor refactoring, track count of aborted requests for better progress 2020-07-07 22:39:43 +02:00
Ice3man543 299a6d5194 Always check when update-templates passed 2020-07-07 18:33:35 +05:30
Ice3man543 5447387e71 Added feature to check for updates daily 2020-07-07 18:24:00 +05:30
bauthard 63b020f693
Update banner.go 2020-07-07 17:53:37 +05:30
Ice3man543 8f38faa579 Fixed an issue with requests 2020-07-07 17:48:25 +05:30
bauthard d830294168
Update banner.go 2020-07-07 11:23:58 +05:30
Mzack9999 606c46ba29
Merge pull request #132 from projectdiscovery/dsl-fix
DSL Fix
2020-07-06 20:51:45 +02:00
Mzack9999 e25ee652c0 Merge branch 'fix-default-matchers-condition' of github.com:manuelbua/nuclei into manuelbua-fix-default-matchers-condition 2020-07-06 20:48:31 +02:00
Mzack9999 d60a59d2c6 removed problematic early optimization 2020-07-06 20:11:19 +02:00
bauthard 257d9685af Fixed error with auto-updates 2020-07-06 12:30:02 +05:30
Manuel Bua 6f894b718a Abort on error, bar will be properly recreated 2020-07-06 00:13:13 +02:00
Manuel Bua d7b440a4ac Disambiguate error message 2020-07-05 23:52:47 +02:00
Manuel Bua d03fbd1932 Encapsulate behavior and clean up 2020-07-05 23:38:58 +02:00
Manuel Bua 6afd949060 Clean up 2020-07-05 23:21:20 +02:00
Manuel Bua 20758a14a5 Use shortcut method 2020-07-05 22:48:05 +02:00
Manuel Bua 3a4d7ba3e1 No need to abort the only running bar anymore 2020-07-05 22:47:07 +02:00
Manuel Bua d903115634 Use simple average instead of ewma-based, reduce cpu/mem footprint 2020-07-05 22:46:34 +02:00
Manuel Bua 02bf957c13 Better handling of the timeout context 2020-07-05 21:39:04 +02:00
Manuel Bua 683de150bd Remove any completed bar so far 2020-07-05 20:11:53 +02:00
Manuel Bua e59ac01c65 Experimental single progress bar 2020-07-05 17:22:21 +02:00
Manuel Bua 9afd9bc4c2 Refactoring to compute workload in advance 2020-07-05 17:17:04 +02:00
Manuel Bua 6d68b09863 Initial progress bar implementation 2020-07-04 23:00:11 +02:00
dw1 fc434f13e0 🪲 Bug fixes #128 2020-07-04 14:34:41 +07:00
Ice3man543 458dbc1b38 v2 update 2020-07-01 16:17:24 +05:30