Commit Graph

2383 Commits (57b2ec7ade9334c03b4eb19bf135843e1a405ead)

Author SHA1 Message Date
Mzack9999 a7c1f43b41 adding speed to workflows 2020-07-27 19:36:40 +02:00
Manuel Bua 8718d52546 Ensure input URLs and requests are present before tracking any progress 2020-07-27 18:47:23 +02:00
Manuel Bua 3916983bdb Merge branch 'master' into enumeration-progressbar 2020-07-27 00:25:36 +02:00
Manuel Bua 06cffee6aa Add support for workflows 2020-07-27 00:00:34 +02:00
Mzack9999 f45e01eda6 multi-matcher correction 2020-07-26 22:07:03 +02:00
Mzack9999 c659d432b3 hacky silent output within workflow 2020-07-26 21:30:03 +02:00
Mzack9999 573088450f adding missing input flag 2020-07-26 21:17:42 +02:00
Mzack9999 8c437fbd40 adding workflow's same directory path search for template variables 2020-07-26 20:14:05 +02:00
Manuel Bua 660c8420c1 Remove unused code 2020-07-26 16:57:19 +02:00
Manuel Bua baa1715c44 Better input sanitization, skip empty lines 2020-07-26 16:56:53 +02:00
Manuel Bua 4d8131c8d8 Add support for DNS requests 2020-07-26 16:36:01 +02:00
Manuel Bua ea6229300d Merge branch 'master' into enumeration-progressbar 2020-07-26 15:46:21 +02:00
Manuel Bua 3cc79c2c49 Properly close file after use 2020-07-26 15:35:26 +02:00
Manuel Bua 32e20b13e6 Use single progressbar, tweak styling 2020-07-26 15:31:31 +02:00
Manuel Bua b33a2b6d16 Update outdated comment 2020-07-25 23:22:09 +02:00
Manuel Bua a9560336f4 Refactoring and unused code removal 2020-07-25 23:13:58 +02:00
Manuel Bua 48cf65b563 Automatically de-duplicate supplied user input 2020-07-25 23:02:24 +02:00
Mzack9999 5eb92abb80
Merge pull request #174 from projectdiscovery/bugfix-engine-rework
engine parallelism fix
2020-07-25 22:48:48 +02:00
Manuel Bua c967997a5a Merge branch 'bugfix-engine-rework' into enumeration-progressbar-next 2020-07-25 22:47:32 +02:00
Mzack9999 731241c90e fsm in-memory deduplication 2020-07-25 22:25:21 +02:00
Manuel Bua 24341bf34c Merge branch 'bugfix-engine-rework' into enumeration-progressbar-next 2020-07-25 22:18:28 +02:00
Manuel Bua 65d7246b2f Use custom formatter for percentage, ensure fixed string size is used 2020-07-25 22:15:45 +02:00
Mzack9999 3a8ee75b93 wip - output 2020-07-25 21:44:43 +02:00
Mzack9999 4cc9ad9100 wip - output 2020-07-25 21:15:28 +02:00
Manuel Bua d1c8527f1f Merge branch 'bugfix-engine-rework' into enumeration-progressbar-next 2020-07-25 21:05:47 +02:00
Manuel Bua d8e69cacf2 Pad bar name with spaces (approx. size due to escape codes) 2020-07-25 21:03:38 +02:00
Mzack9999 c05cf9f8c6 adding missing result bool 2020-07-25 20:45:38 +02:00
Manuel Bua bb24061628 Merge branch 'bugfix-engine-rework' into enumeration-progressbar-next 2020-07-25 20:33:01 +02:00
Manuel Bua 436e7223d8 Update output coloring, set a maximum length for template names 2020-07-25 20:29:12 +02:00
Mzack9999 5fa309cb82 2nd race bugfix 2020-07-25 20:11:46 +02:00
Manuel Bua dad381e66e Cleanup pluralization 2020-07-25 19:58:17 +02:00
Manuel Bua 88e683c0b5 Always use a global progress bar, protect input file reading 2020-07-25 15:16:41 +02:00
Manuel Bua 3bfdd0c381 Increment progress instead of altering totals, wrap more std streams 2020-07-25 00:46:18 +02:00
Manuel Bua 8aecbeb821 Experimental support for new parallelism 2020-07-24 22:30:15 +02:00
Wyatt Dahlenburg 6362fad61f Updated runner to ensure relative path was respected in configuration file when using wildcards 2020-07-24 15:29:08 -05:00
Manuel Bua c7d234fe5a Merge branch 'bugfix-engine-rework' into enumeration-progressbar-next 2020-07-24 20:29:04 +02:00
Mzack9999 ab2f96c9e9 more experimental parallelism 2020-07-24 18:12:16 +02:00
Mzack9999 ffdaffd9a7 wip - code refactor 2020-07-24 13:37:01 +02:00
Mzack9999 9f1ac66032 wip - new engine parallelism fix 2020-07-23 23:28:34 +02:00
Manuel Bua 362fee27a4 Initial update to latest updates 2020-07-23 20:19:19 +02:00
Wyatt Dahlenburg 959711edb0 Support template wildcarding 2020-07-23 13:06:58 -05:00
Manuel Bua 04875a55dd Merge branch 'master' into enumeration-progressbar-next
# Conflicts:
#	v2/internal/runner/runner.go
#	v2/pkg/executor/executer_http.go
2020-07-22 22:28:26 +02:00
Florian Pfitzer b3fb45d381 json-request option for request/response output in JSON matches 2020-07-22 11:17:58 +02:00
Mzack9999 eee452a8cd progress tracker fix 2020-07-21 20:51:58 +02:00
Mzack9999 b4aabf4b34 fixed return point 2020-07-20 21:23:04 +02:00
Mzack9999 5f0edcd8aa experimental new syntax 2020-07-20 01:37:07 +02:00
Mzack9999 148c0dec89 updated go modules 2020-07-20 00:50:25 +02:00
Mzack9999 c2433fdb34 merged master 2020-07-20 00:47:04 +02:00
Mzack9999 53363ec5fc fixed port rerouting in raw requests 2020-07-20 00:41:31 +02:00
Mzack9999 2299a0163a some code refactor 2020-07-19 19:25:05 +02:00
Manuel Bua 838f844837 Better naming 2020-07-19 14:24:43 +02:00
Manuel Bua 0983e8b9fa Ensure path is resolved before using it 2020-07-19 14:04:49 +02:00
Mzack9999 fc355c79ca improved dynamic bindings 2020-07-19 03:14:19 +02:00
Mzack9999 57a496203f code refactor 2020-07-18 21:42:23 +02:00
Manuel Bua 97069afb3c Merge branch 'master' into fix-147-support-multiple-input-templates 2020-07-18 21:27:39 +02:00
Manuel Bua c161a385b0 Do not report per-template empty results 2020-07-18 20:32:00 +02:00
Manuel Bua 0833e21a17 Avoid adding empty array, ensure warning is shown to the user. 2020-07-18 18:55:52 +02:00
Manuel Bua 29cd509809 Merge branch 'master' into fix-147-support-multiple-input-templates 2020-07-17 18:21:40 +02:00
Mzack9999 4b95842723 b 2020-07-17 16:57:53 +02:00
Mzack9999 c195bdb2f8 allow non canonical header name 2020-07-17 16:04:13 +02:00
Mzack9999 c346087926 limited support for same template extraction+reuse 2020-07-17 15:07:34 +02:00
Mzack9999 66713434f8 added in-workflow cookie reuse 2020-07-16 16:32:42 +02:00
Mzack9999 a256a56993 adding in-template cookie reuse 2020-07-16 16:16:26 +02:00
Mzack9999 a4ac439790 typo 2020-07-16 14:27:52 +02:00
Mzack9999 a75ea4644b adding external headers support within workflow 2020-07-16 14:22:59 +02:00
Mzack9999 a23031a2e6 using native go for cookies 2020-07-16 12:58:56 +02:00
Mzack9999 037aa356a6 small correction 2020-07-16 12:47:38 +02:00
Mzack9999 ef8146cfd5 typo global correction 2020-07-16 10:57:28 +02:00
Mzack9999 075509f91f finalized + corrected typo 2020-07-16 10:32:00 +02:00
Mzack9999 98dad65652 prototyping 2020-07-15 00:47:01 +02:00
Mzack9999 34224c537b
Merge pull request #140 from ankh2054/master
Add description to Template files
2020-07-14 23:18:38 +02:00
Mzack9999 457cfe91ba
Merge pull request #142 from projectdiscovery/feature-workflow-improvement
improved workflow logic
2020-07-14 23:16:15 +02:00
bauthard 73832708fd
Update banner.go 2020-07-14 12:49:48 +05:30
Mzack9999 eb03174a94 workflow now returns last matched payloads value 2020-07-14 03:54:57 +02:00
Mzack9999 f10ebeb070 removed useless assignment 2020-07-14 03:28:18 +02:00
Mzack9999 120637b31f adding array return type 2020-07-14 03:26:21 +02:00
Manuel Bua f376a7d9cd Move no-results logic as Workflows will not return any 2020-07-14 00:10:08 +02:00
Manuel Bua 9d7303549b Refactor enumeration driver and streamline input processing 2020-07-14 00:04:19 +02:00
Manuel Bua 54ada7735d Add generic multiStringFlag option flag, update usage string 2020-07-14 00:01:46 +02:00
Manuel Bua e9d27edf88 Fix #148
Include template name in error string when walking a template list.
2020-07-13 19:55:16 +02:00
bauthard 661c813fdc
Update banner.go 2020-07-13 15:55:41 +05:30
Mzack9999 6fa236e205 expanded variable type support 2020-07-13 03:30:07 +02:00
Manuel Bua 987e9ccf38 Merge branch 'master' into enumeration-progressbar 2020-07-12 19:55:36 +02:00
Manuel Bua d0174c047c Respect color output user flag, remove unused Progress WaitGroup 2020-07-12 18:09:29 +02:00
Ice3man d72b57e857
Merge pull request #129 from dwisiswant0/improves-raw-requests
Improvements HTTP RAW Requests
2020-07-12 06:39:43 -07:00
Manuel Bua e877d613e4 More clear comment wording 2020-07-11 23:52:45 +02:00
Manuel Bua de82c1617e Remove unused code 2020-07-11 23:50:59 +02:00
Manuel Bua 97901f36b4 Refactor code, add meaningful comments 2020-07-11 22:57:44 +02:00
Mzack9999 2dee3507c0 adding script external variables support 2020-07-11 03:20:27 +02:00
Manuel Bua 002daadf46 Try avoid using time.After when reading captured output. 2020-07-10 23:42:49 +02:00
Mzack9999 44e726515c Closes #144 2020-07-10 22:29:49 +02:00
Manuel Bua 2fe3d354c3 Group url/host-bound log messages to avoid segmentation in the output.
This is the simple and memory-friendly way. The alternative would be to
build a map[host]strings.Builder to allow for seamless per-host output
construction and output, but memory usage would probably be higher even
with a pool of builders.
2020-07-10 22:14:39 +02:00
Mzack9999 b4bf6cf4ac improved workflow logic 2020-07-10 09:04:38 +02:00
Manuel Bua e5949c8eba Precompute total number of HTTP requests when using multiple templates 2020-07-09 21:21:37 +02:00
Manuel Bua 81eed093d2 Minor refactoring, rename mutex 2020-07-09 21:20:00 +02:00
Manuel Bua cbb8b11784 Use atomic operations to alter and track the progress 2020-07-09 21:18:06 +02:00
Manuel Bua 0ff138a477 Do not use mpb.Progress for logging
This will cause sync issues with very fast output and will defeat the
purpose of logging.

Instead, buffer both stdout/stderr and show their output at the end.
2020-07-09 20:57:24 +02:00
Manuel Bua 6c43aab488 Try use multiple mpb.Progress for distinct stdout/stderr output 2020-07-08 23:13:53 +02:00
Manuel Bua 333809f3d6 Safer stdio capturing 2020-07-08 23:13:05 +02:00
Charles Holtzkampf f2f0a1726b
Update workflows.go 2020-07-08 20:01:21 +01:00
Charles Holtzkampf 73068f5fb2
Update output_http.go 2020-07-08 19:56:18 +01:00
Charles Holtzkampf 9476b72b2e
Update output_dns.go 2020-07-08 19:55:38 +01:00
Charles Holtzkampf 2ab7b0b8f8
Update http_utils.go 2020-07-08 19:54:36 +01:00
Charles Holtzkampf 5b60336883
Add description field to template.yaml.
To describe the nature of the test being performed by template.
2020-07-08 19:21:08 +01:00
Manuel Bua 96f1f29328 Merge branch 'master' into enumeration-progressbar 2020-07-08 17:33:44 +02:00
Ice3man543 d6fb6bda51 Fixed update template issue 2020-07-08 17:25:29 +05:30
Manuel Bua df501136a6 Abort single request 2020-07-07 23:11:33 +02:00
Manuel Bua 32db612f62 Merge branch 'master' into enumeration-progressbar 2020-07-07 22:51:40 +02:00
Manuel Bua bd274cf1d6 Remove unused code 2020-07-07 22:42:34 +02:00
Manuel Bua 9e1b39549f Minor refactoring, track count of aborted requests for better progress 2020-07-07 22:39:43 +02:00
Ice3man543 299a6d5194 Always check when update-templates passed 2020-07-07 18:33:35 +05:30
Ice3man543 5447387e71 Added feature to check for updates daily 2020-07-07 18:24:00 +05:30
bauthard 63b020f693
Update banner.go 2020-07-07 17:53:37 +05:30
Ice3man543 8f38faa579 Fixed an issue with requests 2020-07-07 17:48:25 +05:30
bauthard d830294168
Update banner.go 2020-07-07 11:23:58 +05:30
Mzack9999 606c46ba29
Merge pull request #132 from projectdiscovery/dsl-fix
DSL Fix
2020-07-06 20:51:45 +02:00
Mzack9999 e25ee652c0 Merge branch 'fix-default-matchers-condition' of github.com:manuelbua/nuclei into manuelbua-fix-default-matchers-condition 2020-07-06 20:48:31 +02:00
Mzack9999 d60a59d2c6 removed problematic early optimization 2020-07-06 20:11:19 +02:00
bauthard 257d9685af Fixed error with auto-updates 2020-07-06 12:30:02 +05:30
Manuel Bua 6f894b718a Abort on error, bar will be properly recreated 2020-07-06 00:13:13 +02:00
Manuel Bua d7b440a4ac Disambiguate error message 2020-07-05 23:52:47 +02:00
Manuel Bua d03fbd1932 Encapsulate behavior and clean up 2020-07-05 23:38:58 +02:00
Manuel Bua 6afd949060 Clean up 2020-07-05 23:21:20 +02:00
Manuel Bua 20758a14a5 Use shortcut method 2020-07-05 22:48:05 +02:00
Manuel Bua 3a4d7ba3e1 No need to abort the only running bar anymore 2020-07-05 22:47:07 +02:00
Manuel Bua d903115634 Use simple average instead of ewma-based, reduce cpu/mem footprint 2020-07-05 22:46:34 +02:00
Manuel Bua 02bf957c13 Better handling of the timeout context 2020-07-05 21:39:04 +02:00
Manuel Bua 683de150bd Remove any completed bar so far 2020-07-05 20:11:53 +02:00
Manuel Bua e59ac01c65 Experimental single progress bar 2020-07-05 17:22:21 +02:00
Manuel Bua 9afd9bc4c2 Refactoring to compute workload in advance 2020-07-05 17:17:04 +02:00
Manuel Bua 6d68b09863 Initial progress bar implementation 2020-07-04 23:00:11 +02:00
dw1 fc434f13e0 🪲 Bug fixes #128 2020-07-04 14:34:41 +07:00
Ice3man543 458dbc1b38 v2 update 2020-07-01 16:17:24 +05:30