infosecn1nja
/
HELK
mirror of https://github.com/infosecn1nja/HELK.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
329 Commits
4 Branches
12 Tags
262 MiB
Commit Graph

329 Commits (42b71c83d9eff7c2e0a990198d572476a8ed85e2)

Author SHA1 Message Date
neu5ron 42b71c83d9 add cluster settings 2019-05-20 18:58:54 -04:00
neu5ron af774feaac a little bit more mem for small builds 2019-05-20 17:45:16 -04:00
neu5ron 97c9c23c5b a little bit more mem for small builds 2019-05-20 17:41:44 -04:00
neu5ron a9fc554625 use global ordinals on high cardinality fields 2019-05-20 17:25:36 -04:00
neu5ron ce483b4b83 more affective way to remove nested user field from under winlog. 2019-05-20 16:33:57 -04:00
Roberto Rodriguez 9ecbd823bf
Merge pull request #250 from Cyb3rWard0g/winlogbeat7 
Winlogbeat7
 2019-05-20 10:52:29 -07:00
Roberto Rodriguez 513227fc38 Testing ELK 7.0.1 
- Updated Spark to 2.4.3
- Updated Docker compose files
- Updating Elastalert
 2019-05-17 14:51:56 -04:00
neu5ron 19d1ddfa6d optional remove the nest if need be, just uncomment the line 2019-05-17 03:25:31 -04:00
neu5ron 5a45d4c7df make a bit more efficient by moving the add field into the main purposed mutate 2019-05-17 03:17:40 -04:00
neu5ron a759ce1342 support for Winlogbeat 7 and keep backwards compatibility for Winlogbeat 6 2019-05-17 03:16:54 -04:00
neu5ron b03e9432de Merge branch 'dev' of https://github.com/Cyb3rWard0g/HELK into winlogbeat7 2019-05-16 13:40:21 -04:00
Roberto Rodriguez 71777555d8 Towards ELK 7.0.1 2019-05-14 11:05:55 -04:00
neu5ron b3febc8452 Merge branch 'master' of https://github.com/Cyb3rWard0g/HELK into murmur3-to-sha1 2019-05-06 12:18:51 -04:00
neu5ron c5fb59d515 switch from murmur3 to sha1, reference https://github.com/Cyb3rWard0g/HELK/issues/231 2019-04-17 17:45:33 -04:00
neu5ron c18aac2f51 switch from murmur3 to sha1, reference https://github.com/Cyb3rWard0g/HELK/issues/231 2019-04-17 15:40:09 -04:00
Nate Guagenti 489adb9c57
Update winlogbeat.yml 
winlogbeat 7.x field name changes will require some pipeline rewrites. propose users to use version 6.x of winlogbeat
 2019-04-13 22:44:06 -04:00
Roberto Rodriguez 2ef0bd0bed Update kibana-setup.sh 
fix https://github.com/Cyb3rWard0g/HELK/issues/228
 2019-04-06 19:40:18 -07:00
Roberto Rodriguez bb1f4ce8ca
Merge pull request #227 from aarju/master 
Adding incident Response Dashboards for investigating alerts
 2019-04-06 21:33:19 -04:00
Aaron Jewitt bf16cfb54d Added Incident Response investigation dashboards 2019-04-06 20:24:49 +02:00
Roberto Rodriguez 2b30994493 Updated a few configs 2019-04-06 13:21:29 -04:00
Aaron Jewitt b669cd5fa8
updated DFIR_Dashboards.json 
attempting to format the json for kibana API injest
 2019-04-03 22:50:27 +02:00
Aaron Jewitt 38c0936ad6
Update DFIR_Dashboards.json 2019-04-01 22:14:26 +02:00
aaron 57e9008f66 Added the DFIR_Dashboards.json file 2019-03-28 13:10:36 -07:00
Nate Guagenti b331afdfb8 Update 0099-all-fingerprint-hash-filter.conf 2019-03-23 10:44:54 -04:00
Nate Guagenti 91f761fee3 Update 0099-all-fingerprint-hash-filter.conf 2019-03-23 10:44:54 -04:00
Nate Guagenti 9ed4539a53 Update 0099-all-fingerprint-hash-filter.conf 2019-03-23 10:44:54 -04:00
Nate Guagenti b268b38c0e Update 0099-all-fingerprint-hash-filter.conf 
better fingerprint-hashing for deduplication.
more specific for both winlogbeat and nxlog
 2019-03-23 10:44:54 -04:00
Roberto Rodriguez 98e32e2e87 Resources- Images 2019-03-16 14:30:12 -04:00
Roberto Rodriguez e819329f7a [HOT FIX] Mainly Jupyter and Logstash Updates 
HELK-JUPYTER
+ Miniconda3 to handle python packages
+ Python 3.7
+ Container not running as root
+ new entrypoint and cmd scripts
+ postgres not running as root and under the same container
+ Spark Jar and Python dependencies provided offline (not downloading from maven directly - Sometimes this fails)
+ Jupyter PySpark kernel using conda to run ipykernel module
+ PYSPARK_PYTHON Python 3.7

HELK-LOGSTASH
+ Fix https://github.com/Cyb3rWard0g/HELK/issues/217
 2019-03-11 09:00:54 -04:00
Roberto Rodriguez 1389aae218 [HOT FIX] 03042019 
fix https://github.com/Cyb3rWard0g/HELK/issues/215
- Logstash plugins offline install (default)
- Logstash mutate statements update
- ES Memory Calculation fix
- Compose files typo
 2019-03-04 10:03:39 -05:00
Roberto Rodriguez cfb9b98894 [HOT FIX] v0.1.7-alpha02262019 - Logstash Pipeline 
helk-logstash
+ Added offline plugins file
+ Updated win security conversion
+ cleaned process-name filter & process-name-split configs
+ cleaned process-id filter & proces-id conversion configs
+ set kafka max poll records to 500
+ updated SOURCE_ & TARGET_ field names from process entity to be renamed process_source_ and process_target. Following the basic `entity_context_property` from OSSEM CIM
 2019-02-26 00:33:31 -05:00
Roberto Rodriguez 65131b2c65 [Alpha] v0.1.7-alpha02242019 2019-02-24 17:27:03 -05:00
Roberto Rodriguez 5986ff4e2b KSQL Images version update 
Updated KSQL Server and CLI to 5.1.2
 2019-02-24 16:00:57 -05:00
Roberto Rodriguez c6b6d7c881 [HOT FIX] Jupyter & Logstash 
helk-Jupyter
+ Deleted several notebooks that were repeating code and exercises
+ Consolidated notebooks to show the basics of python, pandas, Spark SQL, Pyspark and Graphframes
+ Updated pip libraries

helk-logstash
+ removed 999 pipeline output config since it was affecting logstash start
+ added z_originial_message condition when fingerprinting events. That helps for when I want to replicate events that have been already parsed by helk-logstash
 2019-02-23 19:40:01 -05:00
Roberto Rodriguez cb5950ae32 [HOT-FIX] Logstash & Nginx 
fix https://github.com/Cyb3rWard0g/HELK/issues/195
fix https://github.com/Cyb3rWard0g/HELK/issues/197
fix https://github.com/Cyb3rWard0g/HELK/issues/196
 2019-02-22 10:33:30 -05:00
Roberto Rodriguez fbe9ca8e9e
Merge pull request #181 from nicholasaleks/bugfix/issue104-jupyterlab-throws-403 
Bugfix/issue104 jupyterlab throws 403
 2019-02-22 08:11:13 -05:00
Roberto Rodriguez e34dad52e0
Merge pull request #193 from neu5ron/master 
What in the heeeeeeeeeeeelk
 2019-02-22 07:45:03 -05:00
neu5ron 81912acef1 2 new default mappings 2019-02-22 04:24:44 -05:00
neu5ron 41e36572a0 full nxlog support, with ability to merge directly with winlogbeat so full HELK pipeline is support for windows logs coming from both winlogbeat or nxlog or both :) 2019-02-22 03:57:34 -05:00
neu5ron f230e6d2c3 revisit catchall... 2019-02-22 03:57:25 -05:00
neu5ron a77419060b #175 
#126
- spacing & newline cleanup
 2019-02-22 03:22:20 -05:00
neu5ron b8ba2c6ef4 #175 
#115
- drastically reduced minimum compute
- additional logic for heap related to very little computer for people testing
- spacing & newline cleanup
 2019-02-22 03:22:06 -05:00
neu5ron 9499ca9de9 #176 
#175
- drastically reduce minimum requirements
- update docker-ce
- automatically choose option 1 if not enough computer for option 2, warns user as well
- spacing & newline cleanup
- a bit better variabling for echo'ing messages/info
- an additional systctl vm.max_map_count modification for really large deployments
 2019-02-22 03:21:06 -05:00
neu5ron b1dc331a49 - update docker-ce 
- spacing & newline cleanup
 2019-02-22 03:19:34 -05:00
neu5ron 1012287c22 #100 2019-02-22 03:15:28 -05:00
neu5ron 97b271b00f ELK 6.6.1 :) 
also logstash port 8531 for nxlog tcp input :)
 2019-02-22 03:13:14 -05:00
neu5ron f611302830 only keep enabled winlogbeat configuration parameters for readability - as discussed. 2019-02-22 03:12:13 -05:00
neu5ron 56b9da4994 ELK 6.6.1 :) 2019-02-22 03:11:29 -05:00
neu5ron 192b88f724 #171 
also catches other scenarios in future that would lead to fingerprint/hasing issue
 2019-02-22 03:08:36 -05:00
neu5ron 85023608e0 #118 
#121
 2019-02-22 03:06:08 -05:00