Commit Graph

315 Commits (c18aac2f5136cf86e906ca67f89c673e5895e54e)

Author SHA1 Message Date
neu5ron c18aac2f51 switch from murmur3 to sha1, reference https://github.com/Cyb3rWard0g/HELK/issues/231 2019-04-17 15:40:09 -04:00
Nate Guagenti 489adb9c57
Update winlogbeat.yml
winlogbeat 7.x field name changes will require some pipeline rewrites. propose users to use version 6.x of winlogbeat
2019-04-13 22:44:06 -04:00
Roberto Rodriguez 2ef0bd0bed Update kibana-setup.sh
fix https://github.com/Cyb3rWard0g/HELK/issues/228
2019-04-06 19:40:18 -07:00
Roberto Rodriguez bb1f4ce8ca
Merge pull request #227 from aarju/master
Adding incident Response Dashboards for investigating alerts
2019-04-06 21:33:19 -04:00
Aaron Jewitt bf16cfb54d Added Incident Response investigation dashboards 2019-04-06 20:24:49 +02:00
Roberto Rodriguez 2b30994493 Updated a few configs 2019-04-06 13:21:29 -04:00
Aaron Jewitt b669cd5fa8
updated DFIR_Dashboards.json
attempting to format the json for kibana API injest
2019-04-03 22:50:27 +02:00
Aaron Jewitt 38c0936ad6
Update DFIR_Dashboards.json 2019-04-01 22:14:26 +02:00
aaron 57e9008f66 Added the DFIR_Dashboards.json file 2019-03-28 13:10:36 -07:00
Nate Guagenti b331afdfb8 Update 0099-all-fingerprint-hash-filter.conf 2019-03-23 10:44:54 -04:00
Nate Guagenti 91f761fee3 Update 0099-all-fingerprint-hash-filter.conf 2019-03-23 10:44:54 -04:00
Nate Guagenti 9ed4539a53 Update 0099-all-fingerprint-hash-filter.conf 2019-03-23 10:44:54 -04:00
Nate Guagenti b268b38c0e Update 0099-all-fingerprint-hash-filter.conf
better fingerprint-hashing for deduplication.
more specific for both winlogbeat and nxlog
2019-03-23 10:44:54 -04:00
Roberto Rodriguez 98e32e2e87 Resources- Images 2019-03-16 14:30:12 -04:00
Roberto Rodriguez e819329f7a [HOT FIX] Mainly Jupyter and Logstash Updates
HELK-JUPYTER
+ Miniconda3 to handle python packages
+ Python 3.7
+ Container not running as root
+ new entrypoint and cmd scripts
+ postgres not running as root and under the same container
+ Spark Jar and Python dependencies provided offline (not downloading from maven directly - Sometimes this fails)
+ Jupyter PySpark kernel using conda to run ipykernel module
+ PYSPARK_PYTHON Python 3.7

HELK-LOGSTASH
+ Fix https://github.com/Cyb3rWard0g/HELK/issues/217
2019-03-11 09:00:54 -04:00
Roberto Rodriguez 1389aae218 [HOT FIX] 03042019
fix https://github.com/Cyb3rWard0g/HELK/issues/215
- Logstash plugins offline install (default)
- Logstash mutate statements update
- ES Memory Calculation fix
- Compose files typo
2019-03-04 10:03:39 -05:00
Roberto Rodriguez cfb9b98894 [HOT FIX] v0.1.7-alpha02262019 - Logstash Pipeline
helk-logstash
+ Added offline plugins file
+ Updated win security conversion
+ cleaned process-name filter & process-name-split configs
+ cleaned process-id filter & proces-id conversion configs
+ set kafka max poll records to 500
+ updated SOURCE_ & TARGET_ field names from process entity to be renamed process_source_ and process_target. Following the basic `entity_context_property` from OSSEM CIM
2019-02-26 00:33:31 -05:00
Roberto Rodriguez 65131b2c65 [Alpha] v0.1.7-alpha02242019 2019-02-24 17:27:03 -05:00
Roberto Rodriguez 5986ff4e2b KSQL Images version update
Updated KSQL Server and CLI to 5.1.2
2019-02-24 16:00:57 -05:00
Roberto Rodriguez c6b6d7c881 [HOT FIX] Jupyter & Logstash
helk-Jupyter
+ Deleted several notebooks that were repeating code and exercises
+ Consolidated notebooks to show the basics of python, pandas, Spark SQL, Pyspark and Graphframes
+ Updated pip libraries

helk-logstash
+ removed 999 pipeline output config since it was affecting logstash start
+ added z_originial_message condition when fingerprinting events. That helps for when I want to replicate events that have been already parsed by helk-logstash
2019-02-23 19:40:01 -05:00
Roberto Rodriguez cb5950ae32 [HOT-FIX] Logstash & Nginx
fix https://github.com/Cyb3rWard0g/HELK/issues/195
fix https://github.com/Cyb3rWard0g/HELK/issues/197
fix https://github.com/Cyb3rWard0g/HELK/issues/196
2019-02-22 10:33:30 -05:00
Roberto Rodriguez fbe9ca8e9e
Merge pull request #181 from nicholasaleks/bugfix/issue104-jupyterlab-throws-403
Bugfix/issue104 jupyterlab throws 403
2019-02-22 08:11:13 -05:00
Roberto Rodriguez e34dad52e0
Merge pull request #193 from neu5ron/master
What in the heeeeeeeeeeeelk
2019-02-22 07:45:03 -05:00
neu5ron 81912acef1 2 new default mappings 2019-02-22 04:24:44 -05:00
neu5ron 41e36572a0 full nxlog support, with ability to merge directly with winlogbeat so full HELK pipeline is support for windows logs coming from both winlogbeat or nxlog or both :) 2019-02-22 03:57:34 -05:00
neu5ron f230e6d2c3 revisit catchall... 2019-02-22 03:57:25 -05:00
neu5ron a77419060b #175
#126
- spacing & newline cleanup
2019-02-22 03:22:20 -05:00
neu5ron b8ba2c6ef4 #175
#115
- drastically reduced minimum compute
- additional logic for heap related to very little computer for people testing
- spacing & newline cleanup
2019-02-22 03:22:06 -05:00
neu5ron 9499ca9de9 #176
#175
- drastically reduce minimum requirements
- update docker-ce
- automatically choose option 1 if not enough computer for option 2, warns user as well
- spacing & newline cleanup
- a bit better variabling for echo'ing messages/info
- an additional systctl vm.max_map_count modification for really large deployments
2019-02-22 03:21:06 -05:00
neu5ron b1dc331a49 - update docker-ce
- spacing & newline cleanup
2019-02-22 03:19:34 -05:00
neu5ron 1012287c22 #100 2019-02-22 03:15:28 -05:00
neu5ron 97b271b00f ELK 6.6.1 :)
also logstash port 8531 for nxlog tcp input :)
2019-02-22 03:13:14 -05:00
neu5ron f611302830 only keep enabled winlogbeat configuration parameters for readability - as discussed. 2019-02-22 03:12:13 -05:00
neu5ron 56b9da4994 ELK 6.6.1 :) 2019-02-22 03:11:29 -05:00
neu5ron 192b88f724 #171
also catches other scenarios in future that would lead to fingerprint/hasing issue
2019-02-22 03:08:36 -05:00
neu5ron 85023608e0 #118
#121
2019-02-22 03:06:08 -05:00
neu5ron 34d51bc8da #118
#121
2019-02-22 03:04:17 -05:00
neu5ron a2dafe5a4b #173 2019-02-22 03:02:34 -05:00
neu5ron aeeb195a7c #182 & #183 2019-02-22 03:02:00 -05:00
neu5ron ad7817f3ea not ip index management for pipeline we had, but hand't set elasticsearch template - now we do 2019-02-22 03:01:30 -05:00
neu5ron 1939fa58c4 #178 2019-02-22 03:00:31 -05:00
nicholasaleks 1851fd3af2 Create helk admin user home dir to store notebooks 2019-02-18 18:04:36 -05:00
nicholasaleks f7266986cb Relocated jupyterhub notebooks to user home directory (seems like standard practice according to documentation) 2019-02-17 19:02:56 -05:00
nicholasaleks 6c8b81b316 Added user to the Jupyter spawner and notebook_dir 2019-02-17 17:52:06 -05:00
Roberto Rodriguez 4184706206 [HOT-FIX] 02022019
helk-Elasticsearch
- Adjusted ES JAVA OPTs (Heap size) calculations

helk-jupyter
+ Upgraded image to 0.1.0
+ Updated graphframes to 0.7.0
+ fix https://github.com/Cyb3rWard0g/HELK/issues/161
+ fix https://github.com/Cyb3rWard0g/HELK/issues/163

helk-logstash
+ fix https://github.com/Cyb3rWard0g/HELK/issues/162
2019-02-02 03:17:25 -05:00
Roberto Rodriguez 5318e9e37a
Merge pull request #158 from devdua/update-script
Made subscription and build choice menu same as install script
2019-01-31 14:56:15 -05:00
Roberto Rodriguez c7086ab9c6 [HOT FIX] 01312019
helk ELK
Updated to version 6.5.4

helk-logstash
fix https://github.com/Cyb3rWard0g/HELK/issues/156
+ Pipeline Updated
++ More security events
++ Reduced regex complexity to split process paths to process names
++ Enabled Kafka output again for Win Security and Win Sysmon logs
++ Added more win security conversion events

helk-elastalert
fix https://github.com/Cyb3rWard0g/HELK/issues/157
fix https://github.com/Cyb3rWard0g/HELK/issues/159

ELK:
+ Consolidated ELK scripts to one per container instead of trial and basic

helk-sigma
+ Updated own fork

helk-jupyter
+ Updated Elastic ES-Hadoop to 6.5.4

helk-jupyter
+ jupyterlab-manager widgets
+ Updated pandas 0.24.0
+ Updated altair 2.3.0
2019-01-31 11:29:49 -05:00
Dev Dua c5f206debf Made subscription and build choice menu same as install script 2019-01-29 10:31:10 +05:30
Roberto Rodriguez 9b7d224661
Merge pull request #152 from devdua/improved-git-handling-update-script
Added git checks for stable update experience
2019-01-24 11:54:47 -05:00
Roberto Rodriguez fa1c3aa5f3
Merge pull request #153 from richiercyrus/master
OSQuery Intergration (MacOS Support)
2019-01-24 11:53:42 -05:00