Commit Graph

329 Commits (42b71c83d9eff7c2e0a990198d572476a8ed85e2)

Author SHA1 Message Date
neu5ron 34d51bc8da #118
#121
2019-02-22 03:04:17 -05:00
neu5ron a2dafe5a4b #173 2019-02-22 03:02:34 -05:00
neu5ron aeeb195a7c #182 & #183 2019-02-22 03:02:00 -05:00
neu5ron ad7817f3ea not ip index management for pipeline we had, but hand't set elasticsearch template - now we do 2019-02-22 03:01:30 -05:00
neu5ron 1939fa58c4 #178 2019-02-22 03:00:31 -05:00
nicholasaleks 1851fd3af2 Create helk admin user home dir to store notebooks 2019-02-18 18:04:36 -05:00
nicholasaleks f7266986cb Relocated jupyterhub notebooks to user home directory (seems like standard practice according to documentation) 2019-02-17 19:02:56 -05:00
nicholasaleks 6c8b81b316 Added user to the Jupyter spawner and notebook_dir 2019-02-17 17:52:06 -05:00
Roberto Rodriguez 4184706206 [HOT-FIX] 02022019
helk-Elasticsearch
- Adjusted ES JAVA OPTs (Heap size) calculations

helk-jupyter
+ Upgraded image to 0.1.0
+ Updated graphframes to 0.7.0
+ fix https://github.com/Cyb3rWard0g/HELK/issues/161
+ fix https://github.com/Cyb3rWard0g/HELK/issues/163

helk-logstash
+ fix https://github.com/Cyb3rWard0g/HELK/issues/162
2019-02-02 03:17:25 -05:00
Roberto Rodriguez 5318e9e37a
Merge pull request #158 from devdua/update-script
Made subscription and build choice menu same as install script
2019-01-31 14:56:15 -05:00
Roberto Rodriguez c7086ab9c6 [HOT FIX] 01312019
helk ELK
Updated to version 6.5.4

helk-logstash
fix https://github.com/Cyb3rWard0g/HELK/issues/156
+ Pipeline Updated
++ More security events
++ Reduced regex complexity to split process paths to process names
++ Enabled Kafka output again for Win Security and Win Sysmon logs
++ Added more win security conversion events

helk-elastalert
fix https://github.com/Cyb3rWard0g/HELK/issues/157
fix https://github.com/Cyb3rWard0g/HELK/issues/159

ELK:
+ Consolidated ELK scripts to one per container instead of trial and basic

helk-sigma
+ Updated own fork

helk-jupyter
+ Updated Elastic ES-Hadoop to 6.5.4

helk-jupyter
+ jupyterlab-manager widgets
+ Updated pandas 0.24.0
+ Updated altair 2.3.0
2019-01-31 11:29:49 -05:00
Dev Dua c5f206debf Made subscription and build choice menu same as install script 2019-01-29 10:31:10 +05:30
Roberto Rodriguez 9b7d224661
Merge pull request #152 from devdua/improved-git-handling-update-script
Added git checks for stable update experience
2019-01-24 11:54:47 -05:00
Roberto Rodriguez fa1c3aa5f3
Merge pull request #153 from richiercyrus/master
OSQuery Intergration (MacOS Support)
2019-01-24 11:53:42 -05:00
richiercyrus 5c75079d7c added keys under root true to config. 2019-01-24 10:53:15 +00:00
Dev Dua eb0e6faba6 Added git checks for stable update experience 2019-01-24 12:58:45 +05:30
Roberto Rodriguez 5f303c83ae Update helk_install.sh
Fix https://github.com/Cyb3rWard0g/HELK/issues/149
2019-01-11 19:02:24 -05:00
richiercyrus d372ef452d Troubleshooting the output file. 2019-01-08 15:38:57 -08:00
richiercyrus 12e4de9996 Added osquery filter to parse json message. 2019-01-08 08:19:37 -08:00
Roberto Rodriguez f4e323790d Update sysmon-join.commands 2019-01-07 13:05:11 -05:00
Roberto Rodriguez 8b28eb3b0c
Merge pull request #146 from devdua/update-script-decision-block-rearrangement
Update script decision block rearrangement
2019-01-07 09:28:53 -05:00
Nate Guagenti 5864371375
Update helk_install.sh
bash is no better than python :)
https://github.com/Cyb3rWard0g/HELK/issues/148
thanks @roberix
2019-01-07 09:26:47 -05:00
richiercyrus 240a8262ff Initial filebeat changes for osquery intergration 2019-01-05 11:00:10 -05:00
Dev Dua 6dd269c5b5 Refactored decision making to pull updates 2019-01-04 12:20:08 +05:30
Dev Dua a133a68f5f Merge branch 'master' of https://github.com/Cyb3rWard0g/HELK 2019-01-04 12:17:15 +05:30
Ryan G 5ac63ca1e9 Added missing exit and wiki reference 2019-01-04 01:14:55 -05:00
Ryan G 03ce2ff74e Added a pre-install check for x86_64 architecture to docker/helk_install.sh, as it is required for the project to run properly after the install script is completed. 2019-01-04 01:14:55 -05:00
Roberto Rodriguez 48fb280398 Update helk_install.sh
fix https://github.com/Cyb3rWard0g/HELK/issues/142
2018-12-25 15:30:09 -05:00
Roberto Rodriguez eecd5f6c09 Update KSQL Post Additions 2018-12-24 15:12:56 -05:00
Roberto Rodriguez ccbee9f6fb
Merge pull request #140 from leechristensen/patch-1
Downloads docker installer script over HTTPS
2018-12-20 12:39:08 -05:00
Lee Christensen 1d6b5e82e9
Downloads docker installer script over HTTPS 2018-12-20 09:29:36 -08:00
Roberto Rodriguez 380030515b Merge branch 'master' of https://github.com/Cyb3rWard0g/HELK 2018-12-14 10:29:20 -05:00
Roberto Rodriguez 64e00259fd README update and resources
- Added KSQL logo to HELK Ecosystem
- Updated features section with KSQL description
- Deleted a few images
2018-12-14 10:29:12 -05:00
Dev Dua 09c4b83af6 Merge branch 'christmas-release-update-script-revision' 2018-12-14 20:32:51 +05:30
Roberto Rodriguez 28cae7d58a
Merge pull request #135 from devdua/christmas-release-update-script-revision
Revise update script to align with Christmas Release
2018-12-14 18:01:44 +03:00
Roberto Rodriguez 6cc8a6bf3a Updated a few typos
fix https://github.com/Cyb3rWard0g/HELK/issues/134
fix https://github.com/Cyb3rWard0g/HELK/issues/133
2018-12-14 09:59:02 -05:00
Dev Dua 9549c86723 Revise update script to align with Christmas Release 2018-12-14 17:05:30 +05:30
Roberto Rodriguez dbcf5a1983 Updating README
Added Elastalert and Sigma information
2018-12-14 00:33:05 +03:00
Roberto Rodriguez 181c851a9e v0.1.6-alpha12132018
HELK base image
+ Updated to 0.0.3

HELK ELK Version
+ Now using 6.5.3 official ELK Docker Images (https://www.elastic.co/blog/elastic-stack-6-5-3-released)

helk_install
+ Users can now select between two deployments:
++ helk-kibana-analysis (KAFKA + KSQL + ELK + NGNIX + ELASTALERT)
++ helk-kibana-notebooks (KAFKA + KSQL + ELK + NGNIX + ELASTALERT + SPARK + JUPYTER)
+ Fixed https://github.com/Cyb3rWard0g/HELK/issues/131 . Users can now set up the Kibana UI User password during installation. Also, user can set the Elasticsearch elastic account password when using the Trial license option.

helk-elastalert
+ Elastalert deployed and ready to use with SIGMA integration. Blog available at https://medium.com/@Cyb3rWard0g

helk-elasticsearch
+ consolidated main configs in one
+ added more environment variables for ELASTIC_PASSWORD and default values in case it is not used to be compatible with the default values applied to HELK.

helk-logstash
+ updated to 6.5.3
+ simplified pipeline to have only one folder
+ logstash-entrypoint script can now enable elastic password on all logstash output conf files.
+ New environment variables (ELASTIC_PASSWORD, ELASTIC_HOST, ELASTIC_PORT)

helk-nginx
+ split the default config for the two deployment options (helk-kibana-analysis (trial/base) and helk-kibana-notebook-analysis (trial/base)

helk-kibana
+ Updated to version 6.5.3
+ Added new environment variables (ELASTICSEARCH_URL, SERVER_HOST, SERVER_PORT, ELASTIC_PASSWORD, ELASTIC_HOST, ELASTIC_PORT, ELASTICSEARCH_USERNAME, ELASTICSEARCH_PASSWORD, KIBANA_UI_PASSWORD) and logic to make the build more dynamic

helk-jupyter
+ updated Jupyterlab to 0.35.4
+ updated jupyterhub to 0.9.4
+ updated jupyterlab hub extension to 0.12.0
+ updated ES_HADOOP to 6.5.3
+ updated org.apache.spark:spark-sql-kafka-0-10_2.11:2.4.0
+ Added extra notebooks to test deployment and provide more information for analyst experiencing Jupyter for the first time

helk-kafka-base
+ reduced docker container size
+ updated Kafka to 2.1.0 (this affects Kafka brokers and zookeeper)

helk-kafka-broker
+ User can now define a list of topics to be created via the new environment variable KAFKA_CREATE_TOPICS. That needs to be defined either in the docker-compose file or while running the docker container on its own.

helk-zookeeper
+ reduced size of container
+ updated build to kafka 2.1.0

helk-KSQL
+ initial integration of KSQL
+ KSQL Server and KSQL CLI are available
+ Blog post coming soon ;)
2018-12-14 00:27:17 +03:00
Roberto Rodriguez bb9946331f
Merge pull request #129 from devdua/revise-helk-update
Minor improvement to update check
2018-12-10 09:10:56 +03:00
Dev Dua 98f8b06085 Minor improvement to update check 2018-12-10 11:35:43 +05:30
neutron cbecc30cb6 - fix for large import of settings like dashboards/visualizations
- future fix for other applications like CyberChef that will generate long URIs
2018-12-06 01:24:45 -05:00
neutron fbe105ead7 - previous sysctl for vm.max_map_count was not persistent
- custom sysctl file for future tuning/settings
- typo correction
2018-12-06 01:08:54 -05:00
neutron 7b51b81d7d - better ulimit settings
- set so that we will use calculation of host memory to set the JVM options -- otherwise users can uncomment the JAVA_OPTS in this config and will ignore our memory check.
2018-12-06 01:05:15 -05:00
neutron 8d93f770ff get host available memory and divide that by 4, use megabytes in case dividing by 4 is too low of a number like 0 :) 2018-12-06 01:03:11 -05:00
neutron 16cccca2f3 get host available memory and set that by half, however not higher than 31GB 2018-12-06 01:02:04 -05:00
Roberto Rodriguez 9dabef7ad8
Merge pull request #66 from devdua/add-helk-update-script
Add helk update script
2018-10-03 12:46:43 -04:00
Roberto Rodriguez bcabb02257
Merge pull request #106 from devdua/modify-ip-extraction-method-linux
Modify host IP extraction system for Linux kernel
2018-10-03 12:42:23 -04:00
Dev Dua 68c33d5ed5 Modify host IP extraction system for Linux kernel 2018-10-03 21:41:56 +05:30
Dev Dua 00b194427f Merge branch 'master' of https://github.com/Cyb3rWard0g/HELK into add-helk-update-script 2018-10-01 11:49:12 +05:30