HELK-JUPYTER
+ Miniconda3 to handle python packages
+ Python 3.7
+ Container not running as root
+ new entrypoint and cmd scripts
+ postgres not running as root and under the same container
+ Spark Jar and Python dependencies provided offline (not downloading from maven directly - Sometimes this fails)
+ Jupyter PySpark kernel using conda to run ipykernel module
+ PYSPARK_PYTHON Python 3.7
HELK-LOGSTASH
+ Fix https://github.com/Cyb3rWard0g/HELK/issues/217
helk-Jupyter
+ Deleted several notebooks that were repeating code and exercises
+ Consolidated notebooks to show the basics of python, pandas, Spark SQL, Pyspark and Graphframes
+ Updated pip libraries
helk-logstash
+ removed 999 pipeline output config since it was affecting logstash start
+ added z_originial_message condition when fingerprinting events. That helps for when I want to replicate events that have been already parsed by helk-logstash
helk ELK
Updated to version 6.5.4
helk-logstash
fix https://github.com/Cyb3rWard0g/HELK/issues/156
+ Pipeline Updated
++ More security events
++ Reduced regex complexity to split process paths to process names
++ Enabled Kafka output again for Win Security and Win Sysmon logs
++ Added more win security conversion events
helk-elastalert
fix https://github.com/Cyb3rWard0g/HELK/issues/157
fix https://github.com/Cyb3rWard0g/HELK/issues/159
ELK:
+ Consolidated ELK scripts to one per container instead of trial and basic
helk-sigma
+ Updated own fork
helk-jupyter
+ Updated Elastic ES-Hadoop to 6.5.4
helk-jupyter
+ jupyterlab-manager widgets
+ Updated pandas 0.24.0
+ Updated altair 2.3.0
HELK base image
+ Updated to 0.0.3
HELK ELK Version
+ Now using 6.5.3 official ELK Docker Images (https://www.elastic.co/blog/elastic-stack-6-5-3-released)
helk_install
+ Users can now select between two deployments:
++ helk-kibana-analysis (KAFKA + KSQL + ELK + NGNIX + ELASTALERT)
++ helk-kibana-notebooks (KAFKA + KSQL + ELK + NGNIX + ELASTALERT + SPARK + JUPYTER)
+ Fixed https://github.com/Cyb3rWard0g/HELK/issues/131 . Users can now set up the Kibana UI User password during installation. Also, user can set the Elasticsearch elastic account password when using the Trial license option.
helk-elastalert
+ Elastalert deployed and ready to use with SIGMA integration. Blog available at https://medium.com/@Cyb3rWard0g
helk-elasticsearch
+ consolidated main configs in one
+ added more environment variables for ELASTIC_PASSWORD and default values in case it is not used to be compatible with the default values applied to HELK.
helk-logstash
+ updated to 6.5.3
+ simplified pipeline to have only one folder
+ logstash-entrypoint script can now enable elastic password on all logstash output conf files.
+ New environment variables (ELASTIC_PASSWORD, ELASTIC_HOST, ELASTIC_PORT)
helk-nginx
+ split the default config for the two deployment options (helk-kibana-analysis (trial/base) and helk-kibana-notebook-analysis (trial/base)
helk-kibana
+ Updated to version 6.5.3
+ Added new environment variables (ELASTICSEARCH_URL, SERVER_HOST, SERVER_PORT, ELASTIC_PASSWORD, ELASTIC_HOST, ELASTIC_PORT, ELASTICSEARCH_USERNAME, ELASTICSEARCH_PASSWORD, KIBANA_UI_PASSWORD) and logic to make the build more dynamic
helk-jupyter
+ updated Jupyterlab to 0.35.4
+ updated jupyterhub to 0.9.4
+ updated jupyterlab hub extension to 0.12.0
+ updated ES_HADOOP to 6.5.3
+ updated org.apache.spark:spark-sql-kafka-0-10_2.11:2.4.0
+ Added extra notebooks to test deployment and provide more information for analyst experiencing Jupyter for the first time
helk-kafka-base
+ reduced docker container size
+ updated Kafka to 2.1.0 (this affects Kafka brokers and zookeeper)
helk-kafka-broker
+ User can now define a list of topics to be created via the new environment variable KAFKA_CREATE_TOPICS. That needs to be defined either in the docker-compose file or while running the docker container on its own.
helk-zookeeper
+ reduced size of container
+ updated build to kafka 2.1.0
helk-KSQL
+ initial integration of KSQL
+ KSQL Server and KSQL CLI are available
+ Blog post coming soon ;)
Roberto Rodriguez
neu5ron
richiercyrus