Cancel
Start searching
This search is based on elasticsearch and can look through several thousand pages in miliseconds.
Learn more
This search is based on elasticsearch and can look through several thousand pages in miliseconds.
Learn more
The Elastic Stack has become an integral part of our projects and my day-to-day work. It’s hard to imagine how, before we started using Elasticsearch and Kibana for analysis, we were able to keep track of and evaluate the massive amounts of information that our clients’ servers write to log files every day. In addition to the classic ELK Stack for technical log file analysis, we can now also offer our clients customized real-time analyses of business data based on the Elastic Stack.
Against the impressive backdrop of the Kapitol Theater and Concert Hall in Frankfurt-Offenbach, users of the Elastic Stack and the developers behind these products came together for a day. In short and concise presentations, the respective core developers first demonstrated what is already possible today with version 5.0 of Elasticsearch, Kibana, Beats, Logstash, and X-Pack, as well as what is planned for 2017.
Special attention was paid to the Timelion module. What began as a hobby project by the original author of Kibana has now come of age and, using a powerful query language, allows users to perform complex analyses on the data and generate informative graphs. The Timelion charts can then be used in the familiar Kibana dashboards and, of course, interact with the widgets available there.
Next, Tech Lead Steve Dodson introduced Prelert. The behavioral analysis program, originally developed by his independent company, was recently acquired by Elastic and will be the most exciting new Kibana module for me.
In an impressive presentation, Steve demonstrated how Prelert uses “unsupervised machine learning” to identify patterns in time-series data, highlight deviations from those patterns, and generate alerts—all before a serious outage even occurs. In doing so, it learns correlations and relationships even between highly diverse data—such as logs from different services and servers—and thus helps investigate the root causes of a problem.
The release of the updated Kibana module is expected in mid-2017. However, it won't actually be available to everyone. The Prelert license requires at least one Elastic Stack Platinum subscription.
Two workshops used specific scenarios from the field of security to demonstrate the full range of possibilities offered by the modules in the Elastic Stack.
The first scenario imported log data from SSH authentication using FileBeat and distinguished between successful and failed login attempts by valid users and a brute-force attack by an attacker. Upon detecting a successful brute-force attack, the Watcher module sends an email containing relevant details.
The second scenario addressed the detection of access to websites containing malware—both in real time and for subsequent analysis. To achieve this, PacketBeat was used to collect network packets from DNS queries directly at the proxy, which were then forwarded to Elasticsearch via Logstash. Using Perlocator queries, the documents can be analyzed directly within the Elasticsearch cluster, and alerts can be generated in real time when matches are found.
Of course, the documents stored in Elasticsearch can also be analyzed at any time afterward. This way, as new threats become known, clients on the network that have accessed the compromised URLs in the past can be immediately identified, and appropriate measures can be taken.
In addition to the presentations and workshops, there was, of course, plenty of opportunity for discussions among users of Elasticsearch products. The numerous “Ask-Me-Anything” stations were particularly helpful, where Elasticsearch employees were able to competently answer even the most specific questions.
Overall, the “small” Elastic conference was a well-rounded and highly recommended event.
