Gartner Blog Network


Cool Vendor Pick: Graylog

by Jonah Kowall  |  January 27, 2015  |  20 Comments

There has been a lot of interest over the last 12 months in products based on open source for monitoring and management. In the area of log analysis, Elasticsearch has been a player which has strengthened with the growing investments in the space. The awareness has been greatly increased in the past year. While the popular Kibana frontend to Elasticsearch has been the main GUI. These two projects are paired with Logstash for ingest, combined these make up the ELK stack. There is another great open source project to take a look at. The focus of this weeks write-up is on this alternative to ELK.

The company behind Graylog is Torch out of Hamburg Germany (https://www.torch.sh/) they do consulting around the product. The open source site is https://www.graylog2.org/ the project is an ElasticSearch based product, but unlike Kibana it also has additional features:

  • Take inputs directly into the Graylog server processes
  • Output from the server to multiple backends based on output plugins, right now the main one is for ElasticSearch
  • Alerting based on matching or other criteria are integrated into the Graylog project along with a stream processing capability

The supported data comes in the form of plugins which include syslog or GELF (Graylog Extended Log Format) or other plugins. GELF allows for several enhancement from typical syslog.

  • No length limitations for messages (syslog is 1024 bytes)
  • Data types (string, number)
  • Variation in syslog implementation
  • Compression via gzip or zlib

The nice thing is that you don’t need to do any extractions once the messages have been added via GELF. They have 72 such plugins including many GELF libraries (See: https://www.graylog2.org/supported-sources?perPage=100)

On the site you can sign up for a self-service trial of the software, I did this in early November, there has been another release since then. These screenshots may be a little out of date:

Image1

 

Image2

There can be multiple backend nodes connected to the frontend. There is some good management within the GUI of the connections. The main dashboard when you login shows you information about the cluster, components, and the status. There is a query box.

Image4

 

Some other administrative views. Many of the log management tools, especially in open source neglect the day to day maintenance and administration. Being a systems and operations person myself I always dig into the internals needed for day to day administration. Graylog has a lot of what’s been missing across open source ElasticSearch management tools. Some additional views:

 

Image19

 

Image20

They have a data generator in the demo so you’ll see there are plenty of events in the data store.

Image14

Image4

Here is a query for smtp in the last 30 minutes.

Image5

 

You can also see inside the queries being sent to ElasticSearch, here are the JSON objects being passed to the engine:

Image6

Value breakdowns of the results quickly

Image7

 

Graylog has the notion of stream as illustrated below

Image8

 

What these are is a way to pass realtime rules against the data coming into the Graylog server before they are committed to elasticsearch, this real time processing provides a differentiator to Kibana based systems

Image9

Image10

 

Image17

 

Some sample sinks of what you can do with a proper eventing system, such as alerting:

Image11

The requisite dashboarding for any monitoring tool. Everyone loves dashboards, users are always asking for more dashboards, and they clearly do sell monitoring products. The value they provide are typically pretty limited. If the actual analytics in our software were better the computer would be doing the analysis versus a user looking at graphical displays of data. I digress…

Image12

You cannot share the same backend between Kibana/Logstash and Graylog since they use a different schema for the log data in ElasticSearch. Hence you’ll have to make a decision which tool you want to use when setting up ElasicSearch. Please leave comments or questions below on @jkowall on Twitter.

 

Additional Resources

Category: analytics  itoa  logfile  monitoring  olm  

Jonah Kowall
Research Vice President
3.5 years with Gartner
20 years IT industry

Jonah Kowall is a research Vice President in Gartner's IT Operations Research group. He focuses on application performance monitoring (APM), Unified Monitoring, Network Performance Monitoring and Diagnostics (NPMD), Infrastructure Performance Monitoring (IPM), IT Operations Analytics (ITOA), and general application and infrastructure availability and performance monitoring technologies. Read Full Bio


Thoughts on Cool Vendor Pick: Graylog


  1. Arie says:

    Nice review Jonah,

    It is a great tool in our systems management environment. What makes this great is the ease of use of this toolset. We started using it to collect monitoring data (nagios/check_mk) into it, to get better insight in what is happening in time. The smart configurable stream alerts are very useful to send out alerts. Another thing we do is collect windows eventlogs with nxlog in GELF format.

    It is possible to look at the data in ES with Kibana-V3, to make (management)-dashboards to present statistical views and search thru the data, and find this correlating events that matter.

    The next thing is to get logfile data into it so we have everything in one place to look at known and unknown issues that arise on us, sold problems or detect them before they arise.

  2. […] Kowall, Research VP for Gartner.   The overview, published on January 27, 2015 and titled, “Cool Vendor Pick: Graylog,” provides a brief overview and several screen shots of […]

  3. As I see it, the huge roadblock to progress is still the digital marketing talent shortage. Most companies have a limited ‘talent puddle’ of skilled practitioners that are able to work on progressive market development strategies. Meanwhile, the majority of their old-school marketers have not attempted to learn the required new skills — so marketing organizations are dominated by staff that view the world through their legacy media-buyer mindset. To them, digital marketing merely means buying Google Ads or advertising placements on Facebook and LinkedIn. What can a CMO do when 80+ of their current team are not skilled for today’s demands? Clearly, it’s a big ongoing challenge.

  4. xem phim sex says:

    Thank So Much For Sharing

  5. It is a great tool in our systems management environment. What makes this great is the ease of

  6. It is a great tool in our systems management environment. What makes this great is the

  7. video says:

    It is a great tool in our systems management environment.

  8. Great content quality! lookout our website too, thank you.

  9. It is really helpful. thank u

  10. مشاهده، بررسی و خرید انواع دستگاه و صندلی ماساژور در وبسایت آی‌رست ♥

  11. Helpful contents! thanks for announcing.



Leave a Reply

Your email address will not be published. Required fields are marked *

Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.