Gartner Blog Network

Monitoring simplification

by Jonah Kowall  |  August 5, 2011  |  17 Comments

Many of the clients I speak with on a regular basis want to simplify their monitoring technologies. It’s too time consuming, error prone, and complex to run much of the software deployed currently. Many of them want to outsource the management, some of them want SaaS models, and others want appliance or simple monitoring systems. Obviously you make sacrifices when you take this approach, but does it really affect downtime? Leveraging agentless monitoring is a good way to increase coverage and reduce complexity involved in agent management. In many cases, and on critical systems agents are required. This is something that is discussed in a note which was published a few days ago: How to Augment APM with conventional monitoring

Customer demand is changing a lot of portfolios inside companies, and vendors are responding. I am seeing smaller vendors go up against large incumbent solutions in a more regular manner. I believe this trend will continue, and one of my upcoming research notes on SaaS monitoring solutions will provide a good overview of the options if you want to have control, but simplify the technology and implementation.

Alongside this trend there is a strengthening interest in open source monitoring tools, saving cost, and simplifying management of the tools. The ease of open source is somewhat of a misnomer, which is why I am close to publishing 2 notes on open source and commercial open source options.

Category: eca  it-operations  

Jonah Kowall
Research Vice President
3.5 years with Gartner
20 years IT industry

Jonah Kowall is a research Vice President in Gartner's IT Operations Research group. He focuses on application performance monitoring (APM), Unified Monitoring, Network Performance Monitoring and Diagnostics (NPMD), Infrastructure Performance Monitoring (IPM), IT Operations Analytics (ITOA), and general application and infrastructure availability and performance monitoring technologies. Read Full Bio

Thoughts on Monitoring simplification

  1. Charley Rich says:

    Good points.

    Also to consider is the full user story for this monitoring scenario. What would be the next action after your monitoring tool produces an alert? If it is problem isolation and you go into still a different tool and figure out what to fix, ok. But, if you want to get enough depth of information from the monitoring tool producing the alert to know what to correct, in most cases you will find it quite difficult to do without local agents.

    I think the two step process will affect downtime as it so often leads to the “all hands meeting” where someone is left standing when the music stops and it is their job to fix the issue.

    Charley Rich
    crich @

  2. Jonah Kowall says:

    Yes, deep dive tools are often needed, but there is an overall need for monitoring tools which are simple, and provide “just enough” data. This is more the case in other monitoring and not APM. The buyers of APM tools often need depth and richness. The APM market is evolving rapidly, it is not yet a mature, consolidated space. There are still some interesting success stories in APM of vendors who do “just enough” as well, but they are not yet mature enough to determine if the approach will stick with the ever expanding complexity of our application delivery infrastructures.

    -Jonah Kowall

  3. Ben Clonts says:

    Very good topic. I have seen this play out in many industries. The real challenge I see with monitoring is the human component. It takes real experts to set up and maintain the monitoring systems. Not just the installation of the software, but establishing and maintaining the alert thresolds so they yield valuable and timely information.

    Unfortunately it is not a set it and forget it scenario.

    Keeping it simple is critical to success, as it will dramatically decrease the required maintance (read support manhours) and establish the value of the monitoring system early.

    Most efforts die out quickly because the organization cannot dedicate the needed expert support manhours. Everyone becomes disenchanted and the system gets shelved.

  4. Jonah Kowall says:

    Ben, I believe this is changing with the advent of more complex analytics and behavior engines which are finding their way into all of the product portfolios. It will create monitoring which understands fluctuations in usage, environment changes (vm, cloud). Use of multivariate analysis on metrics is something which is becoming commonplace.

  5. App Man says:

    Totally agree Jonah.

    Application complexity is on the up for the customer. You can’t manage this complexity with more complexity (incumbent monitoring technology). You can only get value from an APM solution if that solution is easy to deploy and easy to use otherwise it quickly becomes shelf-ware.

    APM promises to let organizations manage application performance and availability, yet most spend their time and money managing the APM solution itself. APM solutions need to work harder to simplify the complexity of agent deployment and configuration whilst delivering the right data to the right user at the right time.

    As applications become more distributed through SOA and cloud computing, APM complexity will only get worse unless many vendors acknowledge their weaknesses so they can alleviate the management and services cost associated with their solutions.

    APM needs more software and less services if it is to make a real impact on helping organizations manage application performance.

  6. Jonah Kowall says:

    Agreed, aside from that APM helps organizations manage performance, but in itself it doesn’t do that directly. Some solutions are quite simple, while others are complex. When you start incorporating other aspects of APM then it becomes difficult to do so in a simple manner. This post is about all monitoring aspects, not just APM.

    I think many vendors are realizing that services/consulting need to be reduced, and many are doing so. The issue is that businesses want to have an expert on hand to help them diagnose issues, hence the reason services are sold. Its almost like renting a performance expert or using a consultant to help fix a hot issue.

  7. Many APM vendors are promoting simplification but all they have done is simplified their engineering effort, limited the application of their solution, and their favorite create a lip-stick-on-pig mgmt dashboard to sell rather than solve.

    We need to make a distinction between surface (UI, model, API) complexity (or simplicity) and internal complexity (intelligent, adaptive,…) of APM solutions.

    I expect internal complexity to increase as the M in APM becomes more management than what it is today which is measurement/monitoring. The application and runtimes needs to become aware of their own behavior and cost (latency, liability, lease) profiles. We refer to this under the banner of CARS: Cost Aware Runtimes & Services.

    The Ultimate Feedback Loop

    Automated Performance Management starts with Software’s Self Observation

    QoS for Applications

  8. I would also recommend reading the following article to understand why the current crop of APM solutions have many of their capabilities touted in their marketing material disabled in production because of a very poor cost benefit analysis.

    Intelligent Activity Metering (IAM)

  9. Jonah Kowall says:

    I asked you for a briefing on the product, but didn’t get one. I’m interested to see more about how the UI works for the users.

    I believe the UI is very important in products, there are way too many difficult to use products on the market.

  10. Hi Jonah,

    Was that last message directed at me? I did respond to your tweet but I never received any follow-up. Sorry for the disconnect. I was wondering what had happened.

  11. Jonah Kowall says:

    Sorry for the disconnect, anyone can brief an analyst (of course if they are interested in speaking with the company) by filling out this form :

    Please do so and we can talk. Would be good to include Will Cappelli on the briefing as well as myself.


  12. Doug McClure says:

    Is this a top down simplification discussion (budget, costs, FTEs), outside in pressure (LoB, end users, ease of use, quality data, etc) or inside out improvement (the monitoring group maturing, optimizing, automating, streamlining)?

    Where is the tipping point for this simplification? At what size and scale does it really apply to? Is that a number of monitored things, a number of end users, a singular technology type or type of functional monitoring? When is it practical for a Fortune 50 type company to simplify things on a broad scale? Can they?

    I see this happening with certain types of companies (relatively young, Internet era companies) for sure but I don’t see it happening on a broad scale in legacy companies that much yet (piggy back on new project/initiative ultimately leading to a stand alone non-integrated tool).

    So in terms of broader based monitoring simplification, what needs to happen with traditional vendors and their approaches to be viable in the future across all types of companies?

    Is this really a ‘best of breed’ versus ‘good enough’ discussion in the long run? I think so in many cases when there is pressure from the business side to simplify. I need 20% of those features w/o paying for the other 80% that come in the product/platform/portfolio…

    Interesting times for sure!


  13. Jonah Kowall says:

    Its more about simplicity of management and implementation. It doesn’t pertain to businesses which are operating at a higher maturity level (for example ones that are implementing BSM). Unfortunately many of the businesses are very immature, and cannot manage the monitoring systems in place.

    Thanks for the comments Doug.

  14. Doug I strongly believe that simplification must first happen at the underlying measurement model & then to surface naturally without too much transformation in the UI. Today’s APM tools seem to create multiple distinct and disconnected information & management models for each new system, technology, sub-system or component resulting in a hodgepodge of tabs and views which are inconsistent and not interconnected mentally or from an interaction perspective. Clearly when you need 10’s-100’s of views just to navigate data collected you have go a problem. A few views with advanced filtering & querying would at least be indicative of a model that is versatile and hopefully universal. That is not the case and its not because we are dealing with different points in the processing from client to backend. This happens even within one tier as engineers of such solutions realize their model can’t model X and are then forced at kludge and confuse everyone.

    I believe the model in the UI and the model in the runtime should be from a user perspective very similar if not identical. Users should first learn how a solution instruments then measures to make sense of the UI and to have confidence and understand of the information and its quality.

    This also important when it comes to cost traceability which I recently touched on in this blog entry:

    What is this universal model? Well you might have guessed its metering (activities=>probes + resources=>meters).

  15. xem phim sex says:

    I asked you for a briefing on the product, but didn’t get one. I’m interested to see more about how the UI works for the users.

  16. believe this trend will continue, and one of m

  17. believe this trend will continue

Leave a Reply

Your email address will not be published. Required fields are marked *

Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.