Jonah Kowall

A member of the Gartner Blog Network

Jonah Kowall
Research Vice President
3.5 years with Gartner
20 years IT industry

Jonah Kowall is a research Vice President in Gartner's IT Operations Research group. He focuses on application performance monitoring (APM), Unified Monitoring, Network Performance Monitoring and Diagnostics (NPMD), Infrastructure Performance Monitoring (IPM), IT Operations Analytics (ITOA), and general application and infrastructure availability and performance monitoring technologies. Read Full Bio

Coverage Areas:

Application Monitoring is not Application Performance Monitoring (APM)

by Jonah Kowall  |  February 14, 2014  |  18 Comments

There is a common issue I deal with when speaking to end users trying to monitor applications. This confusion is partially created by vendors who would like to position themselves in the hot APM market, yet they clearly don’t enable performance monitoring. These vendors are slowly starting to correct the messaging, but many have poor market understanding, and continue to confuse buyers.

There are two types of monitoring technologies, one is availability monitoring and the other performance monitoring. Before embarking on a performance monitoring journey (this applied to both application performance monitoring and network performance monitoring) there should be a good foundation of availability monitoring. Availability monitoring is the easier of the two, it’s inexpensive, effective in catching major issues, and should be the staple of any monitoring initiative. We recommend unified monitoring tools (See Post : http://blogs.gartner.com/jonah-kowall/2013/11/12/unified-monitoring-note-presentation-and-client-interest/) to handle availability monitoring across technologies with a single offering.

When looking at server monitoring tools, they do more than monitor the server and OS components, but also handle the collection of data from instances of applications on the OS instances. The data collected includes metrics and often times log data which shows major issues in application availability or health. This is often what people are looking for, and many vendors call these requirements “APM”, but that’s incorrect. We call this server monitoring and/or application instance monitoring. These are availability tools and not performance monitoring tools.

The area APM tools differ from server monitoring tools is in multiple ways. APM tools live within the application and provide end user experience data from the user through the distributed application components. They are able to monitor and trace transactions across tiers of the application. Similarly other tools which monitor application performance can reside on the network, while these don’t have the level of granularity when tracing transactions and getting internals of applications they certainly can detect performance deviations of application components and often times can handle other application technologies.

Hopefully this helps clear things up, and please reply here or contact me on Twitter @jkowall

18 Comments »

Category: APM IT Operations Logfile Monitoring NPM     Tags:

18 responses so far ↓

  • 1 Application Monitoring is not Application Performance Monitoring (APM) | All that Cuteness   February 14, 2014 at 6:28 pm

    [...] By Jonah Kowall [...]

  • 2 Lionel Johnstone   February 14, 2014 at 7:34 pm

    Interesting post. Thanks. My view is that the enterprise IT landscape is becoming more diverse with mission critical applications being delivered out of not only the enterprise data centre but also 3rd party data centres (Salesforce, Workday etc). With this in mind, surely the only way to clearly understand how IT is impacting the end-user’s ability to do his or her job is to monitor from the point of view of that end-user.

    Installing APM and other monitoring tools in the corporate dc tells only part of the story. The only way to understand the real end-user experience is to monitor performance on all the devices he or she uses across all of the activities carried out in all of the apps being consumed. Is that something that you see emerging at all?

  • 3 Jonah Kowall   February 14, 2014 at 8:04 pm

    I know your angle there Lionel. Yes EUE is important, but its by no means the only area that APM tools must address. We have a five dimensional model, and EUE is one dimension. Within most I&O organizations they are responsible for delivery of applications from end to end, and not from end… Hence they need EUE integrated with other capabilities.

    The cost for desktop EUE implementations is high, and assumes the organization has other APM tools to provide the missing perspective. This is not feasible for most organizations due to budgetary constraints or other elements affecting the desktop landscape such as BYOD. Additionally the desktop EUE tools do not help manage the business delivery of applications to their external customers which is extremely critical.

    Just an example, I do not run Gartner software on my compute devices, hence a desktop focused measurement tool would not work for me or our current BYOD policies.

  • 4 GP   February 15, 2014 at 12:04 pm

    Jonah – thanks for helping clarify the difference. In all of our implementations we have seen APM help “instrumentize” Business SLA’s associated with IT applications.

    Arguably the SLA reporting construct in many IT organizations consists of manual collection and processing of data on excel spreadsheets etc. This delays getting that data into hands of decision makers.

    APM offers an automated data collection and translation engine which is extremely valuable even more so given that now we are able to actually see the cost of our IT through AWS IaaS for example. This means that I can use the APM technology to derive insights like – it cost me $5,000 to process 1,000 customer transactions with a sub-5 second response time and generated $15,000 in revenue. APM helps us tie that various aspects of the overall system together and make better decisions.

    We have come up with a simple framework that categorizes the metrics and SLA’s into 4 buckets SysOps, AppOps, SecOps and FinOps.

    Thanks again for your blog post on APM.

  • 5 Jonah Kowall   February 15, 2014 at 3:31 pm

    GP, agreed that SLAs most use and track are built on availability, which obviously has it’s flaws since the users expect services to not only be available but be performant.

    I’m not following your logic on the cost side of thing most of the techniques I’ve seen which try to build cost models for AWS tend to relate cloudwatch metric data with usage. I haven’t seen anyone use an APM tool to build costing models. Have you actually done that for your clients with APM tooling?

  • 6 GP   February 18, 2014 at 2:32 am

    Jonah – we are working on a Continuous Process Optimization (CPO) program where we work with the customer to review key operational parameters to optimize performance given that we use IaaS/AWS. We are currently in the middle of defining the cost model based on key customer metrics (not the traditional systems oriented ones).

  • 7 Kevin Bailey   February 18, 2014 at 8:00 pm

    Jonah – Very interesting comments. If you had to segment one from the other could it be simplified to APM versus AANPM – Application Aware Network Performance Monitoring?

  • 8 Jonah Kowall   February 19, 2014 at 1:38 am

    Not quite, as the AA-NPM tooling is specific to network subject matter experts.

  • 9 MK   February 23, 2014 at 4:41 am

    Jonah – biggest problem I have seen with apm tools are they live in small space and not provide a holistic view of apps+network performance affecting business. In other words a true convergence with Business Trx performance with Apps+Network performance. is your evaluation criteria segregate the 2 world or will there be any one true convergence in future. ?

  • 10 Jonah Kowall   February 23, 2014 at 11:00 pm

    Michael you will be interested in my next research note, but modern or leading APM tools definitely handle business transaction monitoring.

  • 11 Mike   March 5, 2014 at 12:48 pm

    Hi Jonah,

    Thank you for trying to clear this up. We’d like to help spread the right message as well. I have a couple questions regarding:

    Comment: APM tools live within the application and provide end user experience data from the user through the distributed application components.
    Question: Can you give a couple examples of what “end user experience” you are referencing? I was also confused by “distributed application components”? How would we see these implemented in a commercial product? Can you give some metric examples?

    Comment: They are able to monitor and trace transactions across tiers of the application.
    Question: To me, this is sort of vague. If you could point me to something that gives specifics that would be great. An actual example would be helpful.

    Again, thank you for your efforts.

  • 12 Jonah Kowall   March 6, 2014 at 2:42 pm

    APM tools meet our 5 dimensional criteria and provide these technologies and visibility. I would get your hands on a copy of our APM Magic Quadrant and you’ll better understand how these tools that live within the application runtime work versus those which collect application metrics from the OS (as you guys do). There is a lot of confusion where buyers compare tools which monitor servers and application instances and those which monitor users, transactions, and distributed application environments.

  • 13 Muhammad Saeed   May 14, 2014 at 5:34 pm

    Would you please differentiate the difference between application performance management and database performance management?

  • 14 Jonah Kowall   May 15, 2014 at 2:33 pm

    Muhammad, we don’t use the words “management” for the products out there, since they essentially monitor. We do consider database management tools separate since they do a lot more than monitoring. The products which focus on database performance monitoring are a subset of APM, but APM tools are concerned with the complete application (what we cover in our 5 dimensional model) and not just a single component (database).

  • 15 Irit Gillath   June 24, 2014 at 4:07 pm

    We also see a great confusion between code-profiling level APM and unified application monitoring. We see different users looking for different things due to to the lack of clear definitions or the evolution that application support is going through. We would love to talk to you further about it

  • 16 Jonah Kowall   June 30, 2014 at 1:18 am

    Sure, if you are a client Irit I’d be happy to better explain that, but we’re quite clear that server monitoring includes application instances for health, while APM tools focused on distributed applications, transaction visibility, and end user experience monitoring.

  • 17 Manisha   July 7, 2014 at 10:58 am

    Appreciate your effort in explaining and clarifying this concept Jonah.

    In WSO2, they have a product for Business Monitoring namely WSO2 Business Activity Monitor, http://wso2.com/products/business-activity-monitor/
    There, it is capable of doing all the functionalities that you have mentioned in your post. It can collect data from the server instances by setting up data agents in those particular servers. Once the data are collected, the BAM server is capable of analyzing data performing a hive query on top it using Hadoop processing. Finally the data is shown in a dashboard with a set of gadgets.

    Here, the user can define their own gadgets according to their own KPIs, so as to capture the number of requests coming to a particular server or to a particular service or an application etc.

    This is a good example for Application Performance Monitoring in my understanding.

  • 18 Jonah Kowall   July 7, 2014 at 5:10 pm

    If you want to build your own tools, which most do not want to do. APM tools are primarily used to solve tangible problems not just explore data collected. I’ve never spoken to your clients nor have your briefed us. I was not aware of this solution.

Leave a Comment