Market Share or Market Adoption?

End user experience monitoring is the most important dimension of APM. In order for IT to become business and user aligned they must understand user experience, user journey, and their customer and user constituency. Gartner’s recent APM survey “Survey Analysis: End-User Experience Monitoring Is the Critical Dimension for Enterprise APM Consumers” by Cameron Haight finds the same thing; 46% of survey respondents ranked end user experience monitoring as #1. I found the same thing across the thousands of end user calls I took as an analyst.

We’ve seen massive shifts in open source over the last decade, driven by highly robust projects driven by dedicated companies and individual contributors. Open source causes issues when trying to judge market opportunity. The analyst firms focus on looking at revenue, but there is an untracked ecosystem out there.

The best way to analyze market share and market opportunity, and many times what execution should be is to analyze what is actually in use versus what is likely sitting on a shelf collecting dust. Thankfully today’s user experience products can actually be measured by crawling the web, which is exactly what companies like Datanyze, Netcraft, Builtwith, and Similartech do.

The best open source resource for technology usage on the web is the http archive due to the fact that it’s backed by leading companies like Google, but increasingly is being sponsored by APM companies. The data is also publicly available and can be analyzed by anyone. I use this resource often when testing hypotheses about the market as it provides an easy way to get questions answered about technologies in use, trends, and other changes on the web.

This past weekend I decided to do some trend analysis on the “market leading” APM companies out there to see what the adoption trends are. While many vendors sell a lot of software, much of it is unfortunately shelfware. End users increasingly pay for it due to bundles which are common with vendors like IBM, CA, BMC, and HP.

I began by looking at various javascript libraries included on various publicly accessible websites. My interest was in analyzing usage of the following products:

• AppDynamics Browser
• BMC Truesight EUEM
• CA Technologies APM (Specifically BRUM)
• Dynatrace Application Monitoring
• Dynatrace Gomez Real User Monitoring
• Dynatrace Ruxit Real User Monitoring
• HP Diagnostics
• IBM Tealeaf and APM (which has the same Javascript used for both APM and Marketing use cases)
• New Relic Browser

CA Technologies

I was unable to find a consistent way CA products were deployed for JavaScript instrumentation. The CA website itself doesn’t even use their own product (that should tell you something). If someone shares the secret here, I can do the analysis. CA still doesn’t have much in terms of SaaS delivered real user monitoring or transactional APM.

HP Diagnostics

Similar to CA, while there are specific filenames in use for the HP products; there are none which are found in the data within HTTP archive. This either means no one is using the products publicly (which wouldn’t surprise me), or they are not doing it via JavaScript includes which would be a typical best practice. HP still lacks SaaS delivered real user monitoring and transactional APM is in beta.

HP and CA both have customers using packet analysis for APM, but most continue to move off those platforms based on large hardware expenditures (packet aggregation, switching, and server hardware to do the analysis). Additionally these technologies don’t work in modern applications (web or mobile based), especially those behind a CDN, hosted on public cloud, or in highly virtualized or container based infrastructure.

Dynatrace

Dynatrace as a company has it’s own set of issues with loads of overlapping technologies. You can even see in the single use case of end user experience monitoring, they have 3 distinct technologies and product offerings showing the level of portfolio fragmentation. Where will they invest? Which one is the right choice? The portfolio is an increasing mix of overlapping technologies, which must be corrected if they wish to remain competitive with leading vendors with a unified strategy (AppDynamics and New Relic).

Here are the charts showing the remaining vendors analysis. We have no way to differentiate between paid, trial, and freemium offerings here, so please keep that in mind.

New Relic has a massive installed base, since they launched their browser product in 2013. You can see momentum has slowed throughout 2015. In the last earnings call New Relic only had 5,285 customers paying more than $5,000 per year. The base New Relic Browser product costs $2,388 per year for 500,000 page views, which is quite a small site.

If we remove the New Relic numbers we get this graphic.

You’ll firstly notice that legacy vendor technologies are all on the decline, this includes BMC, Dynatrace Gomez, IBM Tealeaf. The clear investment is in the newer technologies such as AppDynamics, Ruxit (which is new, but gaining traction), and some Dynatrace installs.

If anyone wants the queries I used to collect this data, it’s all open data available on Google BigQuery, or you can download the data, and load it into your own MySQL database.

Please leave comments!

Update : 2/6/16

Here are some added players in the final graph. I included Pingdom, Akamai RUM, and SOASTA. Acceleration by SOASTA, slowdowns of Pingdom (Similar to what we're seeing with New Relic), and Akamai is still rather small.

I've also posted the source code for my queries here : https://github.com/jkowall/APM-BigQuery