This just in from the folks at Gartner: DataOps is a market! Wait, haven’t we known this all along? Didn't we name our company DataOps? Read on, and I’ll share my takeaways from this key milestone in one analyst firm’s coverage of the DataOps space.
And if you're in Orlando for the 2023 Gartner Data & Analytics Summit, happening March 20-22nd, swing by Booth #1528 to learn firsthand (I'll be there!) how you can build, test + deploy data products and applications on Snowflake!
On December 5, Gartner delivered its first DataOps Market Guide. What does that mean? Objectively it means that, in their eyes, the DataOps software category is now a a market. That’s a key milestone, as many technologies never get that far. They remain destined to be categorized as a ‘niche’ or a ‘feature’. Still, why does that matter? Surprisingly, it matters quite a bit. Until Gartner “blesses” a new software category and defines it as a Market, many organizations are unwilling to move forward in any significant way in terms of adopting or deploying that software. That may seem unfair, but as those of us in the industry know, Gartner has a disproportionate influence on corporate IT strategy. So, we are thrilled that Gartner has taken the step to define DataOps as a market! Before we get into the findings in the Market Guide, I thought I should take a moment to further explain “The Gartner Progression”, for those of you who might not be familiar with it:
The Gartner Progression
It starts with a category or capability. Gartner can literally spend years delivering research on these—often with little objective analysis. At some point there’s enough critical mass (interest from inquiries and more) to warrant a Market Guide. Some categories/capabilities never get that far, some stall out and don’t go any farther. Assuming we continue to see growth and development in the DataOps market, we could see delivery of a DataOps Magic Quadrant in the next year or so. No commitment at present but all indications seem to point in that direction.
So we’re at an important step—and DataOps live is represented fairly well in this research. Our involvement with this document began in August when they circulated a first cut at defining the capabilities involved. We provided some commentary, suggestions and added several core criteria to their doc—so in effect, we helped make it better!
Key Findings
One of the common questions I hear is “Why DataOps?”. Gartner says that many clients tell them that they see the need to simplify things—fewer niche tools and more automation—with streamlining being the objective. So it seems ironic that many clients start their DataOps journey by trying a DIY approach with one or two niche tools—later coming to the realization that these tools were never intended to work together. The result is that the “house of cards” they’ve assembled takes more effort in care and feeding than the initial effort sought to alleviate. Gartner hints at this by their mention of “Market Cynicism” for standalone tools.
At the highest level, the objectives of DataOps tools are clear and align with what we’ve been positioning from the beginning—to streamline data delivery and improve productivity. Gartner then highlighted five core capabilities: Orchestration, Observability, Environment Management, Deployment Automation and Test Automation. These core capabilities line up very well against ours; honestly, I don’t see any required capabilities that aren’t being delivered in the current DataOps.live platform or our near-term roadmap.
Statistics and a Quote
Let’s look at the numbers according to Gartner:
- By 2025, they assert a 10x productivity edge for DataOps-driven data engineering teams.
- Inquiry data increases on both the DataOps topic and tools—more folks asking about DataOps
- Market penetration for DataOps is still relatively low—1 to 5% of target audiences
I was pleased to see some of our language directly represented in the guide—specifically our use of the term “single pane of glass” not once, but TWICE jumped out at me. We’ve been seeing a lot of noise from point/niche tools that clearly do not fulfill the Seven Pillars of #TrueDataOps—so by that measure we’re the only solution in the space able to deliver that integrated view across data pipeline lifecycles—the “single pane of glass”.
But by far the best part was to see a direct quote from OUR customer, the only customer reference in the guide—Roche Diagnostics is quoted that they were able to do “120 releases per month”—using DataOps.live—versus one every three months a year ago—that’s BIG.
Looking at the Market
The guide broke the market into three categories: Orchestrators, Specialists and Portfolio Players. Clearly DataOps.live is in the latter category but it’s also important to consider that we partner quite well with Data Observability and Governance/Catalog solutions—in effect helping make them better by serving up rich, high-fidelity operational metadata spanning the entire data pipeline lifecycle.
They also called out a category they once referred to as “Servware”—tools that are predicated by a very substantial Professional Services (PS) component—basically providing a toolset that is then implemented by the PS teams. I’ve only seen one vendor in that category, DataKitchen—and their philosophies line up well with our #TrueDataOps’ model—but their approach and market they focus on (cloud platforms vs. cloud data stores) are different than ours.
There were a couple items noticeably absent from the guide. There is literally zero mention of governance in any manner. Does this mean that it’s not important? Absolutely not—quite the opposite. They cite its importance in their definition of DataOps yet it missed the cut here. Another pair of items left out are CI/CD and Agile. Two concepts that are critical to DevOps and by association, DataOps.
What’s in a TAM?
Total Addressable Market—defined as the total market for DataOps products and services is calculated both top-down and bottom up. I particularly like how the top-down model used the application development market as a basis—as this is where we saw DevOps originate. This conveniently led to an identified TAM of $3B. By comparison, the global data integration market is expected to reach $29.21B by 2020 according to Grand View Research. Lots of room to grow!
Conclusion
I think that this is a great start—and up until this, Gartner has been writing and delivering research docs in the absence of a consistent, objective analysis of the core capabilities in our market. So, I believe that this work offers more credibility than some of the prior pieces. It’s a good affirmation for the space but clearly DataOps.live is positioned well both by making the list and seeing our messaging and customer value reflected throughout. According to my sources, the next step will be for Gartner to produce a detailed Market Guide later in 2023 with a detailed analysis of each vendor’s capabilities. From there? Onward and upward—to the Magic Quadrant!
Need some more background on DataOps? Connect with us at DataOps.live—“The folks who wrote the book on DataOps”. Here’s a link to the book—DataOps for Dummies. I’d also recommend reading the Roche Case Study, OneWeb Case Study and be sure to register for our Podcast Series hosted by Kent Graziano, The Data Warrior.
