- Gartner's report discusses several MC+A strategic partners. We acknowledge our bias concerning these partnerships as we choose our partners thoughtfully through an evaluation process like Gartner's report evaluation.
- This article is an editorial assessment and contains opinions.
3 key takeaways for customers and buyers.
- Gartner's scoring does not reflect the reality of the products nor what customers are asking for.
- The Gartner Magic Quadrant report appears to be out of date as soon as it is released.
- Buyers need to do independent research and understand the bias of the report.
Welcome Back Gartner.
It appears that we are approaching the early stages of a return to (albeit a new) normalcy. Cancelled events and conferences from a year ago are now virtual and planning for in-person conferences is on the horizon. A similar sign in our industry is the release of the annual Gartner Magic Quadrant for Insight Engines. The report’s beginnings trace back to the Gartner Magic Quadrant of Enterprise Search report, which tried to capture the many facets (pun) of search technology capabilities available to the enterprise. Gartner skipped releasing the report last year, making this report a much-anticipated update to the previous version of the Quadrant released in late 2019. That report featured MC+A as a strategic partner and reseller of Attivio
Same Magic Quadrant. Same Questions.
Firstly, having read our share of these reports, we provide our assessment of it as third-party validation, with the disclosure that we have an obvious bias towards our partners. That said, this report is challenging to validate as a third party because of our understanding of the products, what Gartner has stated the scoring weights to be, and what Gartner has commented on in its strengths and weakness. Customers who would like a detailed assessment should contact our offices for a search technology assessment. These assessments allow us to fully detail the impact of any of these technologies on your business.
The Value of the Magic Quadrant for Insight Engines?
The Need for Context
However, given that shortly after the last report was released, ServiceNow acquired Attivio after its placement in the upper right of the quadrant. This lead would many customers to question the scoring system and the due diligence Gartner does. We recommend that companies in the market for this technology look at sources outside of Gartner to validate the use case and advise them through this process.
Buyers should also understand that participation in the Quadrant report requires a buy-in. Additionally, some have suggested that the Quadrant report is biased , with Gartner itself stating it is purely opinion . While the grid oversimplifies a complicated task of fitting technology to your use case. Lastly, it’s unclear how Gartner applies weighting to the report criteria. For example, how does Gartner score the Natural Language Processing (NLP) capabilities and relevancy of the system by default and over time? These are critical components that seem to be at best in one to two of the eight categories: completeness of vision.
What change from 2019 to 2020?
Much has changed between the releases of the Magic Quadrant for Insight Engines. It is no surprise that there is a movement by vendors around the quadrant. For example, in 2019, IBM took the top of the scoring in the category of “ability to execute,” with Google surprisingly second. In that report, Lucidworks was first in “completeness of vision” with Coveo in second (both of which were not surprising). The rest of the field ended up clustered in the lower section of the upper quadrant. Some speculation in the industry “sewing circle” was that the clustering intended to give room for scoring improvements over time as this was a new system/field.
The movement in the quadrant should not mean that these vendors did not keep up with the field. Instead, it is more likely that Gartner’s scoring model, which is not open for review, is how redistribution occurs. A comparison in evaluation criteria between 2019 and 2020:
Ability to execute
|Product or Servce||High||High|
|Sales Execution / Pricing||Low||Medium|
|Market Responsiveness /Record||Low||Not Rated|
Completeness of Vision
|Offering (Product) Strategy||High||High|
|Vertical / Industry Strategy||Low||Medium|
Mindbreeze Inspire - Leader
For an example of the scoring question, let’s look at Mindbreeze, which scored incredibly well. While nota partner of MC+A, other agencies (aka competitors of MC+A) who represent Mindbreeze speakvery favorably of them, and their continued presence in the quadrant reflects well on the company and its executive team. However, we noticed Gartner’s comment about Mindbreeze: “the greatest flexibility in deployment options.” Mindbreeze’s solutions are historically delivered in an appliance form factor. Given that it can run on-prem and in their cloud, it is more flexible than on-prem or in the cloud. However, if you search “Hipaa” (HIPPA is a privacy regulation for health care information in the United States and any cloud provider must accommodate for it) on Mindbreeze’s website, you see:
This apparent gap in capabilities in their cloud deployment options is striking. It is not what we should expect from a technology leader.
Additionally, there is not any mention of which public clouds Mindbreeze deploys in, its platform’s integrity, etc. These are particularly important and known to anyone who is ever filled out InfoSec security forms. The other items in the report that raise questions are the inclusion of Gartner’s Peer Insights scores and ‘incorporating third party technology.’ Gartner Peer Insights is the equivalent of Yelp or Amazon reviews, are not officially validated, and are ignored by anyone serious about the technology space.
Does this mean that Mindbreeze should not score well? No, quite the contrary. Our criticism is of Gartner and their scoring criteria. We are aware of many companies having deployed Mindbreeze are pleased with the technology’s business outcome. Mindbreeze is late to the cloud, but their adoption of the cloud is excellent for customer choice. Going to the cloud is incredibly complex, and everyone, including Google after the GSA, has struggled to do it well. The cloud’s inherent complexity (security, availability zones, multi-tenancy, K8, etc.) highlights that customers should perform their due diligence beyond just reading these reports.
Google Cloud Search - Challenger
We were stunned to see Google’s placement in the report in any meaningful way. Similar signs in the field suggest a similar fate for Google Cloud Search to that of the Google Search Appliance, particularly when you consider Google’s lack of focus on launching new products. (See Google Stadia, Google Wave, etc.) (Read: Spring cleaning is upon us) Our observation is that Google Cloud Search lacks marketing behind the product. We are experiencing customers coming to us for guidance, stating that “Google isn’t selling Google Cloud Search.” Is this product going to be 2021 Attivio? If this ends up the case, what does the “Overall Viability” category mean?
Google’s commercial search leadership provides a free (spending billions on Google.com has benefits) perception advantage with potentials customers. “Let’s just call Google” is still a thing. But we fail to see how Cloud Search supports data enrichment, deliveries results to various touch points or allows for query input flexibility on its own. These gaps require you to rely on additional Google Cloud Services. Google Cloud provides impressive capabilities, but many other vendors can also tap into those APIs for their product. (See Mindbreeze for reference when product company leverages third parties)
IBM Watson Discovery - Leader
We have spent a good amount of project time over the last 18 months replacing IBM technology. Like Google, IBM gets a lot of free mindshare with customers (multimillion-dollar TV ad campaigns and a turn on Jeopardy help). On one report, they received top placement in the “ability to execute” category. In the subsequent report, they are top in the “completeness of vision.” Why?
What is likely to change in 2021?
Stay tuned. We are planning follow-ups on this topic—the market opportunity for Insight Engine technology. eCommerce, Customer Support, and Digital Workplace are some of the biggest beneficiaries of budget allocations in the past 12 months. As life resumes to ‘normal,’ the need for better relevancy within the experience of using these systems will continue to increase.
Find the Quadrant Confusing?
MC+A can help you make sense of the rankings.
In conclusion, the Magic Quadrant for Insight Engines can be confusion to align to your business use cases. We are partnered with many of the companies on the Gartner Magic Quadrant and have done hundreds of implementations of search technologies and use cases. This makes us well versed in the technical capabilities and business impact of the technologies in this category. We can help you select the technology platform that is right for your organization. We work in many areas of the world and focus on being non-biased towards products but instead focused on measurable business outcomes.a
Go Further with Expert Consulting
Launch your technology project with confidence. Our experts allow you to focus on your project’s business value by accelerating the technical implementation with a best practice approach. We provide the expert guidance needed to enhance your users’ search experience, push past technology roadblocks, and leverage the full business potential of search technology.