- Gartner's report discusses several MC+A strategic partners. We acknowledge our bias concerning these partnerships as we choose our partners thoughtfully through an evaluation process like Gartner's report evaluation.
- This article is an editorial assessment and contains opinions.
3 key takeaways for customers and buyers.
- Gartner's scoring does not reflect the reality of the products nor what customers are asking for.
- The Gartner Magic Quadrant report appears to be out of date as soon as it is released.
- Buyers need to do independent research and understand the bias of the report.
Welcome Back Gartner.
It appears that we are approaching the early stages of a return to (albeit a new) normalcy. Cancelled events and conferences from a year ago are now virtual, and planning for in-person conferences is on the horizon. A similar sign in our industry is the release of the annual Gartner Magic Quadrant for Insight Engines. The report’s beginnings trace back to the Gartner Magic Quadrant of Enterprise Search report, which tried to capture the many facets (pun) of search technology capabilities available to the enterprise. Gartner skipped releasing the report last year, making this report a much-anticipated update to the previous version of the Quadrant, released in late 2019. That report featured MC+A as a strategic partner and reseller of Attivio.
Same Magic Quadrant. Same Questions.
Every quadrant report generates questions related to how vendors plot to the quadrants. Typically, all vendors who spent significant time with Gartner place well. There is considerable pride for the vendors that do well and questions on ranking criteria for those that did not, as the quadrant ranking reflects their management and vision. Additionally, customers will often use this report alone to shortlist any search-specific ‘search’ projects. So, there are immediate benefits to a strong placement on the Quadrant. The Quadrant coming out later than usual did not reduce questions, even with our lower expectations for the report due to its short absence.
Firstly, having read our share of these reports, we provide our assessment of it as third-party validation, with the disclosure that we have an obvious bias towards our partners. That said, this report is challenging to validate as a third party because of our understanding of the products, what Gartner has stated the scoring weights to be, and what Gartner has commented on in its strengths and weaknesses. Customers who would like a detailed assessment should contact our offices for a search technology assessment. These assessments allow us to fully detail the impact of these technologies on your business.
The Value of the Magic Quadrant for Insight Engines?
This report alone shapes much of the sourcing for enterprises throughout the year. The technology vendors, many of whom are our partners, spend a great deal of time and care to ensure that the Gartner analysts and others are fully aware of their products, company, and partners’ capabilities. Given that attention, along with the nonpublic information shared with analysts, the report is supposed to be the most up-to-date snapshot that the industry has of itself. This information allows purchasing departments to shortlist vendors for their RFPs as they become prescreened quickly.
The Need for Context
Buyers should also understand that participation in the Quadrant report requires a buy-in. Additionally, some have suggested that the Quadrant report is biased, with Gartner itself stating it is purely opinion, while the grid oversimplifies a complicated task of fitting technology to your use case. Lastly, it’s unclear how Gartner applies weighting to the report criteria. For example, how does Gartner score the Natural Language Processing (NLP) capabilities and relevancy of the system by default and over time? These are critical components that seem to be missing from the completeness of vision category.
We recommend that companies in the market for this technology look at sources outside of Gartner to validate the use case and advise them through this process.
What changed from 2019 to 2020?
Much has changed between the releases of the Magic Quadrant for Insight Engines. It is no surprise that there is a movement by vendors around the quadrant. For example, in 2019, IBM took the top of the scoring in the category of “ability to execute,” with Google surprisingly second. In that report, Lucidworks was first in “completeness of vision” with Coveo in second (neither of which were surprising). The rest of the field ended up clustered in the lower section of the upper quadrant. Some speculation in the industry “sewing circle” was that the clustering intended to give room for scoring improvements over time, as this was a new system/field.
The movement in the quadrant should not mean that these vendors did not keep up with the field. Instead, it is more likely that Gartner’s scoring model, which is not open for review, is how redistribution occurs. A comparison in evaluation criteria between 2019 and 2020:
Ability to execute
| Category | 2019 | 2020 |
|---|---|---|
| Operations | Medium | Medium |
| Customer Experience | High | High |
| Product or Servce | High | High |
| Overall Viability | Medium | Medium |
| Sales Execution / Pricing | Low | Medium |
| Market Responsiveness /Record | Low | Not Rated |
Completeness of Vision
| Category | 2019 | 2020 |
|---|---|---|
| Market Understanding | High | High |
| Marketing Strategy | Low | Low |
| Sales Strategy | N/A | Low |
| Offering (Product) Strategy | High | High |
| Business Model | Low | Low |
| Vertical / Industry Strategy | Low | Medium |
| Innovation | Medium | Medium |
| Geographic Strategy | Low | Low |
Mindbreeze Inspire - Leader
For an example of the scoring question, let’s look at Mindbreeze, which scored incredibly well. While not a partner of MC+A, other agencies (aka competitors of MC+A) that work with Mindbreeze speak very favorably of them, and their continued presence in the quadrant reflects well on the company and its executive team. However, we noticed Gartner’s remark about Mindbreeze: “the greatest flexibility in deployment options.” Mindbreeze’s solutions are historically delivered in an appliance form factor. Given that it can run on-premise and in their cloud, it is more flexible than on-premise or in the cloud. However, if you search “HIPAA” (HIPAA is a privacy regulation for health care information in the United States and any cloud provider must accommodate for it) on Mindbreeze’s website, you see:
It is striking that there is no mention of HIPAA compliance is a significant gap in capabilities and specification regarding their cloud deployment options. It is not what we should expect from a technology leader.
Additionally, there is not any mention of which public clouds Mindbreeze deploys in, its platform’s integrity, etc. These are particularly important and known to anyone who is ever filled out InfoSec security forms. The other items in the report that raise questions are the inclusion of Gartner’s Peer Insights scores and ‘incorporating third-party technology.’ Gartner Peer Insights is the equivalent of Yelp or Amazon reviews, are not officially validated, and are ignored by anyone serious about the technology space.
Does this mean that Mindbreeze should not score well? No, quite the contrary. Our criticism is of Gartner and their scoring criteria. We are aware of many companies to have deployed Mindbreeze and are pleased with the technology’s business outcome. Mindbreeze is late to the cloud, but their adoption of the cloud is excellent for customer choice. Going to the cloud is incredibly complex, and everyone, including Google after the GSA, has struggled to do it well. The cloud’s inherent complexity (security, availability zones, multi-tenancy, K8, etc.) highlights that customers should perform their due diligence beyond just reading these reports.
Google Cloud Search - Challenger
We were stunned to see Google’s placement in the report in any meaningful way. Similar signs in the field suggest a similar fate for Google Cloud Search to that of the Google Search Appliance, particularly when you consider Google’s lack of focus on launching new products. (See Google Stadia, Google Wave, etc.) (Read: Spring cleaning is upon us) Our observation is that Google Cloud Search lacks marketing behind the product. We are experiencing customers coming to us for guidance, stating that “Google isn’t selling Google Cloud Search.” Will this product be the 2021 Attivio? If this ends up the case, what does the “Overall Viability” category mean?
Google’s commercial search leadership provides a free (spending billions on Google.com has benefits) perception advantage with potentials customers. “Let’s just call Google” is still a thing. But we fail to see how Cloud Search supports data enrichment, delivers results to various touchpoints or allows for query input flexibility on its own. These gaps require you to rely on additional Google Cloud Services. Google Cloud provides impressive capabilities, but many other vendors can also tap into those APIs for their product. (See Mindbreeze for reference when product company leverages third parties)
IBM Watson Discovery - Leader
We have spent a good amount of project time over the last 18 months replacing IBM technology. Like Google, IBM gets a lot of free mind share with customers (multimillion-dollar TV ad campaigns and a turn on Jeopardy help). On one report, they received top placement in the “ability to execute” category. In the subsequent report, they are top in the “completeness of vision.” Why?
Attivio
In a nontraditional acquisition, ServiceNow acquired the R&D team and the intellectual property of Attivio at the end of 2019. Customers are aware that Lucidworks acquired the remaining assets (namely the customer contracts and staff). The report makes no mention of it, presumably because there is an NDA consideration preventing its disclosure. Like those inappropriate opinions expressed by a subset of our relatives during the Holiday gatherings (back when we could have those safely), the absence of acknowledging what happened to Attivio is painfully awkward.
Why does this matter? At the time of the acquisition, Lucidworks had the assets to maintain and migrate customers. (MC+A is a Lucidworks partner and previously was an Attivio partner.)
What is likely to change in 2021?
Stay tuned. We are planning follow-ups on this topic—the market opportunity for Insight Engine technology. eCommerce, Customer Support, and Digital Workplace are some of the biggest beneficiaries of budget allocations in the past 12 months. As life resumes to ‘normal,’ the need for better relevancy within the experience of using these systems will continue to increase.
Find the Quadrant Confusing?
MC+A can help you make sense of the rankings.
In conclusion, the Magic Quadrant for Insight Engines can be confusing to align to your business use cases. We are partnered with many of the companies on the Gartner Magic Quadrant and have done hundreds of implementations of search technologies and use cases. This makes us well versed in the technical capabilities and business impact of the technologies in this category. We can help you select the technology platform that is right for your organization. We work in many areas of the world and focus on being non-biased towards products but instead focused on measurable business outcomes.
Go Further with Expert Consulting
Launch your technology project with confidence. Our experts allow you to focus on your project’s business value by accelerating the technical implementation with a best practice approach. We provide the expert guidance needed to enhance your users’ search experience, push past technology roadblocks, and leverage the full business potential of search technology.