Top 5 insights from Gartner’s Magic Quadrant for Insight Engines

Improving Search

This article is a cross post. Originally posted at

Gartner has migrated the “Gartner Magic Quadrant for Enterprise Search” to the “Magic Quadrant for Insight Engines”. This migration follows a trend set by Google who announced the effectively its exit to the enterprise search space later to reveal a insight engine embedded within gSuite called Cloud Search. Gartner describes an Insight Engine as:
“Insight engines provide more-natural access to information for knowledge works and other constituents in ways that enterprise search has not.”
Which is somewhat false because insight engines are typically built on a similar technology stack that enterprise search is built on. Regardless of the name change some fundamental changes in the market exist and some key insights from this in my few are:

1 – We are moving beyond keyword Search

At MC+A we made some predictions for 2017 for the cognitive area in an insight article published earlier this year. I have heard a few times in the past couple of months that the go forward strategy is that the CMS has keyword search and therefore there isn’t a need for an insight engine.

A keyword based search index makes retrieving a known document fairly easily and efficient. If you do not know the combination or issue the wrong keyword then you are not going to be successful. Contrary to this, an insight engine takes some many more scores beyond keyword matches.

2 – Your CMS having a Lucene based index is not going to cut it

As noted in numerous articles, CMS’ are designed for the people who buy them and not for the people who use them. The search feature is not something that is typically tested properly because most searches are in the long tail and are hard to fully understand. A simple reverse index of keywords matched with some stems is not the experience that consumers of your system want.

3 – Natural Language Processing is the gateway to real time interactions

Natural Language is a thing.
Search and Assist can take cues from navigation paths, signals processing and other ML techniques. It’s not going to be as useful as someone telling you exactly what they are looking for.

4 – We’re going to need a bigger boat

Needing a bigger boat
Machine Learning and AI algorithms need lots of computing cycles. Disks are slow even if they are SDD and memory is fast. Analytics and Signals are generated at an exponential rate to search queries. Additionally indexing your content with NLP can be quite consuming. (Cloud services and SAAS offerings obfuscate this from consumers.)

5 – When it’s all said and done, there will still be something that looks like a search box

Count the number of people outside of an Apple commercial who use Siri and then think how awful the office will become is people repeating endlessly their voice requests. As far as Human Computer Interfaces go, the search box is here to stay near term. Searchengineland predicted its death last year.

If you read the article, you’ll note that most of the searching is still done through a box although with an advance assist feature which allows for autocomplete.