Building Intelligent Experiences – Insight Engines vs Microsoft Search
Part 1 – Breaking Down Microsoft Cognitive Services
Artificial Intelligence (AI) and Machine Learning (ML) in business have become as synonymous as salt and pepper on any primetime cooking program you might watch on the Food Network. Regardless of what solution you are ‘cooking up,’ it will likely benefit from applying it. Gartner collectively referrers to the technology section as Insights Engines, stating, “Insight engines augment search technology with artificial intelligence to deliver insights — in context and using various modalities — derived from the full range of enterprise content and data.” Over the past several years, organizations are growing aware of something we have understood for a long while. Search is vital as a component of almost every user interaction that creates value. Search technology, enhanced with ML & AI, is essential to organizations seeking to grow revenue, enable users, mitigate risk, or reduce cost.
The quest to realize this value has led, unsurprisingly, to organizations creating ‘task forces’ and initiatives to rollout ML and AI solutions. This top-down drive is making too many cooks and not enough seasoning, to continue the previous analogy. The general complexity of ML and AI solutions compounded when excited task forces buy into Microsoft’s promise that if they only plug into their grid, insights will abound.
Insight engines augment search technology with artificial intelligence to deliver insights … derived from the full range of enterprise content and data.
Gartner
Microsoft Cognitive Services – The definition of a Data Silo
The current offering of (and a few recently announced) Microsoft Search comes in two forms: services associated with Microsoft 365 (formerly Office 365 or SharePoint Search) and services associated with Azure. The Microsoft 365 Services focused on two things: 1) SharePoint Search and 2) The Office Knowledge Graph. Comparatively, the Azure services fall into three areas: 1) Azure Cognitive Services, 2) Azure Cognitive Search, and 3) Azure Machine Learning.
Looking at these separately, here are some of the high-level challenges your organization might face while working toward that promise of AI on the platform.
Common Challenges with Microsoft’s Search Offerings
| Microsoft Search Common Challenges | Azure Search Common Challenges |
| Difficulty getting non-Microsoft 365 content indexed | No unified data ingestion framework |
| Very few native connectors / connectivity | Additional costs for third-party connectors |
| Additional costs for third party connectors | Primitive security restrictions |
| Slow roadmap (tied to SharePoint) | Not what you would see in enterprise content systems |
| Office 365 Admin Center limit how changes are implemented | Not a cohesive solution (Piecemeal approach) |
| Not Currently Universal | Services alone expensive compared to commercial Insight Engines |
| Minimal Analytics | Lacks Enterprise Integrations |
| No ML Tuning of Signals | BYOM (Build Your Own Models) for Prediction |
| No Custom Recommendations | Cognitive features are consumer-orientated (Is a transformer a car that turns into a Robot or an electrical device?) |
| You have to with the SharePoint Admin Team to configure changes. |
With the high-level challenges out of the way, we will focus on delving more deeply into the two platforms.
Microsoft 365 In-Depth
Digging into the Microsoft 365 platform a little deeper, some boxes allow searching within the ecosystem. These search boxes and their associated autocompletes attempt to realize the low search maturity goal of a unified index with a variety of results retrieved. Microsoft is gradually improving the breadth of this search to its credit, but Yammer and Email are not present in the search within the Office portal.
Both autocomplete and the search provide rich results, such as in these examples but is limited Out of the Box to just 365 content. If you want to bring in other data, you’ll need to:
- Source the connector (There are 100+ third party connectors)
- Work with your Azure administrator
- Work with your Microsoft 365 Administrator
Autocomplete example within SharePoint
Search results example.
Project Cortex
Project Cortex is every knowledge manager’s dream. It provides taxonomy services that promote a structured approach to organizing information, allowing building a knowledge graph (link). Once created, Cortex leverages this organizational graph in three areas:
Topic Cards
Topic cards provide context based on entities it detects from the underlying Knowledge Graph.
Topic Pages
For known entities, Topic Pages are created and can be curated. Topic page are essentially a full page views of a topic card.
Knowledge Center
There is a new Knowledge Center that aggregates this information.
Azure Services In-Depth (not really)
As we’ve stated, the Azure Services are simply a bunch of programming APIs. There is no interface or default user interface for configuring them.
Sample Use Case: Employee Lifecycle Management
Let’s compare the ‘build your own’ style of Microsoft Cognitive Search to off-the-shelf options such as Coveo or Lucidworks. For this example, we will focus our comparison on Coveo, as we’ve recently begun implementing a major US retailer’s digital workplace initiative. To guide the comparison, we will take some of the experiences from that ongoing project and some future recommendations we have given them.
Integrating a unified experience for Employee Lifecycle Management is a common task as part of any Digital Workplace Transformation.
Like many companies today, our client uses various tools from multiple vendors to stitch together the workplace experience. The intranet is built in Adobe Experience Manager (AEM), while there is an Office 365 initiative. Active Directory is in place to populate and manage SharePoint access profiles. The integration complexity did not matter in the least to the 100k+ retail store employees – this tool is critical to task completion. Successful task completion for employees is often minimized by the teams designing and wiring together these experiences. The solution to address these needs and move the organization forward was to leverage AI.
With the help of AI technology, we created an “Employee 360” Portal
Employee 360 Portal Requirements
- Provide curated content from HR on the AEM platform
- Curated courses from the Learning Management System (LMS)
- Access to weekly wrap-sheet of sales and promotions stored in Box.com
- Expert finders for employees looking for:
- Offer experts to help intuitional knowledge
- Offer previous project assets to accelerate work
- Provide Employee Life Cycle hints:
- What training should the employee take
- What articles are people in their role interested in
- Success measured by employee satisfaction (eSat)
- Intelligently identify content gaps
Key Capabilities of an Insight Engine:
- Unify Your Data Access
- Understanding Your Audience
- Provide Recommendations
- Users like you took this training
- Users like you went to this page when looking at this page
- Adjust the ranking based on how you use to the system
- Improve Relevancy: time, place, person, message
- Based on journeys
- Based on successful outcomes
- Based on understanding a particular intent
- Omnichannel / Omni Present user experience
- Exposed in the main search page
- Exposed in widgets on pages (“Recommend training”)
- Exposed other structure queries (“Recommend experts”)
- Rich analytics
- Auditable experience
- Tracking across the various silos
There is no PaaS (Platform as a Service) with these capabilities.
Systems like Microsoft Azure, Google Cloud, and Amazon Web Services have dozens of microservices that perform pieces of these features, but they are far from turnkey. Building on these PaaS environment is a bespoke process versus implementing a commercial Insight Engine. Bespoke is fine for wine and cheese, but building a customized ML/AI solution not only breaks DRY principles by developing something custom that already exists. It opens your organization to additional risks, like your custom security, which will require additional vetting. PaaS requires you to wire together APIs, which requires substantial code, and when launched, your IT team has inherited the support of a completely custom system.
Using a SaaS offering such as Coveo, you are pushing this burden and expense to Coveo’s platform. The platform is SOC2 compliant and is running within some of the largest organizations in the world. Additionally, as an Insight Engine, it’s agnostic to the end-user platform that delivers the insights.
Lastly, while Cortex provides some useful features, it falls short in delivering the Employee 360 use case’s requirements. It significantly improves the underlying graph capabilities available in the system, but it has no concept of a user’s journey or context. It does not provide meaningful analytics. Like many things with Microsoft SharePoint, it enhances your users’ experience in Microsoft 365. However, since most of your experiences (aka systems) are not Microsoft, it starts to become more of a PaaS, and therefore is not an optimal choice, even for those significantly invested in the Microsoft 365 platform.
Check back for our next article in the series.
Learn More
Contact us today to learn how MC+A can help your organization deliver solutions that provide an intelligent experience for your customers and employees.
[hubspot portal=”5146609″ id=”c2194749-4aba-48de-a71d-e05046bab13d” type=”form”]