What is Human-Guided Machine Learning?

Editor’s Note: This post was originally published in April 2019. We’ve updated the content to reflect the latest information and best practices so you can stay up to date with the most relevant insights on the topic.

As artificial intelligence (AI) continues to transform modern businesses, one concept remains a critical factor to deliver the actionable insights companies need to make better decisions: human-guided machine learning. By uniting the precision and efficiency of AI with human expertise, human-guided machine learning plays a pivotal role in improving data quality, resolving ambiguities, and ensuring that data management processes align with organizational goals. But what exactly is human-guided machine learning, and how does it complement both AI and master data management (MDM)? 

There are three general types of machine learning: supervised, unsupervised, and reinforcement learning. Human-guided machine learning is a type of supervised learning, which uses a set of human-labeled training data to develop a model. In supervised learning, the algorithm learns a set of inputs along with corresponding correct outputs. The training data used to create a machine learning model is assumed to be ground truth, meaning that its validity is not questioned. For that reason, the importance of starting with high-quality data cannot be overstated. After all, if the data you use to train the ML model is inaccurate, incomplete, inconsistent, or outdated, your results will be flawed. As well, to avoid introducing incorrect or biased results, the model must still be tested for accuracy before it can be deployed. 

There are also subsets of supervised learning known as active learning, or semi-supervised learning, where the machine learning model is improved with each additional correction or piece of information collected. This is where humans come in.

Human-guided machine learning is a process whereby subject matter experts accelerate the learning process by teaching the technology in real time. For example, if the machine learning model comes across a piece of data it is uncertain about, a human can be asked to weigh in and give feedback. The model then learns from this input, and uses it to make a more accurate prediction the next time. Human-guided machine learning works from the bottom up by first using algorithms to conduct the heavy lifting of identifying relationships within the data, and engaging humans when necessary for training or validation. This means that, inevitably, the amount of time a human needs to spend performing a specific task will decrease as the machine learning accuracy increases.

This is important, mainly because of the sheer volume and variety of datasets that enterprises are tasked with managing today. Given the right solution, mastering large, diverse datasets through machine learning is significantly easier than creating and managing a network of custom rules and formulas, which is the approach of traditional MDM solutions. And, with human-guided machine learning, technical or data science knowledge isn’t even required. All that’s needed are subject matter experts who know the ins and outs of your data, and can tell the model whether, in fact, two similar records in your database are actually the same entity

There are numerous benefits to human-guided machine learning, including the following:

• Trust: The involvement of subject matter experts in the training and validation of datasets produced by human-guided machine learning means that businesses can have confidence in the reports built on resulting datasets.
• Speed: Since human-guided machine learning puts the heavy lifting on machine learning algorithms, data can be ready in a matter of days or weeks instead of months (or years).
• Scale: With deterministic rules-based approaches to unifying data, there are scale limitations: humans can only write and understand so many rules. Machine learning eliminates this issue and enables you to leverage all of your organizational data.

To learn more about how Tamr uses human-guided machine learning to improve entity resolution and data unification, please schedule a demo. And to begin your journey towards achieving the clean, connected, and enriched data your business demands, please download our latest ebook, The MDM Journey: From Trustworthy Data to Operationalization.