Why you need to explain machine learning models | Google Cloud Blog (2023)

Many companies are actively using AI today or plan to integrate it into their future strategies.76%of organizations are now prioritizing AI and ML over other initiatives in their IT budgets, and the global AI industry is expected to continue catching up$260 billionby 2027. However, as AI and advanced analytics become more ubiquitous, the need for greater transparency into how AI technologies work will be critical.

In this post, we'll explore why Explainable AI (XAI) is essential for mainstream AI adoption, common XAI practices, and how Google Cloud can help.

Why you need to explain ML models

AI technology suffers from the so-called black box problem. In other words, you may know the question or the data (the input) but lack insight into the steps or processes that deliver the final answer (the output). This is particularly problematic with deep learning approaches and artificial neural networks, which contain many hidden layers of nodes that "learn" through pattern recognition.

Often stakeholders are reluctant to trust ML projects because they don't understand what they are doing. It's difficult for decision makers to relinquish control to a mysterious machine learning model, especially when it's responsible for making critical decisions. Artificial intelligence systems make predictions that have profound implications, and in some industries like healthcare or self-driving cars, they could mean the difference between life and death.

It is often difficult to get support that a model can be trusted to make decisions, let alone make them.improvethan a human can, especially when there is no explanation as to how that decision was made. How did the AI ​​model predict or make a decision? How can you be sure that the algorithms have no bias? Is there enough transparency and interpretability to trust the model decision?

Decision makers want to know the reasons behind an AI-powered decision so they can be sure it's the right decision. In fact, after aPwC Research, the majority of CEOs (82%) believe that AI-supported decisions must be explainable in order to be credible.

What is explainable AI?

explainable AI(XAI) is a set of tools and frameworks that you can use to understand how your machine learning models make decisions. This is not to be confused with depicting a complete step-by-step deconstruction of an AI model, which can be nearly impossible when trying to track down the millions of parameters used in deep learning algorithms. Rather, the purpose of XAI is to provide insight into how the models work so that human experts can understand the logic of decision-making.

When applied successfully, XAI offers three key benefits:

1. Increase confidence in ML models
When decision makers and other stakeholders have more insight into the results of an ML model, they are more likely to trust AI-powered systems. Explainable AI tools can be used to provide clear and understandable explanations of the reasoning that led to the model's outcome. Assuming you are using a deep learning model to analyze medical images such as x-rays, you can use explainable AI to create bump maps (i.e. heatmaps) that highlight the pixels used to create the diagnosis. For example, an ML model that classifies a fracture would also highlight the pixels used to determine if the patient has a fracture.

2. Improves general troubleshooting
Explainability in AI can also allow you to debug a model and troubleshoot how it works. Let's imagine your model is able to identify animals in images. Over time, you find that the model continues to classify images of dogs playing in the snow as foxes. Explainable AI tools make it easy to figure out why this error keeps popping up. When you look at the explainable AI models you use to show how a prediction is made, you see that this ML model uses the background of an image to differentiate between dogs and foxes. The model learned incorrectly that dogs serve as a domestic background and snow in an image means that the image contains a fox.

3. Eliminate bias and other potential AI pitfalls
The XAI is also useful for identifying sources of distortion. For example, you could have a model to detect when cars are turning left illegally. When asked what the injury is based on in an image, you find that the model detected bias in the training data. Instead of focusing on cars making an illegal left turn, look to see if there's a pothole. This impact could be caused by a skewed data set containing a large number of images captured on bad roads, or even real bias where a ticket is more likely to be issued in an underfunded area of ​​a city.

Where does explainability fit into the ML lifecycle?

Explainable AI shouldn't be an afterthought at the end of your ML workflow. Instead, explainability must be built in and applied at every step, from data collection and processing to model training, evaluation, and service.

There are a few ways to build explainability into your ML lifecycle. This could mean using explainable AI to identify imbalances in datasets, ensuring model behavior satisfies certain rules and fairness metrics, or showing model behavior locally and globally. For example, if a model was trained using synthetic data, you need to ensure that it behaves the same way using real data. Or, as we discussed before with deep learning models for medical imaging, a common way of explainability is to create heat maps to identify the pixels used for image classification.

Another tool you can use is segmented assessments of machine learning model performance. according to ourAI principles, you must avoid creating or reinforcing unfair prejudice. AI algorithms and datasets can often reflect or reinforce unfair bias. When you find that a model isn't working well in a small minority of cases, it's important to address any issues of fairness.

Split assessments allow you to explore how different parts of a data set might affect your results. Image models allow you to search for different images based on factors such as poor lighting or overexposure.

We also recommend creating model cards, which can help explain any limitations and tradeoffs you may need to make to improve performance, and then provide a way to test how the model works.

Explainable AI methods

When we talk about explainable AI methods, it is important to understand the difference between global and local methods.

• a world methodis to understand the general structure of how a model makes a decision.

• a local methodis to understand how the model made decisions for a single instance.

For example, a global approach might be to look at a table listing all the resources used, ordered by their overall importance to a decision. Feature importance tables are commonly usedexplain structured data modelsHelping people understand how specific input variables affect the final output of a model.

But how about explaining how a model makes a decision for an individual prediction or a specific person? This is where local methods come into play.

For the purpose of this post, we'll cover local methods based on how they can be usedExplanation of model predictions on imagery.

Here are the most common explainable local AI methods:

• Model Agnostic Interpretable Local Explanation (LIME)
• Kernel Shapley Additive Explanations (KernalSHAP)
• Integrated gradients (IG)
• AI Explainable Explanations (XRAI)

BothLIME and Kernal ShapSplitting an image into patches that are randomly sampled from the preview to produce a series of perturbed (i.e., modified) images. The image will look like the original, but some parts of the image have been erased. The disturbed images are then fed to the trained model and asked to make a prediction.

In the example below, the model would be asked: Is this picture a frog or not?

The model would then provide the probability that the image is a frog. Based on the patches selected, you can rank the importance of each patch by ultimate probability. Both methods can be used to explain local meaning when determining whether the image contained a frog.

built-in gradientsis a technique used to assign importance values ​​based on the gradients of the final output. IG takes the reference images and compares them to the actual pixel value of the images that contain the information that the corresponding model was designed to identify. The idea behind this is that if the image contains what the model was trained for, the accuracy of the value will be improved. It helps determine how much a gradient changes from the reference image to the point where it makes a prediction, and provides an attribution mask that helps determine what the image uses to classify an image.

XRAIis a technique that combines the three methods above and combines patch identification with built-in gradients to show prominent areas that have the greatest impact on a decision, rather than individual pixels. Larger regions with this approach tend to give better results.

Another new method we're integrating with Google Cloud isTracIn– a simple, scalable approach that estimates the impact of training data. The quality of the ML model training data can have a major impact on a model's performance. TracIn tracks mislabeled examples and outliers from multiple data sets and helps explain predictions by assigning an influence value to each training example.

When training a model to predict whether images contain zucchini, observe the gradient changes to see what reduces loss (defender) and what increases loss (opponent). With TracIn you can see which images the model can use to learn to identify a zucchini and which ones are used to distinguish what is not a zucchini.

Using explainable AI on Google Cloud

We start Vertexexplainable AIto help data scientists not only improve their models, but also provide insights that make them more accessible to decision makers. Our goal is to provide a set of useful tools and frameworks that can help data science teams in many ways, such as: B. Explain how ML models reach a conclusion, debug models, and fight bias.

With Vertex's Explainable AI Platform, you can:

• Interpretable and inclusive AI design. Build AI systems from the ground up with Vertex's explainable AI tools designed to detect and fix biases, anomalies, and other gaps in data and models. WithAI explanations, data scientists can useAutoML tables,Apex predictions, Ofnotebooksto explain how much a factor contributed to the model's predictions, thereby improving the datasets and model architecture. HeTool what if?It allows you to examine model performance across a variety of features, tweak strategies, and even manipulate individual data point values.
  

• Deliver ML models safely by providing user-friendly explanations. By deploying a model in AutoML Tables or Vertex AI, you can mirror patterns found in your training data to get a real-time assessment and prediction of how different factors affected the final result.

• Streamline model management with performance monitoring and training. With you can easily monitor forecasts and provide basic information labels for forecast inputscontinuous developmentspecial feature.Vertex data labelcompares predictions with truth flags to add feedback and optimize model performance.

AI remains an exciting frontier that will continue to shape and inspire the future of businesses across all industries. However, for AI to reach its full potential and gain broader adoption, everyone involved, not just data scientists, must understand how ML models work. That's why we remain committed to ensuring that the future of AI serves everyone, whether they are customers, business users or decision makers.

Next Steps

Learn to offer explanations along with predictions by doing thisnotebook jupyteron the artificial intelligence cloud platform. step by stepinstructionsThey are also available on Qwiklabs. And if you're interested in what's next for machine learning in the next five years, check out oursApplied Machine Learning Summitto hear Spotify, Google, Kaggle, Facebook and other leaders in the machine learning community.