Explainability

Overview

Explainability is a model’s ability to return inference details that explain how it came to a result. The details returned depend on the explainable input type. Some types include image, audio, and video.

This feature is tested during the model deployment. Check out the model container specifications for more details.

Whitebox explainability

This category includes models that have a built-in explainability algorithm.

These models return JSON outputs with mask values for explainable results. Mask values include the prediction results and the explainability results (pixel values that motivate the prediction made by the model).

With whitebox explainability, job results return three objects: a results object, an explainability object, and a model type object. The model type object, modelType, describes the type of data the model can process. The results object, result, contains the prediction results in a classPredictions array. The explainability object, explanation, varies with each model type as described below.

To check if a model has whitebox explainability, send a request to get model details and look for the built-in-explainability feature.

Explanation by model type

imageClassification

In this case, the explanation object contains parameters that provide details on how the model came to the results in a maskRLE array and dimension parameters height and width. The maskRLE follows a column-major order (Fortran order).

JSON

{
  "modelType": "",
  "result": {
    "classPredictions": []
  },
  "explanation": {
    "maskRLE": [],
    "dimensions": {
      "height": "",
      "width": ""
    }
  }
}

textClassification

Text classification models with the explainability feature return JSON outputs with word importance values for explainable results. It includes the prediction results and explainability results (score values that motivate the prediction made by the model).

JSON

{
  "modelType": "textClassification",
  "result": {
    "classPredictions": []
  },
  "explanation": {
    "wordImportances": {},
    "explainableText": {}
  }
}