Overview
The model operations service provides a clear insight into AI performance and usage across the installation. Its APIs report model usage (by model and by user), jobs, queue details, and model drift.
Model Ops monitors model performance metrics and alerts when retraining should be considered.
By monitoring the distribution of data feeding a model and detecting when it changes, you can learn if predictions are different than normal, faster. You can identify why models are making wrong predictions: if the models are getting wrong data or simply are no longer up to date.
The model drift object
{
"modelIdentifier": "...",
"modelVersion": "...",
"modelName": "...",
"lastActiveDatetime": "...",
"driftCalculations": 1,
"avgChiSquaredValue": 0,
"avgPValue": 0,
}
| Parameter | Type | Description |
|---|---|---|
| modelIdentifier | string | A model identifier provided by Modzy. It contains 10 random alphanumeric characters. |
| modelVersion | string | The model’s version number. It follows the semantic versioning format. |
| modelName | string | The model’s name. It may only contain alphanumeric, dash, and underscore characters. |
| lastActiveDatetime | string | The last time the model was used in ISO8601 (YYYY-MM-DDThh:mm:ss.sTZD) format. |
| driftCalculations | number | The number of drift calculations made for the model’s version. |
| avgChiSquaredValue | number | The average chiSquaredValue. The chiSquaredValue determines how likeliness that any differences arose by chance between the baseline dataset and a given inference dataset. |
| avgPValue | number | The average pValue. The pValue measures the strength of the evidence provided by the chiSquaredValue. |