Introduction

Target drift refers to changes in the distribution of the target variable over time, which can affect the performance of machine learning models.

Basic concepts: 

• ‍Baseline: Users can define the baseline basis on ‘Tag’ or segment of data based on ‘date’.
• ‍Frequency: Users can define how frequently they want to calculate the monitoring metrics
• ‍Alerts frequency: Users can configure how frequently they want to be notified about the alerts

Create drift

Target Drift dashboard creation shares similar inputs with 'Data Drift'. However, unlike 'Data Drift', you must specify the Baseline and Current data parameters, along with the True label, predicted label, and the model type. You also need to select the compute option based on the specific server requirements for the target drift monitor.

Drift Metrics: AryaXAI offers various statistical tests to analyze target drift, including the Chi-square test, Jensen-Shannon distance, Kolmogorov-Smirnov (K-S) test, Kullback-Leibler Divergence, Population Stability index (Psi), Wasserstein distance, and Z-test.

You can learn more about these tests in our wiki section. 

Mixing Multiple Tags: To merge data from different tags, you can easily select multiple tags within the segment (baseline/current).

Target drift monitor

Within AryaXAI, you can easily track target drifts and receive notifications upon detecting any identified drift in your target data. To create a target drift monitor:

• Navigate to the 'Monitors’ tab in ML Monitoring and select 'Create Monitor’.
• Assign a name for the monitor and choose Target drift' as the monitor type. Select the email list to which you want the alerts to be sent.
• You can select the compute option based on the specific server requirements for the target drift task
• Choose the statistical test and set thresholds
• From the dropdown, select the baseline and current true label. Utilize tags to define the baseline and current data 'Current' typically represents production data for tracking drift in your production environment.
• Utilize the date features to further segment your baseline and set the monitoring frequency.

In the event of identified drift, alerts are generated and delivered through both the web application and email based on the specified frequency.

Dashboard Logs

Any new dashboard created for target drift analysis will be listed in the Dashboard Logs, where you can view details such as the baseline and associated tags, the creation date and name of the dashboard, the owner,  baseline and true label, model type and the statistical test used for detecting target drift. In the Actions column, you have options to expand or collapse the dashboard to view or hide detailed information, configure alerts based on the specific dashboard configuration, or delete the dashboard from the logs.

NOTE: The detailed dashboard in the Actions column will only be visible if the status shows as ‘Completed’. If the status shows ‘Failed’ the reason will be specified in the ‘Error’ column.

For all the logs listed here, users can configure automatic alerts based on the dashboard log. This option is available in the 'Alerts' column.

Alerts

Any triggered alert is promptly displayed as a notification in the top-right corner of the web application interface. All notifications can be accessed and cleared from the dedicated tab.

Navigate to the 'Alert' tab adjacent to the 'Monitoring' sub-tab to access a list of triggered alerts. Clicking 'View trigger info' provides detailed insights into the trigger, including current data size, triggered data drift, drift percentage, and more.