The Scenario Planning Tool (SPT) allows users to explore a machine learning model’s new prediction in the case that one or more of an instance’s features changed.

This alternate “scenario” is analyzed by the Virtualitics AI Platform to:

• Determine whether the scenario is likely to occur
• Identify the difference in the model output
• Plot the new point in an interactive graph tool
• Display the waterfall plot for the new scenario

Using the Scenario Planning Tool (SPT)

The example code below incorporates previously-created Model and Explainer Assets and creating the XAIDashboard element to display on a Page.

# Get previously created assets
xgb = store_interface.get_model(label="example", name="remaining useful life predictor") #finance
data_train = store_interface.get_dataset(label="example", name="explainer training set")
data_test = store_interface.get_dataset(label="example", name="explainer test set")
explainer = store_interface.get_explainer(label="example", name="ensemble model")


data_train_df = data_train.get_object()
train_mins = data_train_df.min()
train_maxs = data_train_df.max()


bounds = {key: [train_mins[key], train_maxs[key]] for key in data_train_df.columns}


dash = XAIDashboard(xgb, explainer, data_test, "Credit Score", "Probability of Default in next 30 Days (Predicted)",
                    "Probability of Default in next 30 Days (Predicted)", bounds=bounds,
                    description=XAIDashboard.xai_dashboard_description(),
                    expected_title="Average Probability of Default in next 30 days.",
                    predicted_title="Predicted Probability of Default in next 30 days.", train_data=data_train_ds,
                    encoding=DataEncoding.ONE_HOT)
 
page.add_content_to_section(dash, "Interactive Scenario Planning")

What to Expect

The previous code creates the following Scatter Plot:

The user can then select a point to visualize it in the point editor on the right. The selected point is increased in size and the other points have decreased saturation.

The user can then edit attributes of this instance. Here, the credit score was changed from 544 to 800 to explore how this person’s probability of defaulting would change if they had a higher credit score.

Once the “Predict” button is pressed, the point is moved to its new location on the plot, showing a decreased probability of defaulting.

If the user clicks the cog wheel on the top right of the point editor panel to, they can enable rendering of a waterfall plot. The waterfall plot appears directly below the SPT and shows the explanation for the newly created scenario.

To the left, the user will see an additional Infographic gives information about the scenario we created. Here, the user can see the model’s output for the scenario, the difference from the original point, the number of attributes that we edited that fell out of range of realistic data, and how likely the scenario we created is.

In this case, we can see that our scenario is unlikely. This is because we significantly increased the person’s credit score, but did not change any other input factors. It's likely that someone with low credit would have other negative attributes, like late payments or a shorter account lifetime.

Luckily, the SPT notifies us that this scenario where the credit score was increased without other factors changing is unlikely to occur.