When is this applicable?

The Scenario Planning Tool (SPT) allows users to explore a machine learning model’s new prediction in the case that one or more of an instance’s features changed. This alternate “scenario” is analyzed by our platform to determine whether the scenario is likely to occur, the difference in the model output, plotting the new point in an interactive graph tool, and displays the waterfall plot for the new scenario.

How-To

• Have a Model and Explainer Asset (See the article: Store and Load a Model Asset)
• Create the XAIDashboard element and display it on a Page

# Get previously created assets
xgb = store_interface.get_model(label="example", name="remaining useful life predictor") #finance
data_train = store_interface.get_dataset(label="example", name="explainer training set")
data_test = store_interface.get_dataset(label="example", name="explainer test set")
explainer = store_interface.get_explainer(label="example", name="ensemble model")

data_train_df = data_train.get_object()
train_mins = data_train_df.min()
train_maxs = data_train_df.max()

bounds = {key: [train_mins[key], train_maxs[key]] for key in data_train_df.columns}

dash = XAIDashboard(xgb, explainer, data_test, "Credit Score", "Probability of Default in next 30 Days (Predicted)",
					"Probability of Default in next 30 Days (Predicted)", bounds=bounds,
					description=XAIDashboard.xai_dashboard_description(),
					expected_title="Average Probability of Default in next 30 days.",
					predicted_title="Predicted Probability of Default in next 30 days.", train_data=data_train_ds,
					encoding=DataEncoding.ONE_HOT)
					
page.add_content_to_section(dash, "Interactive Scenario Planning")

What to Expect (Validation)

The previous code creates the following scatter plot:

Then we can select a point to visualize it in the point editor on the right. The selected point is increased in size and the other points have decreased saturation:

Then we can edit attributes of this instance. Here, we changed the credit score from 544 to 800 to explore how this person’s probability of defaulting would change if they had a higher credit score.

After we press “Predict”, the point is moved to its new location on the plot, showing a decreased probability of defaulting!

We can also press the cog wheel to enable rendering of the waterfall plot.

The waterfall plot appears directly below the SPT and shows the explanation for the newly created scenario.

Below, an additional Infographic gives information about the scenario we created. Here we can see the model’s output for the scenario, the difference from the original point, the number of attributes that we edited that fell out of range of realistic data, and how likely the scenario we created is. We can see that our scenario is unlikely. This is because we significantly increased the person’s credit score, but did not change any other input factors. It's likely that someone with low credit would have other negative attributes, like late payments or a shorter account lifetime. Luckily, the SPT notifies us that this scenario where the credit score was increased without other factors changing is unlikely to occur.