Smart Sampling suggests a subsample of data to perform further analysis and can take into account similarity across data points, parity with the original dataset, and anomalous data points.

Why Is This Important?

Customers that have large datasets want to use all of their data for analysis, however the tools that are currently available have limitations that prevent this, and there is no standard approach to selecting subsamples of data.

How?

Connections

A data source connection is a prerequisite for creating a Smart Sampling Flow. If you do not see the connection you would like to use then select Connections in the side navigation panel to create one.

Steps for running Smart Sampling:

From the Predict Home page open the Smart Sampling Flow
(Optional) Enter a Flow Label and / or Flow Notes
Click the Start Flow button
Select an existing distributed compute platform connection from the dropdown menu and then click the Update button. Note: If you do not see the connection you would like to use then select Connections in the side navigation panel to create one.
Select a data source connection from the dropdown menu and then click the Update button
Enter a table name or SQL query for the Dataset Connection and then click the Update button
The Dataset Preview card will show and updated with the first 100 rows of data
Click the Next button to go to the Dataset Filtering page
Select any columns to keep from the dropdown menu and then click the Update button
Select any columns to filter on from the dropdown menu and then click the Update button
Select any values for any of the columns that were selected in the previous step and then click the Update button
The Filtered Dataset Preview card will show and updated with the first 100 rows of data
Click the Next button to go to the Sampling Configuration page
Select the units for the sample size selection (number of rows or percent to keep) the click the Update button
Use the slider or enter a value for the desired sample size
Select an algorithm that is available from the dropdown menu and then click the Update button
Select additional algorithm specific parameters from the available dropdown(s)
Select the output format(s) for the dataset
(Optional) Provide an output name for the dataset then click the Next button
Select any action for the output format that was selected
- Download the Dataset - select Cancel or Download
- Intelligent Exploration - click the Explore Data button, then Submit. Note: You must have an active connection to Explore
- Save to Database - select an output data source connection from the dropdown menu then click the Update button

Additional Details

The following algorithms are available:

Random Sampling - This sampling method is the fastest option. Use this when you want to get a sample quickly and using a random sample is sufficient.
Spatial Sampling - This sampling method takes longer to run than random sampling because it distributes the sample spatially across the selected columns. Use this when you want to prioritize selecting points in your sample that provide spatial coverage over the selected columns.
Validation-Based Sampling - This sampling method takes longer to run than random sampling because it performs custom-built validation of the sample. Use this when you want to prioritize that the sample and original dataset have similar relationships to a target (or KPI), and have a similar proportion of anomalies.

Using Prebuilt Applications

Using Prebuilt Applications