Machine Learning Introduction

Prev Next

Before you start

Read this section before setting up a Machine Learning data activity. The choices you make here shape your setup from the start.

It helps to understand your data before you begin: how values are spread across each column, which columns are related, and whether any values are rare or missing. This page covers the basics and should help you profile your data and pick sensible starting parameters for your first training run.

Expect the first run to be a starting point rather than the finished result. Once you've generated synthetic data, compare it with your source data, adjust the parameters based on what you find, and train again.

What is machine learning?

In the Curiosity Machine Learning data activity, machine learning means training to produce a model on your existing data so that it learns the data's statistical patterns:

  • How values are distributed in each column

  • How columns relate to each other

  • How related tables connect

Once trained, the model generates new synthetic records that follow those same patterns or characteristics. The records are newly generated rather than copied from the source, so you get realistic, representative test data without exposing the real records it learned from.

Generative Engines

The generative engine is the machine learning algorithm that learns the patterns in your training data. Training produces a model, which you then use to generate new rows with the same characteristics. The Generative Engine list shows every engine available to you.

Pick one based on what your data looks like:

Engine

Type

Data size

Use it when

bayesian_network

Statistical

Small (up to around 25,000 rows)

You want speed, and to see which columns depend on which

tvae

Neural (variational autoencoder)

Medium to large (over 25,000 rows)

Your data is mostly numeric, smooth distributions and complex numeric relationships

ctgan

Neural (GAN)

Medium to large (over 25,000 rows)

Your data is heavily categorical, has many distinct labels, or has very imbalanced categories

adsgan

Neural (GAN) with privacy

—

Privacy is a priority. It balances how realistic the data is against how easily a synthetic record could be linked back to a real person

arf

Tree-based (adversarial random forests)

—

You want high-fidelity tabular data and the neural engines struggle to converge on your data

ddpm

Neural (diffusion)

—

You want high-quality output on data with many columns. It learns to reverse a process of adding noise to data

dpgan

Neural (GAN) with differential privacy

—

You need a differential privacy guarantee. Noise is added during training, so no single record's presence in the training data can be inferred

pategan

Neural (GAN) with differential privacy

—

You need strong differential privacy guarantees. It trains several discriminators on separate parts of the data and learns from their combined, noisy votes

nflow

Neural (normalizing flow)

—

You want a model that learns the exact probability of each row, by transforming a simple distribution into your data's distribution

privbayes

Statistical (Bayesian network with differential privacy)

—

You want a bayesian_network-style dependency graph with a mathematical privacy guarantee

rtvae

Neural (variational autoencoder)

—

A variant of tvae. Use it in the same situations: small to medium tables where GAN-based engines are unstable

timevae

Neural (variational autoencoder) for sequences

—

Your data is a sequence over time, and you need to keep its trends and seasonal patterns

timegan

Neural (GAN) for sequences

—

Your data is a sequence over time, and you need to keep how values change from one time step to the next

bayesian_network models your data as a graph of column dependencies. It's fast, and the relationships it learns are explicit: each column depends on specific other columns. It can struggle with very wide tables.

tvae compresses each row and learns to rebuild it. It doesn't use adversarial training, so it's usually more stable than ctgan. It tends to produce average values, so outliers can get smoothed away.

ctgan trains two networks against each other. The generator makes rows and the discriminator tries to tell them from real ones. It specifically targets under-represented categories, but it's slower to train and harder to tune.

adsgan is a GAN that adds a privacy penalty during training, so synthetic rows are harder to link back to real individuals. Raising the penalty gives more privacy at some cost to realism.

arf (adversarial random forests) uses trees instead of a neural network. It keeps splitting the data until real and synthetic rows can't be told apart.

ddpm is a diffusion model. It learns to reverse a process that gradually adds noise to the data, and can give high-quality output on data with many columns.

dpgan is a GAN with differential privacy. It adds noise during training so that no single record's presence can be worked out from the model.

pategan gives differential privacy in a different way: several teacher models are trained on separate parts of the data, and the generator learns from their combined, noisy votes.

nflow (normalizing flows) transforms a simple distribution into your data's distribution through a series of reversible steps, which lets it calculate the exact probability of each row.

privbayes is a Bayesian network with differential privacy. It builds a dependency graph like bayesian_network, then adds noise to the probabilities to give a mathematical privacy guarantee.

rtvae is a variant of tvae whose training is more resistant to outliers.

timevae and timegan are built for sequential data, such as readings or transactions recorded over time. timevae uses a variational autoencoder and timegan uses a GAN. Both learn how values change from one time step to the next, including trends and seasonal patterns.

All engines

Parameter

What it does

When to change it

encoder_max_clusters (data_encoder_max_clusters on tvae)

How finely continuous columns are divided before training

Raise it (for example from 10 to 20) if values come out too coarse, or if less common values go missing

sampling_patience

How many attempts the engine gets to produce a valid row before it fails

Raise it if generation fails with "failed to meet constraints"

bayesian_network - Training Guide

Parameter

What it does

When to change it

struct_learning_search_method

How it searches for column relationships

Choose to suit your data. tree_search gives each column at most one parent; mmhc, hillclimb and pc allow several; exhaustive suits a handful of columns. See the Bayesian Training Guide

struct_learning_score

How each candidate structure is scored: bic or k2

bic keeps the structure compact. k2 can find more links but takes longer. Ignored by tree_search

struct_max_indegree

Maximum number of columns any one column can depend on

Higher is more accurate but much slower. For tables with more than 20 columns, keep it at 2–3

struct_learning_n_iter

Maximum number of search steps

Raise it if relationships are missing from the generated data. Ignored by tree_search

tvae - Training Guide

Parameter

What it does

When to change it

n_iter

Number of training passes

Raise it for a better fit. Lower it if training takes too long

weight_decay

Discourages the model from memorising individual rows

Increase it if the output is noisy. Reduce it if outliers disappear

lr

Learning rate: how big a step each update takes

Lower it if training fails with NaN errors

batch_size

Rows processed per update

Don't set it too small on small data, because training can become unstable

ctgan - Training Guide

Parameter

What it does

When to change it

n_iter

Number of training passes. The default is 2000, which is heavy on a CPU

Lower it for a first run

generator_n_layers_hidden

Depth of the generator network

2-3 is usually enough

discriminator_n_iter

How many times the discriminator updates for each generator update

Raise it if the generator's output is poor and doesn't improve

discriminator_dropout

Randomly switches off part of the discriminator during training

Increase it if the discriminator is too strong for the generator to learn

lr / batch_size

Learning rate and rows per update

If ctgan keeps producing the same values, lower lr

adsgan - Training Guide (coming soon)

Parameter

What it does

When to change it

n_iter

Number of training passes. The default is 2000, which is heavy on a CPU

Lower it for a first run

lambda_identifiability_penalty

How strongly the engine avoids rows that resemble real individuals

Raise it for more privacy. Lower it if the output loses realism

discriminator_n_iter

How many times the discriminator updates for each generator update

Raise it if the output is poor and doesn't improve

batch_size

Rows processed per update

Don't set it too small on small data, because training can become unstable

arf - Training Guide (coming soon)

Parameter

What it does

When to change it

num_trees

Number of trees in the forest

Raise it for better fidelity and more stable results

min_node_size

Minimum number of rows in each leaf of a tree

Raise it if the output follows the source too closely. Lower it for finer detail

max_iters

Maximum number of training iterations

Raise it if training stops before the output is good enough

ddpm - Training Guide (coming soon)

Parameter

What it does

When to change it

n_iter

Number of training passes

Lower it for a first run

num_timesteps

Number of steps used to add noise to the data and remove it again

More steps improve quality but slow training and generation. Lower it for a quicker first run

model_type

The network design: mlp, resnet or tabnet

Start with the default

batch_size

Rows processed per update. The default is 1024

Lower it on small tables

dpgan - Training Guide (coming soon)

Parameter

What it does

When to change it

epsilon

Privacy budget

Lower it for stronger privacy. Raise it if the output is too far from the real data

delta

The small chance that the privacy guarantee is exceeded

Keep it very small. Lower is stricter

dp_max_grad_norm

Limits how much any one row can affect each training update

Lower it for more privacy, at the cost of more noise

n_iter

Number of training passes. The default is 2000

Lower it for a first run

pategan - Training Guide (coming soon)

Parameter

What it does

When to change it

epsilon

Privacy budget

Lower it for stronger privacy. Raise it if the output is too far from the real data

n_teachers

Number of teacher models that vote on the training data

Raise it to strengthen privacy

teacher_template

The model type used for each teacher, such as xgboost or linear

Start with the default

n_iter

Number of training passes

Raise it if the output doesn't look realistic

nflow - Training Guide (coming soon)

Parameter

What it does

When to change it

n_iter

Number of training passes

Lower it for a first run

num_transform_blocks

Number of building blocks in each layer of the flow

Raise it to capture more complex patterns

num_bins

Number of bins the flow uses to shape numeric values

Raise it for finer numeric detail

n_units_hidden

Number of neurons in each hidden layer

Raise it if the output misses detail

privbayes - Training Guide (coming soon)

Parameter

What it does

When to change it

epsilon

Privacy budget

Lower it for stronger privacy. Raise it if the output is too far from the real data

K

Maximum number of columns any one column can depend on. 0 means it's chosen automatically

Leave it at 0 to start. Set a low value on wide tables

n_bins

Number of bins numeric columns are divided into

Raise it for finer numeric detail

mi_thresh

Minimum strength for keeping a relationship between columns

Raise it to keep only the stronger relationships

rtvae - Training Guide (coming soon)

Parameter

What it does

When to change it

robust_divergence_beta

How resistant training is to outliers

Raise it if outliers in your data distort the output

Other parameters

The same as tvae

See tvae above

timevae - Training Guide (coming soon)

Parameter

What it does

When to change it

mode

The type of sequence model: LSTM, GRU or RNN

Start with the default

n_iter

Number of training passes

Lower it for a first run

moments_penalty

How closely the generated statistics must match the real series

Raise it if generated values drift from the real series' averages and spread

batch_size

Rows processed per update

Don't set it too small on small data

timegan - Training Guide (coming soon)

Parameter

What it does

When to change it

mode

The type of sequence model: LSTM, GRU or RNN

Start with the default

n_iter

Number of training passes

Lower it for a first run

gamma_penalty

How closely generated sequences must follow the real patterns from one time step to the next

Raise it if generated sequences don't follow the real patterns over time

moments_penalty

How closely the generated statistics must match the real series

Raise it if generated values drift from the real series' averages and spread