Before you start
Read this section before setting up a Machine Learning data activity. The choices you make here shape your setup from the start.
It helps to understand your data before you begin: how values are spread across each column, which columns are related, and whether any values are rare or missing. This page covers the basics and should help you profile your data and pick sensible starting parameters for your first training run.
Expect the first run to be a starting point rather than the finished result. Once you've generated synthetic data, compare it with your source data, adjust the parameters based on what you find, and train again.
What is machine learning?
In the Curiosity Machine Learning data activity, machine learning means training to produce a model on your existing data so that it learns the data's statistical patterns:
How values are distributed in each column
How columns relate to each other
How related tables connect
Once trained, the model generates new synthetic records that follow those same patterns or characteristics. The records are newly generated rather than copied from the source, so you get realistic, representative test data without exposing the real records it learned from.
Generative Engines
The generative engine is the machine learning algorithm that learns the patterns in your training data. Training produces a model, which you then use to generate new rows with the same characteristics. The Generative Engine list shows every engine available to you.
Pick one based on what your data looks like:
Engine | Type | Data size | Use it when |
|---|---|---|---|
Statistical | Small (up to around 25,000 rows) | You want speed, and to see which columns depend on which | |
Neural (variational autoencoder) | Medium to large (over 25,000 rows) | Your data is mostly numeric, smooth distributions and complex numeric relationships | |
Neural (GAN) | Medium to large (over 25,000 rows) | Your data is heavily categorical, has many distinct labels, or has very imbalanced categories | |
adsgan | Neural (GAN) with privacy | — | Privacy is a priority. It balances how realistic the data is against how easily a synthetic record could be linked back to a real person |
arf | Tree-based (adversarial random forests) | — | You want high-fidelity tabular data and the neural engines struggle to converge on your data |
ddpm | Neural (diffusion) | — | You want high-quality output on data with many columns. It learns to reverse a process of adding noise to data |
dpgan | Neural (GAN) with differential privacy | — | You need a differential privacy guarantee. Noise is added during training, so no single record's presence in the training data can be inferred |
pategan | Neural (GAN) with differential privacy | — | You need strong differential privacy guarantees. It trains several discriminators on separate parts of the data and learns from their combined, noisy votes |
nflow | Neural (normalizing flow) | — | You want a model that learns the exact probability of each row, by transforming a simple distribution into your data's distribution |
privbayes | Statistical (Bayesian network with differential privacy) | — | You want a bayesian_network-style dependency graph with a mathematical privacy guarantee |
rtvae | Neural (variational autoencoder) | — | A variant of tvae. Use it in the same situations: small to medium tables where GAN-based engines are unstable |
timevae | Neural (variational autoencoder) for sequences | — | Your data is a sequence over time, and you need to keep its trends and seasonal patterns |
timegan | Neural (GAN) for sequences | — | Your data is a sequence over time, and you need to keep how values change from one time step to the next |
bayesian_network models your data as a graph of column dependencies. It's fast, and the relationships it learns are explicit: each column depends on specific other columns. It can struggle with very wide tables.
tvae compresses each row and learns to rebuild it. It doesn't use adversarial training, so it's usually more stable than ctgan. It tends to produce average values, so outliers can get smoothed away.
ctgan trains two networks against each other. The generator makes rows and the discriminator tries to tell them from real ones. It specifically targets under-represented categories, but it's slower to train and harder to tune.
adsgan is a GAN that adds a privacy penalty during training, so synthetic rows are harder to link back to real individuals. Raising the penalty gives more privacy at some cost to realism.
arf (adversarial random forests) uses trees instead of a neural network. It keeps splitting the data until real and synthetic rows can't be told apart.
ddpm is a diffusion model. It learns to reverse a process that gradually adds noise to the data, and can give high-quality output on data with many columns.
dpgan is a GAN with differential privacy. It adds noise during training so that no single record's presence can be worked out from the model.
pategan gives differential privacy in a different way: several teacher models are trained on separate parts of the data, and the generator learns from their combined, noisy votes.
nflow (normalizing flows) transforms a simple distribution into your data's distribution through a series of reversible steps, which lets it calculate the exact probability of each row.
privbayes is a Bayesian network with differential privacy. It builds a dependency graph like bayesian_network, then adds noise to the probabilities to give a mathematical privacy guarantee.
rtvae is a variant of tvae whose training is more resistant to outliers.
timevae and timegan are built for sequential data, such as readings or transactions recorded over time. timevae uses a variational autoencoder and timegan uses a GAN. Both learn how values change from one time step to the next, including trends and seasonal patterns.
All engines
Parameter | What it does | When to change it |
|---|---|---|
encoder_max_clusters (data_encoder_max_clusters on tvae) | How finely continuous columns are divided before training | Raise it (for example from 10 to 20) if values come out too coarse, or if less common values go missing |
sampling_patience | How many attempts the engine gets to produce a valid row before it fails | Raise it if generation fails with "failed to meet constraints" |
bayesian_network - Training Guide
Parameter | What it does | When to change it |
|---|---|---|
struct_learning_search_method | How it searches for column relationships | Choose to suit your data. |
struct_learning_score | How each candidate structure is scored: |
|
struct_max_indegree | Maximum number of columns any one column can depend on | Higher is more accurate but much slower. For tables with more than 20 columns, keep it at 2–3 |
struct_learning_n_iter | Maximum number of search steps | Raise it if relationships are missing from the generated data. Ignored by |
tvae - Training Guide
Parameter | What it does | When to change it |
|---|---|---|
n_iter | Number of training passes | Raise it for a better fit. Lower it if training takes too long |
weight_decay | Discourages the model from memorising individual rows | Increase it if the output is noisy. Reduce it if outliers disappear |
lr | Learning rate: how big a step each update takes | Lower it if training fails with NaN errors |
batch_size | Rows processed per update | Don't set it too small on small data, because training can become unstable |
ctgan - Training Guide
Parameter | What it does | When to change it |
|---|---|---|
n_iter | Number of training passes. The default is 2000, which is heavy on a CPU | Lower it for a first run |
generator_n_layers_hidden | Depth of the generator network | 2-3 is usually enough |
discriminator_n_iter | How many times the discriminator updates for each generator update | Raise it if the generator's output is poor and doesn't improve |
discriminator_dropout | Randomly switches off part of the discriminator during training | Increase it if the discriminator is too strong for the generator to learn |
lr / batch_size | Learning rate and rows per update | If ctgan keeps producing the same values, lower lr |
adsgan - Training Guide (coming soon)
Parameter | What it does | When to change it |
|---|---|---|
n_iter | Number of training passes. The default is 2000, which is heavy on a CPU | Lower it for a first run |
lambda_identifiability_penalty | How strongly the engine avoids rows that resemble real individuals | Raise it for more privacy. Lower it if the output loses realism |
discriminator_n_iter | How many times the discriminator updates for each generator update | Raise it if the output is poor and doesn't improve |
batch_size | Rows processed per update | Don't set it too small on small data, because training can become unstable |
arf - Training Guide (coming soon)
Parameter | What it does | When to change it |
|---|---|---|
num_trees | Number of trees in the forest | Raise it for better fidelity and more stable results |
min_node_size | Minimum number of rows in each leaf of a tree | Raise it if the output follows the source too closely. Lower it for finer detail |
max_iters | Maximum number of training iterations | Raise it if training stops before the output is good enough |
ddpm - Training Guide (coming soon)
Parameter | What it does | When to change it |
|---|---|---|
n_iter | Number of training passes | Lower it for a first run |
num_timesteps | Number of steps used to add noise to the data and remove it again | More steps improve quality but slow training and generation. Lower it for a quicker first run |
model_type | The network design: mlp, resnet or tabnet | Start with the default |
batch_size | Rows processed per update. The default is 1024 | Lower it on small tables |
dpgan - Training Guide (coming soon)
Parameter | What it does | When to change it |
|---|---|---|
epsilon | Privacy budget | Lower it for stronger privacy. Raise it if the output is too far from the real data |
delta | The small chance that the privacy guarantee is exceeded | Keep it very small. Lower is stricter |
dp_max_grad_norm | Limits how much any one row can affect each training update | Lower it for more privacy, at the cost of more noise |
n_iter | Number of training passes. The default is 2000 | Lower it for a first run |
pategan - Training Guide (coming soon)
Parameter | What it does | When to change it |
|---|---|---|
epsilon | Privacy budget | Lower it for stronger privacy. Raise it if the output is too far from the real data |
n_teachers | Number of teacher models that vote on the training data | Raise it to strengthen privacy |
teacher_template | The model type used for each teacher, such as xgboost or linear | Start with the default |
n_iter | Number of training passes | Raise it if the output doesn't look realistic |
nflow - Training Guide (coming soon)
Parameter | What it does | When to change it |
|---|---|---|
n_iter | Number of training passes | Lower it for a first run |
num_transform_blocks | Number of building blocks in each layer of the flow | Raise it to capture more complex patterns |
num_bins | Number of bins the flow uses to shape numeric values | Raise it for finer numeric detail |
n_units_hidden | Number of neurons in each hidden layer | Raise it if the output misses detail |
privbayes - Training Guide (coming soon)
Parameter | What it does | When to change it |
|---|---|---|
epsilon | Privacy budget | Lower it for stronger privacy. Raise it if the output is too far from the real data |
K | Maximum number of columns any one column can depend on. 0 means it's chosen automatically | Leave it at 0 to start. Set a low value on wide tables |
n_bins | Number of bins numeric columns are divided into | Raise it for finer numeric detail |
mi_thresh | Minimum strength for keeping a relationship between columns | Raise it to keep only the stronger relationships |
rtvae - Training Guide (coming soon)
Parameter | What it does | When to change it |
|---|---|---|
robust_divergence_beta | How resistant training is to outliers | Raise it if outliers in your data distort the output |
Other parameters | The same as tvae | See tvae above |
timevae - Training Guide (coming soon)
Parameter | What it does | When to change it |
|---|---|---|
mode | The type of sequence model: LSTM, GRU or RNN | Start with the default |
n_iter | Number of training passes | Lower it for a first run |
moments_penalty | How closely the generated statistics must match the real series | Raise it if generated values drift from the real series' averages and spread |
batch_size | Rows processed per update | Don't set it too small on small data |
timegan - Training Guide (coming soon)
Parameter | What it does | When to change it |
|---|---|---|
mode | The type of sequence model: LSTM, GRU or RNN | Start with the default |
n_iter | Number of training passes | Lower it for a first run |
gamma_penalty | How closely generated sequences must follow the real patterns from one time step to the next | Raise it if generated sequences don't follow the real patterns over time |
moments_penalty | How closely the generated statistics must match the real series | Raise it if generated values drift from the real series' averages and spread |