This guide covers everything you need to train a CTGAN (Conditional Tabular Generative Adversarial Network) model and use it to generate synthetic data with the same characteristics as your source data.
Section | What it covers |
|---|---|
Step-by-step: create the activity and train a model | |
What CTGAN is, how it works and when to choose it | |
Activity settings and every ctgan parameter, with tuning advice | |
What training produces, how to read the log and when to retrain | |
Which columns to train on and which to leave out | |
Generating data, checking it against the source and fixing issues |
New to CTGAN? Follow section 1 first. It links to the other sections wherever you need more detail.
The example dataset
New to machine learning? Load our ML Sample Dataset - Lending into your database first, then work through this guide with the
CUSTOMERtable on its own, then add the other tables once you're comfortable.
The examples in this guide use the CUSTOMER table from a synthetic consumer-lending dataset. It has 12,000 rows, one per customer, and 11 columns:
Column | Type | What it holds |
|---|---|---|
| Text | Unique identifier, e.g. |
| Date | When the customer relationship started |
| Whole number | Months since |
| Category | Six bands, |
| Category | Five bands. About 7% of values are missing |
| Category |
|
| Category | Eight regions across Ireland and the UK |
| Category |
|
| Whole number | 473–850 |
| Decimal | 1,000–40,000, in steps of 500 |
| Decimal | Debt-to-income ratio, 0.05–0.65 |
It mixes categories, whole numbers and decimals, includes missing values, and has strong relationships between columns. For example, customers with higher credit scores have higher credit limits. Those are the things you'll check the synthetic data for.
1. Training
Before you start, you need:
A connection to your source database, scanned into the Data Dictionary
A Database Definition built from that connection
Create the data activity
1. Create a Machine Learning data activity. Go to Data Activities and create a new activity of type Machine Learning.
2. Add a name and description. Use a name you'll recognise later, such as Customer – CTGAN. In the description, note the table and engine you're training.
3. Select a Database Definition and rule set. Select your Database Definition, then choose New Rule Set. If you already have a training rule set for this data, choose Existing Rule Set instead.
Configure the activity
4. Select the source connection. On the Configuration tab, select the Source Connection the training data is read from.
5. Leave Find Rules off. It adds significant training time, and on a first run you want to confirm that everything completes. See Activity settings.
6. Set Imputation Strategy to Random Sample. It keeps each column's real proportions and works for every column type. See Activity settings.
7. Set Generative Engine to ctgan. See 2. Engine for why you'd choose it.
8. Set the engine parameters. For a first run, use the values in Example values.
Choose tables and columns
9. Select your tables. On the Tables tab, check the table(s) you want and click Add Selected. Start with one table. It's the quickest way to confirm everything works before you add related tables.
10. Finish the wizard. Click Next, then Finish. Your new data activity opens.
11. Run Analyze (optional). From the Actions panel, choose Analyze. It confirms the data can be read and suggests post-processing rules, such as the valid range of each numeric column. Relationship suggestions only appear when your rule set includes related tables. See Analyze.
12. Set which columns are active. Open your training rule set and expand your table. Make repeated information inactive. In the example, open_date is inactive because tenure_months is calculated from it. If you train related tables together, keep the key columns that link them active. See 5. Column Inclusion / Exclusion Guide.
13. Check the table-level engine (optional). Each table in the rule set uses the engine from the Configuration tab unless you set a different one for it. To use a different engine or different parameters for one table, edit the table and set them there. A table-level engine overrides the Configuration tab for that table.
Train
14. Train the model. Go back to the dashboard and select Train Model.
15. Check the log. Shortly after training starts, open the job log and find the line beginning Fit internal model. Confirm it says ctgan and shows the parameters you set. If it doesn't, stop the job and check steps 7 and 13. See Reading the training log.
When training finishes, the model is attached to the activity and its output files are available to download. See 4. Model.
2. Engine
What CTGAN is
CTGAN trains two neural networks against each other:
The generator creates synthetic rows from random input.
The discriminator looks at a mix of real and synthetic rows and tries to tell them apart.
Each network learns from the other. As the discriminator gets better at spotting synthetic rows, the generator gets better at producing rows that look real. When training finishes, the generator is kept, and generation passes random input through it to create new rows that follow the patterns of your data without copying any real row.
Before training, each column is converted into a form the networks can use:
Categories are split into one yes/no value per category.
Numbers are grouped into clusters, and each value is stored as its cluster plus its position within that cluster. This helps with columns whose values bunch into several groups. The number of clusters is set by
encoder_max_clusters. See 3. Hyperparameters.
Why "conditional"
During training, CTGAN picks a category column and a value from it, and asks the generator for a row with that value. It picks values evenly, not in proportion to how common they are, so rare categories are practised as often as common ones. This is what makes CTGAN a good choice when some categories appear in only a small share of rows. At generation time, the categories come back in their real proportions.
How CTGAN compares with the other engines
ctgan | tvae | bayesian_network | |
|---|---|---|---|
Type | Neural network (two competing networks) | Neural network (autoencoder) | Probability graph |
Best for | Imbalanced data with many categories | Small to medium tables with mixed categories and numbers | Small to medium tables where you want to see the relationships |
Training stability | Can be unstable | Good | Good |
Speed (platform rating) | 6/10 | 7/10 | 8/10 |
Can you see what it learned? | No | No | Yes, as a graph of dependencies |
When to choose CTGAN
Choose CTGAN when:
Your table has category columns with rare values that must still appear in the output
Your table mixes categories and numbers, and you want a neural model
You're training related tables together and want the relationships between them kept
Choose something else when:
You need to explain or inspect the relationships the model found. Use bayesian_network.
Training is erratic on your data even after tuning. Try tvae.
Global engine vs table-level engine
You can set the engine in two places:
Configuration tab: the default engine for the activity.
Table level, in the training rule set: the engine for that table.
The table-level setting wins. Leave it blank to use the Configuration tab engine, or set it to use a different engine for one table. Confirm which engine each table used in the log (see Reading the training log).
Training related tables
CTGAN works well across related tables. When you train more than one table, keep the key columns that link them active: each foreign key, and the primary key it points to. The engine from the Configuration tab applies to every table unless a table has its own engine set, so check the log to confirm every table uses ctgan.
To learn how the tables relate, the platform copies some of the parent table's columns into each child table before training. They appear in the log with names beginning __cond__. These come from the correlations switched on for the definition in the Data Dictionary. Keep them selective: a few strong correlations per child table work better than many, and give the engine enough rows to learn each pattern well. When you run Analyze, at most 2 correlations per table are switched on by default.
When you generate related tables, switch on Use Sequential ID for each table's primary key. Every key is then unique, every child row links to a parent row, and each parent has a similar number of children to the source. See Generate data.
CPU vs GPU
CTGAN runs on the CPU unless the server has a GPU. Training on CPU takes longer. If the log includes CUDA libraries not found, you're on CPU. Use a lower n_iter for a first run so you get a result sooner.
Do I need a target column?
No. CTGAN learns how all the columns relate to each other, so no single column is treated as the answer.
3. Hyperparameters
A hyperparameter is a setting you choose before training. It controls how the model learns, as opposed to what it learns from the data. Changing a hyperparameter means you need to retrain.
Activity settings
These two settings are on the Configuration tab and apply to the whole activity, whichever engine you use.
Find Rules. Searches for mathematical formulas linking your columns, such as one column being another divided by a third. It adds significant time to training. Leave it off for a first run, and turn it on later if you suspect columns are calculated from each other.
Imputation Strategy. Decides how missing values are filled before training.
Option | How it fills a gap | Use for |
|---|---|---|
Random Sample | A value drawn from the column's existing values | Any column. Recommended default |
Mode | The most common value | Categories |
Least Frequent | The least common value | Rarely useful |
Mean | The average | Numbers only |
Median | The middle value | Numbers only |
Zero | 0 | Numbers where missing genuinely means zero |
Random Sample keeps each column's proportions closest to the real data. Whichever option you choose, the model also records which values were originally missing, in an extra column called <column>__missing_indicator. This lets it learn the pattern of what's missing as well as the values.
ctgan parameters
Training length and pace
Parameter | Example value | What it does |
|---|---|---|
| 300 | Number of training rounds (epochs). Each round passes through all the training data once. More rounds give the networks longer to learn, and take longer to run |
| 500 | Rows the networks look at before each update. Larger is faster per round but can learn less detail. Keep it well below your row count |
| 0.001 | Learning rate: how big a change the networks make at each update. Too high and training is erratic; too low and it's slow to improve |
| 0.001 | Small penalty that discourages the networks from relying on extreme internal values. Helps prevent memorising the training data |
| 1 | Number of discriminator updates for each generator update |
| 5 | Number of checks with no improvement before training stops early |
| 100 | Minimum number of rounds before training can stop early |
| 50 | How often, in rounds, training progress is checked and written to the log |
Network size
Parameter | Example value | What it does |
|---|---|---|
| 2 | Number of layers in the generator |
| 500 | Size of each generator layer |
| 2 | Number of layers in the discriminator |
| 500 | Size of each discriminator layer |
|
| Mathematical function applied between layers. Rarely needs changing |
More layers and larger layers let the networks capture more complex patterns. They also take longer to train, and make it easier to memorise rows on a small table.
Regularisation and stability
Parameter | Example value | What it does |
|---|---|---|
| 0.1 | Share of the generator's connections randomly switched off during each update. Reduces memorising |
| 0.1 | The same, for the discriminator |
| 10 | Keeps the discriminator's feedback smooth, which keeps training stable. Rarely needs changing |
| 1 | Caps how large a single update can be, to stop training becoming unstable |
Data encoding and generation
Parameter | Example value | What it does |
|---|---|---|
| 10 | Maximum number of clusters each numeric column is split into. More clusters capture more detail, at the cost of more training time |
| Off | Nudges category proportions in the generated data towards the proportions in the training data |
| Off | Reduces the data before training. Leave off, so the output can be compared with the source |
| 500 | Number of attempts the engine makes to produce a valid row before giving up. Only matters if generation returns fewer rows than you asked for |
Example values
The values in the tables above are the ones used for the example table (around 12,000 rows, a mix of category and numeric columns, CPU only). n_iter of 300 and batch_size of 500 give a first result in reasonable time on CPU. Use them as a starting point, then adjust for your data.
Tuning: which setting to change
Change one setting at a time and compare each run against the source.
What you see in the output | Try |
|---|---|
Distributions or relationships are weak across most columns | Increase |
Rare categories are missing or too rare | Increase |
Many rows look alike, with little variety | Lower |
Numeric columns look too smooth, or a sharp step in the data is lost | Increase |
The networks still miss relationships after longer training | Increase the layer sizes |
Synthetic rows are copies, or near-copies, of real rows | Increase the dropout values or |
Training is erratic | Lower |
Training is too slow | Lower |
4. Model
What a trained model is
A trained model is the result of a training job, attached to your data activity. It contains everything the engine learned from your data. It takes a snapshot of the training rule set when the job starts, so changes you make to the rule set afterwards don't affect a model that's already trained.
You use the model by selecting it in a generation rule set (see 6. Generate and Validate).
Each new training run attaches its model to the activity in place of the previous one. Before you generate, check that the activity and the generation rule set point at the model you want.
Output files
When training finishes, open the trained model from the training rule set. The trained model page shows:
Configuration files:
training.jsonandmetadata.json, which you can download.Location: the folder on your host server where the model files are saved.
Engine parameters: the engine settings the model was trained with.
metadata.json is the most useful configuration file: it shows the query used to load the training data, so you can confirm which columns were trained. See Training output files for what each file contains.
Reading the training log
Open the job log shortly after training starts and check these lines.
Engine and parameters:
Fit internal model 'ctgan' for table '<table>' with params: {...}Confirm the engine is ctgan and the parameters match what you set.
Columns and types:
type_metadata_mapping: {'CUSTOMER': {'integer_cols': [...],
'float_cols': [...], 'categorical_cols': [...]}}Check that each column is classed as you'd expect. A number stored as text, for example, would appear under categorical_cols. The table's primary key is always listed, even when it's inactive, because the platform uses it to keep track of rows.
Category encoding:
VERIFY [CUSTOMER][channel_cd]: [('BRANCH', 0), ('BROKER', 1), ('ONLINE', 2)]Each category is listed with the number it's stored as. Check for unexpected values, such as a misspelling or a stray blank. Missing values appear as -1.
Missing values: columns with gaps get an extra <column>__missing_indicator column. In the example, you'd see income_band__missing_indicator.
Progress: while the networks train, the log shows a progress bar for the training rounds, for example 45%| 135/300. Use it to see how far through training the job is.
Finished:
Synthcity relational model trained and saved.Messages you can ignore:
Message | Why it's safe |
|---|---|
| No GPU. Training runs on CPU |
| An unrelated engine failing to load. It doesn't affect ctgan |
| The primary key has a different value in almost every row, so it isn't mapped to categories |
Checking what the model learned
You can't look inside a CTGAN model to see what it learned. The only way to check it is to generate data and compare it against the source. See Compare with the source.
When to retrain
Situation | Retrain? |
|---|---|
You changed an engine parameter | Yes |
You made columns active or inactive | Yes |
The source data has changed | Yes |
The model missed a relationship or distribution | Yes, with adjusted settings |
Values break a fixed rule (range, steps, date order) | No, add a post-processing rule |
You want more or fewer rows | No, set the row count at generation |
5. Column Inclusion / Exclusion Guide
Every active column is something the model has to learn. Columns that carry no useful information slow training down and can make the output worse. Columns that give too much away can teach the model the wrong thing.
Exclude these
Columns calculated from another column. If one column is worked out from another, keep one of them. The model would otherwise have to learn the calculation, and will get it slightly wrong. Example: tenure_months is calculated from open_date, so open_date is inactive. If you need both, recalculate the second one with a post-processing rule.
Columns that give away another column's value. A column set directly from another column's value. For example, a status column that is set to DEFAULT whenever a default flag is 1. It adds nothing, and it can hide the real relationships.
Free text. Notes, comments and descriptions. Each distinct text is treated as a separate category, so the engine can't produce new text and may repeat real text. Mask or generate these separately.
Personal details. Names, emails, phone numbers and addresses. Excluding them keeps real personal details out of the model. Generate them with generator functions.
Unique reference numbers that aren't keys. Any column other than a key with a different value in every row. There's no pattern to learn from it. Generate it with a post-processing rule or generator function instead.
Include these
Columns with the patterns you want to keep. Any column involved in a relationship you care about must be active, on both sides of that relationship. Example: credit_score and credit_limit both stay active, so the model can learn that they rise together.
Columns with missing values. Keep them. The model learns where values are missing as well as what they are, and the pattern often carries information. Example: income_band is missing more often for BROKER customers. Keeping the column keeps that pattern.
Category columns with rare values. This is where CTGAN does best. Keep them active so the rare values are learned.
Keys that link tables. When you train related tables together, keep each foreign key and the primary key it points to active. In the example, that's customer_id in CUSTOMER and LOAN_ACCOUNT, and account_id in LOAN_ACCOUNT and ACCOUNT_BEHAVIOUR. They're what lets each child row link to the right parent.
Think before including these
Column type | Guidance |
|---|---|
Dates | The platform splits a date column into parts (year, month, day and so on) before training, so the model can learn patterns such as busier months. This happens by default for date columns with at least 100 distinct values. A date column with fewer distinct values is treated as a category. Add |
Two columns that say nearly the same thing | Keep one, and recalculate the other with a post-processing rule |
Categories with hundreds of values | Each value adds to training time. Group rare values into an |
Columns filled by a generator function | Exclude from training. They're filled at generation time |
Quick checklist
Column | Active? | Why |
|---|---|---|
Primary key that another table points to (multi-table) | ✓ | Keeps parent–child links |
Foreign key (multi-table) | ✓ | Keeps parent–child links |
Unique reference number that isn't a key | ✗ | No pattern to learn |
Category, code or band | ✓ | Rare values are CTGAN's strength |
Number | ✓ | Unless calculated from another column |
Date | ✓ | Add date-order rules afterwards |
Calculated from another column | ✗ | Recalculate with a post-processing rule |
Gives away another column's value | ✗ | Adds nothing, hides real relationships |
Free text | ✗ | Can't be generated; may repeat real text |
Personal details | ✗ | Use generator functions |
Column with missing values | ✓ | The pattern of what's missing is information |
6. Generate and Validate
Generate data
1. Create a generation rule set. Select your trained model in its configuration.
2. Review the rule set. The table and column settings are copied from training. Leave post-processing rules out for now, so you can see what the model produces on its own.
Edit each table in the generation rule set:
Default Count. Set the number of rows. If you trained related tables, set it on the parent table only and leave it empty for child tables. Their row counts come from the parent, following the number of children per parent in the source.
Use Sequential ID. Switch this on for each table's primary key, select the key in Column, and enter a Start ID, such as 1. Each row then gets a unique key, numbered in order. Don't switch it on for foreign keys: they're filled from the parent rows, which keeps the links between tables. Use Use Composite Sequential ID instead for a primary key made of more than one column.
Sequential IDs are numbers. If your keys have a format, such as C000001, add a custom_python rule to apply it. See Improve the output.
3. Generate a small sample. Select Generate and enter a small row count, such as 500. Check that the job completes, the right columns are filled, and data types and formats are what you expect.
Large outputs. You can generate millions of rows. Check a small sample first, then scale up.
4. Generate a full run. Generate again with the same number of rows as your source table (12,000 in the example). Matching the row count makes the comparison fair. The job log shows the row count and where the output file was saved. See Generation reports.
The Generate Data dialog has two further options:
Random seed. Enter a whole number to get the same output every time you generate with the same settings. This makes runs easy to compare: any difference comes from the model or the rules, not from chance. Leave it blank for different output each run.
Generate report charts. Saves charts next to the output file, with an
index.htmlpage to view them all.
Each generation from the same model overwrites the previous output file. Download the output before you generate again if you want to keep it.
Generation reports
Each generation saves reports next to the output files. They compare the generated data with the source, so they're the quickest place to start checking it.
Report | What it shows |
|---|---|
| An overall score and grade, a score for each table, the weakest columns and any risks found |
| How closely each column matches the source, how well relationships between columns are kept, privacy checks, and whether each check passed |
| Duplicate or missing primary keys, child rows whose parent is missing, values outside the source range and unknown categories |
| How much of each column's range and categories the generated data covers |
| How to read the reports and charts, and a plain-English verdict |
Start with the executive summary. Then check the integrity report for key problems, and the relationship check in the evaluation report.
Compare with the source
Download the generated data from your host server and compare it with the source data. The reports cover most of the checks below.
Compare on:
Distributions. Categories should appear in similar proportions. Numeric columns should have similar ranges, averages and shape.
Rare values. Rare categories and extreme values should still appear, in roughly the same amounts.
Relationships. Columns that move together in the source should still move together. In the example, credit limit should still rise with credit score.
Missing values. Each column should have a similar share of missing values. About 7% of
income_bandvalues should be empty.Valid values. Every value should be possible: known categories, sensible ranges, and business rules that still hold.
Links between tables. If you generated more than one table, every child row should point to a real parent row, and each parent should have a similar number of children.
Similarity to real records. No synthetic row should be a copy of a real one.
Leave inactive columns and generator-filled columns out of the comparison.
What to expect. A close match, not an exact one. Proportions and missing-value rates are usually within a few percent of the source, and relationships between columns can come back a little softer. A small difference across many columns is normal. If one column or relationship is far off, improve the output as described below.
The model learns patterns, not rules. Formats and business rules, such as credit_limit always being a multiple of 500, or a loan never being larger than the customer's credit limit, are applied with post-processing rules. See Improve the output.
Improve the output
There are two ways to improve the output. You can use either or both.
Retrain the model if the model missed something, such as a weak relationship or a distorted distribution. See Tuning: which setting to change. Also check that the relevant columns were active.
Add post-processing rules to fix things the model can't be expected to learn, such as fixed steps, ranges, key formats or date order. Rules are applied after the model has run, so you can add them and generate again without retraining. Rules run in the order they're listed, so put a rule that recalculates a value after any rules that fix its inputs.
Problem in the output | Rule |
|---|---|
Values outside the real range |
|
Values that should be in fixed steps, such as multiples of 500 |
|
Keys that need a format, such as a prefix |
|
Dates in the wrong order |
|
Child rows pointing to parents that don't exist |
|
Parent totals that don't match their child rows |
|
Values that should be calculated from other columns |
|
Synthetic rows too similar to real ones |
|
Duplicate primary keys, or keys that should be numbered in order | Use Sequential ID on the table in the generation rule set |
For the example, these rules give an output that matches the source format:
# | Rule | Settings |
|---|---|---|
1 |
|
|
2 |
|
|
3 |
|
|
Rule 2 creates new identifiers from C100001 upwards, so none match a real customer. Rule 3 recalculates open_date from tenure_months, counting back from the dataset's snapshot date (August 2026). It's accurate to the month, so every date falls on the 1st.
See Post-processing rules reference for every rule and its settings.
Repeat until you're happy
Retrain and regenerate until the output meets your needs. Change one thing at a time and compare each run against the source, so you can see which change made the difference.