| Title: | Synthetic Data Doubles for Safer Prototyping |
|---|---|
| Description: | Creates synthetic data doubles from real datasets for prototyping, teaching, 'shiny' development, and AI-assisted programming. Provides data profiling, role detection, configurable synthesis, utility comparison, and disclosure-risk warnings. Synthetic outputs are intended to reduce direct disclosure risk, not to guarantee privacy. |
| Authors: | Lennon Li [aut, cre] |
| Maintainer: | Lennon Li <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 0.7.0 |
| Built: | 2026-07-22 23:44:57 UTC |
| Source: | https://github.com/lennon-li/dataganger |
Cross-tabulates the quasi-identifier columns and reports how many records
fall in combinations (equivalence classes) smaller than k. NA is treated
as a distinct level so that missing values cannot mask a small cell.
assess_kanonymity(data, qi_cols, k = 5)assess_kanonymity(data, qi_cols, k = 5)
data |
A data frame. |
qi_cols |
Character vector of quasi-identifier column names. |
k |
Minimum acceptable cell size (default 5). |
A list with no_qi (logical), smallest_cell (integer),
n_below, pct_below, and worst_cells (a tibble of the smallest
combinations and their counts).
Evaluates whether code written against the synthetic development twin will
run against the original data without errors. Checks column presence, R
class compatibility, factor level coverage, all-NA columns, zero-variance
columns, missingness spikes, and ID uniqueness. haven_labelled columns
currently round-trip as character in synthetic data, so that class change is
expected for now.
check_code_readiness(original, synthetic, roles = NULL)check_code_readiness(original, synthetic, roles = NULL)
original |
The original data frame. |
synthetic |
The synthetic data frame (from |
roles |
Optional; a |
An S3 object of class dataganger_code_readiness with
components:
A tibble with one row per check: check, scope,
column, status ("pass"/"warn"/"fail"), message.
List with n_pass, n_warn, n_fail,
ready (TRUE when n_fail == 0).
List with dimensions and generated_at.
orig <- data.frame(x = 1:10, y = factor(letters[1:10])) spec <- synth_spec(purpose = "demo") syn <- synthesize_data(orig, spec) check_code_readiness(orig, syn)orig <- data.frame(x = 1:10, y = factor(letters[1:10])) spec <- synth_spec(purpose = "demo") syn <- synthesize_data(orig, spec) check_code_readiness(orig, syn)
Compares an original dataset with its synthetic double across dataset-level
dimensions, numeric distributions, categorical distributions, and numeric
correlations. Returns a structured dataganger_comparison object.
compare_synthetic(original, synthetic, roles = NULL)compare_synthetic(original, synthetic, roles = NULL)
original |
The original data frame. |
synthetic |
The synthetic data frame (from |
roles |
Optional; a |
An S3 object of class dataganger_comparison, a list with
components dataset, numeric, categorical, relationship, interaction,
privacy_flags, and meta.
dat <- data.frame(x = 1:10, y = letters[1:10]) spec <- synth_spec(purpose = "demo") syn <- synthesize_data(dat, spec) compare_synthetic(dat, syn)dat <- data.frame(x = 1:10, y = letters[1:10]) spec <- synth_spec(purpose = "demo") syn <- synthesize_data(dat, spec) compare_synthetic(dat, syn)
Testable command-line entrypoint used by the installed exec/dataganger shim.
dataganger_cli(args = commandArgs(trailingOnly = TRUE), quit = FALSE)dataganger_cli(args = commandArgs(trailingOnly = TRUE), quit = FALSE)
args |
Character vector of trailing command-line arguments. |
quit |
Logical. When |
Integer status code: 0 success, 1 processing error, 2 syntax error.
Applies heuristic-based role detection to every column in a data frame.
Roles include a recommended synthesis role plus the two primary disclosure
axes used by the Configure step: whether a column points to a person
(identifies) and whether it is sensitive. The legacy single
disclosure_role value is retained as derived compatibility metadata for
existing synthesis/export/CLI paths.
detect_roles(data, profile = NULL)detect_roles(data, profile = NULL)
data |
A data frame. |
profile |
Optional; a |
An S3 object of class dataganger_roles, a tibble with columns:
Column name.
R class of the column.
Role detected by heuristic.
User-supplied override (initially NA).
How the column is treated during synthesis.
Justification for the recommended role.
Whether the column points to a person: "none", "combination", or "direct".
Logical flag for whether the column is sensitive if revealed.
User-supplied override for identifies (initially NA).
User-supplied override for sensitive (initially NA).
Disclosure role. NA (unselected) is the
conservative default whenever detection is not confident; the user must
choose a role before generating. "direct" and "sensitive" are the
only values auto-assigned (confident identifier / known-sensitive name).
"quasi" and "none" are user-assigned choices only.
Justification for the auto-assigned disclosure role.
df <- data.frame( id = 1:50, date = as.Date("2020-01-01") + 0:49, city = rep(c("Toronto", "Vancouver", "Montreal"), length.out = 50), cat = factor(rep(letters[1:3], length.out = 50)) ) detect_roles(df)df <- data.frame( id = 1:50, date = as.Date("2020-01-01") + 0:49, city = rep(c("Toronto", "Vancouver", "Montreal"), length.out = 50), cat = factor(rep(letters[1:3], length.out = 50)) ) detect_roles(df)
Shapes the synthetic output so that no quasi-identifier combination appears
in fewer than k records. Direct identifiers are removed. Quasi-identifiers
are coarsened step-by-step and any residual cell still below k has its QI
values blanked (NA). Operates on the output only.
enforce_kanon(synthetic, roles, k = 5, max_steps = 6L, max_suppress_frac = 0.2)enforce_kanon(synthetic, roles, k = 5, max_steps = 6L, max_suppress_frac = 0.2)
synthetic |
A synthetic data frame. |
roles |
A roles object/data frame with |
k |
Minimum cell size (default 5). |
max_steps |
Maximum coarsening iterations (default 6). |
max_suppress_frac |
Feasibility backstop. If satisfying |
The shaped synthetic data frame, with an attribute kanon
recording the achieved state (smallest_cell, suppressed_cells,
suppressed_rows, suppressed_row_frac, qi_cols, k, infeasible).
suppressed_rows/suppressed_row_frac count actual blanked rows across
the QI columns – distinct from suppressed_cells, which counts the
number of distinct QI combinations folded into suppression. The two can
differ a lot: reaching k can require absorbing a few whole neighbouring
cells (suppression works at cell granularity, not row granularity), and
a handful of small cells sitting next to one dominant cell can end up
suppressing most or all of a QI column even though only a few original
cells were actually below k.
A realistic-but-fictional administrative claims dataset with 300 synthetic records. Contains claim identifiers, procedure codes (haven-labelled), costs, and provider locations. No real patient data.
example_admin_claimsexample_admin_claims
A tibble with 300 rows and 9 columns:
Integer. Claim identifier.
Character. Patient identifier.
Date. Date of service.
Factor. Diagnosis code.
haven_labelled. Procedure type (Consult / Surgery / Lab / Imaging).
Numeric. Claim cost in dollars, some missing.
Logical. Whether the claim was approved, some missing.
Character. City of the service provider.
Character. Forward sortation area (FSA).
A realistic-but-fictional health survey dataset with 200 synthetic records. Contains demographics, clinical measures, and haven-labelled smoking status. No real patient data.
example_health_surveyexample_health_survey
A tibble with 200 rows and 10 columns:
Character. Record identifier.
Numeric. Age in years.
Factor. Biological sex (Male / Female).
Numeric. Body mass index, with some missing values.
haven_labelled. Smoking status (Current / Former / Never).
Numeric. Systolic blood pressure, some missing.
Numeric. Diastolic blood pressure.
Date. Date of survey response.
Factor. Canadian province abbreviation.
Character. Free-text comments, some missing.
A realistic-but-fictional disease registry dataset with 150 synthetic records. Contains enrollment data, disease staging (haven-labelled), biomarker values, and patient status. No real patient data.
example_registryexample_registry
A tibble with 150 rows and 10 columns:
Character. Subject identifier.
Date. Date of enrollment.
Numeric. Age at enrollment in years.
haven_labelled. Disease stage (Stage I-IV).
Numeric. Biomarker A level, some missing.
Numeric. Biomarker B level, some missing.
Factor. Current patient status.
Date. Date of last follow-up visit, some missing.
Factor. Geographic region.
Character. Clinical notes, some missing.
Profiles a data frame and writes a diagnostic_view.json describing
column roles, sensitivity, and exposure levels. Does not synthesise data.
Intended for agent pre-inspection and Lens ingestion.
export_diagnostic_package( data, path, roles = NULL, profile = NULL, overwrite = FALSE )export_diagnostic_package( data, path, roles = NULL, profile = NULL, overwrite = FALSE )
data |
A data frame to describe. |
path |
Output path for the JSON file. |
roles |
Optional; a |
profile |
Optional; a |
overwrite |
Logical. When |
Invisibly, the written JSON path.
dat <- data.frame(age = c(34, 29, 41), grp = c("a", "b", "c")) export_diagnostic_package(dat, path = tempfile(fileext = ".json"))dat <- data.frame(age = c(34, 29, 41), grp = c("a", "b", "c")) export_diagnostic_package(dat, path = tempfile(fileext = ".json"))
Writes a reviewable export bundle containing the synthetic data, a human guide, an optional comparison report, a combined agent recipe, the packaged agent instructions, and a manifest. By default the bundle is written as a zip archive.
export_synthetic( synthetic, original = NULL, comparison = NULL, privacy = NULL, path, format = c("zip", "dir"), sanitize_for_spreadsheets = TRUE, purpose = NULL, roles = NULL, include_original_names = NULL, fail_on_exact_match = FALSE, include_report = TRUE, kanon_acknowledged = FALSE, include_dictionary = TRUE, code_readiness = NULL, compact = FALSE, overwrite = FALSE )export_synthetic( synthetic, original = NULL, comparison = NULL, privacy = NULL, path, format = c("zip", "dir"), sanitize_for_spreadsheets = TRUE, purpose = NULL, roles = NULL, include_original_names = NULL, fail_on_exact_match = FALSE, include_report = TRUE, kanon_acknowledged = FALSE, include_dictionary = TRUE, code_readiness = NULL, compact = FALSE, overwrite = FALSE )
synthetic |
A synthetic data frame, typically from |
original |
Optional original data frame. When provided, used for the data dictionary, comparison fallback, privacy fallback, and exact-row guard. |
comparison |
Optional |
privacy |
Optional |
path |
Output path. Required. For |
format |
Character. One of |
sanitize_for_spreadsheets |
Logical. When |
purpose |
Optional purpose label for README text. Defaults to the
purpose recorded in |
roles |
Optional recorded role decisions as a |
include_original_names |
Logical or |
fail_on_exact_match |
Logical. When |
include_report |
Logical. When |
kanon_acknowledged |
Logical. Records whether a human explicitly
acknowledged exporting a bundle whose k-anonymity backstop was infeasible.
Interactive UI export uses this to clear the bundle blocker; non-interactive
exports leave it |
include_dictionary |
Deprecated, ignored. |
code_readiness |
Optional |
compact |
Deprecated, ignored. |
overwrite |
Logical. When |
Invisibly, the written bundle path.
dat <- data.frame( age = 21:70, score = seq(10, 59), grp = rep(LETTERS[1:5], each = 10) ) spec <- synth_spec(purpose = "demo", seed = 1, engine = "internal") syn <- synthesize_data(dat, spec) export_synthetic(syn, path = tempfile(fileext = ".zip"), include_report = FALSE)dat <- data.frame( age = 21:70, score = seq(10, 59), grp = rep(LETTERS[1:5], each = 10) ) spec <- synth_spec(purpose = "demo", seed = 1, engine = "internal") syn <- synthesize_data(dat, spec) export_synthetic(syn, path = tempfile(fileext = ".zip"), include_report = FALSE)
A synthetically generated dataset of 50 regional summary records for use
as sample input in the DataGangeR Shiny app. Simulates public-health
surveillance data aggregated by region. Generated with set.seed(42).
geographic_samplegeographic_sample
A data frame with 50 rows and 5 columns:
Region identifier (Region_01 through Region_50)
Regional population count
Event rate per 100,000 population
Area classification (Urban / Suburban / Rural)
Assigned risk level (Low / Medium / High)
Synthetically generated via data-raw/Geographic_sample.R
A synthetically generated dataset of 200 individual records for use as
sample input in the DataGangeR Shiny app. Contains demographic and health
variables with realistic distributions. Generated with set.seed(42).
individual_sampleindividual_sample
A data frame with 200 rows and 7 columns:
Integer record identifier
Age in years (18–85)
Sex (Male / Female / Other)
Annual income in dollars (log-normal, some NAs)
Highest education level
Logical smoking status
Body mass index
Synthetically generated via data-raw/individual_sample.R
Disclosure control assumes individual-level microdata. A positive result should drive a non-blocking warning, not a separate policy.
looks_aggregated(data)looks_aggregated(data)
data |
A data frame. |
A list with aggregated (logical) and reason (character).
Reads a data file, profiles it, detects column roles, synthesizes data, and exports a zip bundle suitable for passing to an AI agent.
make_agent_bundle( file, out, purpose = "development", seed = NULL, overwrite = FALSE, ... )make_agent_bundle( file, out, purpose = "development", seed = NULL, overwrite = FALSE, ... )
file |
Path to the input data file. Passed to |
out |
Path for the output zip file. |
purpose |
Synthesis purpose preset. Defaults to |
seed |
Optional integer random seed for reproducible synthesis. |
overwrite |
Logical. When |
... |
Additional arguments passed to |
Invisibly, the written bundle path.
dat <- data.frame( age = c(24, 28, 35, 41), score = c(88, 91, 84, 95), grp = c("A", "A", "B", "B") ) path <- tempfile(fileext = ".csv") readr::write_csv(dat, path) make_agent_bundle( file = path, out = tempfile(fileext = ".zip"), purpose = "demo", seed = 1 )dat <- data.frame( age = c(24, 28, 35, 41), score = c(88, 91, 84, 95), grp = c("A", "A", "B", "B") ) path <- tempfile(fileext = ".csv") readr::write_csv(dat, path) make_agent_bundle( file = path, out = tempfile(fileext = ".zip"), purpose = "demo", seed = 1 )
Produces two bar charts comparing original and synthetic data:
standardized differences for numeric columns and total variation
distance for categorical columns. Requires ggplot2 (Suggests).
plot_comparison(comparison)plot_comparison(comparison)
comparison |
A |
Invisibly, a list with two ggplot objects: numeric and
categorical. Each is NULL if no columns of that type exist.
dat <- data.frame(x = 1:10, y = letters[1:10]) spec <- synth_spec(purpose = "demo") syn <- synthesize_data(dat, spec) cmp <- compare_synthetic(dat, syn) if (requireNamespace("ggplot2", quietly = TRUE)) { plot_comparison(cmp) }dat <- data.frame(x = 1:10, y = letters[1:10]) spec <- synth_spec(purpose = "demo") syn <- synthesize_data(dat, spec) cmp <- compare_synthetic(dat, syn) if (requireNamespace("ggplot2", quietly = TRUE)) { plot_comparison(cmp) }
Scans original and (optionally) synthetic data for disclosure-risk flags.
Supports two stages: "pre" (before synthesis, requires only the
original dataset and roles) and "post" (after synthesis, requires both
original and synthetic).
privacy_check( original, synthetic = NULL, roles = NULL, stage = c("pre", "post"), spec = NULL )privacy_check( original, synthetic = NULL, roles = NULL, stage = c("pre", "post"), spec = NULL )
original |
The original data frame. |
synthetic |
Optional; the synthetic data frame (required for
|
roles |
Optional; a |
stage |
Character. |
spec |
Optional; a |
An S3 object of class dataganger_privacy_check, a tibble with
columns variable, flag, severity, stage, and recommendation.
df <- data.frame(id = 1:50, x = rnorm(50), city = rep("Toronto", 50)) roles <- detect_roles(df) privacy_check(df, roles = roles, stage = "pre")df <- data.frame(id = 1:50, x = rnorm(50), city = rep("Toronto", 50)) roles <- detect_roles(df) privacy_check(df, roles = roles, stage = "pre")
Profiles each column in a data frame, detecting type, computing summary statistics, missingness, cardinality, and flags for free-text, dates, and haven-labelled vectors.
profile_data(data)profile_data(data)
data |
A data frame or tibble. |
An S3 object of class dataganger_profile, which is a list
containing:
profile: a tibble with one row per column.
n_rows: total number of rows.
n_cols: total number of columns.
generated_at: POSIXct timestamp of when profiling ran.
df <- data.frame( id = 1:5, name = letters[1:5], score = c(10.1, 15.2, NA, 13.8, 11.0) ) profile_data(df)df <- data.frame( id = 1:5, name = letters[1:5], score = c(10.1, 15.2, NA, 13.8, 11.0) ) profile_data(df)
Reads CSV, Excel (.xlsx, .xls), SAS (.sas7bdat), and XPT (.xpt)
files into a tibble. Dispatches on file extension.
read_input(file, sheet = NULL, encoding = NULL, col_select = NULL, ...)read_input(file, sheet = NULL, encoding = NULL, col_select = NULL, ...)
file |
Path to the data file. |
sheet |
For Excel files, the sheet name or index to read. Passed to
|
encoding |
Character encoding for CSV files (e.g. |
col_select |
Optional character vector of column names to keep. For CSV
files the selection is passed directly to |
... |
Additional arguments passed to the underlying reader
( |
A tibble::tibble(). SAS/XPT imports preserve haven_labelled
vectors as-is.
path <- tempfile(fileext = ".csv") readr::write_csv(data.frame(id = 1:3, grp = c("a", "b", "c")), path) read_input(path)path <- tempfile(fileext = ".csv") readr::write_csv(data.frame(id = 1:3, grp = c("a", "b", "c")), path) read_input(path)
Prints a pre-filled GitHub issue you can copy into your browser for the
lennon-li/dataganger repository, with package and R environment details
already populated. Use this to report a bug, suggest a feature, or send
general feedback without copying session details by hand.
report_issue( message = NULL, context = NULL, type = c("feedback", "bug", "feature") )report_issue( message = NULL, context = NULL, type = c("feedback", "bug", "feature") )
message |
Character. A short description of the problem or suggestion.
If |
context |
Character. Optional context about where the issue happened,
such as |
type |
Character. One of |
Invisibly, the GitHub issue URL that was printed.
if (interactive()) { report_issue( message = "The export step was unclear when I skipped the dictionary.", context = "Shiny app", type = "feedback" ) }if (interactive()) { report_issue( message = "The export step was unclear when I skipped the dictionary.", context = "Shiny app", type = "feedback" ) }
Opens the DataGangeR interactive workflow in a local Shiny app. Requires
the shiny, bslib, DT, and plotly packages.
run_app(max_upload_mb = 50, launch = interactive(), port = NULL, ...)run_app(max_upload_mb = 50, launch = interactive(), port = NULL, ...)
max_upload_mb |
Maximum file upload size in megabytes. Default 50. |
launch |
Whether to launch the app. Default |
port |
Port to pass to |
... |
Additional arguments passed to |
Invisibly NULL.
if (interactive()) { run_app() }if (interactive()) { run_app() }
Given a profile_data() profile (which carries cross-column coverage
information), suggests how many rows to synthesize so that the synthetic
data can still represent every category combination and every category
level observed in the original data, without blindly matching a large
original row count.
suggest_min_rows( profile, roles = NULL, data = NULL, k = 5L, threshold = 1000L, cap = 5000L )suggest_min_rows( profile, roles = NULL, data = NULL, k = 5L, threshold = 1000L, cap = 5000L )
profile |
A |
roles |
Optional; a |
data |
Optional; the original data frame. When provided alongside
|
k |
Reserved for a future k-anonymity-style cell-size floor; unused by the current coverage rule. |
threshold |
Row count at or above which a reduction is suggested. |
cap |
Maximum suggested row count from combination coverage. |
The rule (coverage-based) is:
For small inputs (fewer than threshold rows, default 1000) the
original row count is kept — there is nothing to gain from reducing.
Otherwise the suggestion is the number of observed cross-column
category combinations, capped at cap (default 5000) to avoid
suggesting millions of rows on wide data, and floored at the largest
per-column distinct count so every level remains representable. The
suggestion never exceeds the original row count.
Continuous columns are covered by preserving their min/max (already handled by the synthesis engine); they do not raise the suggested count.
A list with:
Suggested integer row count.
Human-readable explanation.
Original row count.
Observed category-combination count (or NA).
Per-column distinct floor used (or NA).
TRUE if the cap bound the suggestion.
TRUE if the suggestion is below the original count.
p <- profile_data(datasets::iris) suggest_min_rows(p)p <- profile_data(datasets::iris) suggest_min_rows(p)
Builds a synthesis specification from a purpose preset with optional
user overrides. The specification records the synthesis parameters but
does not check engine availability - that is done by synthesize_data().
synth_spec( purpose, level = NULL, n = NULL, roles = NULL, privacy = NULL, name_strategy = NULL, seed = NULL, engine = NULL, acknowledge_risk = FALSE, ... )synth_spec( purpose, level = NULL, n = NULL, roles = NULL, privacy = NULL, name_strategy = NULL, seed = NULL, engine = NULL, acknowledge_risk = FALSE, ... )
purpose |
Character. A single non-missing string: |
level |
Character or |
n |
Integer or |
roles |
A |
privacy |
A |
name_strategy |
Character or |
seed |
Integer or |
engine |
Character or |
acknowledge_risk |
Logical. Required to be |
... |
Additional decision parameters passed to the spec list. These are the same settings exposed under Synthesis Settings in the app:
|
An S3 object of class dataganger_spec (a named list).
synth_spec(purpose = "demo") synth_spec(purpose = "development", n = 200, seed = 42) synth_spec(purpose = "analytics", acknowledge_risk = TRUE)synth_spec(purpose = "demo") synth_spec(purpose = "development", n = 200, seed = 42) synth_spec(purpose = "analytics", acknowledge_risk = TRUE)
Creates a synthetic copy of a dataset using the specified specification and engine. The internal engine supports schema-only (Level 1) and marginal (Level 2) synthesis. The optional synthpop engine is used for objectives that request moderate or high relationship preservation.
synthesize_data(data, spec, roles = NULL, engine = NULL)synthesize_data(data, spec, roles = NULL, engine = NULL)
data |
A data frame to synthesize from. |
spec |
A |
roles |
Optional; a |
engine |
Character or |
An S3 object of class dataganger_synthetic, a tibble with
attributes spec, original_dims, seed_used, and generated_at.
Set options(dataganger.disable_synthpop = TRUE) to steer
auto-derived synthesis onto the internal engine even when synthpop is
installed. This is intended for environments where a synthpop synthesis
is undesirable or can hang unattended (for example continuous
integration). An explicit engine = "synthpop" request is still
honoured; only objective-derived routing is affected.
dat <- data.frame(x = 1:5, y = letters[1:5]) spec <- synth_spec(purpose = "demo") syn <- synthesize_data(dat, spec)dat <- data.frame(x = 1:5, y = letters[1:5]) spec <- synth_spec(purpose = "demo") syn <- synthesize_data(dat, spec)
A synthetically generated dataset of 365 daily records for use as sample
input in the DataGangeR Shiny app. Simulates environmental monitoring
data across multiple sites. Generated with set.seed(42).
temporal_sampletemporal_sample
A data frame with 365 rows and 5 columns:
Measurement date (daily from 2023-01-01)
Site identifier (SITE_A through SITE_E)
Numeric measurement value (some NAs)
Ambient temperature in degrees Celsius
Logical quality-control flag
Synthetically generated via data-raw/temporal_sample.R