A simpler column-role taxonomy, a de-identify-by-default policy for identifier columns, honest date/time synthesis, and a batch of Configure-page privacy and usability fixes.
simulation = "scramble" — it is kept in the output but de-identified
(its letters and digits are reordered while delimiters and length are
preserved, so no original value survives) rather than silently removed.
Dropping a column is now an explicit action (simulation = "drop").
Recipes or scripts that relied on the old drop-by-default behaviour will
now see the column present (scrambled) unless they set simulation: drop
explicitly; regenerate affected bundles and update recipes as needed."5") or a run of
identical digits ("11"), so a plain integer ID column left its smallest
values in place — a re-identification leak. Such values are now replaced
with random digits of the same width instead, guaranteeing every scrambled
value differs from the original. Genuine alphanumeric IDs are still
reordered (their character multiset is preserved) as before."01/08/2020" or "14:30" were previously detected as
dates but fell through to generic categorical resampling — the original
values were reshuffled across rows with no date-range synthesis and no
coarsen_dates protection. They are now parsed with their own format,
synthesised through the date/datetime/time-of-day machinery, and
reformatted back to the source pattern. Character-stored dates also now
receive the same quasi disclosure default as native Date columns.kanon
attribute, manifest.json, and human/human.md carry suppressed_rows
and suppressed_row_frac, and dataganger inspect reports k-anonymity
status. A single below-k residual cell can cascade into suppressing much
more of a quasi-identifier column than its cell count implies; this is
existing, intended behaviour, now surfaced rather than hidden.stat risk styling now renders.dg_max_comparable_levels()), and the
former "pseudo identifier" type is merged into a single "alphanumeric ID"
identifier catch-all.CRAN resubmission changes requested on 2026-07-10.
'shiny' correctly in the
Description field.read_input() examples are now self-contained, executable, and no longer
use commented-out code.export_synthetic(), make_agent_bundle(), and
export_diagnostic_package() examples now write only to temporary paths
and no longer use \dontrun{}.tempfile() /
tempdir().One minimal export bundle, a Configure page with no silent defaults, and a package-wide audit pass: privacy fixes, an honest engine story, honest privacy wording, and a lighter dependency footprint.
name_strategy = "generic", generic renaming ran first and recorded the
name map before enforce_kanon() dropped direct identifiers and applied
suppression — so bundles could under-protect. Both engine paths now
enforce k-anonymity first. Bundles generated with generic names on
earlier versions may be under-protected; regenerate them.export_synthetic() uses the same roles-derived exclusion rule
as privacy_check_post().decimal_places() samples long
columns with a deterministic stride instead of sample(), removing an
RNG side effect.bslib::font_google() replaced with plain family strings served from
the packaged self-hosted files); the no-network source guard now also
scans inst/app/.synthetic_data.csv at the root,
human/ (human.md with the privacy report folded in, plus optional
comparison_report.html), and agent/ (recipe.yaml = combined
spec + roles, AGENT.md, manifest.json). data_dictionary.csv,
load_data.R, analysis.qmd, ai-readme.md, privacy_report.txt, and
the separate spec.yaml/roles.yaml are gone. CLI synthesize gains
--recipe <recipe.yaml>; --spec/--roles remain supported. compact
and include_dictionary are deprecated no-ops.export_synthetic() honors its code_readiness argument: when
supplied, the bundle gains agent/code_readiness_report.json;
make_agent_bundle() computes it automatically, so every agent bundle
now ships the structural-compatibility report.AGENT.md in every bundle and
using-dataganger-bundles) rewritten to the minimal bundle contract."auto" is a real engine alias in synth_spec(), synthesize_data(),
the CLI (--engine <auto|internal|synthpop>), and spec YAML. An explicit
"auto" behaves exactly like leaving the engine unset: the engine is
derived from the objective and dataganger.disable_synthpop is
respected.engine_required spec field is retired; spec printing
reports the explicit engine or auto (derived from objective).synthpop is unavailable, the upload attestation recommends installing
it for correlation-aware synthesis.user_identifies / user_sensitive) so CLI synthesis is
unaffected.income; flagged columns get a "potential identifier" pill and
semantically-coloured actions.detect_roles() documents the two-axis columns; assorted roxygen and
vignette corrections; the startup message is reduced to version +
run_app().purrr, tidyr, and vctrs dropped from Imports (unused); plotly
moved to Suggests with an install gate in the Compare module.check_code_readiness() now documents and reports that
haven_labelled → character is the expected round-trip for now.Privacy gating, UI/CLI parity, an agent skill, and a provable no-network guarantee.
spec.yaml and roles.yaml, and CLI
synthesize --roles can reuse the full role matrix so UI and CLI runs
reproduce byte-identical output with the same seed.none / combination. The two questions are
framed as the remaining risks after direct identifiers: linkage (combination)
and sensitivity.SKILL.md plus dataganger skill [--out <file>]
so an AI can drive the package to generate synthetic data without ever
reading the real data; fixed ai-readme.md so dropped columns are not listed
as NA (NA).report_issue() prints a copy-paste
GitHub issue instead of opening a browser (the Shiny button shows a copyable
modal). A shipped runtime trap test and source guard prove the package makes
no network calls, and a Linux unshare -rn CI job runs the suite with no
network at all.vignette("privacy-and-ai-workflow") documents the privacy gating ladder,
the two ways to use the package with AI, and the no-network guarantee.Configure redesign around two intrinsic privacy questions.
identifies: none / combination /
direct) and is it sensitive (sensitive) — and derives the treatment
rather than asking the user to pick it. The two questions are shown
prominently above the per-column table.NA bins.synth_spec() documentation for the current settings surface.export_synthetic(compact = ) supports two bundle variants: the compact app
download and the full CLI / agent bundle.Generation, comparison, and export clarity pass:
report_issue() helper plus an in-app Report a problem button open
a pre-filled GitHub issue with environment details, without sending anything
automatically.synthpop (auto))
after generation, instead of always showing auto.:::
self-reference (clearing an R CMD check NOTE); spelling WORDLIST expanded.Configure page clarity pass:
127, not
127.00) by detecting whole-valued numerics, not just R integer storage.... (recommended)), drops the
redundant recommended_role column, and adds a per-row info tooltip with a
plain-English reason and the storage type.+/- affordance.pkgload removed from Suggests (uses .__DEVTOOLS__ namespace check directly).cran-comments.md refreshed with accurate 0/0/2 NOTE explanations.pkgdown site generation via GitHub Actions (_pkgdown.yml + workflows).docs/superpowers/, todo.md) migrated out of the package root.detect_roles() is now conservative: it only auto-assigns a disclosure role
when confident (a clear direct identifier, or a known-sensitive column name).
All other columns are left unselected rather than defaulted to
quasi-identifier. This fixes the root cause of the 100%-NA synthetic output:
measures, counts, dates, and low-cardinality categoricals are no longer
silently treated as quasi-identifiers.None is a valid explicit choice; empty is not.max_suppress_frac feasibility backstop is retained as defense-in-depth.disclosure_roles: mapping (column -> role) so
disclosure decisions are reproducible from the command line.Bug fix — CUSUM hang (Bug 5): Synthesis no longer hangs on datasets with
character-stored date columns (e.g. "Jun 8, 2019") or other high-cardinality
character columns. The root cause was two-fold: (1) character date strings were
classified "unknown" and passed to synthpop as a 2000+-level factor, causing
CART to enumerate billions of split candidates; (2) even moderate-cardinality
character columns (>20 distinct values) used as CART predictors trigger the
same 2^(k-1) blowup at k=34. Fix: detect_roles() now detects date strings
(ISO, "Mon DD YYYY", MM/DD/YY) via regex and classifies them as "date"; and a
new synthpop_bridge_cols() function excludes any character column with >20
distinct values from synthpop's CART, synthesizes it independently via the
marginal engine, and stitches it back into the output in the original column
order. Generation on the CUSUM test file (41 k rows, 14 columns) now completes
in under 10 seconds at both 50-row and 5000-row target sizes.
P1 — Configure busy indicator: the Upload page now shows a "Profiling data…" / "Detecting column roles…" progress bar while the app analyses the uploaded file, so users know the app is working rather than frozen.
P2 — Row count first: the Row count (n) input is now the first item in the Configure advanced-settings panel.
P3 — Role-reactive row suggestion: suggest_min_rows() gains a data
parameter; when called with data and roles, it recomputes the coverage
estimate over only the columns that are still being synthesized. Dropping an
ID or excluded column now immediately lowers the suggested row count on the
Configure page.
P4 — Case IDs render as character: columns detected as ID candidates are coerced to character in the data-panel preview, so a numeric case ID displays as "1078541" instead of "1,078,541.00".
P5 — Column summary stats: the Configure page now shows a per-column summary section below the synthesis settings. Continuous columns get a min / Q1 / median / Q3 / max / mean / SD table; categorical columns get a top-5 frequency table with counts and percentages.
P6 — Generation progress bar + timer: while synthesis is running, the
Generation page displays a MM:SS elapsed-time counter and an animated
progress bar. Both update every second.
Cancellable background synthesis: the Shiny Generation step now runs the
synthesize -> compare -> privacy pipeline in a background process (via
callr), so the app stays responsive and a Cancel button can stop a long or
stuck run. Falls back to synchronous in-process generation when callr is
unavailable or the package is dev-loaded. The dataganger.synthesis_async
option forces the deterministic synchronous path for tests and CI.
Coverage-based row-count suggestion: profile_data() now carries
cross-column coverage (distinct joint combinations of categorical columns
plus the largest per-column level count), and the new suggest_min_rows()
function turns that into a sufficient synthetic row count - capped at 5000,
floored at the largest level count, and never above the original. The
Configuration row-count slider pre-fills with the suggestion, shows an
inline hint, and warns when set below the coverage floor. The Upload step
shows a coverage-summary card.
Diagnostics for long-running synthesis: dg_log / dg_timeit emit
per-phase progress to the R console when options(dataganger.verbose = TRUE), and check_cancel() polls options(dataganger.cancel) at column
boundaries for cooperative cancellation. .onAttach() prints a startup
hint with the package version and how to launch the app and CLI.
Free-text detection now head-samples to 1000 rows, bounding a hot path that could slow the transition into Configure on wide or long-string data.
synthesize_marginal() trusts the detected role instead of recomputing the
free-text heuristic, removing a redundant pass over character columns.
The Comparison step no longer shows a stale full table on first transition: the first comparable variable renders correctly without needing a click.
The Configuration step shows inline help for each disclosure role (None, Direct identifier, Quasi-identifier, Sensitive) with a short example.
callr and pkgload added to Suggests.
The Column Roles step now shows a non-blocking notice when the uploaded data looks like an aggregated counts table rather than individual records, since disclosure control assumes individual-level microdata.
New option dataganger.disable_synthpop: set
options(dataganger.disable_synthpop = TRUE) to steer objective-derived
synthesis onto the internal engine even when synthpop is installed. Intended
for environments where a synthpop synthesis is undesirable or can hang
unattended (for example continuous integration). An explicit
engine = "synthpop" request is still honoured.
Shiny app interface refinements: the Objective step shows each purpose's details on selection and uses consistent "more bars = stronger" meters, including an Anonymity meter for resistance to re-identification. Each synthesis engine (auto, internal, synthpop) now has a plain-language explainer. The Generation step shows a configuration recap (including advanced settings) with an "Adjust settings" shortcut back to Configuration. The Comparison step has a wrapping variable grid and a clearer original vs. synthetic explainer. The Export step downloads a single bundle (synthetic data as CSV, documentation, comparison report, and an analysis notebook), with an optional save-to-folder for sessions run locally.
Every bundle now includes analysis.qmd, a Quarto report with runnable R
code and reference Python code to read both the original and synthetic data
and compare them (summary statistics, distribution plots, and DataGangeR's
fidelity metrics).
The Shiny synthesis spec exposes an engine selector (auto / internal / synthpop) with an inline note on whether synthpop is installed.
Ship an agent skill (inst/skills/using-dataganger-bundles/) describing how
AI agents should consume a bundle: never access the real data, ask the human
for a go-ahead before touching the synthetic data, and where to save work.
Fixed interpolate() so bundle helper files (e.g. load_data.R) render
their templates instead of shipping literal placeholders.
DataGangeR now routes relationship-preserving objectives to the optional synthpop engine when installed. Please cite: Nowok B, Raab GM, Dibben C (2016). "synthpop: Bespoke Creation of Synthetic Data in R." Journal of Statistical Software, 74(11), 1-26. doi:10.18637/jss.v074.i11
Initial CRAN-ready scaffold.