Skip to main content

What's New

This page tracks the most recent product-level changes that matter to users of DataGen.

v0.11.0​

This release strengthens representative enterprise evidence and the release boundary around external generation plugins. It is intended for teams that use DataGen to exercise analysis, export, and integration workflows with datasets that remain repeatable and explicit about what they do and do not prove.

What changed​

  • Representative management paths without provider lock-in Opt-in management observations now model the common management surface for each endpoint cohort, with a bounded set of missing, stale, alternative-provider, and hosted-compute cases. Existing scenarios keep their legacy fixed per-company sample unless they opt in, so a routine upgrade does not quietly change their expected population.
  • Policy-setting timestamps that travel with the data Policy settings now receive deterministic source timestamps when upstream fields are incomplete. DataGen derives a missing value from explicit neighboring bounds and the supplied generation time, then exports the completed when_created, when_modified, observed_at, and retrieved_at values. Consumers no longer need to fill those fields from their own clock.
  • Windows publisher metadata with explicit release evidence Hosted Windows CI continuously verifies the portable publisher-metadata contract. Release publication separately requires a fresh signed attestation from the prepared workstation's real D: NTFS and G: ReFS regressions. Source-dependent checks run from a deterministic archive of the clean committed main object, rather than the mutable working tree, while each metadata probe resolves beneath a validated root on the volume it is claiming. The signer parses and checks the preflight evidence before producing a canonical payload that binds the commit, tree id, archive hash, complete snapshot-manifest hash, and passed D/G filesystem claims; GitHub verifies those claims against a repository-pinned public certificate before building or publishing. This prevents a transient working-tree edit from changing the tested source, but does not defend against compromise of the prepared account, Git object database, private key, or signing process memory. This is pinned-key verification, not CA-chain validation or operator identity proof. Current assertions cover owner, group, restrictive DACL, file attributes, and creation/access/write timestamps. SACL and mandatory integrity-label handling is best-effort implementation behavior when Windows privileges permit it and is outside the current regression evidence.
  • A bounded external-plugin intake boundary Plugin manifests and catalogs are read from the authenticated bytes of validated file handles, with size, cumulative-size, parsed-row, and JSON-depth limits. PowerShell and assembly-host output is streamed through bounded capture, giving operators a clear failure instead of allowing an unbounded plugin payload to consume the host.
  • Assembly packages proved as a complete unit An approved assembly plugin is staged from validated handles, including the dependencies it needs. Its deterministic package inventory must reproduce the approved content hash before execution. Required cleanup failures stop the operation rather than allowing a partially cleaned staging area to be treated as success.

Trust boundary​

The new controls authenticate and bound plugin inputs and prove the staged package content. They do not turn an approved plugin into an operating-system sandbox: in-process PowerShell remains trusted code, and an approved assembly plugin runs under the host operating-system identity. Use only reviewed plugins and run DataGen with an account appropriate for that trust decision.

v0.10.1​

This patch corrects a correlation error in representative server-management observations. Earlier generation could label an arbitrary selected server as hosted even when that server's own hosting facts described it as on-premises. Consumers could therefore receive a plausible management row that contradicted the server it referenced.

What changed​

  • Hosting facts now govern the observation DataGen selects hosted examples from servers that are actually cloud-hosted and copies the generated provider from that server. It selects non-hosted examples from the on-premises population and does not invent a provider.
  • Requested coverage remains representative The configured hosted percentage is still honored where the generated population permits it. When one hosting category has too few servers, DataGen fills the remaining observation budget from the available category without changing those servers' facts.
  • Out-of-band capability stays truthful A supported out-of-band guest deployment path is attached only to a real hosted server. On-premises observations remain unavailable, and additional hosted observations remain unknown unless the scenario provides stronger evidence.
  • Selection remains repeatable The same scenario, seed, generated time, and server population produce the same observations. When several hosted providers are present, the bounded sample represents different providers before taking additional servers from one provider.

The normalized schema is unchanged. Consumers should regenerate synthetic datasets that rely on representative management observations so the corrected server-to-observation correlation is present in the data.

v0.10.0​

This release makes representative endpoint-management observations useful for lifecycle and correction workflows without turning DataGen into a provider-specific connector. It adds a small, bounded history surface to the existing opt-in observations and gives downstream consumers the fields needed to distinguish a current fact from the older fact it replaces.

What changed​

  • Provider-neutral lifecycle history Endpoint-management observations remain generic enterprise facts. They retain a management-provider value for realism, but do not introduce a vendor product, tenant, or consumer-specific status model. A current row has lifecycle_state Current; an earlier row has Historical.
  • Explicit currentness and supersession The normalized endpoint_management_observations table now includes lifecycle_state, is_current, and superseded_by_observation_id. Historical rows set is_current to false and point to the current observation that governs them. Current rows set is_current to true and leave the supersession field empty. Use these fields instead of inferring lifecycle from a stale check-in, reachability, or capability value.
  • A separately bounded history budget infrastructure.representativeManagementHistoryObservationCount controls history independently of representativeManagementObservationCount. It is an additional per-company limit, not a multiplier of the current-observation count. Set it to 0 when a scenario needs only current facts; leaving representative observations disabled still emits neither current nor historical management observations.
  • Deterministic chronology For the same scenario, seed, and generated timestamp, the selected current observations and their historical predecessors are stable. Each historical observation is dated before its successor and has an earlier last check-in. Consumers should order a history view by observed_at_utc and then id; they should not rely on the physical row order of an export file.
  • Compatible adoption path The public model defaults remain current-only, and the three normalized columns are additive. Existing current-state adapters can initially filter on is_current = true; history-aware adapters can retain every row and follow superseded_by_observation_id. Accept missing or unfamiliar lifecycle values conservatively when reading exports from an older DataGen release, and keep provider-specific interpretation in the consumer adapter.

For the normalized table and processing guidance, see Normalized Export.

v0.9.4​

This patch corrects the opt-in representative relationship facts introduced for enterprise discovery and integration testing. Generated history now points to real same-company entities and follows the application topology already present in the world.

Highlights​

  • Graph-backed application placement InstalledOn history starts from a real application, follows one of its generated services to a server host, and requires software inventory on that server. Active and removed observations use the same truthful application/server pair.
  • Ownership without invention A missing owner no longer suppresses valid application/server history. DataGen emits Owns observations only when it can select a real same-company person from the application's owning department; otherwise ownership remains absent rather than being fabricated.
  • Exact opt-in behavior Representative relationship history follows the same switch and count as representative management observations. Disabling the feature or setting infrastructure.representativeManagementObservationCount to 0 produces neither surface.
  • Stable consumer boundary The public models and normalized export schema are unchanged. Existing adapters still translate product-neutral DataGen facts, but datasets generated with representative management observations should be regenerated when adopting v0.9.4 so they no longer contain the invalid v0.9.3 relationship references.
  • Bounded website dependency acceptance The current website lockfile still resolves image-size@2.0.2 through Docusaurus, and npm audit does not pass because no compatible upstream fix is available. The build guard and read-only pull-request CI support a bounded acceptance through 2026-09-30; see the repository dependency disposition for the affected surfaces, controls, residual risk, and re-review triggers.

v0.9.3​

This patch makes repeatable generation evidence useful across separate processes and machines without presenting it as signed release attestation.

Highlights​

  • Stable seeded generation Generation choices that previously depended on process-salted hashes now use stable hashing. With the same scenario, seed, fixed times, DataGen version, source identity, and catalog content, supported generation paths no longer inherit per-process hash variation.
  • Portable artifact provenance Snapshots and normalized exports carry portable provenance. Canonical output uses ordinal cross-platform ordering and normalized newlines, while a single deterministic export manifest represents the artifact set.
  • Independent-run receipt tooling The new invocation-contract, wrapper, and receipt scripts preissue two parent challenges, bind each run to its inputs and tooling, then compare canonical payload inventories. The resulting receipt records whether both independently invoked candidates matched.
  • Deliberate evidence boundary This is trusted-operator, unsigned QA evidence. It helps catch stale inputs, accidental reruns, copied candidates, and ordinary process mistakes; it does not authenticate an operator or resist a writer who can alter the evidence files. Declared sensitive arguments are redacted before serialization and hashing, so they are intentionally outside the equality claim.
  • Guarded documentation image inputs The supported npm run build command scans the local website source tree and rejects symlinked paths plus ICNS, JXL, and HEIF-family extensions or signatures before it starts Docusaurus. It is a compensating control for normal builds, not an audit suppression or a defense against someone who intentionally invokes Docusaurus directly: npm continues to report the upstream advisory path until a supported patched release exists.

For the complete operating pattern, see Repeatable Lab Runs.

v0.9.2​

This security patch refreshes the locked documentation toolchain dependencies used to build and preview the DataGen website.

Highlights​

  • Clean website dependency audit The site lockfile now resolves patched transitive versions for the npm advisories identified by repository security scanning, including the critical shell-quote advisory and the related high, moderate, and low findings.
  • Stable documentation workflow The direct Docusaurus dependency set and the generated static-site behavior remain unchanged. The update is limited to compatible transitive dependency releases selected by npm within the existing supported ranges.

v0.9.1​

This security patch refreshes DataGen's SQLite provider stack without changing its generation, export, or PowerShell workflows.

Highlights​

  • Patched portable SQLite provider DataGen now resolves the maintained SQLitePCLRaw 3.x bundle and its cross-platform native SQLite runtime. The update removes the previously vulnerable native package while retaining the same supported Windows, Linux, and macOS operating environments.
  • No workflow changes Catalog generation, normalized exports, and the published PowerShell module keep their existing commands and contracts. This release is intended to be a dependency-security update that is safe to take before generating new datasets.

v0.9.0​

This minor release adds a portable enterprise-data contract for management observations and makes repeatable release datasets easier to generate from the public PowerShell module.

Highlights​

  • Provider-neutral management observations Scenarios can opt into representative endpoint-management facts covering registration, capability, check-in, and hosted-compute state. The model remains generic: it does not encode a downstream product, tenant, engagement, or deployment scenario.
  • Normalized observation exports The normalized profile now emits endpoint_management_observations and relationship_history_observations, giving consumer adapters a stable, documented source surface instead of requiring private generator knowledge.
  • Deterministic module inputs New-SEEnterpriseWorld, Save-SEEnterpriseWorld, and Export-SEEnterpriseWorld now accept explicit generation, snapshot, and export times. Automation can reproduce a dataset and its associated receipt without relying on the clock of the machine running the command.
  • Additive by design Management observations are opt-in scenario data. Existing generation remains valid, and consumers remain responsible for translating the generic facts into their own contracts.

v0.8.4​

This patch release focuses on target-state policy modeling and more useful walkthroughs for turning generated data into actual lab environments.

Highlights​

  • Source and target environment roles Identity stores, OUs, containers, policies, policy settings, and policy target links now carry environment-role metadata so normalized exports can distinguish current-source evidence from modeled target-state controls.
  • Target Active Directory and GPO slice Generated worlds now include a modeled target AD/GPO slice with workstation, server, and privileged-access baselines. This gives downstream policy-parity, migration-planning, and validation workflows a concrete target surface to compare against.
  • Stronger policy export parity Normalized exports now include environment-role fields for the affected identity and policy artifacts, and registry-backed policy settings receive a stable policy-path fallback when no explicit path is present.
  • True walkthrough depth The AD, Entra, and hybrid walkthroughs now continue beyond DataGen export into native PowerShell or Microsoft Graph population patterns, validation checks, cleanup guidance, and repeatable lab refresh workflows.

v0.8.3​

This patch release tightens Active Directory realism and gives lab-population workflows explicit control over generated environment defaults.

Highlights​

  • Optional AD defaults Scenarios can now set identity.includeEnvironmentDefaults to include or omit generated default containers, built-in groups, default accounts, and tenant/default sync objects. AD lab examples disable those defaults so generated create-lists do not collide with an existing domain.
  • More realistic directory shape Generated hybrid AD environments now include multiple top-level OUs, default containers such as CN=Users and CN=Builtin, built-in administrative groups, default accounts, Domain Controllers OU placement, and Entra Connect sync evidence when defaults are enabled.
  • Cleaner password-expiration signals User-focused accounts no longer default to PasswordNeverExpires; that signal is reserved for more intentional deviations such as service, shared mailbox, machine, and selected privileged account patterns.
  • Duckburg freeze refresh The Duckburg ingestion freeze was regenerated from the updated generator output and verified against the deterministic Duckburg scenario.

v0.8.2​

This patch release focuses on stronger Active Directory realism, especially around where OU placement belongs and how delegated administration shows up in the generated directory.

Highlights​

  • Correct endpoint vs directory-object separation Physical workstations and servers no longer carry direct OU placement metadata. OU residency now remains on the corresponding directory accounts, while the endpoint objects themselves stay anchored to location and machine-account relationships.
  • Location-aware OU population Hybrid user and machine accounts are now placed into the intended location-specific OU branches, so city and branch OUs actually contain realistic directory objects instead of reading like mostly empty scaffolding.
  • Broader AD delegation evidence Generated environments now include a richer delegated-administration ACL surface across workstation, server, user, group, service-account, and administrative OUs, which makes downstream directory and control-plane analysis more believable.

v0.8.1​

This release focuses on repository realism cleanup for collaboration-heavy enterprises, especially around what should and should not count as a true file share.

Highlights​

  • Realistic hidden-root share modeling User homes and profile paths are no longer exported as one top-level share per employee. DataGen now models a small number of real hidden roots such as users$ and profiles$, which is much closer to how modern environments are actually run.
  • Limited owner-specific exceptions The generator still emits a small number of direct-owner restricted shares for realistic edge cases like executive, legal, HR, and finance working files, but those now appear as exceptions instead of dominating the repository surface.
  • Cleaner downstream repository counts Flagship datasets such as Duckburg now present a far more believable file-share footprint for SharePoint- and Teams-heavy organizations, which improves downstream CMDB, coverage, and repository analytics.

v0.8.0​

This release focuses on realism expansion beyond the flagship Duckburg bundle, with particular emphasis on topology completeness, CMDB quality, and more believable access/resource surfaces.

Highlights​

  • AD sites and subnet realism Generated environments now include Active Directory sites, site links, site memberships, internal network subnets, and realistic IP allocation for workstations, servers, telephony, and network assets.
  • Stronger CMDB operating shape Configuration items now carry a more believable criticality spread across platform, application, infrastructure, collaboration, software, and data surfaces, reducing the flat analytical profile that broke downstream realism.
  • Cleaner resource and access realism Shared resources and major enterprise applications now tell a clearer group-centric access story, with fewer synthetic naming artifacts in repositories, collaboration workspaces, and supporting ACL surfaces.
  • Duckburg refresh The DTED demo package was regenerated again from the improved source contract so topology, CMDB, repository, and plugin-record realism all start from the newer baseline.

v0.7.0​

This release focuses on realism hardening in the flagship Duckburg dataset, especially around identity, access, endpoint coverage, and release-quality naming.

Highlights​

  • Group-centric access realism Major enterprise applications, shared mailboxes, and file-share resources now surface clearer governing groups instead of leaning on direct assignment patterns or synthetic-looking ACL labels.
  • Identity and endpoint cleanup Device-account display names, sAMAccountName uniqueness, account OU repair behavior, and endpoint/security agent coverage are all tighter and more believable across hybrid and cloud-facing identity surfaces.
  • Stronger organization/output polish Team and resource naming was refined further, late-stage synthetic suffix artifacts were removed, and Duckburg now reads more like an enterprise environment and less like a procedurally stitched demo bundle.
  • Broader realism validation The quality sweep now catches more release-stage issues earlier, including duplicate directory account names, weak access-label patterns, and OU-reference cleanup noise.

v0.6.0​

This release focuses on realism hardening and stronger downstream export evidence, especially for DTED-oriented validation and demo datasets.

Highlights​

  • Organization and reporting realism Large scenarios now generate cleaner department and team structures, more believable reporting lines, and tighter person-to-team alignment instead of fragmented manager spray or breadcrumb team names.
  • Richer DTED-facing evidence Normalized exports now carry stronger policy-setting provenance and CMDB evidence, including typed source and behavior on policy settings plus fqdn, unc_path, rto_hours, and rpo_hours on configuration items.
  • Identity and access alignment Account evidence remains consumer-agnostic, but the generated data is now better shaped for downstream bridges that need lifecycle, state, and non-AD identity-store association cues.
  • Duckburg refresh The flagship Duckburg package was regenerated with the newer realism, CMDB, policy, organizational, and plugin-record surfaces so downstream testing starts from a cleaner baseline.

v0.5.1​

This patch release focuses on docs toolchain security hygiene.

Highlights​

  • Patched website transitive dependency The docs stack no longer pulls the vulnerable uuid path through sockjs; the site now uses a vendored sockjs patch backed by Node's built-in crypto.randomUUID().
  • Clean audit surface The website lockfile was refreshed so npm audit is clean again without waiting on an upstream Docusaurus or webpack-dev-server release.
  • Verified docs runtime Both the static site build and the local Docusaurus development server were validated against the patched dependency tree.

v0.5.0​

This release focuses on source realism and evidence quality for downstream consumers such as DTED, without turning DataGen itself into a consumer-specific adapter.

Highlights​

  • Identity realism cleanup Large flagship scenarios no longer emit dense clusters of duplicate person display names, and account/device identity evidence is more coherent across people, devices, and machine accounts.
  • Stronger export evidence Normalized exports now carry richer lifecycle and classification signals, including account creation/modification timestamps, application type/deployment type, and improved policy-setting path data.
  • Identity store and application realism AD, Entra, and Okta naming/domain surfaces are cleaner and more believable, and application metadata is stronger for downstream typing and relationship construction.
  • Richer policy corpus Policy generation now produces a broader, more enterprise-like management surface with better path realism and stronger scope evidence across Group Policy, Intune, and Conditional Access.
  • Duckburg acquisition scenario The flagship Duckburg scenarios now include an acquired-company path so downstream discovery and migration tooling can exercise integration-oriented company-to-company relationships.

v0.4.4​

This patch release corrects the release-tag lineage so GitHub Actions runs the intended fixed revision.

Highlights​

  • Clean release tag The v0.4.4 tag points at the corrected release commit, so the release workflow uses the fixed flagship acceptance test and portability guardrail changes.
  • No functional regression This release carries forward the same runtime fixes from v0.4.3; the primary change is publishing them under an unambiguous release tag.

v0.4.3​

This patch release focuses on release reliability and test portability.

Highlights​

  • Portable flagship acceptance tests The flagship realism acceptance suite now uses only repo-stable example scenarios, so release builds no longer depend on local generated artifacts.
  • Hardened portability validation The repo portability validator no longer self-matches on its own detection pattern, which keeps CI and release workflows from failing on the guardrail itself.

v0.4.2​

This patch release focuses on release portability and contributor guardrails.

Highlights​

  • Repo portability validation DataGen now includes a validator that checks tracked files for machine-specific absolute paths before they break CI or release workflows.
  • Optional pre-push hook Contributors can enable a repo-managed pre-push hook to run the portability check automatically before publishing changes.
  • Stable realism review defaults The realism review script now defaults only to repo-stable scenario inputs instead of depending on local generated artifacts.

v0.4.1​

This patch release focuses on security and release automation hygiene.

Highlights​

  • Secure machine-account password generation Machine-account passwords now use cryptographically secure randomness instead of the general generator random source.
  • Explicit CI workflow permissions The CI workflow now declares explicit read-only permissions to satisfy current GitHub Actions security policy and keep release automation unblocked.

v0.4.0​

This release is the point where DataGen moved from a stronger enterprise generator into a broader synthetic operating environment platform.

Highlights​

  • Bundled domain packs DataGen now includes first-party packs for ITSM, SecOps, and BusinessOps through the native scenario packs model.
  • Temporal simulation foundations Generated worlds can now include temporal events, drift, and snapshot-oriented export artifacts.
  • Productized scenario authoring Archetypes, persona presets, richer overlays, and an archetype-first wizard now shape the preferred authoring workflow.
  • Major realism hardening Recent work improved organization structure, geography, groups, policies, repositories, CMDB artifacts, applications, infrastructure, and external ecosystem modeling.
  • Built-in quality validation Generation results now include structured quality reports, and the realism review tooling can emit both markdown and JSON for CI and repeatable review loops.

Practical impact​

For most users, this means:

  • easier scenario setup
  • more believable default output
  • better operating-domain coverage
  • stronger validation and demo datasets
  • clearer automation and CI checks around generation quality

Recommended entry points​

If you are starting fresh with the current platform, begin with:

If you are extending the platform, also read:

Notes on release scope​

DataGen continues to treat downstream import shaping as an external concern.

The work in this release deepens the generated source environment itself:

  • richer structure
  • more realistic operating domains
  • temporal behavior
  • stronger quality and realism diagnostics

It does not turn the core product into a collection of consumer-specific adapters.