Govern access, sensitive data, and resource lifecycle

Event history is valuable because it is durable. That same durability raises the cost of weak access controls, accidental deletion, and unplanned sensitive-data ingestion.

Governance must begin when the event model is designed, not after production data has accumulated.

This guide covers four related concerns:

1. Data-core access control
2. Least-privilege IAM
3. Sensitive-data handling
4. Retirement, truncation, and deletion

Separate data-core access from IAM

Flowcore data cores have an access-control setting:

• private limits the data core to its tenant boundary.
• public allows intentional cross-tenant access subject to platform permissions.

IAM answers the more detailed question: which user, role, service identity, or API key may perform which action on which resource?

A private data core does not remove the need for IAM. It narrows the boundary, while policies and bindings control actions inside it.

For production, default to:

spec:
  accessControl: private
  deleteProtection: true

Use public access only when sharing is a documented platform requirement with reviewed consumers and ownership.

Design least-privilege application access

Separate identities by responsibility.

Publishers

A publisher may need permission to ingest specific event types. It normally does not need to delete a data core, administer IAM, read all history, or reset Data Pathways.

Consumers

A consumer may need event read or pathway-processing permissions for selected sources. It may not need permission to publish new facts or create platform resources.

Provisioning automation

A bootstrap or CI identity may need to create and update data cores, flow types, event types, or pathway definitions. Keep this credential separate from the normal runtime key where practical.

Administrators

Human or automation administrators may perform IAM and destructive lifecycle operations. These credentials should not be embedded in application workloads.

Create separate keys per application and environment. A shared tenant-wide key makes ownership, rotation, and incident containment difficult.

Record an access model for each data core

Document:

• The owning team
• Approved publishers
• Approved consumers
• Provisioning owner
• Replay operators
• IAM administrators
• Destructive-operation approvers
• Data classification
• Retention requirements

Review this model when a new event type or consumer is added. Access often expands gradually; periodic review prevents unused permissions from becoming permanent.

Enable delete protection in production

Delete protection prevents accidental removal of a data core. It should be enabled for production resources even when IAM already restricts deletion.

Delete protection adds an explicit lifecycle step. A deletion requires an intentional update to disable protection before the delete request can succeed.

Use a reviewed sequence:

1. Identify the exact tenant and data core.
2. Confirm the business owner approved deletion.
3. Inventory child flow types, event types, pathways, projections, and consumers.
4. Confirm retention, legal, audit, and backup requirements.
5. Stop new production writes if appropriate.
6. Capture final positions and required exports.
7. Disable delete protection in a separate reviewed change.
8. Apply and verify that change.
9. Request deletion.
10. Verify completion and downstream cleanup.
11. Preserve the approval and verification record.

Do not combine “disable protection” and “delete everything” into an unreviewed routine deployment.

Identify sensitive data before ingestion

An immutable event stream should not become a convenient place to copy every field from an incoming request.

Classify payload fields before publishing:

• Public or non-sensitive operational data
• Internal data
• Personal data
• Authentication or authorization data
• Financial or regulated data
• Secrets that must never be events

Passwords, bearer tokens, API keys, private cryptographic material, raw card credentials, and similar secrets should not be published at all.

For personal or regulated data, decide whether the event needs the raw value. Prefer stable references, tokens, classifications, or minimized fields when consumers do not need the original value.

Configure Sensitive Data intentionally

Flowcore event types can include sensitive-data configuration such as sensitiveDataEnabled and a sensitiveDataMask.

At the SDK level, event-type create and update commands accept sensitive-data settings. The mask describes protected structure and supports platform handling of sensitive fields.

A simplified resource example is:

- name: customer.email-changed.0
  description: A customer's email address changed
  sensitiveDataEnabled: true
  sensitiveDataMask:
    key: customer-contact
    schema:
      type: object
      properties:
        email:
          type: string

Sensitive Data configuration is not a substitute for minimization, IAM, transport encryption, or secure application logging. It is one part of the governance design.

Test mask behavior with representative payloads before production. Invalid mask schemas can cause resource validation failures, while incomplete masks can leave fields outside the intended controls.

Plan field removal and scrambling workflows

Flowcore provides event-type operations for removing or scrambling sensitive fields and for listing previous sensitive-data removals.

Use these targeted workflows when compliance requires a protected field to be removed while the surrounding event history remains useful.

A removal request should record:

• The legal or policy basis
• The event type and field path
• The affected application or subject scope
• Whether the operation removes or scrambles data
• The requester and approver
• The execution status
• Verification that projections and derived stores were also handled

Removing a field from Flowcore does not automatically remove copies from every projection, export, cache, search index, warehouse, or backup. Coordinate the complete data lifecycle.

Do not use broad event-type truncation as the default privacy mechanism when a targeted Sensitive Data workflow is appropriate.

Retire an event type without deleting history

Retirement normally means producers stop writing an event version. It does not mean historical events disappear.

Use a controlled retirement process:

1. Mark the event type deprecated in contract documentation.
2. Identify all producers and consumers.
3. Introduce the replacement contract when required.
4. Migrate consumers and replay tooling.
5. Stop old-version publication.
6. Monitor for unexpected writes.
7. Preserve the old schema, fixtures, and migration code.
8. Record the retirement date and retained-history policy.

Keep the old event type available while its history is needed for replay, audit, or migration.

Understand destructive operations

Flowcore exposes asynchronous lifecycle requests for operations such as:

• Truncating an event type's history
• Deleting an event type
• Deleting a flow type
• Deleting a data core

SDK commands can optionally wait for asynchronous completion. A successful request is not equivalent to a complete business cleanup; operators must verify the resulting state and downstream effects.

Truncation

Truncation removes retained events for an event type while the event-type resource remains.

It can break:

• Replay from old checkpoints
• Projection rebuilds
• Audit history
• Historical analytics
• Consumer recovery
• Incident investigation

Use truncation only for an approved lifecycle requirement, test data cleanup, or another explicitly justified case.

Event-type deletion

Deleting an event type removes the platform resource and can invalidate subscriptions, manifests, code registrations, and operational tooling.

Retiring production is usually safer than deleting history and the resource immediately.

Flow-type and data-core deletion

Parent deletion affects child resources and consumers. Data-core deletion is protected when delete protection is enabled and should require the strongest review.

Inventory dependencies before issuing any parent delete request.

Treat deletion as an asynchronous workflow

Destructive commands may accept options such as waitForTruncate or waitForDelete. Whether you wait in the caller or poll later, design for an asynchronous operation:

1. Submit the reviewed request.
2. Capture the operation result.
3. Wait or poll for completion.
4. Verify the resource or history state.
5. Confirm consumers and projections behave as expected.
6. Escalate partial or failed cleanup.

Do not run destructive requests from an application request handler or as an automatic consequence of a missing declaration.

Use an explicit destructive-change checklist

Before truncating or deleting:

• The exact tenant and resource IDs were independently verified.
• The business owner approved the operation.
• IAM permits only the intended operator.
• Delete protection status is understood.
• Producers are stopped or redirected.
• Consumers and Data Pathways are inventoried.
• Replay and projection dependencies are documented.
• Retention, legal, and audit requirements are satisfied.
• Required exports or backups are complete and tested.
• Sensitive Data alternatives were considered.
• The asynchronous completion plan is documented.
• Post-operation verification and rollback limitations are understood.

Keep an audit record

For sensitive-data removal, truncation, or deletion, preserve:

• Who requested and approved the operation
• The reason and policy basis
• Exact resource identifiers
• The command or API operation used
• Start and completion times
• Before-and-after verification
• Downstream systems checked
• Residual risks or incomplete cleanup

The audit record should not contain the sensitive values being removed.

Governance checklist for new event types

Before an event type reaches production:

• The data core is the correct governance boundary.
• Access control is private unless public sharing is approved.
• Publisher and consumer IAM are least privilege.
• Provisioning credentials are separated where practical.
• The payload excludes secrets and minimizes personal data.
• Sensitive fields and masks are reviewed and tested.
• Retention and retirement policies are documented.
• Production delete protection is enabled.
• Replay owners and destructive-operation approvers are known.
• Derived stores are included in privacy workflows.

Return to Define and evolve event contracts for schema and version migration guidance.

For a practical development pipeline, use Build your first event pipeline.