Diagnose common Flowcore failures

Flowcore failures are easiest to diagnose by following one event from producer to projection. Begin with evidence instead of restarting services or widening permissions. The location where the event disappears usually identifies the responsible layer.

Use this order: confirm publication, find the stored event, inspect pathway delivery, inspect handler completion, and finally inspect the projection or downstream side effect. A problem after storage is not fixed by changing ingestion, and a stale UI does not prove that Flowcore lost the event.

Publication fails before an event is stored

A schema error, unknown event path, or oversized payload is deterministic. Correct the contract or data rather than retrying indefinitely. A network timeout or temporary 5xx can be retried with bounded backoff, but keep the same source or command identity because Flowcore may have accepted the first request before the response was lost.

A 401 usually means the credential is missing, invalid, expired, or used with an endpoint that expects another authentication mode. A 403 means the identity authenticated but lacks the required action or resource scope. Check effective permissions for the exact API key or user; do not grant administrator access simply to make the error disappear.

Verify the tenant, data core, flow type, and event type from configuration. Environment mismatches often look like missing data because the producer successfully wrote to a development tenant while operators inspected production.

A Pathways write hangs or times out

When PathwaysBuilder.write() publishes an event that the same process does not handle, the call may wait for local processing unless fire-and-forget behavior is enabled. The event can already be stored while the application reports a timeout and retries it.

Use the supported fire-and-forget option for outbound paths that have no local handler. Keep a stable command identity so application retries remain safe.

Also confirm that baseUrl points to the webhook ingestion service for event writes. The general platform API and webhook ingestion API serve different purposes.

The event exists but never reaches the handler

For a managed Data Pathway, inspect enabled state, subscribed flow and event types, endpoint URL, authentication headers, and recent delivery errors. A renamed secret or unreachable internal hostname can stop delivery while event storage remains healthy.

For a virtual pump, inspect actual checkpoint progress rather than relying on process health. Common PostgreSQL-backed installations can reveal the last processed event and per-flow cursor with:

SELECT max(created_at) AS last_processed_at
FROM pathway_state;
 
SELECT flow_type, time_bucket, event_id
FROM pathway_pump_state
ORDER BY flow_type;

If recent matching events exist in Flowcore but these positions remain old, the pump is stalled. Leader heartbeats only prove that cluster coordination is alive; they do not prove that handlers are advancing.

A new flow type with no checkpoint deserves special attention. Some virtual-pump versions initialize near the current time and can skip events emitted before the pump started.

Delivery repeatedly fails on one event

Repeated failure of the same event usually indicates invalid data, an unsupported event version, or deterministic handler logic. Increasing concurrency only expands the retry storm.

Inspect the failing event ID and handler error without copying sensitive payloads into logs or tickets. Reproduce the event in a controlled environment, fix the handler or publish a correcting fact, and then restart from a position that includes the failed event.

If the event cannot be processed safely, use the platform's approved skip or failed-event workflow and document the resulting projection gap. Skipping is a data decision, not routine error handling.

The handler runs but the projection is stale

A handler can return success after swallowing a failed database write, or update a table that the query API no longer reads. Compare the latest projection sourceEventId with a recent stored event and verify that the query path points to the expected database and schema.

Required projection work should run in a transaction and throw on failure. A 2xx transform response tells managed delivery that processing succeeded and may advance its position. Returning success while required work failed can create a permanent gap.

Caching can also hide a correct projection. Query the projection directly before changing pathway configuration, then check application and CDN cache behavior separately.

Duplicate events or side effects appear

At-least-once delivery makes duplicate handling an application responsibility. A database projection should use a unique event ID or transactional processed-event marker. External API calls should use the event ID or a stable business operation as an idempotency key.

During migration, confirm that a legacy strand and new managed Data Pathway are not both delivering the same events. Also check whether the service subscribed to an event that it publishes, which can create a delivery loop.

Do not remove duplicates from Flowcore history merely to repair a projection. Make the consumer idempotent and rebuild the derived state if necessary.

Validation starts failing after a deployment

Compare the producer and consumer contract versions. Removing a field, changing its meaning, or making an optional field required is a breaking change even when TypeScript still compiles in one repository.

Deploy additive event-type versions in consumer-first order. If several services share schemas, verify the actual package version in each deployed image rather than assuming lockfiles are aligned.

Provisioning or manifest apply fails

Review the diff and the applying identity. Runtime application keys often lack permission to create or update data cores, event types, IAM, and pathways—and usually should lack it.

Move shared resource creation into manifests or a bootstrap job. Check that referenced parent resources exist and that legacy names have not been mixed with v2 names. A 403 during provision() is not evidence that the runtime needs a tenant-wide role.

Local behavior differs from production

Development commonly uses a virtual pump while production uses managed delivery. Differences in endpoint routing, checkpoint storage, auto-provisioning, and secret injection can therefore remain hidden locally.

Log the selected tenant, data core, delivery mode, and application version at startup. Test production-like delivery in staging and verify a real event end to end after deployment.

When recording an incident, include the event ID, affected path, last known checkpoint, application version, and first failing timestamp. That information makes recovery reproducible and avoids broad, speculative changes.

Continue with Recover delivery, replay, and projection gaps after identifying where processing stopped.