Delivery retries and recovery

Pismo provides an at-least-once delivery guarantee. If an event cannot be delivered to your destination, the Fanout service automatically retries delivery based on the configured retry policy.

Events are processed in batches of up to 10 events. When a delivery failure occurs, only the affected events are re-queued and retried. Other events continue through the delivery pipeline without interruption.

Every delivery attempt, whether successful or unsuccessful, is recorded in the Delivery Audit Log, including timestamps, destination information, and delivery responses.

Delivery audit logging

To support monitoring and troubleshooting, the Fanout service records every delivery attempt in an immutable audit log.

Audit logging operates independently of event delivery. If an audit log write fails, the platform automatically retries the persistence operation without affecting event processing or delivery.

Failures when writing audit records do not prevent events from being delivered to their configured destinations.

Credential management

For supported cross-account integrations, the Fanout service caches delivery credentials and refreshes them before expiration.

If a delivery failure is detected, the credential cache is automatically reset. This forces the platform to obtain fresh credentials during the next processing cycle, helping prevent repeated delivery failures caused by expired or invalid credentials.

Security

All outbound connections use TLS 1.2 or later.

Depending on the connector type, the Fanout service supports mutual TLS (mTLS) JWT-based authentication End-to-end payload encryption using AES symmetric encryption with RSA public key wrapping

For connector-specific authentication requirements, refer to the connector reference documentation.

Retry and recovery flow

The following diagram illustrates how the Fanout service handles delivery failures, retries, credential refresh, and audit logging throughout the recovery process.

When a delivery attempt fails, the Fanout service records the failure, queues the event for retry processing, and attempts redelivery according to the configured retry policy until delivery succeeds or the retry limit is reached.

Timeouts

To detect delivery failures quickly, each delivery attempt is subject to a connector-specific timeout. When a destination does not respond within the timeout period, the delivery attempt is marked as failed and automatically scheduled for retry.

In addition to connector-specific timeouts, each batch has a maximum processing window of 300 seconds (5 minutes). This limit acts as a safeguard to prevent stalled processing. For HTTP destinations, failed requests are typically detected within 10 seconds, allowing retries to begin quickly.