Setting up an additional event subscriber

Configuring an additional subscriber enables the sharing of event data from one organization (the direct subscriber) with one or more additional organizations (additional subscriber). Additional subscribers can be either:

  • A different organization on the Pismo platform
  • An external system than can consume SNS events

An organization can be an additional subscriber to many other organizations to receive their events data. In addition, you can use event filtering to customize the specific events you are interested in. Access to event data between subscribers can consume either streaming events or batch event files. The event data from one or more organization is sent to another organization.

πŸ“˜

For clarity, this guide shows integrations between AWS clouds. However, it is possible for events to be shared among other cloud providers, such as Azure and GCP. For more information, see Setting up real-time event delivery.

Image illustrating how event filtering works.

A partner organization subscribes to event data from one or more organizations, either by streaming events or batch event-file delivery. The following diagram illustrates the data flows of an additional subscriber organization receiving streaming events or event files from two other direct subscriber organizations:

If there is more than one direct subscriber linked to an additional subscriber, the additional subscribers may receive duplicate events in the same file of the additional subscriber organization. The additional subscriber only reads/receives files with event content from its linked direct subscribers.

The following example code shows event data from two direct subscribers in a file of an additional subscriber:

{
    "event_id": "252212bf-cde2-43f9-b28f-33ea2b452b32",
    "domain": "onboarding",
    "event_type": "succeeded",
    "schema_version": 1,
    "org_id": "TN-a1db4e4e-315f-4a67-9036-ecacd370b561",
    "cid": "8a3f650b-c80c-451f-993d-7368dba4c715",
    "timestamp": "2019-07-20T19:03:00.000Z",
    "data": {}
}
{
    "event_id": "562212bf-cde2-43f9-b28f-33ea2b452b13",
    "domain": "onboarding",
    "event_type": "succeeded",
    "schema_version": 1,
    "org_id": "TN-1234e4e-531f-jhft-5896-asades55655",
    "cid": "9a3f650b-c80c-451f-993d-7368dba4c714",
    "timestamp": "2019-07-20T19:03:00.000Z",
    "data": {}
}

In a case like this, the additional subscriber is able to distinguish events between the two direct subscribers by using their org_id. The names of the direct subscriber files differ from those of the additional subscriber. (Note that there is no standard for file naming.) Though the examples show integrations only between AWS clouds, the integrations with other cloud services are available. For more information, see the Setting up event file delivery guide.

Conditions

Discuss all of the following conditions needed to ask the Data Team (for more information, see Data and reporting overview).

Preconditions

  • The direct subscriber must be on the Pismo Data Platform.

  • The additional subscriber must be on the Pismo Data Platform and must have data integration configured to be able to read/receive SNS events.

Post-conditions

  • Nothing changes for the direct subscriber.

  • The additional subscriber is able to read/receive events data from one of the clouds that the Pismo Data Platform currently integrates (GCP, Azure or AWS).

The sending of the event (or the writing of the file to the bucket) occurs asynchronously. So, it is possible for the additional subscriber to receive access to the file/event before the direct subscriber does.