The Pismo API data platform consumes, treats, and validates events that occur during the course of normal processing (account actions, transactions, payments, auditing, and so on). The data from these events is delivered to you in real-time or via files that you can download.

Delivery works on an at least once delivery guarantee. This means that the platform keeps attempting to deliver a message until at least one attempt is successful.

🚧

Idempotency of processing is recommended because the at least once delivery guarantee can result in duplicate events.

Pismo provides JSON schemas to document and validate events and their possible combinations, payloads, and event types. For more information, see Event schemas in this article.

Understanding Pismo event data

Pismo event payload data is formatted in JavaScript Object Notation (JSON). For example:

{
    "event_id": "252212bf-cde2-43f9-b28f-33ea2b452b32",
    "domain": "account",
    "event_type": "customer-creation-1",
    "schema_version": 1,
    "org_id": "<Your Org ID>",
    "cid": "7c7737f0-53d4-9b49-9833-e4c47483634e",
    "timestamp": "2019-07-20T19:03:00.000Z",
    "data": {<Varies according to domain, event_type, and schema_version>}
}

• event_id - Uniquely identifies each event.

• domain - Represents a Pismo application domain. Examples include account, authorization, cards, and so on. You can see a complete domain list in the navigation tab when you view Pismo's JSON schemas in HTML.

• event_type - Represents a processing action within a domain. For example, for account, these could include customer-creation-1, limit-change-1, and status-change-1. The number at the end represents the schema version. You can see a complete event type list in the navigation tab when you view Pismo's JSON schemas in HTML.

• schema_version - Represents the event's JSON schema version. The schema version number appears at the end of the event_type field, for example, limit-change-1.

• cid - The correlation identifier field is used to link related API requests and events. For example, if a user makes an authorization request with 10 installments, this generates 1 authorization event and 10 transaction events all containing the same correlation ID.

• timestamp - Indicates when the event occurred in ISO-8601 format.

• data - Contains the event data itself. This content varies according to domain, event_type and schema_version. See Pismo's JSON schemas in HTML for details on event data.

Event schemas

Every event generated by an API has an associated contract. Contracts take the form of JSON schemas, which you can use for validation and to understand event payload data.

📘

Pismo currently uses the 2019-09 JSON schema version.

The fields common to all events comprise the base contract. What differentiates one event from another is its data or payload field.

You can view available event schemas in these ways:

• JSON schemas in HTML
• JSON schemas in S3 bucket

JSON schemas in HTML

You can view event schemas in navigable HTML. It's the same information, but more user-friendly than reading a JSON document.

While this HTML is useful to view and understand events, for integration, you should use the S3 bucket JSON schemas.

JSON schemas in S3 bucket

If you have an Amazon Web Services (AWS) cloud account, you can retrieve JSON schemas from the Pismo AWS documentation S3 bucket. You will need to periodically check for updates, as no update notification is sent.

Bucket access

In order to access the JSON Schemas S3 Bucket, you have to complete the AWS Files Integration for your org. There, the bucket access is described and can be tested using the testing script provided in the Testing your AWS Integration section.

You can also obtain S3 bucket JSON schemas using AWS SDKs. To know what SDKs are available, see Tools to Build on AWS.

Bucket path structure

The Pismo documentation S3 bucket has the following path structure:

• json_schemas/ - Parent folder for Pismo JSON schemas.

• json_schemas/<domain>/ - These folders represent Pismo application domains such as account, cards, authorization, and so on, and correspond to an event's domain field.

• json_schemas/<domain>/<event type>.json - Within each domain folder are the schema JSON files, named according to an event's event_type field. For example: json_schemas/account/customer-creation-1.json. Note that the schema version is part of the event_type field.

AWS CLI command to list all schemas:

aws s3 ls --recursive s3://pismo-data-dev-dataplatform-docs/json_schemas

AWS CLI command to list all schemas from a specific domain (account):

aws s3 ls --recursive s3://pismo-data-dev-dataplatform-docs/json_schemas/account

New event file notification

Event notifications and data are sent in real-time and via files. For event file delivery, event data is batched for 5 minutes or until it reaches 10MB in size and then saved as a new AWS S3 bucket object to the /main_stream/ path. When this occurs, a new bucket object event is generated and sent with data containing the object's path, size, and name.

As with all events, the dataplatform/new_bucket_object event generates a real-time notification and a new event saved as part of the next bucket object.

Sample new file event payload

{
  "org_id": "TN-f770039e-aabd-498c-9134-9fde4cac4d51", 
  "cid": "d426f496-9fae-491e-a8b0-257eb6b8db16", 
  "event_id": "e03e7919-486e-4552-a7da-ae0b5a2b9a1e", 
  "domain": "dataplatform", 
  "event_type": "new_bucket_object", 
  "schema_version": 1, 
  "timestamp": "2020-04-11T19:22:37.946Z", 
  "data": {
    "object_size": 538, 
    "object_name": "main_stream/2020/04/11/19/dataplatform-tn-f770039e-aabd-498c-9134-9fde4cac4d51-1-2020-06-19-19-17-42-45baae47-28d6-4ef9-9f31-9cd1891d469b", 
    "bucket_name": "pismo-dataplatform-tn-f770039e-aabd-498c-9134-9fde4cac4d51"
  }
}

Although this object doesn't have a .json extension, it contains JSON event data and you can download it directly.

AWS CLI command to download bucket object:

aws s3 cp s3://<your bucket name>/main_stream/YYYY/MM/DD/HH/<object name> <download destination path>