Incident lifecycle

Incident resolution has two objectives:

  • Mitigation: Restore normal platform operation.
  • Resolution: Remedy any negative effects (such as adjusting incorrect balances) and take action to avoid future recurrences.

📘

The on-call support team aims to restore normal service operation as quickly as possible. In general, the team acknowledges incident requests within 15 minutes. Mitigation can take up to 4 hours for most incidents, and up to 72 hours low-priority incidents. For more information, see Incident request status.


Incident request status

Every incident request is automatically escalated, no matter the severity. It's Pismo's responsibility to evaluate the incident and determine its priority based on the impact that it has. The following table describes the available priority levels for an incident. For more information, see Request status.

Incident priorityDescription
P1 - CriticalSevere impact:
  • Affects many services or critical processes.
  • Prevents normal operations or causes critical damage.
P2 - HighSignificant impact:
  • Affects some services or processes.
  • Interferes with normal operations.
P3 - MediumPartial degradation of services or financial processes.
P4 - LowMinor impacts to platform performance.

Incident resolution process

After your request is submitted and acknowledged, the resolution process proceeds as follows:

  • A new service request receives the status In Dev. You can add comments during this stage, to provide evidence or additional information.

📘

Reopening requests

If you are not satisfied with a resolution, you should reopen your request instead of submitting a new one. For more information, see Track a service request.

  • When the support team has mitigated the issue, they update the status of the request to Restored.
  • The support team may decline an incident request if:
    • They are unable to reproduce the reported issue.
    • The issue is outside the agreed-upon parameters.
    • The issue is caused by a third party or is otherwise beyond Pismo's control. (However, the support team will clearly communicate the reason for declining an incident request.)

      📘

      The status of a declined incident is Resolved with the resolution Won’t Do.

  • When the support team has resolved the issue and corrected any negative effects, they update the status of the request to Resolved.
  • For resolved P1 and P2 incidents, Post Mortem status indicates that Pismo is investigating how and why it occurred. For more information, see Root cause analysis (RCA).
  • Incidents with Pending status indicate that the support team needs more information or approval to proceed with the planned actions. Pending requests move automatically to Resolved status after 5 days without activity.
  • Closed status represents that the incident is fully concluded and cannot be reopened.

Root cause analysis (RCA)

RCA refers to the process that the Pismo engineering team follows after an incident is resolved. This process determines why the incident occurred and develops the necessary steps to ensure that it doesn't happen again.

🚧

Details about the root cause that contain sensitive or internal information may be omitted.

Incidents identified internally

Incidents that are discovered internally are reported on the Pismo operations status page, along with updates about its resolution. You are welcome to open an incident even if there's already an incident posted on the status page, in case you encounter any other impacts.

📘

Scheduled maintenance activity

Pismo posts notification messages in advance of all scheduled maintenance on the on the Pismo operations status page. In case of an incident related to a pre-communicated maintenance activity, Pismo reserves the right to decline the request but will provide an explanation for the decision.