- Notice history

All systems operational

Sending emails in Gmail/Inbox - Operational

Web app - Operational

API - Operational

Rules - Operational

Sidebar - Operational

Authentication & Login - Operational

100% - uptime
Mar 2020
Apr 2020
May 2020

Billing & Plan Entitlements - Operational

100% - uptime
Mar 2020
Apr 2020
May 2020

Notice history

May 2020

Delays in Reply detection
  • Resolved
    UTC
    Resolved

    We've finished backfilling replies for all affected customers since 9:30 PM PDT on May 13, 2020. Rules, Reminders, Sidechat replies, reply analytics, activity syncing, and sequence recipients should be fully up to date.

  • Update
    UTC
    Update

    We have backfilled half of affected customers and are continuing backfill the remaining replies.

  • Monitoring
    UTC
    Monitoring

    We have identified the root cause and are backfilling replies for affected customers. We'll have an update in 30 minutes with the latest status.

  • Identified
    UTC
    Identified

    Starting May 13, 2020 9:30 PM PDT the following areas may have experienced issues with reply detection. These include: * Sequence recipients that replied during the affected timeframe may not have exited from the sequence. * Reply analytics and activity syncing to Salesforce may also have been delayed during the affected time. * Reminders scheduled during the affected time may have triggered, despite a reply. * Rules triggered on sending or receiving emails would not have triggered. * Sidechat replies would not have synced to collaborators.

Apr 2020

Increased latency retrieving Salesforce data
  • Resolved
    UTC
    Resolved

    Delayed rule processing has caught up, and all systems are fully operational.

  • Update
    UTC
    Update

    Processing of delayed rules is still ongoing. All other delayed processing has caught up, and our systems continue to be fully operational.

  • Monitoring
    UTC
    Monitoring

    We've implemented a fix which resolves the latency issues, and are monitoring while our systems process delayed jobs. No data has been lost.

  • Update
    UTC
    Update

    We're continuing to work with our third-party provider to remediate the issue.

  • Identified
    UTC
    Identified

    Some customers are experiencing delays when using functionality that retrieves data from Salesforce. We have traced this issue to a bug in third-party software, and are actively pursuing remediation.

Increased latency sending emails due to Gmail API delays
  • Resolved
    UTC
    Resolved

    This incident has been resolved.

  • Monitoring
    UTC
    Monitoring

    Latency has returned to normal levels on Google's end. We're monitoring their fix.

  • Identified
    UTC
    Identified

    For some users, sending email from Gmail has high latency, both with and without Mixmax. Sequences and rules are still sending, albeit with delays. Google has acknowledged issues with Google Cloud https://status.cloud.google.com/incident/zall/20005 which we believe are causing downstream issues with Gmail. We're monitoring for a fix on their end.

Mar 2020

Increased latency sending emails due to Gmail API delays.
  • Resolved
    UTC
    Resolved

    Google has restored Gmail service for all users: https://www.google.com/appsstatus#hl=en&v=issue&sid=1&iid=1bac5e7a50db851d55b9d50a04512ed0. We are no longer seeing latency when sending emails.

  • Update
    UTC
    Update

    Google has restored Gmail service for some users, and expects a resolution for all users within the hour: https://www.google.com/appsstatus#hl=en&v=issue&sid=1&iid=1bac5e7a50db851d55b9d50a04512ed0. We are no longer seeing latency sending messages, and will continue to monitor the issue until Google resolves the issue on their end.

  • Monitoring
    UTC
    Monitoring

    Latency when sending messages is dropping; we are continuing to monitor the issue.

  • Identified
    UTC
    Identified

    Google has acknowledged issues with GSuite: https://www.google.com/appsstatus#hl=en&v=status. We're continuing to monitor on our end.

  • Update
    UTC
    Update

    We are continuing to investigate this issue.

  • Investigating
    UTC
    Investigating

    Sequences and other messages will continue to send, albeit with delays. We are currently investigating the issue.

Delays in Salesforce rule processing
  • Resolved
    UTC
    Resolved

    We're experiencing delays in rules triggered by Salesforce events.

Elevated error rate
  • Resolved
    UTC
    Resolved

    We experienced a spike in error rates for incoming webhooks. This impacted rules with: - Salesforce triggers - Greenhouse triggers - Incoming webhook triggers This is now resolved.

Mar 2020 to May 2020

Next