The hotfix was a resounding success! The intermittent slowness has dissipated and the servers are humming along nicely. We humbly apologize for the hiccup and any inconvenience it may have caused. Our team will be performing a post-mortem in the coming days to help us identify and squash similar bugs *before* they make it to the production servers. Cheers!
Posted Feb 03, 2021 - 15:25 MST
Monitoring
After sifting through stacks of code, error logs and memory dumps, we have high confidence that the offending bug has been identified. A hotfix has just been deployed and we are now monitoring the servers to confirm the remedy is working as expected.
Posted Feb 03, 2021 - 14:26 MST
Update
As we push into the evening hours here in the United States, our engineers are still working to identify the underlying issue. The good news is that the earlier increase in server capacity has contributed greatly to the stability of the platform. Unfortunately, the as-yet unidentified issue is still triggering short bouts of slowness for a small number of users. Though it's proving to be a troublesome bugger to track down, we're 100% committed to resolving this issue and taking any necessary steps to mitigate it form happening again in the future.
Posted Feb 02, 2021 - 16:43 MST
Update
Our engineering sleuths are still deep diving into the server logs looking for troublemakers. In the meantime, we've added additional servers to handle site demand, which has largely stabilized performance. We'll keep you posted on any new developments.
Posted Feb 02, 2021 - 13:46 MST
Update
Not much new to report. For most folks, the site is operating normally. There are still random fluctuations in server performance that may cause intermittent slownesses. Typically, these hiccups resolve themselves pretty quickly. Will share more information as we learn more.
Posted Feb 02, 2021 - 12:34 MST
Update
Server performance remains inconsistent as the team continues to troubleshoot. This one is a real head scratcher folks. Will continue to update as we learn more.
Posted Feb 02, 2021 - 11:36 MST
Update
Site stability is better, but we have not yet identified the root cause of the issue. We have convened a virtual War Room to enhance our troubleshooting efforts. Based on the flurry of excited techno jargon coming out of these discussions, I'm optimistic we'll soon identify the culprit. Thank you for bearing with us.
Posted Feb 02, 2021 - 10:23 MST
Investigating
We're seeing an uptick in degraded server performance again, which may result in slow loading times. Apologies for the ongoing instability. We appreciate your patience while we continue to investigate.
Posted Feb 02, 2021 - 09:45 MST
Update
We are continuing to monitor the health of the servers. Response times are much better but still fluctuating a bit. Our engineers are continuing to hunt for the root cause of the issue.
Posted Feb 02, 2021 - 09:12 MST
Monitoring
Our crack team has deployed a speculative fix that appears to have improved site stability. Users should no longer experience slow loading times. We’ll continue monitoring just to be safe.
Posted Feb 02, 2021 - 08:38 MST
Investigating
We’re currently experiencing a service disruption affecting all platforms. Our engineering team is working to identify the root cause and implement a solution. All consumer and business users may see slow response times. We will send an additional update in 20 minutes.
Posted Feb 02, 2021 - 08:19 MST
This incident affected: iOS Applications (iOS Business) and Android Applications (Android Consumer).