April 21, 2023

Update on the 420 Ecommerce outage

By
Tim Barash
April 21, 2023

Dispensary Partners,

‍

We let you down yesterday.

‍

Dutchie experienced a significant outage which impacted our E-Commerce services. Β Early indications from our investigation are that the outage was caused by an issue with our database provider that powers many of the largest companies in the world.

‍

Let us be clear, regardless of root cause, we understand that Dutchie did not deliver for you, your team, and your customers on the biggest day of the year. Β We know that we need to do better, and we are relentlessly committed to stability now and in the years ahead.

‍

While we are still working on a root cause analysis detailing the factors that led to the outage, we wanted to share early findings with you today. Β For context, Dutchie E-Commerce utilizes a high availability database with multiple redundant nodes to protect against any single points of failure. Despite having multiple layers of resiliency and multiple redundant systems, a series of cascading failures within our database provider resulted in the outage. Our database partner powers many Fortune 100 companies and we have had no issues with them historically of any magnitude. Β We do not have any indications that this outage was caused by increased traffic due to 4/20 and unfortunately could have occurred under normal usage.

‍

Our team and system provider will continue to monitor this system closely to ensure continued stability and we will redouble our efforts to reduce the possibility of these types of failures from recurring. To see the latest updates, please subscribe to our status page.

‍

We know it won’t make up for the frustration, but we will be crediting your account with an amount equivalent to our estimate of your lost profits on orders associated with E-Commerce service interruptions on 4/20. We will provide you with the details of the calculation, as well as a more detailed technical explanation of what happened in the coming days.

‍

Despite what happened, yesterday was still a record-breaking day for our industry. This is a testament to your passion and grit, as well as the product and experience you provide. Our customers saw record setting numbers across all of our platforms, with over 1.2 million transactions processed on point-of-sale, which operated without disruption throughout the day, and over 250 thousand on E-Commerce.

‍

We will update this post when more information is available.

‍

Thank you for your continued partnership,

About the author
Tim Barash
CEO, Dutchie