Previous incidents

We want to follow up on yesterday's incident.

This was unlike anything we have ever seen before. The trigger of the incident was a customer performing a re-seed on a medium to large-sized tenant. Especially one of the schemas was crafted very, unfortunately. Almost all source entities would trigger the schema and because the schema had an action to trigger its parent entity, that cascaded into millions of view generations in just 2 hours.

A lot of the view generation was in this scena...

After yesterday's incident, we have worked to understand the cause and scope of the issue.

The issue affected any tenants utilising the partial schema feature and that updated content in a roughly 90-minute period. The exact issues depended on the specific content and configuration of the customer solution.

Internally we categorise this incident as level 2 "High", and when these incidents happen we are not satisfied. This type of issue should never happen, but when they do we want to ...

We are very sorry for the inconvenience caused by this incident. We understand that our Delivery API is the backbone of your website and any full or partial outages are not up to the standard we hold ourselves to.

We are still working through the details of what exactly happened, but we want to let you know what we know so far. Two aspects of our process failed in this situation.

The first issue is of course that we deployed a release with a serious defect. The problem was not discovere...