On Wednesday, 3 December, Enterspeed experienced a temporary service disruption that affected the processing of views shortly after a new JavaScript transformation component was deployed.

What Happened

Earlier the same morning, we introduced an infrastructure change that passed review and testing. However, this change contained a hidden issue that only appeared when combined with the later deployment of the transformation component. The issue did not show up in our test environment due to a different deployment sequence.

Impact

• View processing was interrupted starting at 13:04 CET.
• No data was lost.
• All affected jobs were re-queued and successfully processed.
• The full backlog completed at 15:37 CET.

How We Responded

Our monitoring detected the problem immediately. The engineering team assembled within minutes, identified the underlying infrastructure change as the cause, and restored the previous version at 13:34 CET, after which processing resumed. We then re-processed all pending updates to ensure system consistency, with all jobs completing by 15:04 CET.

What We Are Doing to Prevent This

We are reviewing and improving our deployment and testing processes to ensure that sequencing differences between environments cannot hide potential issues. This includes tightening validation around infrastructure changes and improving how production-like deployment sequences are simulated in testing.

Questions or Concerns?

If you have any questions about this incident or its impact, we’re here to help — please reach out in our Slack support channel or email us at support@enterspeed.com.

We apologise for the disruption this incident caused and appreciate your understanding.

We have now re-queued all failed processing jobs to ensure that all data affected during the incident is fully restored.

We apologise for the inconvenience. We will update this status post with a full incident report once we have completed our investigation and identified the key learnings.

If you have any questions, please contact us via your Slack support channel or at support@enterspeed.com.

The service is now restored.

We are now shifting focus to restore the failed processing jobs and will report back when we have complete that. Not all views are up to date yet, but when the restore of the failed processing jobs are complete, no data will be lost.

We believe to have identified the root cause and have applied a fix. We are currently monitoring to see how the system responds.

We are still investigating but have confirmed that all JavaScript schemas are currently not generating views.

The engineering team is fully engaged in understanding and fixing the issue.

Our monitoring systems are currently reporting errors in JavaScript processing. We are investigating.