Previous incidents

Today's issues was a result of a configuration issue in our deployment process that caused a secret to be set incorrectly. While we believed the issue had been fixed last week, a later update unintentionally reverted the change. Today a service was restarted after our system automatically recovered from a failed probe. This caused one of our services to fail in connecting with the data store as the secret was incorrectly configured.

Resolution:
We manually updated the configuration to restor...

We have investigated the cause of the errors earlier today.

The cause of the problems was due to a new JSON serialiser that was part of today's deployment. As a side note, we can share that we process a lot of JSON, and we can gain a significant performance improvement with the new JSON serialiser, but we digress.

In one specific use case, we had not tested the new code, which resulted in 0.83% of Ingest API requests returning an error response in a scenario where the requests were supposed...

The Ingest API has recovered.

During a routine deployment, the Enterspeed Ingest API did not deploy correctly. The Ingest API service was then manually restarted and recovered after approximately five minutes of downtime.

The error appeared without any warnings from deployments to development and testing environments. We will review the deployment process for any changes that can minimise the likelihood of a similar problem.

If you have any questions, please reach out on your support Slack...