Emergency Process

Let's get it sorted...

First things first, if you have a genuine emergency we’ll do our best to help. We'll be glad that you asked us. Just remember, that emergency (out-of-hours) support is offered on a best efforts basis unless you specifically have a Semantic "Out-of-Hours" contract.

If you know that you're likely to see a significant traffic peak, please try to give us a heads up in advance then we can scale server capacity temporarily.

Initial checks

We have LOOP sites on monitoring so that we can attempt basic troubleshooting.

• 99.9% of the time, a site is probably quickly restarting
  If you see a blank screen, or a Cloudflare error, then this is likely to be the problem.
  An alert is triggered, and we log any repeated alerts for investigation. 1 - 2 restarts per month is normal as Microsoft can deploy tweaks or adjustments to the infrastructure (these are typically overnight)
• If you see an error
  A 500 error is unilkely to fix itself, but 99% of the time a manual restart from us will help.  If an outage is longer than 10 minutes, we get an alert on our phones and will do our best to help with basic troubleshooting (unless you have a full extended hours SLA add-on contract).

If you're emailing us - please try to include: 

• Any recent changes / updates, and what page was updated
• Any pages affected

We ensure code compiles and runs before it's deployed, and avoid deployments just before weekends to minimise the risk of code updates causing any problems.

Sometimes emergencies are caused by external service provider issues or outages beyond our control.  These are rare, but you might like to check the Azure Status Page or Cloudflare Status Page for known issues.

The big tech providers can take a while to update their status pages, so Down Detector is another good resource to check.

You can ping the whole team using our emergency@semantic.co.uk email address.

See our 'Handling Outages and Downtime Checklist' for more tips on maximising guest engagement and making sure you're prepared.

Created by Mollie Young, Modified on Tue, 6 Oct at 2:42 PM by Neil Lewin