Incident Affecting Site Uptime Data
Spending hours maintaining client sites?
WP Remote will streamline your WordPress maintenance process and save you at least 4 hours per site every week.
Summary
Here is a short summary of the incident:
Timeline
- 6th May – Issues started
- 7th May – Changed servers
- 8th May – Blocked requests peaked; we received 5x of these alerts
- 10th May – Alerts started waning
- 12th May – Cleaned all customer data to reflect accurate stats
What happened
Between 6th to 12th of May, we experienced issues with external firewalls blocking our plugin requests.
A standalone firewall, Imunify 360, had implemented an invalid rule that was blocking our requests. Imunify 360 is used directly by several web hosts, and therefore this was a widespread issue.
As a consequence of these blocked requests, many customers were receiving incorrect alerts about site downtime.
The sites weren’t actually down, but our uptime requests were blocked and therefore reports have incorrectly shown downtime.
How we dealt with the issue
As this is the nature of dynamic IPs on the internet, we have proactively built in failsafes to avoid these situations. Therefore, when our requests are blocked, we move to different servers. Usually this is sufficient to resolve any issues, and service carries on uninterrupted.
However, in this particular instance, Imunify 360 quickly blocked even the new IPs. Therefore we reached out to their support team, and were assured that they would be rolling out a fix.
The fix itself took a few days to implement, and therefore the incident took longer to resolve, and more customers were affected as a result.
Impact on uptime stats
Many customers were incorrectly alerted to site downtime because of the blocked requests.
We identified the affected sites, and removed the incorrect data. Therefore, all reports currently have the correct information.
However, as some automated reports had been triggered before the data was cleaned, there is a segment of customers who will receive incorrect reports about their site’s uptime.
We recommend regenerating client reports, so they show the correct data. Or sending this incident report as an explanation of what occurred.
Conclusion
Please reach out to us for any clarification. We are happy to assist in any way.
Tags:
Share it:
You may also like
How to Manage Appointment-Based WordPress Websites
Having a strong online presence is crucial for businesses, especially those that rely on appointments. WordPress, a popular and versatile content management system (CMS), is often the go-to platform for…
Insult to injury? AI bots come for your content and site resources
Generative AI is changing content as we speak. The trouble is that none of us know what is going to happen. And now there is yet another spanner in the…
Automate SSL Checks with WP Remote
Have you ever had one or more of your managed websites showing an “insecure” warning to visitors? Or worse, has a client of yours come running to you with a…
How do you manage your websites?
Managing multiple WordPress websites can be time consuming and error-prone. WP Remote will save you hours every day while providing you complete peace of mind.
Managing everything yourself
But it’s too time-consuming, complicated and stops you from achieving your full potential. You don’t want to put your clients’ sites at risk with inefficient management.
Putting together multiple tools
But these tools don’t work together seamlessly and end up costing you a lot more time and money.
