Previous incidents

May 2025
May 19, 2025
1 incident

High w_await on RX nodes

Degraded

Created May 19 at 12:05am MSK

Our monitoring systems recorded a sharp increase in the w_await indicator on some servers of the RX cluster. This indicator reflects the response time of NVMe drives during write operations.

Since we use RAID1 on our servers to increase the reliability of data storage, its speed depends on the "slowest disk" itself. If at least one disk from the array starts working incorrectly, this is reflected in the entire array.

During an internal investigation, we found that all the problematic drives...

May 02, 2025
1 incident

CloudFlare Network Unavailability

Degraded

Resolved May 03 at 07:03pm MSK

Since we did not receive a clear response from CloudFlare, what exactly is the reason for this incident, we decided to replace the IP addresses with others. The replacement was successful, this subnet has been decommissioned, and customers have no problems working with the CloudFlare network.

1 previous update

April 2025
Apr 30, 2025
1 incident

Service Degradation Due to High CPU Usage in BillManager

Degraded

Resolved Apr 30 at 12:05pm MSK

The issue affecting system performance due to a bug in the BillManager software has been resolved. The abnormal CPU usage has returned to normal levels, and service response times have stabilized. We continue to monitor the situation closely.

The root cause has been identified as a microcode bug in BillManager, and the incident is under further analysis by ISPsystem. No further impact is expected at this time.

We apologize for the inconvenience caused and thank you for your understanding.

1 previous update

Apr 27, 2025
1 incident

Web Protection Issues: 405 Not Allowed Errors

Degraded

Resolved Apr 28 at 12:52pm MSK

At the moment, the incident has been resolved, no repeats are expected.

3 previous updates

March 2025
No incidents reported