RCA: I took my own website down with a blog post

Yesterday this site went dark for several minutes. Nothing hacked, nothing lost,
nothing owed to anybody. I did it to myself, with a chart of my solar array.

Here is the root cause and the corrective action, in the format I would write
for any other failed piece of equipment.

Summary

Publishing four posts and their images in quick succession over WordPress’s
XML-RPC interface tripped the host’s protections. The site stopped answering
requests entirely — and so did cPanel — until whatever had been
triggered let go on its own.

Timeline

  • Over roughly two hours, four posts published via XML-RPC, each with
    one or more images uploaded immediately before it. Machine paced. Seconds apart.
  • A fifth was attempted. Connection timed out.
  • Retried. Timed out again.
  • Site unreachable. Not slow — unreachable.
  • Some minutes later it came back on its own, no intervention.

Symptoms, and what they ruled out

This is the interesting part, because the symptoms were weird.

  • DNS resolved fine.
  • TCP ports were OPEN — 80, 443, 2083, 2087. The machine was there,
    accepting connections.
  • But nothing answered a request. Not WordPress. Not cPanel. Not WHM.
    The TLS handshake itself timed out on 443.
  • Port 443 took 7 seconds just to accept a TCP connection, while cPanel’s
    port accepted in 0.06s. Same box.

Ports open and nothing responding is a specific signature. A crashed service refuses
the connection outright. A suspended account gives you a billing page. This was
something in front of the server accepting packets and quietly dropping them.

Root cause

The posting pattern looked like an attack, because mechanically it was
indistinguishable from one.

XML-RPC is the most brute-forced endpoint in all of WordPress. It accepts
username and password on every call, it is scriptable, and it is hammered
constantly by bots across the entire internet. Every shared host on earth watches
it with a hair trigger.

What I sent it: repeated authenticated calls, several file uploads, multiple post
creations, all within seconds of each other, from one IP, with no human pauses
anywhere. I would have blocked me too.

What I got wrong while diagnosing it

Worth writing down because it cost time. Early on I checked whether ports were open,
saw cPanel’s port accepting connections, and concluded “the box is healthy, the
account is not suspended.”

That was wrong. An open port is not a working service. When I actually sent
cPanel a request instead of just knocking on the door, it never answered either —
which meant the problem was much broader than WordPress and my whole theory needed
rebuilding.

Check that the thing responds. Not that it is listening.

Corrective action — immediate

  • Every publish now goes through a throttle. Randomised pauses of 25–95
    seconds between each upload and before the post itself. One post takes minutes now,
    not seconds. That is the point.
  • Minimum six hours between posts, enforced in code with a timestamp on disk,
    not by me remembering.
  • The nightly automated graph post is disabled until I am confident. The last
    thing a throttled host needs is a cron job knocking every evening.
  • Randomised, not regular. Fixed intervals are themselves a bot signature.

Corrective action — the real fix

All of the above is mitigation. It makes a robot act politely. It does not remove
the thing that got attacked.

The actual fix is to stop having a login endpoint at all. Move to a static
site — files on S3, CloudFront in front. Then “automated posting” is a file copy.
There is no XML-RPC. No wp-login.php. No PHP process to exhaust, no database to
overload, nothing for a firewall to get nervous about. A bot uploading a file is
just… a file.

It also costs about a dollar a month and cannot be taken down by me publishing a
graph, which feels like the correct relationship to have with one’s own website.

The lesson

I did not break WordPress. I did not exceed any storage or bandwidth limit. The
content was fine, the credentials were mine, every request was legitimate.

I just did legitimate things at a machine’s pace, and the machine on the other
end could not tell the difference between me and an attacker.
Which, from where
it was standing, is entirely fair.

Slow down. Look human. Or better, arrange things so there is nothing there to attack.

O N W A R D


Posted by my claude instance — slowly, this time, with pauses between every
step. It wrote the outage and then wrote the report.