A few days ago, I received an email from a reader of one of my books. Among other things, he said that he was having trouble getting to one of the websites I'd referenced in the book. I responded that I would check it out. Usually, something like this is due to a misprinted URL in the referring article or book, or it could be that I'd deleted or changed a page on my website.
That was not the case this time. When I clicked on the link to my website, I was faced with—horror of horrors—an online casino.
I thought this would turn out to be a simple case of DNS man-in-the-middle or something similar. Certainly, nothing would be wrong on my own server.
Finding the problem
I use Google Domains as my registrar. Before doing anything else, I checked to ensure that my IP addresses were correct. They were.
I logged into a remote Linux host that I have access to, and performed a traceroute with MTR (Matt's TraceRoute). That indicated that the route to my host was correct.
This did not look good.
Next, I looked at my
httpd.conf and verified that it was correct. I did find a couple of non-related configuration issues, and fixed those, but they didn't affect the problem at hand. I isolated my network from the internet, and tried my website again. I have internal DNS that works for that. I still came up with the invasive website. That was proof positive that the problem was an infection of my own server.
None of that took long. I was just working under the assumption that the problem was elsewhere rather than on my own server. Silly me!
I finally looked at my server's WordPress installation. I was hoping that the database hadn't been infected. I could have recovered from anything by wiping it all out and restoring from backups, but I was hoping to avoid that if possible. Unfortunately, the
html directory of my website had some noticeable, "extra" files and one new directory. The
html/wp-admin/admin.php file had also been replaced.
I was fortunate to have multiple other websites that weren't infected, so it was easy to compare file dates and sizes. I also keep complete daily backups of my websites in order to recover from problems such as this.
Fixing the problem
The fix, in this case, was surprisingly easy. WordPress is quite easy to install, backup, move, reinstall, and restore. I started by deleting the obvious extra files and directory. I copied the known good files from my backups over the infected ones. I could have simply restored everything from the last known good backup and that would have worked as well. I compared the good backup with the recovered website and all looked good.
The database for the website was not affected in any way, which I verified with manual review of a data dump.
The real problem
After analyzing the problem, I realized that I was the root cause. My failure to ensure that WordPress was properly updated for this website allowed this to happen. I use separate instances of WordPress for each of my websites, so the others were not affected because they were being updated automatically.
A series of issues led to this failure of mine.
The affected website had been set up with a different email address that I'd stopped using a few months ago. This prevented me from getting the usual notices that upgrades were available.
I'd also failed to configure that site for automatic updates from WordPress.
And I didn't bother to check to see whether the site was being updated.
When it was attacked, the site was at least one full release level behind what was available. The ones that were kept up to date were not affected by this attack.
What I learned
Having written many books and articles in which I discuss the necessity to keep systems updated with the latest versions of operating system and application software, I'm frankly embarrassed by this. However, it has been a good learning experience for me and a reminder that I must not become complacent. I almost didn't write this article! I didn't want to admit to being negligent with one of my own systems. And yet, I felt compelled to write about it in the hope that you learn from my experience.
So, as I have learned from painful experience, it is critical to keep our systems updated. It's one of the most vital steps in the continuing battle to prevent the computers under our care from being infected. The specific details of the infection I experienced are less important than the fact that there are always attacks taking place against our systems. Complacency is one of the attack vectors that crackers can count on to aid their efforts.