I suppose these things are sent to try us. Initially I thought I had been hacked like last year, but a quick sweep around the system showed me that nothing was wrong. You see the server would run fine for a day and then die at around 19:00. Weird.
So I backed up and bit the bullet to reformat and start again…I figured it wouldn’t take me all that long and it would give me the opportunity to upgrade all of the software at the same time. Something I have been meaning to do for a while anyway.
That was the plan anyway. Some of you may have noticed a distinct Wolf shaped gap in the blogosphere over the last week…so clearly the plan failed. Not only did it fail, it failed spectacularly. Add to the 19:00 failure and necessary restart to the fact that I couldn’t request a remote reset from my control panel and had to wait hours each time for the (undoubtedly busy) datacenter technicians to reboot the server manually. I eventually decided to get them to reboot into recovery mode so that I could backup and start the reformat.
This is where things really started going wrong. Firstly I decided to ask them to check the physical server for errors. Something that took some time, but ultimately paid off when they mailed me to say that the server had been rebuilt as the hard disk had been replaced. They, of course, waited until I began a vitriol against how “stable” linux was on twitter. Ah well..
So I start installing the necessary packages and restoring my data. Except I couldn’t access my sites. Odd, thinks I…maybe I should check a few logs and see what is happening. Turns out a piece of software that is pretty much just “Install and run” is failing to start properly. In true style, it couldn’t be something basic and unnecessary. Oh no dear readers, it had to be something complicated and absolutely necessary to the operation of the server. I figured I had done something wrong, so cue reformat and install from scratch again…still not working….ok, reformat and install different linux distro…still not working.
Hmm.. I try the old version of my distro. The troublesome software springs into life. Yey, thinks I…until I look at the vital other software I need to install…where the first line says “If you are running the old version, upgrade first….” – Fuck!
I remembered an offer from the company I rent the server from to install the server and their software for me, so I finally give up after delving into the rather scary linux world of compiling your own software from source.. and probably starting the next world war instead of getting a successful installation. Checking the news regularly shows me that the world is no more fucked up than usual, so I assume I have gotten away with it and ask the company to put me out of my misery.
They do this and dutifully mail me to advise that it is up, running and working. 2 minutes later I discover that they are, in fact, wrong. I gleefully inform them of this and they decide to install manually from CD for me the next morning.
So a mere 5 days after the server went down, it was back up and running..leaving me the joyful task of restoring everything.
Then came the test and everything was working…sort of. WordPress decided that it wouldn’t allow half of my plugins to work properly. It was in no way and shape due to me forgetting to install a PHP Module…nosireebob. I would never do that.
So there you have it, The Laughing Wolf is back…now I just need to write something. I feel another server outage coming on ;-)