Confessions of a Six Figure Professional Blogger

Fighting The Wordpress Dragons

October 8, 2009 | Webmaster Crap

What follows is a quick tale into the technical details of Wordpress that I’d rather not think about, but was forced to…

Over on PCMech, I have been having a cyclical and perplexing issue with server load. The end result was that PCMech disappeared from the Internet for a few minutes at a time, several times per day.

high-server-load-wordpress Extremely frustrating to say the least. Especially when you pay your web host about $700 per month to keep these sites online.

A few weeks ago, I started host shopping. I am with Pair Networks now, but I started talking with the guys at Rackspace. Both great companies. Rackspace is certainly more proficient with their marketing, but they also back it up. I’m not even a customer and I can tell that their “fanatical support” is indeed what it says. That said, Pair is no slouch either. Pair is always spot on with their email replies and they get the job done.

At the end of the day, I’ve got a business to run, however. As good as Pair is, if they can’t get it done for me, I switch. The prospect of moving all of my sites to Rackspace wasn’t exactly one I wanted to confront. At the last minute, fortunately Pair came through for me and offered to throw a lot more hardware in my direction. Essentially, it was an offer I couldn’t refuse.

As I had left it, Wordpress was still proving to be a beast, but we threw so much hardware at it that it was acceptable. Still, PCMech was slower than other sites.

Fast forward to this morning. PCMech is gone. Buh bye. Wordpress is delivering nothing but blank pages.

When I check out the cache, it looks like the problems started shortly after 4AM.

I was clueless. And, again, my host was pissing me off. I pay these guys so much money, and essentially I was getting fast responses and some hints about the problem, but no solutions. They told me, essentially, that they weren’t that familiar with Wordpress. This was surprising considering Wordpress is pretty much the most common blog platform on the net.

After wasting several hours on it, all while PCMech was offline, I was beginning to feel pretty damn helpless. I started thinking about my personal network of contacts.

Mark Jaquith came to mind. He seemed ideal. He lives local to me, we know each other, and he also happens to be one of the lead developers for Wordpress. So, I decide to invade his space, look him up on Skype and I interrupt his day. :)

In short, Mark saved my ass. He’s a good guy to know. :)

Come to find out, there are two problems at play here…

Wordpress Options Table

The Wordpress database has a wp_options table. By design, the Wordpress code queries pretty much every row from this table on each page view (specifically, the ones marked “autoload”).

Well, my options table had over 600 entries and was over 23MB in size!

Mark cleared out some bogus entries, shaving the table down considerably. I went in afterward and removed a bunch of crap from old plug-ins.

Seriously, plug-ins add a bunch of CRAP to your options table. When you deactivate the plug-in, that crap remains in there and is queried out of the database each page view. It is excess baggage. I had to go in and remove entries by hand.

This alone brought PCMech back from the dead. The consistent queries which were returning a result set of over 20+ MB was just killing the server, eating up all the memory, and causing Apache to restart cyclically.

Apache Process Size

This issue remains unsolved as of now, but I’m working on it.

Each Apache process is using up about 50MB of memory. That is insanely high. Wordpress itself, we have ascertained, is only using around 5.5 MB. I’m not loading excessive images from the server, and all videos are hosted offsite.

Still to look into is whether we’ve enabled a bunch of Apache modules that are being called up that I’m not even using. I’m also going to look through the code and see about thinning down the number of images.

Every image and every video (essentially, every single file) has to be served by Apache. Lesser is better.

Wordpress Is a Beast

Wordpress is very query-intensive and uses a lot of memory, especially when you have it loaded up with plug-ins. Of course, I am also using vBulletin on this server, and it uses a lot of database queries, too.

When you start subjecting it to high traffic, it can really take your server to its knees. Some other things I have done include:

  1. Getting rid of all unessential plug-ins.
  2. Installing an opcode cache (like APC or Eaccelerator)
  3. Installing WP Super Cache.
  4. Host databases on a separate server. This is the reason I actually have TWO servers with Pair Networks.

I’m going to continue to watch the server and watch that uptime.

I guess this is the kind of stuff you deal with as part of being a pro blogger.

Oh, and thanks again to Mark. You da man!

If you enjoyed this article, you might also like...

  1. Wordpress and Curing High Server Load
  2. Wordpress Borks My Server? Dealing with High Server Load
  3. Digg, The Server Killer And How To Rescue Wordpress From It
  4. My Blog Was Hacked (I Think) – The Story
  5. Solution – Wordpress Theme Reverts to Default (Kubrick)

Learn The Real Story On Blogging As a Business...

  • Get immediate FREE access to the Six Figure Blogger Blueprint
  • Get exclusive stuff NOT available on this blog.
  • Get the latest updates from the weird Risley man who runs this site.

Enter email:

  • This is great information for all of us who use WP,thanks for sharing
  • It's interesting that you wrote this article a little while ago; if I had read it when you actually posted it, the timing would have been perfect! Anyway, I do NOT have a high traffic site, but am always cool with finding a way to get better performance. I learned from a friend on twitter about this wp_options deal, and I found a ton of old entries and it sped page loads up noticeably. Pretty cool. You're absolutely right, when you get rid of a plugin...it's not gone!
  • Hey David,

    How do you access the options table to manually go in and delete entries? I am not sure how to do it.

    Thanks in advance!

    -Adam
  • joebwharton
    This is great information for all of us who use WP. Some of the plugins mentioned in the comments are on my list to check out. Thanks for the info.
  • An extremely useful post and some insightful comments too. It's a shame more plugins don't include an option to remove themselves completely from the database. I'm off to look at my wp-options table now.
  • Waiting for another phenomenal post… It’d be nice to see some content worth reading (your blog is awesome!) since most of them are getting crappy these days.
  • Alberto
    Dude you are scaring me with all this.

    What to do if it happens to me and i dont know all that mumbo jumbo?

    Who would I call? Sounds like Blogmageddon.

    Let your fearful followers know where to turn to if we shalt fall by the same fate?

    For real David, you gotta get back to us on this and post up somthing about who to turn to for quick damage control. That would make a good post. So keep us informed ok. Peace and good luck with it.
  • There is no answer for this. Its just a matter of maintaining Wordpress and optimizing your server. It isn't something you're going to have a really hard time with unless you have a high-traffic blog. And, by that point, chances are you'll know some people in the field who you can turn to. Often, just posting on Twitter for help will get you many volunteers.
  • David,

    While we are talking about Wordpress using up a lot of space on servers here, I've noticed that WP saves all past versions of changes and modifications of posts and pages. Sometimes when I scroll down to the bottom of my post page after making a small edit, I see a very long line of past revisions. Is there a way I can delete all prior changes and modifications in that database? Seems like such a waste of space WP is using there.
  • Yeah. first of all, you can turn that off by putting the following line of code into your wp-config.php file:

    define ('WP_POST_REVISIONS', 0);

    Secondly, I think the WP-Optimize plug-in has ability to remove all post revisions that already exist.
    http://wordpress.org/extend/plugins/wp-optimize/
  • Great Post.. I have been deactivating or removing plugins this past week. Hoping to speed up my blogs load time.

    Looking into the Op Table now.

    Again nice helpful post. TY
  • I have the wp-optimize plugin installed which clears up the options table everytime you optimize the database. I don't pay as much as you do for hosting but I don't have the traffic either.
  • I have WP-Db-Backup plugin doing automatic backups and optimization, and I think it's helping clearing the overhead on the tables (my wp_options is not nearly that big). There's also spesific Optimize Db -plugin and WP-DbManager also does the trick I believe.
  • Thanks for the insight David, good to know the things that you should look out for.

    I realise I'm probably preaching to the converted, but have you experimented with running up a clone of your site on a fresh WP install elsewhere, or is it just the load that's killing it?

    Also, do you use or have you considered using a CDN? I think Yoast had some good things to say about them on his site recently.

    Cheers
  • I don't think my needs are yet high enough for a CDN, but the thought has crossed my mind.
  • Richard60
    Thanks David, I use Wordpress exclusively for all my blogs, my server is Dreamhost and most of what you wrote about was over my head but I did forward the article to my webmaster
  • If a plug-in is de-activated AND removed, does it still leave crap in your wp_options table?

    Thx for this post -- good info.
  • @cbahm:

    A WordPress plugin may or may not leave options in the options table. It depends on the how the plugin developer wrote the code.

    Some plugins give you the option to completely remove all options when you deactivate and remove. The default is to leave the options alone, so even if you delete the plugin files, their database entries are still present.

    There are reasons why this can be a good thing. There are reasons why it's not so good. WordPress has no official policy that I know of.
  • Hi David,

    Just wanted to let you know that my boyfriend, Richard, also consults and does this sort of stuff for a living -- tracking down weird code bugs for high-performance sites. I'm sure he would be happy to work with you to figure out the Apache issue. You can reach him @floatingatoll on Twitter or contact me offline and I will get you his contact info!
  • I just spent an hour this morning culling plugins. Not done yet either, plan on culling a few more. The admin interface is just too slow. Removing a lot of plugins helped, a little.

    Didn't think to check the options table. That will be a next task. I also have a bbPress installation that isn't being used, so that needs to go as well.

    Thanks for such a useful post, this definitely will help me speed up my WordPress.

    -d
blog comments powered by Disqus