Recommend server upgrade

[Darth Wong]

OK ladies and gentlemen, here's the story: the server for this forum has been getting slower and slower over time, as the database gets bigger and bigger (this is why a lot of forums auto-prune or periodically archive old posts). Some of the problems are due to HD wait times, which I can't do much about without spending a lot of money, but other problems are due to CPU loading and RAM limitations, which I should be able to do something about with a little bit of work and not too much money.

Here's the current server spec:

MB: Asus A8N-E Socket 939
CPU: AMD Athlon64 3000+
RAM: 2 GB PC3200 DDR (two 1GB sticks)
HD: 2 WD Raptor 10k rpm 74GB disks
OS: Mandriva 2006.0

I've been thinking of keeping the existing motherboard and upgrading it like so:

CPU: Replace with AMD Athlon 4200+ X2 dual-core
RAM: Add 2 more 1GB PC3200 DDR sticks, for a total of 4 GB
OS: Upgrade to Mandriva 2008.0

This wouldn't cost me too much money, and I'm pretty sure it would make a noticeable difference in performance. Any feedback?

[Ace Pace]

You could probably go with a lower end dual core X2. I don't think the CPU scaling matters that much here.
RAM would work, I suggest that upgrade anyway, but prices for DDR RAM are only going up over time, so I suggest that purchese be done early rather then later.

Not qualified to comment on the OS.

[Darth Wong]

You could probably go with a lower end dual core X2. I don't think the CPU scaling matters that much here.

You're probably right; the second core matters more than the clock speed, especially for a server.

RAM would work, I suggest that upgrade anyway, but prices for DDR RAM are only going up over time, so I suggest that purchese be done early rather then later.

I didn't know that RAM prices were headed upwards. Thanks for the heads-up.

Not qualified to comment on the OS.

Yeah, it's not really so important. It's just an overdue version upgrade that requires taking down the board for a few hours so I keep putting it off, but if I had to take the server off-line to swap out the CPU and add/test RAM anyway, I'd probably do the OS upgrade at the same time. If the board is going to be off-line for at least an hour anyway including a brief RAM test, I might as well take it off-line for a couple of hours.

[Seggybop]

I've read that the dual core socket 939 processors are no longer being made, so it would probably be best to acquire that as soon as possible too.

[Ace Pace]

You could probably go with a lower end dual core X2. I don't think the CPU scaling matters that much here.

You're probably right; the second core matters more than the clock speed, especially for a server.

Newegg seems to have some deals with the 3800+ and RAM but I don't think they ship into canuckistan. Look around though, the 3800+ should be dirt cheap. However, again, looking at Newegg, you might have trouble finding it.

I didn't know that RAM prices were headed upwards. Thanks for the heads-up.

DDR RAM was starting it's phase out years ago and it's starting to spike up now. Not sure how high prices will be in Canada, but in the U.S. prices have passed DDR2 since supply is not being renewed as far as I know. Someone who is actually in the north america region might know more, all I know is that when buying DDR RAM, I ended up paying insane amounts.

[InnocentBystander]

Have you run an explain plan on some of your more time consuming queries? You might be able to improve performance with a couple extra indexes.

[phongn]

You might consider one of the S939 Opterons, as they have 1MB of L2 cache per code. While not as useful for desktop loads, it should be a nice bonus for typical server loads.

[Darth Wong]

Have you run an explain plan on some of your more time consuming queries? You might be able to improve performance with a couple extra indexes.

The worst queries are the search queries; I know that already. The problem is that the search indices are huge. The search word index is 1.2GB, and it isn't getting smaller anytime soon. However, a newer version of MySQL and a couple of extra GB of RAM might help there.

Also, it's worth pointing out that the machine is also used as a desktop computer sometimes, and that draws CPU power away from the server processes, hence the importance of a second CPU core.

[RThurmont]

By the way, if I might ask, what specific version of Mandriva do you use for the board's server? The Free edition? I would assume (hope) its not Mandriva One?

Mandriva One is a nice little distro, but IMO the stripped-down versions of several packages that it contains are really annoying, and the package manager seems to have a hard time doing upgrades on its own. With Mandriva 2007.1 (which I like, due to the fact that Metisse, while for the most part being ugly and totally useless, does cause font rendering to improve considerably), I had to build vim 7 from source in order to replace the useless, stripped down included version.

Speaking of servers, I'm in the process of migrating all of my websites to a dedicated server I recently purchased, and was up until 2:00 AM last night doing work on it. The complex element is that I'm experimenting with virtualization on it, and have been messing around with the idea of running NetBSD and OpenBSD vms atop a Debian base system. However, to my annoyance, the VMWare Server Console is entirely too slow to be used effectively over a remote connection (you can do it, but its just agonizingly slow) so this neccessitates getting the VMs configured locally and then uploading them (and hoping they can be made to work to the point where you can ssh in).

[Uraniun235]

My suggestion would be to consider purchasing used parts; you might be able to get a better deal, especially on the RAM.

Also, are you running a 64-bit version of Mandriva?

[Darth Wong]

By the way, if I might ask, what specific version of Mandriva do you use for the board's server? The Free edition? I would assume (hope) its not Mandriva One?

Mandriva 2008.0 PowerPack download edition. It's just not worth my time to fuck around trying to give the free versions all the functionality that I want, so I paid a few bucks to get the loaded one. Some people are adamantly against paying for Linux because it is nominally free, but time and aggravation are money too, and it's still cheap when you consider the fact that I can install it on every machine I own.

Mandriva One is a nice little distro, but IMO the stripped-down versions of several packages that it contains are really annoying, and the package manager seems to have a hard time doing upgrades on its own. With Mandriva 2007.1 (which I like, due to the fact that Metisse, while for the most part being ugly and totally useless, does cause font rendering to improve considerably), I had to build vim 7 from source in order to replace the useless, stripped down included version.

Yeah, I'm with you there. I hate the stripped-down versions of software you find in certain versions.

Speaking of servers, I'm in the process of migrating all of my websites to a dedicated server I recently purchased, and was up until 2:00 AM last night doing work on it. The complex element is that I'm experimenting with virtualization on it, and have been messing around with the idea of running NetBSD and OpenBSD vms atop a Debian base system. However, to my annoyance, the VMWare Server Console is entirely too slow to be used effectively over a remote connection (you can do it, but its just agonizingly slow) so this neccessitates getting the VMs configured locally and then uploading them (and hoping they can be made to work to the point where you can ssh in).

You can do an awful lot with ssh, but remote X is still pretty damned sluggish even over a fast Internet connection. Luckily, I have physical access to the server so that's not a problem for me. Mind you, I'm not doing anything fancy on it; bbs.stardestroyer.net maps right into the main webroot directory.

My suggestion would be to consider purchasing used parts; you might be able to get a better deal, especially on the RAM.

Not for me. Flaky RAM is annoying as hell and I'd rather get shiny new RAM. Besides, I already ran out and bought a pair of OCX RAM sticks today, along with an Athlon 3800+ X2. That was the fastest one they had in stock for my old Socket939 board.

Also, are you running a 64-bit version of Mandriva?

Yes.

[Ace Pace]

Before installing the CPU, check your BIOS version and compatibility. There is quite a chance that you will need to flash your BIOS in order for it to properly recognise the dual core. If not, either it will not boot, or just recognise one core.

I'm not sure what the flashing procedure is though for you, I imagine burn to disc, since I doubt ASUS cared to provide a Linux based flashing utility like they do for windows.

I imagine you'll want to do the same Prime95 and Memtest you did for the last upgrade. Just remember that any Prime95 test or so must either be run twice(for both cores) or use a version that spreads across cores.

[Pu-239]

Since the server is on Linux, he'll be using mprime instead of prime95. Same idea of running two instances applies though (you'll have to make two copies of the extracted folder and run each mprime simultaneously in it's own directory w/ the -t flag).

[RThurmont]

Mandriva 2008.0 PowerPack download edition. It's just not worth my time to fuck around trying to give the free versions all the functionality that I want, so I paid a few bucks to get the loaded one. Some people are adamantly against paying for Linux because it is nominally free, but time and aggravation are money too, and it's still cheap when you consider the fact that I can install it on every machine I own.

I have no problems with the idea of paying for a Linux distro...indeed, I do it all the time, in the form of Linux Format magazine, which I find to be a great way to randomly peruse the various different distributions out there. One similiar thing to what you described, though, that annoys me, is the increasing militance of the *Ubuntu-using segments of the community...for example, within the local Linux user group, Kubuntu is the standard and everything else tends to be perceived as crap (except for Debian, which seems to be the preferred server variant). I however personally find Kubuntu to be vastly overrated, and Mandriva and Suse to be vastly underrated in the whole grand scheme of things.

You can do an awful lot with ssh, but remote X is still pretty damned sluggish even over a fast Internet connection. Luckily, I have physical access to the server so that's not a problem for me. Mind you, I'm not doing anything fancy on it; bbs.stardestroyer.net maps right into the main webroot directory.

VMWare Server actually uses some modified variant of VNC+X for communication, I'm not real clear on what it is, exactly, but I do know that it is not straight X (but having X is a prerequisite for installation onto a Linux box). The GUI portions of the interface are fine over the Internet, but the actual window through which you view and operate on the guest OS is just too darn slow; you type a character into the shell and wait half a second for it to appear.

[Darth Wong]

I have no problems with the idea of paying for a Linux distro...indeed, I do it all the time, in the form of Linux Format magazine, which I find to be a great way to randomly peruse the various different distributions out there. One similiar thing to what you described, though, that annoys me, is the increasing militance of the *Ubuntu-using segments of the community...for example, within the local Linux user group, Kubuntu is the standard and everything else tends to be perceived as crap (except for Debian, which seems to be the preferred server variant). I however personally find Kubuntu to be vastly overrated, and Mandriva and Suse to be vastly underrated in the whole grand scheme of things.

To be honest, the anti-corporate attitude of a lot of Linux users reminds me of the anti-establishment urban black youth attitudes toward "whitey". You're an "Uncle Tom" if you use a Linux brand that is associated with a corporation, so SuSE's association with Novell and the fact that Mandriva incorporated itself and tries to get business deals are big no-nos as far as the Linux snob set goes. Hence, they will run them down for any excuse they can find. There's nothing about Linux which is inherently anti-corporate; it may be very much against Microsoft's way of doing things but that doesn't mean it has to be anti-business or anti-corporate. But there's a real Bohemian attitude to a lot of the Linux fan base.

VMWare Server actually uses some modified variant of VNC+X for communication, I'm not real clear on what it is, exactly, but I do know that it is not straight X (but having X is a prerequisite for installation onto a Linux box). The GUI portions of the interface are fine over the Internet, but the actual window through which you view and operate on the guest OS is just too darn slow; you type a character into the shell and wait half a second for it to appear.

I keep my remote admin simple: X over ssh. And my ssh is locked down tighter than a nun's cunt: restricted by IP address and forced to use keypair authentication rather than text passwords.

[The Kernel]

If you are having trouble with the performance of the database, might I also suggest you think about a getting a third Raptor drive and doing a RAID 5 array? This will significantly improve your read performance while at the same time giving you an extra bit of security in the event of a drive failure.

As for the CPU upgrade, it's hard to recommend an S939 upgrade right now due to it being a completely dead platform. However that being said, you are probably best putting in a cheap CPU and RAM upgrade and then getting a new machine around this time next year when Nehalem hits.

[Darth Wong]

If you are having trouble with the performance of the database, might I also suggest you think about a getting a third Raptor drive and doing a RAID 5 array? This will significantly improve your read performance while at the same time giving you an extra bit of security in the event of a drive failure.

Is a RAID 5 array a good idea for a database server? I haven't used one for years, but I recall they always had very good read performance coupled with shitty write performance. Also, I'd have to reformat all the drives, which could get messy.

As for the CPU upgrade, it's hard to recommend an S939 upgrade right now due to it being a completely dead platform. However that being said, you are probably best putting in a cheap CPU and RAM upgrade and then getting a new machine around this time next year when Nehalem hits.

Yeah, I was thinking that the dual-core CPU and RAM upgrade will take this platform to pretty much the limit of what it can do. After that, it's time for the forklift upgrade.

[InnocentBystander]

I don't know about the performance of mySql, but what about moving to a more robust database? I'd wager an oracle license is no more expensive than a new CPU.

[Ace Pace]

Amusingly, if you finsh the upgrade, your rig will be pretty near mine.

The Anandtech primer on RAID has this to say on RAID5.

In its optimal form, RAID 5 provides substantially faster read performance than a single drive or RAID 1 configuration, but write performance suffers due to the need to write (and sometimes recalculate) the parity information for the majority of writes performed. While this write performance is still faster than a single disk configuration in most cases, the true performance benefit of a RAID 5 array is in its read ability.

[snip]



RAID 5 finds a comfortable home in most "read often, write infrequently" server applications which require long periods of uptime, such as web servers, file/print servers, and database servers. Dedicated RAID 5 controllers that include large amounts of RAM can negate much of the write performance penalty, but such setups are quite a bit more costly. Note also that simplified RAID 5 controllers exist that require the CPU to perform the parity calculations, which can result in write performance that is lower than a single drive.

Pros:
Good usable amount of data (capacity is the sum of all but one drive in the set)
Fault-tolerant - can survive one disk failure without impact to users
Strong read performance
Cons:
Write performance (without a large controller cache) is substantially below that of RAID 0
Expensive (either in terms of controller cost or CPU usage) due to parity calculations

EDIT: as Phongn said below me, not so usful for a DB like a message board.

[phongn]

Is a RAID 5 array a good idea for a database server? I haven't used one for years, but I recall they always had very good read performance coupled with shitty write performance. Also, I'd have to reformat all the drives, which could get messy.

No! RAID5 is very bad for database loads. If you need to increase the speed, you'd probably have to go RAID10 (not RAID 0+1, which is what some controllers do anyways when you select "RAID10"), ideally on a good hardware controller.

I don't know about the performance of mySql, but what about moving to a more robust database? I'd wager an oracle license is no more expensive than a new CPU.

MySQL is actually quite fast, but it traditionally has achieved that speed by discarding many of the traditional database conventions in very bad ways. Of course, for a message board, it's not bad.

EDIT: Also, lighttpd is gaining popularity as a web server, and is supposed to be quite fast, even under PHP-type loads. You might want to look into it.

[The Kernel]

No! RAID5 is very bad for database loads. If you need to increase the speed, you'd probably have to go RAID10 (not RAID 0+1, which is what some controllers do anyways when you select "RAID10"), ideally on a good hardware controller.

Why is it bad for database loads? Do you have some benchmarks to show this? I've never actually heard that and I would assume that for a situation like SDN read performance of the database would be more critical than write performance and RAID5 is excellent at random read performance.

[The Kernel]

Is a RAID 5 array a good idea for a database server? I haven't used one for years, but I recall they always had very good read performance coupled with shitty write performance. Also, I'd have to reformat all the drives, which could get messy.

Not "shitty" write performance, but yes the main advantage is in random read performance which I assume is probably what is putting the most strain on your database.

As for reformatting the drives, you should just be able to image your current system and reload it once the RAID array is up.

Yeah, I was thinking that the dual-core CPU and RAM upgrade will take this platform to pretty much the limit of what it can do. After that, it's time for the forklift upgrade.

Yes, and given the fact that both AMD and Intel are moving to entirely new mobo platforms next year, it's probably a good idea to sit tight until then. Intel's new socket combined with CSI and a Nehalem based core will probably make a fantastic upgrade in the late 2008 - early 2009 time frame.

[The Kernel]

For anyone who is interested, PC Guide has a good comparison of the various RAID levels.

PC Guide Comparison of RAID levels

[phongn]

Why is it bad for database loads? Do you have some benchmarks to show this? I've never actually heard that and I would assume that for a situation like SDN read performance of the database would be more critical than write performance and RAID5 is excellent at random read performance.

It's the random-write performance that I'm worried about, coupled with the parity cost if Mike has to go software RAID (which I'm certain he would). Don't forget that not only is that machine a server, but it'll face interactive loads as well, further compounding the issue.

In fact, I've never heard anyone seriously advocate RAID 5 for database loads until you brought it up. It's usually RAID 1 or RAID 10 that I hear bandied around. I like RAID 5 (I have a 1TB soft-RAID 5 configuration) but not for databases.

[Pu-239]

Perhaps this mod will be useful?
http://www.phpbb.com/community/viewtopi ... 6&t=531915

Phrase search would be extremely useful in addition to performance.