StarDestroyer.Net BBS

I was thinking of building a NAS, complete with RAID 5 and a number of 500+ gb drives giving me somewhere on the order of 1.5 tb of reliable storage. At the same time, I was thinking that perhaps, just maybe, some media PC functionality could be incorporated. After all, I have a lot of DVDs I'd like to see stored digitally instead of optically. The idea of watching movies without ever inserting a disc is very appealing.

I've spec'd out a couple things, including this ready to go NAS vs Do it Yourself: this RAID card and associated quiet-box from Antec.

What I'm curious to know is whether I'm on the right track at all; is it worthwhile to build my own NAS box and then bundle in some sort of media output functionality, or just go right ahead and buy a ready-to-go NAS and forget about digitizing all those DVD's? I'm trying to keep the cost under $1000, but I could go as high as 1500.

Thoughts?

A NAS makes more sense if you have a home LAN that has more than one machine needing access to the same data. If you only have one PC hooked up it's not really worth the hassle or the cost.

There is also another advantage to the NAS being that you can bring SUBSTANTIAL storage to another site (I presume for serious matters) without uprooting the whole machine. For casual purposes any USB driven portable disk would suffice.

Do you really need RAID? It's not really a backup solution, and your performance bottleneck is probably your ethernet speed, unless you have gigabit. Plus Linux softraid is probably good enough even if you do need it (I'm assuming DIY)- I'd DIY it, since a fileserver can do a lot of other things than a dedicated NAS- more maintenance though (I'm biased since I already run a fileserver).

Pu-239 wrote:Do you really need RAID? It's not really a backup solution, and your performance bottleneck is probably your ethernet speed, unless you have gigabit. Plus Linux softraid is probably good enough even if you do need it (I'm assuming DIY)- I'd DIY it, since a fileserver can do a lot of other things than a dedicated NAS- more maintenance though (I'm biased since I already run a fileserver).

But a backup solution for 1.5TB is either going to suck a lot of ass (god damn that is a lot of DVD burning you'd have to do!) or be prohibitively expensive - i.e. tape drives. RAID-5 is a nice middle ground between real backups and no backups - if a hard drive dies, it doesn't mean you automatically lose the data that was on it, but you're still really vulnerable if you get unlucky and have a second hard drive die while you're still rebuilding the array.

(Plus, technically, it's not like he'd actually be losing any data if the drive died; he'd just have to sit through and re-rip his DVDs, which would be only slightly more tedious as sitting down and pulling ripped DVD images from DVD-R backups. In this instance, I think it's very safe to assume that RAID is being explored as a convenience.)

I agree on going with the fileserver and the softRAID; for the most part, your server isn't going to be doing much else, so who cares if it hits the CPU for the RAID calculations? Plus, if your OS gets hosed somehow, you can easily transplant the RAID array to another OS install pretty easily, whereas if a hardware RAID controller shits its pants (and miraculously doesn't trash all the array in the process) you'll almost certainly have to replace it with a controller of the very same model. Better hope they haven't stopped making them if that happens!

As for the file server, seriously, for the price of that Infrant box (which doesn't include drives), you could have the server and a couple of hard drives. Plus, you could plug a projector into the server and have it play movies, or plug speakers into it and play music, or run a webserver or FTP off it if you like, or run a game server, or whatever. It gives you hell of more options than something which just sits on the network and serves files and nothing else.

Windows Server 2003 (and, no, I'm not advocating breaking copyright law; I've heard of some schools which practically give it away) also comes with RAID-5 support (which seems to work pretty decently; I use it myself at home) and I've also heard that Windows XP has RAID-5 support, but that you have to tweak the registry in order for it to show up.

Uraniun235 wrote:

Pu-239 wrote:Do you really need RAID? It's not really a backup solution, and your performance bottleneck is probably your ethernet speed, unless you have gigabit. Plus Linux softraid is probably good enough even if you do need it (I'm assuming DIY)- I'd DIY it, since a fileserver can do a lot of other things than a dedicated NAS- more maintenance though (I'm biased since I already run a fileserver).

But a backup solution for 1.5TB is either going to suck a lot of ass (god damn that is a lot of DVD burning you'd have to do!) or be prohibitively expensive - i.e. tape drives. RAID-5 is a nice middle ground between real backups and no backups - if a hard drive dies, it doesn't mean you automatically lose the data that was on it, but you're still really vulnerable if you get unlucky and have a second hard drive die while you're still rebuilding the array.

(Plus, technically, it's not like he'd actually be losing any data if the drive died; he'd just have to sit through and re-rip his DVDs, which would be only slightly more tedious as sitting down and pulling ripped DVD images from DVD-R backups. In this instance, I think it's very safe to assume that RAID is being explored as a convenience.)

I agree on going with the fileserver and the softRAID; for the most part, your server isn't going to be doing much else, so who cares if it hits the CPU for the RAID calculations? Plus, if your OS gets hosed somehow, you can easily transplant the RAID array to another OS install pretty easily, whereas if a hardware RAID controller shits its pants (and miraculously doesn't trash all the array in the process) you'll almost certainly have to replace it with a controller of the very same model. Better hope they haven't stopped making them if that happens!

As for the file server, seriously, for the price of that Infrant box (which doesn't include drives), you could have the server and a couple of hard drives. Plus, you could plug a projector into the server and have it play movies, or plug speakers into it and play music, or run a webserver or FTP off it if you like, or run a game server, or whatever. It gives you hell of more options than something which just sits on the network and serves files and nothing else.

Windows Server 2003 (and, no, I'm not advocating breaking copyright law; I've heard of some schools which practically give it away) also comes with RAID-5 support (which seems to work pretty decently; I use it myself at home) and I've also heard that Windows XP has RAID-5 support, but that you have to tweak the registry in order for it to show up.

Well, I was thinking external drives, or just scheduling rsync or some other syncing program every night or at least- that said, if it's just DVD images, who really cares if the HDDs go tits up....? Also, afaik, Windows doesn't support softraid at installation, while I believe ubuntu,debian, et al do (not really a big deal if you seperate your OS/data partition though, run whichever OS you might need to run your other things).

Uraniun235 wrote:

Pu-239 wrote:Do you really need RAID? It's not really a backup solution, and your performance bottleneck is probably your ethernet speed, unless you have gigabit. Plus Linux softraid is probably good enough even if you do need it (I'm assuming DIY)- I'd DIY it, since a fileserver can do a lot of other things than a dedicated NAS- more maintenance though (I'm biased since I already run a fileserver).

But a backup solution for 1.5TB is either going to suck a lot of ass (god damn that is a lot of DVD burning you'd have to do!) or be prohibitively expensive - i.e. tape drives. RAID-5 is a nice middle ground between real backups and no backups - if a hard drive dies, it doesn't mean you automatically lose the data that was on it, but you're still really vulnerable if you get unlucky and have a second hard drive die while you're still rebuilding the array.

(Plus, technically, it's not like he'd actually be losing any data if the drive died; he'd just have to sit through and re-rip his DVDs, which would be only slightly more tedious as sitting down and pulling ripped DVD images from DVD-R backups. In this instance, I think it's very safe to assume that RAID is being explored as a convenience.)

I agree on going with the fileserver and the softRAID; for the most part, your server isn't going to be doing much else, so who cares if it hits the CPU for the RAID calculations? Plus, if your OS gets hosed somehow, you can easily transplant the RAID array to another OS install pretty easily, whereas if a hardware RAID controller shits its pants (and miraculously doesn't trash all the array in the process) you'll almost certainly have to replace it with a controller of the very same model. Better hope they haven't stopped making them if that happens!

As for the file server, seriously, for the price of that Infrant box (which doesn't include drives), you could have the server and a couple of hard drives. Plus, you could plug a projector into the server and have it play movies, or plug speakers into it and play music, or run a webserver or FTP off it if you like, or run a game server, or whatever. It gives you hell of more options than something which just sits on the network and serves files and nothing else.

Windows Server 2003 (and, no, I'm not advocating breaking copyright law; I've heard of some schools which practically give it away) also comes with RAID-5 support (which seems to work pretty decently; I use it myself at home) and I've also heard that Windows XP has RAID-5 support, but that you have to tweak the registry in order for it to show up.

I dunno, he said he has around $1000 to spend on this project, so external hdds might be the way to go. I was able to pick up a 500gb Western Digital External drive for around $200 a month or so ago, and a handful of those would easily meet the 1.5tb requirement. Toss in a usb port hub and voila! Storage problem solved for neatly under a grand.

Could wait until Windows Home Server comes out. It'll support continuous backups, along with Single Instance Storage, and a interesting multi disk expansion system (see the Wikipedia article for more details).

I'd love to do something like that, but I don't have the slightest clue how I'd partition it!


If it were FAT32, it'd have to be broken up into dozens of 32 GB partitions, and couldn't have individual files over 4 GB.

If it were NTFS, Mac OS X and Linux can't write to it, only read.

If it were HFS+, Windows can't read or write to it.

Pu-239 wrote: Well, I was thinking external drives, or just scheduling rsync or some other syncing program every night or at least- that said, if it's just DVD images, who really cares if the HDDs go tits up....? Also, afaik, Windows doesn't support softraid at installation, while I believe ubuntu,debian, et al do (not really a big deal if you seperate your OS/data partition though, run whichever OS you might need to run your other things).

Windows doesn't support softRAID, but it's not that difficult to find some spare small hard drive lying around somewhere for use as an OS.

And, I don't know about you, but having to re-rip 500 gigs worth of DVD images sounds pretty godawful to me.

Praxis wrote:I'd love to do something like that, but I don't have the slightest clue how I'd partition it!


If it were FAT32, it'd have to be broken up into dozens of 32 GB partitions, and couldn't have individual files over 4 GB.

If it were NTFS, Mac OS X and Linux can't write to it, only read.

If it were HFS+, Windows can't read or write to it.

Well, if you set it up as a file server with network shares, the other computers accessing the server won't (or at least shouldn't) have to deal with that. Now, this is certainly a problem for an external hard drive you'd like to tote around.

(Also, you can make FAT32 partitions bigger than 32GB, with the right tools.)

No reason to use fat32 anymore, Linux reads/writes NTFS. If you're mac... oh well.... (although someone did port FUSE to MacOS, so maybe the ntfs3g-fuse driver works there too)

EDIT: Indeed: http://shadowofged.blogspot.com/2007/03 ... -os-x.html

Performance is worse on Mac than Linux (where it's already pretty worse than Windows).

EDIT: Actually, not really that bad for Linux: http://www.ntfs-3g.org/performance.html

Another option would be to run whichever OS that you need to read your partition in VMWare or whatever, eg format a partition as Ext3, run Linux in VMware on Windows and share the ext3 using Samba.

EDIT: Ooops, you're still talking about a fileserver- yes, it's a nonissue there, unless you multiboot it which was what I was thinking (why the fuck would anyone want to?).

Beowulf wrote:Could wait until Windows Home Server comes out. It'll support continuous backups, along with Single Instance Storage, and a interesting multi disk expansion system (see the Wikipedia article for more details).

Doesn't Windows already have dynamic disks? (albeit w/o redundancy, or maybe if you use RAID 1/5? ). And Linux can already do LVM over RAID (although it appears if you want to expand, you'd have to do it in increments of 3disk raid5 arrays). There's also ZFS and running Solaris which I hear is quite flexible (although quite frankly, I've been running Solaris in VMWare, and it's not really that user friendly- plus it's bloated and extremely slow- I shudder to think about running it on my PIII-733 server- the filesystem layout is incredibly messy having all this legacy stuff - there are various distros of GNU/Solaris but I wouldn't want to run a server on those).

I'm really against RAID for "backup" purposes- better would be to simply synchronize the drives every night w/ rsync (or a windows equivalent or port- use cygwin?), giving a window of time to save things in case of filesystem corruption (well, in that case, syncing of corrupted filesystem would just not work, so you'd be safe), or accidental deletion. Of course, it'd take two disks, and you'd be only able to use 1/2 of your storage capacity instead of 2/3rds w/ RAID5 - then again, that's only 1/6th of the disk space that you'll keep using RAID5

Something to keep in mind is using RAID1 or RAID10, which performs better (at the cost of 1/6 of usable disk space). That said, again, your bottlenecks are probably elsewhere (esp for just DVD images).

Pu-239 wrote:I'm really against RAID for "backup" purposes- better would be to simply synchronize the drives every night w/ rsync (or a windows equivalent or port- use cygwin?), giving a window of time to save things in case of filesystem corruption (well, in that case, syncing of corrupted filesystem would just not work, so you'd be safe), or accidental deletion. Of course, it'd take two disks, and you'd be only able to use 1/2 of your storage capacity instead of 2/3rds w/ RAID5 - then again, that's only 1/6th of the disk space that you'll keep using RAID5

Something to keep in mind is using RAID1 or RAID10, which performs better (at the cost of 1/6 of usable disk space). That said, again, your bottlenecks are probably elsewhere (esp for just DVD images).

Agreed RAIS sucks for backups, funny story.

Our art department needed a shitload of spaces for photos and stuff, and I told em okay no problem but there will be no backup on that? okay?

they told me sure no problem just put up a RAID0 and we are cool.

See where I am going yet?

Anyways a year or so later one of those disk when sour and took down the set a they where in a uproar. Told em hey I told you no backup no security that was a temporary place before it goes on the web.

Cry moan bitch and so on, and we had raid... like RAID0 anyways they told me that it was a matter of life and death and stuff so I said fine gimme a work order and I will fix it, but it will cost you.

5minutes later I had the work order, sent the disk to IBAS for emergency fix, hey it said emergency on the order...

two days later a 1tb usb disk arrived with all their shit on and I connected it to the network. Also enabled access logging, NO access in one week! even tho I mailed multiple times.

After the bill arived I got to have a chat with the CEO and I shoved him all the mail, the order and the logs.

The bitch in charge got fired (she tried to blame me for the unessesary cost of 100.000kr)

And I got a nice shiny 1tb usb disk on my desk.

That's really not at all relevant to the discussion at hand; RAID-0 shouldn't even really count as a RAID level, since it has absolutely no redundancy at all.

Pu-239 wrote:I'm really against RAID for "backup" purposes- better would be to simply synchronize the drives every night w/ rsync (or a windows equivalent or port- use cygwin?), giving a window of time to save things in case of filesystem corruption (well, in that case, syncing of corrupted filesystem would just not work, so you'd be safe), or accidental deletion. Of course, it'd take two disks, and you'd be only able to use 1/2 of your storage capacity instead of 2/3rds w/ RAID5 - then again, that's only 1/6th of the disk space that you'll keep using RAID5

It's not being proposed as a backup solution; more of a middle-ground (or halfass, if you'd rather see it that way) between absolutely no redundancy whatsoever and a proper backup solution. I'm pretty sure this isn't being floated as a "oh hey I need to make sure this data is preserved" measure but as a "you know it'd be real nice if I didn't have to re-rip a hundred DVDs because one of the hard drives failed" without having to double the cost of storage in order to implement a backup solution.

I mean, come on, if this were a business situation I would be behind you 110%, but this is a home application; if there's any place to cheap out on storage implementation, this would be it.