Q:

Efficient large file delivery over web

In the last few weeks we started serving more and more videos by http, ranging from 50 to 500mb each, and as we didn't had a new server to use it just for that i set it up on our live streaming server.
The box is pretty decent imho (2 x quad at 2ghz, 4gb ram, 4 x 300gb sas 15k rpm in hw RAID1, running Linux 2.6, ext3 fs with deadline scheduler, nginx as webserver) but we still had issues with delivering the files, mainly i suspect because our iowait was ranging from 10 to 45% and util was around 90% most of the times, while cpu was around 20-30%.
From what i've read that should've hold our ~300-400mbps traffic, but it doesn't.. 
Being on a budget we can't rely on a CDN but we can get more processing power if needed and the datacenter can handle our traffic.
Now we got a new server that should do only this but i don't know what would be the best way to setup it, main questions for now being:

what raid level would suit this task more? now have 4 x 300gb + 2 x 146gb, both SAS at 10k rpm, hw raid, would like more space but redundancy is also important
how would i expand it later, lvm a good idea?
how much ram would it need, for now it has 16gb but can add more if it would give us a decent boost, any tips with it?
decided on xfs for filesystem, anything against/better suited for this?
how much could i rely on only this 1 server?

Any other advices are appreciated.
Thank you.

A:

I am not a specialist with this sort of optimization, but here are a couple of thoughts that come to mind:

RAID: Serving up video, I would think you will primarily be performing sustained reads. For that purpose, I tend to think that RAID1 should be fine. A RAID 1+0 configuration might be better because there is more opportunity for parallelization - though I haven't researched the numbers on this. You may also be able to play around with the RAID settings; perhaps a larger stripe size would be better for your situation?
RAM: Why not start with what you have, and then monitor the usage? If you find that the server is doing too much swapping, then you can always add more RAM.

You seem to have a lot of questions, many of which might answer themselves once you throw the switch on the new server. I'd make sure that you can fall back to the old server if something catastrophic happens, and see how the new server performs.

