nginx: I’d like to get some dynamic content rendered faster, so I’m playing with deploying it in front for static files, and using its url proxy feature to have it shoot the cgi/fastcgi/mod_whatever calls through to Apache.

lighttpd: I started to configure it for its fastcgi feature. However, many are the claims that it suffers significant memory leaks under heavy load. Of those making the claim but sticking with the product, the workaround was a cron job that reloaded the daemon at least nightly.

The file sizes were about the same, however I had only a couple thousand different files.

But as others have mentioned, you actually don’t even need TUX for that relatively small amount of hits/second.

But while you don’t NEED TUX it would still make your site *fly*, and leave a LOT of room for growth. Your only concern would be disk I/O (if the images don’t fit all into RAM, or each user requests wildly different images) and your server’s uplink to the internet (100mbps, 1gbps).

On Redhat/Centos, TUX is already there, and after

/etc/rc.d/init.d/tux start
/etc/rc.d/init.d/tux stop

you can start configuring it. You can be up and running within minutes. It runs very well along with Apache on the same machine, too. I.e. if you wanted to server dynamic pages from the same server. In this case TUX will sit in front at port 80 and pass every file with extensions specified by you to Apache in the background at port 8080. Restart of xinetd required to have Apache leave port 80 and let TUX go there. If it won’t work just reboot the whole server.

Only drawback is TUX doesn’t have any hotlink protection (no htaccess). But if that is a concern you can work around that by changing the path to the images with symlinks & cron jobs regularely. Have a symlink point to the real (internal) path, create a new random symlink every hour or day, and delete the old symlink after the new symlink has been active for some time (to prevent broken images for users).

Of course, if you push a lot of bandwidth, usually hotlinking won’t have any noticable effect anyway, so I personally just let away the whole part with path changing.

If you’re concerned about speed, and room for growth, look no further, TUX is second to none for static file serving and will put a smile on your face…

With S3 I’ve had some very bad experience. Bandwidth was very slow. Maybe they solved that now but is it really intended as a content distribution network? I think Limelight is better for content distribution (for example MySpace uses it for pictures). Can’t comment on Akamai.

I’m a fan of mathopd, though I confess that the last time I looked into the “fast static file webserver” space was circa 2003, and the game was pretty much between Apache 1.x, thttpd, mathopd, and maybe one other one I can’t recall. I read a lot of benchmarks other people had done (most of which showed poor understanding of statistical methods), set up my own non-scientific tests, and then read the code of parts of them, including thttpd and mathopd. I picked mathopd at the time because it seemed to win most for my use case and I could understand the code easily, something I couldn’t say for thttpd. Some thttpd security holes had me concerned as well. mathopd has not been free of them, but I sensed less opportunity for them to creep in given the way mathopd was written. None of this may matter much for your uses.

I haven’t compared mathopd to the newer servers others have mentioned here, including lighttpd and ngnix, but if I do, I’ll let you know what I find out.

http://cr.yp.to/publicfile.html

Also you should definitely go to more than 4GB RAM. Opteron systems seem to do better with lots of RAM, provided you are running a 64bit kernel (I use Solaris 10 myself, but Centos x64 should be fine also). Be sure that the RAM is installed properly with equal amounts of RAM in each CPU’s bank of RAM.