More than once, I had the need to do a massive download of files from a URL.

Quick wget command, options are pretty self explanatory:

wget -m --no-prent --reject "index.htm*" http://example.com/directory