Allegro.cc - Online Community

Allegro.cc Forums » Off-Topic Ordeals » Downloading large amounts of files

Credits go to gnolam, jhuuskon, Matthew Leverton, MiquelFire, and ReyBrujo for helping out!
This thread is locked; no one can reply to it. rss feed Print
Downloading large amounts of files
Hard Rock
Member #1,547
September 2001
avatar

So what I want to do is basically download a large amount of files from a website. Now, I don't really want to click on every link, so I'm looking for a quick and easy way to do this.

I think wget is what I'm looking for, but now I'm not sure what language to write the script in to use it.

Basically I want to do this in a scripting language.

path = "website here";
for(int index = 0; index < 100; index++)
{
  url = path + index;
  filename = index +."zip";
  wget url filename
}

I mean I'm sure I could write this code in Java or C++ and have it to the above, but what I'm looking for is a tool that is meant for these kind of tasks as I don't think it makes sense writing separate applications each and every time I want to create something similar.

I think that's basically what I want to do, so what scripting language and or tool would make this the easiest? Anything I can write the above in say 3-5 lines of code in notepad and then just execute from the command line?

_________________________________________________
Hard Rock
[ Stars Dev Company ][ Twitter ][Global Warming: ARA My TINS 07 Entry][Pong Ultra Website][GifAllegS Ver 1.07]
"Well there's also coolwebsearch but we'll let that be an IE exclusive feature" - arielb on the New Browser Plugins "What's better, HTML or Variables?"

gnolam
Member #2,030
March 2002
avatar

Something like wget -r -np -A .zip www.foo.bar/frotz/ should do the trick. I think. I can't recall the exact syntax right now, but it's all in the manual.

--
Move to the Democratic People's Republic of Vivendi Universal (formerly known as Sweden) - officially democracy- and privacy-free since 2008-06-18!

jhuuskon
Member #302
April 2000
avatar

FOR /L %i IN (0,1,99) DO wget whatever options just %i where you want the number

edit4: fixx0red or maybe not.

You don't deserve my sig.

MiquelFire
Member #3,110
January 2003
avatar

If there's a high language script, you could use that, provided you can run it in a command in the case of PHP

Above two works as well. (jhuuskon's is Windows only though)

---
Febreze (and other air fresheners actually) is just below perfumes/colognes, and that's just below dead skunks in terms of smells that offend my nose.
MiquelFire.red
If anyone is of the opinion that there is no systemic racism in America, they're either blind, stupid, or racist too. ~Edgar Reynaldo

ReyBrujo
Moderator
January 2001
avatar

You can also create a list of files you want to download, and then pass it with -i to wget, if I recall correctly.

--
RB
光子「あたしただ…奪う側に回ろうと思っただけよ」
Mitsuko's last words, Battle Royale

Matthew Leverton
Supreme Loser
January 1999
avatar

Hard Rock
Member #1,547
September 2001
avatar

Awesome, thanks all!

_________________________________________________
Hard Rock
[ Stars Dev Company ][ Twitter ][Global Warming: ARA My TINS 07 Entry][Pong Ultra Website][GifAllegS Ver 1.07]
"Well there's also coolwebsearch but we'll let that be an IE exclusive feature" - arielb on the New Browser Plugins "What's better, HTML or Variables?"

Go to: