A Image Crawler I wrote Added GUI (graphical user interface) & packaged into a single .exe file
in 2014, I’ ve written a Python image crawler for a specific website Cure WorldCosplay, a website that attracts cosplayers all over the world post their own pictures . Which has about 10k active members and up to 10 million pictures posted.
The pros is the program is packaged into a single executable file, no programming environment needed. But some virus detection software could report unsafe file.
Here the program is! |
Click names below for download |
With interface |
Without interface |
Theoretically, if you have enough disk space, you can download all the pics of that website (about 9800 Gigabyte), the only limit is your bandwidth. I have deployed 36 crawlers on a Linux server at the same time, they download pics 24/7 at the maximum internet bandwidth.
How to use it: