pyspider

most powerful spider system in python!

Installation

Licensed under the Apache License, Version 2.0

Name	Name	Last commit message	Last commit date
Latest commit binux cronjob support more jobs and support seconds Jul 26, 2014 62e5ef6 · Jul 26, 2014 History 62 Commits
data	data	add test for scheduler	Mar 6, 2014
database	database	add mysql/projectdb	Jul 17, 2014
fetcher	fetcher	cronjob support more jobs and support seconds	Jul 26, 2014
libs	libs	cronjob support more jobs and support seconds	Jul 26, 2014
processor	processor	fix sqlite connection can't shared within thread (not process)	Jul 17, 2014
scheduler	scheduler	cronjob support more jobs and support seconds	Jul 26, 2014
test	test	cronjob support more jobs and support seconds	Jul 26, 2014
webui	webui	add /tasks	Jul 19, 2014
.gitignore	.gitignore	block scripts in iframe, use dataurl to prevent referer content block	Mar 16, 2014
.travis.yml	.travis.yml	add --allow-all-external	Jul 17, 2014
LICENSE	LICENSE	Initial commit	Feb 21, 2014
README.md	README.md	update readme and ...	Mar 8, 2014
logging.conf	logging.conf	add cdn option to change cdn service	Apr 3, 2014
requirements.txt	requirements.txt	add --allow-all-external	Jul 17, 2014
run.py	run.py	add database/mysql/taskdb	Jul 17, 2014
runtest.sh	runtest.sh	add projectdb	Feb 9, 2014
webui.py	webui.py	add task page	Jul 16, 2014