Skip to content

ElegantCloud/feiying

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

51 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Prequest

1) make sure git and python is installed.

2) in an Ununtu or Debian system, you can install them easily by aptitude.

3) you can visit the help page of github to learn how to setup ssh to access github

Setup

1) 'aptitude install python-virtualenv' and 'aptitude install python-dev'

2) when virtualenv is installed, run 'virtualenv scrapy' to create an new python env.

3) a new dir named 'scrapy' is created is step 2 is completed successfully.

4) run 'cd scrapy', then you can see three dirs there: bin, include and lib.

5) run 'source bin/activate' to setup the python env.  

6) use 'pip install scrapy' to install scrapy. depended packages such twisted and w3lib will be install automatically.
   # Sometimes you may have to install libxml2-dev and libxslt-dev by aptitude when encountering errors.

7) run 'scrapy startproject feiying' to create a new scrapy project.

8) 'cd feiying' and then 'git clone [email protected]:huuguanghui/feiying.git' 

9) run 'scrapy server' to schedule the crawlling task. For test, you can run 'scrapy crawl youku_video', it will crawl once. 


About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages