Commits

Branch selector

 master

User selector

All users

Datepicker

All time

Commit History

Commits on Jul 16, 2014

• Returned default settings to use old http/https download handlers.

  Mimino666

  committed
  e9e0507

Commits on Jul 15, 2014

• Added more ignored extenstions to link extractors.

  Mimino666

  committed
  2862e8a

• Removed 'x-gzip' from Accept-Encoding.

  Mimino666

  committed
  083a21d

• Fixed gz.gunzip() broken after changes from Python 2.7.3 to 2.7.4.

  Mimino666

  committed
  755bc35

• Implemented http_1_1 handler.

  Mimino666

  committed
  ea19948

• Removed OpenSSL from optional features - it is now required.

  Show description for 1c3700e
  Mimino666

  committed
  1c3700e

• Fixes for Console class.

  Mimino666

  committed
  48376c4

• Fixed case-sensitivnes test for is_url_from_any_domain method.

  Mimino666

  committed
  12123bd

• Added few monkeypathces to fix external libraries.

  Mimino666

  committed
  2452fda

• Removed bad processing of request 'POST' method.

  Mimino666

  committed
  7c27841

• Added MANIFEST file.

  Show description for c00549d
  Mimino666

  committed
  c00549d

Commits on Jun 23, 2014

• Added POST method processing.

  Mimino666

  committed
  7e35b43

Commits on May 18, 2014

• Added <style> regex.

  Mimino666

  committed
  0c28174

Commits on Apr 1, 2014

• Log list of disabled middlewares in DEBUG level.

  Mimino666

  committed
  f1d66b5

Commits on Feb 4, 2014

• Added fix of relative paths in url in canonicalize_url method.

  Mimino666

  committed
  633c27c

Commits on Jan 29, 2014

• Crawlmi supports cookies. Yay!

  Mimino666

  committed
  def7dba

• Fixed http headers class.

  Show description for fa54669
  Mimino666

  committed
  fa54669

Commits on Nov 23, 2013

• Optimized S classes with precompiled xpath expressions.

  Mimino666

  committed
  cc72c31

• Deleted xpath_exists() method from S class.

  Mimino666

  committed
  1d18b13

Commits on Oct 31, 2013

• Added correction of relative paths to link extractor - it is needed way

  Show description for 323e6fb
  Mimino666

  committed
  323e6fb

Commits on Oct 21, 2013

• Created Canonical pipeline mw.

  Mimino666

  committed
  1cae25f

• Little optimization in response's base_url - use raw body instead of text.

  Mimino666

  committed
  b285032

Commits on Oct 20, 2013

• Fixed DOWNLOAD_DELAY settings - use float instead of integer value for it.

  Mimino666

  committed
  0714d4a

Commits on Sep 21, 2013

• Added filter option to S class.

  Mimino666

  committed
  6288f29

Commits on Sep 17, 2013

• Added first() filter method to SP class.

  Mimino666

  committed
  f853eac

Commits on Sep 16, 2013

• Added convenience parse() arguments to SP class to parse directly out of

  Show description for 1f4d220
  Mimino666

  committed
  1f4d220

Commits on Sep 15, 2013

• Added option to use css selectors in S class.

  Mimino666

  committed
  1db4138

Commits on Sep 14, 2013

• Initialize namespaces for XML selector in XmlResponse.

  Mimino666

  committed
  9b40f6d

• Added absolute_url callback method to S class - very useful.

  Mimino666

  committed
  7fdd4b6

Commits on Jul 2, 2013

• Added strip_www option to canonicalize_url().

  Mimino666

  committed
  7413359

Commits on Jun 20, 2013

• Added tags and signal to clear the storage to DuplicateFilter mw.

  Mimino666

  committed
  7c9108c

• Improved settings command to show PIPELINE and EXTENSION dicts merge

  Show description for 86d6117
  Mimino666

  committed
  86d6117

• Fixed usage of rstrip() function - it doesn't cut the argument from

  Show description for aa4605d
  Mimino666

  committed
  aa4605d

Commits on Jun 17, 2013

• Fixed SitemapSpider's "calm zone", when waiting between the batches of

  Show description for 9f0038e
  Mimino666

  committed
  9f0038e

Commits on Jun 11, 2013

• Update README.md

  Mimino666

  committed
  2b27c72

Pagination

Previous

Next