2016-05-0 0.101.0: Fixed German model

* Fixed bug that prevented German parses from being deprojectivised.
* Bug fixes to sentence boundary detection.
* Add rich comparison methods to the Lexeme class.
* Add missing Doc.has_vector and Span.has_vector properties.
* Add missing Span.sent property.

2016-04-05 v0.100.7: German!

----------------------------

spaCy finally supports another language, in addition to English. We're lucky to have Wolfgang Seeker on the team, and the new German model is just the beginning.
Now that there are multiple languages, you should consider loading spaCy via the load() function. This function also makes it easier to load extra word vector data for English:

.. code:: python

    import spacy
    en_nlp = spacy.load('en', vectors='en_glove_cc_300_1m_vectors')
    de_nlp = spacy.load('de')

To support use of the load function, there are also two new helper functions: spacy.get_lang_class and spacy.set_lang_class.
Once the German model is loaded, you can use it just like the English model:

.. code:: python

    doc = nlp(u'''Wikipedia ist ein Projekt zum Aufbau einer Enzyklopädie aus freien Inhalten, zu dem du mit deinem Wissen beitragen kannst. Seit Mai 2001 sind 1.936.257 Artikel in deutscher Sprache entstanden.''')

    for sent in doc.sents:
        print(sent.root.text, sent.root.n_lefts, sent.root.n_rights)

    # (u'ist', 1, 2)
    # (u'sind', 1, 3)

The German model provides tokenization, POS tagging, sentence boundary detection, syntactic dependency parsing, recognition of organisation, location and person entities, and word vector representations trained on a mix of open subtitles and Wikipedia data. It doesn't yet provide lemmatisation or morphological analysis, and it doesn't yet recognise numeric entities such as numbers and dates.

Bugfixes
--------

* spaCy < 0.100.7 had a bug in the semantics of the Token.__str__ and Token.__unicode__ built-ins: they included a trailing space.
* Improve handling of "infixed" hyphens. Previously the tokenizer struggled with multiple hyphens, such as "well-to-do".

* Improve handling of periods after mixed-case tokens

* Improve lemmatization for English special-case tokens

* Fix bug that allowed spaces to be treated as heads in the syntactic parse

* Fix bug that led to inconsistent sentence boundaries before and after serialisation.

* Fix bug from deserialising untagged documents.

Add support for GloVe vectors

This release offers improved support for replacing the word vectors used by spaCy.

To install Stanford's GloVe vectors, trained on the Common Crawl, just run

    sputnik --name spacy install en_glove_cc_300_1m_vectors

To reduce memory usage and loading time, we've trimmed the vocabulary down to 1m entries.

This release also integrates all the code necessary for German parsing. A German model will be
released shortly. To assist in multi-lingual processing, we've added a load() function. To load the English model with the GloVe vectors:

    spacy.load('en', vectors='en_glove_cc_300_1m_vectors')

Fix incorrect use of header file, caused from problem with thinc

Fix OSX problem introduced in 0.100.3

Small correction to right_edge calculation

Release 0.100.3

Support multi-threading, via the .pipe method. spaCy now releases the GIL around the
parser and entity recognizer, so systems that support OpenMP should be able to do
shared memory parallelism at close to full efficiency.

We've also greatly reduced loading time, and fixed a number of bugs.

v0.100.2

Fix data version lock that affected v0.100.1

Fix install for OSX

v0.100 included header files built on Linux that caused installation to fail on OSX.
This should now be corrected. We also update the default data distribution, to
include a small fix to the tokenizer.

Version 0.100: Revise setup.py, better model downloads, bug fixes

* Redo setup.py, and remove ugly headers_workaround hack. Should result in fewer install problems.
* Update data downloading and installation functionality, by migrating to the Sputnik data-package manager. This will allow us to offer finer grained control of data installation in future.
* Fix bug when using custom entity types in Matcher. This should work by default when using the
  English.__call__ method of running the pipeline. If invoking Parser.__call__ directly to do NER,
  you should call the Parser.add_label() method to register your entity type.
* Fix head-finding rules in Span.
* Fix problem that caused doc.merge() to sometimes hang
* Fix problems in handling of whitespace

Improve span merging, internal refactoring

* Merging multi-word tokens into one, via the doc.merge() and span.merge() methods, no longer invalidates existing Span objects. This makes it much easier to merge multiple spans, e.g. to merge all named entities, or all base noun phrases. Thanks to @andreasgrv for help on this patch.
* Lots of internal refactoring, especially around the machine learning module, thinc. The thinc API has now been improved, and the spacy._ml wrapper module is no longer necessary.
* The lemmatizer now lower-cases non-noun, noun-verb and non-adjective words.
* A new attribute, .rank, is added to Token and Lexeme objects, giving the frequency rank of the word.