forked from apache/tika
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathCHANGES.txt
1061 lines (805 loc) · 35.9 KB
/
CHANGES.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Apache Tika Change Log
======================
Release 1.1 - Current Development
---------------------------------
* Link Extraction: The rel attribute is now extracted from
links per the LinkConteHandler. (TIKA-824)
* MP3: Fixed handling of UTF-16 (two byte) ID3v2 tags (previously
the last character in a UTF-16 tag could be corrupted) (TIKA-793)
* Performance: Loading of the default media type registry is now
significantly faster. (TIKA-780)
* PDF: Allow controlling whether overlapping duplicated text should
be removed. Disabling this (the default) can give big
speedups to text extraction and may workaround cases where
non-duplicated characters were incorrectly removed (TIKA-767).
Allow controlling whether text tokens should be sorted by their x/y
position before extracting text (TIKA-612); this is necessary for
certain PDFs. Fixed cases where too many </p> tags appear in the
XHTML output, causing NPE when opening some PDFs with the GUI
(TIKA-778).
* RTF: Fixed case where a font change would result in processing
bytes in the wrong font's charset, producing bogus text output
(TIKA-777). Don't output whitespace in ignored group states,
avoiding excessive whitespace output (TIKA-781). Binary embedded
content (using \bin control word) is now skipped correctly;
previously it could cause the parser to incorrectly extract binary
content as text (TIKA-782).
* CLI: New TikaCLI option "--list-detectors", which displays the
mimetype detectors that are available, similar to the existing
"--list-parsers" option for parsers. (TIKA-785).
* Detectors: The order of detectors, as supplied via the service
registry loader, is now controlled. User supplied detectors are
prefered, then Tika detectors (such as the container aware ones),
and finally the core Tika MimeTypes is used as a backup. This
allows for specific, detailed detectors to take preference over
the default mime magic + filename detector. (TIKA-786)
* Microsoft Project (MPP): Filetype detection has been fixed,
and basic metadata (but no text) is now extracted. (TIKA-789)
* Outlook: fixed NullPointerException in TikaGUI when messages with
embedded RTF or HTML content were filtered (TIKA-801).
Release 1.0 - 11/4/2011
---------------------------------
The most notable changes in Tika 1.0 over previous releases are:
* API: All methods, classes and interfaces that were marked as
deprecated in Tika 0.10 have been removed to clean up the API
(TIKA-703). You may need to adjust and recompile client code
accordingly. The declared OSGi package versions are now 1.0, and
will thus not resolve for client bundles that still refer to 0.x
versions (TIKA-565).
* Configuration: The context class loader of the current thread is
no longer used as the default for loading configured parser and
detector classes. You can still pass an explicit class loader
to the configuration mechanism to get the previous behaviour.
(TIKA-565)
* OSGi: The tika-core bundle will now automatically pick up and use
any available Parser and Detector services when deployed to an OSGi
environment. The tika-parsers bundle provides such services based on
for all the supported file formats for which the upstream parser library
is available. If you don't want to track all the parser libraries as
separate OSGi bundles, you can use the tika-bundle bundle that packages
tika-parsers together with all its upstream dependencies. (TIKA-565)
* RTF: Hyperlinks in RTF documents are now extracted as an <a
href=...>...</a> element (TIKA-632). The RTF parser is also now
more robust when encountering too many closing {'s vs. opening {'s
(TIKA-733).
* MS Word: From Word (.doc) documents we now extract optional hyphen
as Unicode zero-width space (U+200B), and non-breaking hyphen as
Unicode non-breaking hyphen (U+2011). (TIKA-711)
* Outlook: Tika can now process also attachments in Outlook messages.
(TIKA-396)
* MS Office: Performance of extracting embedded office docs was improved.
(TIKA-753)
* PDF: The PDF parser now extracts paragraphs within each page
(TIKA-742) and can now optionally extract text from PDF
annotations (TIKA-738). There's also an option to enable (the
default) or disable auto-space insertion (TIKA-724).
* Language detection: Tika can now detect Belarusian, Catalan,
Esperanto, Galician, Lithuanian (TIKA-582), Romanian, Slovak,
Slovenian, and Ukrainian (TIKA-681).
* Java: Tika no longer ships retrotranslated Java 1.4 binaries along
with the normal ones that work with Java 5 and higher. (TIKA-744)
* OpenOffice documents: header/footer text is now extracted for text,
presentation and spreadsheet documents (TIKA-736)
Tika 1.0 relies on the following set of major dependencies (generated using
mvn dependency:tree from tika-parsers):
org.apache.tika:tika-parsers:bundle:1.0
+- org.apache.tika:tika-core:jar:1.0:compile
+- edu.ucar:netcdf:jar:4.2-min:compile
| \- org.slf4j:slf4j-api:jar:1.5.6:compile
+- org.apache.james:apache-mime4j-core:jar:0.7:compile
+- org.apache.james:apache-mime4j-dom:jar:0.7:compile
+- org.apache.commons:commons-compress:jar:1.3:compile
+- commons-codec:commons-codec:jar:1.5:compile
+- org.apache.pdfbox:pdfbox:jar:1.6.0:compile
| +- org.apache.pdfbox:fontbox:jar:1.6.0:compile
| +- org.apache.pdfbox:jempbox:jar:1.6.0:compile
| \- commons-logging:commons-logging:jar:1.1.1:compile
+- org.bouncycastle:bcmail-jdk15:jar:1.45:compile
+- org.bouncycastle:bcprov-jdk15:jar:1.45:compile
+- org.apache.poi:poi:jar:3.8-beta4:compile
+- org.apache.poi:poi-scratchpad:jar:3.8-beta4:compile
+- org.apache.poi:poi-ooxml:jar:3.8-beta4:compile
| +- org.apache.poi:poi-ooxml-schemas:jar:3.8-beta4:compile
| | \- org.apache.xmlbeans:xmlbeans:jar:2.3.0:compile
| \- dom4j:dom4j:jar:1.6.1:compile
+- org.apache.geronimo.specs:geronimo-stax-api_1.0_spec:jar:1.0.1:compile
+- org.ccil.cowan.tagsoup:tagsoup:jar:1.2.1:compile
+- asm:asm:jar:3.1:compile
+- com.drewnoakes:metadata-extractor:jar:2.4.0-beta-1:compile
+- de.l3s.boilerpipe:boilerpipe:jar:1.1.0:compile
+- rome:rome:jar:0.9:compile
\- jdom:jdom:jar:1.0:compile
The following people have contributed to Tika 1.0 by submitting or commenting
on the issues resolved in this release:
Andrzej Bialecki
Antoni Mylka
Benson Margulies
Chris A. Mattmann
Cristian Vat
Dave Meikle
David Smiley
Dennis Adler
Erik Hetzner
Ingo Renner
Jeremias Maerki
Jeremy Anderson
Jeroen van Vianen
John Bartak
Jukka Zitting
Julien Nioche
Ken Krugler
Mark Butler
Maxim Valyanskiy
Michael Bryant
Michael McCandless
Nick Burch
Pablo Queixalos
Uwe Schindler
Žygimantas Medelis
See http://s.apache.org/Zk6 for more details on these contributions.
Release 0.10 - 09/25/2011
-------------------------
The most notable changes in Tika 0.10 over previous releases are:
* A parser for CHM help files was added. (TIKA-245)
* TIKA-698: Invalid characters are now replaced with the Unicode
replacement character (U+FFFD), whereas before such characters were
replaced with spaces, so you may need to change your processing of
Tika's output to now handle U+FFFD.
* The RTF parser was rewritten to perform its own direct shallow
parse of the RTF content, instead of using RTFEditorKit from
javax.swing. This fixes several issues in the old parser,
including doubling of Unicode characters in certain cases
(TIKA-683), exceptions on mal-formed RTF docs (TIKA-666), and
missing text from some elements (header/footer, hyperlinks,
footnotes, text inside pictures).
* Handling of temporary files within Tika was much improved
(TIKA-701, TIKA-654, TIKA-645, TIKA-153)
* The Tika GUI got a facelift and some extra features (TIKA-635)
* The apache-mime4j dependency of the email message parser was upgraded
from version 0.6 to 0.7 (TIKA-716). The parser also now accepts a
MimeConfig object in the ParseContext as configuration (TIKA-640).
Tika 0.10 relies on the following set of major dependencies (generated using
mvn dependency:tree from tika-parsers):
org.apache.tika:tika-parsers:bundle:0.10
+- org.apache.tika:tika-core:jar:0.10:compile
+- edu.ucar:netcdf:jar:4.2-min:compile
| \- org.slf4j:slf4j-api:jar:1.5.6:compile
+- org.apache.james:apache-mime4j-core:jar:0.7:compile
+- org.apache.james:apache-mime4j-dom:jar:0.7:compile
+- org.apache.commons:commons-compress:jar:1.1:compile
+- commons-codec:commons-codec:jar:1.4:compile
+- org.apache.pdfbox:pdfbox:jar:1.6.0:compile
| +- org.apache.pdfbox:fontbox:jar:1.6.0:compile
| +- org.apache.pdfbox:jempbox:jar:1.6.0:compile
| \- commons-logging:commons-logging:jar:1.1.1:compile
+- org.bouncycastle:bcmail-jdk15:jar:1.45:compile
+- org.bouncycastle:bcprov-jdk15:jar:1.45:compile
+- org.apache.poi:poi:jar:3.8-beta4:compile
+- org.apache.poi:poi-scratchpad:jar:3.8-beta4:compile
+- org.apache.poi:poi-ooxml:jar:3.8-beta4:compile
| +- org.apache.poi:poi-ooxml-schemas:jar:3.8-beta4:compile
| | \- org.apache.xmlbeans:xmlbeans:jar:2.3.0:compile
| \- dom4j:dom4j:jar:1.6.1:compile
+- org.apache.geronimo.specs:geronimo-stax-api_1.0_spec:jar:1.0.1:compile
+- org.ccil.cowan.tagsoup:tagsoup:jar:1.2.1:compile
+- asm:asm:jar:3.1:compile
+- com.drewnoakes:metadata-extractor:jar:2.4.0-beta-1:compile
+- de.l3s.boilerpipe:boilerpipe:jar:1.1.0:compile
+- rome:rome:jar:0.9:compile
\- jdom:jdom:jar:1.0:compile
The following people have contributed to Tika 0.10 by submitting or commenting
on the issues resolved in this release:
Alain Viret
Alex Ott
Alexander Chow
Andreas Kemkes
Andrew Khoury
Babak Farhang
Benjamin Douglas
Benson Margulies
Chris A. Mattmann
chris hudson
Chris Lott
Cristian Vat
Curt Arnold
Cynthia L Wong
Dave Brosius
David Benson
Enrico Donelli
Erik Hetzner
Erna de Groot
Gabriele Columbro
Gavin
Geoff Jarrad
Gregory Kanevsky
gunter rombauts
Henning Gross
Henri Bergius
Ingo Renner
Ingo Wiarda
Izaak Alpert
Jan H√∏ydahl
Jens Wilmer
Jeremy Anderson
Joseph Vychtrle
Joshua Turner
Jukka Zitting
Julien Nioche
Karl Heinz Marbaise
Ken Krugler
Kostya Gribov
Luciano Leggieri
Mads Hansen
Mark Butler
Matt Sheppard
Maxim Valyanskiy
Michael McCandless
Michael Pisula
Murad Shahid
Nick Burch
Oleg Tikhonov
Pablo Queixalos
Paul Jakubik
Raimund Merkert
Rajiv Kumar
Robert Trickey
Sami Siren
samraj
Selva Ganesan
Sjoerd Smeets
Stephen Duncan Jr
Tran Nam Quang
Uwe Schindler
Vitaliy Filippov
See http://s.apache.org/vR for more details on these contributions.
Release 0.9 - 02/13/2011
------------------------
The most notable changes in Tika 0.9 over previous releases are:
* A critical bugfix preventing metadata from printing to the
command line when the underlying Parser didn't generate
XHTML output was fixed. (TIKA-596)
* The 0.8 version of Tika included a NetCDF jar file that pulled
in tremendous amounts of redundant dependencies. This has
been addressed in Tika 0.9 by republishing a minimal NetCDF
jar and changing Tika to depend on that. (TIKA-556)
* MIME detection for iWork, and OpenXML documents has been
improved. (TIKA-533, TIKA-562, TIKA-588)
* A critical backwards incompatible bug in PDF parsing that
was introduced in Tika 0.8 has been fixed. (TIKA-548)
* Support for forked parsing in separate processes was added.
(TIKA-416)
* Tika's language identifier now supports the Lithuanian
language. (TIKA-582)
Tika 0.9 relies on the following set of major dependencies (generated using
mvn dependency:tree from tika-parsers):
org.apache.tika:tika-parsers:bundle:0.9
+- org.apache.tika:tika-core:jar:0.9:compile
+- edu.ucar:netcdf:jar:4.2-min:compile
| \- org.slf4j:slf4j-api:jar:1.5.6:compile
+- commons-httpclient:commons-httpclient:jar:3.1:compile
| +- commons-logging:commons-logging:jar:1.1.1:compile (version managed from 1.0.4)
| \- commons-codec:commons-codec:jar:1.2:compile
+- org.apache.james:apache-mime4j:jar:0.6:compile
+- org.apache.commons:commons-compress:jar:1.1:compile
+- org.apache.pdfbox:pdfbox:jar:1.4.0:compile
| +- org.apache.pdfbox:fontbox:jar:1.4.0:compile
| \- org.apache.pdfbox:jempbox:jar:1.4.0:compile
+- org.bouncycastle:bcmail-jdk15:jar:1.45:compile
+- org.bouncycastle:bcprov-jdk15:jar:1.45:compile
+- org.apache.poi:poi:jar:3.7:compile
+- org.apache.poi:poi-scratchpad:jar:3.7:compile
+- org.apache.poi:poi-ooxml:jar:3.7:compile
| +- org.apache.poi:poi-ooxml-schemas:jar:3.7:compile
| | \- org.apache.xmlbeans:xmlbeans:jar:2.3.0:compile
| \- dom4j:dom4j:jar:1.6.1:compile
+- org.apache.geronimo.specs:geronimo-stax-api_1.0_spec:jar:1.0.1:compile
+- org.ccil.cowan.tagsoup:tagsoup:jar:1.2:compile
+- asm:asm:jar:3.1:compile
+- com.drewnoakes:metadata-extractor:jar:2.4.0-beta-1:compile
+- de.l3s.boilerpipe:boilerpipe:jar:1.1.0:compile
+- rome:rome:jar:0.9:compile
\- jdom:jdom:jar:1.0:compile
The following people have contributed to Tika 0.9 by submitting or commenting
on the issues resolved in this release:
Alex Skochin
Alexander Chow
Antoine L.
Antoni Mylka
Benjamin Douglas
Benson Margulies
Chris A. Mattmann
Cristian Vat
Cyriel Vringer
David Benson
Erik Hetzner
Gabriel Miklos
Geoff Jarrad
Jukka Zitting
Ken Krugler
Kostya Gribov
Leszek Piotrowicz
Martijn van Groningen
Maxim Valyanskiy
Michel Tremblay
Nick Burch
paul
Paul Pearcy
Peter van Raamsdonk
Piotr Bartosiewicz
Reinhard Schwab
Scott Severtson
Shinsuke Sugaya
Staffan Olsson
Steve Kearns
Tom Klonikowski
≈Ωygimantas Medelis
See http://s.apache.org/qi for more details on these contributions.
Release 0.8 - 11/07/2010
------------------------
The most notable changes in Tika 0.8 over previous releases are:
* Language identification is now dynamically configurable,
managed via a config file loaded from the classpath. (TIKA-490)
* Tika now supports parsing Feeds by wrapping the underlying
Rome library. (TIKA-466)
* A quick-start guide for Tika parsing was contributed. (TIKA-464)
* An approach for plumbing through XHTML attributes was added. (TIKA-379)
* Media type hierarchy information is now taken into account when
selecting the best parser for a given input document. (TIKA-298)
* Support for parsing common scientific data formats including netCDF
and HDF4/5 was added (TIKA-400 and TIKA-399).
* Unit tests for Windows have been fixed, allowing TestParsers
to complete. (TIKA-398)
Tika 0.8 relies on the following set of major dependencies (generated using
mvn dependency:tree from tika-parsers):
org.apache.tika:tika-parsers:bundle:0.8
+- org.apache.tika:tika-core:jar:0.8:compile
+- edu.ucar:netcdf:jar:4.2:compile
| \- org.slf4j:slf4j-api:jar:1.5.6:compile
+- commons-httpclient:commons-httpclient:jar:3.1:compile
| +- commons-logging:commons-logging:jar:1.1.1:compile (version managed from 1.0.4)
| \- commons-codec:commons-codec:jar:1.2:compile
+- org.apache.commons:commons-compress:jar:1.1:compile
+- org.apache.pdfbox:pdfbox:jar:1.3.1:compile
| +- org.apache.pdfbox:fontbox:jar:1.3.1:compile
| \- org.apache.pdfbox:jempbox:jar:1.3.1:compile
+- org.bouncycastle:bcmail-jdk15:jar:1.45:compile
+- org.bouncycastle:bcprov-jdk15:jar:1.45:compile
+- org.apache.poi:poi:jar:3.7:compile
+- org.apache.poi:poi-scratchpad:jar:3.7:compile
+- org.apache.poi:poi-ooxml:jar:3.7:compile
| +- org.apache.poi:poi-ooxml-schemas:jar:3.7:compile
| | \- org.apache.xmlbeans:xmlbeans:jar:2.3.0:compile
| \- dom4j:dom4j:jar:1.6.1:compile
+- org.apache.geronimo.specs:geronimo-stax-api_1.0_spec:jar:1.0.1:compile
+- org.ccil.cowan.tagsoup:tagsoup:jar:1.2:compile
+- asm:asm:jar:3.1:compile
+- com.drewnoakes:metadata-extractor:jar:2.4.0-beta-1:compile
+- de.l3s.boilerpipe:boilerpipe:jar:1.1.0:compile
+- rome:rome:jar:0.9:compile
\- jdom:jdom:jar:1.0:compile
The following people have contributed to Tika 0.8 by submitting or commenting
on the issues resolved in this release:
Łukasz Wiktor
Adam Wilmer
Alex Baranau
Alex Ott
André Ricardo
Andrey Barhatov
Andrey Sidorenko
Antoni Mylka
Arturo Beltran
Attila Kir√°ly
Brad Greenlee
Bruno Dumon
Chris A. Mattmann
Chris Bamford
Christophe Gourmelon
Dave Meikle
David Weekly
Dmitry Kuzmenko
Erik Hetzner
Geoff Jarrad
Gerd Bremer
Grant Ingersoll
Jan H√∏ydahl
Jean-Philippe Ricard
Jeremias Maerki
Joao Garcia
Jukka Zitting
Julien Nioche
Ken Krugler
Liam O'Boyle
Mads Hansen
Marcel May
Markus Goldbach
Martijn van Groningen
Maxim Valyanskiy
Mike Hays
Miroslav Pokorny
Nick Burch
Otis Gospodnetic
Peter van Raamsdonk
Peter Wolanin
Piotr Bartosiewicz
Radek
Rajiv Kumar
Reinhard Schwab
rick cameron
Robert Muir
Sanjeev Rao
Simon Tyler
Sjoerd Smeets
Slavomir Varchula
Staffan Olsson
Tom De Leu
Uwe Schindler
Victor Kazakov
See http://s.apache.org/ab0 for more details on these contributions.
Release 0.7 - 3/31/2010
-----------------------
The most notable changes in Tika 0.7 over previous releases are:
* MP3 file parsing was improved, including Channel and SampleRate
extraction and ID3v2 support (TIKA-368, TIKA-372). Further, audio
parsing mime detection was also improved for the MIDI format. (TIKA-199)
* Tika no longer relies on X11 for its RTF parsing functionality. (TIKA-386)
* A Thread-safe bug in the AutoDetectParser was discovered and
addressed. (TIKA-374)
* Upgrade to PDFBox 1.0.0. The new PDFBox version improves PDF parsing
performance and fixes a number of text extraction issues. (TIKA-380)
The following people have contributed to Tika 0.7 by submitting or commenting
on the issues resolved in this release:
Adam Rauch
Benson Margulies
Brett S.
Chris A. Mattmann
Daan de Wit
Dave Meikle
Durville
Ingo Renner
Jukka Zitting
Ken Krugler
Kenny Neal
Markus Goldbach
Maxim Valyanskiy
Nick Burch
Sami Siren
Uwe Schindler
See http://tinyurl.com/yklopby for more details on these contributions.
Release 0.6 - 01/20/2010
------------------------
The most notable changes in Tika 0.6 over the previous release are:
* Mime-type detection for HTML (and all types) has been improved, allowing malformed
HTML files and those HTML files that require a bit more observed content
before the type is properly detected, are now correctly identified by
the AutoDetectParser. (TIKA-327, TIKA-357, TIKA-366, TIKA-367)
* Tika now has an additional OSGi bundle packaging that includes all the
required parser libraries. This bundle package makes it easy to use all
Tika features in an OSGi environment. (TIKA-340, TIKA-342)
* The Apache POI dependency used for parsing Microsoft Office file formats
has been upgraded to version 3.6. The most visible improvement in this
version is the notably reduced ooxml jar file size. The tika-app jar size
is now down to 15MB from the 25MB in Tika 0.5. (TIKA-353)
* Handling of character encoding information in input metadata and HTML
<meta> tags has been improved. When no applicable encoding information is
available, the encoding is detected by looking at the input data.
(TIKA-332, TIKA-334, TIKA-335, TIKA-341)
* Some document types like Excel spreadsheets contain content like
numbers or formulas whose exact text format depends on the current locale.
So far Tika has used the platform default locale in such cases, but
clients can now explicitly specify the locale by passing a Locale instance
in the parse context. (TIKA-125)
* The default text output encoding of the tika-app jar is now UTF-8
when running on Mac OS X. This is because the default encoding used
by Java is not compatible with the console application in Mac OS X.
On all other platforms the text output from tika-app still uses
the platform default encoding. (TIKA-324)
* A flash video (video/x-flv) parser has been added. (TIKA-328)
* The handling of Number and Date cell formatting within the Microsoft Excel
documents has been added. This include currencies, percentages and
scientific formats. (TIKA-103)
The following people have contributed to Tika 0.6 by submitting or commenting
on the issues resolved in this release:
Andrzej Bialecki
Bertrand Delacretaz
Chris A. Mattmann
Dave Meikle
Erik Hetzner
Felix Meschberger
Jukka Zitting
Julien Nioche
Ken Krugler
Luke Nezda
Maxim Valyanskiy
Niall Pemberton
Peter Wolanin
Piotr B.
Sami Siren
Yuan-Fang Li
See http://tinyurl.com/yc3dk67 for more details on these contributions.
Release 0.5 - 11/14/2009
------------------------
The most notable changes in Tika 0.5 over the previous release are:
* Improved RDF/OWL mime detection using both MIME magic as well as
pattern matching (TIKA-309)
* An org.apache.tika.Tika facade class has been added to simplify common
text extraction and type detection use cases. (TIKA-269)
* A new parse context argument was added to the Parser.parse() method.
This context map can be used to pass things like a delegate parser or
other settings to the parsing process. The previous parse() method
signature has been deprecated and will be removed in Tika 1.0. (TIKA-275)
* A simple ngram-based language detection mechanism has been added along
with predefined language profiles for 18 languages. (TIKA-209)
* The media type registry in Tika was synchronized with the MIME type
configuration in the Apache HTTP Server. Tika now knows about 1274
different media types and can detect 672 of those using 927 file
extension and 280 magic byte patterns. (TIKA-285)
* Tika now uses the Apache PDFBox version 0.8.0-incubating for parsing PDF
documents. This version is notably better than the 0.7.3 release used
earlier. (TIKA-158)
The following people have contributed to Tika 0.5 by submitting or commenting
on the issues resolved in this release:
Alex Baranov
Bart Hanssens
Benson Margulies
Chris A. Mattmann
Daan de Wit
Erik Hetzner
Frank Hellwig
Jeff Cadow
Joachim Zittmayr
Jukka Zitting
Julien Nioche
Ken Krugler
Maxim Valyanskiy
MRIT64
Paul Borgermans
Piotr B.
Robert Newson
Sascha Szott
Ted Dunning
Thilo Goetz
Uwe Schindler
Yuan-Fang Li
See http://tinyurl.com/yl9prwp for more details on these contributions.
Release 0.4 - 07/14/2009
------------------------
The most notable changes in Tika 0.4 over the previous release are:
* Tika has been split to three different components for increased
modularity. The tika-core component contains the key interfaces and
core functionality of Tika, tika-parsers contains all the adapters
to external parser libraries, and tika-app bundles everything together
in a single executable jar file. (TIKA-219)
* All the three Tika components are packaged as OSGi bundles. (TIKA-228)
* Tika now uses the new Commons Compress library for improved support
of compression and packaging formats like gzip, bzip2, tar, cpio,
ar, zip and jar. (TIKA-204)
* The memory use of parsing Excel sheets with lots of numbers
has been considerably reduced. (TIKA-211)
* The AutoDetectParser now has basic protection against "zip bomb"
attacks, where a specially crafted input document can expand to
practically infinite amount of output text. (TIKA-216)
* The ParsingReader class can now use a thread pool or a more complex
execution model (java.util.concurrent.Executor) for the background
parsing task. (TIKA-215)
* Automatic type detection of text- and XML-based documents has been
improved. (TIKA-225)
* Charset detection functionality from the ICU4J library was inlined
in Tika to avoid the dependency to the large ICU4J jar. (TIKA-229)
* Composite parsers like the AutoDetectParser now make sure that any
RuntimeExceptions, IOExceptions or SAXExceptions unrelated to the given
document stream or content handler are converted to TikaExceptions
before being passed to the client. (TIKA-198, TIKA-237)
The following people have contributed to Tika 0.4 by submitting or commenting
on the issues resolved in this release:
Chris A. Mattmann
Daan de Wit
Dave Meikle
David Weekly
Jeremias Maerki
Jonathan Koren
Jukka Zitting
Karl Heinz Marbaise
Keith R. Bennett
Maxim Valyanskiy
Niall Pemberton
Robert Burrell Donkin
Sami Siren
Siddharth Gargate
Uwe Schindler
See http://tinyurl.com/mgv9o3 for more details on these contributions.
Release 0.3 - 03/09/2009
------------------------
The most notable changes in Tika 0.3 over the previous release are:
* Tika now supports mime type glob patterns specified using
standard JDK 1.4 (and beyond) syntax via the isregex attribute
on the glob tag. See:
http://java.sun.com/j2se/1.4.2/docs/api/java/util/regex/Pattern.html
for more information. (TIKA-194)
* Tika now supports the Office Open XML format used by
Microsoft Office 2007. (TIKA-152)
* All the metadata keys for Microsoft Office document properties are now
included as constants in the MSOffice interface. Clients should use
these constants instead of the raw string values to refer to specific
metadata items. (TIKA-186)
* Automatic detection of document types in Tika has been improved.
For example Tika can now detect plain text just by looking at the first
few bytes of the document. (TIKA-154)
* Tika now disables the loading of all external entities in XML files
that it parses as input documents. This improves security and avoids
problems with potentially broken references. (TIKA-185)
* Tika now replaces all invalid XML characters in the extracted text
content with spaces. This prevents problems when output from Tika
is processed with XML tools. (TIKA-180)
* The Tika CLI now correctly flushes its buffers when invoked with the
--text argument. This prevents the end of the text output from being
lost. (TIKA-179)
* Embedded text in MIDI files is now extracted. For example many karaoke
files contain song lyrics embedded as MIDI text.
* The text content of Microsoft Outlook message files no longer appears as
multiple copies in the extracted text. (TIKA-197)
* The ParsingReader class now makes most document metadata available
already before any of the extracted text is consumed. This makes it
easier for example to construct Lucene Document instances that contain
both extracted text and metadata. (TIKA-203)
See http://tinyurl.com/tika-0-3-changes for a list of all changes in Tika 0.3.
The following people have contributed to Tika 0.3 by submitting or commenting
on the issues resolved in this release:
Andrzej Rusin
Chris A. Mattmann
Dave Meikle
Georger Ara√∫jo
Guillermo Arribas
Jonathan Koren
Jukka Zitting
Karl Heinz Marbaise
Kumar Raja Jana
Paul Borgermans
Peter Becker
Sébastien Michel
Uwe Schindler
See http://tinyurl.com/tika-0-3-contributions for more details on
these contributions.
Release 0.2 - 12/04/2008
------------------------
1. TIKA-109 - WordParser fails on some Word files (Dave Meikle)
2. TIKA-105 - Excel parser implementation based on POI's Event API
(Niall Pemberton)
3. TIKA-116 - Streaming parser for OpenDocument files (Jukka Zitting)
4. TIKA-117 - Drop JDOM and Jaxen dependencies (Jukka Zitting)
5. TIKA-115 - Tika package with all the dependencies (Jukka Zitting)
6. TIKA-97 - Tika GUI (Jukka Zitting)
7. TIKA-96 - Tika CLI (Jukka Zitting)
8. TIKA-112 - Use Commons IO 1.4 (Jukka Zitting)
9. TIKA-127 - Add support for Visio files (Jukka Zitting)
10. TIKA-129 - node() support for the streaming XPath utility (Jukka Zitting)
11. TIKA-130 - self-or-descendant axis does not match self in streaming XPath
(Jukka Zitting)
12. TIKA-131 - Lazy XHTML prefix generation (Jukka Zitting)
13. TIKA-128 - HTML parser should produce XHTML SAX events (Jukka Zitting)
14. TIKA-133 - TeeContentHandler constructor should use varargs (Jukka Zitting)
15. TIKA-132 - Refactor Excel extractor to parse per sheet and add
hyperlink support (Niall Pemberton)
16. TIKA-134 - mvn package does not produce packages for bin/src
(Karl Heinz Marbaise)
17. TIKA-138 - Ignore HTML style and script content (Jukka Zitting)
18. TIKA-113 - Metadata (such as title) should not be part of content
(Jukka Zitting)
19. TIKA-139 - Add a composite parser (Jukka Zitting)
20. TIKA-142 - Include application/xhtml+xml as valid mime type for XMLParser
(mattmann)
21. TIKA-143 - Add ParsingReader (Jukka Zitting)
22. TIKA-144 - Upgrade nekohtml dependency (Jukka Zitting)
23. TIKA-145 - Separate NOTICEs and LICENSEs for binary and source packages
(Jukka Zitting)
24. TIKA-146 - Upgrade to POI 3.1 (Jukka Zitting)
25. TIKA-99 - Support external parser programs (Jukka Zitting)
26. TIKA-149 - Parser for Zip files (Dave Meikle & Jukka Zitting)
27. TIKA-150 - Parser for tar files (Jukka Zitting)
28. TIKA-151 - Stream compression support (Jukka Zitting)
29. TIKA-156 - Some MIME magic patterns are ignored by MimeTypes
(Jukka Zitting)
30. TIKA-155 - Java class file parser (Dave Brosius & Jukka Zitting)
31. TIKA-108 - New Tika logos (Yongqian Li & Jukka Zitting)
32. TIKA-120 - Add support for retrieving ID3 tags from MP3 files
(Dave Meikle & Jukka Zitting)
33. TIKA-54 - Outlook msg parser
(Rida Benjelloun, Dave Meikle & Jukka Zitting)
34. TIKA-114 - PDFParser : Getting content of the document using
"writer.ToString ()" , some words are stuck together
(Dave Meikle)
35. TIKA-161 - Enable PMD reports (Jukka Zitting)
36. TIKA-159 - Add support for parsing basic audio types: wav, aiff, au, midi
(Sami Siren)
37. TIKA-140 - HTML parser unable to extract text
(Julien Nioche & Jukka Zitting)
38. TIKA-163 - GUI does not support drag and drop in Gnome or KDE (Dave Meikle)
39. TIKA-166 - Update HTMLParser to parse contents of meta tags (Dave Meikle)
40. TIKA-164 - Upgrade of the nekohtml dependency to 1.9.9 (Jukka Zitting)
41. TIKA-165 - Upgrade of the ICU4J dependency to version 3.8 (Jukka Zitting)
42. TIKA-172 - New Open Document Parser that emits structured XHTML content
(Uwe Schindler & Jukka Zitting)
43. TIKA-175 - Retrotranslate Tika for use in Java 1.4 environments (Jukka Zitting)
44. TIKA-177 - Improvements to build instruction in README (Chris Hostetter & Jukka Zitting)
45. TIKA-171 - New ContentHandler for plain text output that has no problem with
missing white space after XHTML block tags (Uwe Schindler & Jukka Zitting)
Release 0.1-incubating - 12/27/2007
-----------------------------------
1. TIKA-5 - Port Metadata Framework from Nutch (mattmann)
2. TIKA-11 - Consolidate test classes into a src/test/java directory tree (mattmann)
3. TIKA-15 - Utils.print does not print a Content having no value (jukka)
4. TIKA-19 - org.apache.tika.TestParsers fails (bdelacretaz)
5. TIKA-16 - Issues with data files used for testing by TestParsers (bdelacretaz)
6. TIKA-14 - MimeTypeUtils.getMimeType() returns the default mime type for
.odt (Open Office) file (bdelacretaz)
7. TIKA-12 - Add URL capability to MimeTypesUtils (jukka)
8. TIKA-13 - Fix obsolete package names in config.xml (siren)
9. TIKA-10 - Remove MimeInfoException catch clauses and import from TestParsers (siren)
10. TIKA-8 - Replaced the jmimeinfo dependency with a trivial mime type detector (jukka)
11. TIKA-7 - Added the Lius Lite code. Added missing dependencies to POM (jukka)
12. TIKA-18 - "Office" interface should be renamed "MSOffice" (mattmann)
13. TIKA-23 - Decouple Parser from ParserConfig (jukka)
14. TIKA-6 - Port Nutch (or better) MimeType detection system into Tika (J. Charron & mattmann)
15. TIKA-25 - Removed hardcoded reference to C:\oo.xml in OpenOfficeParser (K. Bennett & jukka)
16. TIKA-17 - Need to support URL's for input resources. (K. Bennett & mattmann)
17. TIKA-22 - Remove @author tags from the java source (mattmann)
18. TIKA-21 - Simplified configuration code (jukka)
19. TIKA-17 - Rename all "Lius" classes to be "Tika" classes (jukka)
20. TIKA-30 - Added utility constructors to TikaConfig (K. Bennett & jukka)
21. TIKA-28 - Rename config.xml to tika-config.xml or similar (mattmann)
22. TIKA-26 - Use Map<String, Content> instead of List<Content> (jukka)
23. TIKA-31 - protected Parser.parse(InputStream stream,
Iterable<Content> contents) (jukka & K. Bennett)
24. TIKA-36 - A convenience method for getting a document's content's text
would be helpful (K. Bennett & mattmann)
25. TIKA-33 - Stateless parsers (jukka)
26. TIKA-38 - TXTParser adds a space to the content it reads from a file (K. Bennett & ridabenjelloun)
27. TIKA-35 - Extract MsOffice properties, use RereadableInputStream devloped by K. Bennett (ridabenjelloun & K. Bennett)
28. TIKA-39 - Excel parsing improvements (siren & ridabenjelloun)
29. TIKA-34 - Provide a method that will return a default configuration
(TikaConfig) (K. Bennett & mattmann)
30. TIKA-42 - Content class needs (String, String, String) constructor (K. Bennett)
31. TIKA-43 - Parser interface (jukka)
32. TIKA-47 - Remove TikaLogger (jukka)
33. TIKA-46 - Use Metadata in Parser (jukka & mattmann)
34. TIKA-48 - Merge MS Extractors and Parsers (jukka)
35. TIKA-45 - RereadableInputStream needs to be able to read to
the end of the original stream on first rewind. (K. Bennett)
36. TIKA-41 - Resource files occur twice in jar file. (jukka)
37. TIKA-49 - Some files have old-style license headers, fixed (Robert Burrell Donkin & bdelacretaz)
38. TIKA-51 - Leftover temp files after running Tika tests, fixed (bdelacretaz)