Skip to content

Commit

Permalink
BUG: None in read_fwf colspec should work like empty slice
Browse files Browse the repository at this point in the history
DOC: update read_fwf docs
  • Loading branch information
TomAugspurger committed May 8, 2014
1 parent fa1f585 commit b7b6b1f
Show file tree
Hide file tree
Showing 4 changed files with 35 additions and 2 deletions.
2 changes: 2 additions & 0 deletions doc/source/io.rst
Original file line number Diff line number Diff line change
Expand Up @@ -809,6 +809,8 @@ two extra parameters:
String value 'infer' can be used to instruct the parser to try detecting
the column specifications from the first 100 rows of the data. Default
behaviour, if not specified, is to infer.
As with regular python slices, you can slice to the end of the line
with ``None``, e.g. ``colspecs = [(0, 1), (1, None)]``.
- ``widths``: A list of field widths which can be used instead of 'colspecs'
if the intervals are contiguous.

Expand Down
3 changes: 3 additions & 0 deletions doc/source/release.rst
Original file line number Diff line number Diff line change
Expand Up @@ -479,6 +479,9 @@ Bug Fixes
claim that they contained all the things (:issue:`7066`).
- Bug in ``DataFrame.boxplot`` where it failed to use the axis passed as the ``ax`` argument (:issue:`3578`)
- Bug in the ``XlsxWriter`` and ``XlwtWriter`` implementations that resulted in datetime columns being formatted without the time (:issue:`7075`)
were being passed to plotting method
- :func:`read_fwf` treats ``None`` in ``colspec`` like regular python slices. It now reads from the beginning
or until the end of the line when ``colspec`` contains a ``None`` (previously raised a ``TypeError``)

pandas 0.13.1
-------------
Expand Down
5 changes: 3 additions & 2 deletions pandas/io/parsers.py
Original file line number Diff line number Diff line change
Expand Up @@ -2232,10 +2232,11 @@ def __init__(self, f, colspecs, delimiter, comment):
"input was a %r" % type(colspecs).__name__)

for colspec in self.colspecs:

if not (isinstance(colspec, (tuple, list)) and
len(colspec) == 2 and
isinstance(colspec[0], (int, np.integer)) and
isinstance(colspec[1], (int, np.integer))):
isinstance(colspec[0], (int, np.integer, type(None))) and
isinstance(colspec[1], (int, np.integer, type(None)))):
raise TypeError('Each column specification must be '
'2 element tuple or list of integers')

Expand Down
27 changes: 27 additions & 0 deletions pandas/io/tests/test_parsers.py
Original file line number Diff line number Diff line change
Expand Up @@ -2326,6 +2326,33 @@ def test_fwf_colspecs_is_list_or_tuple_of_two_element_tuples(self):
'Each column specification must be.+'):
read_fwf(StringIO(self.data1), [('a', 1)])

def test_fwf_colspecs_None(self):
# GH 7079
data = """\
123456
456789
"""
colspecs = [(0, 3), (3, None)]
result = read_fwf(StringIO(data), colspecs=colspecs, header=None)
expected = DataFrame([[123, 456], [456, 789]])
tm.assert_frame_equal(result, expected)

colspecs = [(None, 3), (3, 6)]
result = read_fwf(StringIO(data), colspecs=colspecs, header=None)
expected = DataFrame([[123, 456], [456, 789]])
tm.assert_frame_equal(result, expected)

colspecs = [(0, None), (3, None)]
result = read_fwf(StringIO(data), colspecs=colspecs, header=None)
expected = DataFrame([[123456, 456], [456789, 789]])
tm.assert_frame_equal(result, expected)

colspecs = [(None, None), (3, 6)]
result = read_fwf(StringIO(data), colspecs=colspecs, header=None)
expected = DataFrame([[123456, 456], [456789, 789]])
tm.assert_frame_equal(result, expected)


def test_fwf_regression(self):
# GH 3594
#### turns out 'T060' is parsable as a datetime slice!
Expand Down

0 comments on commit b7b6b1f

Please sign in to comment.