csv: Inconsistency re QUOTE_NONNUMERIC

[tlotze]

BPO	30046
Nosy	@serhiy-storchaka, @zhangyangyu, @tlotze, @corona10

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2017-04-11.20:41:55.773>
labels = ['type-bug', 'library']
title = 'csv: Inconsistency re QUOTE_NONNUMERIC'
updated_at = <Date 2017-04-19.07:11:22.021>
user = 'https://github.com/tlotze'

bugs.python.org fields:

activity = <Date 2017-04-19.07:11:22.021>
actor = 'corona10'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2017-04-11.20:41:55.773>
creator = 'tlotze'
dependencies = []
files = []
hgrepos = []
issue_num = 30046
keywords = []
message_count = 5.0
messages = ['291516', '291590', '291862', '291864', '291865']
nosy_count = 4.0
nosy_names = ['serhiy.storchaka', 'xiang.zhang', 'tlotze', 'corona10']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue30046'
versions = []

Linked PRs

• gh-74232: Adds note about csv quoting of booleans #134422
• gh-74232: Add a note about roundtrip of non-float numerics in CSV #134963
• [3.14] gh-74232: Add a note about roundtrip of non-float numerics in CSV (GH-134963) #135048
• [3.13] gh-74232: Add a note about roundtrip of non-float numerics in CSV (GH-134963) #135049

[tlotze]

A csv.writer with quoting=csv.QUOTE_NONNUMERIC does not quote boolean values, which makes a csv.reader with the same quoting behaviour fail on that value:

-------- csv.py ----------

import csv
import io


f = io.StringIO()

writer = csv.writer(f, quoting=csv.QUOTE_NONNUMERIC)
writer.writerow(['asdf', 1, True])

f.seek(0)
reader = csv.reader(f, quoting=csv.QUOTE_NONNUMERIC)
for row in reader:
    print(row)

---

$ python3 csvbug.py 
Traceback (most recent call last):
  File "csvbug.py", line 12, in <module>
    for row in reader:
ValueError: could not convert string to float: 'True'

---

I'd consider this inconsistency a bug, but in any case something that needs documenting.

[zhangyangyu]

boolean is not quoted since in Python it's a subclass of int so True and False are numeric. This is also the case with numeric objects defining __int__ or __float__ but doesn't get a corresponding string representation.

Since QUOTE_NONNUMERIC will converts data to float when reading, I think we may force the converting even when writing so the inconsistency would disappear. Or document this limitation.

[serhiy-storchaka]

This issue is not easy, it needs a thoughtful design before starting coding. I agree with Xiang's analysis and proposed solutions. But if just convert numbers to float we can get an overflow for large integers or lost precision in case of Decimal. The consumer of the CSV file may be not Python and it may support larger precision than Python float.

And it is not clear what would be better solution for enums. Should they be serialized by name or by value?

[corona10]

I would like to solve this issue.
Is there any other way than casting specifically for the bool object?
(e.g For general ways?)

[corona10]

Oh, I read the Serhiy Storchaka 's comment just right now.

[aterrel]

I don't think it is reasonable to change the type system and this isn't a type bug perse, more of a difference in type expectations.

I've added the note to the docs but this is my first doc fix in CPython so not sure if I did it correctly.

[serhiy-storchaka]

Other example is Fraction. 1/2 cannot be converted to float.

We need a parameter for the reader to control how to convert non-quoted fields to numeric. Something like parse_float in json, but it will work not only with floats.

We may add also add parameters for the reader to control how to convert numerics and other types to strings.