Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

to_binary_s should always return ASCII-8BIT encoded strings #120

Merged
merged 1 commit into from
Feb 21, 2020
Merged

to_binary_s should always return ASCII-8BIT encoded strings #120

merged 1 commit into from
Feb 21, 2020

Conversation

tenderlove
Copy link
Contributor

StringIO in Ruby <= 2.6.X didn't respect Encoding.default_internal, so
calling read on the StringIO wouldn't tag the string with the same
encoding. Ruby 2.7 fixes this bug. The problem is that to_binary_s
is returning a binary string and should always return an ASCII-8BIT
encoded string.

This patch changes the method to always return the ASCII-8BIT string
regardless of the value of Encoding.default_internal.

Here is the behavior of StringIO on Ruby 2.6:

$ ruby -v -rstringio -e'Encoding.default_internal = Encoding::UTF_8; io = StringIO.new("foo".force_encoding("ASCII-8BIT")); io.rewind; p io.read.encoding'
ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-darwin19]
-e:1: warning: setting Encoding.default_internal
 #<Encoding:ASCII-8BIT>

Here it is on 2.7:

$ ruby -v -rstringio -e'Encoding.default_internal = Encoding::UTF_8; io = StringIO.new("foo".force_encoding("ASCII-8BIT")); io.rewind; p io.read.encoding'
ruby 2.7.0p0 (2019-12-25 revision 647ee6f091) [x86_64-darwin19]
-e:1: warning: setting Encoding.default_internal
  #<Encoding:UTF-8>

We can see 2.7 respects default_internal, but unfortunately that means
that to_binary_s would return the wrong encoding.

@coveralls
Copy link

coveralls commented Feb 20, 2020

Coverage Status

Coverage decreased (-2.0e-05%) to 99.91% when pulling 26d35a1 on tenderlove:always-return-ascii-8bit into 2f22031 on dmendel:master.

oreoshake added a commit to github/darrrr that referenced this pull request Feb 20, 2020
StringIO in Ruby <= 2.6.X didn't respect `Encoding.default_internal`, so
calling `read` on the StringIO wouldn't tag the string with the same
encoding.  Ruby 2.7 fixes this bug.  The problem is that `to_binary_s`
is returning a binary string and should always return an ASCII-8BIT
encoded string.

This patch changes the method to always return the ASCII-8BIT string
regardless of the value of `Encoding.default_internal`.

Here is the behavior of StringIO on Ruby 2.6:

```
$ ruby -v -rstringio -e'Encoding.default_internal = Encoding::UTF_8; io = StringIO.new("foo".force_encoding("ASCII-8BIT")); io.rewind; p io.read.encoding'
ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-darwin19]
-e:1: warning: setting Encoding.default_internal
 #<Encoding:ASCII-8BIT>
```

Here it is on 2.7:

```
$ ruby -v -rstringio -e'Encoding.default_internal = Encoding::UTF_8; io = StringIO.new("foo".force_encoding("ASCII-8BIT")); io.rewind; p io.read.encoding'
ruby 2.7.0p0 (2019-12-25 revision 647ee6f091) [x86_64-darwin19]
-e:1: warning: setting Encoding.default_internal
  #<Encoding:UTF-8>
```

We can see 2.7 respects `default_internal`, but unfortunately that means
that `to_binary_s` would return the wrong encoding.
@dmendel dmendel merged commit 8acb1b2 into dmendel:master Feb 21, 2020
@dmendel
Copy link
Owner

dmendel commented Feb 21, 2020

Thanks! New version of BinData with this fix will be out shortly

@tenderlove tenderlove deleted the always-return-ascii-8bit branch February 21, 2020 16:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants