BUG: Fixed assign failure when with Copy-on-Write #60941
Conversation
|
Since there has been no feedback for two weeks, just a friendly ping @mroeschke @rhshadrach @WillAyd , thanks 🙏 |
pandas/tests/indexing/test_iloc.py
Outdated
| # Check that view is modified correctly | ||
| expected_view = DataFrame( | ||
| {"B": [2, 2, 2, 2], "C": [3, 2, 1, 2]}, index=df.index | ||
| ) | ||
| tm.assert_frame_equal(df, expected_view) |
There was a problem hiding this comment.
Nit: Once the assignment is done, this is no longer a view. Can you change to just "df" instead of "view".
There was a problem hiding this comment.
Sure, sovled in fcffbf2
doc/source/whatsnew/v2.3.0.rst
Outdated
| updated to work correctly with NumPy >= 2 (:issue:`57739`) | ||
| - :meth:`Series.str.decode` result now has ``StringDtype`` when ``future.infer_string`` is True (:issue:`60709`) | ||
| - :meth:`~Series.to_hdf` and :meth:`~DataFrame.to_hdf` now round-trip with ``StringDtype`` (:issue:`60663`) | ||
| - The :meth:`DataFrame.iloc` now works correctly with ``copy_on_write`` option (:issue:`60309`) |
There was a problem hiding this comment.
Can you give a little more detail here. Perhaps adding
after subsetting the columns of a DataFrame and using a slice
There was a problem hiding this comment.
Sure, Solved in 725b41b
pandas/core/internals/managers.py
Outdated
| if isinstance(indexer[1], slice) and indexer[1] == slice(None): | ||
| col_indexer = slice(None) | ||
| else: | ||
| col_indexer = np.arange(len(blk_loc)) | ||
| self.blocks[0].setitem((indexer[0], col_indexer), value) |
There was a problem hiding this comment.
It's not clear to me why it's correct to pass through slice(None) but not other cases, e.g. slice(0, 3, 2) or [0, 1].
There was a problem hiding this comment.
I think you're right. My current changes failed under the following test case. I'll mark the PR as a draft until I properly handle the bug and will also add more test cases. Thanks!
dftest3 = pd.DataFrame(
{"A": [1, 4, 1, 5], "B": [2, 5, 2, 6], "C": [3, 6, 1, 7], "D": [8, 9, 10, 11]}
)
df3 = dftest3[["B", "C", "D"]]
df3.iloc[[1, 3], 0:3:2] = [[2, 2], [2, 2]]
|
This pull request is stale because it has been open for thirty days with no activity. Please update and respond to this comment if you're still interested in working on this. |
|
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen. |
|
@chilin0525 I wanted to fix this bug and decided to start from this PR to reuse the test and whatsnew, so pushed some changes here |
jorisvandenbossche
added
Bug
Indexing
|
This special case of splitting the block (to avoid having the copy the entire block) when a copy for CoW was triggered was clearly not covered very well. After expanding the test coverage (and updating some existing setitem tests to be parametrized on having a reference or not), had to do some more fixes. So this turned out to be more complicated than anticipated .. |
|
Owee, I'm MrMeeseeks, Look at me. There seem to be a conflict, please backport manually. Here are approximate instructions:
And apply the correct labels and milestones. Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon! Remember to remove the If these instructions are inaccurate, feel free to suggest an improvement. |
|
Thanks @chilin0525 and @jorisvandenbossche |
Co-authored-by: Joris Van den Bossche <[email protected]>
|
Manual backport -> #62409 |
…60941) (#62409) Co-authored-by: ChiLin Chiu <[email protected]>
doc/source/whatsnew/v2.3.0.rstfile if fixing a bug or adding a new feature.