Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Preserve nanosecond resolution when encoding/decoding times #7827

Merged
merged 26 commits into from
Sep 17, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
db94bb9
preserve nanosecond resolution when encoding/decoding times.
kmuehlbauer Aug 31, 2023
82b74ec
Apply suggestions from code review
kmuehlbauer Sep 4, 2023
8da55ac
use emit_user_level_warning
kmuehlbauer Sep 4, 2023
96f60fc
move time alignment for nc3 to encode_nc3_variable
kmuehlbauer Sep 4, 2023
4f6440a
fix test for encode_cf_timedelta
kmuehlbauer Sep 4, 2023
f8e961f
fix CFMaskCoder for time-like (also allow timedelta64), add first tests
kmuehlbauer Sep 4, 2023
c97d7b4
Merge branch 'main' into nanoseconds-resolution
kmuehlbauer Sep 5, 2023
06ed4bf
Merge branch 'main' into nanoseconds-resolution
kmuehlbauer Sep 5, 2023
5e4a5ee
Merge branch 'main' into nanoseconds-resolution
kmuehlbauer Sep 7, 2023
2365da0
Merge branch 'main' into nanoseconds-resolution
kmuehlbauer Sep 11, 2023
d020bbc
rename to _unpack_time_units_and_ref_date as suggested in review
kmuehlbauer Sep 10, 2023
a8c6057
refactor delta -> time_units as suggested in review
kmuehlbauer Sep 10, 2023
9b96ff7
refactor out function _time_units_to_timedelta64, reorder flow and re…
kmuehlbauer Sep 10, 2023
5adb58e
import _is_time_like from coding.variables
kmuehlbauer Sep 11, 2023
87fbb1a
adapt tests, add _numpy_to_netcdf_timeunit-conversion function
kmuehlbauer Sep 11, 2023
1fbc881
adapt tests, add _numpy_to_netcdf_timeunit-conversion function
kmuehlbauer Sep 11, 2023
8a6ecb5
Merge branch 'main' into nanoseconds-resolution
kmuehlbauer Sep 11, 2023
c1f810c
Merge branch 'main' into nanoseconds-resolution
kmuehlbauer Sep 11, 2023
7f2b8b4
Merge branch 'main' into nanoseconds-resolution
kmuehlbauer Sep 13, 2023
d538ea9
adapt test as per review, remove arm_xfail for backend test
kmuehlbauer Sep 13, 2023
a75e4b8
add whats-new.rst entry
kmuehlbauer Sep 13, 2023
d4a71cd
Update doc/whats-new.rst
kmuehlbauer Sep 14, 2023
4dca66e
Update doc/whats-new.rst
kmuehlbauer Sep 14, 2023
ee1b939
Merge remote-tracking branch 'origin/main' into nanoseconds-resolution
kmuehlbauer Sep 16, 2023
ebb00b8
fix whats-new.rst
kmuehlbauer Sep 16, 2023
c4d4a92
Merge remote-tracking branch 'origin/main' into nanoseconds-resolution
kmuehlbauer Sep 17, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
adapt test as per review, remove arm_xfail for backend test
  • Loading branch information
kmuehlbauer committed Sep 13, 2023
commit d538ea9355bbd1c7aefa9444b235e8a18be62663
2 changes: 0 additions & 2 deletions xarray/tests/test_backends.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,6 @@
from xarray.core.options import set_options
from xarray.core.pycompat import array_type
from xarray.tests import (
arm_xfail,
assert_allclose,
assert_array_equal,
assert_equal,
Expand Down Expand Up @@ -526,7 +525,6 @@ def test_roundtrip_string_encoded_characters(self) -> None:
assert_identical(expected, actual)
assert actual["x"].encoding["_Encoding"] == "ascii"

@arm_xfail
def test_roundtrip_numpy_datetime_data(self) -> None:
times = pd.to_datetime(["2000-01-01", "2000-01-02", "NaT"])
expected = Dataset({"t": ("t", times), "t0": times[0]})
Expand Down
15 changes: 10 additions & 5 deletions xarray/tests/test_coding_times.py
Original file line number Diff line number Diff line change
Expand Up @@ -1321,16 +1321,21 @@ def test_roundtrip_timedelta64_nanosecond_precision_warning() -> None:

def test_roundtrip_float_times() -> None:
fill_value = 20.0
t0 = "2000-01-01 12:00:00"
times = [np.datetime64(t0, "ns"), np.datetime64("NaT", "ns")]
times = [np.datetime64("2000-01-01 12:00:00", "ns"), np.datetime64("NaT", "ns")]

units = "days since 2000-01-01"
var = Variable(
["time"], times, encoding=dict(dtype=np.float64, _FillValue=fill_value)
["time"],
times,
encoding=dict(dtype=np.float64, _FillValue=fill_value, units=units),
)

encoded_var = conventions.encode_cf_variable(var)
decoded_var = conventions.decode_cf_variable("foo", encoded_var)
np.testing.assert_array_equal(encoded_var, np.array([0.5, 20.0]))
assert encoded_var.attrs["units"] == units
assert encoded_var.attrs["_FillValue"] == fill_value

decoded_var = conventions.decode_cf_variable("foo", encoded_var)
assert_identical(var, decoded_var)
assert decoded_var.encoding["units"] == f"days since {t0}"
assert decoded_var.encoding["units"] == units
assert decoded_var.encoding["_FillValue"] == fill_value