Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP]: Add Pyodide support and CI jobs for Zarr #1903

Draft
wants to merge 24 commits into
base: main
Choose a base branch
from
Draft

[WIP]: Add Pyodide support and CI jobs for Zarr #1903

Changes from 1 commit
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
9f5a110
Add CI job to test out-of-tree Pyodide builds
agriyakhetarpal May 22, 2024
29282fc
Add `[msgpack]` dependency for `numcodecs`
agriyakhetarpal May 23, 2024
d465742
Bump to Pyodide 0.26.0, update comments
agriyakhetarpal May 27, 2024
cdf0bb2
Try to run tests without async
agriyakhetarpal May 27, 2024
dfe0321
Move shared file to rootdir, outside v2 and v3
agriyakhetarpal May 27, 2024
b100ec9
Move `fasteners` import inside ThreadSynchronizer
agriyakhetarpal May 27, 2024
b0dddca
Make the tests directory importable, fix `_shared`
agriyakhetarpal May 27, 2024
d227728
Import list of greetings from `numcodecs`
agriyakhetarpal May 27, 2024
fdb2bef
Skip some tests that use threading
agriyakhetarpal May 27, 2024
621077a
Skip some tests that use `fcntl`
agriyakhetarpal May 27, 2024
7ae9a97
Skip tests that require `dbm`
agriyakhetarpal May 27, 2024
22eb6da
Move `IS_WASM` logic to internal `zarr` API
agriyakhetarpal May 27, 2024
6836947
Skip a few tests trying to import `multiprocessing`
agriyakhetarpal May 27, 2024
fe3bf27
Skip tests that use async and threading code
agriyakhetarpal May 27, 2024
08997ec
Improve `asyncio_tests_wrapper`, fix test imports
agriyakhetarpal May 27, 2024
9bfc860
Skip entire `test_codecs.py` file
agriyakhetarpal May 27, 2024
9bcb350
Skip yet another test that requires threads
agriyakhetarpal May 27, 2024
9985abb
xfail test where array's fill values are different
agriyakhetarpal May 27, 2024
7ea12ef
xfail test because Emscripten FS
agriyakhetarpal May 27, 2024
a6565de
Skip last test that tries to run threads
agriyakhetarpal May 27, 2024
85f621c
Another test that tries to run threads
agriyakhetarpal May 27, 2024
1a64255
xfail another array's differing `fill_values` test
agriyakhetarpal May 27, 2024
c8cb38b
Skip entire sync file under WASM, no threading
agriyakhetarpal May 27, 2024
eb36d40
Restore pytest config options, remove when needed
agriyakhetarpal May 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Skip entire test_codecs.py file
  • Loading branch information
@agriyakhetarpal
agriyakhetarpal committed May 27, 2024
commit 9bfc860c2f84b0667172154cea95400626dd0dda
11 changes: 5 additions & 6 deletions tests/v3/test_codecs.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,11 @@
from zarr.store import MemoryStore, StorePath
from zarr.testing.utils import IS_WASM, assert_bytes_equal

# Skip entire file if running on WASM platforms, see
# 1. https://github.com/pyodide/pyodide/issues/2221
# 2. https://github.com/pyodide/pyodide/issues/237
pytestmark = pytest.mark.skipif(IS_WASM, reason="Can't test async code in WASM")


@dataclass(frozen=True)
class _AsyncArrayProxy:
Expand Down Expand Up @@ -65,7 +70,6 @@ def order_from_dim(order: Literal["F", "C"], ndim: int) -> tuple[int, ...]:
return tuple(range(ndim))


@pytest.mark.skipif(IS_WASM, reason="Can't start new threads in WASM")
@pytest.mark.parametrize("index_location", ["start", "end"])
def test_sharding(
store: Store, sample_data: np.ndarray, index_location: ShardingCodecIndexLocation
Expand Down Expand Up @@ -96,7 +100,6 @@ def test_sharding(
assert np.array_equal(sample_data, read_data)


@pytest.mark.skipif(IS_WASM, reason="Can't test async code in WASM")
@pytest.mark.parametrize("index_location", ["start", "end"])
def test_sharding_partial(
store: Store, sample_data: np.ndarray, index_location: ShardingCodecIndexLocation
Expand Down Expand Up @@ -130,7 +133,6 @@ def test_sharding_partial(
assert np.array_equal(sample_data, read_data)


@pytest.mark.skipif(IS_WASM, reason="Can't test async code in WASM")
@pytest.mark.parametrize("index_location", ["start", "end"])
def test_sharding_partial_read(
store: Store, sample_data: np.ndarray, index_location: ShardingCodecIndexLocation
Expand Down Expand Up @@ -158,7 +160,6 @@ def test_sharding_partial_read(
assert np.all(read_data == 1)


@pytest.mark.skipif(IS_WASM, reason="Can't test async code in WASM")
@pytest.mark.parametrize("index_location", ["start", "end"])
def test_sharding_partial_overwrite(
store: Store, sample_data: np.ndarray, index_location: ShardingCodecIndexLocation
Expand Down Expand Up @@ -195,7 +196,6 @@ def test_sharding_partial_overwrite(
assert np.array_equal(data, read_data)


@pytest.mark.skipif(IS_WASM, reason="Can't test async code in WASM")
@pytest.mark.parametrize(
"outer_index_location",
["start", "end"],
Expand Down Expand Up @@ -303,7 +303,6 @@ async def test_order(
assert_bytes_equal(await (store / "order/0.0").get(), z._store["0.0"])


@pytest.mark.skipif(IS_WASM, reason="Can't test async code in WASM")
@pytest.mark.parametrize("input_order", ["F", "C"])
@pytest.mark.parametrize("runtime_write_order", ["F", "C"])
@pytest.mark.parametrize("runtime_read_order", ["F", "C"])
Expand Down