Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Read mode: Distinguish random access from streaming/linear access #1234

Open
franzpoeschel opened this issue Mar 22, 2022 · 0 comments
Open

Comments

@franzpoeschel
Copy link
Contributor

Is your feature request related to a problem? If so, please describe.

Two angles to this:

  1. Calling Series("data.bp", Access::READ) will implicitly parse the entire Series in random-access mode. If using series.readIterations() afterwards, this is unnecessary. Using a streaming read mode allows us to skip this, possibly simplifying the control flow in the ADIOS2 backend.
  2. The upcoming BP5 engine in ADIOS2 will also distinguish between these two kinds of access modes. Random-accessing a series written via the streaming API will then only be possible by explicitly using the random access read mode.

Describe the solution you'd like
The existing Access::READ_ONLY keeps its semantics. a new mode READ_LINEAR or READ_STREAMING is added.

Describe alternatives you've considered
Deferred iteration parsing does similar things and will probably share some logic. It's not the same though:

  1. A READ_LINEAR mode affects the moment when BeginStep() is first called during reading.
  2. A READ_LINEAR mode will affect the mode that will be used to open the ADIOS2 engine.
  3. Deferred iteration parsing does not change the behavior of the backend at all, it just does not descend into the single iterations while parsing initially.

Additional context
Further questions:

  • What should be the behavior of this mode in file-based iteration encoding, group-based iteration encoding without steps, or other backends?
    -> Probably similar to deferred iteration parsing
  • As a perspective, we might merge deferred iteration parsing with this feature. But this will probably require an API addition for explicit random access (series.jumpToStep() or sth). For now, I would keep them separate.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant