Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom Hierarchies #1432

Open
wants to merge 26 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
c3522f0
JSON backend: Fail when trying to open non-existing groups
franzpoeschel May 30, 2023
506363f
Insert CustomHierarchy class to Iteration
franzpoeschel Apr 12, 2023
a57bf26
Help older compilers deal with this
franzpoeschel May 25, 2023
56363a5
Add vector variants of meshes/particlesPath
franzpoeschel Jul 28, 2023
83d2e12
Move meshes and particles over to CustomHierarchies class
franzpoeschel Jul 28, 2023
0ee50f4
Move dirtyRecursive to CustomHierarchy
franzpoeschel May 30, 2023
10b6b9e
Move Iteration reading logic to CustomHierarchy
franzpoeschel Jul 28, 2023
0b3bb43
Move Iteration flushing logic to CustomHierarchy class
franzpoeschel Jul 31, 2023
90cd659
Support for custom datasets
franzpoeschel Jul 31, 2023
b321772
Treat "meshes"/"particles" as normal subgroups
franzpoeschel Jul 31, 2023
9335e13
Regex-based list of meshes/particlesPaths
franzpoeschel Jul 31, 2023
1415e90
More extended testing
franzpoeschel Jul 31, 2023
6303b14
Fix Python bindings without adding new functionality yet
franzpoeschel Aug 1, 2023
9468600
Add simple Python bindings and an example
franzpoeschel Aug 1, 2023
1876759
Replace Regexes with Globbing
franzpoeschel Oct 13, 2023
01e59a7
Move .meshes and .particles back to Iteration class
franzpoeschel Oct 24, 2023
3849c74
Some fixes in read error handling
franzpoeschel Oct 26, 2023
4acd1f8
More symmetric design for container types
franzpoeschel Oct 26, 2023
0637683
Don't write unitSI in custom datasets
franzpoeschel Nov 13, 2023
a869c3d
Discouraged support for custom datasets inside the particlesPath
franzpoeschel Dec 21, 2023
8dbe1c2
Fix after rebase: dirtyRecursive
franzpoeschel Mar 26, 2024
a4512d7
Fixes to the dirty/dirtyRecursive logic
franzpoeschel May 24, 2024
59a4a1c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 7, 2024
583ec1b
Some cleanup in CustomHierarchies class
franzpoeschel Aug 14, 2024
317cc0f
Use polymorphism for meshes/particlesPath in Python
franzpoeschel Nov 6, 2024
1032573
Remove hasMeshes / hasParticles logic
franzpoeschel Nov 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add vector variants of meshes/particlesPath
  • Loading branch information
franzpoeschel committed Dec 17, 2024
commit 56363a576d186ddbd90ba8ecd01e4b719a522786
6 changes: 4 additions & 2 deletions include/openPMD/Series.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -424,6 +424,7 @@ class Series : public Attributable
* <CODE>basePath</CODE>.
*/
std::string meshesPath() const;
std::vector<std::string> meshesPaths() const;
/** Set the path to <A
* HREF="https://github.com/openPMD/openPMD-standard/blob/latest/STANDARD.md#mesh-based-records">mesh
* records</A>, relative(!) to <CODE>basePath</CODE>.
Expand All @@ -434,6 +435,7 @@ class Series : public Attributable
* @return Reference to modified series.
*/
Series &setMeshesPath(std::string const &meshesPath);
Series &setMeshesPath(std::vector<std::string> const &meshesPath);

/**
* @throw no_such_attribute_error If optional attribute is not present.
Expand Down Expand Up @@ -467,6 +469,7 @@ class Series : public Attributable
* <CODE>basePath</CODE>.
*/
std::string particlesPath() const;
std::vector<std::string> particlesPaths() const;
/** Set the path to groups for each <A
* HREF="https://github.com/openPMD/openPMD-standard/blob/latest/STANDARD.md#particle-records">particle
* species</A>, relative(!) to <CODE>basePath</CODE>.
Expand All @@ -477,6 +480,7 @@ class Series : public Attributable
* @return Reference to modified series.
*/
Series &setParticlesPath(std::string const &particlesPath);
Series &setParticlesPath(std::vector<std::string> const &particlesPath);

/**
* @throw no_such_attribute_error If optional attribute is not present.
Expand Down Expand Up @@ -877,8 +881,6 @@ OPENPMD_private
iterations_iterator end,
internal::FlushParams const &flushParams,
bool flushIOHandler = true);
void flushMeshesPath();
void flushParticlesPath();
void flushRankTable();
/* Parameter `read_only_this_single_iteration` used for reopening an
* Iteration after closing it.
Expand Down
19 changes: 19 additions & 0 deletions include/openPMD/backend/Attribute.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -292,6 +292,25 @@ namespace detail
}
}
}
// conversion cast: turn a 1-element vector into a single value
else if constexpr (auxiliary::IsVector_v<T>)
{
if constexpr (std::is_convertible_v<typename T::value_type, U>)
{
if (pv->size() != 1)
{
return {std::runtime_error(
"getCast: vector to scalar conversion requires "
"single-element vectors")};
}
return {U(*pv->begin())};
}
else
{
return {std::runtime_error(
"getCast: no vector to scalar conversion possible.")};
}
}
else
{
return {std::runtime_error("getCast: no cast possible.")};
Expand Down
2 changes: 0 additions & 2 deletions src/Iteration.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -358,7 +358,6 @@ void Iteration::flushIteration(internal::FlushParams const &flushParams)
if (!s.containsAttribute("meshesPath"))
{
s.setMeshesPath("meshes/");
s.flushMeshesPath();
}
meshes.flush(s.meshesPath(), flushParams);
for (auto &m : meshes)
Expand All @@ -374,7 +373,6 @@ void Iteration::flushIteration(internal::FlushParams const &flushParams)
if (!s.containsAttribute("particlesPath"))
{
s.setParticlesPath("particles/");
s.flushParticlesPath();
}
particles.flush(s.particlesPath(), flushParams);
for (auto &species : particles)
Expand Down
104 changes: 80 additions & 24 deletions src/Series.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,27 @@ Series &Series::setBasePath(std::string const &bp)

std::string Series::meshesPath() const
{
return getAttribute("meshesPath").get<std::string>();
auto res = meshesPaths();
if (res.empty())
{
throw no_such_attribute_error("meshesPath");
}
/*
* @todo: Verify that meshesPath has canonical form
*/
return res.at(0);
}

std::vector<std::string> Series::meshesPaths() const
{
if (containsAttribute("meshesPath"))
{
return getAttribute("meshesPath").get<std::vector<std::string>>();
}
else
{
return {};
}
}

Series &Series::setMeshesPath(std::string const &mp)
Expand All @@ -212,6 +232,23 @@ Series &Series::setMeshesPath(std::string const &mp)
setDirty(true);
return *this;
}
Series &Series::setMeshesPath(std::vector<std::string> const &mp)
{
// @todo if already written, then append
switch (mp.size())
{
case 0:
return *this;
case 1:
setAttribute("meshesPath", *mp.begin());
break;
default:
setAttribute("meshesPath", mp);
break;
}
dirty() = true;
return *this;
}

#if openPMD_HAVE_MPI
chunk_assignment::RankMeta Series::rankTable(bool collective)
Expand Down Expand Up @@ -479,7 +516,27 @@ void Series::flushRankTable()

std::string Series::particlesPath() const
{
return getAttribute("particlesPath").get<std::string>();
auto res = particlesPaths();
if (res.empty())
{
throw no_such_attribute_error("particlesPath");
}
/*
* @todo: Verify that particlesPath has canonical form
*/
return res.at(0);
}

std::vector<std::string> Series::particlesPaths() const
{
if (containsAttribute("particlesPath"))
{
return getAttribute("particlesPath").get<std::vector<std::string>>();
}
else
{
return {};
}
}

Series &Series::setParticlesPath(std::string const &pp)
Expand All @@ -502,6 +559,23 @@ Series &Series::setParticlesPath(std::string const &pp)
setDirty(true);
return *this;
}
Series &Series::setParticlesPath(std::vector<std::string> const &pp)
{
// @todo if already written, then append
switch (pp.size())
{
case 0:
return *this;
case 1:
setAttribute("particlesPath", *pp.begin());
break;
default:
setAttribute("particlesPath", pp);
break;
}
dirty() = true;
return *this;
}

std::string Series::author() const
{
Expand Down Expand Up @@ -1547,26 +1621,6 @@ void Series::flushGorVBased(
}
}

void Series::flushMeshesPath()
{
Parameter<Operation::WRITE_ATT> aWrite;
aWrite.name = "meshesPath";
Attribute a = getAttribute("meshesPath");
aWrite.resource = a.getResource();
aWrite.dtype = a.dtype;
IOHandler()->enqueue(IOTask(this, aWrite));
}

void Series::flushParticlesPath()
{
Parameter<Operation::WRITE_ATT> aWrite;
aWrite.name = "particlesPath";
Attribute a = getAttribute("particlesPath");
aWrite.resource = a.getResource();
aWrite.dtype = a.dtype;
IOHandler()->enqueue(IOTask(this, aWrite));
}

void Series::readFileBased(
std::optional<IterationIndex_t> read_only_this_single_iteration)
{
Expand Down Expand Up @@ -2287,7 +2341,8 @@ void Series::readBase()
aRead.name = "meshesPath";
IOHandler()->enqueue(IOTask(this, aRead));
IOHandler()->flush(internal::defaultFlushParams);
if (auto val = Attribute(*aRead.resource).getOptional<std::string>();
if (auto val = Attribute(*aRead.resource)
.getOptional<std::vector<std::string>>();
val.has_value())
{
/* allow setting the meshes path after completed IO */
Expand Down Expand Up @@ -2325,7 +2380,8 @@ void Series::readBase()
aRead.name = "particlesPath";
IOHandler()->enqueue(IOTask(this, aRead));
IOHandler()->flush(internal::defaultFlushParams);
if (auto val = Attribute(*aRead.resource).getOptional<std::string>();
if (auto val = Attribute(*aRead.resource)
.getOptional<std::vector<std::string>>();
val.has_value())
{
/* allow setting the meshes path after completed IO */
Expand Down