Skip to content

Tags: simw/pipedata

Tags

v0.4

Toggle v0.4's commit message

Verified

This commit was signed with the committer’s verified signature.
simw Simon
Update github actions plugins for release

v0.3

Toggle v0.3's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Refactor code to make simpler (#12)

* Move optypes into separate file

* Rename ChainStart to Chain and StreamStart to Stream

* Add .then and pipe operations, as aliases to flat_map

* Bump version to 0.3

* Remove functions in chain, move to a more functional approach of modifying the functions

* Remove .map, .filter etc functions in Stream, moving to a more functional approach

* Rename batching to batched

v0.2.2

Toggle v0.2.2's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
Add additional logging to read_from_parquet (#11)

v0.2.1

Toggle v0.2.1's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
Add read_from_parquet operation (#10)

v0.2

Toggle v0.2's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
Update zipped_files to return name as well as contents; update json_r…

…ecords and csv_records to take new structure; bump to 0.2 (#9)

* Add more logging; add py.typed file to mark library as typed

* Change zipped_files to return ZippedfileRef instead of just open file, update to version 0.2

* Update json_records and csv_records to either take IO or OpenedFileRef

v0.1.1

Toggle v0.1.1's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
Make parquet file writing start at 1 rather than 0, for ease of readi…

…ng (#8)

v0.1

Toggle v0.1's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
Add data operations for reading from zip, iterating over csv and json…

… records, and writing to parquet (#6)

* Add data operations: reading from zip file, json and csv record parsing, writing to parquet

* Update version to 0.1

* Fix linting; update to run multiple dependency versions

* Run dependency version checks with specific python versions

v0.0.1

Toggle v0.0.1's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
Refactor in order to be able to get more data about the operations ru…

…n in the pipeline (#4)

* Add function to get the counts of all stages of the pipeline

* Update README, fix tuple -> Tuple, add checks that zero values don't end the stream