Tags: simw/pipedata
Tags
Refactor code to make simpler (#12) * Move optypes into separate file * Rename ChainStart to Chain and StreamStart to Stream * Add .then and pipe operations, as aliases to flat_map * Bump version to 0.3 * Remove functions in chain, move to a more functional approach of modifying the functions * Remove .map, .filter etc functions in Stream, moving to a more functional approach * Rename batching to batched
Update zipped_files to return name as well as contents; update json_r… …ecords and csv_records to take new structure; bump to 0.2 (#9) * Add more logging; add py.typed file to mark library as typed * Change zipped_files to return ZippedfileRef instead of just open file, update to version 0.2 * Update json_records and csv_records to either take IO or OpenedFileRef
Add data operations for reading from zip, iterating over csv and json… … records, and writing to parquet (#6) * Add data operations: reading from zip file, json and csv record parsing, writing to parquet * Update version to 0.1 * Fix linting; update to run multiple dependency versions * Run dependency version checks with specific python versions