Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update README.md for rep_removal to mention memory requirement #1106

Open
wants to merge 3 commits into
base: dev
Choose a base branch
from
Open

Update README.md for rep_removal to mention memory requirement #1106

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9ab72da
Update README.md
shahrokhDaijavad Mar 6, 2025
c132340
Merge branch 'dev' into shahrokh_misc
shahrokhDaijavad Mar 10, 2025
ddcae21
Update README.md
shahrokhDaijavad Mar 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Update README.md
  • Loading branch information
@shahrokhDaijavad
shahrokhDaijavad committed Mar 6, 2025
commit 9ab72dac683aacbfa7739e2f88c0b314e8240ff8
3 changes: 3 additions & 0 deletions transforms/universal/rep_removal/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,9 @@ pip install --no-binary :all: psutil
[//]: # (cargo install --path $PACKAGE_LOCATION/dpk_rep_removal/rust)

[//]: # (```)

B) When we tested the transform with a large data file on an M1 Mac, we realized that more than 16GB of memory is needed. If the machine has less than 16GB of memory, we recommend using the [resize](https://github.com/IBM/data-prep-kit/tree/dev/transforms/universal/resize) transform to make the file smaller.

## Input Parameters

The transform can be initialized with the following parameters:
Expand Down