-
Notifications
You must be signed in to change notification settings - Fork 269
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Added info about caching to the documentation
- Loading branch information
Showing
2 changed files
with
19 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
Caching datasets locally | ||
======================== | ||
|
||
In some use cases, it may be desirable to set Fuel's ``data_path`` to | ||
point to a shared network drive. For example, when configuring multiple | ||
machines in a cluster to work on the same data in parallel. | ||
However, this can easily cause network bandwidth to become saturated. | ||
|
||
To avoid this problem, Fuel provides a second configuration variable | ||
named ``local_data_path``, which can be set in ``~/.fuelrc``. This | ||
variable points to a filesystem directory to be used to act as a local | ||
cache for datasets. | ||
|
||
This variable can also be set through an environment variable as follows: | ||
|
||
.. code-block:: bash | ||
$ export FUEL_LOCAL_DATA_PATH="/home/username/my_local_cache" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters