Skip to content

Commit

Permalink
Update file_combinations.md
Browse files Browse the repository at this point in the history
  • Loading branch information
samuell authored May 21, 2019
1 parent f19915d commit d365071
Showing 1 changed file with 36 additions and 8 deletions.
44 changes: 36 additions & 8 deletions docs/howtos/file_combinations.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,24 @@
Sometimes you need to create all the possible combinations of a set of files
that you have as file streams. This you can do with the
[FileCombinator](https://godoc.org/github.com/scipipe/scipipe/components#FileCombinator)
component.
that you have as file streams.

For example, say that you have two file streams:

```
[a.txt b.txt]
[1.txt 2.txt 3.txt]
```

... and you want to process all of the combinations of these two sets of files.
So in other words, what you want is:

```
[a.txt a.txt a.txt b.txt b.txt b.txt]
[1.txt 2.txt 3.txt 1.txt 2.txt 3.txt]
```

This is something you can accomplish with the [FileCombinator](https://godoc.org/github.com/scipipe/scipipe/components#FileCombinator)
component, available in [SciPipe 0.9.1](https://github.com/scipipe/scipipe/releases/tag/v0.9.1)
and later.

## Example

Expand All @@ -15,7 +32,7 @@ numberfile_2.txt
numberfile_3.txt
```

... and you want to create all combinations of the letter files and the number
... and you want to create all combinations of the `letter*` files and the `number*`
files, you can do it as follows:


Expand Down Expand Up @@ -46,15 +63,26 @@ func main() {
}
```

This will generate the following files (excluding the accompanying .audit.json files):
Note that when accessing an in-port on the FileCombinator with the `In(PORTNAME)` method, this port
will be created automatically, together with a corresponding out-port which can be accessed with the
same name, `Out(PORTNAME)`, as can be seen when we connect the fileCombinator to the catenator process
further down in the code.

The program above, if put in a `.go` file and run with `go run file.go`, will generate the following
files (excluding the accompanying .audit.json files):

```
letterfile_b.txt
letterfile_a.txt
letterfile_a.numberfile_1.combined.txt
numberfile_3.txt
numberfile_2.txt
numberfile_1.txt
letterfile_a.numberfile_2.combined.txt
letterfile_a.numberfile_1.combined.txt
letterfile_a.numberfile_3.combined.txt
letterfile_b.txt
letterfile_b.numberfile_1.combined.txt
letterfile_b.numberfile_2.combined.txt
letterfile_b.numberfile_1.combined.txt
letterfile_b.numberfile_3.combined.txt
```

As you can see, all the combinations of the

0 comments on commit d365071

Please sign in to comment.