[Feature] middleware / custom transforms support

[SuperchupuDev]

currently, what is returned after crawling and what is used in filters is always what fdir has processed. i propose a function that allows users to transform those paths with custom logic.

why? because if a user wants to use their custom logic to transform fdir paths they currently have to duplicate it if using filters/excludes/globs (which also means the transform logic is ran twice, which is bad for performance):

const files = await new fdir()
  .withRelativePaths()
  .filter(p => {
    // whatever custom logic they might need
    const path = `../${p}`;

    return someFilteringFunction(path);
  })
  .crawl(root)
  .withPromise()

// the exact same custom logic again
return files.map(p => `../${p}`)

with support for custom transforms, it could look something more like this:

return new fdir()
  .withRelativePaths()
  .transform(p => `../${p}`)
  .filter(someFilteringFunction)
  .crawl(root)
  .withPromise()

// it returns paths with a leading `../`, without duplicating the transform logic. woohoo!

see a real world example of the problem this proposal aims to solve in https://github.com/SuperchupuDev/tinyglobby/blob/1b51f7b90fb0b7db09d6d656b6f80341a0eaeffd/src/index.ts#L72-L73 and https://github.com/SuperchupuDev/tinyglobby/pull/18/files' processPath function

[thecodrr]

Hmm, I don't think fdir is the right place for transformation middleware. Whoever is using this will eventually have to loop over the paths anyway so doing it upfront doesn't really provide any benefit. Secondly, what if the transformation requires async code? It'll mean giving up the sync API which is a no-go.

This is just too out of scope for a directory crawler.

[SuperchupuDev]

Whoever is using this will eventually have to loop over the paths anyway

the whole point of this is to not need to loop over the paths outside fdir more than necessary..?

Secondly, what if the transformation requires async code? It'll mean giving up the sync API which is a no-go.

fair point

[thecodrr]

the whole point of this is to not need to loop over the paths outside fdir more than necessary..?

The user will have to loop over the paths at least once if they intend to do anything with the files. The path transformations can take place there without incurring any extra performance cost. It'll also allow for much more flexible transformations e.g. you want to change the path into an object or an array of segments. Dealing with all of that inside fdir will be a disaster.