Using AWS Mountpoint with Paperclip

[slimdave]

This is more of a "question than a comment" I guess.

We've been using Paperclip backed by S3 for many years, and in general everything is working great for us.

There are a couple of small issues – for example, we build millions of XML fragments, and then combine them into a single larger file of, say, 250MB, which we store and SFTP to third-parties. I think we could maybe stream io from the S3 object into the SFTP client and then onwards, but currently we copy to local file and then send it.

Anyway, I've been looking at AWS Mountpoint: https://docs.aws.amazon.com/AmazonS3/latest/userguide/mountpoint.html

... a high-throughput open source file client for mounting an Amazon S3 bucket as a local file system. With Mountpoint, your applications can access objects stored in Amazon S3 through file system operations, such as open and read. Mountpoint automatically translates these operations into S3 object API calls, giving your applications access to the elastic storage and throughput of Amazon S3 through a file interface.

Although there are limitations (no updates), for attachments where we can work within them this would make some of our operations easier to code – for example we could be combining multiple XML fragments together with a fairly simple cat. Paperclip could be using file storage, and host tools would have direct access to the objects path without needing objects copied to local storage. It could even make testing easier as we'd not need to stub S3 calls.

I'm wondering whether there's something I've not thought of here, or in fact whether anyone has already tried this?

Thanks in advance for any experiences or advice.