Production experience?

[aehlke]

Thanks for the terrific work. Is anyone using this in production / how has it gone? Is there anything newer to look into, recommends are also welcome (besides the paid and online ones like gpt4o of course). thank you

edit: also any experience using this with Japanese / manga content would be great too (where panels may have variety that I don't know whether was included in the inputs to this work).

[pedrovgs]

Hey @aehlke the project was usable when I released the very first version from the technical point of view but I don't think the model is production-ready. What does it mean? The code was executed and working as you would expect but the accuracy is not as good as a company would expect. Why? Because we don't have enough data to train the model. The model distributed with the library right now has been trained with a super small and private data set. Does it mean you can't use it? Nope, you can, but you'll have to train your model with your own data set if you want to get better results. May I ask where you are planning to use this project @aehlke ?

[aehlke]

Thanks very much for the context

I'm adding a manga reader to https://reader.manabi.io which is much anticipated by users. I'd use this library to detect the boundaries in order to let users tap to zoom to next/prev ones, and add scroll/swipe lock. I'll either try this, look into commissioning someone to train something for manga, or continue looking for other solutions (maybe more manga specific ones exist, I haven't looked yet)

edit: btw very impressed with goodnotes iOS architecture! I've been following swift cross-platform options closely. I'm currently excited for https://skip.tools because WASM still seems challenging toolchain wise and compatibility wise but it's definitely the future and something I must find a way to do (get my swift & swiftui apps onto non-apple platforms)

[pedrovgs]

Nice feature and app! You can adapt the model and use it with manga if you want. Just need to use manga for your dataset. If you have a web app you can also use the model from web with onnx.

[aehlke]

@pedrovgs thanks. I tried running your sample project btw and it only detects one panel (the whole page) for your demo images. know how to fix that? all I did was pod install and run in maccatalyst

[pedrovgs]

I never tested on catalyst. Check on the iOS simulator as well please. The demo app should work as the gif shows unless CoreML changed breaking compatibility with the model I exported long ago. You can also test the Android version. The result should be the same.

[aehlke]

Simulator errored, I will try on a real device. On an iPhone, I get the same result as Catalyst...

edit: it worked in Rosetta simulators. I'd need to figure out how to get it working on macOS and actual iOS devices, but this helps me evaluate the model as-is. Thank you