Recommendations for faster AVIF encode times? (single frame, large resolution)

[kevinmershon]

I've been toying with libheif and libavif for a couple of days and both seem to have dreadfully slow encode times for AVIF files. Of the two only libheif seems actually usable, although encode times of a regular sized screenshot, e.g. 1280x800, of about 3.5 seconds on the lower end, up to 10 seconds sometimes.

A regular CanvasRenderTarget in C# encodes a similar sized jpeg in about 15-30ms by comparison, although even that is a tad on the slow side.
On a similar codebase in Swift on a Mac I'm getting encode times of about 7ms for jpeg and ~25ms for AVIF, so some kind of faster (hardware?) encoding tricks must be done which I have not yet figured out.

So far with libheif I've just been using the AOM encoder. I've started trying to build from source with rav1e but either my system is not configured correctly or there seems to be an error in the configuration of the AOM third-party lib:

ninja: error: build.ninja:3335: bad $-escape (literal $ must be written as $$)

[silverbacknet]

At one level, you'll get much better performance out of the SVT-AV1 encoder, and if you wait a bit for 3.0 to drop (most of the big patches are in now, it looks imminent), it'll also have higher quality to size, which AOM is currently the leader of. Downside is it can't do 4:4:4.

But at the high level, you don't present anything about any settings you used. Did you use just the default settings and let them go from there? It's unclear what you were using at all on the Mac. There are a few settings that will dramatically affect speed -- different codecs are set at different spots by default too, instead of all adjusted to be nearly equivalent.

Some external GPUs do have hardware encoders, but no Apple CPU does yet.

Default speeds of various codecs, higher is faster/lower quality:
libheif/aom: 6 (of 10)
libavif/aom: 6 also
libheif/svt-av1: 12 (of 13)
libheif/rav1e: 8 (of 10)

10-bit vs 8-bit and 4:4:4 vs 4:2:0 are the other really dramatic options. Quality sorta does, with some codecs adjusting to go slower at higher quality levels, if you don't manually specify.

[kevinmershon]

@silverbacknet Thank you for the quick reply! On Mac I'm not using libheif at all. I'll paste some code here to show what I'm doing for comparison. I've tried basically every setting I can think of. On the latest version of libheif setting the speed parameter throws an exception for me.

 private static bool setupFired = false;

 private static HeifContext ctx;
 private static HeifEncoder encoder;
 private static void setupHeifEncoder() {
     // we only ever want to call this once
     if (setupFired) {
         return;
     }
     setupFired = true;

     ctx = new HeifContext();
     encoder = ctx.GetEncoder(HeifCompressionFormat.Av1);
 }

 public byte[] EncodeAsAVIF() {
     if (encoder == null) {
         setupHeifEncoder();
     }
     var start = DateTime.Now;
     byte[] avifData;
     using (var memoryStream = new MemoryStream()) {
         // renderTarget is a CanvasRenderTarget with a screenshot's worth of pixel data in RGB.
         // this line takes ~6-12ms
         byte[] pixelBytes = this.renderTarget.GetPixelBytes();
         
         var encodingOptions = new HeifEncodingOptions {
             SaveAlphaChannel = false,
             WriteTwoColorProfiles = false
         };
         encoder.SetLossyQuality(20);
         encoder.SetLossless(false);
         encoder.SetParameter("chroma", "444");
         encoder.SetParameter("threads", "10");
         // I cannot set "speed" here or it throws an exception on AOM

         using (var image = new HeifImage((int)this.Width, (int)this.Height, HeifColorspace.Rgb, HeifChroma.InterleavedRgb24)) {
             image.AddPlane(HeifChannel.Interleaved, image.Width, image.Height, 24);

             // helper to load the pixels that takes ~8ms, I couldn't figure out how to just push directly to the HeifImage yet
             CopyRgb(pixelBytes, image, (int)this.Width, (int)this.Height);

             // this step is taking seconds to complete and should take < 50ms
             ctx.EncodeImage(image, encoder, encodingOptions);

             ctx.WriteToStream(memoryStream);
             avifData = memoryStream.ToArray();
         }
     }

     this.renderTarget.Dispose();
     return avifData;
 }

For comparison here's the swift code I have that is taking ~15-25ms but still so much faster than libheif on windows. This is using avif.swift which has both AOM and svtav1enc but I'm using AOM specifically. I've compared with svtav1enc and found AOM produced significantly better compression for about a 1ms performance hit, well within my tolerance.

do {
    let nsImage = NSImage(cgImage: image!, size: .init(width: CGFloat(width), height: CGFloat(height)))
    data = try Self.encoding.encode(nsImage, speed: 10, quality: 0.2, preferredCodec: .AOM)
} catch {
    data = Self.encodeToJPEG(cgImage: image!)
}

[kevinmershon]

Over the weekend I was able to get libavif-sharp working and the average time to render with it is about 50-350ms. Drastically longer than jpeg encoding, but still 10X faster than libheif, for comparison. I felt like this might be helpful to share as a comparable benchmark.