KHR_texture_ktx

[rsahlin]

Development of this PR is halted until #1956 is finialized.
This extension started out as needed by KHR_lights_environment but the functionality is now moved into that extension

The purpose of this PR is to add support for textures using KTX v2 images, the intention is not to provide support for already supported formats such as the typical 8 bpp formats suppoted by JPEG or PNG, but rather to add support for formats that have increased precision and/or range.

However, the 'usual suspects' are included as means to strengthen KTX as an image container choice (over PNG/JPG).
You may see these as a means to 'cover all bases' in the glTF/KTX ecosystem as currently the only supported format is basisu supercompression.

This extension shall be seen as a building block for future extensions that will require HDR image support - for instance #1956
A similar extension aims at introducing HDR support using compressed textures : #1955

I am working on a CLI tool that can read .HDR (RGBE format) images and save out as rgb float format using KTX v2.
Currently very experimental and only has limited support for reading .HDR and saving in 32 bit float format.
Tool can also be used to pass a number of images as faces or layers, for instance when generating a cubemap.
Equirectangular images can be saved out as cubemap.

[lexaknyazev]

Given how glTF handles existing PNG/JPEG/Basis textures, this extension could only be used with non-color texture slots (e.g. normals or ORM) for the following reason.

None of the proposed pixel formats support sRGB transfer because they already have high enough precision. Yet, all texture slots that use color data are currently locked to 8-bit sRGB inputs. Down-sampling data at runtime would subvert the whole reason of having it stored in high precision. If the goal of this extension is to allow any existing texture slot to support high-precision inputs, the extension spec should redefine that explicitly by stating that in those cases a linear transfer function is used.

However the extension still cannot go beyond 0.0 ... 1.0 range as values outside of it are undefined for all existing color and non-color texture slots (ignoring IBL for now).

[rsahlin]

Right - we have a spec that declares sRGB transfer function for baseColorTexture and emissiveTexture, this is a property of the material object.
We also have extensions that may provide additional color texture sources.

What if:
We add wording to this extension that changes the baseColorTexture and emissiveTexture transfer function so that it is sRGB only when the image source is image/png or image/png.
This would be similar to what KHR_texture_basisu does to image.mimetype (it 'adds' the use of image/ktx2).

When this extension is used it is allowed to provide color textures that does not use sRGB transfer function for texture sources refered to by material source objects.
In the core spec this affects baseColorTextureand emissiveTexture, in extensions it will affect any color texture source.
This means that when this extension is used a texture that stores color values shall not be sRGB encoded, if it contains higher range than 8 bits per color component.

[atteneder]

Thanks for this draft,

The list of supported vkFormat values is a specific subset (meaning KTX2 supports more than that). I assume this was chosen, because those are uncompressed and universally supported across platforms, right?

Imagine upcoming demand to broaden the KTX2 support even further (e.g. to compressed formats like ASTC):

• Would it make sense to include the in this PR? (or even lift the limit to a subset)
• Should each set of formats have a dedicated extension (e.g. KHR_texture_astc) ? If so, should the naming of this extension (KHR_texture_ktx) be made more specific?
• Does that even make sense (supporting non-universally supported formats)?

Regards,
Andi

[rsahlin]

Thanks for your comment @atteneder

Yes, the formats are choses as either:

• A substitute for the (uncompressed) formats used to store PNG/JPEG source images.
• Uncompressed formats that deliver increased range or precision compared to PNG/JPEG source images.

I am not sure how to handle (directly) compressed formats such as ASTC - instead of using these the intention is to use KHR_texture_basisu - which can then be transcoded to ASTC on the target device.
Is there a reason why you do not think that KHR_texture_basisu can deliver the same result as ASTC?

So, in conclusion - I see this extension as just providing uncompressed texture formats that are universally available (or super simple to convert to/from)

Hope this answers your question?

[lexaknyazev]

Is there a reason why you do not think that KHR_texture_basisu can deliver the same result as ASTC?

The only reason is that ASTC supports many more configurations and block sizes than the ASTC subset used for Basis Universal.

Still, ASTC should not be included in this extension because emulating it on devices without hardware support is far from simple - the format is very complex.

[atteneder]

Hope this answers your question?

Indeed, thanks a lot!

Admitted, the KHR_texture_astc example does not make much sense in the presence of BasisU. But even if I cannot come up with some sane example/use-case right now, I guess one extension per KTX2 format subset is the direction this is taking.

[rsahlin]

Ah, ok I see thanks for the explanation @lexaknyazev
I agree - to provide support for a specific compressed texture format is not part of this extension :-)

[rsahlin]

@atteneder Please feel free to chime in on what VK_ formats that should be included in this extension.

[aaronfranke]

This PR has been superseded by KHR_texture_basisu, which adds support for KTX v2 .ktx2 images that can be used as textures. This PR should be closed.

I do see that this PR has "layer", while KHR_texture_basisu does not have this. What is the use case for this? If it's still needed, perhaps this can be reworked into another extension that builds upon KHR_texture_basisu.

[donmccurdy]

@aaronfranke this PR doesn't overlap with the features of KHR_texture_basisu, which allows only “Basis Universal” UASTC and ETC1S compressed textures. KTX2 is also capable of supporting dozens of other compressed and uncompressed formats, including floating-point textures used in image base lighting or lightmaps. KHR_texture_ktx was proposed to allow (some of) these other texture formats that KHR_texture_basisu does not.

But I think the OP is correct that it makes sense to continue with this extension only in conjunction with adding features to glTF, like IBL, that require some of these formats. And fully specifying an IBL extension is non-trivial.

[mosra]

A use case I have for this extension is large-ish texture atlases using texture arrays, i.e., using a 2D array KTX2 file and the "layer" in this extension. And not using it for anything fancy like IBL or any fancy format besides (block-compressed) 8-bit sRGB, just in places where regular 2D images would be used for regular textures.

What's nice about how the extension is made is that it still allows a fallback to 2D images if the importer doesn't support KTX2 files or texture arrays, and how there isn't anything else to be specified to make interaction with e.g. KHR_texture_transform work. For me, and together with offering an alternative to MSFT_texture_dds with a better format, that's the main selling point of this extension.

I feel like expanding the pixel format or texture format support should be a responsibility of some other extension, not this one, and I'm fine with it restricting the allowed pixel formats to a subset compatible with PNG/JPEG by default. Same for disallowing e.g. cube map KTX files. I think just even allowing non-Basis (array) KTX2 files to be used warrants existence of this extension, and the extras could be tangential KHR_texture_cubemap, KHR_texture_half_float etc. extensions, without tying a particular feature to a concrete file format.