add QCOM_motion_estimation (KhronosGroup#383)

* add QCOM_motion_estimation

* minor edits to the extension text

* remove header updates

extensions/QCOM/QCOM_motion_estimation.txt

@@ -0,0 +1,185 @@
+Name
+
+    QCOM_motion_estimation
+
+Name Strings
+
+    GL_QCOM_motion_estimation
+
+Contributors
+
+    Jonathan Wicks
+    Sam Holmes
+    Jeff Leger
+
+Contacts
+
+    Jeff Leger  <[email protected]>
+
+Status
+
+    Complete
+
+Version
+
+    Last Modified Date: March 19, 2020
+    Revision: 1.0
+
+Number
+
+    OpenGL ES Extension #326
+
+Dependencies
+
+    Requires OpenGL ES 2.0
+
+    This extension is written against the OpenGL ES 3.2 Specification.
+
+    This extension interacts with OES_EGL_image_external.
+
+Overview
+
+    Motion estimation, also referred to as optical flow, is the process of
+    producing motion vectors that convey the 2D transformation from a reference
+    image to a target image.  There are various uses of motion estimation, such as
+    frame extrapolation, compression, object tracking, etc.
+
+    This extension adds support for motion estimation in OpenGL ES by adding
+    functions which take the reference and target images and populate an
+    output texture containing the corresponding motion vectors.
+
+New Procedures and Functions
+
+    void TexEstimateMotionQCOM(uint ref,
+                               uint target,
+                               uint output);
+
+    void TexEstimateMotionRegionsQCOM(uint ref,
+                                      uint target,
+                                      uint output,
+                                      uint mask);
+
+New Tokens
+
+    Accepted by the <pname> parameter of GetIntegerv, GetInteger64v, and GetFloatv:
+
+        MOTION_ESTIMATION_SEARCH_BLOCK_X_QCOM    0x8C90
+        MOTION_ESTIMATION_SEARCH_BLOCK_Y_QCOM    0x8C91
+
+Additions to the OpenGL ES 3.2 Specification
+
+    Add two new rows in Table 21.40 "Implementation Dependent Values"
+
+    Get Value                              Type     Get Command  Minimum Value   Description           Sec
+    ---------                              ----     -----------  -------------   -----------           ------
+    MOTION_ESTIMATION_SEARCH_BLOCK_X_QCOM   Z+      GetIntegerv  1               The block size in X   8.19
+    MOTION_ESTIMATION_SEARCH_BLOCK_Y_QCOM   Z+      GetIntegerv  1               The block size in Y   8.19
+
+Additions to Chapter 8 of the OpenGL ES 3.2 Specification
+
+    The commands
+
+    void TexEstimateMotionQCOM(uint ref,
+                               uint target,
+                               uint output);
+
+    void TexEstimateMotionRegionsQCOM(uint ref,
+                                      uint target,
+                                      uint output,
+                                      uint mask);
+
+    are called to perfom the motion estimation based on the contents of the two input
+    textures, <ref> and <target>.  The results of the motion estimation are stored in
+    the <output> texture.
+
+    The <ref> and <target> must be either be GL_R8 2D textures, or backed by EGLImages where
+    the underlying format contain a luminance plane.  The <ref> and <target> dimension must
+    be identical and must be an exact multiple of the search block size.  While <ref> and <target>
+    can have multiple levels, the implementation only reads from the base level.
+
+    The resulting motion vectors are stored in a 2D texture <output> of the format GL_RGBA16F,
+    ready to be used by other application shaders and stages.  While <output> can have multiple
+    levels, the implementation only writes to the base level.  The <output> dimensions
+    must be set as follows so that it can hold one vector per search block:
+
+        output.width  = ref.width  / MOTION_ESTIMATION_SEARCH_BLOCK_X_QCOM
+        output.height = ref.height / MOTION_ESTIMATION_SEARCH_BLOCK_Y_QCOM
+
+    Each texel in the <output> texture represents the estimated motion in pixels, for the supported
+    search block size, from the <ref> texture to the <target> target texture.  Implementations may
+    generate sub-pixel motion vectors, in which case the returned vector components may have fractional
+    values.  The motion vector X and Y components are provided in the R and G channels respectively.
+    The B and A components are currently undefined and left for future expansion.  If no motion is
+    detected for a block, or if the <mask> texture indicates that the block should be skipped, then
+    the R and G channels will be set to zero, indicating no motion.
+
+    The <mask> texture is used to control the region-of-interest which can help to reduce the
+    overall workload.  The <mask> texture dimensions must exactly match that of the <output>
+    texture and the format must be GL_R8UI.  While <mask> can have multiple levels, the
+    implementation only reads from the base level.  For any texel with a value of 0 in the <mask>
+    motion estimation will not be performed for the corresponding block.  Any non-zero texel value
+    will produce a motion vector result in the <output> result.  The <mask> only controls the vector
+    basepoint.  Therefore it is possible for an unmasked block to produce a vector that lands in the
+    masked block.
+
+Errors
+
+    INVALID_OPERATION is generated if any of the textures passed in are invalid
+
+    INVALID_OPERATION is generated if the texture types are not TEXTURE_2D or TEXTURE_EXTERNAL_OES
+
+    INVALID_OPERATION is generated if <ref> is not of the format GL_R8, or when backed by an EGLImage,
+    when the underlying internal format does not contain a luminance plane.
+
+    INVALID_OPERATION is generated if <target> is not of the format GL_R8, or when backed by an EGLImage,
+    when the underlying internal format does not contain a luminance plane.
+
+    INVALID_OPERATION is generated if the <ref> and <target> textures do not have
+    identical dimensions
+
+    INVALID_OPERATION is generated if the <output> texture is not of the format GL_RGBA16F
+
+    INVALID_OPERATION is generated if the <mask> texture is not of the format GL_R8UI
+
+    INVALID_OPERATION is generated if the <output> or <mask> dimensions are not
+    ref.[width/height] / MOTION_ESTIMATION_SEARCH_BLOCK_[X,Y]_QCOM
+
+Interactions with OES_EGL_image_external
+
+    If OES_EGL_image_external is supported, then the <ref> and/or <target> parameters to
+    TexEstimateMotionQCOM and TexEstimateMotionRegionsQCOM may be backed by an EGLImage.
+
+Issues
+
+    (1) What should the pixel data of the input textures <ref> and <target> contain?
+
+    Resolved: Motion estimation tracks the brightness across the input textures.  To produce
+    the best results, it is recommended that the texels in the <ref> and <target> textures
+    represent some measure of the luminance/luma.  OpenGL ES does not currently expose
+    a Y8 or Y plane only format, so GL_R8 can be used.  Alternatively, a texture backed by
+    and EGLImage, which has an underlying format where luminance is contained in a separate plane,
+    can also be used.  If starting with an RGBA8 texture one way to convert it to GL_R8 would be
+    to perform a copy and use code such as the following:
+
+        fragColor = rgb_2_yuv(texture(tex, texcoord).rgb, itu_601_full_range).r;\n"
+
+    (2) Why use GL_RGBA16F instead of GL_RG16F for storing the motion vector output?
+
+    Resolved: While only the R and G channels are currently used, it was decided to use
+    a format with more channels for future expansion.  A floating point format was chosen
+    to support implementations with sub-pixel precision without enforcing any particular precision
+    requirements other than what can be represented in a 16-bit floating point number.
+
+    (3) Why is the motion estimation quality not defined?
+
+    Resolved: The intention of this specification is to estimate the motion between
+    the two input textures.  Implementations should aim to produce the highest quality estimations
+    but since the results are estimations there are no prescribed steps for how the vectors
+    must be generated.
+
+
+Revision History
+
+      Rev.  Date        Author          Changes
+      ----  ----------  --------        -----------------------------------------
+      1.0   03/19/2020  Jonathan Wicks  Initial public version

extensions/esext.php

@@ -677,4 +677,6 @@
 </li>
 <li value=325><a href="extensions/QCOM/QCOM_shading_rate.txt">GL_QCOM_shading_rate</a>
 </li>
+<li value=326><a href="extensions/QCOM/QCOM_motion_estimation.txt">GL_QCOM_motion_estimation</a>
+</li>
 </ol>

extensions/registry.py

@@ -4484,6 +4484,11 @@
         'flags' : { 'public' },
         'url' : 'extensions/QCOM/QCOM_framebuffer_foveated.txt',
     },
+    'GL_QCOM_motion_estimation' : {
+        'esnumber' : 326,
+        'flags' : { 'public' },
+        'url' : 'extensions/QCOM/QCOM_motion_estimation.txt',
+    },
     'GL_QCOM_performance_monitor_global_mode' : {
         'esnumber' : 56,
         'flags' : { 'public' },

xml/gl.xml

@@ -9526,7 +9526,8 @@ typedef unsigned int GLhandleARB;
     </enums>
 
     <enums namespace="GL" start="0x8C90" end="0x8C9F" vendor="QCOM" comment="For Affie Munshi. Reassigned from AMD to QCOM (bug 5874)">
-            <unused start="0x8C90" end="0x8C91" vendor="QCOM"/>
+        <enum value="0x8C90" name="GL_MOTION_ESTIMATION_SEARCH_BLOCK_X_QCOM" group="GetPName"/>
+        <enum value="0x8C91" name="GL_MOTION_ESTIMATION_SEARCH_BLOCK_Y_QCOM" group="GetPName"/>
         <enum value="0x8C92" name="GL_ATC_RGB_AMD"/>
         <enum value="0x8C93" name="GL_ATC_RGBA_EXPLICIT_ALPHA_AMD"/>
             <unused start="0x8C94" end="0x8C9F" vendor="QCOM"/>
@@ -28864,6 +28865,19 @@ typedef unsigned int GLhandleARB;
             <param group="TextureEnvParameter"><ptype>GLenum</ptype> <name>pname</name></param>
             <param len="COMPSIZE(pname)">const <ptype>GLfixed</ptype> *<name>params</name></param>
         </command>
+        <command>
+            <proto>void <name>glTexEstimateMotionQCOM</name></proto>
+            <param group="Texture"><ptype>GLuint</ptype> <name>ref</name></param>
+            <param group="Texture"><ptype>GLuint</ptype> <name>target</name></param>
+            <param group="Texture"><ptype>GLuint</ptype> <name>output</name></param>
+        </command>
+        <command>
+            <proto>void <name>glTexEstimateMotionRegionsQCOM</name></proto>
+            <param group="Texture"><ptype>GLuint</ptype> <name>ref</name></param>
+            <param group="Texture"><ptype>GLuint</ptype> <name>target</name></param>
+            <param group="Texture"><ptype>GLuint</ptype> <name>output</name></param>
+            <param group="Texture"><ptype>GLuint</ptype> <name>mask</name></param>
+        </command>
         <command>
             <proto>void <name>glTexFilterFuncSGIS</name></proto>
             <param group="TextureTarget"><ptype>GLenum</ptype> <name>target</name></param>
@@ -50956,6 +50970,15 @@ typedef unsigned int GLhandleARB;
                 <command name="glFramebufferFoveationParametersQCOM"/>
             </require>
         </extension>
+        <extension name="GL_QCOM_motion_estimation" supported="gles2">
+            <require>
+                <enum name="GL_MOTION_ESTIMATION_SEARCH_BLOCK_X_QCOM"/>
+                <enum name="GL_MOTION_ESTIMATION_SEARCH_BLOCK_Y_QCOM"/>
+                <enum name="GL_FOVEATION_SCALED_BIN_METHOD_BIT_QCOM"/>
+                <command name="glTexEstimateMotionQCOM"/>
+                <command name="glTexEstimateMotionRegionsQCOM"/>
+            </require>
+        </extension>
         <extension name="GL_QCOM_texture_foveated" supported="gles2">
             <require>
                 <enum name="GL_FOVEATION_ENABLE_BIT_QCOM"/>