[HW Accel Support]: GPU accel for semantic search (AMD Ryzen 7 5700U + ROCM image)

[gonzalezcalleja]

Describe the problem you are having

Hi,

I'm using frigate 73fee63-rocm image on a AMD Ryzen 7 5700U + 64GB RAM, with an USB coral accelerator. I would like use HW accel from GPU only for semantic search so I changed from small to large model but i found this error on startup:

frigate.util.model             WARNING : OpenVINO failed to build model, using CPU instead: Exception from src/inference/src/cpp/core.cpp:121

go2rtc is using GPU HW accel.

Version

73fee63-rocm

Frigate config file

detectors:
  coral_usb:
    type: edgetpu
    device: usb:0

semantic_search:
  enabled: true
  reindex: false
  model_size: large

docker-compose file or Docker CLI command

services:
  frigate:
    container_name: frigate
    network_mode: host
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    stop_grace_period: 30s 
    image: ghcr.io/blakeblackshear/frigate:73fee63-rocm
    shm_size: "4gb" 
    devices:
      - /dev/dri/renderD128 
      - /dev/dri/card0 
      - /dev/bus/usb:/dev/bus/usb
      - /dev/kfd
    volumes:
      - /home/jgonzalez/docker/frigate/config:/config
      - /etc/localtime:/etc/localtime:ro
      - /mnt/videovigilancia:/media/frigate
      - type: tmpfs
        target: /tmp/cache
        tmpfs:
          size: "5gb"

Relevant Frigate log output

frigate  | 2025-02-11 23:56:54.853588384  [2025-02-11 23:56:54] frigate.util.model             WARNING : OpenVINO failed to build model, using CPU instead: Exception from src/inference/src/cpp/core.cpp:121:
frigate  | 2025-02-11 23:56:54.853594530  Check '!m_device_map.empty()' failed at src/plugins/intel_gpu/src/plugin/plugin.cpp:383:
frigate  | 2025-02-11 23:56:54.853598372  [GPU] Can't get OPTIMIZATION_CAPABILITIES property as no supported devices found or an error happened during devices query.
frigate  | 2025-02-11 23:56:54.853602702  [GPU] Please check OpenVINO documentation for GPU drivers setup guide.
frigate  | 2025-02-11 23:56:54.853605496
frigate  | 2025-02-11 23:56:54.853607382
frigate  | 2025-02-11 23:56:54.853610665
frigate  | 2025-02-11 23:56:55.564313773  INFO: Created TensorFlow Lite XNNPACK delegate for CPU.

Relevant go2rtc log output

-

FFprobe output from your camera

-

Install method

Docker Compose

Object Detector

Coral

Network connection

Wired

Camera make and model

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

Output for: docker exec -it frigate /opt/rocm/bin/rocminfo

ROCk module is loaded
=====================
HSA System Attributes
=====================
Runtime Version:         1.1
System Timestamp Freq.:  1000.000000MHz
Sig. Max Wait Duration:  18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count)
Machine Model:           LARGE
System Endianness:       LITTLE
Mwaitx:                  DISABLED
DMAbuf Support:          YES

==========
HSA Agents
==========
*******
Agent 1
*******
  Name:                    AMD Ryzen 7 5700U with Radeon Graphics
  Uuid:                    CPU-XX
  Marketing Name:          AMD Ryzen 7 5700U with Radeon Graphics
  Vendor Name:             CPU
  Feature:                 None specified
  Profile:                 FULL_PROFILE
  Float Round Mode:        NEAR
  Max Queue Number:        0(0x0)
  Queue Min Size:          0(0x0)
  Queue Max Size:          0(0x0)
  Queue Type:              MULTI
  Node:                    0
  Device Type:             CPU
  Cache Info:
    L1:                      32768(0x8000) KB
  Chip ID:                 0(0x0)
  ASIC Revision:           0(0x0)
  Cacheline Size:          64(0x40)
  Max Clock Freq. (MHz):   4372
  BDFID:                   0
  Internal Node ID:        0
  Compute Unit:            16
  SIMDs per CU:            0
  Shader Engines:          0
  Shader Arrs. per Eng.:   0
  WatchPts on Addr. Ranges:1
  Features:                None
  Pool Info:
    Pool 1
      Segment:                 GLOBAL; FLAGS: FINE GRAINED
      Size:                    61618512(0x3ac3950) KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Alignment:         4KB
      Accessible by all:       TRUE
    Pool 2
      Segment:                 GLOBAL; FLAGS: KERNARG, FINE GRAINED
      Size:                    61618512(0x3ac3950) KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Alignment:         4KB
      Accessible by all:       TRUE
    Pool 3
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED
      Size:                    61618512(0x3ac3950) KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Alignment:         4KB
      Accessible by all:       TRUE
  ISA Info:
*******
Agent 2
*******
  Name:                    gfx90c
  Uuid:                    GPU-XX
  Marketing Name:          AMD Radeon Graphics
  Vendor Name:             AMD
  Feature:                 KERNEL_DISPATCH
  Profile:                 BASE_PROFILE
  Float Round Mode:        NEAR
  Max Queue Number:        128(0x80)
  Queue Min Size:          64(0x40)
  Queue Max Size:          131072(0x20000)
  Queue Type:              MULTI
  Node:                    1
  Device Type:             GPU
  Cache Info:
    L1:                      16(0x10) KB
    L2:                      1024(0x400) KB
  Chip ID:                 5708(0x164c)
  ASIC Revision:           0(0x0)
  Cacheline Size:          64(0x40)
  Max Clock Freq. (MHz):   1900
  BDFID:                   1280
  Internal Node ID:        1
  Compute Unit:            8
  SIMDs per CU:            4
  Shader Engines:          1
  Shader Arrs. per Eng.:   1
  WatchPts on Addr. Ranges:4
  Features:                KERNEL_DISPATCH
  Fast F16 Operation:      TRUE
  Wavefront Size:          64(0x40)
  Workgroup Max Size:      1024(0x400)
  Workgroup Max Size per Dimension:
    x                        1024(0x400)
    y                        1024(0x400)
    z                        1024(0x400)
  Max Waves Per CU:        40(0x28)
  Max Work-item Per CU:    2560(0xa00)
  Grid Max Size:           4294967295(0xffffffff)
  Grid Max Size per Dimension:
    x                        4294967295(0xffffffff)
    y                        4294967295(0xffffffff)
    z                        4294967295(0xffffffff)
  Max fbarriers/Workgrp:   32
  Packet Processor uCode:: 471
  SDMA engine uCode::      40
  IOMMU Support::          None
  Pool Info:
    Pool 1
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED
      Size:                    30809256(0x1d61ca8) KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Alignment:         4KB
      Accessible by all:       FALSE
    Pool 2
      Segment:                 GLOBAL; FLAGS:
      Size:                    30809256(0x1d61ca8) KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Alignment:         4KB
      Accessible by all:       FALSE
    Pool 3
      Segment:                 GROUP
      Size:                    64(0x40) KB
      Allocatable:             FALSE
      Alloc Granule:           0KB
      Alloc Alignment:         0KB
      Accessible by all:       FALSE
  ISA Info:
    ISA 1
      Name:                    amdgcn-amd-amdhsa--gfx90c:xnack+
      Machine Models:          HSA_MACHINE_MODEL_LARGE
      Profiles:                HSA_PROFILE_BASE
      Default Rounding Mode:   NEAR
      Default Rounding Mode:   NEAR
      Fast f16:                TRUE
      Workgroup Max Size:      1024(0x400)
      Workgroup Max Size per Dimension:
        x                        1024(0x400)
        y                        1024(0x400)
        z                        1024(0x400)
      Grid Max Size:           4294967295(0xffffffff)
      Grid Max Size per Dimension:
        x                        4294967295(0xffffffff)
        y                        4294967295(0xffffffff)
        z                        4294967295(0xffffffff)
      FBarrier Max Size:       32
*** Done ***

Ubuntu server 24.04 + 6.12.12-x64v3-xanmod1 kernel

[NickM-27]

You're trying to use a dev build which we do not provide support for. In this case you are using a feature that is not currently supported until we can build onnxruntime for rocm with python3.11

[gonzalezcalleja]

Great! I will downgrade to 0.15 and test it.

Thanks for your support.

[gonzalezcalleja]

After downgrade to 0.15.0-rocm, i'm getting this error message, with same config:

frigate  | 2025-02-12 11:50:38.284118193  [ERROR:[email protected]] global cap.cpp:164 open VIDEOIO(CV_IMAGES): raised OpenCV exception:
frigate  | 2025-02-12 11:50:38.284123431
frigate  | 2025-02-12 11:50:38.284126784  OpenCV(4.9.0) /io/opencv/modules/videoio/src/cap_images.cpp:274: error: (-215:Assertion failed) number < max_number in function 'icvExtractPattern'
frigate  | 2025-02-12 11:50:38.284129717

frigate  | 2025-02-12 12:08:44.108272467  rocBLAS error: Cannot read /opt/rocm/lib/rocblas/library/TensileLibrary.dat: Illegal seek for GPU arch : gfx90c
frigate  | 2025-02-12 12:08:44.108298659   List of available TensileLibrary Files :
frigate  | 2025-02-12 12:08:44.108644463  "/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx908.dat"
frigate  | 2025-02-12 12:08:44.109222224  "/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx942.dat"
frigate  | 2025-02-12 12:08:44.109306527  "/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx940.dat"
frigate  | 2025-02-12 12:08:44.109417232  "/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx1101.dat"
frigate  | 2025-02-12 12:08:44.109598481  "/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx906.dat"
frigate  | 2025-02-12 12:08:44.109618107  "/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx1102.dat"
frigate  | 2025-02-12 12:08:44.110100459  "/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx90a.dat"

Testing with reindex: True, don't pass from here:

[NickM-27]

looks like you haven't set the appropriate environment args as described in the documentation
https://docs.frigate.video/configuration/object_detectors#docker-settings-for-gpu-access

[NickM-27]

Also keep in mind that in general ROCm is only recommended for discrete AMD GPUs

[gonzalezcalleja]

I have HSA_OVERRIDE_GFX_VERSION: "9.0.0" as docker environment.

amdgpu_top show frigate.embeddings_manager using a lot of GPU: