Add genai backend for ollama to run generative AI models using OpenVINO Runtime.

mysteries about Hardware in software engineer's eyes

This is a simple demonstration of more advanced, agentic patterns built on top of the Realtime API.

MiniCPM-o 2.6: A GPT-4o Level MLLM for Vision, Speech and Multimodal Live Streaming on Your Phone

Using OpenVINO to speed up MeloTTS inference

Using OpenVINO to speed up moondream2 inference

🐸💬 - a deep learning toolkit for Text-to-Speech, battle-tested in research and production

🤗 Optimum Intel: Accelerate inference with Intel optimization tools

🤱🏻 Turn any webpage into a desktop app with Rust. 🤱🏻 利用 Rust 轻松构建轻量级多端桌面应用

llm deploy project based mnn. This project has merged into MNN.

Triton backend that enables pre-process, post-processing and other logic to be implemented in Python.

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.

BladeDISC is an end-to-end DynamIc Shape Compiler project for machine learning workloads.

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.

AutoKernel 是一个简单易用，低门槛的自动算子优化工具，提高深度学习算法部署效率。

The Triton backend for TensorRT.

Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.

Build Container Images In Kubernetes

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

OpenVINO backend for Triton.

Representation and Reference Lowering of ONNX Models in MLIR Compiler Infrastructure