Skip to content

Latest commit

 

History

History
81 lines (64 loc) · 3.57 KB

README.md

File metadata and controls

81 lines (64 loc) · 3.57 KB

Nexa SDK

The Nexa SDK is a comprehensive toolkit for supporting ONNX and GGML models. It supports text generation, image generation, vision-language models (VLM), and text-to-speech (TTS) capabilities. Additionally, it offers an OpenAI-compatible API server with JSON schema mode for function calling and streaming support, and a user-friendly Streamlit UI.

Features

  • Model Support:

    • ONNX & GGML models
    • Conversion Engine
    • Inference Engine:
      • Text Generation
      • Image Generation
      • Vision-Language Models (VLM)
      • Text-to-Speech (TTS)

Detailed API documentation is available here.

  • Server:
    • OpenAI-compatible API
    • JSON schema mode for function calling
    • Streaming support
  • Streamlit UI for interactive model deployment and testing

Installation

For CPU version

pip install nexaai --no-cache-dir

For GPU version

pip install nexaai-gpu --no-cache-dir

Nexa CLI commands

Model Commands

NLP Models

Model Type Format Command
octopus-v2 NLP GGUF nexa gen-text octopus-v2
octopus-v4 NLP GGUF nexa gen-text octopus-v4
tinyllama NLP GGUF nexa gen-text tinyllama
llama2 NLP GGUF/ONNX nexa gen-text llama2
llama3 NLP GGUF/ONNX nexa gen-text llama3
llama3.1 NLP GGUF/ONNX nexa gen-text llama3.1
gemma NLP GGUF/ONNX nexa gen-text gemma
gemma2 NLP GGUF nexa gen-text gemma2
qwen1.5 NLP GGUF nexa gen-text qwen1.5
qwen2 NLP GGUF/ONNX nexa gen-text qwen2
mistral NLP GGUF/ONNX nexa gen-text mistral
codegemma NLP GGUF nexa gen-text codegemma
codellama NLP GGUF nexa gen-text codellama
codeqwen NLP GGUF nexa gen-text codeqwen
deepseek-coder NLP GGUF nexa gen-text deepseek-coder
dolphin-mistral NLP GGUF nexa gen-text dolphin-mistral
nomic-embed-text NLP GGUF nexa gen-text nomic-embed-text
phi2 NLP GGUF nexa gen-text phi2
phi3 NLP GGUF/ONNX nexa gen-text phi3

Multimodal Models

Model Type Format Command
nanollava Multimodal GGUF nexa vlm nanollava
llava-phi3 Multimodal GGUF nexa vlm llava-phi3
llava-llama3 Multimodal GGUF nexa vlm llava-llama3
llava1.6-mistral Multimodal GGUF nexa vlm llava1.6-mistral
llava1.6-vicuna Multimodal GGUF nexa vlm llava1.6-vicuna

Computer Vision Models

Model Type Format Command
stable-diffusion-v1-4 Computer Vision GGUF nexa gen-image sd1-4
stable-diffusion-v1-5 Computer Vision GGUF/ONNX nexa gen-image sd1-5
lcm-dreamshaper Computer Vision GGUF/ONNX nexa gen-image lcm-dreamshaper