[🏆Outstanding Paper Award at ACL 2024] MMToM-QA: Multimodal Theory of Mind Question Answering

LAVIS - A One-stop Library for Language-Vision Intelligence

The official repo for "SpatialBot: Precise Spatial Understanding with Vision Language Models.

[ECCV 2024] Official implementation of the paper "Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection"

API for Grounding DINO 1.5: IDEA Research's Most Capable Open-World Object Detection Model Series

Awesome-LLM: a curated list of Large Language Model

code for "Strengthening Multimodal Large Language Model with Bootstrapped Preference Optimization"

A brief introduction to the quaternions and its applications in 3D geometry.

Mathematical Visual Instruction Tuning for Multi-modal Large Language Models

GRUtopia: Dream General Robots in a City at Scale

[ICML 2024] 3D-VLA: A 3D Vision-Language-Action Generative World Model

Cambrian-1 is a family of multimodal LLMs with a vision-centric design.

PyTorch Implementation of "V* : Guided Visual Search as a Core Mechanism in Multimodal LLMs"

😎 up-to-date & curated list of awesome 3D Visual Grounding papers, methods & resources.

Official Repo for Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning

Kolmogorov Arnold Networks

OpenEQA Embodied Question Answering in the Era of Foundation Models

[ICLR 2024] Fine-tuning LLaMA to follow Instructions within 1 Hour and 1.2M Parameters

[CVPR 2024 Oral] InternVL Family: A Pioneering Open-Source Alternative to GPT-4o. 接近GPT-4o表现的开源多模态对话模型

Code for "Biological Sequence Design with GFlowNets", 2022

Awesome-LLM-3D: a curated list of Multi-modal Large Language Model in 3D world Resources