Everything about the SmolLM2 and SmolVLM family of models

Evaluate Multimodal LLMs as Embodied Agents

Implementation of Alphafold 3 from Google Deepmind in Pytorch

Code and benchmark for the paper: "A Practitioner's Guide to Continual Multimodal Pretraining" [NeurIPS'24]

🚀 Efficient implementations of state-of-the-art linear attention models in Torch and Triton

[NeurIPS 2024] ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimization

[ICML24] Official Implementation of "ETHER: Efficient Finetuning of Large-Scale Models with Hyperplane Reflections"

Official codebase used to develop Vision Transformer, SigLIP, MLP-Mixer, LiT and more.

Pretrained deep learning models for Jax/Flax: StyleGAN2, GPT2, VGG, ResNet, etc.

Official repository of Evolutionary Optimization of Model Merging Recipes

Accelerating the development of large multimodal models (LMMs) with one-click evaluation module - lmms-eval.

(ECCV 2024) Code for V-IRL: Grounding Virtual Intelligence in Real Life

VLM Evaluation: Benchmark for VLMs, spanning text generation tasks from VQA to Captioning

A flexible and efficient codebase for training visually-conditioned language models (VLMs)

Official repository for "Fantastic Gains and Where to Find Them: On the Existence and Prospect of General Knowledge Transfer between Any Pretrained Model" [ICLR 2024 spotlight]

Unofficial implementation of "SODA: Bottleneck Diffusion Models for Representation Learning"

[ICLR 2024] Official repository for "Vision-by-Language for Training-Free Compositional Image Retrieval"

✨✨Latest Advances on Multimodal Large Language Models

Consistency Distilled Diff VAE

DataComp: In search of the next generation of multimodal datasets

A curated list of plugins that you can add to your FiftyOne install!

Refine high-quality datasets and visual AI models

Python package to download and use the SSB datasets

{KFAC,EKFAC,Diagonal,Implicit} Fisher Matrices and finite width NTKs in PyTorch

This is the repository for the Photorealistic Unreal Graphics (PUG) datasets for representation learning.

Official implementation of "Controlling Text-to-Image Diffusion by Orthogonal Finetuning".

Implementation of Discrete Key / Value Bottleneck, in Pytorch