Awesome-LLM-RAG

This repo aims to record advanced papers of Retrieval Agumented Generation (RAG) in LLMs.

We strongly encourage the researchers that want to promote their fantastic work to the LLM RAG to make pull request to update their paper's information!

---

Contents

• Resources
  • Workshops and Tutorials
• Papers
  • Survey and Benchmark
  • Retrieval-enhanced LLMs
  • RAG Instruction Tuning
  • RAG In-Context Learning
  • RAG Embeddings
  • RAG Search
  • RAG Long-text and Memory

---

Resources

Workshops and Tutorials

Personalized Generative AI
Zheng Chen, Ziyan Jiang, Fan Yang, Zhankui He, Yupeng Hou, Eunah Cho, Julian McAuley, Aram Galstyan, Xiaohua Hu, Jie Yang
CIKM 23 – Oct 2023 [link]

First Workshop on Recommendation with Generative Models
Wenjie Wang, Yong Liu, Yang Zhang, Weiwen Liu, Fuli Feng, Xiangnan He, Aixin Sun
CIKM 23 – Oct 2023 [link]

First Workshop on Generative Information Retrieval
Gabriel Bénédict, Ruqing Zhang, Donald Metzler
SIGIR 23 – Jul 2023 [link]

Retrieval-based Language Models and Applications
Akari Asai, Sewon Min, Zexuan Zhong, Danqi Chen
ACL 23 – Jul 2023 [link]

Papers

Survey and Benchmark

Benchmarking Large Language Models in Retrieval-Augmented Generation
Jiawei Chen, Hongyu Lin, Xianpei Han, Le Sun
arXiv 2023. [Paper][Github]
4 Sep 2023

Retrieval-enhanced LLMs

Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models
Wenhao Yu, Hongming Zhang, Xiaoman Pan, Kaixin Ma, Hongwei Wang, Dong Yu
arxiv - Nov 2023 [Paper]

REST: Retrieval-Based Speculative Decoding
Zhenyu He, Zexuan Zhong, Tianle Cai, Jason D Lee, Di He
arXiv - Nov 2023 [Paper][Github]

Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection
Anonymous
ICLR 24 – October 23 [paper]

Retrieval meets Long Context Large Language Models
Anonymous
ICLR 24 – October 23 [paper]

DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines
Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, Christopher Potts
arXiv – October 2023 [paper] [code]

Active Retrieval Augmented Generation
Zhengbao Jiang, Frank F. Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, Graham Neubig
arXiv – May 2023 [paper] [code]

REPLUG: Retrieval-Augmented Black-Box Language Models
Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, Wen-tau Yih
arXiv – Jan 2023 [paper]

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, Douwe Kiela NeurIPS 2020 - May 2020 [Paper]

RAG Instruction Tuning

RA-DIT: Retrieval-Augmented Dual Instruction Tuning
Anonymous
ICLR 24 – October 23 [paper]

InstructRetro: Instruction Tuning post Retrieval-Augmented Pretraining
Boxin Wang, Wei Ping, Lawrence McAfee, Peng Xu, Bo Li, Mohammad Shoeybi, Bryan Catanzaro
arXiv - October 23 [paper]

RAG In-Context Learning

In-Context Retrieval-Augmented Language Models
Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, Yoav Shoham
AI21 Labs – Jan 2023 [paper] [code]

RAG Embeddings

Text Embeddings Reveal (Almost) As Much As Text
John X. Morris, Volodymyr Kuleshov, Vitaly Shmatikov, Alexander M. Rush
EMNLP 2023 - Oct 2023 [Paper][Github]

Jina Embeddings 2: 8192-Token General-Purpose Text Embeddings for Long Documents
Michael Günther, Jackmin Ong, Isabelle Mohr, Alaeddine Abdessalem, Tanguy Abel, Mohammad Kalim Akram, Susana Guzman, Georgios Mastrapas, Saba Sturua, Bo Wang, Maximilian Werk, Nan Wang, Han Xiao
arXiv - Oct 2023. [Paper][Model]

RAG Search

RAG Long-text and Memory

Understanding Retrieval Augmentation for Long-Form Question Answering
Hung-Ting Chen, Fangyuan Xu, Shane A. Arora, Eunsol Choi
arXiv 2023 - Oct 2023 [Paper]