Integrating External Data and User-Provided Context in llama.cpp with LARS

[gnusupport]

I'm excited to share my vision for constructing a Retrieval-Augmented Generation (RAG) system using llama.cpp. I aim for this implementation to be directly integrated, enabling llama.cpp to make additional requests or execute scripts to incorporate external data seamlessly. This capability would greatly enhance the flexibility and responsiveness of the system. That is however for llama.cpp

For LARS:

Currently, I observe that there is a requirement for users to place documents in a specific directory. However, this approach doesn't fit my use case. It feels like a constraint that limits the system's potential. Instead, I envision an external script managing those documents, offering users the choice to request a RAG list prior to receiving the final output from the Large Language Model (LLM). This would allow every user to have their preferred method of supplying documents.

How about just providing a list of file paths? That would be better than having user to move files or create symlinks. Files could remain where they are.

My embeddings are stored in PostgreSQL's pgvector, which has proven to be incredibly fast and efficient. The results are superior compared to searching by name, though PostgreSQL's native text search also performs admirably.

My documents are numerous, encompassing database entries, files, and more, all distributed throughout the file system but organized within the Dynamic Knowledge Repository, as conceptualized by Doug Engelbart. Separating some documents into a specific directory for LARS to process would be a lengthy task. I prefer to generate embeddings effortlessly and maintain this streamlined process.

I'm curious if there's a way for users to supply their own embeddings to this RAG system with LARS, allowing LARS to incorporate user-provided RAG inputs as context, rather than relying solely on its internal functionality. This would open up new possibilities for customization and efficiency in handling diverse data sources.