Abstract
This paper explores how to improve information retrieval for Large Language Models (LLMs) in environments with limited computational resources. Motivated by a real-world use case at Brandeis University’s offices, we built a lightweight Retrieval-Augmented Generation (RAG) system to process large archives of documents. To overcome challenges related to limited context size and the unstructured data, we implement a graph-based RAG approach (GraphRAG) that organizes document chunks and named entities into a knowledge graph. This method improves retrieval accuracy without increasing computational cost. We also propose an evaluation method based on document relevance, avoiding the need for expensive LLM-based validation. Our experiments focused on testing RAG-based information retrieval, notably
without relying on LLMs due to cost limitations. While it is outperformed by classic RAG techniques in terms of recall, our findings suggest that GraphRAG can be an effective and practical solution in environments where low cost, speed, and resource efficiency are prioritized over maximum retrieval completeness. Our results show that in a computationally constrained setting, GraphRAG is more efficient and takes half the time to run.