To main content

Towards Requirements Engineering for RAG Systems

Abstract

This short paper explores how a maritime company develops and integrates large-language models (LLM). Specifically by looking at the requirements engineering for Retrieval Augmented Generation (RAG) systems in expert settings. Through a case study at a maritime service provider, we demonstrate how data scientists face a fundamental tension between user expectations of AI perfection and the correctness of the generated outputs. Our findings reveal that data scientists must identify context-specific "retrieval requirements" through iterative experimentation together with users because they are the ones who can determine correctness. We present an empirical process model describing how data scientists practically elicited these "retrieval requirements" and managed system limitations. This work advances software engineering knowledge by providing insights into the specialized requirements engineering processes for implementing RAG systems in complex domain-specific applications.
Read the publication

Category

Academic chapter

Language

English

Affiliation

  • SINTEF Digital / Software Engineering, Safety and Security
  • Norwegian University of Science and Technology

Year

2025

Publisher

Association for Computing Machinery (ACM)

Book

EASE '25: Proceedings of the 29th International Conference on Evaluation and Assessment in Software Engineering, June 17 to 20, 2025, Istanbul, Turkey

ISBN

9798400713859

Page(s)

782 - 788

View this publication at Norwegian Research Information Repository