GenAI: Developing and Deploying a Specialized Underwriting AI Assistant

Описание: Reinsurance companies provide expert medical and underwriting knowledge to insurance companies by distributing Underwriting Manuals. Swiss Re's manual, Life Guide, receives over 23 million hits per year from Life & Health Underwriters (UWs). It provides them with guidance to review a Life or Health insurance application, helping them understand and translate risk into rating adjustments for each individual applicant. Taking the right decision in a timely manner becomes a challenge due to the vast diversity in medical records, history, and profiles of the applicants. In this talk, we present a novel add-on to Life Guide that provides the ability to interact in a human-like fashion, akin to having a specialized assistant by one's side. We cover the major steps needed for creating such a system: from development all the way to productization. To our knowledge it is the first attempt to integrate an advanced chat interface with an underwriting manual in front of UWs for actual decision making.

We start by describing the core of our solution made of a Retrieval-Augmented Generation (RAG) system leveraging state-of-the-art Large Language Model (LLM), namely GPT-4. A crucial first step consists in properly understanding and structuring the Underwriting Manual. This enables us to implement a precise chunking strategy whereby the Manual is split into text chunks and finally embedded into a vector space and stored for later retrieval. We then showcase our information retrieval system used to bring relevant context to the LLM for the final generation of an answer. We give the rationale behind our technical choices and address the specific case of tabular data.

In a second stage, we cover the actual deployment of the solution in front of underwriters. More than the description of a robust production setup, we present our attempts at solving issues specifically related to the use of the RAG technology: adapting to changing LLM versions over time, availability, cost, and latency optimization as well as text chunks management. Besides engineering aspects, we also describe our attempt at creating a trust-inducing User Experience and User Interface despite the hallucinatory nature of LLMs.

From there, we tackle the issue of evaluating our system. We suggest and report on qualitative assessments as well as quantitative metrics gathered from exposing the application to Swiss Re UWs. For instance, we reached a top-5 accuracy of over 90% in the information retrieval step on so-called information pages of the manual.

We finally take a step back to examine the potential impacts of such a system on underwriting practices. We argue that this new interface with Underwriting Manuals has the potential to redefine how UWs interact with them. We also emphasize the importance of mitigating known limitations of LLMs thanks to all available means: from proper evaluation methodologies all the way to User Interfaces and User Experiences.

Speakers:

Louis Douge
Senior Data Scientist
Swiss Re

Robert Simmen
Senior Analytics Manager
Swiss Re