Deploying RAG in Bedrock vs. Local: WattBot 2025 Case Study (2026-02-17)

Описание: Many researchers are exploring retrieval-augmented generation (RAG) to build document-grounded, trustworthy AI tools, but it is often unclear how design choices around models, infrastructure, and deployment play out in practice. In this session, we present lessons learned from replicating the winning RAG system from the WattBot 2025 challenge. The challenge focuses on producing citation-backed energy and sustainability estimates for AI workloads from a fixed corpus of 30+ academic papers — or explicitly abstaining when evidence is missing. After a short overview of the winning approach, Nils Matteson and Blaise Enuh walk through how the system is implemented in practice, including:

1. A cloud deployment using AWS Bedrock
2. Local, open-source deployments (e.g., Hugging Face models on GB10 and Dell PowerEdge R7725 hardware)

The session compares performance, cost, latency, and operational tradeoffs across environments. It also includes a Streamlit-based interface demo for those looking to host their own RAG apps.

Presenters
Nils Matteson: CS/DS Undergraduate Student, UW-Madison (https://nilsmatteson.com/)
Blaise Enuh: Research Associate, Great Lakes Bioenergy Research Center, UW-Madison (  / enuhblaisemanga  )
Chris Endemann: Research Cloud Consultant, DoIT, UW-Madison (  / chris-endemann  )

Links
GitHub: https://github.com/matteso1/KohakuRAG...
WattBot Kaggle challenge: https://www.kaggle.com/competitions/W...
Winning solution: https://github.com/KohakuBlueleaf/Koh...
2025 Machine Learning Marathon: https://ml-marathon.wisc.edu/