The Cloudcast

RAG That Survives Production

• Massive Studios

Share episode

Adam Kamor (@tonicfakedata, Founder & Head of Engineering) talks about building RAG (Retrieval Augmented Generation) systems in production for AI data. 

SHOW: 992

SHOW TRANSCRIPT: The Cloudcast #992 Transcript

SHOW VIDEO: https://youtube.com/@TheCloudcastNET 

NEW TO CLOUD? CHECK OUT OUR OTHER PODCAST: "CLOUDCAST BASICS" 

SHOW NOTES:

Topic 1 - Adam, welcome to the show. Give everyone a brief introduction.

Topic 2: Our topic today is RAG systems, specifically RAG in production. Let’s start with customization sources and types. When it comes to customizing off-the-shelf LLMs, RAG is one option, as is an MCP connection to a SQL database, and there is pre- and post-training, as well as fine-tuning. How does an organization decide what path is best for customization?

Topic 3 - RAG came on the scene as the savior for organizations that want to use customer AI without the need for fine-tuning and additional training. It has either gone through or is currently still in the trough of disillusionment. What are your thoughts on RAG's evolution and the challenges it faces?

Topic 4 - Let’s walk through the basics of validation. Once you set up RAG, how would an organization know it works? How is accuracy measured and validated? Are you looking for hallucinations? Context quality?

Topic 5 - What is Tonic Validate, and where does it fit into this stack? Is it in band? Out of band? Built into the CI workflow?

Topic 6 - Accuracy is one aspect, but we hear more and more about ROI for Enterprises. How should ROI, risk, and compliance be measured?

Topic 7 - Where and how does security fit into all of this? Also, your thoughts on synthetic data for training vs. real data?


FEEDBACK?

Head Shot

Brian Gracely

Host
Head Shot

Aaron Delp

Co-host