Skip to content
Back to case studies
SanitizedGenAI / AI2024

GenAI RAG Platform

30+ min document hunts → sub-10s answers; hybrid RAG + semantic cache cut LLM spend ~60%.

Reported engagement figures. See the results section for measurement context and limitations.

Role
Senior AWS Solution Architect
Context
Enterprise / SaaS
Duration
5 months
Team
3 engineers + 1 architect (me)

Tech stack

  • AWS Bedrock
  • Claude
  • OpenSearch Serverless
  • Lambda
  • S3
  • ElastiCache
  • Step Functions
  • Python
  • React
  • Terraform
Executive summary

Business problem and production context.

Business problem

Enterprise client had 2M+ internal documents (PDFs, Confluence, Slack threads) spread across systems with no unified search. Engineers spent 30+ minutes finding answers to questions already documented. Traditional keyword search missed semantic context and returned irrelevant results.

Architecture decision

Built an end-to-end RAG pipeline on AWS Bedrock with Claude as the LLM. Implemented document ingestion via Lambda + S3 triggers, chunking with overlap strategies, and vector embeddings stored in Amazon OpenSearch Serverless. Added semantic caching with ElastiCache to reduce repeated LLM calls by 60%. Built a React chat interface with streaming responses and source attribution.

Result

Before and after.

The retrieval, cost, productivity and adoption figures are reported in the engagement account. The benchmark dataset, observation period and attribution method are not included in this public case study. These results should not be read as a forecast for another system.

Time to Find Answerreported
Before30+ minutes
After<10 seconds
LLM Cost vs no cachingreported
Beforebaseline
After~60% lower
Search Relevance (MRR@10)reported
Before0.31 (keyword)
After0.78 (hybrid RAG)
Engineer Productivityreported
BeforeBaseline
After+35% reported; method not published
Architecture

Hybrid retrieval answers internal questions with cited sources and reuses answers to repeated questions.

The ingestion pipeline, hybrid index, semantic cache and streaming chat path. Auth, query-handling compute and monitoring are not shown.

Simplified view

Main components only; supporting services are left out.

  1. 1. Ingestion

    • S3

      Document sources and triggers

    • Lambda connectors

      Incremental sync per source

    • Step Functions

      Runs the chunking pipeline

  2. 2. Index

    • Bedrock (Amazon Titan)

      Generates chunk embeddings

    • OpenSearch Serverless

      Hybrid vector and keyword search

  3. 3. Answering

    • React chat

      Streams answers with sources

    • API Gateway WebSocket

      Carries streamed responses

    • ElastiCache

      Semantic answer cache

    • Bedrock (Claude)

      Writes answers from retrieved chunks

How it works, step by step

  1. 1S3 events trigger Lambda connectors that sync documents from S3, Confluence and Slack exports incrementally.
  2. 2A Step Functions pipeline extracts PDF text, splits it into overlapping chunks and generates Titan embeddings.
  3. 3Chunks and vectors are stored in OpenSearch Serverless, which serves hybrid vector and keyword search.
  4. 4A question from the React chat arrives over a WebSocket API, and a semantically similar earlier question returns its cached answer from ElastiCache.
  5. 5Otherwise Claude on Bedrock answers from the retrieved chunks and streams the response with the source chunks it used.

Guardrails

  • Every answer shows the document chunks it came from
  • Cache entries for a document are flushed when it changes
  • Cache returns an answer only above a high similarity threshold
  • Chunking strategy kept explicit and relevance compared before rollout
  • Vector store kept inside the VPC

Considered and not used

  • Pinecone: OpenSearch Serverless was AWS-native, kept data in the VPC and supported hybrid keyword and vector search.
  • GPT-4: Claude had a longer context window, better instruction following and native Bedrock availability.
What Rahul owned

My part of the work, separate from the team's.

The team behind the engagement is in the facts row above. These are the parts I was responsible for, and the artifacts that show the work.

Owned scope

  • RAG architecture on AWS Bedrock with ingestion, chunking, vector storage, semantic cache, and streaming UI boundaries
  • Retrieval quality and source-attribution path so engineers could trust the answer, not only the generated text
  • Cost-per-query controls using cache hits, context sizing, and repeated-query reduction

Evidence artifacts

Hybrid RAG designOpenSearch Serverless vector storeSource attributionSemantic cacheBefore/after retrieval metrics
Constraints

What would fail first?

Every project has constraints. The useful work is naming them early enough that rollback, cost, and ownership are designed before an incident.

Data volume

2M+ documents across mixed internal sources required incremental ingestion and traceable source chunks.

Trust

Enterprise users needed source attribution before they would rely on AI answers in daily work.

Cost

Repeated questions would have made model usage expensive without semantic caching and context control.

Decision table

Trade-offs accepted on purpose.

OpenSearch Serverless vs Pinecone
Chose OpenSearch for AWS-native integration, no data leaving the VPC, and hybrid keyword+vector search.
Claude vs GPT-4
Claude selected for longer context window, better instruction following, and AWS Bedrock native availability.
Semantic caching vs no caching
Added ElastiCache layer.
60% cache hit rate on repeated queries, cutting LLM costs by around 60%.
Chunking strategy
512-token chunks with 50-token overlap, balanced between retrieval precision and context preservation.
Risk table

Production risks and how they were controlled.

Wrong answer looked confident
Returned source chunks, tracked retrieval quality, and treated unsupported answers as product risk.
AI platform + product owner
Embedding or chunk changes broke relevance
Kept chunking strategy explicit and compared relevance before rollout.
Platform engineer
LLM cost scaled with repeated questions
Added semantic cache and context-size control before exposing broader usage.
Platform + finance
Implementation approach

Small steps, visible changes, fewer surprises.

  1. 01

    Cataloged document sources. 2M+ files across S3 buckets, Confluence spaces, and Slack exports. Built Lambda-based connectors for each source with incremental sync.

  2. 02

    Implemented chunking pipeline using Step Functions: PDF extraction → text splitting (512 tokens, 50-token overlap) → embedding generation via Amazon Titan → vector storage in OpenSearch Serverless.

  3. 03

    Built semantic search layer combining vector similarity search with keyword filtering: a hybrid approach that outperformed pure vector search by 23% on relevance benchmarks.

  4. 04

    Added ElastiCache-based semantic cache layer, comparing query embeddings and returning cached responses for semantically similar questions (cosine similarity > 0.95).

  5. 05

    Deployed React chat interface with streaming responses via WebSocket API Gateway, including source attribution showing exactly which document chunks informed each answer.

Takeaway

What carries over to your system.

  • Chunk size tuning was the single biggest lever for retrieval quality. We tested 256, 512, and 1024 tokens. 512 with 50-token overlap gave the best balance of precision and recall.
  • Semantic caching saved ~60% on LLM costs but required careful invalidation. When source documents were updated, we had to flush related cache entries. We built a document-to-cache mapping for this.
  • Source attribution was non-negotiable for enterprise adoption. Engineers wouldn't trust answers without seeing the exact document and page number. This feature alone drove 80% of user adoption.

Rollback path

Ingestion, retrieval, and answer generation were separable, so a bad retrieval change could be rolled back without disabling the full chat surface.

Cost considerations

Semantic caching and context discipline were the main cost levers; the visible result was about 60% lower repeated LLM spend in the sanitized case story.

Observability notes

Useful signals were retrieval latency, cache hit rate, no-source answers, failed ingestion jobs, and user feedback on source relevance.

Services built on this work

Have a similar production problem?

Send the current architecture, bill concern, deployment pain, or GenAI reliability issue. I will help you find the first real bottleneck.

Scroll or pinch to zoom, drag to move100%