GenAI RAG Platform
30+ min document hunts → sub-10s answers; hybrid RAG + semantic cache cut LLM spend ~60%.
Reported engagement figures. See the results section for measurement context and limitations.
- Role
- Senior AWS Solution Architect
- Context
- Enterprise / SaaS
- Duration
- 5 months
- Team
- 3 engineers + 1 architect (me)
Tech stack
- AWS Bedrock
- Claude
- OpenSearch Serverless
- Lambda
- S3
- ElastiCache
- Step Functions
- Python
- React
- Terraform
Business problem and production context.
Business problem
Enterprise client had 2M+ internal documents (PDFs, Confluence, Slack threads) spread across systems with no unified search. Engineers spent 30+ minutes finding answers to questions already documented. Traditional keyword search missed semantic context and returned irrelevant results.
Architecture decision
Built an end-to-end RAG pipeline on AWS Bedrock with Claude as the LLM. Implemented document ingestion via Lambda + S3 triggers, chunking with overlap strategies, and vector embeddings stored in Amazon OpenSearch Serverless. Added semantic caching with ElastiCache to reduce repeated LLM calls by 60%. Built a React chat interface with streaming responses and source attribution.
Before and after.
The retrieval, cost, productivity and adoption figures are reported in the engagement account. The benchmark dataset, observation period and attribution method are not included in this public case study. These results should not be read as a forecast for another system.
Hybrid retrieval answers internal questions with cited sources and reuses answers to repeated questions.
The ingestion pipeline, hybrid index, semantic cache and streaming chat path. Auth, query-handling compute and monitoring are not shown.
Main components only; supporting services are left out.
1. Ingestion
S3
Document sources and triggers
Lambda connectors
Incremental sync per source
Step Functions
Runs the chunking pipeline
2. Index
Bedrock (Amazon Titan)
Generates chunk embeddings
OpenSearch Serverless
Hybrid vector and keyword search
3. Answering
React chat
Streams answers with sources
API Gateway WebSocket
Carries streamed responses
ElastiCache
Semantic answer cache
Bedrock (Claude)
Writes answers from retrieved chunks
How it works, step by step
- 1S3 events trigger Lambda connectors that sync documents from S3, Confluence and Slack exports incrementally.
- 2A Step Functions pipeline extracts PDF text, splits it into overlapping chunks and generates Titan embeddings.
- 3Chunks and vectors are stored in OpenSearch Serverless, which serves hybrid vector and keyword search.
- 4A question from the React chat arrives over a WebSocket API, and a semantically similar earlier question returns its cached answer from ElastiCache.
- 5Otherwise Claude on Bedrock answers from the retrieved chunks and streams the response with the source chunks it used.
Guardrails
- Every answer shows the document chunks it came from
- Cache entries for a document are flushed when it changes
- Cache returns an answer only above a high similarity threshold
- Chunking strategy kept explicit and relevance compared before rollout
- Vector store kept inside the VPC
Considered and not used
- Pinecone: OpenSearch Serverless was AWS-native, kept data in the VPC and supported hybrid keyword and vector search.
- GPT-4: Claude had a longer context window, better instruction following and native Bedrock availability.
My part of the work, separate from the team's.
The team behind the engagement is in the facts row above. These are the parts I was responsible for, and the artifacts that show the work.
Owned scope
- RAG architecture on AWS Bedrock with ingestion, chunking, vector storage, semantic cache, and streaming UI boundaries
- Retrieval quality and source-attribution path so engineers could trust the answer, not only the generated text
- Cost-per-query controls using cache hits, context sizing, and repeated-query reduction
Evidence artifacts
What would fail first?
Every project has constraints. The useful work is naming them early enough that rollback, cost, and ownership are designed before an incident.
Data volume
2M+ documents across mixed internal sources required incremental ingestion and traceable source chunks.
Trust
Enterprise users needed source attribution before they would rely on AI answers in daily work.
Cost
Repeated questions would have made model usage expensive without semantic caching and context control.
Trade-offs accepted on purpose.
Production risks and how they were controlled.
Small steps, visible changes, fewer surprises.
- 01
Cataloged document sources. 2M+ files across S3 buckets, Confluence spaces, and Slack exports. Built Lambda-based connectors for each source with incremental sync.
- 02
Implemented chunking pipeline using Step Functions: PDF extraction → text splitting (512 tokens, 50-token overlap) → embedding generation via Amazon Titan → vector storage in OpenSearch Serverless.
- 03
Built semantic search layer combining vector similarity search with keyword filtering: a hybrid approach that outperformed pure vector search by 23% on relevance benchmarks.
- 04
Added ElastiCache-based semantic cache layer, comparing query embeddings and returning cached responses for semantically similar questions (cosine similarity > 0.95).
- 05
Deployed React chat interface with streaming responses via WebSocket API Gateway, including source attribution showing exactly which document chunks informed each answer.
What carries over to your system.
- Chunk size tuning was the single biggest lever for retrieval quality. We tested 256, 512, and 1024 tokens. 512 with 50-token overlap gave the best balance of precision and recall.
- Semantic caching saved ~60% on LLM costs but required careful invalidation. When source documents were updated, we had to flush related cache entries. We built a document-to-cache mapping for this.
- Source attribution was non-negotiable for enterprise adoption. Engineers wouldn't trust answers without seeing the exact document and page number. This feature alone drove 80% of user adoption.
Rollback path
Ingestion, retrieval, and answer generation were separable, so a bad retrieval change could be rolled back without disabling the full chat surface.
Cost considerations
Semantic caching and context discipline were the main cost levers; the visible result was about 60% lower repeated LLM spend in the sanitized case story.
Observability notes
Useful signals were retrieval latency, cache hit rate, no-source answers, failed ingestion jobs, and user feedback on source relevance.
Further reading
Services built on this work
Have a similar production problem?
Send the current architecture, bill concern, deployment pain, or GenAI reliability issue. I will help you find the first real bottleneck.
Request the reliability audit