The enterprise RAG thesis: data, computation, and LLM on the client's own server
'Datalab was giving us 25% discount in exchange for my PDF — for my data that might be proprietary.' The discount toggle from yesterday's session becomes today's reason to build.
Perplexity and NotebookLM already exist, so the differentiated use case is the one they structurally can't serve: an enterprise whose documents must never train someone else's model. The whole architecture follows from that constraint — 'host this on your client's server, where the data, the computation, and the LLM all sit inside your client's very own server.' OpenAI appears in the demo purely as a stand-in ('my Mac does not support bigger models'); Ollama is designed in as the first-class interchangeable provider from the first spec message, 'not an afterthought.'
The business model rides along: sell the same repurposable codebase to clients per vertical ('RAG is a RAG — there's just gonna be a very simple change in a system prompt'), and monetize the self-hosted deploy as a retainer/monthly-maintenance contract since there's no per-seat SaaS to bill. Today's vertical, crowd-chosen: HR/talent management — 'internal Perplexity for an HR company' — with candidate data as exactly the kind of PII that justifies the on-prem shape.
The use-case brainstorm itself: fintech invoices, legal/insurance, hospital records, HR — each pitched by the cohort, each valued precisely because the data is proprietary.
It's the clearest statement in the course of when NOT to use hosted AI — and the counterpart to the sprint's open-source doctrine: privacy as the buying reason.
Enterprise AI means a bigger OpenAI contract.
For proprietary-data clients it often means the opposite — no hosted API at all; open models on their own hardware, with the hosted key used only to demo.
Datalab was giving us 25% discount in exchange for my data that might be proprietary.
RAG is a RAG. There's just gonna be a very simple change in a system prompt.
What they're able to visually see can be our moat.
Maps onto your standing rule set: this is the client conversation where 'where does the data live' decides the architecture before any feature does.
Go deeper
In one line: Enterprise RAG = retrieval system where documents, vector store, and inference all run inside the client's infrastructure (Ollama or equivalent local LLM); hosted APIs allowed only as demo scaffolding; sold as build + retainer, repurposed across verticals by swapping the system prompt.
Rajesh's live challenge — 'if OpenAI is used, how is this enterprise?' — answered directly: OpenAI is demo-only, Ollama is the production path ()
Chat-only vs workspace-library upload semantics taught as a product decision: per-conversation ephemeral vs persistent knowledge base ()
Ravindra's RBAC caveat accepted: a shared workspace library needs role-based access control, deferred as out of MVP scope ()
'What they're able to visually see can be our moat' — the interface, not the RAG, is the defensible part, since the RAG is open source ()
▶ Watch this taught:
Answer from memory first — the recall attempt is what makes it stick. Then reveal.
Why is Ollama in the spec from message one rather than added later?
Because provider-swappability is the product's entire enterprise claim; retrofitting it after building around OpenAI's shapes would rework every seam. 'First-class interchangeable provider, not an afterthought.'
