← All sessionsHomeSearch
C7 EST | 14 Day AI Sprint·Day 1 | Fundamentals of Gen AI & Deploying Your Own AI Playground·4:40:00

Day 1: APIs, Webhooks, How LLMs Predict, CoStar vs Meta Prompting, Local Models, and a Live RAG Build

David Bajaj Day mentor - data and applied scientist at Microsoft on the Teams Copilot team (~6 years); built 'edu.ai' (2023) which became 'Shiksha Copilot' inside MS Teams; prior IoT at Air India and VMware; claims a published paper on rule-based tree-of-thoughts prompting · Uthappa Host - housekeeping, LMS walkthrough, breakout rooms, reflection/badge process
▶LMS lesson

The short version

  1. Everything you will build - prompts, automations, agents - eventually talks to a model through an API: a request with a unique URL, an authorization key, and a structured body, returning a response. 'Every request is my 1 API call.' Webhooks are the push-based cousin: 'a doorbell' instead of checking the mailbox.
  2. An LLM in three steps: tokenize (rule of thumb 75 words = 100 tokens), embed (numbers that encode meaning and relationships), predict one token at a time by probability - which is why the same prompt gives different wording every run. Vocabulary is finite; sentences are infinite.
  3. Two prompting techniques 'solve 99% of tasks': CoStar (Context, Objective, Style, Tone, Audience, Response - one prompt per task) and meta prompting - a reusable prompt-that-writes-prompts built in three layers from Google's, OpenAI's, and Anthropic's own prompting guides. Rule: never fill more than 50-60% of the context window.
  4. Local/private models are the answer when data cannot leave the building (Goldman, JP Morgan run in-house). Practical floor is 8 GB RAM; a GPU makes it faster, not possible. Pick models with artificialanalysis.ai, compare them in OpenRouter.
  5. RAG fixes two model limits - knowledge cutoff and no access to your documents - without a $3B retrain: upload -> chunk -> embed into a vector store -> retrieve matching chunks -> generate ONLY from them. Live build: a Tesla-manual bot with a cheap classifier in front so off-topic questions never hit the expensive retrieval path.

The concepts

01

API vs webhook: the waiter and the doorbell

You never walk into the kitchen. You tell the waiter, and the food comes back to you. That waiter is the API.

An API (application programming interface) takes a request, validates and authorizes it, and returns a response. Two analogies carry the session: the postal service (the address and stamp are the request structure, the postman is the delivery) and the restaurant (the waiter bridges you and the kitchen). Live against OpenAI's API docs the recurring anatomy is a unique URL, an authorization key, and a request/response body - and every tool you connect (Notion, Telegram, WhatsApp) needs its own key.

A webhook is the 'smarter' push variant. An API is a pull: you keep asking whether the order is ready. A webhook is the system telling you the moment something changes - 'a doorbell', or the parent calling 'food is ready' instead of the child asking every five minutes. Webhooks are what n8n triggers ride on from Day 3 onward.

Why it matters

Prompting, automations, and agents are all 'advanced versions' of this one idea; the Day 5 MCP session extends the same waiter analogy to a concierge.

This idea elsewhereextendsDileep reuses the waiter analogy: API = you check, webhook = they call, MCP = a concierge handles the whole exchange
Go deeper

In one line: API = on-demand request/response through a URL + auth key + body (pull); webhook = the remote system pushes a notification to you when an event happens.

API anatomy: unique URL, authorization/API key, structured request and response (l3186007 0:52)

Every integrated tool needs its own API key (l3186007 0:45)

'Every request is my 1 API call' - the unit that billing counts (l3186007 0:59)

API = pull (you check); webhook = push (it notifies you) (l3186007 0:56-0:57)

Webhooks are the trigger mechanism for the automation sessions (l3186007 0:55)

▶ Watch this taught:

02

Tokenize, embed, predict: why the same prompt never answers the same way

Run the identical prompt twice and read two different answers. Nothing is broken - you are watching probability.

Step one, tokenization: words are broken into tokens; the working rule is 75 words = 100 tokens (his 'explain LLM to a 5-year-old' prompt was 11 tokens). Vocabulary is finite (~170,000 English words) but sentence combinations are infinite - 'my training data can be infinite... but my words are always finite.' Step two, embeddings: each token becomes a vector of numbers encoding meaning and relationships, so 'apple' lights up Macintosh and fruit, 'democratic' lights up parliamentary and socialist - shown live in an embedding visualizer.

Step three, prediction: the model emits the single most probable next token, then the next, so wording varies run to run while the theme holds. Two useful corollaries: pricing everywhere is per token consumed, and there are two kinds of AI user - model builders (Microsoft training Phi) and application builders using someone else's model, which is who this course is for.

Why it matters

Token math explains cost, context-window limits, and the 50-60% rule in meta prompting; prediction explains why evals (later in Paul's KB) measure a spread not a value.

This idea elsewhereprerequisiteThe Evals sprint's 'system that does not give the same answer twice' is this prediction step seen from the testing side
Go deeper

In one line: LLM = tokenizer + embedding space + next-token predictor; output is sampled by probability, so variation is intrinsic, and cost scales with tokens.

Rule of thumb: 75 words ~ 100 tokens (l3186007 1:06)

Embeddings are numeric vectors that encode meaning and relationships between words (l3186007 1:11-1:12)

Prediction is one token at a time by probability - hence run-to-run variability (l3186007 1:13)

Finite vocabulary (~170k words), infinite sentences (l3186007 1:10)

AI product pricing is measured in tokens per session (l3186007 1:06)

▶ Watch this taught:

03

CoStar: one structured prompt per task

Screenshot the framework, hand it to the model with a one-line ask, and let it write the prompt for you.

CoStar breaks a prompt into Context, Objective, Style, Tone, Audience, and Response format (he groups the letters loosely while teaching - context, objective, steps, tools, actions, reflection). The live build is a fitness coach 'Ria' for a fictional 'Healthify You' app, modeled on HealthifyMe's real bot: give the model the CoStar image plus minimal instructions and it expands a full prompt - weekly plans as tables, and a reflection step that makes the model verify details before answering ('quick and simple and effective. All 3').

The limitation is the point of the next concept: a CoStar prompt is single-use. New task, new prompt from scratch.

Why it matters

It is the fastest reliable prompt structure for a one-off job, and the reflection/self-check line is a cheap guardrail worth copying into any prompt.

Go deeper

In one line: CoStar = Context, Objective, Style, Tone, Audience, Response - a per-task prompt skeleton, best produced by having the model expand it from a minimal brief.

Six parts: Context, Objective, Style, Tone, Audience, Response format (l3186007 1:21)

Workflow: upload the framework image, give a minimal ask, let the model expand it (l3186007 1:24-1:26)

Specify output format explicitly - e.g. weekly plans as tables (l3186007 1:27)

Add a reflection step: verify details internally before the final answer (l3186007 1:28)

Limitation: one CoStar prompt per use case (l3186007 1:30)

▶ Watch this taught:

Check yourself

Answer from memory first — the recall attempt is what makes it stick. Then reveal.

You need the same assistant behaviour for ten different departments. CoStar or meta prompt?

Meta prompt - CoStar would mean ten hand-written prompts; a meta prompt is a generator you point at each department with one line.

04

Meta prompting: a prompt that writes prompts, built from the labs' own guides

Instead of asking how to make pasta, tell the model what is in your fridge and let it decide the recipe. That is the difference between a prompt and a meta prompt.

Meta prompting is 'designing prompts that think how to think': a generic generator you reuse for any task. Built live in Google AI Studio (free; Gemini 3 Flash) in three layers. Layer 1: upload Google's Prompt Engineering whitepaper (~38k of a ~1M-token window) and ask for a generic meta-prompt structure. Layer 2: feed the GPT-5 prompting-best-practices URL (URL-context toggle on) to fold in OpenAI's techniques such as stop conditions. Layer 3: feed Anthropic's guide, which adds XML tagging. The result is a 'master prompt' - 'we have now covered the big 3 of AI.'

Use: pair the master prompt with a one-line task ('act as a market researcher') and it asks clarifying questions before drafting the final prompt. Hard rule: keep the conversation under 50-60% of the context window or 'context will deviate.' Model-agnostic - AI Studio was chosen for cost, not necessity.

Why it matters

This is the reusable asset from Day 1 - every later mentor says 'use your meta prompt' when a learner asks for 'the best prompt' for a tool.

People get this wrong

A bigger context window means you should put more in.

Past roughly half the window answers drift; the window is headroom, not a target.

Go deeper

In one line: Meta prompt = reusable prompt generator synthesized from vendor prompting guides (Google structure + OpenAI stop conditions + Anthropic XML tags); apply with a one-line task; stay under ~60% of context.

Recipe-generator analogy: generic builder vs single-recipe CoStar (l3186007 1:32-1:33)

Layer 1: Google prompting whitepaper -> generic structure (l3186007 1:39-1:42)

Layer 2: GPT-5 best-practices URL via URL-context toggle (l3186007 1:44-1:45)

Layer 3: Anthropic guide adds XML-tag convention (l3186007 1:47-1:50)

Never exceed 50-60% of the context window (l3186007 1:41, 1:49)

Result asks clarifying questions before producing the task prompt (l3186007 1:52-1:58)

▶ Watch this taught:

05

Local LLMs: when data can't leave, and what hardware actually works

Goldman and JP Morgan do not paste into ChatGPT. Neither should your client's contracts.

Cloud models send data off-device; regulated firms run models in-house. For an individual the equivalent is an open-weights model on your laptop: offline use, unlimited free tokens, confidential documents queried locally - 'completely private... No 1 can see.' Hardware: 4 GB RAM is 'really difficult' even for ~1B-parameter models; 8 GB is the practical minimum; a GPU raises tokens/second but is not required. His MacBook (M3, 16 GB) ran Phi-4 Mini (~4.4 GB) and a DeepSeek reasoning model (~2 GB) at ~12 tokens/s in LM Studio.

Tools: Msty (hybrid local + cloud, down that night), LM Studio (local-only, his preference), GPT4All and Ollama mentioned. Choosing a model: OpenRouter lists 606 models and can run several side by side on one prompt; artificialanalysis.ai benchmarks intelligence, speed, cost, coding, and context window - at recording GPT-5.2 led intelligence, Claude was priciest, Grok 4.1 claimed a ~2M-token window.

Why it matters

The Msty/Ollama walkthrough was promised again later because the live demo failed - a recurring thread through the office-hours days.

For your projects

Paul's Hermes/Ollama experiments in the Catalyst OH XII record are the grown-up version of this - same privacy argument, agent on top.

Go deeper

In one line: Run open-weights models locally (LM Studio / Msty / Ollama) for privacy and zero marginal cost; 8 GB+ RAM floor; GPU optional; pick models via artificialanalysis.ai and test via OpenRouter.

Privacy driver: regulated firms (Goldman Sachs, JP Morgan) run in-house LLMs (l3186007 2:01-2:02)

Under 4 GB RAM impractical; 8 GB+ minimum; GPU speeds but is not required (l3186007 2:04)

LM Studio is his pick; Msty hybrid was down; GPT4All and Ollama mentioned (l3186007 2:05-2:14)

Live: Phi-4 Mini and DeepSeek reasoning model at ~12 tok/s on a 16 GB M3 (l3186007 2:15-2:19)

OpenRouter: 606 models, parallel comparison; artificialanalysis.ai for benchmarks (l3186007 2:21-2:27)

▶ Watch this taught:

06

RAG end to end, with a cheap classifier gate (Tesla-manual build in Dify)

how-to

Retraining ChatGPT costs about three billion dollars. Attaching your PDF to it costs nothing - that is RAG.

Two model limits motivate RAG: knowledge cutoff (a live query showed two chats disagreeing on their cutoff) and no access to private documents. The pipeline: upload the knowledge base (PDF/CSV/site) -> chunk it (the model cannot swallow a whole document) -> embed chunks into a vector store -> at query time retrieve the matching chunks -> generate an answer ONLY from them - 'if the response is present... in the data, then only your model is responding.' Uploading a file to ChatGPT, and even its web search, are RAG in disguise.

The Dify build: the 'question-answer classifier' template with two branches. A cheap model (GPT-3.5) first decides whether the question is Tesla-related. Yes -> knowledge retrieval over the 306-page Cybertruck manual -> answer. No ('price of iPhone 17') -> canned refusal, never touching the expensive path. Use cases listed: support/FAQ bots, legal assistants, medical-record Q&A, study helpers, product policy bots.

Do it in this order
Why it matters

The gate-then-retrieve shape reappears in Day 4's n8n support agent (classifier -> switch -> agent) and is the cheapest reliability lever in any RAG bot.

This idea elsewhereextendsDay 4 rebuilds the same gate-then-retrieve shape in n8n with Pinecone and a real Gmail inbox
Go deeper

In one line: RAG = knowledge base -> chunk -> embed -> retrieve -> generate-from-retrieved-only; put a cheap classifier in front so irrelevant questions never reach retrieval.

RAG solves knowledge cutoff and private-document access without retraining (l3186007 2:36-2:38)

Pipeline: upload -> chunk -> embed -> retrieve -> generate from retrieved context only (l3186007 2:44-2:51)

Retraining a frontier model cited at ~$3B; RAG attaches data at inference time instead (l3186007 2:39-2:40)

File upload and web search in ChatGPT are themselves RAG (l3186007 2:43, 2:47-2:48)

Cheap classifier gate before retrieval saves cost and blocks off-topic queries (l3186007 2:56-3:01)

▶ Watch this taught:

Every concept, three clicks deep

The same concepts as a quick reference: the closed row is the glance, open is the study card, and every timestamp jumps into the recording.

01API vs webhook: the waiter and the doorbellAPI = on-demand request/response through a URL + auth key + body (pull);›

API = on-demand request/response through a URL + auth key + body (pull); webhook = the remote system pushes a notification to you when an event happens.

API anatomy: unique URL, authorization/API key, structured request and response (l3186007 0:52)

Every integrated tool needs its own API key (l3186007 0:45)

'Every request is my 1 API call' - the unit that billing counts (l3186007 0:59)

API = pull (you check); webhook = push (it notifies you) (l3186007 0:56-0:57)

Webhooks are the trigger mechanism for the automation sessions (l3186007 0:55)

02Tokenize, embed, predict: why the same prompt never answers the same wayLLM = tokenizer + embedding space + next-token predictor;›

LLM = tokenizer + embedding space + next-token predictor; output is sampled by probability, so variation is intrinsic, and cost scales with tokens.

Rule of thumb: 75 words ~ 100 tokens (l3186007 1:06)

Embeddings are numeric vectors that encode meaning and relationships between words (l3186007 1:11-1:12)

Prediction is one token at a time by probability - hence run-to-run variability (l3186007 1:13)

Finite vocabulary (~170k words), infinite sentences (l3186007 1:10)

AI product pricing is measured in tokens per session (l3186007 1:06)

03CoStar: one structured prompt per taskCoStar = Context, Objective, Style, Tone, Audience, Response - a per-task prompt skeleton, best produced by…›

CoStar = Context, Objective, Style, Tone, Audience, Response - a per-task prompt skeleton, best produced by having the model expand it from a minimal brief.

Six parts: Context, Objective, Style, Tone, Audience, Response format (l3186007 1:21)

Workflow: upload the framework image, give a minimal ask, let the model expand it (l3186007 1:24-1:26)

Specify output format explicitly - e.g. weekly plans as tables (l3186007 1:27)

Add a reflection step: verify details internally before the final answer (l3186007 1:28)

Limitation: one CoStar prompt per use case (l3186007 1:30)

04Meta prompting: a prompt that writes prompts, built from the labs' own guidesMeta prompt = reusable prompt generator synthesized from vendor prompting guides (Google structure + OpenAI…›

Meta prompt = reusable prompt generator synthesized from vendor prompting guides (Google structure + OpenAI stop conditions + Anthropic XML tags); apply with a one-line task; stay under ~60% of context.

Recipe-generator analogy: generic builder vs single-recipe CoStar (l3186007 1:32-1:33)

Layer 1: Google prompting whitepaper -> generic structure (l3186007 1:39-1:42)

Layer 2: GPT-5 best-practices URL via URL-context toggle (l3186007 1:44-1:45)

Layer 3: Anthropic guide adds XML-tag convention (l3186007 1:47-1:50)

Never exceed 50-60% of the context window (l3186007 1:41, 1:49)

Result asks clarifying questions before producing the task prompt (l3186007 1:52-1:58)

05Local LLMs: when data can't leave, and what hardware actually worksRun open-weights models locally (LM Studio / Msty / Ollama) for privacy and zero marginal cost;›

Run open-weights models locally (LM Studio / Msty / Ollama) for privacy and zero marginal cost; 8 GB+ RAM floor; GPU optional; pick models via artificialanalysis.ai and test via OpenRouter.

Privacy driver: regulated firms (Goldman Sachs, JP Morgan) run in-house LLMs (l3186007 2:01-2:02)

Under 4 GB RAM impractical; 8 GB+ minimum; GPU speeds but is not required (l3186007 2:04)

LM Studio is his pick; Msty hybrid was down; GPT4All and Ollama mentioned (l3186007 2:05-2:14)

Live: Phi-4 Mini and DeepSeek reasoning model at ~12 tok/s on a 16 GB M3 (l3186007 2:15-2:19)

OpenRouter: 606 models, parallel comparison; artificialanalysis.ai for benchmarks (l3186007 2:21-2:27)

06RAG end to end, with a cheap classifier gate (Tesla-manual build in Dify)RAG = knowledge base -> chunk -> embed -> retrieve -> generate-from-retrieved-only;›

RAG = knowledge base -> chunk -> embed -> retrieve -> generate-from-retrieved-only; put a cheap classifier in front so irrelevant questions never reach retrieval.

RAG solves knowledge cutoff and private-document access without retraining (l3186007 2:36-2:38)

Pipeline: upload -> chunk -> embed -> retrieve -> generate from retrieved context only (l3186007 2:44-2:51)

Retraining a frontier model cited at ~$3B; RAG attaches data at inference time instead (l3186007 2:39-2:40)

File upload and web search in ChatGPT are themselves RAG (l3186007 2:43, 2:47-2:48)

Cheap classifier gate before retrieval saves cost and blocks off-topic queries (l3186007 2:56-3:01)

Tools referenced

ToolCoverageMomentContext
Google AI StudiodemonstratedFree meta-prompting workbench; Gemini 3 Flash; URL-context toggle
ChatGPTdemonstratedToken counts, embeddings, prediction variability demos
MstydemonstratedHybrid local+cloud runner; server down during the demo, promised again later
LM StudiodemonstratedLocal-only runner, his preference; Phi-4 Mini and DeepSeek live
OpenRouterdemonstrated606 models; run several in parallel to compare
Artificial AnalysisdemonstratedBenchmark site for intelligence/speed/cost/context choice
DifydemonstratedRAG chatbot build with question-answer classifier template
Phi-4demonstratedMicrosoft open-weights model run locally
DeepSeekdemonstratedReasoning model run locally in LM Studio
OpenAI APIexplainedAPI docs used to show URL + key + body anatomy; Playground needs paid credits
GPT4AllmentionedAlternative local chat app
OllamamentionedNamed, not demoed

Action items

    Extraction notes

    This page was built from an auto-generated transcript, which garbles product and people's names. Those were corrected silently in everything above and logged here for transparency. The warnings flag claims that were true on the recording day but change fast.

    Transcript corrections applied

    The transcript saysThe trainer actually means
    Divi / Diffie / V rackDify (dify.ai)
    Misty Studio / Misty AIMsty
    5 4 / Phi 4Microsoft Phi-4 / Phi-4 Mini
    GurokGrok
    EntropicAnthropic
    byte piece / sentence pieceByte-Pair Encoding and SentencePiece tokenizers
    Healthy find meHealthifyMe (real app); 'Healthify You' is his fictional example

    True on recording day — verify before relying