←

# llm engineering

// [38] real interview questions. Answers and sources live in the practice app.

practice this topic
  1. Your RAG-powered support assistant confidently gives a wrong answer that isn't in any of your documents. Walk through where a RAG pipeline can fail and how you'd localize the fault.(senior)
  2. How do you choose a chunking strategy for RAG, and what are the tradeoffs of small vs large chunks?(mid)
  3. When would keyword/BM25 retrieval beat embedding-based retrieval, and why is hybrid search common in production RAG?(senior)
  4. How do you know your LLM feature actually works? Design an evaluation approach for an open-ended generation feature before and after shipping.(senior)
  5. You're using LLM-as-a-judge to evaluate outputs. What are the failure modes, and how do you make the judge trustworthy?(senior)
  6. Your LLM agent reads documents fetched from the web and a user's uploaded files. Explain the indirect prompt injection risk and how you defend against it.(senior)
  7. What is hallucination in LLMs, and what concrete mitigations reduce it in a production feature?(mid)
  8. Your downstream code needs to parse JSON from the LLM, but sometimes it returns malformed or extra text. How do you make structured output reliable in production?(mid)
  9. Design the tool-use interface for an LLM agent that can query a database and send emails. What makes tool design succeed or fail?(senior)
  10. You have a large, growing knowledge base to expose through an LLM. Should you fine-tune, use RAG, or rely on long context? How do you decide?(senior)
  11. Your LLM feature feels slow. Break down LLM latency and explain how streaming changes the UX and where it doesn't help.(mid)
  12. Explain the three kinds of LLM caching — exact, prompt/prefix, and semantic — and when each is worth the complexity.(senior)
  13. How does prompt (prefix) caching actually work, and what breaks a cache hit? Give the rules you'd follow to maximize hit rate.(mid)
  14. Your LLM API bill is growing faster than usage. How do you control cost at scale without hurting quality?(senior)
  15. A prompt change that helped one case silently regressed several others in production. How do you version and manage prompts so this doesn't happen?(mid)
  16. Design input and output guardrails for an LLM assistant. What goes on each side and what are the tradeoffs?(senior)
  17. Design an agent that plans and executes multi-step tasks with tools. What are the failure modes and how do you contain them?(senior)
  18. What is a model gateway / router in front of LLM calls, and why do serious LLM products put one there?(mid)
  19. How do you make an LLM feature observable? What do you log and which metrics actually matter?(mid)
  20. Explain the economics of the context window. Why isn't 'just stuff everything into a long context' the answer even when it fits?(senior)
  21. What is a re-ranker in RAG, when do you add one, and what does it cost you?(mid)
  22. How would you evaluate a RAG system specifically, separating retrieval quality from generation quality?(senior)
  23. Your embedding-based retrieval returns plausible but subtly off results. What can go wrong with embeddings and vector search in practice?(mid)
  24. You need to upgrade to a newer/better base model. Why is this risky for an existing LLM product, and how do you do it safely?(senior)
  25. What is the 'composability gap' in multi-step LLM pipelines, and how do you test a chain of LLM steps?(mid)
  26. What are temperature and top-p (sampling parameters), and how do you set them for a factual extraction task vs a creative task?(junior)
  27. How would you defend an LLM feature against jailbreaks and abuse, and what's realistic to expect from those defenses?(senior)
  28. When should you reach for an orchestration/agent framework, and when is it a mistake?(mid)
  29. Design a customer-support chatbot that can read a user's account data and internal docs. Walk the full production design: retrieval, guardrails, actions, fallback.(senior)
  30. How do you build a golden/regression eval set for an LLM feature, and why is it more urgent than in normal software?(mid)
  31. You're deciding whether to self-host an open-weights model or call a hosted API. What actually drives that decision?(senior)
  32. Few-shot examples in your prompt sometimes help and sometimes hurt. How do you use them well and detect when they backfire?(mid)
  33. How do you catch and handle bad LLM outputs at runtime — empty, malformed, toxic, or off-brand — without tanking latency?(senior)
  34. What does an LLM agent's memory look like, and how do short-term (context) and long-term memory differ in a real system?(mid)
  35. An LLM product that worked at launch is quietly getting worse in production over weeks, with no code change. What could cause silent quality drift and how do you catch it?(senior)
  36. How do you estimate and set token/cost budgets for an LLM feature before shipping, and enforce them at runtime?(mid)
  37. When you ask ChatGPT something, the answer appears word by word instead of all at once. Why does it stream like that, and is it just a UI animation?(mid)
  38. With an AI chatbot the first word takes a noticeable beat to appear, then the rest of the answer flows quickly. Why is the first token slow and the rest fast?(senior)