←

# ml systems

// [50] real interview questions. Answers and sources live in the practice app.

practice this topic
  1. For any user on Twitter, how would you suggest who they should follow? What do you do when that user is new, and what are the limitations of data-driven recommender systems?(senior)
  2. You run an e-commerce website. Sometimes users want to buy an item that is no longer available. Build a recommendation system to suggest replacement items.(mid)
  3. How do you build a system to display the top 10 results when a user searches for rental listings in a certain location on Airbnb?(senior)
  4. When you enter a search query on Google, you're shown a list of related searches. How would you generate a list of related searches for each query?(mid)
  5. Each question on Quora often gets many different answers. How do you create a model that ranks all these answers? How computationally intensive is this model?(mid)
  6. When you type a question on StackOverflow, you're shown a list of similar questions to make sure your question hasn't been asked before. How do you build such a system?(mid)
  7. Autocompletion: how would you build an algorithm to finish your sentence when you text?(mid)
  8. Given a dataset of credit card purchases where each record is labelled fraudulent or safe, how would you build a fraud detection system?(junior)
  9. If you were building a Netflix clone, how would you build a system that predicts when a user stops watching a TV show — whether they are tired of the show or just taking a break?(senior)
  10. On social networks like Facebook, users can list their high schools. Can you estimate what percentage of listed high schools are real? How do you find, and deploy at scale, a way of detecting invalid schools?(senior)
  11. Imagine you work on iPhone. Every time users open their phone, you want to suggest the one app they are most likely to open first, with 90% accuracy. How would you approach it?(senior)
  12. An e-commerce company wants to minimize the time it takes customers to purchase their selected items. As a machine learning engineer, what can you do to help?(senior)
  13. How would you design a question answering system that extracts an answer from a large collection of documents given a user query?(mid)
  14. Duolingo wants to recommend increasingly difficult stories to language learners. How would you measure the difficulty level of a story, and how would you edit a story to make it easier or harder?(mid)
  15. Design a newsfeed ranking system for a social network.(senior)
  16. Design a video recommendation system for a platform like YouTube or Netflix.(senior)
  17. Design an ads click prediction (CTR) system.(senior)
  18. Design a harmful content detection system for a social platform.(senior)
  19. Before designing an ML system, how do you decide whether the problem needs ML at all? When does a heuristic beat a model?(mid)
  20. Your new ranking model improves offline metrics (AUC, nDCG) significantly, but the online A/B test shows flat or negative business metrics. How do you investigate, and why does this happen?(senior)
  21. What is training-serving skew, what causes it, and how do you prevent and detect it in a production ML system?(senior)
  22. Your model's production performance is degrading over weeks. How do you detect and handle data distribution shift?(senior)
  23. What is a feature store, when does a team actually need one, and what problems does it solve?(senior)
  24. Batch prediction vs online prediction: how do you choose, and what does a hybrid look like?(junior)
  25. How do you safely deploy a new model version to production? Compare shadow deployment, canary release, and A/B testing.(mid)
  26. In recommendation and feed systems trained on implicit feedback, why is 'the user didn't click' not a negative label, and how do you construct training labels correctly?(mid)
  27. How should you split time-correlated data into train/validation/test sets, and what data leakage traps come up in practice?(junior)
  28. For an LLM-powered product, how do you decide between RAG, fine-tuning, and long-context prompting for injecting knowledge?(senior)
  29. Your LLM-backed feature is too slow and too expensive. How do you optimize serving latency and cost?(senior)
  30. How would you build an evaluation system for an LLM product before and after shipping?(senior)
  31. You're deploying an LLM customer-support chatbot that can access user account data and internal docs. What guardrails and failure handling do you design?(senior)
  32. Design an agentic AI assistant that plans and executes multi-step tasks with tools. What are the failure modes and how do you contain them?(senior)
  33. A feature store sounds clean in theory but teams hit real problems adopting one. What goes wrong in practice, and what do you actually get from it?(senior)
  34. What is a model registry, what does it store beyond the model file, and how does it enable safe rollback and reproducibility?(mid)
  35. You want to test a new model's real-world behavior with zero user risk before serving it. Design a shadow deployment and explain exactly what it can and cannot tell you.(senior)
  36. How do you A/B test a model change correctly? Cover randomization unit, metrics, power, and the traps specific to ML experiments.(senior)
  37. Monitoring fires: your production model's inputs have drifted. Walk through a drift-response playbook — how you triage and what you actually do.(senior)
  38. You need to serve embeddings for hundreds of millions of items with low-latency nearest-neighbor retrieval. What are the hard parts of embeddings at scale?(senior)
  39. Users complain your search/ranking results 'feel worse' but nDCG on your offline eval looks fine. How do you debug ranking metrics and connect them to real quality?(senior)
  40. When labels arrive late or barely at all, how do you monitor and evaluate a production model? Contrast natural labels with delayed/absent ones.(mid)
  41. When should a model retrain — on a schedule, on a trigger, or continually? What decides the cadence, and what can go wrong with retraining too often?(mid)
  42. A recommender optimizing engagement is measurably 'working' but slowly narrowing what users see and amplifying popular items. How do you diagnose and break the feedback loop?(senior)
  43. Why is a two-stage retrieval-then-ranking architecture almost universal in large recommendation and search systems, and what breaks if you skip a stage?(senior)
  44. Your classifier outputs probabilities that feed a downstream decision (thresholding, ranking, an auction bid). It ranks well but the probabilities are wrong. Why does calibration matter and how do you fix it?(mid)
  45. You have a new user or a brand-new item with no interaction history. How do you handle cold-start in a recommendation system, on both the user side and the item side?(senior)
  46. You inherit an ML pipeline and can't reproduce a model that's running in production. What reproducibility practices should have been in place, and how do you retrofit them?(mid)
  47. Compare multi-armed bandits with classic A/B testing for rolling out model or content variants. When is a bandit the better tool and when is it the wrong one?(mid)
  48. How do you decide what to log from a production ML system so that today's traffic becomes tomorrow's training and evaluation data — without poisoning it?(senior)
  49. Shazam recognizes a song in a few seconds in a noisy café, even with people talking over it. How would you design a system that identifies a song from a short, noisy audio clip?(mid)
  50. Your photos app quietly groups every picture of the same person into one album, even people you never tagged. How would you build a system that clusters photos by identity without labels?(senior)