AI and ML interview questions for experienced candidates

Twelve questions for experienced candidates

1. Your model's offline metrics are great but production performance is poor. What do you check?
2. How do you handle a heavily imbalanced dataset?
3. How do you detect and prevent data leakage?
4. How do you design an evaluation that reflects business value?
5. How would you run an A/B test for a new model?
6. When would you choose a gradient-boosted tree over a neural network?
7. A stakeholder wants to replace your classifier with an LLM. How do you decide?
8. How do you reduce inference cost by half without hurting quality much?
9. How do you monitor a model in production?
10. Tell me about a model you decided not to ship.
11. How do you prioritise between improving the model and improving the data?
12. How have you mentored or led other ML practitioners?

Offline good, production bad

Work through the usual suspects in order: training–serving skew (features computed differently), leakage (training used information unavailable at prediction time), distribution shift (production traffic differs from the training sample), label delay or definition mismatch (what you measured offline isn't what the business measures), and feedback loops. Compare feature distributions between training and live data, replay production inputs through the offline pipeline, and check the metric definition with stakeholders. See MLOps interview questions.

Imbalance and leakage

• Imbalance: choose metrics that reflect it (precision–recall curves, recall at a fixed precision), adjust class weights or resample the training set only, tune the decision threshold to business costs, and collect more minority examples where possible.
• Leakage: split by time or by entity rather than randomly when data is temporal or grouped; audit features for anything recorded after the prediction moment; be suspicious of any single feature that's dramatically predictive.

Model choice and the LLM question

For “should we replace this with an LLM?”, compare on the same evaluation set: quality, latency, cost per prediction at expected volume, failure modes, and operational risk. LLMs often win on flexibility and cold start; small specialised models often win on cost and predictability at scale. A hybrid — the LLM labels data to train a small model — is a common answer interviewers like. See generative AI interview questions.

Cost and experimentation

Halving inference cost: quantisation, distillation into a smaller model, batching, caching repeated requests, routing easy cases to a cheaper model, and pruning unused features that need expensive lookups. For A/B tests, define the primary metric and guardrails beforehand, size the test for the effect you care about, randomise at the right unit (user rather than request if users see multiple predictions), and watch for novelty effects.

Project deep dives

Expect 20 to 30 minutes on one project. Prepare the business problem, your specific role, the data challenges, the modelling choices and alternatives you rejected, how you evaluated and launched it, the measured impact, and what went wrong. “A model I decided not to ship” is a favourite because it tests judgement; have a real example where evaluation or risk changed your mind.

Practising

Reading answers is not the same as saying them. Pick ten of the questions above and answer them aloud against a timer; a free AI interview assistant like CoPilot Interview can ask senior ML questions in a mock interview and flag answers that ran long or lacked an example. Rehearse two project deep dives until you can handle five levels of “why?” on each. See also machine learning engineer interview questions.

Disclosure: CoPilot Interview is our product, so weigh what we say about it accordingly. Everything else on this page applies whichever tool you use, or none.

Frequently asked questions

What ML questions are asked to experienced candidates?

Scenario questions on production failures, imbalanced data, leakage, evaluation design, A/B testing, model selection, LLM adoption and cost, plus deep dives into projects you led and examples of mentoring.

How do you debug a model that works offline but not in production?

Check for training–serving skew, data leakage, distribution shift, metric or label mismatches and feedback loops, by comparing live and training feature distributions and replaying production inputs offline.

When should you use an LLM instead of a traditional model?

When the task is open-ended language work with little labelled data or needs flexibility. For high-volume narrow tasks with labels and tight cost or latency budgets, a smaller specialised model is often better.

How do I prepare for a senior ML project deep dive?

Prepare the business problem, your role, data challenges, choices and rejected alternatives, evaluation, launch, measured impact and failures — and practise answering repeated follow-up questions on each.