Foundations

Evaluation interview questions

Evaluation questions are where interviewers find out whether you can connect a metric to a decision. Precision, recall, ROC AUC and log loss each answer a different question, and picking one means knowing what a false positive and a false negative cost.

Expect to be asked what you’d compare a model against (a simple baseline often settles more arguments than a better AUC), how you’d choose a classification threshold, why a model that ranks well can still give badly calibrated probabilities, and which regression metric to report when a few large errors matter more than many small ones.

Try one now

Friendly

An airline's operations team wants a model to predict which flights will leave late. A colleague says a few simple rules would do. How do you decide?

Free, no sign-up. The interviewer asks follow-ups on your reasoning, then rates your answer.

What the 23 questions cover

  • Choosing a classification threshold9
  • Regression metrics (MAE, RMSE, MAPE, R²)6
  • Calibration6
  • Precision and recall5
  • ROC AUC5
  • Baseline models4
  • Precision-recall curve4
  • Confusion matrix2
  • Log loss / cross-entropy2
  • Lift and gain charts2
  • KS statistic2
  • Gini coefficient1

A question can cover more than one topic, so the counts add up to more than 23.

How the practice works

  1. The AI interviewer asks the question and follows up on your answer, the way an interviewer would.
  2. Stuck? Ask for a hint. Each one gives a little more away, so you can get unstuck without being handed the answer.
  3. Get a written debrief. Four ratings, each from Needs work to Interview-ready, tied to what you actually said, and what to study next.

More foundations questions

See all categories