Statistics interview questions for data science and analytics

Fifteen common statistics interview questions

1. What is a p-value, and what is it not?
2. Explain type I and type II errors and statistical power.
3. How do you interpret a 95% confidence interval?
4. What does the central limit theorem say, and why does it matter?
5. When would you use the binomial, Poisson and normal distributions?
6. Explain Bayes' theorem with an example.
7. How do you calculate the sample size for an A/B test?
8. What is the problem with peeking at A/B test results?
9. What is the multiple comparisons problem?
10. What are the assumptions of linear regression?
11. Correlation versus causation — how would you establish causation?
12. What is Simpson's paradox?
13. What is selection bias? Survivorship bias?
14. When would you use a t-test, a chi-square test or a Mann–Whitney test?
15. How do you handle outliers?

Hypothesis testing, precisely

• p-value: the probability of seeing data at least as extreme as observed if the null hypothesis were true. It is not the probability that the null hypothesis is true.
• Type I error: a false positive — rejecting a true null; its rate is the significance level α. Type II error: a false negative; its rate is β.
• Power: 1 − β, the probability of detecting a real effect of a given size; it rises with sample size and effect size.
• Confidence interval: if you repeated the study many times, 95% of intervals built this way would contain the true value. It is not a 95% probability that this particular interval contains it.

Distributions and the CLT

The central limit theorem says the mean of many independent samples is approximately normally distributed regardless of the underlying distribution, given finite variance. That's why z- and t-tests on averages work even for skewed data with enough samples.

Bayes' theorem

A test for a condition affecting 1% of people is 90% sensitive with a 5% false positive rate. Of 1,000 people, 10 have it and 9 test positive; of 990 without it, about 50 test positive. So a positive result means roughly 9 ÷ 59 ≈ 15% chance of having the condition. Interviewers use this to check you account for base rates.

A/B testing pitfalls

• Peeking: checking repeatedly and stopping when significant inflates false positives. Fix the sample size in advance or use sequential methods designed for it.
• Multiple comparisons: testing many metrics or variants guarantees some false positives; pre-register a primary metric and correct (for example, Bonferroni or false discovery rate).
• Wrong randomisation unit: randomise by user, not page view, when users see multiple pages.
• Novelty effects and seasonality: run full weeks and watch trends over time.
• Sample ratio mismatch: if a 50/50 split comes out 52/48, investigate before trusting results.

Causation and paradoxes

Correlation can come from confounders, reverse causation or chance. Establish causation with randomised experiments, or with quasi-experimental methods — difference-in-differences, regression discontinuity, instrumental variables — when experiments aren't possible. Simpson's paradox is when a trend in every subgroup reverses in the aggregate because group sizes differ; always check segment-level results. See data analyst interview questions.

Practising

Interviewers follow up on the most specific thing you say, so rehearse the second question as well as the first. CoPilot Interview, a free AI interview assistant, runs statistics mock interviews with follow-ups and produces a report of likely next-round questions. Practise explaining p-values and confidence intervals to a non-technical listener; it's the most common follow-up. Quant candidates should also see quant interview prep.

Disclosure: CoPilot Interview is our product, so weigh what we say about it accordingly. Everything else on this page applies whichever tool you use, or none.

Frequently asked questions

What statistics questions are asked in data science interviews?

P-values, type I and II errors, power, confidence intervals, the central limit theorem, distributions, Bayes' theorem, A/B test design and pitfalls, regression assumptions, causation and biases such as selection and survivorship.

What is a p-value in simple terms?

The probability of getting results at least as extreme as yours if there were really no effect. A small p-value suggests the data are unlikely under no effect; it is not the probability that there's no effect.

Why is peeking at A/B tests a problem?

Checking repeatedly and stopping as soon as a result looks significant greatly increases the false positive rate. Fix the sample size beforehand or use sequential testing methods.

What is Simpson's paradox?

A pattern where a trend appears in every subgroup but reverses when the groups are combined, usually because the groups differ in size or composition.