Generative AI interview questions with answer outlines

Fifteen common generative AI interview questions

1. How does self-attention work, and why did transformers replace RNNs?
2. What is tokenisation, and how does it affect cost and behaviour?
3. Explain temperature, top-k and top-p sampling.
4. What causes hallucinations, and how do you reduce them?
5. How do diffusion models generate images?
6. How do GANs work, and what is mode collapse?
7. What is a VAE, and how does it differ from a GAN?
8. How do multimodal models like CLIP align text and images?
9. When would you prompt, use retrieval, or fine-tune?
10. What is LoRA, and why is it popular?
11. How do you evaluate a generative model's outputs?
12. What is the KV cache, and why does it matter for latency?
13. What is quantisation, and what does it trade off?
14. What is prompt injection, and how do you defend against it?
15. How would you reduce the cost of a generative AI feature?

Fundamentals, answered briefly

• Self-attention: each token builds a weighted mix of every other token's representation, with weights from query–key similarity. It parallelises across the sequence, unlike RNNs, and captures long-range dependencies directly.
• Tokenisation: text is split into subword units; cost, context limits and some quirks (arithmetic, rare words, non-English text using more tokens) follow from it.
• Sampling: temperature scales the probability distribution — low is focused, high is varied. Top-k keeps the k likeliest tokens; top-p keeps the smallest set whose probability sums to p.
• Hallucination: the model generates plausible text without grounding. Reduce it with retrieval and citations, constrained output formats, lower temperature for factual tasks, and verification steps.

Beyond language models

Prompting, retrieval or fine-tuning?

Start with prompting: cheapest and fastest to iterate. Add retrieval when the model needs private or fresh knowledge — it grounds answers and lets you cite sources. Fine-tune when you need consistent style, format or behaviour that prompting can't reliably produce, or to make a smaller model match a larger one on a narrow task. LoRA fine-tunes small low-rank adapter matrices instead of all weights, cutting memory and storage so many task-specific adapters can share one base model. Interviewers like hearing that these combine: a fine-tuned model that also uses retrieval is common. See RAG interview questions.

Evaluation and production

• Evaluation: task-specific test sets, human ratings, model-graded evaluation with a clear rubric (checked against humans), and online metrics such as acceptance or edit rate. Automatic overlap metrics like BLEU and ROUGE correlate poorly with quality for open-ended text.
• Latency: the KV cache stores attention keys and values for previous tokens so each new token doesn't recompute them; streaming output improves perceived speed.
• Cost: shorter prompts, caching repeated context, routing easy requests to smaller models, batching, and quantised self-hosted models where volume justifies it.
• Safety: content filtering, prompt-injection defences (separate instructions from untrusted data, restrict tool permissions), and red-teaming before launch.

How deep interviews go by role

Practising

Answer these out loud, in two minutes or less each, and expect a follow-up on every claim. A free AI interview assistant such as CoPilot Interview can run generative AI questions as a mock interview built from your résumé and score each answer. Pair this page with LLM interview questions and AI engineer interview questions for the full loop.

Disclosure: CoPilot Interview is our product, so weigh what we say about it accordingly. Everything else on this page applies whichever tool you use, or none.

Frequently asked questions

What are the most common generative AI interview questions?

How attention and transformers work, tokenisation, sampling parameters, hallucination, diffusion versus GANs versus VAEs, prompting versus retrieval versus fine-tuning, LoRA, evaluation, latency and cost, and prompt injection.

How do diffusion models work, in an interview answer?

They are trained to reverse a gradual noising process. At generation time they start from noise and denoise step by step, often guided by a text embedding, to produce an image.

When should you fine-tune instead of using RAG?

Fine-tune for consistent style, format or behaviour, or to specialise a smaller model. Use retrieval for private or changing knowledge and when you need citations. They are often combined.

How do you evaluate generative AI output?

With task-specific test sets, human ratings, rubric-based model grading validated against humans, and online signals such as acceptance or edit rates. Overlap metrics like BLEU are weak for open-ended text.