NLP interview questions: from tokenisation to transformers

Fifteen common NLP interview questions

1. What preprocessing steps would you apply to text, and when would you skip them?
2. Stemming versus lemmatisation?
3. How do subword tokenisers like BPE and WordPiece work?
4. What is TF-IDF?
5. How do word2vec embeddings work?
6. What is the difference between static and contextual embeddings?
7. Compare BERT and GPT.
8. What is masked language modelling?
9. How does attention improve sequence-to-sequence models?
10. How would you build a text classifier?
11. What is named entity recognition, and how is it framed?
12. What are BLEU, ROUGE and perplexity?
13. How do you handle out-of-vocabulary words?
14. How do you handle multilingual text?
15. When would you use a classical model instead of an LLM?

Classical foundations

• Preprocessing: lower-casing, removing noise, handling punctuation and stop words. Modern transformer pipelines often skip most of it, because the tokeniser and model handle raw text better.
• Stemming vs lemmatisation: stemming chops suffixes by rule (“studies” → “studi”); lemmatisation maps to dictionary forms (“studies” → “study”) using vocabulary and part of speech.
• TF-IDF: weights a word by its frequency in a document times the log of how rare it is across documents, so distinctive words count more than common ones.
• N-grams: sequences of n tokens that capture some word order in bag-of-words models.

Embeddings and tokenisation

Word2vec learns a vector per word by predicting words from their context (CBOW) or context from a word (skip-gram), so words in similar contexts get similar vectors. These are static: “bank” has one vector whether it means a river bank or a financial one. Contextual embeddings from transformers give each occurrence its own vector depending on the sentence. Subword tokenisers such as BPE and WordPiece build a vocabulary of frequent character sequences, so rare and new words are split into known pieces — largely solving the out-of-vocabulary problem.

BERT versus GPT

Tasks and evaluation

• Text classification: start with TF-IDF and logistic regression as a baseline, then fine-tune a small transformer if needed; evaluate with precision, recall and F1 per class.
• NER: usually framed as token classification with BIO tags (beginning, inside, outside) and evaluated with entity-level F1.
• BLEU: n-gram precision against references, common for translation. ROUGE: n-gram recall, common for summarisation. Both correlate weakly with quality for open-ended generation.
• Perplexity: how surprised a language model is by held-out text; lower is better, but it doesn't measure usefulness.

Multilingual text

For multilingual products, mention that tokenisers split some languages into many more tokens than English, raising cost and shrinking effective context; that multilingual encoders allow training in one language and applying to others with some loss; and that evaluation needs native-speaker review per language, because automatic metrics and translated test sets both hide quality problems.

Classical model or LLM?

For high-volume, narrow tasks with labelled data — routing support tickets, detecting spam — a small fine-tuned encoder or even TF-IDF with logistic regression is cheaper, faster and more predictable. LLMs shine with little labelled data, many tasks at once, or open-ended outputs. A common good answer: use an LLM to bootstrap labels, then train a small model for production. See generative AI interview questions.

Practising

The fastest way to find gaps is to be asked. Run a spoken mock round on NLP topics — a free AI interview assistant such as CoPilot Interview builds one from your résumé and the job description and scores every answer — then study whatever you stumbled on. Have one end-to-end text project ready to discuss, with the baseline you beat and by how much.

Disclosure: CoPilot Interview is our product, so weigh what we say about it accordingly. Everything else on this page applies whichever tool you use, or none.

Frequently asked questions

What are common NLP interview questions?

Preprocessing, stemming versus lemmatisation, subword tokenisation, TF-IDF, word2vec and contextual embeddings, BERT versus GPT, attention, text classification, NER, evaluation metrics and when to prefer classical models over LLMs.

What is the difference between BERT and GPT?

BERT is an encoder trained with masked language modelling using context from both sides, best for understanding tasks. GPT is a decoder trained to predict the next token left to right, best for generation.

What is TF-IDF in simple terms?

A score that weights a word by how often it appears in a document and how rare it is across all documents, so distinctive words matter more than common ones.

Is classical NLP still relevant in interviews?

Yes. Interviewers value knowing when a TF-IDF baseline or small fine-tuned model beats an LLM on cost, latency and predictability for narrow, high-volume tasks.