Artificial Intelligence Natural Language Processing

Question 1

What is tokenization in natural language processing?

Accepted Answer

Splitting text into meaningful units such as words or subwords

Answer

Converting text to uppercase

Answer

Measuring the sentiment of a sentence

Answer

Removing stop words from a document

Question 2

What does TF-IDF measure in information retrieval and NLP?

Accepted Answer

The importance of a word in a document relative to a corpus

Answer

The grammatical structure of a sentence

Answer

The translation probability between two languages

Answer

The semantic similarity between two documents

Question 3

Which word embedding model learns vector representations by predicting surrounding words (skip-gram) or predicting a word from context (CBOW)?

Accepted Answer

Word2Vec

Answer

GloVe

Answer

BERT

Answer

FastText

Question 4

What NLP task involves labeling each token in a sentence with its grammatical role (noun, verb, etc.)?

Accepted Answer

Part-of-speech tagging

Answer

Named entity recognition

Answer

Sentiment analysis

Answer

Dependency parsing

Question 5

What is the 'attention mechanism' in neural NLP models?

Accepted Answer

A method that allows the model to weight the importance of different input tokens when producing an output

Answer

A technique for removing irrelevant tokens

Answer

A way to speed up tokenization

Answer

A rule for choosing the correct part of speech

Question 6

What is named entity recognition (NER) in NLP?

Accepted Answer

Identifying and classifying real-world entities such as people, organizations, and locations in text

Answer

Detecting the language of a text

Answer

Counting the frequency of words in a corpus

Answer

Generating summaries of long documents