NLP Basic Concepts

Stem is the invariable part of a word (volver –> volv).

Lemma is the canonical form of a lexeme (volvieron –> volver). Lexeme is, in english, more or less the same thingh but not in spanish. Therfore do not use Lexeme to refer Stem nor Lemma.

Chunk: Phrase chunking is a natural language process that separates and segments sentences into its subconstituents, i.e. noun, verb and prepositional phrases.

T-test is a mesure that indicates if an improvement is statistically significant (EXTEND WIKIPEDIA).

Precision is the number of correct answer instances divided by all answered instances.

Recall is the number of correct answer instances divided by all instances.

F1-mesure is F1= (2*precision*recall)/(precision+recall) - Normalizes the previous measures

MRR (Mean Reciprocal Rank) is a mesure used in TREC for QA.The score for an individual question was the reciprocal of the rank at which the first correct answer was returned or 0 of no correct response was returned. is a statistic for evaluating any process that produces a list of possible responses to a query, ordered by probability of correctness. The reciprocal rank of a query response is the multiplicative inverse of the rank of the correct answer. The mean reciprocal rank is the average of the reciprocal ranks of a sample of queries.

Example

For example, suppose we have the following three sample queries for a system that tries to translate English words to their plurals. In each case, the system makes three guesses, with the first one being the one it thinks is most likely correct:

Query Results Correct response Rank Reciprocal rank
cat catten, cati, cats cats 3 1/3
torus torii, tori, toruses tori 2 1/2
virus viruses, virii, viri viruses 1 1

Given those three samples, we could calculate the mean reciprocal rank as (1/3 + 1/2 + 1)/3 = 11/18 or about 0.61.

This basic definition does not specify what to do if (1) none of the proposed results are correct (use reciprocal rank 0), or if (2) there are multiple correct answers in the list (consider using Mean Average Precision: MAP).

Improvement Measure

The improvement of one value over another is calculated: (better*100)/worse

Error Reduction Measure

The error (100-score) reduction of one value over another is calculated: (((100-worse)-(100-better))*100)/(100-worse)