What NaN means¶
Short answer¶
nan means "I cannot compute this number for this text." It is not an
error; it is a missing-data marker.
The library does not invent a number from insufficient data. If it did, your table would contain a column that looks real and means nothing, with no way to tell.
How often¶
In a three-word English text:
short = tlf.analyze("A short sentence.", lang="en")
nans = [k for k, v in short.items() if isinstance(v, float) and v != v]
len(short), len(nans)
34 of 181 features are nan. Examples:
['aspect_imp_ratio', 'aspect_perf_ratio', 'aspect_prog_ratio',
'case_abl_ratio', 'case_acc_ratio', 'case_dat_ratio', 'case_gen_ratio',
'case_loc_ratio']
In the same text, ttr still returns a number:
Why it happens¶
Three reasons.
1. The text is too short¶
Every feature has a minimum, and it is stated in the registry:
A word here is the library's default word
(Concepts). segment_text's segment_size counts the
same word, so a 100-word segment meets a 100-word minimum.
Below the minimum you get nan. Some minimums come from the source
(mtld: "texts as short as 100 tokens can be used"), others from the
mathematics: with a single window, mattr collapses to plain TTR and stops
being a moving average — hence the 2 × window threshold.
2. The required structure is absent¶
zeyrek_derivational_suffix_ratio has a zero denominator if the text contains no
derivational suffixes. parse_depth_mean cannot produce a value if no
sentence parses. verb_dist_mean cannot be measured unless the text has at least two verbs.
The punct_…_ratio features cannot be computed if the text has no punctuation marks.
3. An optional package is missing¶
Without wordfreq, wordfreq_mean and wordfreq_rare_ratio return nan
and the library raises a MissingDependencyWarning.
warn=False silences the warning only; the feature is still nan. The same
flag also silences ParagraphStructureWarning.
Not nan, but misleading: paragraphs¶
Paragraph boundaries are found from blank lines. If your text has no blank lines,
the whole text counts as one paragraph and para_len_mean becomes the word count
of the entire text: the value is not nan, but it says nothing about paragraphs.
Details: limitations §8.
Why nan and not None¶
nan is a float. That means every value in the returned dictionary has
the same type, and the table goes straight into pandas, numpy, R or
CSV. With None, the column dtype becomes object, arithmetic breaks, and
in a CSV an empty cell becomes hard to distinguish from a real zero.