Ana içeriğe geç

Compare two versions

Measuring two states of the same text — before and after an edit, a translation against its original, a draft against the final — is the most direct use of this library. The comparison is fair because the same text goes through the same pipeline twice.

Method

import turkish_linguistic_features as tlf

before = tlf.analyze(before_text, lang="en")
after  = tlf.analyze(after_text,  lang="en")

print(f"{'key':22s} {'before':>10} {'after':>10} {'diff':>10}")
for k in ("sent_len_mean", "word_len_mean", "flesch_reading_ease",
          "flesch_kincaid_grade", "ari", "coleman_liau", "ttr", "long_word_ratio"):
    print(f"{k:22s} {before[k]:10.4f} {after[k]:10.4f} {after[k]-before[k]:+10.4f}")

A real example

An academic paragraph in its heavy form and its plain form:

The objective of the present investigation is the comparative evaluation of the formulas utilised in the determination of the readability levels of English texts, together with the establishment of the degrees of concordance obtaining between those formulas. In this connection, three distinct formulas in widespread use within the relevant literature have been examined.

This study compares English readability formulas. The aim is to show how far they agree. We looked at three formulas that are widely used.

Output:

key                        before      after       diff
sent_len_mean             26.5000     8.0000   -18.5000
word_len_mean              5.9434     4.6667    -1.2767
flesch_reading_ease       10.7375    68.2900   +57.5525
flesch_kincaid_grade      18.3450     5.7217   -12.6233
ari                       20.1689     5.1387   -15.0302
coleman_liau              18.0302     7.9400   -10.0902
ttr                        0.6792     0.9583    +0.2791
long_word_ratio            0.4528     0.2083    -0.2445

How to read it:

  • Mean sentence length 26.5 → 8 words. This is the real change; two long sentences became three short ones.
  • Flesch Reading Ease 10.7 → 68.3. On Flesch's bands, 10.7 is "very difficult" (postgraduate) and 68.3 is "standard". A 57-point jump.
  • All three grade-level formulas agree on the direction and roughly on the size: FKGL −12.6, ARI −15.0, Coleman-Liau −10.1. Agreement in direction is reassuring, but not proof: all three rest on word and sentence length.
  • long_word_ratio 0.45 → 0.21. "investigation", "concordance", "determination" are gone.
  • ttr 0.68 → 0.96 but this means nothing. The second text is shorter, so TTR rose. That is length, not style.

Many pairs at once

If you have a set of pairs, build a table:

import turkish_linguistic_features as tlf

rows = []
for name, before_text, after_text in pairs:
    for label, text in (("before", before_text), ("after", after_text)):
        rows.append({"label": label, "source": name, "segment_id": 0,
                     **tlf.analyze(text, lang="en")})

tlf.save_csv(rows, "before-after.csv")

Because label holds before/after, the CSV feeds straight into a paired analysis.