Fuzzy Rough Set based Evaluations of Summaries

Note: This same article appears in my median.com account as well

#AI-EXERCISE #RESEARCH-Excercise

This short article lays emphasis on how to evaluate summaries produced from Text Summarization. The toolkit used here is Fuzzy Rough sets. The reference summary and the system summary are evaluated and compared for similarity using Fuzzy Rough Set based lower similarity and upper similarity. However, this has not been evaluated yet. The evaluation needs comparisons of results with typical ROUGE based recall scores for n-grams. The intuition is basically based on the fact that the computation of lower and upper approximation require more than an n-gram based model. This is much more than n-gram model.

The definition of Fuzzy Rough Set based lower and upper approximation is given as follows:

Definition. A generalized definition of lower and upper approximations of Fuzzy Rough Set, where R be the fuzzy equivalence relation, is as follows:

Let two summary produced be E1 and let the reference summary be R1. The two kinds of similarities are computed:

1. Lower Similarity

2. Upper Similarity

These accounts for how much similar are the system generated summary and the reference gold summary. Compute similarity between the system summary and the reference summary and then compute the ROUGE scores, and see the correlation and similarity between scores.

This was the guideline for your Research Exercise, which can be taken as a AI Exercise as well.

Published by Nidhika

Hi, Apart from profession, I have inherent interest in writing especially about Global Issues of Concern, fiction blogs, poems, stories, doing painting, cooking, photography, music to mention a few! And most important on this website you can find my suggestions to latest problems, views and ideas, my poems, stories, novels, some comments, proposals, blogs, personal experiences and occasionally very short glimpses of my research work as well.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: