A Comparison of Recent Information Retrieval Term-Weighting Models Using Ancient Datasets

Abstract

With the development of technology, human computer interaction is continuously increasing. Parallel to this, information from web sites, social media, blogs and other applications reach enormous dimensions. It becomes a big problem to obtain the desired information from this mass of data. One way of solving this problem is to keep the information correctly indexed and searched by using information retrieval methods. Information retrieval is the study of finding documents of unstructured material which should satisfy users' information needs. Various term-weighting models have been proposed for information retrieval. This work is carried out to analyze and evaluate the retrieval effectiveness of recently developed term-weighting models (after the 2000s) using the earlier datasets (dating back as far as the 1980s) with the motivation of such comparison has not been done. The open source library Apache Lucene is used for all experiments and evaluation. As a result, we observe that the DFIC model is in general more effective than the other models. We note also that, although one model can be the most effective for one dataset, the same model can be the least effective for another dataset.