Document indexing: a concept-based approach to term weight estimation

Traditional index weighting approaches for information retrieval from texts depend on the term frequency based analysis of the text contents. A shortcoming of these indexing schemes, which consider only the occurrences of the terms in a document, is that they have some limitations in extracting sema...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Information processing & management Jg. 41; H. 5; S. 1065 - 1080
Hauptverfasser: Kang, Bo-Yeong, Lee, Sang-Jo
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Oxford Elsevier Ltd 01.09.2005
Elsevier Science
Elsevier Science Ltd
Schlagworte:
Automatic indexing
Cluster analysis
Concepts
Data mining
Exact sciences and technology
Formalism
Index weight estimation
Indexes
Indexing
Information and communication sciences
Information processing and retrieval
Information retrieval
Information retrieval. Man machine relationship
Information science. Documentation
Lexical semantics
Lexicon
Online information retrieval
Query formulation
Ranking
Research process. Evaluation
Retrieval
Sciences and techniques of general use
Search output
Semantics
Studies
Terms
Wall Street
Weighting
Weighting function
United States > US
Information retrieval
Index weight estimation
Weighting function
Automatic indexing
Automatic classification
Weighting
Estimation
Cluster
Experimentation
Algorithm
Index (documentation)
ISSN:0306-4573, 1873-5371
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Traditional index weighting approaches for information retrieval from texts depend on the term frequency based analysis of the text contents. A shortcoming of these indexing schemes, which consider only the occurrences of the terms in a document, is that they have some limitations in extracting semantically exact indexes that represent the semantic content of a document. To address this issue, we developed a new indexing formalism that considers not only the terms in a document, but also the concepts. In this approach, concept clusters are defined and a concept vector space model is proposed to represent the semantic importance degrees of lexical items and concepts within a document. Through an experiment on the TREC collection of Wall Street Journal documents, we show that the proposed method outperforms an indexing method based on term frequency (TF), especially in regard to the few highest-ranked documents. Moreover, the index term dimension was 80% lower for the proposed method than for the TF-based method, which is expected to significantly reduce the document search time in a real environment.
Bibliographie:SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 14
ObjectType-Article-1
ObjectType-Feature-2
content type line 23
ISSN:0306-4573
1873-5371
DOI:10.1016/j.ipm.2004.08.005