Distributed Identification of Top-l Inner Product Elements and its Application in a Peer-to-Peer Network

The inner product measures how closely two feature vectors are related. It is an important primitive for many popular data mining tasks, for example, clustering, classification, correlation computation, and decision tree construction. If the entire data set is available at a single site, then comput...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on knowledge and data engineering Jg. 20; H. 4; S. 475 - 488
Hauptverfasser: Das, K., Bhaduri, K., Kun Liu, Kargupta, H.
Format: Journal Article
Sprache:Englisch
Veröffentlicht: New York, NY IEEE 01.04.2008
IEEE Computer Society
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:
Algorithms
Algorithms for data and knowledge management
Applied sciences
Classification tree analysis
Computation
Computer science; control theory; systems
Computer systems and distributed systems. User interface
Data mining
Data processing. List processing. Character string processing
Decision trees
Exact sciences and technology
Knowledge management applications
Large-scale systems
Mathematical analysis
Memory organisation. Data processing
Mining methods and algorithms
Network servers
Networks
Partitioning algorithms
Peer to peer computing
Probabilistic algorithms
Scalability
Software
Statistical distributions
Statistics
Studies
Vectors (mathematics)
Probabilistic algorithms
Knowledge management applications
Algorithms for data and knowledge management
Data mining
Mining methods and algorithms
Cluster analysis
Distributed data mining
Data analysis
Correlation
Peer to peer
Statistical analysis
Probabilistic approach
Scalability
Matrix product
pier-to-peer network
Information extraction
Distributed system
Decision tree
Modeling
Classification
inner product
Localization
ISSN:1041-4347, 1558-2191
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The inner product measures how closely two feature vectors are related. It is an important primitive for many popular data mining tasks, for example, clustering, classification, correlation computation, and decision tree construction. If the entire data set is available at a single site, then computing the inner product matrix and identifying the top (in terms of magnitude) entries is trivial. However, in many real-world scenarios, data is distributed across many locations and transmitting the data to a central server would be quite communication intensive and not scalable. This paper presents an approximate local algorithm for identifying top-l, inner products among pairs of feature vectors in a large asynchronous distributed environment such as a peer-to-peer (P2P) network. We develop a probabilistic algorithm for this purpose using order statistics and the Hoeffding bound. We present experimental results to show the effectiveness and scalability of the algorithm. Finally, we demonstrate an application of this technique for interest-based community formation in a P2P environment.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ObjectType-Article-2
ObjectType-Feature-1
content type line 23
ISSN:1041-4347
1558-2191
DOI:10.1109/TKDE.2007.190714