Distributed Sparse Random Projection Trees for Constructing K-Nearest Neighbor Graphs
A random projection tree that partitions data points by projecting them onto random vectors is widely used for approximate nearest neighbor search in high-dimensional space. We consider a particular case of random projection trees for constructing a k-nearest neighbor graph (KNNG) from high-dimensio...
Uložené v:
| Vydané v: | Proceedings - IEEE International Parallel and Distributed Processing Symposium s. 36 - 46 |
|---|---|
| Hlavní autori: | , , |
| Médium: | Konferenčný príspevok.. |
| Jazyk: | English |
| Vydavateľské údaje: |
IEEE
01.05.2023
|
| Predmet: | |
| ISSN: | 1530-2075 |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | A random projection tree that partitions data points by projecting them onto random vectors is widely used for approximate nearest neighbor search in high-dimensional space. We consider a particular case of random projection trees for constructing a k-nearest neighbor graph (KNNG) from high-dimensional data. We develop a distributed-memory Random Projection Tree (DRPT) algorithm for constructing sparse random projection trees and then running a query on the forest to create the KNN graph. DRPT uses sparse matrix operations and a communication reduction scheme to scale KNN graph constructions to thousands of processes on a supercomputer. The accuracy of DRPT is comparable to state-of-the-art methods for approximate nearest neighbor search, while it runs two orders of magnitude faster than its peers. DRPT is available at https://github.com/HipGraph/DRPT. |
|---|---|
| AbstractList | A random projection tree that partitions data points by projecting them onto random vectors is widely used for approximate nearest neighbor search in high-dimensional space. We consider a particular case of random projection trees for constructing a k-nearest neighbor graph (KNNG) from high-dimensional data. We develop a distributed-memory Random Projection Tree (DRPT) algorithm for constructing sparse random projection trees and then running a query on the forest to create the KNN graph. DRPT uses sparse matrix operations and a communication reduction scheme to scale KNN graph constructions to thousands of processes on a supercomputer. The accuracy of DRPT is comparable to state-of-the-art methods for approximate nearest neighbor search, while it runs two orders of magnitude faster than its peers. DRPT is available at https://github.com/HipGraph/DRPT. |
| Author | Azad, Ariful Rahman, Md Khaledur Ranawaka, Isuru |
| Author_xml | – sequence: 1 givenname: Isuru surname: Ranawaka fullname: Ranawaka, Isuru email: isjarana@iu.edu organization: Indiana University,Bloomington,IN,USA – sequence: 2 givenname: Md Khaledur surname: Rahman fullname: Rahman, Md Khaledur email: khaledrahman@meta.com organization: Meta Inc – sequence: 3 givenname: Ariful surname: Azad fullname: Azad, Ariful email: azad@iu.edu organization: Indiana University,Bloomington,IN,USA |
| BookMark | eNotjF1PwjAYRqvRRED-gSb9A8O3X-t6aYYgkSARuCbd-hZKZCPtuPDfu0SvnuSck2dI7pq2QUKeGUwYA_OyWE_XGyWNMhMOXEwAgMkbMjbaFEKBEDrP-S0ZMCUg46DVAxmmdALgIKQZkN00pC6G6tqho5uLjQnpl21ce6br2J6w7kLb0G1ETNS3kZZt0_fXHjcH-pGt0EZMHV1hOByr3s-jvRzTI7n39jvh-H9HZDd725bv2fJzvihfl1ngILus8gJz4ZnGinGpVC24Es5i4fJC1bVXVnNlmGIuR-O5yWvBQDrra68LdE6MyNPfb0DE_SWGs40_ewZMa8lA_ALa51Sm |
| CODEN | IEEPAD |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IL CBEJK RIE RIL |
| DOI | 10.1109/IPDPS54959.2023.00014 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Xplore POP ALL IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP All) 1998-Present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Xplore Digital Library url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISBN | 9798350337662 |
| EISSN | 1530-2075 |
| EndPage | 46 |
| ExternalDocumentID | 10177410 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: Advanced Scientific Computing Research funderid: 10.13039/100006192 – fundername: U.S. Department of Energy funderid: 10.13039/100000015 |
| GroupedDBID | 29O 6IE 6IF 6IH 6IK 6IL 6IN AAJGR AAWTH ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IPLJI OCL RIE RIL |
| ID | FETCH-LOGICAL-i204t-bf3e63f17eb12455c3253dae8d685ccf5a7259151d6e9f296c3104dafcf78edd3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 2 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001035517300005&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 02:11:46 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i204t-bf3e63f17eb12455c3253dae8d685ccf5a7259151d6e9f296c3104dafcf78edd3 |
| PageCount | 11 |
| ParticipantIDs | ieee_primary_10177410 |
| PublicationCentury | 2000 |
| PublicationDate | 2023-May |
| PublicationDateYYYYMMDD | 2023-05-01 |
| PublicationDate_xml | – month: 05 year: 2023 text: 2023-May |
| PublicationDecade | 2020 |
| PublicationTitle | Proceedings - IEEE International Parallel and Distributed Processing Symposium |
| PublicationTitleAbbrev | IPDPS |
| PublicationYear | 2023 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0020349 |
| Score | 1.8451526 |
| Snippet | A random projection tree that partitions data points by projecting them onto random vectors is widely used for approximate nearest neighbor search in... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 36 |
| SubjectTerms | Approximation algorithms Costs Data visualization Distributed databases distributed memory algorithms k nearest neighbor graph Nearest neighbor methods parallel algorithm Parallel processing Scalability |
| Title | Distributed Sparse Random Projection Trees for Constructing K-Nearest Neighbor Graphs |
| URI | https://ieeexplore.ieee.org/document/10177410 |
| WOSCitedRecordID | wos001035517300005&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3JTsMwELWg4sCpLEXs8oGraRYnds6UAgJFEW2l3qrEHqMikVZN4fuZSdLChQO3KFJkeZ15zrz3GLsJpQ2tAV9gbm0E7sREJKClMNpJTHfBl0Ut4vqi0lRPp0nWktVrLgwA1MVncEuP9b98uzCfdFXWp-WDERAR-q5SqiFrbdEVCa20FB3fS_pP2SAbIfiJiI0SkIypR0ydXxYqdQQZdv_Z9gHr_XDxeLaNModsB8oj1t2YMfB2bx6zyYAkcMm9CiwfLRGvAn_NS7v4oM_f64qrko9XABXHRJWTU2ejHVu-8WeRkpZtteYpXZXiuuAPpGRd9dhkeD--exStZ4KYB55ci8KFEIfOV3gGBzKKTBhEoc1B21hHxrgoVwh4MMzbGBIXJLHB_E7a3BmnNFgbnrBOuSjhlHFMPExhnTYGAlkg0IqtKZwqrI4TUMo7Yz0aptmykcWYbUbo_I_3F2yfZqKpFrxkHewjXLE987WeV6vrejK_ATRVodk |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NT8JAEN0YNNETfmD8dg9eV2i7bbdnESFg0wgk3Ei7O2swsRCK_n5n2opePHhrmjSb_Zx523nvMXbnSeMZDY7A3FoL3ImRiEBJoZWVmO6CI7NSxHUUxrGazaKkJquXXBgAKIvP4J4ey3_5Zqk_6KqsTcsHIyAi9F1fStep6FpbfEVSKzVJx-lE7UHSTcYIf3zio7gkZNohrs4vE5UyhvSa_2z9kLV-2Hg82caZI7YD-TFrftsx8Hp3nrBpl0Rwyb8KDB-vELECf0lzs3ynz9_KmqucT9YABcdUlZNXZ6Uem7_yoYhJzbbY8JguS3Fl8CfSsi5abNp7nDz0Re2aIBZuR25EZj0IPOuEeAq70ve15_qeSUGZQPlaWz8NEfJgoDcBRNaNAo0ZnjSp1TZUYIx3yhr5MoczxjH10JmxSmtwZYZQKzA6s2FmVBBBGHbOWYuGab6qhDHm3yN08cf7W7bfnzyP5qNBPLxkBzQrVe3gFWtgf-Ga7enPzaJY35QT-wXMWKUg |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Proceedings+-+IEEE+International+Parallel+and+Distributed+Processing+Symposium&rft.atitle=Distributed+Sparse+Random+Projection+Trees+for+Constructing+K-Nearest+Neighbor+Graphs&rft.au=Ranawaka%2C+Isuru&rft.au=Rahman%2C+Md+Khaledur&rft.au=Azad%2C+Ariful&rft.date=2023-05-01&rft.pub=IEEE&rft.eissn=1530-2075&rft.spage=36&rft.epage=46&rft_id=info:doi/10.1109%2FIPDPS54959.2023.00014&rft.externalDocID=10177410 |