HCI∧2 framework: a software framework for multimodal human-computer interaction systems
This paper presents a novel software framework for the development and research in the area of multimodal human-computer interface (MHCI) systems. The proposed software framework, which is called the HCI∧2 Framework, is built upon publish/subscribe (P/S) architecture. It implements a shared-memory-b...
Gespeichert in:
| Veröffentlicht in: | IEEE transactions on cybernetics Jg. 43; H. 6; S. 1593 |
|---|---|
| Hauptverfasser: | , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
United States
01.12.2013
|
| Schlagworte: | |
| ISSN: | 2168-2275, 2168-2275 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | This paper presents a novel software framework for the development and research in the area of multimodal human-computer interface (MHCI) systems. The proposed software framework, which is called the HCI∧2 Framework, is built upon publish/subscribe (P/S) architecture. It implements a shared-memory-based data transport protocol for message delivery and a TCP-based system management protocol. The latter ensures that the integrity of system structure is maintained at runtime. With the inclusion of bridging modules, the HCI∧2 Framework is interoperable with other software frameworks including Psyclone and ActiveMQ. In addition to the core communication middleware, we also present the integrated development environment (IDE) of the HCI∧2 Framework. It provides a complete graphical environment to support every step in a typical MHCI system development process, including module development, debugging, packaging, and management, as well as the whole system management and testing. The quantitative evaluation indicates that our framework outperforms other similar tools in terms of average message latency and maximum data throughput under a typical single PC scenario. To demonstrate HCI∧2 Framework's capabilities in integrating heterogeneous modules, we present several example modules working with a variety of hardware and software. We also present an example of a full system developed using the proposed HCI∧2 Framework, which is called the CamGame system and represents a computer game based on hand-held marker(s) and low-cost camera(s). |
|---|---|
| AbstractList | This paper presents a novel software framework for the development and research in the area of multimodal human-computer interface (MHCI) systems. The proposed software framework, which is called the HCI∧2 Framework, is built upon publish/subscribe (P/S) architecture. It implements a shared-memory-based data transport protocol for message delivery and a TCP-based system management protocol. The latter ensures that the integrity of system structure is maintained at runtime. With the inclusion of bridging modules, the HCI∧2 Framework is interoperable with other software frameworks including Psyclone and ActiveMQ. In addition to the core communication middleware, we also present the integrated development environment (IDE) of the HCI∧2 Framework. It provides a complete graphical environment to support every step in a typical MHCI system development process, including module development, debugging, packaging, and management, as well as the whole system management and testing. The quantitative evaluation indicates that our framework outperforms other similar tools in terms of average message latency and maximum data throughput under a typical single PC scenario. To demonstrate HCI∧2 Framework's capabilities in integrating heterogeneous modules, we present several example modules working with a variety of hardware and software. We also present an example of a full system developed using the proposed HCI∧2 Framework, which is called the CamGame system and represents a computer game based on hand-held marker(s) and low-cost camera(s). This paper presents a novel software framework for the development and research in the area of multimodal human-computer interface (MHCI) systems. The proposed software framework, which is called the HCI∧2 Framework, is built upon publish/subscribe (P/S) architecture. It implements a shared-memory-based data transport protocol for message delivery and a TCP-based system management protocol. The latter ensures that the integrity of system structure is maintained at runtime. With the inclusion of bridging modules, the HCI∧2 Framework is interoperable with other software frameworks including Psyclone and ActiveMQ. In addition to the core communication middleware, we also present the integrated development environment (IDE) of the HCI∧2 Framework. It provides a complete graphical environment to support every step in a typical MHCI system development process, including module development, debugging, packaging, and management, as well as the whole system management and testing. The quantitative evaluation indicates that our framework outperforms other similar tools in terms of average message latency and maximum data throughput under a typical single PC scenario. To demonstrate HCI∧2 Framework's capabilities in integrating heterogeneous modules, we present several example modules working with a variety of hardware and software. We also present an example of a full system developed using the proposed HCI∧2 Framework, which is called the CamGame system and represents a computer game based on hand-held marker(s) and low-cost camera(s).This paper presents a novel software framework for the development and research in the area of multimodal human-computer interface (MHCI) systems. The proposed software framework, which is called the HCI∧2 Framework, is built upon publish/subscribe (P/S) architecture. It implements a shared-memory-based data transport protocol for message delivery and a TCP-based system management protocol. The latter ensures that the integrity of system structure is maintained at runtime. With the inclusion of bridging modules, the HCI∧2 Framework is interoperable with other software frameworks including Psyclone and ActiveMQ. In addition to the core communication middleware, we also present the integrated development environment (IDE) of the HCI∧2 Framework. It provides a complete graphical environment to support every step in a typical MHCI system development process, including module development, debugging, packaging, and management, as well as the whole system management and testing. The quantitative evaluation indicates that our framework outperforms other similar tools in terms of average message latency and maximum data throughput under a typical single PC scenario. To demonstrate HCI∧2 Framework's capabilities in integrating heterogeneous modules, we present several example modules working with a variety of hardware and software. We also present an example of a full system developed using the proposed HCI∧2 Framework, which is called the CamGame system and represents a computer game based on hand-held marker(s) and low-cost camera(s). |
| Author | Shen, Jie Pantic, Maja |
| Author_xml | – sequence: 1 givenname: Jie surname: Shen fullname: Shen, Jie – sequence: 2 givenname: Maja surname: Pantic fullname: Pantic, Maja |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/24235258$$D View this record in MEDLINE/PubMed |
| BookMark | eNpNkE1OwzAQhS1URKH0AGyQl2xS_JvE7CACWqkSm7LoKnIcRwTiONiOqt6Am3AvToIlimAW8-Z9enqLOQOT3vYagAuMFhgjcb0ptncLgjBdEJJhntIjcEpwmifR8sm_ewrm3r-iOHlEIj8BU8II5YTnp2C7LFZfH58ENk4avbPu7QZK6G0TdtLpPwob66AZu9AaW8sOvoxG9omyZhiDdrDt45YqtLaHfu-DNv4cHDey83p-0Bl4frjfFMtk_fS4Km7XyYAZDolEhCAq0rTGdSYQ16jmteIqOtoIVpFUEK4QqXLFsoxzSRHBkXOERcVyTWfg6qd3cPZ91D6UpvVKd53stR19iTlijLOU0Ri9PETHyui6HFxrpNuXv--g34W6ZSg |
| ContentType | Journal Article |
| DBID | CGR CUY CVF ECM EIF NPM 7X8 |
| DOI | 10.1109/TCYB.2013.2271563 |
| DatabaseName | Medline MEDLINE MEDLINE (Ovid) MEDLINE MEDLINE PubMed MEDLINE - Academic |
| DatabaseTitle | MEDLINE Medline Complete MEDLINE with Full Text PubMed MEDLINE (Ovid) MEDLINE - Academic |
| DatabaseTitleList | MEDLINE MEDLINE - Academic |
| Database_xml | – sequence: 1 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: 7X8 name: MEDLINE - Academic url: https://search.proquest.com/medline sourceTypes: Aggregation Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Sciences (General) |
| EISSN | 2168-2275 |
| ExternalDocumentID | 24235258 |
| Genre | Research Support, Non-U.S. Gov't Journal Article |
| GroupedDBID | 0R~ 4.4 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACIWK AENEX AGQYO AGSQL AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CGR CUY CVF EBS ECM EIF EJD HZ~ IFIPE IPLJI JAVBF M43 NPM O9- OCL PQQKQ RIA RIE RIG RNS 7X8 |
| ID | FETCH-LOGICAL-p141t-a02203966d1d7905e0d5dc5c1d73f94b26925c02b8c47755a302194b5019b48e3 |
| ISICitedReferencesCount | 8 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000327647500009&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 2168-2275 |
| IngestDate | Thu Oct 02 05:31:54 EDT 2025 Mon Jul 21 05:41:48 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 6 |
| Language | English |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-p141t-a02203966d1d7905e0d5dc5c1d73f94b26925c02b8c47755a302194b5019b48e3 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 |
| PMID | 24235258 |
| PQID | 1504454643 |
| PQPubID | 23479 |
| ParticipantIDs | proquest_miscellaneous_1504454643 pubmed_primary_24235258 |
| PublicationCentury | 2000 |
| PublicationDate | 2013-Dec 20131201 |
| PublicationDateYYYYMMDD | 2013-12-01 |
| PublicationDate_xml | – month: 12 year: 2013 text: 2013-Dec |
| PublicationDecade | 2010 |
| PublicationPlace | United States |
| PublicationPlace_xml | – name: United States |
| PublicationTitle | IEEE transactions on cybernetics |
| PublicationTitleAlternate | IEEE Trans Cybern |
| PublicationYear | 2013 |
| SSID | ssj0000816898 |
| Score | 2.0253475 |
| Snippet | This paper presents a novel software framework for the development and research in the area of multimodal human-computer interface (MHCI) systems. The proposed... |
| SourceID | proquest pubmed |
| SourceType | Aggregation Database Index Database |
| StartPage | 1593 |
| SubjectTerms | Biofeedback, Psychology - methods Computer Graphics Humans Imaging, Three-Dimensional - methods Information Storage and Retrieval - methods Man-Machine Systems Software Software Design User-Computer Interface Video Games |
| Title | HCI∧2 framework: a software framework for multimodal human-computer interaction systems |
| URI | https://www.ncbi.nlm.nih.gov/pubmed/24235258 https://www.proquest.com/docview/1504454643 |
| Volume | 43 |
| WOSCitedRecordID | wos000327647500009&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE/IET Electronic Library customDbUrl: eissn: 2168-2275 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000816898 issn: 2168-2275 databaseCode: RIE dateStart: 20130101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV3datRAFB5s9cIbsdZqtZYpVLFI6mZ-dma8q0tLhbIWTGF7FSaTCWyh2XWz1foGfZO-V5_EM5NJNmIL9cKbkBzID_MdTs6Z8_MhtO2YaEyfq0hLySBAsSRShS4iQ4tcZvAD6QvtySbEcChHI3Uc6ucrTycgylJeXqrpf4UaZAC2a539B7jbh4IAzgF0OALscLwX8IeDL76CQb7dE-RD0RRf1V3NFVjdn67Yq5X7OkNfVng-yV1jpNvVj0wge_DjJGaBT7zqTDcP_qyLFR3NRMM57pMP5ldmZ6Xrjmwd9m9NE8i4VaRjwHRs6n6hM93dfYhpp5LDGykS92H5Sc1-smtvkQUrWw9jCtrUNZngT9HbbbkfhZoMTj-7Cjy6C0-EYJMuflxNsn74NT04OTpKk_1R8m76PXKUYi71HvhVltBD-BYV18197QacZxzxXMnt94asN7z441-vvTsC8Z5I8hQ9CSEE3quhX0EPbPkMrQQjXeH3YZL4zio6BV24ubomuEX7E9a40YGFFIMO4IUO4D91AHd0AAcdeI5ODvaTwWEUuDSiaczieaRdRzWF2DaPczeTzfZynhtu4IoWimWkrwg3PZJJw4TgXFNw_kDOIQTImLR0DS2Xk9K-RLjQwmQZ48oozoSFmD3vaQqOcy8XmVV2HW01K5WCrXIJKF3ayUWVQvDBGGfgBK-jF_USptN6qErq_HpOuHx1j7tfo8cLXdxAy_PZhX2DHpkf83E120RLYiQ3Pdi_ARzFaRA |
| linkProvider | IEEE |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=HCI%E2%88%A72+framework%3A+a+software+framework+for+multimodal+human-computer+interaction+systems&rft.jtitle=IEEE+transactions+on+cybernetics&rft.au=Shen%2C+Jie&rft.au=Pantic%2C+Maja&rft.date=2013-12-01&rft.issn=2168-2275&rft.eissn=2168-2275&rft.volume=43&rft.issue=6&rft.spage=1593&rft_id=info:doi/10.1109%2FTCYB.2013.2271563&rft.externalDBID=NO_FULL_TEXT |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2168-2275&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2168-2275&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2168-2275&client=summon |