Learning deep networks with crowdsourcing for relevance evaluation
In this paper, we propose a novel relevance evaluation method using labels collected from crowdsourcing. The proposed method not only predicts the relevance between query texts and responses in information retrieval systems but also performs the label aggregation tasks simultaneously. It first merge...
Saved in:
| Published in: | EURASIP journal on wireless communications and networking Vol. 2020; no. 1; pp. 1 - 11 |
|---|---|
| Main Authors: | , , , , , , |
| Format: | Journal Article |
| Language: | English |
| Published: |
Cham
Springer International Publishing
25.04.2020
Springer Nature B.V SpringerOpen |
| Subjects: | |
| ISSN: | 1687-1499, 1687-1472, 1687-1499 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | In this paper, we propose a novel relevance evaluation method using labels collected from crowdsourcing. The proposed method not only predicts the relevance between query texts and responses in information retrieval systems but also performs the label aggregation tasks simultaneously. It first merges two kinds of heterogeneous data (i.e., image and query text) and constructs a CNN-like deep neural network. Then, on the top of its softmax layer, an additional layer was built to model the crowd workers. Finally, classification models for relevance prediction and aggregated labels for training examples can be simultaneously learned from noisy labels. Experimental results show that the proposed method significantly outperforms other state-of-the-art methods on a real-world dataset. |
|---|---|
| AbstractList | In this paper, we propose a novel relevance evaluation method using labels collected from crowdsourcing. The proposed method not only predicts the relevance between query texts and responses in information retrieval systems but also performs the label aggregation tasks simultaneously. It first merges two kinds of heterogeneous data (i.e., image and query text) and constructs a CNN-like deep neural network. Then, on the top of its softmax layer, an additional layer was built to model the crowd workers. Finally, classification models for relevance prediction and aggregated labels for training examples can be simultaneously learned from noisy labels. Experimental results show that the proposed method significantly outperforms other state-of-the-art methods on a real-world dataset. Abstract In this paper, we propose a novel relevance evaluation method using labels collected from crowdsourcing. The proposed method not only predicts the relevance between query texts and responses in information retrieval systems but also performs the label aggregation tasks simultaneously. It first merges two kinds of heterogeneous data (i.e., image and query text) and constructs a CNN-like deep neural network. Then, on the top of its softmax layer, an additional layer was built to model the crowd workers. Finally, classification models for relevance prediction and aggregated labels for training examples can be simultaneously learned from noisy labels. Experimental results show that the proposed method significantly outperforms other state-of-the-art methods on a real-world dataset. |
| ArticleNumber | 82 |
| Author | Feng, Xinqi Zhang, Jing Cao, Qi Li, Qianmu Shen, Haiyuan Wu, Ming Yin, Xiaochun |
| Author_xml | – sequence: 1 givenname: Ming surname: Wu fullname: Wu, Ming organization: School of Computer Science and Engineering, Nanjing University of Science and Technology – sequence: 2 givenname: Xiaochun surname: Yin fullname: Yin, Xiaochun organization: Facility Horticulture Laboratory of Universities in Shandong, WeiFang University of Science & Technology – sequence: 3 givenname: Qianmu surname: Li fullname: Li, Qianmu email: qianmu@njust.edu.cn organization: School of Cyber Science and Engineering, Nanjing University of Science and Technology, Intelligent Manufacturing Department, Wuyi University – sequence: 4 givenname: Jing surname: Zhang fullname: Zhang, Jing organization: School of Computer Science and Engineering, Nanjing University of Science and Technology – sequence: 5 givenname: Xinqi surname: Feng fullname: Feng, Xinqi organization: Center Of Informationization Construction And Management, Nanjing Sport Institute – sequence: 6 givenname: Qi surname: Cao fullname: Cao, Qi organization: Academy of Science and Technology Strategic Consulting, Chinese Academy of Science – sequence: 7 givenname: Haiyuan surname: Shen fullname: Shen, Haiyuan organization: Jiangsu Zhongtian Internet Technology Co, Ltd |
| BookMark | eNp9UMtOwzAQtBBIQOAHOEXiHPCrjn2EikelSlzgbG1sp6QEu9gpFX-PaUAgDj3t7mhmNDvHaN8H7xA6I_iCECkuE2GCyQpTXGEiVF3RPXREhKwrwpXa_7MfouOUlhgzxhU9QtdzB9F3flFa51ald8MmxJdUbrrhuTQxbGwK62i-CG2IZXS9ewdvXJlHv4ahC_4EHbTQJ3f6PQv0dHvzOL2v5g93s-nVvDKcqqECYsA0QG3DhZQTBpPWMWoEtvmytDYgXKs4J43FDGogkjIGkmYUlCXACjQbfW2ApV7F7hXihw7Q6S0Q4kJDHDrTOw0T1UhjaqCC8AY3DViBubQm3xnn2et89FrF8LZ2adDL_KbP8TVligvGJrnQAsmRlYtIKbpWm27Y_jxE6HpNsP5qX4_t69y-3rafPQpE_0l_Au8UsVGUMtkvXPxNtUP1CUSmmjk |
| CitedBy_id | crossref_primary_10_3390_rs13061211 crossref_primary_10_1108_AJIM_08_2021_0215 crossref_primary_10_3390_agriculture11030273 |
| Cites_doi | 10.1109/TKDE.2015.2504974 10.1109/CVPR.2015.7298935 10.1109/JIOT.2019.2911669 10.1109/TMM.2020.2976573 10.1109/TSE.2016.2624293 10.1016/j.neucom.2019.01.075 10.1109/CLOUD.2015.40 10.1007/s11036-019-01448-8 10.1109/TCSS.2019.2906925 10.1145/2093346.2093356 10.1109/TSC.2019.2963301 10.1016/j.comcom.2019.10.012 10.1007/978-3-030-33702-5_8 10.1016/j.future.2018.02.050 10.1109/TCAD.2018.2883993 10.1016/j.future.2019.01.012 10.1109/TC.2019.2935042 10.1109/JBHI.2019.2891526 10.1145/1480506.1480508 10.3115/1613715.1613751 10.1007/s11227-019-03027-w 10.26599/BDMA.2018.9020028 10.1109/TMI.2016.2528120 10.1007/s11227-019-03011-4 10.1145/2557642.2563670 10.26599/BDMA.2018.9020031 10.1109/TSE.2017.2681667 10.1186/s13638-015-0498-8 10.2307/2346806 10.1145/3377876 10.1007/s11280-019-00684-y 10.1007/s11036-019-01445-x |
| ContentType | Journal Article |
| Copyright | The Author(s) 2020 The Author(s) 2020. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
| Copyright_xml | – notice: The Author(s) 2020 – notice: The Author(s) 2020. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
| DBID | C6C AAYXX CITATION 3V. 7SC 7SP 7XB 8AL 8FD 8FE 8FG 8FK ABUWG AFKRA ARAPS AZQEC BENPR BGLVJ CCPQU DWQXO GNUQQ HCIFZ JQ2 K7- L7M L~C L~D M0N P5Z P62 PHGZM PHGZT PIMPY PKEHL PQEST PQGLB PQQKQ PQUKI PRINS Q9U DOA |
| DOI | 10.1186/s13638-020-01697-2 |
| DatabaseName | Springer Nature Link CrossRef ProQuest Central (Corporate) Computer and Information Systems Abstracts Electronics & Communications Abstracts ProQuest Central (purchase pre-March 2016) Computing Database (Alumni Edition) Technology Research Database ProQuest SciTech Collection ProQuest Technology Collection ProQuest Central (Alumni) (purchase pre-March 2016) ProQuest Central (Alumni) ProQuest Central UK/Ireland Advanced Technologies & Computer Science Collection ProQuest Central Essentials ProQuest Central ProQuest Technology Collection ProQuest One ProQuest Central Korea ProQuest Central Student SciTech Premium Collection ProQuest Computer Science Collection Computer Science Database Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional Computing Database Advanced Technologies & Aerospace Database ProQuest Advanced Technologies & Aerospace Collection Proquest Central Premium ProQuest One Academic (New) ProQuest Publicly Available Content Database ProQuest One Academic Middle East (New) ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Applied & Life Sciences ProQuest One Academic (retired) ProQuest One Academic UKI Edition ProQuest Central China ProQuest Central Basic DOAJ Directory of Open Access Journals |
| DatabaseTitle | CrossRef Publicly Available Content Database Computer Science Database ProQuest Central Student Technology Collection Technology Research Database Computer and Information Systems Abstracts – Academic ProQuest One Academic Middle East (New) ProQuest Advanced Technologies & Aerospace Collection ProQuest Central Essentials ProQuest Computer Science Collection Computer and Information Systems Abstracts ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College ProQuest Central China ProQuest Central ProQuest One Applied & Life Sciences ProQuest Central Korea ProQuest Central (New) Advanced Technologies Database with Aerospace Advanced Technologies & Aerospace Collection ProQuest Computing ProQuest Central Basic ProQuest Computing (Alumni Edition) ProQuest One Academic Eastern Edition Electronics & Communications Abstracts ProQuest Technology Collection ProQuest SciTech Collection Computer and Information Systems Abstracts Professional Advanced Technologies & Aerospace Database ProQuest One Academic UKI Edition ProQuest One Academic ProQuest One Academic (New) ProQuest Central (Alumni) |
| DatabaseTitleList | Publicly Available Content Database CrossRef |
| Database_xml | – sequence: 1 dbid: DOA name: DOAJ Directory of Open Access Journals url: https://www.doaj.org/ sourceTypes: Open Website – sequence: 2 dbid: PIMPY name: ProQuest Publicly Available Content Database url: http://search.proquest.com/publiccontent sourceTypes: Aggregation Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 1687-1499 |
| EndPage | 11 |
| ExternalDocumentID | oai_doaj_org_article_a59b8cc7a2614b0bbad6048dc2618cc4 10_1186_s13638_020_01697_2 |
| GroupedDBID | -A0 .4S .DC 0R~ 29G 2WC 3V. 4.4 40G 5GY 5VS 6OB 8FE 8FG 8R4 8R5 AAFWJ AAJSJ AAKKN AAKPC ABDBF ABEEZ ABFTD ABUWG ACACY ACGFS ACUHS ACULB ADBBV ADDVE ADINQ ADMLS AENEX AFGXO AFKRA AFPKN AHBYD AHYZX ALMA_UNASSIGNED_HOLDINGS AMKLP ARAPS ARCSS AZQEC BCNDV BENPR BGLVJ BPHCQ C24 C6C CCPQU CS3 DU5 DWQXO E3Z EAD EAP EAS EBLON EBS EDO EMK ESX GNUQQ GROUPED_DOAJ HCIFZ HZ~ I-F K6V K7- KQ8 M0N M~E OK1 P2P P62 PIMPY PQQKQ PROAC Q2X RHU RNS RSV SEG SOJ TUS U2A XSB AASML AAYXX CITATION OVT 7SC 7SP 7XB 8AL 8FD 8FK JQ2 L7M L~C L~D PHGZM PHGZT PKEHL PQEST PQGLB PQUKI PRINS Q9U |
| ID | FETCH-LOGICAL-c429t-a1cacba2db468853a5fe32c60d885d27ca6ef9441bd03a7a18233a826efa9d1a3 |
| IEDL.DBID | BENPR |
| ISICitedReferencesCount | 3 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000528580000001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1687-1499 1687-1472 |
| IngestDate | Tue Oct 14 18:18:30 EDT 2025 Sat Oct 11 05:46:36 EDT 2025 Sat Nov 29 01:43:53 EST 2025 Tue Nov 18 20:55:44 EST 2025 Fri Feb 21 02:35:32 EST 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 1 |
| Keywords | Crowdsourcing Deep learning Information retrieval Relevance evaluation |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c429t-a1cacba2db468853a5fe32c60d885d27ca6ef9441bd03a7a18233a826efa9d1a3 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| OpenAccessLink | https://www.proquest.com/docview/2394633536?pq-origsite=%requestingapplication% |
| PQID | 2394633536 |
| PQPubID | 237293 |
| PageCount | 11 |
| ParticipantIDs | doaj_primary_oai_doaj_org_article_a59b8cc7a2614b0bbad6048dc2618cc4 proquest_journals_2394633536 crossref_citationtrail_10_1186_s13638_020_01697_2 crossref_primary_10_1186_s13638_020_01697_2 springer_journals_10_1186_s13638_020_01697_2 |
| PublicationCentury | 2000 |
| PublicationDate | 2020-04-25 |
| PublicationDateYYYYMMDD | 2020-04-25 |
| PublicationDate_xml | – month: 04 year: 2020 text: 2020-04-25 day: 25 |
| PublicationDecade | 2020 |
| PublicationPlace | Cham |
| PublicationPlace_xml | – name: Cham – name: New York |
| PublicationTitle | EURASIP journal on wireless communications and networking |
| PublicationTitleAbbrev | J Wireless Com Network |
| PublicationYear | 2020 |
| Publisher | Springer International Publishing Springer Nature B.V SpringerOpen |
| Publisher_xml | – name: Springer International Publishing – name: Springer Nature B.V – name: SpringerOpen |
| References | KumarS.SinghM.Big data analytics for healthcare industry: impact, applications, and toolsBig Data Min. Analytics201821485710.26599/BDMA.2018.9020031 XuX.MoR.DaiF.LinW.WanS.DouW.Dynamic resource provisioning with fault tolerance for data-intensive meteorological workflows in cloudIEEE Trans. Ind. Inform.2019201911 GaoZ.LiY.WanS.Exploring deep learning for view-based 3D model retrievalACM Trans. Multimed. Comput. Commun. Appl. (TOMM)202016112110.1145/3377876 WhitehillJ.WuT. -f.BergsmaJ.MovellanJ. R.RuvoloP. L.Whose vote should count more: optimal integration of labels from labelers of unknown expertiseAdvances in Neural Information Processing Systems20092220352043 DawidA. P.SkeneA. M.Maximum likelihood estimation of observer error-rates using the em algorithmAppl. Stat.1979281202810.2307/2346806 WanS.LiX.XueY.LinW.XuX.Efficient computation offloading for Internet of Vehicles in edge computing-assisted 5G networksJ. Supercomput.20197510130https://doi.org/10.1007/s11227-019-03011-4 WangY.HeQ.YeD.YangY.Formulating criticality-based cost-effective fault tolerance strategies for multi-tenant service-based systemsIEEE Trans. Softw. Eng.201744329130710.1109/TSE.2017.2681667 ZhouJ.HuX. S.MaY.SunJ.WeiT.HuS.Improving availability of multicore real-time systems suffering both permanent and transient faultsIEEE Trans. Comput.20196812178518010715909310.1109/TC.2019.2935042 QiL.ChenY.YuanY.FuS.ZhangX.XuX.A QOS-aware virtual machine scheduling method for energy conservation in cloud-based cyber-physical systemsWorld Wide Web20202311275129710.1007/s11280-019-00684-yhttps://doi.org/10.1007/s11280-019-00684-y WangB.MaH.WangX.DengG.YangY.WanS.Vulnerability assessment method for cyber-physical system considering node heterogeneityJ. Supercomput.20197510121https://doi.org/10.1007/s11227-019-03027-w ZhangY.ChenX.ZhouD.JordanM. I.Spectral methods meet em: a provably optimal algorithm for crowdsourcingJ. Mach. Learn. Res.20161713537358035435081367.68241 LiuH.KouH.YanC.QiL.Link prediction in paper citation network to construct paper correlation graphEURASIP J. Wirel. Commun. Netw.20192019111210.1186/s13638-015-0498-8 B. Ionescu, A. -L. Radu, M. Menéndez, H. Müller, A. Popescu, B. Loni, Div400: a social image retrieval result diversification dataset. Proceedings of the 5th ACM Multimedia Systems Conference, 29–34 (2014). P. Lai, Q. He, G. Cui, X. Xia, M. Abdelrazek, F. Chen, J. Hosking, J. Grundy, Y. Yang, Edge user allocation with dynamic quality of service. International Conference on Service-Oriented Computing, 86–101 (2019). ZhaoY.LiH.WanS.SekuboyinaA.HuX.TettehG.PiraudM.MenzeB.Knowledge-aided convolutional neural network for small organ segmentationIEEE J. Biomed. Health Inform.20192341363137310.1109/JBHI.2019.2891526 WanS.GuZ.NiQ.Cognitive computing and wireless communications on the edge for healthcare service robotsComput. Commun.20191499910610.1016/j.comcom.2019.10.012 XuX.XueY.QiL.YuanY.ZhangX.UmerT.WanS.An edge computing-enabled computation offloading method with privacy preservation for internet of connected vehiclesFutur. Gener. Comput. Syst.2019968910010.1016/j.future.2019.01.012 KongC.LuoG.TianL.CaoX.Disseminating authorized content via data analysis in opportunistic social networksBig Data Min. Analytics201821122410.26599/BDMA.2018.9020028 SimonyanK.ZissermanA.Very deep convolutional networks for large-scale image recognitionarXiv preprint arXiv:1409.155620142014114 O. Vinyals, A. Toshev, S. Bengio, D. Erhan, Show and tell: a neural image caption generator. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3156–3164 (2015). QiL.ZhangX.DouW.HuC.YangC.ChenJ.A two-stage locality-sensitive hashing based approach for privacy-preserving mobile service recommendation in cross-platform edge environmentFutur. Gener. Comput. Syst.20188863664310.1016/j.future.2018.02.050 XuY.QiL.DouW.YuJ.Privacy-preserving and scalable service recommendation based on simhash in a distributed cloud environmentComplexity2017201719 R. Snow, B. O’Connor, D. Jurafsky, A. Y. Ng, Cheap and fast—but is it good?: evaluating non-expert annotations for natural language tasks (Association for Computational Linguistics). Proceedings of the Conference on Empirical Methods in Natural Language Processing, 254–263 (2008). MikolovT.ChenK.CorradoG.DeanJ.Efficient estimation of word representations in vector spacearXiv preprint arXiv:1301.378120132014112 RongX.word2vec parameter learning explainedarXiv preprint arXiv:1411.273820142014121 QiL.HeQ.ChenF.DouW.WanS.ZhangX.XuX.Finding all you need: Web APIs recommendation in web of things through keywords searchIEEE Trans. Comput. Soc. Syst.2019651063107210.1109/TCSS.2019.2906925 HoweJ.The rise of crowdsourcingWired Mag.200614614 DingS.QuS.XiY.WanS.Stimulus-driven and concept-driven analysis for image caption generationNeurocomputing201935111010.1016/j.neucom.2019.01.075 XuX.HeC.XuZ.QiL.WanS.BhuiyanM. Z. A.Joint optimization of offloading utility and privacy for edge computing enabled IoTIEEE Internet Things J.2019201911 ZhangJ.ShengV. S.NicholsonB. A.WuX.Ceka: a tool for mining the wisdom of crowdsJ. Mach. Learn. Res.2015161285328583450526 GongW.QiL.XuY.Privacy-aware multidimensional mobile service quality prediction and recommendation in distributed fog environmentWirel. Commun. Mob. Comput.2018201818 ZhouJ.SunJ.ZhouX.WeiT.ChenM.HuS.HuX. S.Resource management for improving soft-error and lifetime reliability of real-time MPSoCsIEEE Trans. Comput. Aided Des. Integr. Circ. Syst.2018381222152810.1109/TCAD.2018.2883993 X. Xu, X. Liu, Z. Xu, C. Wang, S. Wan, X. Yang, Joint optimization of resource utilization and load balance with privacy preservation for edge services in 5G networks. Mob. Netw. Appl., 1–12 (2019). HeQ.ZhouR.ZhangX.WangY.YeD.ChenF.GrundyJ. C.YangY.Keyword search for building service-based systemsIEEE Trans. Softw. Eng.201643765867410.1109/TSE.2016.2624293 F. Chollet, et al., Keras (2015). https://github.com/fchollet/keras. 10 Accessed 13 June 2015. Z. Junlong, S. Jin, C. Peijin, L. Zhe, W. Tongquan, Z. Xiumin, H. Shiyan, Security-critical energy-aware task scheduling for heterogeneous real-time MPSoCs in IoT. IEEE Trans. Serv. Comput. (TSC) (2019). https://doi.org/10.1109/TSC.2019.2963301. Q. He, J. Han, F. Chen, Y. Wang, R. Vasa, Y. Yang, H. Jin, in 2015 IEEE 8th International Conference on Cloud Computing. QOS-aware service selection for customisable multi-tenant service-based systems: Maturity and approaches (IEEE, 2015), pp. 237–244. AlbarqouniS.BaurC.AchillesF.BelagiannisV.DemirciS.NavabN.Aggnet: deep learning from crowds for mitosis detection in breast cancer histology imagesIEEE Trans. Med. Imaging20163551313132110.1109/TMI.2016.2528120 WanS.XiaY.QiL.YangY. -H.AtiquzzamanM.Automated colorization of a grayscale image with seed points propagationIEEE Trans. Multimed.202020201110.1109/TMM.2020.2976573 WanS.QiL.XuX.TongC.GuZ.Deep learning models for real-time human activity recognition with smartphonesMob. Networks Appl.20197512113https://doi.org/10.1007/s11036-019-01445-x ZhangJ.ShengV. S.WuJ.WuX.Multi-class ground truth inference in crowdsourcing with clusteringIEEE Trans. Knowl. Data Eng.20162841080108510.1109/TKDE.2015.2504974 WanS.GoudosS.Faster R-CNN for multi-class fruit detection using a robotic vision systemComput. Netw.2019168116 GaoZ.XuanH. -Z.ZhangH.WanS.ChooK. -K. R.Adaptive fusion and category-level dictionary learning model for multi-view human action recognitionIEEE Internet Things J.201969280929310.1109/JIOT.2019.2911669 O. Alonso, D. E. Rose, B. Stewart, in ACM SigIR Forum, vol. 42. Crowdsourcing for relevance evaluation (ACM, 2008), pp. 9–15. LeaseM.YilmazE.Crowdsourcing for information retrievalACM SIGIR Forum2012452667510.1145/2093346.2093356 1697_CR16 1697_CR38 Y. Wang (1697_CR5) 2017; 44 1697_CR39 S. Ding (1697_CR35) 2019; 351 S. Wan (1697_CR33) 2019; 149 L. Qi (1697_CR42) 2019; 6 Q. He (1697_CR6) 2016; 43 1697_CR1 S. Kumar (1697_CR3) 2018; 2 S. Albarqouni (1697_CR23) 2016; 35 T. Mikolov (1697_CR30) 2013; 2014 S. Wan (1697_CR32) 2019; 75 1697_CR9 Z. Gao (1697_CR27) 2020; 16 J. Zhang (1697_CR37) 2015; 16 W. Gong (1697_CR44) 2018; 2018 A. P. Dawid (1697_CR17) 1979; 28 1697_CR13 1697_CR36 1697_CR15 J. Whitehill (1697_CR20) 2009; 22 J. Howe (1697_CR7) 2006; 14 L. Qi (1697_CR8) 2020; 23 Z. Gao (1697_CR26) 2019; 6 M. Lease (1697_CR12) 2012; 45 S. Wan (1697_CR24) 2019; 168 X. Xu (1697_CR40) 2019; 2019 L. Qi (1697_CR45) 2018; 88 Y. Xu (1697_CR10) 2017; 2017 C. Kong (1697_CR2) 2018; 2 X. Rong (1697_CR31) 2014; 2014 S. Wan (1697_CR25) 2019; 75 K. Simonyan (1697_CR29) 2014; 2014 H. Liu (1697_CR41) 2019; 2019 X. Xu (1697_CR14) 2019; 96 X. Xu (1697_CR11) 2019; 2019 J. Zhang (1697_CR22) 2016; 28 B. Wang (1697_CR4) 2019; 75 J. Zhou (1697_CR19) 2018; 38 1697_CR43 J. Zhou (1697_CR18) 2019; 68 Y. Zhang (1697_CR21) 2016; 17 Y. Zhao (1697_CR28) 2019; 23 S. Wan (1697_CR34) 2020; 2020 |
| References_xml | – reference: GongW.QiL.XuY.Privacy-aware multidimensional mobile service quality prediction and recommendation in distributed fog environmentWirel. Commun. Mob. Comput.2018201818 – reference: WhitehillJ.WuT. -f.BergsmaJ.MovellanJ. R.RuvoloP. L.Whose vote should count more: optimal integration of labels from labelers of unknown expertiseAdvances in Neural Information Processing Systems20092220352043 – reference: AlbarqouniS.BaurC.AchillesF.BelagiannisV.DemirciS.NavabN.Aggnet: deep learning from crowds for mitosis detection in breast cancer histology imagesIEEE Trans. Med. Imaging20163551313132110.1109/TMI.2016.2528120 – reference: F. Chollet, et al., Keras (2015). https://github.com/fchollet/keras. 10 Accessed 13 June 2015. – reference: KumarS.SinghM.Big data analytics for healthcare industry: impact, applications, and toolsBig Data Min. Analytics201821485710.26599/BDMA.2018.9020031 – reference: Z. Junlong, S. Jin, C. Peijin, L. Zhe, W. Tongquan, Z. Xiumin, H. Shiyan, Security-critical energy-aware task scheduling for heterogeneous real-time MPSoCs in IoT. IEEE Trans. Serv. Comput. (TSC) (2019). https://doi.org/10.1109/TSC.2019.2963301. – reference: XuX.XueY.QiL.YuanY.ZhangX.UmerT.WanS.An edge computing-enabled computation offloading method with privacy preservation for internet of connected vehiclesFutur. Gener. Comput. Syst.2019968910010.1016/j.future.2019.01.012 – reference: O. Alonso, D. E. Rose, B. Stewart, in ACM SigIR Forum, vol. 42. Crowdsourcing for relevance evaluation (ACM, 2008), pp. 9–15. – reference: ZhouJ.HuX. S.MaY.SunJ.WeiT.HuS.Improving availability of multicore real-time systems suffering both permanent and transient faultsIEEE Trans. Comput.20196812178518010715909310.1109/TC.2019.2935042 – reference: ZhouJ.SunJ.ZhouX.WeiT.ChenM.HuS.HuX. S.Resource management for improving soft-error and lifetime reliability of real-time MPSoCsIEEE Trans. Comput. Aided Des. Integr. Circ. Syst.2018381222152810.1109/TCAD.2018.2883993 – reference: WanS.GuZ.NiQ.Cognitive computing and wireless communications on the edge for healthcare service robotsComput. Commun.20191499910610.1016/j.comcom.2019.10.012 – reference: Q. He, J. Han, F. Chen, Y. Wang, R. Vasa, Y. Yang, H. Jin, in 2015 IEEE 8th International Conference on Cloud Computing. QOS-aware service selection for customisable multi-tenant service-based systems: Maturity and approaches (IEEE, 2015), pp. 237–244. – reference: HoweJ.The rise of crowdsourcingWired Mag.200614614 – reference: LeaseM.YilmazE.Crowdsourcing for information retrievalACM SIGIR Forum2012452667510.1145/2093346.2093356 – reference: WanS.GoudosS.Faster R-CNN for multi-class fruit detection using a robotic vision systemComput. Netw.2019168116 – reference: GaoZ.XuanH. -Z.ZhangH.WanS.ChooK. -K. R.Adaptive fusion and category-level dictionary learning model for multi-view human action recognitionIEEE Internet Things J.201969280929310.1109/JIOT.2019.2911669 – reference: O. Vinyals, A. Toshev, S. Bengio, D. Erhan, Show and tell: a neural image caption generator. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3156–3164 (2015). – reference: ZhaoY.LiH.WanS.SekuboyinaA.HuX.TettehG.PiraudM.MenzeB.Knowledge-aided convolutional neural network for small organ segmentationIEEE J. Biomed. Health Inform.20192341363137310.1109/JBHI.2019.2891526 – reference: ZhangJ.ShengV. S.NicholsonB. A.WuX.Ceka: a tool for mining the wisdom of crowdsJ. Mach. Learn. Res.2015161285328583450526 – reference: X. Xu, X. Liu, Z. Xu, C. Wang, S. Wan, X. Yang, Joint optimization of resource utilization and load balance with privacy preservation for edge services in 5G networks. Mob. Netw. Appl., 1–12 (2019). – reference: P. Lai, Q. He, G. Cui, X. Xia, M. Abdelrazek, F. Chen, J. Hosking, J. Grundy, Y. Yang, Edge user allocation with dynamic quality of service. International Conference on Service-Oriented Computing, 86–101 (2019). – reference: ZhangJ.ShengV. S.WuJ.WuX.Multi-class ground truth inference in crowdsourcing with clusteringIEEE Trans. Knowl. Data Eng.20162841080108510.1109/TKDE.2015.2504974 – reference: SimonyanK.ZissermanA.Very deep convolutional networks for large-scale image recognitionarXiv preprint arXiv:1409.155620142014114 – reference: QiL.ZhangX.DouW.HuC.YangC.ChenJ.A two-stage locality-sensitive hashing based approach for privacy-preserving mobile service recommendation in cross-platform edge environmentFutur. Gener. Comput. Syst.20188863664310.1016/j.future.2018.02.050 – reference: RongX.word2vec parameter learning explainedarXiv preprint arXiv:1411.273820142014121 – reference: QiL.HeQ.ChenF.DouW.WanS.ZhangX.XuX.Finding all you need: Web APIs recommendation in web of things through keywords searchIEEE Trans. Comput. Soc. Syst.2019651063107210.1109/TCSS.2019.2906925 – reference: WangB.MaH.WangX.DengG.YangY.WanS.Vulnerability assessment method for cyber-physical system considering node heterogeneityJ. Supercomput.20197510121https://doi.org/10.1007/s11227-019-03027-w – reference: XuX.MoR.DaiF.LinW.WanS.DouW.Dynamic resource provisioning with fault tolerance for data-intensive meteorological workflows in cloudIEEE Trans. Ind. Inform.2019201911 – reference: WanS.QiL.XuX.TongC.GuZ.Deep learning models for real-time human activity recognition with smartphonesMob. Networks Appl.20197512113https://doi.org/10.1007/s11036-019-01445-x – reference: B. Ionescu, A. -L. Radu, M. Menéndez, H. Müller, A. Popescu, B. Loni, Div400: a social image retrieval result diversification dataset. Proceedings of the 5th ACM Multimedia Systems Conference, 29–34 (2014). – reference: DawidA. P.SkeneA. M.Maximum likelihood estimation of observer error-rates using the em algorithmAppl. Stat.1979281202810.2307/2346806 – reference: LiuH.KouH.YanC.QiL.Link prediction in paper citation network to construct paper correlation graphEURASIP J. Wirel. Commun. Netw.20192019111210.1186/s13638-015-0498-8 – reference: GaoZ.LiY.WanS.Exploring deep learning for view-based 3D model retrievalACM Trans. Multimed. Comput. Commun. Appl. (TOMM)202016112110.1145/3377876 – reference: WangY.HeQ.YeD.YangY.Formulating criticality-based cost-effective fault tolerance strategies for multi-tenant service-based systemsIEEE Trans. Softw. Eng.201744329130710.1109/TSE.2017.2681667 – reference: QiL.ChenY.YuanY.FuS.ZhangX.XuX.A QOS-aware virtual machine scheduling method for energy conservation in cloud-based cyber-physical systemsWorld Wide Web20202311275129710.1007/s11280-019-00684-yhttps://doi.org/10.1007/s11280-019-00684-y – reference: WanS.LiX.XueY.LinW.XuX.Efficient computation offloading for Internet of Vehicles in edge computing-assisted 5G networksJ. Supercomput.20197510130https://doi.org/10.1007/s11227-019-03011-4 – reference: DingS.QuS.XiY.WanS.Stimulus-driven and concept-driven analysis for image caption generationNeurocomputing201935111010.1016/j.neucom.2019.01.075 – reference: MikolovT.ChenK.CorradoG.DeanJ.Efficient estimation of word representations in vector spacearXiv preprint arXiv:1301.378120132014112 – reference: KongC.LuoG.TianL.CaoX.Disseminating authorized content via data analysis in opportunistic social networksBig Data Min. Analytics201821122410.26599/BDMA.2018.9020028 – reference: XuY.QiL.DouW.YuJ.Privacy-preserving and scalable service recommendation based on simhash in a distributed cloud environmentComplexity2017201719 – reference: WanS.XiaY.QiL.YangY. -H.AtiquzzamanM.Automated colorization of a grayscale image with seed points propagationIEEE Trans. Multimed.202020201110.1109/TMM.2020.2976573 – reference: HeQ.ZhouR.ZhangX.WangY.YeD.ChenF.GrundyJ. C.YangY.Keyword search for building service-based systemsIEEE Trans. Softw. Eng.201643765867410.1109/TSE.2016.2624293 – reference: R. Snow, B. O’Connor, D. Jurafsky, A. Y. Ng, Cheap and fast—but is it good?: evaluating non-expert annotations for natural language tasks (Association for Computational Linguistics). Proceedings of the Conference on Empirical Methods in Natural Language Processing, 254–263 (2008). – reference: ZhangY.ChenX.ZhouD.JordanM. I.Spectral methods meet em: a provably optimal algorithm for crowdsourcingJ. Mach. Learn. Res.20161713537358035435081367.68241 – reference: XuX.HeC.XuZ.QiL.WanS.BhuiyanM. Z. A.Joint optimization of offloading utility and privacy for edge computing enabled IoTIEEE Internet Things J.2019201911 – volume: 28 start-page: 1080 issue: 4 year: 2016 ident: 1697_CR22 publication-title: IEEE Trans. Knowl. Data Eng. doi: 10.1109/TKDE.2015.2504974 – ident: 1697_CR43 doi: 10.1109/CVPR.2015.7298935 – volume: 6 start-page: 9280 year: 2019 ident: 1697_CR26 publication-title: IEEE Internet Things J. doi: 10.1109/JIOT.2019.2911669 – volume: 2014 start-page: 1 year: 2013 ident: 1697_CR30 publication-title: arXiv preprint arXiv:1301.3781 – volume: 2020 start-page: 1 year: 2020 ident: 1697_CR34 publication-title: IEEE Trans. Multimed. doi: 10.1109/TMM.2020.2976573 – volume: 2014 start-page: 1 year: 2014 ident: 1697_CR31 publication-title: arXiv preprint arXiv:1411.2738 – volume: 2014 start-page: 1 year: 2014 ident: 1697_CR29 publication-title: arXiv preprint arXiv:1409.1556 – volume: 43 start-page: 658 issue: 7 year: 2016 ident: 1697_CR6 publication-title: IEEE Trans. Softw. Eng. doi: 10.1109/TSE.2016.2624293 – volume: 351 start-page: 1 year: 2019 ident: 1697_CR35 publication-title: Neurocomputing doi: 10.1016/j.neucom.2019.01.075 – ident: 1697_CR9 doi: 10.1109/CLOUD.2015.40 – ident: 1697_CR39 doi: 10.1007/s11036-019-01448-8 – ident: 1697_CR36 – volume: 6 start-page: 1063 issue: 5 year: 2019 ident: 1697_CR42 publication-title: IEEE Trans. Comput. Soc. Syst. doi: 10.1109/TCSS.2019.2906925 – volume: 45 start-page: 66 issue: 2 year: 2012 ident: 1697_CR12 publication-title: ACM SIGIR Forum doi: 10.1145/2093346.2093356 – volume: 16 start-page: 2853 issue: 1 year: 2015 ident: 1697_CR37 publication-title: J. Mach. Learn. Res. – volume: 2019 start-page: 1 year: 2019 ident: 1697_CR11 publication-title: IEEE Trans. Ind. Inform. – ident: 1697_CR13 doi: 10.1109/TSC.2019.2963301 – volume: 149 start-page: 99 year: 2019 ident: 1697_CR33 publication-title: Comput. Commun. doi: 10.1016/j.comcom.2019.10.012 – ident: 1697_CR15 doi: 10.1007/978-3-030-33702-5_8 – volume: 88 start-page: 636 year: 2018 ident: 1697_CR45 publication-title: Futur. Gener. Comput. Syst. doi: 10.1016/j.future.2018.02.050 – volume: 38 start-page: 2215 issue: 12 year: 2018 ident: 1697_CR19 publication-title: IEEE Trans. Comput. Aided Des. Integr. Circ. Syst. doi: 10.1109/TCAD.2018.2883993 – volume: 22 start-page: 2035 year: 2009 ident: 1697_CR20 publication-title: Advances in Neural Information Processing Systems – volume: 168 start-page: 1 year: 2019 ident: 1697_CR24 publication-title: Comput. Netw. – volume: 17 start-page: 3537 issue: 1 year: 2016 ident: 1697_CR21 publication-title: J. Mach. Learn. Res. – volume: 96 start-page: 89 year: 2019 ident: 1697_CR14 publication-title: Futur. Gener. Comput. Syst. doi: 10.1016/j.future.2019.01.012 – volume: 68 start-page: 1785 issue: 12 year: 2019 ident: 1697_CR18 publication-title: IEEE Trans. Comput. doi: 10.1109/TC.2019.2935042 – volume: 23 start-page: 1363 issue: 4 year: 2019 ident: 1697_CR28 publication-title: IEEE J. Biomed. Health Inform. doi: 10.1109/JBHI.2019.2891526 – ident: 1697_CR1 doi: 10.1145/1480506.1480508 – ident: 1697_CR16 doi: 10.3115/1613715.1613751 – volume: 75 start-page: 1 issue: 10 year: 2019 ident: 1697_CR4 publication-title: J. Supercomput. doi: 10.1007/s11227-019-03027-w – volume: 2018 start-page: 1 year: 2018 ident: 1697_CR44 publication-title: Wirel. Commun. Mob. Comput. – volume: 2 start-page: 12 issue: 1 year: 2018 ident: 1697_CR2 publication-title: Big Data Min. Analytics doi: 10.26599/BDMA.2018.9020028 – volume: 2017 start-page: 1 year: 2017 ident: 1697_CR10 publication-title: Complexity – volume: 35 start-page: 1313 issue: 5 year: 2016 ident: 1697_CR23 publication-title: IEEE Trans. Med. Imaging doi: 10.1109/TMI.2016.2528120 – volume: 75 start-page: 1 issue: 10 year: 2019 ident: 1697_CR32 publication-title: J. Supercomput. doi: 10.1007/s11227-019-03011-4 – ident: 1697_CR38 doi: 10.1145/2557642.2563670 – volume: 2 start-page: 48 issue: 1 year: 2018 ident: 1697_CR3 publication-title: Big Data Min. Analytics doi: 10.26599/BDMA.2018.9020031 – volume: 44 start-page: 291 issue: 3 year: 2017 ident: 1697_CR5 publication-title: IEEE Trans. Softw. Eng. doi: 10.1109/TSE.2017.2681667 – volume: 2019 start-page: 1 issue: 1 year: 2019 ident: 1697_CR41 publication-title: EURASIP J. Wirel. Commun. Netw. doi: 10.1186/s13638-015-0498-8 – volume: 2019 start-page: 1 year: 2019 ident: 1697_CR40 publication-title: IEEE Internet Things J. – volume: 14 start-page: 1 issue: 6 year: 2006 ident: 1697_CR7 publication-title: Wired Mag. – volume: 28 start-page: 20 issue: 1 year: 1979 ident: 1697_CR17 publication-title: Appl. Stat. doi: 10.2307/2346806 – volume: 16 start-page: 1 issue: 1 year: 2020 ident: 1697_CR27 publication-title: ACM Trans. Multimed. Comput. Commun. Appl. (TOMM) doi: 10.1145/3377876 – volume: 23 start-page: 1275 issue: 1 year: 2020 ident: 1697_CR8 publication-title: World Wide Web doi: 10.1007/s11280-019-00684-y – volume: 75 start-page: 1 issue: 12 year: 2019 ident: 1697_CR25 publication-title: Mob. Networks Appl. doi: 10.1007/s11036-019-01445-x |
| SSID | ssj0033492 |
| Score | 2.2181652 |
| Snippet | In this paper, we propose a novel relevance evaluation method using labels collected from crowdsourcing. The proposed method not only predicts the relevance... Abstract In this paper, we propose a novel relevance evaluation method using labels collected from crowdsourcing. The proposed method not only predicts the... |
| SourceID | doaj proquest crossref springer |
| SourceType | Open Website Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 1 |
| SubjectTerms | Artificial neural networks Communications Engineering Crowdsourcing Deep learning Engineering Evaluation Information retrieval Information Systems Applications (incl.Internet) Labels Machine learning Multi-modal Sensor Data Fusion in Internet of Things Networks Relevance evaluation Signal,Image and Speech Processing |
| SummonAdditionalLinks | – databaseName: DOAJ Directory of Open Access Journals dbid: DOA link: http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV07T8MwELYQYoAB8RSFgjKwgdXWTux4pIiKAVUMgLpZ50cQEgpVW-Dvc3aSPpCAhdHORTndnXMP298Rci5tBj5loQTmgKZOWAqeWWoyhc5TcuXjrbSnOzkc5qORul9q9RXOhFXwwJXgOpApk1srAUP91HSNASfQ6pzFMc5HJNCuVE0yVf2DecDca67I5KIz7XG0MxpSpYA-IilbcUMRrX8lxPy2KxqdzWCHbNdRYnJVcbdL1ny5R7aWsAP3Sb9GRn1OnPfjpKyOc0-TUFhN8KOfLtblAwHGpUnojRJ3-5MFvvcBeRzcPFzf0rohArXoNmYUehasAeZMKnL0s5AVnjMrug5HjkkLwhcKAxzjuhwkYO7AOWAC4QtQrgf8kKyXb6U_Iol3GDcVwprC-7RQysgARsgzw60TaQ4t0mvko22NFh6aVrzqmDXkQlcy1ShTHWWqWYtczN8ZV1gZv1L3g9jnlAHnOk6g9nWtff2X9luk3ShN14tvqkO3d8F5xkWLXDaKXDz-maXj_2DphGyyaGgpZVmbrM8m7_6UbNiP2ct0chbN9Avgz-r1 priority: 102 providerName: Directory of Open Access Journals – databaseName: SpringerLINK dbid: C24 link: http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV07T8MwELZQYYCBNyJQUAY2sGhix0lGWlExoIoBUDfr_EiFhNqqKfD3OTtJoQiQYIxzlqzz-R723XeEnKU6ActjdwVmgHIjNAUba6qSHI1nynLrq9Ieb9PBIBsO87u6KKxsst2bJ0mvqf2xzsRlGTGUFerCHYcgklJUvKsOTswlcvVcjUOlf5nD22vKY76dt2SCPFL_knv55UXUG5r-1v-WuE02a8cyvKokYYes2PEu2fgEN7hHujWY6ig01k7DcZUBXobuLjbEtb4Zf5XvCNCVDV07FZ8gEH5Agu-Th_71fe-G1j0UqEZLM6cQadAKYqO4yNA0Q1JYFmvRMfhl4lSDsEWOPpEyHQYpYLjBGGDMYQvITQTsgLTGk7E9JKE16GoVQqvCWl7kuUodfiFLFNNG8AwCEjVslboGGHd9Lp6lDzQyISv-SOSP9PyRcUDOF3OmFbzGr9Rdt1sLSgeN7Qcms5GsT5qEJFeZ1ilgbMhVRykwAtWU0fiN4zwg7WavZX1eS-kaxAvGEiYCctHs7cfvn5d09DfyY7Iee_HgNE7apDWfvdgTsqZf50_l7NTL8TvMEe3O priority: 102 providerName: Springer Nature |
| Title | Learning deep networks with crowdsourcing for relevance evaluation |
| URI | https://link.springer.com/article/10.1186/s13638-020-01697-2 https://www.proquest.com/docview/2394633536 https://doaj.org/article/a59b8cc7a2614b0bbad6048dc2618cc4 |
| Volume | 2020 |
| WOSCitedRecordID | wos000528580000001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVAON databaseName: DOAJ Directory of Open Access Journals customDbUrl: eissn: 1687-1499 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0033492 issn: 1687-1499 databaseCode: DOA dateStart: 20040101 isFulltext: true titleUrlDefault: https://www.doaj.org/ providerName: Directory of Open Access Journals – providerCode: PRVAVX databaseName: SpringerLINK customDbUrl: eissn: 1687-1499 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0033492 issn: 1687-1499 databaseCode: C24 dateStart: 20041201 isFulltext: true titleUrlDefault: https://link.springer.com/search?facet-content-type=%22Journal%22 providerName: Springer Nature |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LbxMxEB7RlgM98K4aKNEeuIHVrO21d0-IVK1AQLRCgAoXy6-tkFASsgH-PjOOt6FI9MLFkh-7sjzjedn-BuCp9pWNklMILFgmg_LMRu6ZqxpUnlo0Mb1K-_RWz2b1-XnT5oBbn69VDjIxCeqw8BQjP6YU3kqISqgXy--MskbR6WpOobEDe4RUhny-Nz2dte8HWSwIe49cLoVbqZSaD89manXclwJ5j5H7RIgkmvErqikh-F8xO_86KU0K6OzO_079LtzOpmfxcsMr9-BGnN-H_T8ACR_ANMOtXhQhxmUx39wR7wuK1hY4618hBftpABq7BSVcSVcIii1o-EP4eHb64eQVy1kWmEddtGa29NY7y4OTqkblbasuCu7VJGAtcO2til2DVpMLE2G1RYdECIteSexsE0orDmB3vpjHQyhiQGOsU951McquaZwmhENROeGDkrUdQTkssPEZgpwyYXwzyRWpldkQxSBRTCKK4SN4dvnNcgPAce3oKdHtciSBZ6eGxerC5L1obNW42ntt0XuUbuKcDQoFWfBYx3Y5gqOBjibv6N5siTiC5wMnbLv_PaVH1__tMdziiQcl49UR7K5XP-ITuOl_rr_2q3Hm5zHsnHA5TgEDLN9ohmVbfcH-9vW79vNvTFMA0A |
| linkProvider | ProQuest |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1Lb9QwEB6VLRJw4I1YWsAHOIHVje3YyaFCFKi66na1h4LKyfUrFRLaXTYLFX-qv7FjJ-lSJHrrgaOdiZXYn-fhxzcAr5TLTRAsLoF5Q4WXjprAHLV5icZT8TKkW2lfRmo8Lo6OyskanHV3YeKxyk4nJkXtZy6ukW_FFN6S85zLd_MfNGaNirurXQqNBhb74fcphmz19vAjju9rxnY_HX7Yo21WAepQ9y6pyZxx1jBvhSzQWJm8Cpw5OfBY8kw5I0NVopdg_YAbZdAB59ygFx4qU_rMcGz3BqwLBPugB-uT4cHka6f7eeT6iyGexKmbCcW6azqF3KozjlinMVyLDCiKskumMGUMuOTm_rUzmwze7r3_ravuw93WtSbvm7nwANbC9CHc-YNw8RHstHSyJ8SHMCfT5gx8TeJqNMFeOvVpMyMKoDNPYkKZdESCrEjRH8Pna_mJJ9CbzqbhKZDg0dmspLNVCKIqS6sigyPPLXdeisL0IesGVLuWYj1m-viuU6hVSN2AQCMIdAKBZn14c_HOvCEYuVJ6J-LkQjKSg6eK2eJEt7pGm7y0hXPKYHQs7MBa4yUqau-wjPWiD5sdbnSrsWq9Ak0f3nbIWz3-9yc9u7q1l3Br7_BgpEfD8f4G3GYJ_4KyfBN6y8XP8Bxuul_Lb_XiRTuXCBxfNybPAep6WmQ |
| linkToPdf | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V3Nb9MwFH8aG0LswOfQCgN8gBNYbWzHTg4TomwV06aqQoB2M_7KhITarumY-Nf46_bsJCtDYrcdOMZxrNj--X3Yz78H8Eq53ATB4haYN1R46agJzFGbl6g8FS9DupX29UiNx8XxcTlZg9_dXZgYVtnJxCSo_czFPfJ-TOEtOc-57FdtWMRkb_RufkpjBql40tql02ggchh-naP7Vu8e7OFcv2ZstP_5w0faZhigDuXwkprMGWcN81bIAhWXyavAmZMDj0-eKWdkqEq0GKwfcKMMGuOcG7TIQ2VKnxmO7d6CDexSho7fxnB_PPnU6QEeef-iuydxGWdCse7KTiH7dcYR9zS6bpENRVF2RS2m7AFXTN6_TmmT8hvd_5-H7QHca01u8r5ZIw9hLUwfweYfRIyPYdjSzJ4QH8KcTJvY-JrEXWqCI3bu0yFHrIBGPomJZlLoBFmRpW_BlxvpxBNYn86mYRtI8GiEVtLZKgRRlaVVkdmR55Y7L0VhepB1k6tdS70eM4D80MkFK6RuAKEREDoBQrMevLn8Zt4Qj1xbexgxc1kzkoangtniRLcySJu8tIVzyqDXLOzAWuMlCnDv8BnLRQ92OgzpVpLVegWgHrztULh6_e9fenp9ay_hDgJRHx2MD5_BXZaWgqAs34H15eIsPIfb7ufye7140S4rAt9uGpIXli5i_g |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Learning+deep+networks+with+crowdsourcing+for+relevance+evaluation&rft.jtitle=EURASIP+journal+on+wireless+communications+and+networking&rft.au=Wu%2C+Ming&rft.au=Yin+Xiaochun&rft.au=Li+Qianmu&rft.au=Zhang%2C+Jing&rft.date=2020-04-25&rft.pub=Springer+Nature+B.V&rft.issn=1687-1472&rft.eissn=1687-1499&rft.volume=2020&rft.issue=1&rft_id=info:doi/10.1186%2Fs13638-020-01697-2&rft.externalDBID=HAS_PDF_LINK |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1687-1499&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1687-1499&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1687-1499&client=summon |