Uncertainty Evaluation of Object Detection Algorithms for Autonomous Vehicles
The safety of the intended functionality (SOTIF) has become one of the hottest topics in the field of autonomous driving. However, no testing and evaluating system for SOTIF performance has been proposed yet. Therefore, this paper proposes a framework based on the advanced You Only Look Once (YOLO) ...
Gespeichert in:
| Veröffentlicht in: | Automotive innovation (Online) Jg. 4; H. 3; S. 241 - 252 |
|---|---|
| Hauptverfasser: | , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
Singapore
Springer Singapore
01.08.2021
|
| Schlagworte: | |
| ISSN: | 2096-4250, 2522-8765 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | The safety of the intended functionality (SOTIF) has become one of the hottest topics in the field of autonomous driving. However, no testing and evaluating system for SOTIF performance has been proposed yet. Therefore, this paper proposes a framework based on the advanced You Only Look Once (YOLO) algorithm and the mean Average Precision (mAP) method to evaluate the object detection performance of the camera under SOTIF-related scenarios. First, a dataset is established, which contains road images with extreme weather and adverse lighting conditions. Second, the Monte Carlo dropout (MCD) method is used to analyze the uncertainty of the algorithm and draw the uncertainty region of the predicted bounding box. Then, the confidence of the algorithm is calibrated based on uncertainty results so that the average confidence after calibration can better reflect the real accuracy. The uncertainty results and the calibrated confidence are proposed to be used for online risk identification. Finally, the confusion matrix is extended according to the several possible mistakes that the object detection algorithm may make, and then the mAP is calculated as an index for offline evaluation and comparison. This paper offers suggestions to apply the MCD method to complex object detection algorithms and to find the relationship between the uncertainty and the confidence of the algorithm. The experimental results verified by specific SOTIF scenarios proof the feasibility and effectiveness of the proposed uncertainty acquisition approach for object detection algorithm, which provides potential practical implementation chance to address perceptual related SOTIF risk for autonomous vehicles. |
|---|---|
| AbstractList | The safety of the intended functionality (SOTIF) has become one of the hottest topics in the field of autonomous driving. However, no testing and evaluating system for SOTIF performance has been proposed yet. Therefore, this paper proposes a framework based on the advanced You Only Look Once (YOLO) algorithm and the mean Average Precision (mAP) method to evaluate the object detection performance of the camera under SOTIF-related scenarios. First, a dataset is established, which contains road images with extreme weather and adverse lighting conditions. Second, the Monte Carlo dropout (MCD) method is used to analyze the uncertainty of the algorithm and draw the uncertainty region of the predicted bounding box. Then, the confidence of the algorithm is calibrated based on uncertainty results so that the average confidence after calibration can better reflect the real accuracy. The uncertainty results and the calibrated confidence are proposed to be used for online risk identification. Finally, the confusion matrix is extended according to the several possible mistakes that the object detection algorithm may make, and then the mAP is calculated as an index for offline evaluation and comparison. This paper offers suggestions to apply the MCD method to complex object detection algorithms and to find the relationship between the uncertainty and the confidence of the algorithm. The experimental results verified by specific SOTIF scenarios proof the feasibility and effectiveness of the proposed uncertainty acquisition approach for object detection algorithm, which provides potential practical implementation chance to address perceptual related SOTIF risk for autonomous vehicles. |
| Author | Li, Jun Peng, Liang Wang, Hong |
| Author_xml | – sequence: 1 givenname: Liang surname: Peng fullname: Peng, Liang organization: Tsinghua University – sequence: 2 givenname: Hong orcidid: 0000-0002-5127-2941 surname: Wang fullname: Wang, Hong email: hong_wang@tsinghua.edu.cn organization: Tsinghua University – sequence: 3 givenname: Jun surname: Li fullname: Li, Jun organization: Tsinghua University |
| BookMark | eNp9kMtOQjEQhhuDiYi8gKvzAtXp7VyWBPGSaNiI26antHDIoTVtjwlvbwFXLljNn8l8k5nvFo2cdwahewIPBKB6jJwSwTFQggFO6QqNqaAU11UpRjlDU2JOBdygaYw7AGAADWXVGH2snDYhqc6lQ7H4Uf2gUudd4W2xbHdGp-LJpFyOvVm_8aFL230srA_FbEje-b0fYvFltp3uTbxD11b10Uz_6gStnhef81f8vnx5m8_esWZMJNwQVVWWV6I1LfBWUygbynVL17S2itTWKN1ozgTwslR13QI1jVoTnkHFtGITVJ_36uBjDMZK3aXT4SmorpcE5NGMPJuR2Yw8mZGQUfoP_Q7dXoXDZYidoZiH3cYEufNDcPnFS9Qvbd541A |
| CitedBy_id | crossref_primary_10_1109_TIV_2022_3196396 crossref_primary_10_3390_a17110526 crossref_primary_10_1109_TIM_2022_3196954 crossref_primary_10_1007_s42154_022_00196_y crossref_primary_10_1109_TIM_2022_3191724 crossref_primary_10_1109_TIV_2023_3271867 crossref_primary_10_1007_s00500_023_09278_3 crossref_primary_10_1109_TIV_2022_3227921 crossref_primary_10_1016_j_cities_2025_106098 crossref_primary_10_3390_s23052867 crossref_primary_10_1109_TIM_2023_3256459 crossref_primary_10_1109_TITS_2023_3278695 crossref_primary_10_3390_rs14215583 crossref_primary_10_1007_s42154_022_00207_y crossref_primary_10_1007_s42154_023_00245_0 crossref_primary_10_1088_1361_6501_ac87c6 crossref_primary_10_3390_sym14112262 crossref_primary_10_1016_j_aap_2024_107903 crossref_primary_10_1007_s13177_025_00476_x crossref_primary_10_1016_j_iot_2022_100635 crossref_primary_10_1109_TVT_2024_3369100 crossref_primary_10_1109_TITS_2025_3568379 crossref_primary_10_1007_s10664_024_10520_1 crossref_primary_10_1080_15472450_2025_2499591 crossref_primary_10_1109_TITS_2025_3539095 crossref_primary_10_1002_adma_202404763 crossref_primary_10_1109_TIM_2025_3602549 crossref_primary_10_1109_TIV_2023_3341144 crossref_primary_10_1016_j_engappai_2023_107766 crossref_primary_10_1109_TITS_2022_3197602 crossref_primary_10_1371_journal_pone_0275538 crossref_primary_10_1109_TITS_2023_3322166 crossref_primary_10_1109_TVT_2024_3408162 crossref_primary_10_3390_app12188948 crossref_primary_10_7467_KSAE_2024_32_4_331 crossref_primary_10_1007_s42154_023_00270_z crossref_primary_10_1007_s42154_023_00279_4 crossref_primary_10_1109_TIV_2023_3299845 crossref_primary_10_1109_TITS_2023_3321309 |
| Cites_doi | 10.4271/12-03-04-0020 10.1109/CVPR42600.2020.00271 10.1109/TITS.2018.2873921 10.1109/MITS.2019.2953556 10.5244/C.31.57 10.1016/j.ins.2017.01.013 10.1109/ICRA.2019.8793821 10.1016/j.cviu.2019.05.003 10.1016/j.cmpb.2020.105381 10.1109/ICRA.2018.8460700 10.1007/978-3-319-10602-1_48 |
| ContentType | Journal Article |
| Copyright | The Author(s) 2021 |
| Copyright_xml | – notice: The Author(s) 2021 |
| DBID | C6C AAYXX CITATION |
| DOI | 10.1007/s42154-021-00154-0 |
| DatabaseName | Springer Nature OA Free Journals CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | CrossRef |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 2522-8765 |
| EndPage | 252 |
| ExternalDocumentID | 10_1007_s42154_021_00154_0 |
| GrantInformation_xml | – fundername: National Basic Research Program of China (973 Program) grantid: 2020YFB1600303 funderid: http://dx.doi.org/10.13039/501100012166 – fundername: National Science Foundation of China Project grantid: U1964203; 52072215 |
| GroupedDBID | -EM 0R~ 406 AACDK AAHNG AAIAL AAJBT AASML AATNV AAUYE ABAKF ABDZT ABECU ABFTV ABJNI ABKCH ABMQK ABTEG ABTKH ABTMW ABXPI ACAOD ACDTI ACHSB ACMLO ACOKC ACPIV ACZOJ ADKNI ADKPE ADRFC ADTPH ADURQ ADYFF AEFQL AEJRE AEMSY AESKC AFBBN AFQWF AGDGC AGJBK AGMZJ AGQEE AGRTI AIAKS AIGIU AILAN AITGF AJZVZ ALMA_UNASSIGNED_HOLDINGS AMKLP AMXSW AMYLF AXYYD BGNMA C6C CSCUP DPUIP EBLON EBS EJD FIGPU FINBP FNLPD FSGXE GGCAI IKXTQ IWAJR J-C JZLTJ KOV LLZTM M4Y NPVJJ NQJWS NU0 PT4 RLLFE ROL RSV SJYHP SNE SNPRN SOHCF SOJ SRMVM SSLCW STPWE TSG UOJIU UTJUX VEKWB VFIZW ZMTXR AAYXX ABBRH ABDBE ABFSG ABRTQ ACSTC AEZWR AFDZB AFHIU AFOHR AHPBZ AHWEU AIXLP ATHPR AYFIA CITATION |
| ID | FETCH-LOGICAL-c335t-91a77f475beb04bc206924cb2d28fa18feac9c4350466a88b02e9ad1491aa3ca3 |
| IEDL.DBID | RSV |
| ISICitedReferencesCount | 57 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000680568800001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 2096-4250 |
| IngestDate | Sat Nov 29 02:58:22 EST 2025 Tue Nov 18 22:23:16 EST 2025 Fri Feb 21 02:47:42 EST 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 3 |
| Keywords | Uncertainty evaluation Confidence calibration SOTIF Autonomous vehicles |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c335t-91a77f475beb04bc206924cb2d28fa18feac9c4350466a88b02e9ad1491aa3ca3 |
| ORCID | 0000-0002-5127-2941 |
| OpenAccessLink | https://link.springer.com/10.1007/s42154-021-00154-0 |
| PageCount | 12 |
| ParticipantIDs | crossref_citationtrail_10_1007_s42154_021_00154_0 crossref_primary_10_1007_s42154_021_00154_0 springer_journals_10_1007_s42154_021_00154_0 |
| PublicationCentury | 2000 |
| PublicationDate | 20210800 2021-08-00 |
| PublicationDateYYYYMMDD | 2021-08-01 |
| PublicationDate_xml | – month: 8 year: 2021 text: 20210800 |
| PublicationDecade | 2020 |
| PublicationPlace | Singapore |
| PublicationPlace_xml | – name: Singapore |
| PublicationTitle | Automotive innovation (Online) |
| PublicationTitleAbbrev | Automot. Innov |
| PublicationYear | 2021 |
| Publisher | Springer Singapore |
| Publisher_xml | – name: Springer Singapore |
| References | Azevedo, T., de Jong, R., Mattina, M., Maji, P.: Stochastic-YOLO: efficient probabilistic object detection under dataset shifts. In: Paper Presented at the 34th Conference and Workshop on Neural Information Processing Systems, Vancouver, Canada, 6–12 December 2020. Zhu, L., Laptev, N.: Deep and confident prediction for time series at Uber. In: Paper Presented at the 17th IEEE International Conference on Data Mining Workshops, New Orleans, Los Angeles, 18–21 November (2017). Lin, C.H., Hsu, K.C., Johnson, K.R., et al.: Evaluation of machine learning methods to stroke outcome prediction using a nationwide disease registry. Comput. Meth. Programs Biomed. 190,(2020) Geiger, A., Lenz, P. S., Stiller, C., Urtasun, R.: Vision meets robotics: the KITTI dataset. Int. J. Rob. Res. 32(11), 1–6 (2013). Kahn, G., Villaflor, A., Pong, V., Abbeel, P., Levine, S.: Uncertainty-aware reinforcement learning for collision avoidance. Mach. Learn. 1702.01182 (2017). https://arxiv.org/abs/1702.01182v1 Kendall, A., Badrinarayanan, V., Cipolla, R.: Bayesian segNet: model uncertainty in deep convolutional encoder-decoder architectures for scene understanding. In: Paper Presented at the 29th IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Nevada, 27–30 June 2016. Mukhoti, J., Gal, Y.: Evaluating Bayesian deep learning methods for semantic segmentation. In: Paper presented at the 32nd IEEE Conference on Computer Vision and Pattern Recognition Workshops 2019, Long Beach, California, 16–20 June 2019. Miller D, Dayoub F, Milford M, et al.: Evaluating merging strategies for sampling-based uncertainty techniques in object detection. In: Paper Presented at the 2019 IEEE International Conference on Robotics and Automation, Montreal, Quebec, 20–24 May 2019. Zadrozny, B., Elkan, C.: Transforming classifier scores into accurate multiclass probability estimates. In: Paper presented at the 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Association for Computing Machinery, Edmonton, 23–26 July (2002). Gal, Y., Ghahramani, Z.: Dropout as a bayesian approximation: representing model uncertainty in deep learning. In: Paper Presented at the 33rd International Conference on Machine Learning, New York City, 19–24 June (2016). Zadrozny, B., Elkan, C.: Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers. In: Paper presented at the 18th International Conference on Machine Learning, Williams College, Massachusetts, June 28 – July 1 (2001). ShaoLCaiZLiuLLuKPerformance evaluation of deep feature learning for RGB-D image/video classificationInf. Sci.201738526628310.1016/j.ins.2017.01.013 HintonGESrivastavaNKrizhevskyAImproving neural networks by preventing co-adaptation of feature detectorsComput. Sci.201234212223 Miller D, Nicholson L, Dayoub F, et al.: Dropout sampling for robust object detection in open-set conditions. In: Paper Presented at the 2018 IEEE International Conference on Robotics and Automation, Brisbane, Australia, 21–25 May 2018. Bhattacharyya, A., Fritz, M., Schiele, B.: Long-term on-board prediction of people in traffic scenes under uncertainty. In: Paper presented at the 31st IEEE Conference on Computer Vision and Pattern Recognition, Michael Brown, Salt Lake City, 18–22 June (2018). Osband, I.: Risk versus uncertainty in deep learning: bayes, bootstrap and the dangers of dropout. In: Paper presented at the 30th Conference and Workshop on Neural Information Processing Systems, Barcelona, Spain, 5–10 December 2016. International Organization for Standardization: ISO/Pas 21448-road vehicles-safety of the intended functionality. Geneva, Switzerland (2019) PlattJProbabilistic outputs for support vector machines and comparisons to regularized likelihood methodsAdv Large Margin Classif19991036174 WangHHuangYKhajepourAZhangYRasekhipourYCaoDCrash mitigation in motion planning for autonomous vehiclesIEEE Trans. Intell. Transp. Syst.20192093313332310.1109/TITS.2018.2873921 Yu, F., Xian, W., Chen, Y., Liu, F., Liao, M., Madhavan, V., Darrell, T.: Bdd100k: a diverse driving video database for heterogeneous multitask learning. In: Paper Presented at the 33rd IEEE Conference on Computer Vision and Pattern Recognition, Seattle, Washington, 13–19 June 2020. Wang, H., Huang, Y., Khajepour, A., Cao, D., Lv, C.: Ethical decision-making platform in autonomous vehicles with lexicographic optimization based model predictive controller. IEEE Trans. Veh. Technol. 69(8), 8164–8175 (2020). NealRMMCMC using Hamiltonian dynamicsHandbook Markov Chain Monte Carlo.201121121229.65018 WishartJComoSForgioneUWeastJLiterature review of verification and validation activities of automated driving systemsSAE Int. J. Connect. Automat. Veh.20203426732310.4271/12-03-04-0020 Blundell, C., Cornebise, J., Kavukcuoglu, K., Wierstra, D.: Weight uncertainty in neural network. In: Paper presented at the 32nd International Conference on Machine Learning, Lille, France, 6–11 July (2015). Lin, T. Y., Maire, M., Belongie, S., et al.: Microsoft coco: common objects in context. In: Paper presented at the 13th European Conference on Computer Vision, Zurich, Switzerland, 6–12 September 2014. GalYGhahramaniZA theoretically grounded application of dropout in recurrent neural networksAdv. Neural. Inf. Process. Syst.20162910191027 Michelmore, R., Wicker, M., Laurenti, L., Cardelli, L., Gal, Y., Kwiatkowska, M.: Uncertainty quantification with statistical guarantees in end-to-end autonomous driving control. In: Paper presented at the 2020 IEEE International Conference on Robotics and Automation, Paris, France, May 31 – August 31 (2020). Guo, C., Pleiss, G., Sun, Y., Weinberger, K. Q.: On calibration of modern neural networks. In: Paper presented at the 34th International Conference on Machine Learning, International Machine Learning Society, Sydney, 6–11 August (2017). Wang, H., Khajepour, A., Cao, D., Liu, T.: Ethical decision making in autonomous vehicles: challenges and research progress. IEEE Intell. Transp. Syst. Mag. (2020). https://doi.org/10.1109/MITS.2019.2953556 LiSRenWZhangJYuJGuoXSingle image rain removal via a deep decomposition-composition networkComput. Vis. Image. Underst.2019186485710.1016/j.cviu.2019.05.003 Redmon, J., Farhadi, A.: Yolov3: an incremental improvement. In: Paper presented at the 31st IEEE Conference on Computer Vision and Pattern Recognition, Michael Brown, Salt Lake City, 18–22 June 2018. Naeini, M. P., Cooper, G., Hauskrecht, M.: Obtaining well calibrated probabilities using Bayesian binning. In: Paper presented at the 29th AAAI Conference on Artificial Intelligence, American Association for Artificial Intelligence, Austin, 25–30 January (2015). Miller, D., Sünderhauf, N., Zhang H, et al.: Benchmarking sampling-based probabilistic object detectors. In: Paper Presented at the 32nd IEEE Conference on Computer Vision and Pattern Recognition Workshops 2019, Long Beach, California, 16–20 June 2019. Kendall, A., & Gal, Y.: What uncertainties do we need in Bayesian deep learning for computer vision? In: Paper presented at the 30th IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Hawaii, 21–26 July 2017. 154_CR30 154_CR11 154_CR33 154_CR12 154_CR34 S Li (154_CR32) 2019; 186 Y Gal (154_CR22) 2016; 29 J Wishart (154_CR31) 2020; 3 H Wang (154_CR17) 2019; 20 154_CR26 154_CR27 154_CR24 154_CR25 154_CR29 J Platt (154_CR7) 1999; 10 154_CR23 154_CR20 RM Neal (154_CR10) 2011; 2 154_CR9 154_CR8 GE Hinton (154_CR21) 2012; 3 154_CR6 154_CR1 154_CR15 154_CR16 154_CR13 154_CR14 154_CR5 154_CR19 154_CR4 154_CR3 154_CR2 154_CR18 L Shao (154_CR28) 2017; 385 |
| References_xml | – reference: Zadrozny, B., Elkan, C.: Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers. In: Paper presented at the 18th International Conference on Machine Learning, Williams College, Massachusetts, June 28 – July 1 (2001). – reference: Miller D, Nicholson L, Dayoub F, et al.: Dropout sampling for robust object detection in open-set conditions. In: Paper Presented at the 2018 IEEE International Conference on Robotics and Automation, Brisbane, Australia, 21–25 May 2018. – reference: Wang, H., Khajepour, A., Cao, D., Liu, T.: Ethical decision making in autonomous vehicles: challenges and research progress. IEEE Intell. Transp. Syst. Mag. (2020). https://doi.org/10.1109/MITS.2019.2953556 – reference: Yu, F., Xian, W., Chen, Y., Liu, F., Liao, M., Madhavan, V., Darrell, T.: Bdd100k: a diverse driving video database for heterogeneous multitask learning. In: Paper Presented at the 33rd IEEE Conference on Computer Vision and Pattern Recognition, Seattle, Washington, 13–19 June 2020. – reference: Guo, C., Pleiss, G., Sun, Y., Weinberger, K. Q.: On calibration of modern neural networks. In: Paper presented at the 34th International Conference on Machine Learning, International Machine Learning Society, Sydney, 6–11 August (2017). – reference: NealRMMCMC using Hamiltonian dynamicsHandbook Markov Chain Monte Carlo.201121121229.65018 – reference: Michelmore, R., Wicker, M., Laurenti, L., Cardelli, L., Gal, Y., Kwiatkowska, M.: Uncertainty quantification with statistical guarantees in end-to-end autonomous driving control. In: Paper presented at the 2020 IEEE International Conference on Robotics and Automation, Paris, France, May 31 – August 31 (2020). – reference: GalYGhahramaniZA theoretically grounded application of dropout in recurrent neural networksAdv. Neural. Inf. Process. Syst.20162910191027 – reference: Zadrozny, B., Elkan, C.: Transforming classifier scores into accurate multiclass probability estimates. In: Paper presented at the 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Association for Computing Machinery, Edmonton, 23–26 July (2002). – reference: Gal, Y., Ghahramani, Z.: Dropout as a bayesian approximation: representing model uncertainty in deep learning. In: Paper Presented at the 33rd International Conference on Machine Learning, New York City, 19–24 June (2016). – reference: Kendall, A., Badrinarayanan, V., Cipolla, R.: Bayesian segNet: model uncertainty in deep convolutional encoder-decoder architectures for scene understanding. In: Paper Presented at the 29th IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Nevada, 27–30 June 2016. – reference: Miller, D., Sünderhauf, N., Zhang H, et al.: Benchmarking sampling-based probabilistic object detectors. In: Paper Presented at the 32nd IEEE Conference on Computer Vision and Pattern Recognition Workshops 2019, Long Beach, California, 16–20 June 2019. – reference: Wang, H., Huang, Y., Khajepour, A., Cao, D., Lv, C.: Ethical decision-making platform in autonomous vehicles with lexicographic optimization based model predictive controller. IEEE Trans. Veh. Technol. 69(8), 8164–8175 (2020). – reference: WishartJComoSForgioneUWeastJLiterature review of verification and validation activities of automated driving systemsSAE Int. J. Connect. Automat. Veh.20203426732310.4271/12-03-04-0020 – reference: Azevedo, T., de Jong, R., Mattina, M., Maji, P.: Stochastic-YOLO: efficient probabilistic object detection under dataset shifts. In: Paper Presented at the 34th Conference and Workshop on Neural Information Processing Systems, Vancouver, Canada, 6–12 December 2020. – reference: Mukhoti, J., Gal, Y.: Evaluating Bayesian deep learning methods for semantic segmentation. In: Paper presented at the 32nd IEEE Conference on Computer Vision and Pattern Recognition Workshops 2019, Long Beach, California, 16–20 June 2019. – reference: Redmon, J., Farhadi, A.: Yolov3: an incremental improvement. In: Paper presented at the 31st IEEE Conference on Computer Vision and Pattern Recognition, Michael Brown, Salt Lake City, 18–22 June 2018. – reference: Blundell, C., Cornebise, J., Kavukcuoglu, K., Wierstra, D.: Weight uncertainty in neural network. In: Paper presented at the 32nd International Conference on Machine Learning, Lille, France, 6–11 July (2015). – reference: Kendall, A., & Gal, Y.: What uncertainties do we need in Bayesian deep learning for computer vision? In: Paper presented at the 30th IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Hawaii, 21–26 July 2017. – reference: Kahn, G., Villaflor, A., Pong, V., Abbeel, P., Levine, S.: Uncertainty-aware reinforcement learning for collision avoidance. Mach. Learn. 1702.01182 (2017). https://arxiv.org/abs/1702.01182v1 – reference: Miller D, Dayoub F, Milford M, et al.: Evaluating merging strategies for sampling-based uncertainty techniques in object detection. In: Paper Presented at the 2019 IEEE International Conference on Robotics and Automation, Montreal, Quebec, 20–24 May 2019. – reference: Lin, C.H., Hsu, K.C., Johnson, K.R., et al.: Evaluation of machine learning methods to stroke outcome prediction using a nationwide disease registry. Comput. Meth. Programs Biomed. 190,(2020) – reference: Bhattacharyya, A., Fritz, M., Schiele, B.: Long-term on-board prediction of people in traffic scenes under uncertainty. In: Paper presented at the 31st IEEE Conference on Computer Vision and Pattern Recognition, Michael Brown, Salt Lake City, 18–22 June (2018). – reference: Geiger, A., Lenz, P. S., Stiller, C., Urtasun, R.: Vision meets robotics: the KITTI dataset. Int. J. Rob. Res. 32(11), 1–6 (2013). – reference: PlattJProbabilistic outputs for support vector machines and comparisons to regularized likelihood methodsAdv Large Margin Classif19991036174 – reference: Zhu, L., Laptev, N.: Deep and confident prediction for time series at Uber. In: Paper Presented at the 17th IEEE International Conference on Data Mining Workshops, New Orleans, Los Angeles, 18–21 November (2017). – reference: LiSRenWZhangJYuJGuoXSingle image rain removal via a deep decomposition-composition networkComput. Vis. Image. Underst.2019186485710.1016/j.cviu.2019.05.003 – reference: International Organization for Standardization: ISO/Pas 21448-road vehicles-safety of the intended functionality. Geneva, Switzerland (2019) – reference: Lin, T. Y., Maire, M., Belongie, S., et al.: Microsoft coco: common objects in context. In: Paper presented at the 13th European Conference on Computer Vision, Zurich, Switzerland, 6–12 September 2014. – reference: WangHHuangYKhajepourAZhangYRasekhipourYCaoDCrash mitigation in motion planning for autonomous vehiclesIEEE Trans. Intell. Transp. Syst.20192093313332310.1109/TITS.2018.2873921 – reference: HintonGESrivastavaNKrizhevskyAImproving neural networks by preventing co-adaptation of feature detectorsComput. Sci.201234212223 – reference: ShaoLCaiZLiuLLuKPerformance evaluation of deep feature learning for RGB-D image/video classificationInf. Sci.201738526628310.1016/j.ins.2017.01.013 – reference: Osband, I.: Risk versus uncertainty in deep learning: bayes, bootstrap and the dangers of dropout. In: Paper presented at the 30th Conference and Workshop on Neural Information Processing Systems, Barcelona, Spain, 5–10 December 2016. – reference: Naeini, M. P., Cooper, G., Hauskrecht, M.: Obtaining well calibrated probabilities using Bayesian binning. In: Paper presented at the 29th AAAI Conference on Artificial Intelligence, American Association for Artificial Intelligence, Austin, 25–30 January (2015). – ident: 154_CR6 – volume: 3 start-page: 267 issue: 4 year: 2020 ident: 154_CR31 publication-title: SAE Int. J. Connect. Automat. Veh. doi: 10.4271/12-03-04-0020 – ident: 154_CR4 – ident: 154_CR30 doi: 10.1109/CVPR42600.2020.00271 – ident: 154_CR8 – volume: 29 start-page: 1019 year: 2016 ident: 154_CR22 publication-title: Adv. Neural. Inf. Process. Syst. – ident: 154_CR23 – ident: 154_CR29 – ident: 154_CR25 – volume: 20 start-page: 3313 issue: 9 year: 2019 ident: 154_CR17 publication-title: IEEE Trans. Intell. Transp. Syst. doi: 10.1109/TITS.2018.2873921 – ident: 154_CR16 doi: 10.1109/MITS.2019.2953556 – ident: 154_CR24 doi: 10.5244/C.31.57 – volume: 10 start-page: 61 issue: 3 year: 1999 ident: 154_CR7 publication-title: Adv Large Margin Classif – ident: 154_CR14 – ident: 154_CR1 – volume: 3 start-page: 212 issue: 4 year: 2012 ident: 154_CR21 publication-title: Comput. Sci. – volume: 385 start-page: 266 year: 2017 ident: 154_CR28 publication-title: Inf. Sci. doi: 10.1016/j.ins.2017.01.013 – ident: 154_CR3 – ident: 154_CR12 – ident: 154_CR18 – ident: 154_CR33 – ident: 154_CR5 – ident: 154_CR26 doi: 10.1109/ICRA.2019.8793821 – ident: 154_CR9 – volume: 186 start-page: 48 year: 2019 ident: 154_CR32 publication-title: Comput. Vis. Image. Underst. doi: 10.1016/j.cviu.2019.05.003 – volume: 2 start-page: 2 issue: 11 year: 2011 ident: 154_CR10 publication-title: Handbook Markov Chain Monte Carlo. – ident: 154_CR2 doi: 10.1016/j.cmpb.2020.105381 – ident: 154_CR27 doi: 10.1109/ICRA.2018.8460700 – ident: 154_CR19 – ident: 154_CR13 – ident: 154_CR11 – ident: 154_CR20 doi: 10.1007/978-3-319-10602-1_48 – ident: 154_CR15 – ident: 154_CR34 |
| SSID | ssj0003009237 ssib052855666 |
| Score | 2.4066198 |
| Snippet | The safety of the intended functionality (SOTIF) has become one of the hottest topics in the field of autonomous driving. However, no testing and evaluating... |
| SourceID | crossref springer |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 241 |
| SubjectTerms | Automotive Engineering Energy Systems Engineering Mechanical Engineering |
| Title | Uncertainty Evaluation of Object Detection Algorithms for Autonomous Vehicles |
| URI | https://link.springer.com/article/10.1007/s42154-021-00154-0 |
| Volume | 4 |
| WOSCitedRecordID | wos000680568800001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVAVX databaseName: SpringerLINK Contemporary 1997-Present customDbUrl: eissn: 2522-8765 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0003009237 issn: 2096-4250 databaseCode: RSV dateStart: 20180101 isFulltext: true titleUrlDefault: https://link.springer.com/search?facet-content-type=%22Journal%22 providerName: Springer Nature |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV3dS8MwEA-iPuiD3-L8Ig--aSBNkzV9HLrhg05RN_ZW0jR1k9nJ2gn-916ytjiQgb6VkpZyd7n7XfO7O4QuhAy18cOESC4M4amCK08HRNhRLzzwE-VGJ_Tvgm5XDgbhY1kUllds9-pI0nnqutiNQ3TixFIKXOAnkKivQbiTdjs-PfcrKxJMClGB8jeH-SmgGFs3zQCvEzBSWlbP_P7axQi1eDzqok5n-3_fu4O2SpSJW3Oz2EUrJttDmz96D-6j-x6o29EBii_crnt-40mKH2L7cwbfmMLxtDLcGr9OpqNi-J5jwLi4NStsKcRkluO-GTpe3QHqddov17eknK1AtO-LAnycCoKUByI2MeWxZrQJmZiOWcJkqjyZgkMONWApyJ-bSsqYMhOqBPIpTylfK_8QrWaTzBwhzCjcNyFAvyTmXKSS6lDIJEht9uNT00BeJc9Il43H7fyLcVS3THaiikBUkRNVRBvosn7mY952Y-nqq0oFUbkF8yXLj_-2_ARtMKdFy_o7RavFdGbO0Lr-LEb59NzZ3jdUVM9g |
| linkProvider | Springer Nature |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV3dS8MwEA-igvrgtzg_8-CbFrI2WdPHoRsTtym6jb2VNE3dZLaydoL_vZesLQ5koG8lpKXcXe9-1_zuDqErxj2pHC-0OGXKopGAq6p0LaZHvVDXCYUZnTBou90uHw69p7woLC3Y7sWRpPHUZbEbhehELU0pMIHfgkR9Ta9pIt_zy6CwImZzxgpQ_mYwPwEUo-umbcDrFhgpyatnfn_sYoRaPB41Uae587_33UXbOcrE9blZ7KEVFe-jrR-9Bw9Qpw_qNnSA7As3yp7fOInwY6B_zuA7lRmeVozrk9dkOs5G7ykGjIvrs0yXQiSzFA_UyPDqDlG_2ejdtqx8toIlHYdl4OOE60bUZYEKCA2kTWqQicnADm0eiSqPwCF7ErAU5M81wXlAbOWJEPKpqhCOFM4RWo2TWB0jbBNYVx5AvzCglEWcSI_x0I109uMQVUHVQp6-zBuP6_kXE79smWxE5YOofCMqn1TQdXnPx7ztxtLdN4UK_PwTTJdsP_nb9ku00ep12n77vvtwijZto1HNADxDq9l0ps7RuvzMxun0wtjhNxZP0kQ |
| linkToPdf | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV3fS8MwEA6iIvrgb3H-zINvWpa1yZo-DrehOOdAN_ZW0jR1k9mNtRP8771kbdlABuJbKUkpl2vuu-a77xC6YdyTyvFCi1OmLBoJuKpI12K61Qt1nVCY1gm9lttu837f6yxU8Ru2e34kOa9p0CpNcVqehFG5KHyjEKmopekFBgRYkLRvUN00SOfrr73co5jNGcsB-ofB_wQQja6htgG7W-CwJKuk-f2xy9Fq-ajURKDm3v_ffR_tZugT1-bucoDWVHyIdhY0CY_QcxfcwNAE0m_cKLTA8TjCL4H-aYPrKjX8rRjXRu_j6TAdfCYYsC-uzVJdIjGeJbinBoZvd4y6zcbb_YOV9VywpOOwFPY-4boRdVmgAkIDaZMqZGgysEObR6LCI9ioPQkYC_LqquA8ILbyRAh5VkUIRwrnBK3H41idImwTuK88gIRhQCmLOJEe46Eb6azIIaqEKrltfZkJkuu-GCO_kFI2pvLBVL4xlU9K6LaYM5nLcawcfZcvh599msmK4Wd_G36Ntjr1pt96bD-do23bLKgmBl6g9XQ6U5doU36lw2R6ZVzyB0mN2yg |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Uncertainty+Evaluation+of+Object+Detection+Algorithms+for+Autonomous+Vehicles&rft.jtitle=Automotive+innovation+%28Online%29&rft.au=Peng%2C+Liang&rft.au=Wang%2C+Hong&rft.au=Li%2C+Jun&rft.date=2021-08-01&rft.issn=2096-4250&rft.eissn=2522-8765&rft.volume=4&rft.issue=3&rft.spage=241&rft.epage=252&rft_id=info:doi/10.1007%2Fs42154-021-00154-0&rft.externalDBID=n%2Fa&rft.externalDocID=10_1007_s42154_021_00154_0 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2096-4250&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2096-4250&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2096-4250&client=summon |