Search Results - General and reference Cross-computing tools and techniques Evaluation*

Guidelines for Assessing the Accuracy of Log Message Template Identification Techniques by Khan, Zanis Ali, Shin, Donghwan, Bianculli, Domenico, Briand, Lionel

“… While many techniques have been proposed in the literature, only two recent studies provide a comprehensive evaluation and comparison of the techniques using an established benchmark composed of real-world logs…”
Get full text

Breaking the Silence: the Threats of Using LLMs in Software Engineering by Sallou, June, Durieux, Thomas, Panichella, Annibale

“… and research evaluation, and the reproducibility of LLM-based findings. In response, this paper proposes a set of guidelines tailored for SE researchers and Language Model (LM…”
Get full text

Navigating Mobile Testing Evaluation: A Comprehensive Statistical Analysis of Android GUI Testing Metrics by Lan, Yuanhong, Lu, Yifei, Pan, Minxue, Li, Xuandong

“… However, conducting a reliable and robust evaluation based on the two metrics remains challenging, due to the imperfections of the current evaluation system, with a tangle of numerous metric…”
Get full text

How secure is your cache against side-channel attacks? by He, Zecheng, Lee, Ruby B.

“…Security-critical data can leak through very unexpected side channels, making side-channel attacks very dangerous threats to information security. Of these,…”
Get full text

OptiWISE: Combining Sampling and Instrumentation for Granular CPI Analysis by Guo, Yuxin, Chadwick, Alex W., Erdos, Marton, Bora, Utpal, Vougioukas, Ilias, Gabrielli, Giacomo, Jones, Timothy M.

“… Existing profiling tools typically either sample hardware performance counters or instrument the program with extra instructions to analyze its execution…”
Get full text

Fairquant: Certifying and Quantifying Fairness of Deep Neural Networks by Kim, Brian Hyeongseok, Wang, Jingbo, Wang, Chao

“…) receive the same treatment. While there are existing techniques that provide such a guarantee, they tend to suffer from lack of scalability or accuracy as the size and input dimension of the DNN increase…”
Get full text

AIBench Scenario: Scenario-Distilling AI Benchmarking by Gao, Wanling, Tang, Fei, Zhan, Jianfeng, Wen, Xu, Wang, Lei, Cao, Zheng, Lan, Chuanxin, Luo, Chunjie, Liu, Xiaoli, Jiang, Zihan

“…Modern real-world application scenarios like Internet services consist of a diversity of AI and non-AI modules with huge code sizes and long and complicated…”
Get full text

EvalDNN: A Toolbox for Evaluating Deep Neural Network Models by Tian, Yongqiang, Zeng, Zhihua, Wen, Ming, Liu, Yepang, Kuo, Tzu-yang, Cheung, Shing-Chi

“… What is worse, due to the large number of hyper-parameters and inadequate documentation, evaluation results of some deep learning models are hard to reproduce, especially when the models and metrics…”
Get full text

Probing for Requirements Knowledge to Stimulate Architectural Thinking by Anish, Preethu Rose, Balasubramaniam, Balaji, Sainani, Abhishek, Cleland-Huang, Jane, Daneva, Maya, Wieringa, Roel J., Ghaisas, Smita

“…Software requirements specifications (SRSs) often lack the detail needed to make informed architectural decisions. Architects therefore either make…”
Get full text

Mind the gap: can and should software engineering data sharing become a path of less resistance? by Wallace, Ken

“…The facility to process data is, arguably, the defining capability underpinning the transformative power of software: the relationships of each to the other…”
Get full text

Exploring the Impact of the Output Format on the Evaluation of Large Language Models for Code Translation by Macedo, Marcos, Tian, Yuan, Cogo, Filipe R., Adams, Bram

“…, and enhancing software performance. With the recent advances in large language models (LLMs) and their applications to code translation, there is an increasing need for comprehensive evaluation of these models…”
Get full text

An empirical study on selection of Open Source Software - Preliminary results by Hauge, Oyvind, Osterlie, Thomas, Sorensen, Carl-Fredrik, Gerea, Marinela

“…Growing attention on component-based development has inspired the development of several normative methods for selection of software components. Despite these…”
Get full text

Student research poster: A low complexity cache sharing mechanism to address system fairness by Selfa, Vicent, Sahuquillo, Julio, Petit, Salvador, Gomez, Maria E.

“…Shared caches have become, de facto, the common design choice in current multi-cores, ranging from embedded devices to high-performance processors. In these…”
Get full text

LLM-Inference-Bench: Inference Benchmarking of Large Language Models on AI Accelerators by Chitty-Venkata, Krishna Teja, Raskar, Siddhisanket, Kale, Bharat, Ferdaus, Farah, Tanikanti, Aditya, Raffenetti, Ken, Taylor, Valerie, Emani, Murali, Vishwanath, Venkatram

“… Our evaluation includes several LLM inference frameworks and models from LLaMA, Mistral, and Qwen families with 7B and 70B parameters…”
Get full text

Application-Driven Exascale: The JUPITER Benchmark Suite by Herten, Andreas, Achilles, Sebastian, Alvarez, Damian, Badwaik, Jayesh, Behle, Eric, Bode, Mathis, Breuer, Thomas, Caviedes-Voullieme, Daniel, Cherti, Mehdi, Dabah, Adel, Sayed, Salem El, Frings, Wolfgang, Gonzalez-Nicolas, Ana, Gregory, Eric B., Mood, Kaveh Haghighi, Hater, Thorsten, Jitsev, Jenia, John, Chelsea Maria, Meinke, Jan H., Meyer, Catrin I., Mezentsev, Pavel, Mirus, Jan-Oliver, Nassyr, Stepan, Penke, Carolin, Rommer, Manoel, Sinha, Ujjwal, Vieth, Benedikt von St, Stein, Olaf, Suarez, Estela, Willsch, Dennis, Zhukov, Ilya

“…Benchmarks are essential in the design of modern HPC installations, as they define key aspects of system components. Beyond synthetic workloads, it is crucial…”
Get full text

Evaluating the Benefits of Systematic Project Management in Large Public Sector Projects by Guzman, Liliana, Steinbach, Silke, Diebold, Philipp, Zehler, Thomas, Schneider, Klaus, Habbe, Matthias

“…s. We designed a formative evaluation to understand the adequacy and acceptance of the new PM system and to elicit stakeholders' feedback in order to improve the new PM system and the related transfer…”
Get full text

Student research poster - from processing-in-Memory to Processing-in-Storage by Kaplan, Roman

“…This paper studied a new technology and approach - Processing-in-Storage (PrinS). Resistive technologies have shown high-density, high-endurance and low…”
Get full text

Execution-based prediction using speculative slices by Zilles, Craig, Sohi, Gurindar

“…A relatively small set of static instructions has significant leverage on program execution performance. These problem instructions contribute a…”
Get full text

Top-Down Microarchitecture Analysis Approximation Based on Performance Counter Architecture for SiFive RISC-V Processors by Mou, Chan-Yu, Hsiao, Chao-Chieh, Chou, Jerry

“…Modern Out-of-Order RISC-V CPUs have complex mechanisms, making microarchitecture-level performance analysis challenging. Despite increasing Performance…”
Get full text

Ponte Vecchio Across the Atlantic: Single-Node Benchmarking of Two Intel GPU Systems by Applencourt, Thomas, Sadawarte, Aditya, Muralidharan, Servesh, Bertoni, Colleen, Kwack, JaeHyuk, Luo, Ye, Rangel, Esteban, Tramm, John, Ghadar, Yasaman, Tamerus, Arjen, Edsall, Chris, Deakin, Tom

“…Intel Data Center GPU Max 1550, known as Ponte Vecchio (PVC), is a new Intel GPU architecture for high-performance computing. It is the basis of two systems on…”
Get full text