Search Results - Programming Exercise Generation Benchmark

A Survey Study on the State of the Art of Programming Exercise Generation Using Large Language Models by Frankford, Eduard, Hohn, Ingo, Sauerwein, Clemens, Breu, Ruth

“…This paper analyzes Large Language Models (LLMs) with regard to their programming exercise generation capabilities…”
Get full text

JavaBench: A Benchmark of Object-Oriented Code Generation for Evaluating Large Language Models by Cao, Jialun, Chen, Zhiyong, Wu, Jiarong, Cheung, Shing-Chi, Xu, Chang

“…Code generation benchmarks such as HumanEval are widely adopted to evaluate LLMs' capabilities…”
Get full text

On the performance of large language models on introductory programming assignments by Raihan, Nishat, Goswami, Dhiman, Puspo, Sadiya Sayara Chowdhury, Siddiq, Mohammed Latif, Newman, Christian, Ranasinghe, Tharindu, Santos, Joanna C. S., Zampieri, Marcos

“…) have led to the development of a new generation of Large Language Models (LLMs) trained on massive amounts of data…”
Get full text

Dataset of Program Source Codes Solving Unique Programming Exercises Generated by Digital Teaching Assistant by Demidova, Liliya A., Andrianova, Elena G., Sovietov, Peter N., Gorchakov, Artyom V.

“…This paper presents a dataset containing automatically collected source codes solving unique programming exercises of different types…”
Get full text

JavaBench: A Benchmark of Object-Oriented Code Generation for Evaluating Large Language Models by Cao, Jialun, Chen, Zhiyong, Wu, Jiarong, Shing-chi Cheung, Chang, Xu

“…Code generation benchmarks such as HumanEval are widely adopted to evaluate LLMs' capabilities…”
Get full text

Evaluating the quality of scenarios of short-term wind power generation by Pinson, P., Girard, R.

“… ► Guidelines for future evaluation/benchmark exercises. Scenarios of short-term wind power generation are becoming increasingly popular as input to multistage decision-making problems e.g…”
Get full text

DiffCoder: Enhancing Large Language Model on API Invocation via Analogical Code Exercises by Zan, Daoguang, Yu, Ailun, Shen, Bo, Chen, Bei, Li, Wei, Gong, Yongshun, Chen, Xiaolin, Yao, Yafen, Luo, Weihua, Guan, Bei, Liu, Yan, Wang, Yongji, Wang, Qianxiang, Cui, Lizhen

“…The task of code generation aims to generate code solutions based on given programming problems…”
Get full text

SEDGE: Symbolic example data generation for dataflow programs by Kaituo Li, Reichenbach, Christoph, Smaragdakis, Yannis, Diao, Yanlei, Csallner, Christoph

“… Past work demonstrated effective ways to generate small example data sets that exercise operators in the Pig platform, used to generate Hadoop map-reduce programs…”
Get full text

CSEPrompts: A Benchmark of Introductory Computer Science Prompts by Nishat Raihan, Goswami, Dhiman, Sadiya Sayara Chowdhury Puspo, Newman, Christian, Ranasinghe, Tharindu, Zampieri, Marcos

“…Recent advances in AI, machine learning, and NLP have led to the development of a new generation of Large Language Models (LLMs…”
Get full text

A Track-Based Conference Scheduling Problem by Riquelme, Fabian, Montero, Elizabeth, Pérez-Cáceres, Leslie, Rojas-Morales, Nicolás

“…The scheduling of conferences is a challenging task that aims at creating successful conference programs that fulfill an often wide variety of requirements. In…”
Get full text

Single muscle fibre biomechanics and biomechatronics – The challenges, the pitfalls and the future by Friedrich, Oliver, Haug, Michael, Reischl, B, Prölß, G, Kiriaev, Leon, Head, Stewart I, Reid, Michael B

“… We review major standard systems available from research labs and commercial sources, and benchmark those to our recently developed automated MyoRobot biomechatronics platform that provides…”
Get full text

Coordinating self-sizing and self-repair managers for multi-tier systems by Gueye, Soguy Mak-Karé, De Palma, Noël, Rutten, Éric, Tchana, Alain, Berthier, Nicolas

“…Computing systems have become more and more distributed and heterogeneous, making their manual administration difficult and error-prone. The Autonomic…”
Get full text

Strategic Behaviour in a Capacity Market? The New Irish Electricity Market Design by Teirilä, Juha, Ritz, Robert A.

“…The transition to a low-carbon power system requires growing the share of generation from (intermittent…”
Get full text

Evaluating and Aligning CodeLLMs on Human Preference by Yang, Jian, Yang, Jiaxi, Jin, Ke, Miao, Yibo, Zhang, Lei, Yang, Liqun, Cui, Zeyu, Zhang, Yichang, Binyuan Hui, Lin, Junyang

“… Most previous code-related benchmarks, which consist of various programming exercises along with the corresponding test cases, are used as a common measure to evaluate the performance and capabilities of code LLMs…”
Get full text

Swarm inspired test case generation for online C++ programming assessment by Oi-Mean Foong, Quang-Trung Tran, Suet-Peng Yong, Rais, Helmi Md

“… Moreover, they also need to define test cases for different programming exercises in order to assess students' code…”
Get full text

Improving differential evolution through a unified approach by Padhye, Nikhil, Bhardawaj, Piyush, Deb, Kalyanmoy

“…— Initialization, Selection, Generation and Replacement , which are sufficient to describe…”
Get full text

Generation of variational standard plant room solutions by Medjdoub, Benachir, Richens, Paul, Barnard, Nick

“…We have used the object-based CAD programming to take advantage of standardisation to handle the selection, sizing, layout and (potentially…”
Get full text

MixFX-SCORE: Heterogeneous Fixed-Point Compilation of Dataflow Computations by Deheng Ye, Kapre, Nachiket

“… When designing circuits for reconfigurable hardware, we can exercise independent control over bitwidth selection of each variable in the computation…”
Get full text

Test input reduction for result inspection to facilitate fault localization by Hao, Dan, Xie, Tao, Zhang, Lu, Wang, Xiaoyin, Sun, Jiasu, Mei, Hong

“…Testing-based fault-localization (TBFL) approaches often require the availability of high-statement-coverage test suites that sufficiently exercise the areas around the faults…”
Get full text

Health Club Industry Benchmarks Show Some Good and Some Bad by Scudder, Michael Scott

“… Group exercise class attendance increases crested in early 2013 and-though still a healthy part of many clubs' programming-needs a shot of adrenaline and stronger promotion…”
Get full text