Program execution monitoring using deep memory tracing
Uloženo v:
| Název: | Program execution monitoring using deep memory tracing |
|---|---|
| Patent Number: | 11782,819 |
| Datum vydání: | October 10, 2023 |
| Appl. No: | 16/945149 |
| Application Filed: | July 31, 2020 |
| Abstrakt: | A user-annotated reference implementation identifies variable values generated by the reference implementation during its execution. A software implementation under analysis is executed. Variable values in the running memory of the program code of the software implementation under analysis, during its execution, are identified and copied. The variable values traced from the running memory of the program code are compared against the annotated variable values generated by the reference implementation, to determine a similarity between the program code under analysis, and the reference implementation. An output is generated that is indicative of whether the traced variables from the program code under analysis are the same as the annotated variable values generated by the reference implementation. |
| Inventors: | Microsoft Technology Licensing, LLC (Redmond, WA, US) |
| Assignees: | Microsoft Technology Licensing, LLC (Redmond, WA, US) |
| Claim: | 1. A computer system, comprising: at least one processor; and memory storing instructions executable by the at least one processor, wherein the instructions, when executed, cause the computer system to: display a reference software implementation for user annotation; detect a user input annotating a portion of the reference software implementation as a variable value to be monitored; execute the reference software implementation; obtain an expected variable value based on tracing the variable value to be monitored in a running memory used by the reference software implementation during execution of the reference software implementation; trace a running memory, used by a software implementation under analysis, during execution of the software implementation under analysis; generate a comparison result indicative of whether the expected variable value appears in a copy of the running memory used by the software implementation under analysis; and generate a verification output indicative of whether the software implementation under analysis corresponds to the reference software implementation, based on the comparison result, the verification output including a message indicative of whether the expected variable value appears in the running memory used by the software implementation under analysis. |
| Claim: | 2. The computer system of claim 1 wherein the instructions, when executed, cause the computer system to: copy the running memory used by the reference software implementation during execution of the reference software implementation. |
| Claim: | 3. The computer system of claim 1 wherein the instructions, when executed, cause the computer system to: detect a user input identifying a rule that applies a logical operator to one or more variable values generated in the running memory used by the reference software implementation during execution of the reference software implementation and to annotate the reference software implementation with the rule. |
| Claim: | 4. The computer system of claim 3 wherein the instructions, when executed, cause the computer system to: trace the one or more variable values in the running memory used by the reference software application during execution of the reference software implementation to obtain an expected rule result; and apply the rule to variable values in the running memory used by the software implementation under analysis to obtain a rule result under analysis, and generate a comparison result indicative of whether the expected rule result matches the rule result under analysis. |
| Claim: | 5. The computer system of claim 1 wherein the instructions, when executed, cause the computer system to: define the variable value as an input vector based on the user input; and generate the comparison result indicative of whether the input vector appears in the copy of the running memory used by the software implementation under analysis. |
| Claim: | 6. The computer system of claim 5 wherein the variable value to be monitored comprises a set of variable values, including an initial value, an intermediate value, and a final value. |
| Claim: | 7. The computer system of claim 1 wherein the instructions, when executed, cause the computer system to: identify an additional reference software implementation against which the software implementation under analysis is to be compared, and access an additional expected variable value corresponding to the additional reference software implementation; and generate an additional comparison result indicative of whether the additional expected variable value appears in the copy of the running memory used by the software implementation under analysis, and generate an output indicative of whether the software implementation under analysis corresponds to the additional reference software implementation, based on the additional comparison result, along with a message indicative of whether the additional expected variable value appears in the copy of the running memory used by the software implementation under analysis. |
| Claim: | 8. The computer system of claim 1 wherein the instructions, when executed, cause the computer system to, determine whether the software implementation under analysis corresponds to the reference software implementation, based on the comparison result, and, if not, then display a conversion user input mechanism that is actuated to convert the software implementation under analysis to an additional reference software implementation. |
| Claim: | 9. The computer system of claim 8 wherein the instructions, when executed, cause the computer system to: detect user actuation of the conversion user input mechanism, and display the software implementation under analysis for user annotation to identify a portion of the software implementation under analysis as an additional variable value to be monitored, to obtain an additional reference software implementation; and detect user annotation of the software implementation under analysis, and save the additional reference software implementation for comparison against an additional software implementation under analysis. |
| Claim: | 10. The computer system of claim 1 wherein the instructions, when executed, cause the computer system to: detect a change in the variable value in the software implementation under analysis; based on the change in the variable value, obtain a copy of the running memory used by the software implementation under analysis; and generate the comparison result based on the obtained copy of the running memory. |
| Claim: | 11. A computer implemented method, comprising: displaying a reference software implementation for user annotation; detecting a user input annotating a portion of the reference software implementation as a variable value to be monitored; executing the reference software implementation; tracing the variable value to be monitored in a running memory used by the reference software implementation, during execution of the reference software implementation, to obtain an expected variable value; executing a software implementation under analysis; copying a running memory, used by the software implementation under analysis, during execution of the software implementation under analysis; generating a comparison result indicative of whether the expected variable value appears in the copy of the running memory used by the software implementation under analysis; and generating an output indicative of whether the software implementation under analysis corresponds to the reference software implementation, based on the comparison result, along with a message indicative of whether the expected variable value appears in the copy of the running memory used by the software implementation under analysis. |
| Claim: | 12. The computer implemented method of claim 11 wherein tracing comprises: copying the running memory used by the reference software implementation during execution of the reference software implementation. |
| Claim: | 13. The computer implemented method of claim 11 and further comprising: detecting a user input identifying a rule that applies a logical operator to one or more variable values generated in the running memory used by the reference software implementation during execution of the reference software implementation; and annotating the reference software implementation with the rule. |
| Claim: | 14. The computer implemented method of claim 13 and further comprising: tracing the one or more variable values in the running memory used by the reference software application during execution of the reference software implementation to obtain an expected rule result. |
| Claim: | 15. The computer implemented method of claim 14 wherein generating the comparison result comprises: applying the rule to variable values in the running memory used by the software implementation under analysis to obtain a rule result under analysis; and generating a comparison result indicative of whether the expected rule result matches the rule result under analysis. |
| Claim: | 16. The computer implemented method of claim 11 wherein generating a comparison result comprises: identifying an additional reference software implementation against which the software implementation under analysis is to be compared; and accessing an additional expected variable value corresponding to the additional reference software implementation. |
| Claim: | 17. The computer implemented method of claim 16 and further comprising: generating an additional comparison result indicative of whether the additional expected variable value appears in the copy of the running memory used by the software implementation under analysis; and generating an output indicative of whether the software implementation under analysis corresponds to the additional reference software implementation, based on the additional comparison result, along with a message indicative of whether the additional expected variable value appears in the copy of the running memory used by the software implementation under analysis. |
| Claim: | 18. The computer implemented method of claim 11 wherein generating an output comprises: if the software implementation under analysis does not correspond to the reference software implementation, based on the comparison result, then displaying a conversion user input mechanism that is actuated to convert the software implementation under analysis to an additional reference software implementation. |
| Claim: | 19. The computer implemented method of claim 18 and further comprising: detecting user actuation of the conversion user input mechanism; displaying the software implementation under analysis for user annotation to identify a portion of the software implementation under analysis as an additional variable value to be monitored, to obtain an additional reference software implementation; detecting user annotation of the software implementation under analysis; and saving the additional reference software implementation for comparison against an additional software implementation under analysis. |
| Claim: | 20. A computer system, comprising: a test point configuration system that displays a reference software implementation for user annotation and detects a user input annotating a portion of the reference software implementation as a variable value to be monitored; a rule generation and annotation component configured to detect a user input identifying a rule that applies a logical operator to one or more variable values generated in the running memory used by the reference software implementation during execution of the reference software implementation and to annotate the reference software implementation with the rule; an execution system that executes the reference software implementation and that executes a software implementation under analysis; a tracing system that copies the running memory used by the reference software implementation during execution of the reference software implementation to obtain an expected variable value and that traces a running memory, used by the software implementation under analysis, during execution of the software implementation under analysis; a rule application system that generates a comparison result indicative of whether the expected variable value appears in the copy of the running memory used by the software implementation under analysis; and a verification output generator that generates a verification output indicative of whether the software implementation under analysis corresponds to the reference software implementation, based on the comparison result, the verification output including a message indicative of whether the expected variable value appears in the running memory used by the software implementation under analysis. |
| Patent References Cited: | 6249882 June 2001 Testardi 6434738 August 2002 Arnow 6748584 June 2004 Witchsl et al. 8021221 September 2011 Lydon 8261237 September 2012 Zeidman 8353056 January 2013 Han 9207915 December 2015 Asaduilah et al. 20090286218 November 2009 Johnson 20100100774 April 2010 Ding 20100242023 September 2010 Han 20100287539 November 2010 Park 20110289482 November 2011 Bentley 20120101929 April 2012 Howard et al. 20130219374 August 2013 Jain 20150039541 February 2015 Kapur 20150227364 August 2015 Asadullah 20170249231 August 2017 Bofferding 20180129597 May 2018 Anaya 20200334021 October 2020 Bucuvalas 103729580 April 2014 |
| Other References: | F. AlShamsi and A. Elnagar, “An automated assessment and reporting tool for introductory Java programs,” 2011 International Conference on Innovations in Information Technology, 2011, pp. 324-329, doi: 10.1109/INNOVATIONS.2011.5893842. (Year: 2011 ). cited by examiner X. Liu, S. Wang, P. Wang and D. Wu, “Automatic Grading of Programming Assignments: An Approach Based on Formal Semantics ,” 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET),2019, pp. 126-137, doi: 10.1109/ICSE-SEET.2019.00022. (Year: 2019). cited by examiner S. Zougari, M. Tanana and A. Lyhyaoui, “Hybrid assessment method for programming assignments,” 2016 4th IEEE International Colloquium on Information Science and Technology (CiSt), 2016, pp. 564-569, doi: 10.1109/CIST.2016.7805112. (Year: 2016). cited by examiner G. Robles and J. M. Gonzalez-Barahona, “Mining student repositories to gain learning analytics. An experience report,” 2013 IEEE Global Engineering Education Conference (EDUCON), 2013, pp. 1249-1254, doi: 10.1109/EduCon.2013.6530267. (Year: 2013). cited by examiner E. Stankov, M. Jovanov, B. Kostadinov and A. Madevska Bogdanova, “A new model for collaborative learning of programming using source code similarity detection,” 2015 IEEE Global Engineering Education Conference (EDUCON), 2015, pp. 709-715, doi: 10.1109/EDUCON.2015.7096047. (Year: 2015). cited by examiner Xiaohong Su, Jing Qiu, Tiantian Wang and Lingling Zhao, “Optimization and improvements of a Moodie-Based online learning system for C programming,” 2016 IEEE Frontiers in Education Conference (FIE), 2016, pp. 1-8, doi: 10.1109/FIE.2016.7757699. (Year: 2016). cited by examiner “Automate Grading & Personalize Feedback”, Retrieved from: https://okpy.org/, Retrieved Date: Jan. 7, 2020, 4 pages. cited by applicant “Get help with GitHub Classroom”, Retrieved from: https://classroom.github.com/help/auto-grading, Retrieved Date: Jan. 7, 2020, 1 Page. cited by applicant “Otter-Grader Documentation”, Retrieved from: https://otter-grader.readthedecs.io/en/latest/, Retrieved Date: Jan. 7, 2020, 1 Page. cited by applicant “Simplify Teaching”, Retrieved from: http://www.autolabproject.com/, Retrieved Date: Jan. 7, 2020, 9 Pages. cited by applicant Corbett et al., “Locus of Feedback Control in Computer-Based Tutonng Impact on Learning Rate, Achievement and Attitudes”, In Proceedings of the SIGCHI conference on Human factors in computing: systems, Mar. 31, 2001, pp. 245-252. cited by applicant Goldstein, et al. “Understanding How Work Habits Influence Student Performance”, In Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education, Jul. 15, 2019, pp. 154-160. cited by applicant Gupta, et al., “Source Code Plagiarism Detection using Multi Layered Approach for C Language Programs”, In International Journal of Computer Applications, vol. 105. No. 12, Nov. 2014, pp. 5-11. cited by applicant Jhi, et al., “Value-Based Program Characterization and its Application to Software Plagiarism Deteotion”, In Proceedings of the 33rd International Conference on Software Engineering, May 21, 2011, pp. 756-765. cited by applicant Skalka, et al., “Automated Assessment in Learning and Teaching Programming Languages using Virtual Learning Environment”, In Proceedings of IEEE Global Engineering Education Conference, Apr. 8, 2019, 10 Pages. cited by applicant “Tracing (software)”, Retrieved from: https://en.wikipedia.org/wiki/Tracing_(software), Apr. 14, 2016, 4 Pages. cited by applicant “International Search Report and Written Opinion Issued in PCT Application No. PCT/US21/029255”, dated Aug. 16, 2021, 11 Pages. cited by applicant |
| Primary Examiner: | Lyons, Andrew M. |
| Attorney, Agent or Firm: | Volkmann, Christopher J. Kelly, Holt & Christenson, PLLC |
| Přístupové číslo: | edspgr.11782819 |
| Databáze: | USPTO Patent Grants |
| Abstrakt: | A user-annotated reference implementation identifies variable values generated by the reference implementation during its execution. A software implementation under analysis is executed. Variable values in the running memory of the program code of the software implementation under analysis, during its execution, are identified and copied. The variable values traced from the running memory of the program code are compared against the annotated variable values generated by the reference implementation, to determine a similarity between the program code under analysis, and the reference implementation. An output is generated that is indicative of whether the traced variables from the program code under analysis are the same as the annotated variable values generated by the reference implementation. |
|---|