Preventing stack buffer overflow attacks
Uložené v:
| Názov: | Preventing stack buffer overflow attacks |
|---|---|
| Patent Number: | 9,251,373 |
| Dátum vydania: | February 02, 2016 |
| Appl. No: | 13/804056 |
| Application Filed: | March 14, 2013 |
| Abstrakt: | Improved buffer overflow protection for a computer function call stack is provided by placing a predetermined ShadowKEY value on a function's call stack frame and copying the ShadowKEY, a caller EBP, and a return pointer are pushed onto a duplicate stack. The prologue of the function may be modified for this purpose. The function epilogue is modified to compare the current values of the ShadowKEY, caller EBP, and the return pointer on the function stack to the copies stored on the duplicate stack. If they are not identical, an overflow is detected. The preserved copies of these values may be copied back to the function stack frame thereby enabling execution of the process to continue. A function prologue and epilogue may be modified during compilation of the program. |
| Inventors: | Northern Borders University (Arar, Northern Borders Province, SA); AlHarbi, Khalid Nawaf (St. Manamah, SA); Lin, Xiaodong (Waterloo, CA) |
| Claim: | 1. A method of protecting a function stack frame on a computer call stack, the function stack frame corresponding to a function, the method comprising: modifying a prologue of the function wherein, when executed, the prologue performs the steps of: creating in the function stack frame a starting ShadowKEY value, a starting extended base pointer (EBP) value, and a starting return pointer value; and creating a duplicate stack frame on a duplicate stack different from the computer call stack, the duplicate stack frame comprising the starting ShadowKEY value, the starting EBP value, and the starting return pointer value; and modifying an epilogue of the function wherein, when executed, the epilogue performs the steps of: fetching from the function stack frame a finishing ShadowKEY value, a finishing EBP value, and a finishing return pointer value; fetching from the duplicate stack frame the starting ShadowKEY value, the starting EBP value, and the starting return pointer value; comparing the starting ShadowKEY value, the starting EBP value, and the starting return pointer value to the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value, respectively; and if any one of the starting ShadowKEY value, the starting EBP value, and the starting return pointer value is unequal to the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value, respectively, replacing the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value in the function stack frame with the starting ShadowKEY value, the starting EBP value, and the starting return pointer value, respectively; and wherein the starting ShadowKEY value is pushed onto a duplicate stack which is located below the function's stack frame in memory. |
| Claim: | 2. A method of protecting a function stack frame on a computer call stack, the function stack frame corresponding to a function, the method comprising: modifying a prologue of the function wherein, when executed, the prologue performs the steps of: creating in the function stack frame a starting ShadowKEY value, a starting extended base pointer (EBP) value, and a starting return pointer value; and creating a duplicate stack frame on a duplicate stack different from the computer call stack, the duplicate stack frame comprising the starting ShadowKEY value, the starting EBP value, and the starting return pointer value; and modifying an epilogue of the function wherein, when executed, the epilogue performs the steps of: fetching from the function stack frame a finishing ShadowKEY value, a finishing EBP value, and a finishing return pointer value; fetching from the duplicate stack frame the starting ShadowKEY value, the starting EBP value, and the starting return pointer value; comparing the starting ShadowKEY value, the starting EBP value, and the starting return pointer value to the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value, respectively; and if any one of the starting ShadowKEY value, the starting EBP value, and the starting return pointer value is unequal to the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value, respectively, replacing the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value in the function stack frame with the starting ShadowKEY value, the starting EBP value, and the starting return pointer value, respectively; and wherein the duplicate stack frame is located below the function's stack frame in memory and comprises copies of a ShadowKEY, a caller EBP, and a return pointer. |
| Claim: | 3. A method of protecting a function stack frame on a computer call stack, the function stack frame corresponding to a function, the method comprising: modifying a prologue of the function wherein, when executed, the prologue performs the steps of: creating in the function stack frame a starting ShadowKEY value, a starting extended base pointer (EBP) value, and a starting return pointer value; and creating a duplicate stack frame on a duplicate stack different from the computer call stack, the duplicate stack frame comprising the starting ShadowKEY value, the starting EBP value, and the starting return pointer value; and modifying an epilogue of the function wherein, when executed, the epilogue performs the steps of: fetching from the function stack frame a finishing ShadowKEY value, a finishing EBP value, and a finishing return pointer value; fetching from the duplicate stack frame the starting ShadowKEY value, the starting EBP value, and the starting return pointer value; comparing the starting ShadowKEY value, the starting EBP value, and the starting return pointer value to the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value, respectively; and if any one of the starting ShadowKEY value, the starting EBP value, and the starting return pointer value is unequal to the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value, respectively, replacing the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value in the function stack frame with the starting ShadowKEY value, the starting EBP value, and the starting return pointer value, respectively; and wherein the starting ShadowKEY value is specified between the caller EBP and the local variables segments of the stack frame in the duplicate stack located below the function's stack frame in memory. |
| Claim: | 4. A method of protecting a function stack frame on a computer call stack, the function stack frame corresponding to a function, the method comprising: modifying a prologue of the function wherein, when executed, the prologue performs the steps of: creating in the function stack frame a starting ShadowKEY value, a starting extended base pointer (EBP) value, and a starting return pointer value; and creating a duplicate stack frame on a duplicate stack different from the computer call stack, the duplicate stack frame comprising the starting ShadowKEY value, the starting EBP value, and the starting return pointer value; and modifying an epilogue of the function wherein, when executed, the epilogue performs the steps of: fetching from the function stack frame a finishing ShadowKEY value, a finishing EBP value, and a finishing return pointer value; fetching from the duplicate stack frame the starting ShadowKEY value, the starting EBP value, and the starting return pointer value; comparing the starting ShadowKEY value, the starting EBP value, and the starting return pointer value to the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value, respectively; and if any one of the starting ShadowKEY value, the starting EBP value, and the starting return pointer value is unequal to the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value, respectively, replacing the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value in the function stack frame with the starting ShadowKEY value, the starting EBP value, and the starting return pointer value, respectively; and wherein the comparing comprises: comparing the starting ShadowKEY value in the duplicate Stack to the finishing ShadowKEY value of the function Stack; and only if the starting ShadowKEY value is equal to the finishing ShadowKEY value: comparing the starting EBP value in the duplicate Stack to the finishing EBP value of the function Stack; and only if the EBP value is equal to the finishing EBP value: comparing the starting return pointer value in the duplicate Stack to the finishing return pointer value of the function Stack. |
| Claim: | 5. A method of protecting a function stack frame on a computer call stack, the function stack frame corresponding to a function, the method comprising: modifying a prologue of the function wherein, when executed, the prologue performs the steps of: creating in the function stack frame a starting ShadowKEY value, a starting extended base pointer (EBP) value, and a starting return pointer value; and creating a duplicate stack frame on a duplicate stack different from the computer call stack, the duplicate stack frame comprising the starting ShadowKEY value, the starting EBP value, and the starting return pointer value; and modifying an epilogue of the function wherein, when executed, the epilogue performs the steps of: fetching from the function stack frame a finishing ShadowKEY value, a finishing EBP value, and a finishing return pointer value; fetching from the duplicate stack frame the starting ShadowKEY value, the starting EBP value, and the starting return pointer value; comparing the starting ShadowKEY value, the starting EBP value, and the starting return pointer value to the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value, respectively; and if any one of the starting ShadowKEY value, the starting EBP value, and the starting return pointer value is unequal to the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value, respectively, replacing the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value in the function stack frame with the starting ShadowKEY value, the starting EBP value, and the starting return pointer value, respectively; and wherein the starting EBP value is pushed onto a duplicate stack which is located below the function's stack frame in memory after the starting ShadowKEY. |
| Claim: | 6. A method of protecting a function stack frame on a computer call stack, the function stack frame corresponding to a function, the method comprising: modifying a prologue of the function wherein, when executed, the prologue performs the steps of: creating in the function stack frame a starting ShadowKEY value, a starting extended base pointer (EBP) value, and a starting return pointer value; and creating a duplicate stack frame on a duplicate stack different from the computer call stack, the duplicate stack frame comprising the starting ShadowKEY value, the starting EBP value, and the starting return pointer value; and modifying an epilogue of the function wherein, when executed, the epilogue performs the steps of: fetching from the function stack frame a finishing ShadowKEY value, a finishing EBP value, and a finishing return pointer value; fetching from the duplicate stack frame the starting ShadowKEY value, the starting EBP value, and the starting return pointer value; comparing the starting ShadowKEY value, the starting EBP value, and the starting return pointer value to the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value, respectively; and if any one of the starting ShadowKEY value, the starting EBP value, and the starting return pointer value is unequal to the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value, respectively, replacing the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value in the function stack frame with the starting ShadowKEY value, the starting EBP value, and the starting return pointer value, respectively; and wherein the starting return pointer value is pushed onto a duplicate stack which is located below the function's stack frame in memory after the starting EBP value. |
| Claim: | 7. A computer program product comprising a computer-readable memory storing instructions executable by a computer to protect a function stack frame on a computer call stack, the function stack frame corresponding to a function of: modifying a prologue of the function wherein, when executed, the prologue performs the steps of: creating in the function stack frame a starting ShadowKEY value, a starting extended base pointer (EBP) value, and a starting return pointer value; and creating a duplicate stack frame on a duplicate stack different from the computer call stack, the duplicate stack frame comprising the starting ShadowKEY value, the starting EBP value, and the starting return pointer value; and modifying an epilogue of the function wherein, when executed, the epilogue performs the steps of: fetching from the function stack frame a finishing ShadowKEY value, a finishing EBP value, and a finishing return pointer value; fetching from the duplicate stack frame the starting ShadowKEY value, the starting EBP value, and the starting return pointer value; comparing the starting ShadowKEY value, the starting EBP value, and the starting return pointer value to the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value, respectively; and if any one of the starting ShadowKEY value, the starting EBP value, and the starting return pointer value is unequal to the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value, respectively, replacing the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value in the function stack frame with the starting ShadowKEY value, the starting EBP value, and the starting return pointer value, respectively; and wherein the starting ShadowKEY value is pushed onto a duplicate stack which is located below the function's stack frame in memory. |
| Claim: | 8. A computer program product comprising a computer-readable memory storing instructions executable by a computer to protect a function stack frame on a computer call stack, the function stack frame corresponding to a function of: modifying a prologue of the function wherein, when executed, the prologue performs the steps of: creating in the function stack frame a starting ShadowKEY value, a starting extended base pointer (EBP) value, and a starting return pointer value; and creating a duplicate stack frame on a duplicate stack different from the computer call stack, the duplicate stack frame comprising the starting ShadowKEY value, the starting EBP value, and the starting return pointer value; and modifying an epilogue of the function wherein, when executed, the epilogue performs the steps of: fetching from the function stack frame a finishing ShadowKEY value, a finishing EBP value, and a finishing return pointer value; fetching from the duplicate stack frame the starting ShadowKEY value, the starting EBP value, and the starting return pointer value; comparing the starting ShadowKEY value, the starting EBP value, and the starting return pointer value to the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value, respectively; and if any one of the starting ShadowKEY value, the starting EBP value, and the starting return pointer value is unequal to the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value, respectively, replacing the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value in the function stack frame with the starting ShadowKEY value, the starting EBP value, and the starting return pointer value, respectively; and wherein the duplicate stack frame is located below the function's stack frame in memory and comprises copies of a ShadowKEY, a caller EBP, and a return pointer. |
| Claim: | 9. A computer program product comprising a computer-readable memory storing instructions executable by a computer to protect a function stack frame on a computer call stack, the function stack frame corresponding to a function of: modifying a prologue of the function wherein, when executed, the prologue performs the steps of: creating in the function stack frame a starting ShadowKEY value, a starting extended base pointer (EBP) value, and a starting return pointer value; and creating a duplicate stack frame on a duplicate stack different from the computer call stack, the duplicate stack frame comprising the starting ShadowKEY value, the starting EBP value, and the starting return pointer value; and modifying an epilogue of the function wherein, when executed, the epilogue performs the steps of: fetching from the function stack frame a finishing ShadowKEY value, a finishing EBP value, and a finishing return pointer value; fetching from the duplicate stack frame the starting ShadowKEY value, the starting EBP value, and the starting return pointer value; comparing the starting ShadowKEY value, the starting EBP value, and the starting return pointer value to the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value, respectively; and if any one of the starting ShadowKEY value, the starting EBP value, and the starting return pointer value is unequal to the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value, respectively, replacing the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value in the function stack frame with the starting ShadowKEY value, the starting EBP value, and the starting return pointer value, respectively; and wherein the comparing comprises: comparing the starting ShadowKEY value in the duplicate Stack to the finishing ShadowKEY value of the function Stack; and only if the starting ShadowKEY value is equal to the finishing ShadowKEY value: comparing the starting EBP value in the duplicate Stack to the finishing EBP value of the function Stack; and only if the EBP value is equal to the finishing EBP value: comparing the starting return pointer value in the duplicate Stack to the finishing return pointer value of the function Stack. |
| Claim: | 10. A computer program product comprising a computer-readable memory storing instructions executable by a computer to protect a function stack frame on a computer call stack, the function stack frame corresponding to a function of: modifying a prologue of the function wherein, when executed, the prologue performs the steps of: creating in the function stack frame a starting ShadowKEY value, a starting extended base pointer (EBP) value, and a starting return pointer value; and creating a duplicate stack frame on a duplicate stack different from the computer call stack, the duplicate stack frame comprising the starting ShadowKEY value, the starting EBP value, and the starting return pointer value; and modifying an epilogue of the function wherein, when executed, the epilogue performs the steps of: fetching from the function stack frame a finishing ShadowKEY value, a finishing EBP value, and a finishing return pointer value; fetching from the duplicate stack frame the starting ShadowKEY value, the starting EBP value, and the starting return pointer value; comparing the starting ShadowKEY value, the starting EBP value, and the starting return pointer value to the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value, respectively; and if any one of the starting ShadowKEY value, the starting EBP value, and the starting return pointer value is unequal to the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value, respectively, replacing the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value in the function stack frame with the starting ShadowKEY value, the starting EBP value, and the starting return pointer value, respectively; and wherein the starting EBP value is pushed onto a duplicate stack which is located below the function's stack frame in memory after the starting ShadowKEY. |
| Claim: | 11. A computer program product comprising a computer-readable memory storing instructions executable by a computer to protect a function stack frame on a computer call stack, the function stack frame corresponding to a function of: modifying a prologue of the function wherein, when executed, the prologue performs the steps of: creating in the function stack frame a starting ShadowKEY value, a starting extended base pointer (EBP) value, and a starting return pointer value; and creating a duplicate stack frame on a duplicate stack different from the computer call stack, the duplicate stack frame comprising the starting ShadowKEY value, the starting EBP value, and the starting return pointer value; and modifying an epilogue of the function wherein, when executed, the epilogue performs the steps of: fetching from the function stack frame a finishing ShadowKEY value, a finishing EBP value, and a finishing return pointer value; fetching from the duplicate stack frame the starting ShadowKEY value, the starting EBP value, and the starting return pointer value; comparing the starting ShadowKEY value, the starting EBP value, and the starting return pointer value to the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value, respectively; and if any one of the starting ShadowKEY value, the starting EBP value, and the starting return pointer value is unequal to the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value, respectively, replacing the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value in the function stack frame with the starting ShadowKEY value, the starting EBP value, and the starting return pointer value, respectively; and wherein the starting return pointer value is pushed onto a duplicate stack which is located below the function's stack frame in memory after the starting EBP value. |
| Patent References Cited: | 4878221 October 1989 Shimada 5835958 November 1998 Long et al. 6044419 March 2000 Hayek 6301699 October 2001 Hollander 6351780 February 2002 Ecclesine 6456399 September 2002 Ho 6560199 May 2003 Hoshino 6578094 June 2003 Moudgill 6826697 November 2004 Moran 6907068 June 2005 Segawa 7086088 August 2006 Narayanan 7231666 June 2007 Haugh 7251735 July 2007 Howard 7269136 September 2007 Peacock 7272748 September 2007 Conover 7328323 February 2008 Conover 7386886 June 2008 Abrams 7412004 August 2008 Faller 7475220 January 2009 Hastings 7562391 July 2009 Nachenberg 7581089 August 2009 White 7581748 September 2009 Reimers 7631249 December 2009 Borde 7634813 December 2009 Costa 7650640 January 2010 Levy 7669243 February 2010 Conti 7730322 June 2010 Johnson 7793349 September 2010 Abrams 7814333 October 2010 Choi 7853803 December 2010 Milliken 7916640 March 2011 Bishara 7949841 May 2011 Pattabiraman et al. 8078851 December 2011 Chiou 8510596 August 2013 Gupta et al. 2001/0013094 August 2001 Etoh 2003/0217277 November 2003 Narayanan 2004/0133777 July 2004 Kiriansky et al. 2004/0168078 August 2004 Brodley 2004/0255146 December 2004 Asher et al. 2005/0010804 January 2005 Bruening et al. 2005/0044292 February 2005 McKeen 2009/0249289 October 2009 Akritidis et al. 2014/0096247 April 2014 Fischer |
| Other References: | Kuperman et al., “Detection and Prevention of Stack Buffer Overflow Attacks, How to mitigate remote attacks that exploit buffer overflow vulnerabilities on the stack and enable attackers to take control of the program”, Communications of the ACM, Nov. 2005, vol. 48, No. 11, pp. 51-56. cited by applicant C. Cowan, C. Pu, D. Maier, H. Hinton, J. Walpole, P. Bakke, S. Beattie, A. Grie, P.Wagle, and Q. Zhang, “Stackguard: Automatic adaptive detection and prevention of buffer-overflow attacks,” in Proc. of the USENIX Security Symposium, Jan. 1998. cited by applicant Gupta et al., “Dynamic Code Instrumentation to Detect and Recover from Return Address Corruption”, WODA '06, May 23, 2006, Shanghai, China, pp. 65-71. cited by applicant Brandon Bray, “Compiler Security Checks in Depth”, http://msdn.microsoft.com/en-us/library/Aa290051, Feb. 2002, pp. 1-13. cited by applicant MSDN Library, Considerations when Writing Prolog/Epilog Code, http://msdn.microsoft.com/en-us/library/t2wt9aez%28v=vs.71%29.aspx, retrieved on line Jan. 5, 2013, 3 pages. cited by applicant MSDN Library, naked (C++), http://msdn.microsoft.com/en-us/library/h5w1Owxs%28v=vs.80%29.aspx, Visual Studio 2005, retrieved online Jan. 5, 2013, 3 pages. cited by applicant Qiang Zeng. Dinghao Wu. Peng Liu, Cruiser: Concurrent Heap Buffer Overflow Monitoring Using Lock-free Data Structures, http://faculty.ist.psu.edu/wu/papers/cruiser-pldi2011.pdf, PLD'11, Jun. 4-8, 2011, San Jose California, pp. 367-377. cited by applicant Dailey Paulson, Linda, “New Chips Stop Buffer Overflow Attacks”, IEEE Computer, Oct. 2004, pp. 28-30. cited by applicant Canadian Patent Application Serial No. 2,809,516; Office Action dated Sep. 15, 2015. cited by applicant |
| Assistant Examiner: | Wright, Bryan |
| Primary Examiner: | Okeke, Izunna |
| Attorney, Agent or Firm: | Oppedahl Patent Law Firm LLC |
| Prístupové číslo: | edspgr.09251373 |
| Databáza: | USPTO Patent Grants |
| Abstrakt: | Improved buffer overflow protection for a computer function call stack is provided by placing a predetermined ShadowKEY value on a function's call stack frame and copying the ShadowKEY, a caller EBP, and a return pointer are pushed onto a duplicate stack. The prologue of the function may be modified for this purpose. The function epilogue is modified to compare the current values of the ShadowKEY, caller EBP, and the return pointer on the function stack to the copies stored on the duplicate stack. If they are not identical, an overflow is detected. The preserved copies of these values may be copied back to the function stack frame thereby enabling execution of the process to continue. A function prologue and epilogue may be modified during compilation of the program. |
|---|