PR#59: WIP: Fix test_gdb when CET protection is enabled - rpms/python3

rpms / python3

#59 WIP: Fix test_gdb when CET protection is enabled

Closed 5 years ago by vstinner. Opened 5 years ago by vstinner.

rpms/ vstinner/python3 test_gdb_cet into master

Fix test_gdb when CET protection is enabled

Victor Stinner • 5 years ago

360ca47

00309-test_gdb_cet_protection.patch

file added

+87

		`@@ -0,0 +1,87 @@`
		`+ commit 0ce31d340b264a550a3c574e1d6913f4affd4669`
		`+ Author: Miss Islington (bot) <31488909+miss-islington@users.noreply.github.com>`
		`+ Date: Tue Oct 9 08:21:17 2018 -0700`
		`+`
		`+ bpo-32962: Fix test_gdb failure in debug build with -mcet -fcf-protection -O0 (GH-9656)`
		`+`
		`+`
		`+ When Python is built with the intel control-flow protection flags,`
		`+ -mcet -fcf-protection, gdb is not able to read the stack without`
		`+ actually jumping inside the function. This means an extra`
		`+ 'next' command is required to make the $pc (program counter)`
		`+ enter the function and make the stack of the function exposed to gdb.`
		`+`
		`+ Co-Authored-By: Marcel Plch <gmarcel.plch@gmail.com>`
		`+`
		`+ (cherry picked from commit 9b7c74ca32d1bec7128d550a9ab1b2ddc7046287)`
		`+ (cherry picked from commit 79d21331e605fdc941f947621846b8563485aab6)`
		`+`
		`+ Co-authored-by: Victor Stinner <vstinner@redhat.com>`
		`+`
		`+ diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py`
		`+ index 93a2c7dd57..0f950b2325 100644`
		`+ --- a/Lib/test/test_gdb.py`
		`+ +++ b/Lib/test/test_gdb.py`
		`+ @@ -54,6 +54,23 @@ checkout_hook_path = os.path.join(os.path.dirname(sys.executable),`
		`+`
		`+ PYTHONHASHSEED = '123'`
		`+`
		`+ +`
		`+ +def cet_protection():`
		`+ + cflags = sysconfig.get_config_var('CFLAGS')`
		`+ + if not cflags:`
		`+ + return False`
		`+ + flags = cflags.split()`
		`+ + # True if "-mcet -fcf-protection" options are found, but false`
		`+ + # if "-fcf-protection=none" or "-fcf-protection=return" is found.`
		`+ + return (('-mcet' in flags)`
		`+ + and any((flag.startswith('-fcf-protection')`
		`+ + and not flag.endswith(("=none", "=return")))`
		`+ + for flag in flags))`
		`+ +`
		`+ +# Control-flow enforcement technology`
		`+ +CET_PROTECTION = cet_protection()`
		`+ +`
		`+ +`
		`+ def run_gdb(args, *env_vars):`
		`+ """Runs gdb in --batch mode with the additional arguments given by *args.`
		`+`
		`+ @@ -162,6 +179,12 @@ class DebuggerTests(unittest.TestCase):`
		`+ commands += ['set print entry-values no']`
		`+`
		`+ if cmds_after_breakpoint:`
		`+ + if CET_PROTECTION:`
		`+ + # bpo-32962: When Python is compiled with -mcet`
		`+ + # -fcf-protection, function arguments are unusable before`
		`+ + # running the first instruction of the function entry point.`
		`+ + # The 'next' command makes the required first step.`
		`+ + commands += ['next']`
		`+ commands += cmds_after_breakpoint`
		`+ else:`
		`+ commands += ['backtrace']`
		`+ @@ -869,9 +892,17 @@ id(42)`
		`+ id("first break point")`
		`+ l = MyList()`
		`+ ''')`
		`+ + cmds_after_breakpoint = ['break wrapper_call', 'continue']`
		`+ + if CET_PROTECTION:`
		`+ + # bpo-32962: same case as in get_stack_trace():`
		`+ + # we need an additional 'next' command in order to read`
		`+ + # arguments of the innermost function of the call stack.`
		`+ + cmds_after_breakpoint.append('next')`
		`+ + cmds_after_breakpoint.append('py-bt')`
		`+ +`
		`+ # Verify with "py-bt":`
		`+ gdb_output = self.get_stack_trace(cmd,`
		`+ - cmds_after_breakpoint=['break wrapper_call', 'continue', 'py-bt'])`
		`+ + cmds_after_breakpoint=cmds_after_breakpoint)`
		`+ self.assertRegex(gdb_output,`
		`+ r"<method-wrapper u?'__init__' of MyList object at ")`
		`+`
		`+ diff --git a/Misc/NEWS.d/next/Tests/2018-05-10-16-59-15.bpo-32962.S-rcIN.rst b/Misc/NEWS.d/next/Tests/2018-05-10-16-59-15.bpo-32962.S-rcIN.rst`
		`+ new file mode 100644`
		`+ index 0000000000..97328ebafe`
		`+ --- /dev/null`
		`+ +++ b/Misc/NEWS.d/next/Tests/2018-05-10-16-59-15.bpo-32962.S-rcIN.rst`
		`+ @@ -0,0 +1 @@`
		`+ +Fixed test_gdb when Python is compiled with flags -mcet -fcf-protection -O0.`

python3.spec

file modified

+10 -2

		`@@ -14,7 +14,7 @@`
		`# WARNING When rebasing to a new Python version,`
		`# remember to update the python3-docs package as well`
		`Version: %{pybasever}.0`
		`- Release: 10%{?dist}`
		`+ Release: 11%{?dist}`
		`License: Python`


		`@@ -320,6 +320,12 @@`
		`# See: https://bugzilla.redhat.com/show_bug.cgi?id=1609291`
		`Patch308: 00308-tls-1.3.patch`

		`+ # 00309 #`
churchyard commented 5 years ago Patch 309 is for CVE-2018-1000802. See https://fedoraproject.org/wiki/SIGs/Python/PythonPatches
		`+ # Fix test_gdb failure in debug build with "-mcet -fcf-protection -O0".`
		`+ # commit 0ce31d340b264a550a3c574e1d6913f4affd4669`
		`+ # https://bugs.python.org/issue32962`
		`+ Patch309: 00309-test_gdb_cet_protection.patch`
		`+`
		`# (New patches go here ^^^)`
		`#`
		`# When adding new patches to "python" and "python3" in Fedora, EL, etc.,`
		`@@ -1059,7 +1065,6 @@`
		`-wW --slowest --findleaks \`
		`-x test_distutils \`
		`-x test_bdist_rpm \`
		`- -x test_gdb \`
		`%ifarch %{mips64}`
		`-x test_ctypes \`
		`%endif`
		`@@ -1551,6 +1556,9 @@`
		`# ======================================================`

		`%changelog`
		`+ * Wed Oct 10 2018 Victor Stinner <vstinner@redhat.com> - 3.7.0-11`
		`+ - Fix test_gdb when CET protection is enabled on x86_64 (rhbz#1541967)`
		`+`
		`* Thu Sep 27 2018 Petr Viktorin <pviktori@redhat.com> - 3.7.0-10`
		`- Compile the debug build with -Og rather than -O0`

vstinner commented 5 years ago

test_gdb was failing when CET protection is enabled on x86_64
(rhbz#1541967): backport fix from upstream.

churchyard commented on line 14 of python3.spec 5 years ago

Patch 309 is for CVE-2018-1000802. See https://fedoraproject.org/wiki/SIGs/Python/PythonPatches

churchyard commented 5 years ago

Will this be in 3.7.1?

vstinner commented 5 years ago

Will this be in 3.7.1?

I'm not sure, since 3.7.1 has been tagged at 2018-10-09, and I pushed my fixed at 2018-10-09. I hope that it will be part of 3.7.1 :-) I created this PR to reenable test_gdb and to learn how to create a PR also :-)

vstinner commented 5 years ago

Too bad. test_gdb failed on the CI :-(

https://koji.fedoraproject.org/koji/taskinfo?taskID=30181306

Example:

+ echo STARTING: CHECKING OF PYTHON FOR CONFIGURATION: debug
(...)
======================================================================
FAIL: test_NULL_ob_type (test.test_gdb.PrettyPrintTests)
Ensure that a PyObject* with NULL ob_type is handled gracefully
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.7.0/Lib/test/test_gdb.py", line 485, in test_NULL_ob_type
    'set v->ob_type=0')
  File "/builddir/build/BUILD/Python-3.7.0/Lib/test/test_gdb.py", line 456, in assertSane
    cmds_after_breakpoint=cmds_after_breakpoint)
  File "/builddir/build/BUILD/Python-3.7.0/Lib/test/test_gdb.py", line 242, in get_gdb_repr
    self.fail('Unexpected gdb output: %r\n%s' % (gdb_output, gdb_output))
AssertionError: Unexpected gdb output: 'Breakpoint 1 (builtin_id) pending.\n[Thread debugging using libthread_db enabled]\nUsing host libthread_db library "/lib64/libthread_db.so.1".\n'
Breakpoint 1 (builtin_id) pending.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

churchyard commented 5 years ago

Renamed to WIP.

cstratak commented 5 years ago

The issue that happens now is a different one. The PR actually fixes the bug, but this is due to the rebase of gdb which changes some of its python API.

cstratak commented 5 years ago

Reported here:

https://bugs.python.org/issue34811

vstinner commented 5 years ago

test_gdb fails on Fedora Rawhide because of a bug in gdb: https://bugzilla.redhat.com/show_bug.cgi?id=1638798

Metadata Update from @churchyard:
- Pull-request tagged with: WIP, bugfix, tests

5 years ago

Metadata Update from @churchyard:
- Pull-request tagged with: blocked

5 years ago

vstinner commented 5 years ago

This PR is no longer needed, the fix is part of Python 3.7.1 and the python3 package has been updated to Python 3.7.1. I just forgot to close this PR :-)

Pull-Request has been closed by vstinner

5 years ago