_testinternalcapi: optimize_cfg / compiler_codegen const table mismatches cause crashes

[iritkatriel]

Crash report

What happened?

Using _testinternalcapi.compiler_codegen together with _testinternalcapi.optimize_cfg exposed several problems: easy-to-hit segfaults when the const list did not match the instruction stream, metadata["consts"] built at the wrong time so it could miss constants added by _PyCodegen_AddReturnAtEnd (notably None for implicit module return), and a missing error path in the CFG optimizer that left an exception set and then crashed instead of raising ValueError.

from _testinternalcapi import compiler_codegen, optimize_cfg
import ast
# (A) Wrong const list for the real bytecode
insts, meta = compiler_codegen(ast.parse("x = (1, 2)", optimize=1), "<s>", 0)
optimize_cfg(insts, [(1, 2)], 0)  # segfault
# (B) If consts were snapshotted before AddReturnAtEnd, meta["consts"] could be wrong;
# with (B) fixed, this documents the failure mode:
insts, meta = compiler_codegen(ast.parse("pass", mode="exec"), "<s>", 0)
optimize_cfg(insts, [], 0)  # segfault without proper NULL handling after get_const_value

CPython versions tested on:

3.15

Operating systems tested on:

macOS

Output from running 'python -VV' on the command line:

No response

Linked PRs

• gh-148973: fix segfault on mismatch between consts size and oparg in compiler #148974
• [3.14] gh-148973: fix segfault on mismatch between consts size and oparg in compiler (GH-148974) #148980
• [3.13] gh-148973: fix segfault on mismatch between consts size and oparg in compiler (GH-148974) #148997