Mucking with sys.path breaks checking imports

[dmtucker]

Bug Report

If the first 2 entries of sys.path are pwd, mypy seems to miss type checks in imported modules.

To Reproduce

mypy==0.971

# sscce.py
import sys
import mypy.api
sys.path.insert(0, sys.path[0])
print(mypy.api.run(["good.py"])[0])

# good.py
import bad

# bad.py 
foo: str = 5

Expected Behavior

bad.py:1: error: Incompatible types in assignment (expression has type "int", variable has type "str")
Found 1 error in 1 file (checked 1 source file)

Actual Behavior

Success: no issues found in 1 source file

Your Environment

• Mypy version used: 0.971

This issue broke pytest-mypy tests which use pytester.runpytest_subprocess to run pytest in a subprocess (see realpython/pytest-mypy#139).

[dmtucker]

I bisected v0.961..v0.971 and found the failure first reproduces in 2132036, and it still reproduces on master.

[hauntsaninja]

This is a mostly unavoidable side effect of #11143, which changed mypy to have full sys.path support.
#11143 had several bugs, including to do with cwd. For instance, it meant that if you had an editable installed stub package in cwd, mypy wouldn't be able to find it. Moreover, whether or not mypy found it could depend on your invocation of mypy. That is, blanket removal of cwd, as #11143 essentially did, has been proven to not be a viable way forward.

I'm curious if it's easy for you to work around? Note that if you explicitly ask to check bad.py you should get errors from it. Also note that there are many things you could do to sys.path that would make mypy.api.run very unhappy ;-) Also note that if we were to go back to the original #11143, mucking around with sys.path means you'd get inconsistent results between the equivalent of python -m mypy ... and python -m mypy --python-executable $(which python), which is pretty undesirable behaviour.

[dmtucker]

I'm curious if it's easy for you to work around? Note that if you explicitly ask to check bad.py you should get errors from it.

I will probably just xfail the test on any affected mypy versions. The test checks that the plugin behaves correctly when a type check fails in an imported module that wasn't collected by pytest, so checking bad.py directly wouldn't really make sense.

Also note that there are many things you could do to sys.path that would make mypy.api.run very unhappy ;-)

That's fair. I'm not sure why cwd ends up at the front of sys.path twice in runpytest_subprocess calls. 🤔

you'd get inconsistent results between the equivalent of python -m mypy ... and python -m mypy --python-executable $(which python)

Interesting, yes indeed. pytest-mypy only relies on mypy.api.run FWIW.