Argument 2 to pop of MutableMapping has incompatible type

[NiklasRosenstein]

Bug Report

Using MutableMapping.pop(key, default) inconsistently results in an error for the type of the default argument.

To Reproduce

import typing as t
T = t.TypeVar('T')
V = t.TypeVar('V')
def test(key: T, value: V):
  values: t.Dict[T, V] = {}
  print(values.pop(key, None))  # ok
  values.pop(key, value)        # ok
  values.pop(key, None)         # error: Argument 2 to "pop" of "MutableMapping" has incompatible type "None"; expected "V"
  values.pop(key, 42)           # error: Argument 2 to "pop" of "MutableMapping" has incompatible type "int"; expected "V"

Expected Behavior

I expect no errors when checking the above code with Mypy. The definition of MutableMapping.pop() allows using a different type for the default argument value (in which case a union of the map's value type and the default argument's type is returned).

class MutableMapping(Mapping[_KT, _VT], Generic[_KT, _VT]):
    # ....
    @overload
    def pop(self, key: _KT, default: Union[_VT, _T] = ...) -> Union[_VT, _T]: ...

What is curious is that the line with print() does not result in an error, even though the .pop() call is the same as two lines below.

Your Environment

• Mypy version used: 0.782, 0.812
• Mypy command-line flags: N/a
• Mypy configuration options from mypy.ini (and other config files): N/a
• Python version used: CPython 3.7.3
• Operating system and version: WSL Debian 10

[JukkaL]

This only seems to be an issue if the value type is a type variable. There's probably some confusion between type variables in pop and values.

[AlexWaygood]

On 0.942 (and master) you now get three false-positive errors instead of just two:

import typing as t
T = t.TypeVar('T')
V = t.TypeVar('V')
def test(key: T, value: V):
  values: t.Dict[T, V] = {}
  print(values.pop(key, None))  # error: Argument 2 to "pop" of "MutableMapping" has incompatible type "None"; expected "V"
  values.pop(key, value)        # ok
  values.pop(key, None)         # error: Argument 2 to "pop" of "MutableMapping" has incompatible type "None"; expected "V"
  values.pop(key, 42)           # error: Argument 2 to "pop" of "MutableMapping" has incompatible type "int"; expected "V"

[finite-state-machine]

This issue may be fixed in mypy 1.6.0.

mypy-play.net (@NiklasRosenstein's code)
mypy-play.net (shows this affect(s/ed) dicts as well)

[edited: fixed links]

[finite-state-machine]

Still seems fixed in 1.8.0. I suggest closing this issue.

[bhperry]

This does not appear to be fixed for dicts in 1.8.0 (also tried 1.9.0 and 1.10.0), although I'm having trouble figuring out exactly what conditions lead to it happening. Seems fine in simple examples, but in more complex code (some of which uses typevars, other parts do not but still trigger this) I get this

error: Argument 2 to "pop" of "dict" has incompatible type "None"; expected "str"  [arg-type]

[alicederyn]

I can reproduce the old error like this:

import sys

values_by_name = {"one": 1, "two": 2}
for value in values_by_name.values():
  assert value > 0
if value := values_by_name.pop(sys.argv[1], None):
  print(value)
else:
  print("Not found")

example.py:6: error: Argument 2 to "pop" of "dict" has incompatible type "None"; expected "int"  [arg-type]

Significantly, this only happens if mypy has already decided that the LHS of the assignment is of type int, which makes me believe this is just a bug in the error message that's being emitted — something to do with overload resolution? I would expect to see:

example.py:6: error: Incompatible types in assignment (expression has type "int | None", variable has type "int")  [assignment]

[alicederyn]

@bhperry Does this possibly explain what you were seeing?

[bhperry]

@alicederyn Great catch! Yes I think that is exactly what was happening. Didn't think about the LHS because it was a var used in a previous loop as T and then later assigned T | None, always forget that python scoping is a bit loose

[alicederyn]

@finite-state-machine should this be split off to a separate issue, or leave this one open to cover it?

[brianschubert]

Closing since this issue no longer reproduces as of v1.6.0 (specifically after the typeshed sync #15792).

@bhperry @alicederyn Feel free to open a new issue. At a glance, it looks like this falls in the bucket of cases where bidirectional inference can have bad results when there's a union of type variables in the return type. You'll find some related issues under the topic-type-context Type context / bidirectional inference label.