cannot import name 'Mapping' from 'collections'

[paulaceccon]

I'm getting this error when calling:

from bertviz.transformers_neuron_view import BertModel, BertTokenizer

With Python 3.11. The Mapping location has been updated but this is not reflected here.

[jeffnyman]

Just as a note, this happens with Python 3.10 as well. From what I can see, this has to do with urllib3 and requests, which seem to contain the offending code. (Meaning, the imports should be from collections.abc and not collections.) Everything does work on Python 3.9. Making sure all dependencies are up-to-date in the BertViz chain should take care of this.

[jessevig]

Hi @jeffnyman and @paulaceccon, thanks for the issue and apologies for the late reply. I can't seem to replicate the error with python 3.10 or 3.11. How did you install BertViz? Thanks!

[jeffnyman]

How did you install BertViz?

In my case, I'm using Poetry, so I did:

poetry add bertviz

It definitely installs without a problem and I can use it. But when using the BertModel, it looks like requests and urllib3 were referenced specifically as using the older import for collections. So I wasn't sure if this was related to the versions that BertViz was using.

I did try adding both directly:

poetry add urllib3
poetry add requests

But that still seemed to show the issue.

I'll create an entirely new environment and see if I can replicate. I'll include the stack trace as well.

[jeffnyman]

@jessevig I can definitely recreate on a fully new environment, on either Windows or Mac. Here is the script I'm running on Python 3.10.11 and Python 3.11.4:

from transformers import AutoTokenizer
from bertviz.transformers_neuron_view import BertModel
from bertviz.neuron_view import show

model_checkpoint = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
model = BertModel.from_pretrained(model_checkpoint)

text = "a crane flew over the serene lake"

show(model, "bert", tokenizer, text, display_mode="light", layer=0, head=8)

Here is the stack trace:

Traceback (most recent call last):
  File "E:\learning.Repository\ai-blog\zample.py", line 2, in <module>
    from bertviz.transformers_neuron_view import BertModel
  File "E:\learning.Repository\ai-blog\.venv\lib\site-packages\bertviz\transformers_neuron_view\__init__.py", line 2, in <module>
    from .tokenization_bert import BertTokenizer, BasicTokenizer, WordpieceTokenizer
  File "E:\learning.Repository\ai-blog\.venv\lib\site-packages\bertviz\transformers_neuron_view\tokenization_bert.py", line 25, in <module>
    from .tokenization_utils import PreTrainedTokenizer
  File "E:\learning.Repository\ai-blog\.venv\lib\site-packages\bertviz\transformers_neuron_view\tokenization_utils.py", line 25, in <module>
    from .file_utils import cached_path
  File "E:\learning.Repository\ai-blog\.venv\lib\site-packages\bertviz\transformers_neuron_view\file_utils.py", line 19, in <module>
    import boto3
  File "E:\learning.Repository\ai-blog\.venv\lib\site-packages\boto3\__init__.py", line 16, in <module>
    from boto3.session import Session
  File "E:\learning.Repository\ai-blog\.venv\lib\site-packages\boto3\session.py", line 17, in <module>
    import botocore.session
  File "E:\learning.Repository\ai-blog\.venv\lib\site-packages\botocore\session.py", line 26, in <module>
    import botocore.configloader
  File "E:\learning.Repository\ai-blog\.venv\lib\site-packages\botocore\configloader.py", line 19, in <module>
    from botocore.compat import six
  File "E:\learning.Repository\ai-blog\.venv\lib\site-packages\botocore\compat.py", line 25, in <module>
    from botocore.exceptions import MD5UnavailableError
  File "E:\learning.Repository\ai-blog\.venv\lib\site-packages\botocore\exceptions.py", line 15, in <module>
    from botocore.vendored.requests.exceptions import ConnectionError
  File "E:\learning.Repository\ai-blog\.venv\lib\site-packages\botocore\vendored\requests\__init__.py", line 58, in <module>
    from . import utils
  File "E:\learning.Repository\ai-blog\.venv\lib\site-packages\botocore\vendored\requests\utils.py", line 26, in <module>
    from .compat import parse_http_list as _parse_list_header
  File "E:\learning.Repository\ai-blog\.venv\lib\site-packages\botocore\vendored\requests\compat.py", line 7, in <module>
    from .packages import chardet
  File "E:\learning.Repository\ai-blog\.venv\lib\site-packages\botocore\vendored\requests\packages\__init__.py", line 3, in <module>
    from . import urllib3
  File "E:\learning.Repository\ai-blog\.venv\lib\site-packages\botocore\vendored\requests\packages\urllib3\__init__.py", line 10, in <module>
    from .connectionpool import (
  File "E:\learning.Repository\ai-blog\.venv\lib\site-packages\botocore\vendored\requests\packages\urllib3\connectionpool.py", line 38, in <module>
    from .response import HTTPResponse
  File "E:\learning.Repository\ai-blog\.venv\lib\site-packages\botocore\vendored\requests\packages\urllib3\response.py", line 9, in <module>
    from ._collections import HTTPHeaderDict
  File "E:\learning.Repository\ai-blog\.venv\lib\site-packages\botocore\vendored\requests\packages\urllib3\_collections.py", line 1, in <module>
    from collections import Mapping, MutableMapping
ImportError: cannot import name 'Mapping' from 'collections' (E:\appDev\pyenv\.pyenv\pyenv-win\versions\3.10.11\lib\collections\__init__.py)

In looking at this, I now see it looks to be vendored versions of the dependencies that botocore is using. This was discussed in 1865 apparently and was said to be handled by 1922. If I'm reading all this correctly.

If you drop down to Python 3.9, everything is fine.

[jessevig]

Thanks so much for the detailed analysis @jeffnyman! I will take a look at this.

[MrSampson]

I manually added botocore to be the newest version (1.34.6) and it fixed the problem. The problem is in that botocore brings its own dependencies with it in the "vendored" subdirectory.