UnicodeDecodeError: 'ascii' codec can't decode byte

[mschwager]

$ sudo pip install pygit2
[sudo] password for matt: 
Downloading/unpacking pygit2
  Downloading pygit2-0.20.3.tar.gz (146kB): 146kB downloaded
  Running setup.py (path:/tmp/pip_build_root/pygit2/setup.py) egg_info for package pygit2

Installing collected packages: pygit2
  Running setup.py install for pygit2
    building '_pygit2' extension
    x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include -Iinclude -I/usr/include/python2.7 -c src/note.c -o build/temp.linux-x86_64-2.7/src/note.o
    In file included from src/utils.h:34:0,
                     from src/note.c:32:
    src/types.h:83:5: error: unknown type name ‘git_config_iterator’
         git_config_iterator *iter;
         ^
    In file included from /usr/include/python2.7/Python.h:80:0,
                     from src/note.c:29:
    /usr/include/python2.7/object.h:81:12: error: unknown type name ‘git_diff’
         struct _typeobject *ob_type;
                ^
    src/types.h:57:13: note: in expansion of macro ‘PyObject_HEAD’
                 PyObject_HEAD\
                 ^
    src/types.h:103:1: note: in expansion of macro ‘SIMPLE_TYPE’
     SIMPLE_TYPE(Diff, git_diff, list)
     ^
    /usr/include/python2.7/object.h:81:12: error: unknown type name ‘git_blame’
         struct _typeobject *ob_type;
                ^
    src/types.h:57:13: note: in expansion of macro ‘PyObject_HEAD’
                 PyObject_HEAD\
                 ^
    src/types.h:230:1: note: in expansion of macro ‘SIMPLE_TYPE’
     SIMPLE_TYPE(Blame, git_blame, blame)
     ^
    In file included from src/utils.h:34:0,
                     from src/note.c:32:
    src/types.h:255:5: error: unknown type name ‘git_merge_result’
         git_merge_result *result;
         ^
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
    Complete output from command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/pygit2/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-lf_re3-record/install-record.txt --single-version-externally-managed --compile:
    running install

running build

running build_py

creating build

creating build/lib.linux-x86_64-2.7

creating build/lib.linux-x86_64-2.7/pygit2

copying pygit2/repository.py -> build/lib.linux-x86_64-2.7/pygit2

copying pygit2/version.py -> build/lib.linux-x86_64-2.7/pygit2

copying pygit2/credentials.py -> build/lib.linux-x86_64-2.7/pygit2

copying pygit2/settings.py -> build/lib.linux-x86_64-2.7/pygit2

copying pygit2/__init__.py -> build/lib.linux-x86_64-2.7/pygit2

running build_ext

building '_pygit2' extension

creating build/temp.linux-x86_64-2.7

creating build/temp.linux-x86_64-2.7/src

x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include -Iinclude -I/usr/include/python2.7 -c src/note.c -o build/temp.linux-x86_64-2.7/src/note.o

In file included from src/utils.h:34:0,

                 from src/note.c:32:

src/types.h:83:5: error: unknown type name ‘git_config_iterator’

     git_config_iterator *iter;

     ^

In file included from /usr/include/python2.7/Python.h:80:0,

                 from src/note.c:29:

/usr/include/python2.7/object.h:81:12: error: unknown type name ‘git_diff’

     struct _typeobject *ob_type;

            ^

src/types.h:57:13: note: in expansion of macro ‘PyObject_HEAD’

             PyObject_HEAD\

             ^

src/types.h:103:1: note: in expansion of macro ‘SIMPLE_TYPE’

 SIMPLE_TYPE(Diff, git_diff, list)

 ^

/usr/include/python2.7/object.h:81:12: error: unknown type name ‘git_blame’

     struct _typeobject *ob_type;

            ^

src/types.h:57:13: note: in expansion of macro ‘PyObject_HEAD’

             PyObject_HEAD\

             ^

src/types.h:230:1: note: in expansion of macro ‘SIMPLE_TYPE’

 SIMPLE_TYPE(Blame, git_blame, blame)

 ^

In file included from src/utils.h:34:0,

                 from src/note.c:32:

src/types.h:255:5: error: unknown type name ‘git_merge_result’

     git_merge_result *result;

     ^

error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

----------------------------------------
Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/pygit2/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-lf_re3-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/pygit2
Traceback (most recent call last):
  File "/usr/local/bin/pip", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/pip/__init__.py", line 185, in main
    return command.main(cmd_args)
  File "/usr/local/lib/python2.7/dist-packages/pip/basecommand.py", line 161, in main
    text = '\n'.join(complete_log)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 47: ordinal not in range(128)

I've recently updated to Ubuntu 14.04. I didn't have pygit2 installed before then, but that could have something to do with it.

[jdavid]

You need to install libgit2 first

[mschwager]

I have it installed:

$ sudo apt-get install libgit2-dev libgit2-0
Reading package lists... Done
Building dependency tree
Reading state information... Done
libgit2-0 is already the newest version.
libgit2-dev is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.

[jdavid]

Which version? You need 0.20

[jdavid]

Well, the above question is a bit rhetoric, I know Debian/Ubuntu does not have the latest version.

So basically you need to either install libgit2 by hand (best option) or use an older version of pygit2.

[mschwager]

Hmm, I installed libgit2 0.20 and I'm still having the same issue.

$ whereis libgit
libgit: /usr/local/lib/libgit2.so
$ ls -lh /usr/local/lib/libgit2.so
lrwxrwxrwx 1 root root 12 Apr 30 14:33 /usr/local/lib/libgit2.so -> libgit2.so.0
$ ls -lh /usr/local/lib/libgit2.so.0
lrwxrwxrwx 1 root root 17 Apr 30 14:33 /usr/local/lib/libgit2.so.0 -> libgit2.so.0.20.0

[carlosmn]

Which one is "the same" issue. Most of the output you pasted was about not finding the header. If that's still, the case, then you either don't have the right version installed, or you don't have the headers installed.

[mschwager]

The above output shows I have libgit2 0.20 installed. Further proof:

$ sudo make install
[ 23%] Built target git2
[100%] Built target libgit2_clar
Install the project...
-- Install configuration: "Debug"
-- Up-to-date: /usr/local/lib/libgit2.so.0.20.0
-- Up-to-date: /usr/local/lib/libgit2.so.0
-- Up-to-date: /usr/local/lib/libgit2.so
-- Up-to-date: /usr/local/lib/pkgconfig/libgit2.pc
-- Installing: /usr/local/include/git2
-- Up-to-date: /usr/local/include/git2/buffer.h
-- Up-to-date: /usr/local/include/git2/pack.h
-- Up-to-date: /usr/local/include/git2/clone.h
-- Up-to-date: /usr/local/include/git2/revparse.h
-- Up-to-date: /usr/local/include/git2/reflog.h
-- Installing: /usr/local/include/git2/sys
-- Up-to-date: /usr/local/include/git2/sys/reflog.h
-- Up-to-date: /usr/local/include/git2/sys/repository.h
-- Up-to-date: /usr/local/include/git2/sys/odb_backend.h
-- Up-to-date: /usr/local/include/git2/sys/refdb_backend.h
-- Up-to-date: /usr/local/include/git2/sys/commit.h
-- Up-to-date: /usr/local/include/git2/sys/mempack.h
-- Up-to-date: /usr/local/include/git2/sys/filter.h
-- Up-to-date: /usr/local/include/git2/sys/index.h
-- Up-to-date: /usr/local/include/git2/sys/config.h
-- Up-to-date: /usr/local/include/git2/sys/diff.h
-- Up-to-date: /usr/local/include/git2/sys/refs.h
-- Up-to-date: /usr/local/include/git2/strarray.h
-- Up-to-date: /usr/local/include/git2/ignore.h
-- Up-to-date: /usr/local/include/git2/version.h
-- Up-to-date: /usr/local/include/git2/cred_helpers.h
-- Up-to-date: /usr/local/include/git2/refdb.h
-- Up-to-date: /usr/local/include/git2/pathspec.h
-- Up-to-date: /usr/local/include/git2/patch.h
-- Up-to-date: /usr/local/include/git2/graph.h
-- Up-to-date: /usr/local/include/git2/trace.h
-- Up-to-date: /usr/local/include/git2/remote.h
-- Up-to-date: /usr/local/include/git2/errors.h
-- Up-to-date: /usr/local/include/git2/attr.h
-- Up-to-date: /usr/local/include/git2/push.h
-- Up-to-date: /usr/local/include/git2/checkout.h
-- Up-to-date: /usr/local/include/git2/indexer.h
-- Up-to-date: /usr/local/include/git2/repository.h
-- Up-to-date: /usr/local/include/git2/transport.h
-- Up-to-date: /usr/local/include/git2/message.h
-- Up-to-date: /usr/local/include/git2/refspec.h
-- Up-to-date: /usr/local/include/git2/tree.h
-- Up-to-date: /usr/local/include/git2/object.h
-- Up-to-date: /usr/local/include/git2/blame.h
-- Up-to-date: /usr/local/include/git2/odb_backend.h
-- Up-to-date: /usr/local/include/git2/revwalk.h
-- Up-to-date: /usr/local/include/git2/blob.h
-- Up-to-date: /usr/local/include/git2/signature.h
-- Up-to-date: /usr/local/include/git2/types.h
-- Up-to-date: /usr/local/include/git2/merge.h
-- Up-to-date: /usr/local/include/git2/commit.h
-- Up-to-date: /usr/local/include/git2/inttypes.h
-- Up-to-date: /usr/local/include/git2/status.h
-- Up-to-date: /usr/local/include/git2/branch.h
-- Up-to-date: /usr/local/include/git2/cherrypick.h
-- Up-to-date: /usr/local/include/git2/notes.h
-- Up-to-date: /usr/local/include/git2/stash.h
-- Up-to-date: /usr/local/include/git2/stdint.h
-- Up-to-date: /usr/local/include/git2/reset.h
-- Up-to-date: /usr/local/include/git2/filter.h
-- Up-to-date: /usr/local/include/git2/index.h
-- Up-to-date: /usr/local/include/git2/threads.h
-- Up-to-date: /usr/local/include/git2/oid.h
-- Up-to-date: /usr/local/include/git2/common.h
-- Up-to-date: /usr/local/include/git2/config.h
-- Up-to-date: /usr/local/include/git2/submodule.h
-- Up-to-date: /usr/local/include/git2/odb.h
-- Up-to-date: /usr/local/include/git2/diff.h
-- Up-to-date: /usr/local/include/git2/net.h
-- Up-to-date: /usr/local/include/git2/refs.h
-- Up-to-date: /usr/local/include/git2/revert.h
-- Up-to-date: /usr/local/include/git2/tag.h
-- Up-to-date: /usr/local/include/git2.h

Yet I'm still getting:

$ sudo pip install pygit2
...
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/pygit2/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-CKAgyc-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/pygit2
Traceback (most recent call last):
  File "/usr/local/bin/pip", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/pip/__init__.py", line 185, in main
    return command.main(cmd_args)
  File "/usr/local/lib/python2.7/dist-packages/pip/basecommand.py", line 161, in main
    text = '\n'.join(complete_log)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 28: ordinal not in range(128)

[carlosmn]

Assuming you've elided the same error messages from the compiler, it still means it hasn't found the right headers. Do you still have the old package installed somewhere that comes before /usr/local/include in the compiler's search path?

[mschwager]

It looks like it's finding the right headers. One of the errors I'm receiving:

/usr/local/include/git2/signature.h:75:17: note: expected ‘struct git_signature **’ but argument is of type ‘struct git_signature * const’

 GIT_EXTERN(int) git_signature_dup(git_signature **dest, const git_signature *sig);

                 ^

src/blame.c:79:13: error: too few arguments to function ‘git_signature_dup’

             git_signature_dup(hunk->orig_signature) : NULL;

             ^

Which is using /usr/local/include/git2/signature.h and we can see from the installation output above that signature.h get's installed there: -- Up-to-date: /usr/local/include/git2/signature.h

[mschwager]

Here's the full error (it appears as though it's changed since installing libgit2 0.20 instead of the version in apt):

$ sudo pip install pygit2
Downloading/unpacking pygit2
  Downloading pygit2-0.20.3.tar.gz (146kB): 146kB downloaded
  Running setup.py (path:/tmp/pip_build_root/pygit2/setup.py) egg_info for package pygit2

Installing collected packages: pygit2
  Running setup.py install for pygit2
    building '_pygit2' extension
    x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include -Iinclude -I/usr/include/python2.7 -c src/note.c -o build/temp.linux-x86_64-2.7/src/note.o
    src/note.c: In function ‘Note_oid__get__’:
    src/note.c:75:5: warning: implicit declaration of function ‘git_note_oid’ [-Wimplicit-function-declaration]
         return git_oid_to_python(git_note_oid(self->note));
         ^
    src/note.c:75:5: warning: passing argument 1 of ‘git_oid_to_python’ makes pointer from integer without a cast [enabled by default]
    In file included from src/note.c:34:0:
    src/oid.h:38:11: note: expected ‘const struct git_oid *’ but argument is of type ‘int’
     PyObject* git_oid_to_python(const git_oid *oid);
               ^
    x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include -Iinclude -I/usr/include/python2.7 -c src/blob.c -o build/temp.linux-x86_64-2.7/src/blob.o
    x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include -Iinclude -I/usr/include/python2.7 -c src/commit.c -o build/temp.linux-x86_64-2.7/src/commit.o
    x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include -Iinclude -I/usr/include/python2.7 -c src/blame.c -o build/temp.linux-x86_64-2.7/src/blame.o
    src/blame.c: In function ‘wrap_blame_hunk’:
    src/blame.c:73:13: warning: passing argument 1 of ‘git_signature_dup’ from incompatible pointer type [enabled by default]
                 git_signature_dup(hunk->final_signature) : NULL;
                 ^
    In file included from /usr/local/include/git2.h:50:0,
                     from src/error.h:33,
                     from src/blame.c:31:
    /usr/local/include/git2/signature.h:75:17: note: expected ‘struct git_signature **’ but argument is of type ‘struct git_signature * const’
     GIT_EXTERN(int) git_signature_dup(git_signature **dest, const git_signature *sig);
                     ^
    src/blame.c:73:13: error: too few arguments to function ‘git_signature_dup’
                 git_signature_dup(hunk->final_signature) : NULL;
                 ^
    In file included from /usr/local/include/git2.h:50:0,
                     from src/error.h:33,
                     from src/blame.c:31:
    /usr/local/include/git2/signature.h:75:17: note: declared here
     GIT_EXTERN(int) git_signature_dup(git_signature **dest, const git_signature *sig);
                     ^
    src/blame.c:79:13: warning: passing argument 1 of ‘git_signature_dup’ from incompatible pointer type [enabled by default]
                 git_signature_dup(hunk->orig_signature) : NULL;
                 ^
    In file included from /usr/local/include/git2.h:50:0,
                     from src/error.h:33,
                     from src/blame.c:31:
    /usr/local/include/git2/signature.h:75:17: note: expected ‘struct git_signature **’ but argument is of type ‘struct git_signature * const’
     GIT_EXTERN(int) git_signature_dup(git_signature **dest, const git_signature *sig);
                     ^
    src/blame.c:79:13: error: too few arguments to function ‘git_signature_dup’
                 git_signature_dup(hunk->orig_signature) : NULL;
                 ^
    In file included from /usr/local/include/git2.h:50:0,
                     from src/error.h:33,
                     from src/blame.c:31:
    /usr/local/include/git2/signature.h:75:17: note: declared here
     GIT_EXTERN(int) git_signature_dup(git_signature **dest, const git_signature *sig);
                     ^
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
    Complete output from command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/pygit2/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-xs8LeP-record/install-record.txt --single-version-externally-managed --compile:
    running install

running build

running build_py

creating build

creating build/lib.linux-x86_64-2.7

creating build/lib.linux-x86_64-2.7/pygit2

copying pygit2/repository.py -> build/lib.linux-x86_64-2.7/pygit2

copying pygit2/version.py -> build/lib.linux-x86_64-2.7/pygit2

copying pygit2/credentials.py -> build/lib.linux-x86_64-2.7/pygit2

copying pygit2/settings.py -> build/lib.linux-x86_64-2.7/pygit2

copying pygit2/__init__.py -> build/lib.linux-x86_64-2.7/pygit2

running build_ext

building '_pygit2' extension

creating build/temp.linux-x86_64-2.7

creating build/temp.linux-x86_64-2.7/src

x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include -Iinclude -I/usr/include/python2.7 -c src/note.c -o build/temp.linux-x86_64-2.7/src/note.o

src/note.c: In function ‘Note_oid__get__’:

src/note.c:75:5: warning: implicit declaration of function ‘git_note_oid’ [-Wimplicit-function-declaration]

     return git_oid_to_python(git_note_oid(self->note));

     ^

src/note.c:75:5: warning: passing argument 1 of ‘git_oid_to_python’ makes pointer from integer without a cast [enabled by default]

In file included from src/note.c:34:0:

src/oid.h:38:11: note: expected ‘const struct git_oid *’ but argument is of type ‘int’

 PyObject* git_oid_to_python(const git_oid *oid);

           ^

x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include -Iinclude -I/usr/include/python2.7 -c src/blob.c -o build/temp.linux-x86_64-2.7/src/blob.o

x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include -Iinclude -I/usr/include/python2.7 -c src/commit.c -o build/temp.linux-x86_64-2.7/src/commit.o

x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include -Iinclude -I/usr/include/python2.7 -c src/blame.c -o build/temp.linux-x86_64-2.7/src/blame.o

src/blame.c: In function ‘wrap_blame_hunk’:

src/blame.c:73:13: warning: passing argument 1 of ‘git_signature_dup’ from incompatible pointer type [enabled by default]

             git_signature_dup(hunk->final_signature) : NULL;

             ^

In file included from /usr/local/include/git2.h:50:0,

                 from src/error.h:33,

                 from src/blame.c:31:

/usr/local/include/git2/signature.h:75:17: note: expected ‘struct git_signature **’ but argument is of type ‘struct git_signature * const’

 GIT_EXTERN(int) git_signature_dup(git_signature **dest, const git_signature *sig);

                 ^

src/blame.c:73:13: error: too few arguments to function ‘git_signature_dup’

             git_signature_dup(hunk->final_signature) : NULL;

             ^

In file included from /usr/local/include/git2.h:50:0,

                 from src/error.h:33,

                 from src/blame.c:31:

/usr/local/include/git2/signature.h:75:17: note: declared here

 GIT_EXTERN(int) git_signature_dup(git_signature **dest, const git_signature *sig);

                 ^

src/blame.c:79:13: warning: passing argument 1 of ‘git_signature_dup’ from incompatible pointer type [enabled by default]

             git_signature_dup(hunk->orig_signature) : NULL;

             ^

In file included from /usr/local/include/git2.h:50:0,

                 from src/error.h:33,

                 from src/blame.c:31:

/usr/local/include/git2/signature.h:75:17: note: expected ‘struct git_signature **’ but argument is of type ‘struct git_signature * const’

 GIT_EXTERN(int) git_signature_dup(git_signature **dest, const git_signature *sig);

                 ^

src/blame.c:79:13: error: too few arguments to function ‘git_signature_dup’

             git_signature_dup(hunk->orig_signature) : NULL;

             ^

In file included from /usr/local/include/git2.h:50:0,

                 from src/error.h:33,

                 from src/blame.c:31:

/usr/local/include/git2/signature.h:75:17: note: declared here

 GIT_EXTERN(int) git_signature_dup(git_signature **dest, const git_signature *sig);

                 ^

error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

----------------------------------------
Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/pygit2/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-xs8LeP-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/pygit2
Traceback (most recent call last):
  File "/usr/local/bin/pip", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/pip/__init__.py", line 185, in main
    return command.main(cmd_args)
  File "/usr/local/lib/python2.7/dist-packages/pip/basecommand.py", line 161, in main
    text = '\n'.join(complete_log)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 28: ordinal not in range(128)

[carlosmn]

Those errors sound like you've installed the development version of libgit2, instead of the 0.20 release.

[uniphil]

@mschwager are you using virtualenv?

If so, you could try my venvgit2 package, it's in pypi. It will install the right versions of libgit2 and pygit2 into your virtualenv for you with just pip install venvgit2. It's super-hack but it works, at the very least for you maybe as a sanity check to see if a known-good combination if lib/py git will install.

[mschwager]

You were correct Carlos! Thanks guys! :)

If people run into the problem in the future, the solution was:

git checkout v0.20.0
sudo make install
sudo pip install pygit2