PERF: Remove use of generators in Cython code

[mdboom]

Is this a duplicate?

• I confirmed there appear to be no duplicate issues for this request and that I agree to the Code of Conduct

Area

General cuda-python

Is your feature request related to a problem? Please describe.

Generators create very inefficient code in Cython, relative to a simple C for loop, and the overhead is especially bad when the sequences are small.

Describe the solution you'd like

We should remove instances of generators where ever possible.

Describe alternatives you've considered

No response

Additional context

No response

[kkraus14]

Generators create very inefficient code in Cython, relative to a simple C for loop, and the overhead is especially bad when the sequences are small.

Do you have any sense as to how hard it would be to improve this upstream? We should be able to adopt new Cython versions relatively easily.

[mdboom]

I'm not sure. In our specific case, Cython is creating the most general code possible here that could iterate over anything, but we "know" from the type annotations that it's always a list or a tuple (which should be much easier to iterate over). I don't know if the Cython compiler has enough context already to special case for that and make the transformation. I'll poke around and see...

[mdboom]

Some specific examples that create generators now:

if not all(isinstance(_x, (CUlibraryOption)) for _x in libraryOptions):
    raise TypeError("Argument 'libraryOptions' is not instance of type")

is better as:

for _x in libraryOptions:
    if not isinstance(_x, (CUlibraryOption)):
        raise TypeError("Argument 'libraryOptions' is not instance of type")

---

During generated class construction every class has something like:

_dict_CUipcMem_flags = dict(((int(v), v) for k, v in CUipcMem_flags.__members__.items()))

Since the class is code-generated by cython-gen anyway, this would be much better as:

_dict_CUipcMem_flags = { 
    int(CU_IPC_MEM_LAZY_ENABLE_PEER_ACCESS): CU_IPC_MEM_LAZY_ENABLE_PEER_ACCESS 
}

because it can take advantage of knowing the fixed size of the class etc. (But these changes would only affect import times, not run times).

Additionally, if it matters, these could probably benefit from a mapping where the keys are always integers, rather than a general Python dictionary.

[leofang]

Additionally, if it matters, these could probably benefit from a mapping where the keys are always integers, rather than a general Python dictionary.

Does Cython already support optimizing dict access with known kv-types? I am not aware of this capability, if so it'd be great! Or maybe you have in mind c++ dicts?

Another place where we could enjoy optimization is to rely on the fact that these error-code dicts are read-only after construction, but again I am not aware of Cython being able to optimize for this use case.

[mdboom]

Does Cython already support optimizing dict access with known kv-types? I am not aware of this capability, if so it'd be great! Or maybe you have in mind c++ dicts?

No, I don't think it does -- I was suggesting we would pull from somewhere else (probably the C++ stdlib).

Another place where we could enjoy optimization is to rely on the fact that these error-code dicts are read-only after construction, but again I am not aware of Cython being able to optimize for this use case.

Yeah, that's a good point. I /think/ the keys are even compile-time constants, so you could use something like a perfect hash library (though that's probably overkill).

[cpcloud]

I'm happy to take this on as a good first issue. @mdboom Are you actively working on this at the moment?

[mdboom]

I'm happy to take this on as a good first issue. @mdboom Are you actively working on this at the moment?

No, go for it!

[cpcloud]

It seems that generator expressions are not performant, but comparing the codegen for a list comprehension versus an append-in-for-loop approach I couldn't see any meaningful difference in the generated code.