Add `cuda.core.NDBuffer`

[leofang]

Moving the tracking issue to the public. Unlike the existing Buffer which is a 1D contiguous, untyped blob, we want to have a typed buffer that feels more natural to use as underlying memory of tensors/arrays. The requirements include

• Arbitrary dimension
• Holds the underlying pointer
• Holds shape
• Holds strides
• Exposes same attributes as Buffer does

The only bit that needs discussion during design meetings: Is it owning or non-owning?

• I think it should be owning (I was wrong in one of the meetings today) because Buffer is owning too. That'd make this NDBuffer look more like mdarray ([EPIC] Multi-dimensional, heterogeneous data container  cccl#2474; see also Raft's counterpart and p1684).
• If it's non-owning, then it's a view but we already have StridedMemoryView (Lowering StridedMemoryView attribues to typed efficient C/C++/Cython accessible values #180)

[leofang]

Capturing what Satya noted in an offline meeting:

I think NDBuffer is useful for two reasons:

• While it's possible for users to compute the offset themselves, it's better to do that as part of the type so that each user is not burdened with it.
• We can offer efficient copy functions across layouts that users can take advantage of. While it's straightforward to write a copy function, it's hard to write a performant copy function.

[kkraus14]

• I think it should be owning (I was wrong in one of the meetings today) because Buffer is owning too. That'd make this NDBuffer look more like mdarray ([FEA]: Provide mdarray cccl#2474; see also Raft's counterpart and p1684).

The name NDBuffer is very confusing to me as I think of a Buffer as an untyped region of memory. I think using ndarray (more Python than mdarray?) is a more natural name, though we don't want to give the impression we will have all of the functionality that Python users expect of an array and we are purely a data container.

As far as ownership semantics go, we could take the NumPy like approach, where instead of StridedMemoryView that is purely a non-owning class, we could have a single class that can act as both an owning type (where it allocates a Buffer underneath itself on construction as needed) or a view type (where it holds a reference to a Python object that supports __dlpack__ protocol or similar).

[cpcloud]

It seems like we've already gone down the path of the dual-ownership numpy-like model, since StridedMemoryView has an exporting_obj member that is the owning object of the data backing the view.

[leofang]

Yes. This issue could need a bit of update. The tl;dr is: Instead of NDBuffer, we just add StridedLayout. Together with the existing 1D contiguous, untyped Buffer, it can represent what we need with NDBuffer. @stiepan has done this part in #1283 and is working on migrating the ND-copy routine from nvmath-python.

[leofang]

Copy-pasting @kkraus14's offline summary:

• We don't want this to ultimately live in cuda.core, but want it to live in CCCL. We want the implementation in CCCL C++ but with the typical exposure up through CCCL Python that is JITed.
• We will close the PR at the top of the thread and revert the StridedLayout changes made to SMV prior to the 0.5.0 release of cuda.core.
• nvmath-python will continue to rely on their internal implementation until we have a new implementation usable in CCCL-Python.