[C++] Document more of the compute layer

[asfimport]

Ideally, we should add:

• a description and examples of how to call compute functions
• an API reference for concrete C++ functions such as Cast, NthToIndices, etc.

Reporter: Antoine Pitrou / @pitrou

Related issues:

• [C++] Document available functions in compute::FunctionRegistry (depends upon)

_{Note: This issue was originally created as ARROW-9392. Please see the migration documentation for further details.}

[asfimport]

Aldrin Montana / @drin:
Hello  @pitrou! I am interested in helping out with this, at least with the portions that I will be using significantly in the near future. I figured pinging you first to orient myself made sense since you created this issue.

 

I'm not sure how much there is to do here, but I have just had trouble finding documentation myself and wanted to volunteer to contribute. (I posted to the mailing list in case a lot of this already exists: https://lists.apache.org/thread.html/rb0633480a9cf07d311d3a1143c2be1bce3a83e6ae5cf281ebb2cff9b%40%3Cdev.arrow.apache.org%3E)

 

For reference, my usage of the APIs will be related to ARROW-10549, but with a different end goal.

 

Thanks!

[asfimport]

Antoine Pitrou / @pitrou:
Hi @drin ! Your offer to help is very much welcome.

The current documentation, as you probably figured out, is here: https://arrow.apache.org/docs/cpp/compute.html

It is mostly a listing of available compute functions and their semantics. It is probably missing a higher-level description of the compute layer and its design.

It would be useful if you could tell what is missing for your understanding, and you can then start working on that (or you can help us achieve the desired level of understanding).

[asfimport]

Aldrin Montana / @drin:
Thanks! I actually have not seen this particular page (not sure if I missed it, or if it came in a recent-ish update), so I think there is a lot to catch up on here.

 

One thing that I do find confusing, is that my research group tries to use the dataset API and expressions but I don't see any mention of these in the documentation. My impression is that expressions (or maybe the compute API?) came from gandiva, and perhaps the dataset and expression APIs are being coalesced into the compute API?

 

Anyways, I will look over the documentation you linked and figure out what to do next. Thanks again!

[asfimport]

Antoine Pitrou / @pitrou:
Good point that the dataset API is undocumented. I've open ARROW-11677 for it.

[asfimport]

Eduardo Ponce / @edponce:
Hi @drin,

I am currently working on a tutorial for compute functions and will revisit the description of the compute layer, and think it might solve part of this issue.
I want to check with you if you are working on documentation or API for the compute layer. I did not find a pull request for this issue.

[asfimport]

Aldrin Montana / @drin:
Hello! I was planning on it but I have not done anything yet. Let me know if there's anything I can do otherwise to help :)

[asfimport]

Aldrin Montana / @drin:
I am checking back in here since I'm picking up ARROW-8991.

Taking a look around, I believe that this has been mostly addressed. Since this was created there is now a "Compute Functions" section in the C++ API Reference, in addition to the "Compute Functions" section of the C++ User Guide that Antoine initially mentioned. Also, ARROW-11677 has been completed. At the very least, the points mentioned in the description seem to be mostly addressed.

 

@edponce , I think you can close this.

[asfimport]

Antoine Pitrou / @pitrou:
Ok, I'm gonna close for now. Thanks for the heads-up @drin