Multi-threaded inferencing support now lacking a unittest?

[DickJC123]

While we generally insist that a new feature come with unittests at time of introduction, it's also important to ensure those tests are not removed or marked 'skipped' for too long to protect the feature from falling into disrepair.

Back in February, @anirudh2290's PR #16654 introduced multi-threaded inferencing, with new files ./src/imperative/cached_op_threadsafe.{h,cc} and a unittest under ./tests/cpp/thread_safety/. However in June, @eric-haibin-lin's PR #18598 "graph executor c api removal" removed the test files, but left the cached_op_threadsafe.{h,cc} files. Is there some other unittest of multi-threaded inferencing, and with which front end? If not, can the old test be reinstated or replaced?

[DickJC123]

@eric-haibin-lin I work with a version of MXNet that has differences I have yet to PR, e.g. in the files imperative/cached_op.{h,cc}. When I went to merge PR #16654, there were merge conflicts, given the sizable refactoring, that I attempted to correct. However, since the threadsafe unittests were removed, I have no way of testing if my resolved merge preserved the functionality. Do you have a suggestion? Also @anirudh2290?

[szha]

Looks like the removal was mainly due to the dependency on mxnet cpp package in the test. rewriting the test to use c api should be the way to go.

[barry-jin]

Multi_threaded_inference test has been added back in #20131 along with the whole cpp-pakage and will be tested in ci/unix-gpu