Separate backend from hybridize and refactor optimize_for kwargs

[Kh4L]

Description

This PR separates the partitioning backend from hybridize.
hybridize now clears any previous optimization, sets the CachedOp args and activate the hybridization.
optimize_for is responsible of setting the backend, backend options and running the partitioning with backend.

If the user wish to use any partitioning backend, they have use optimize_for.

It also changes the default value of optimize_for's clear arg, making the chaining of the backend the default behavior.

The PR also make the CachedOp kwargs explicit and documented : static_alloc, static_shape, inline_limit, forward_bulk_size, backward_bulk_size.

Examples

How optimize_for changed

Before

blk.optimize_for(x, backend="someBackend", backend_opts={'dedup_subgraph':True})

After

blk.optimize_for(x, backend="someBackend", dedup_subgraph=True)

How hybridize changed

Before

blk.hybridize(backend="someBackend", static_alloc=True)
blk(x)

After

Hybridize can't be used to set the backend anymore, we now have to use optimize_for, which will call hybridize internally.

blk.optimize_for(x, backend="someBackend", static_alloc=True)

How chaining backends changed

Before

blk.optimize_for(x, backend="firstBackend", static_alloc=True)
blk.optimize_for(x, backend="secondBackend", clear=False, dedup_subgraph=True)

After

clear default value is False, we simply chain them

blk.optimize_for(x, backend="firstBackend", static_alloc=True)
blk.optimize_for(x, backend="secondBackend", dedup_subgraph=True)

cc @samskalicky who helped to design the API offline and reviewed the 1.x related PR #19386
@mseth10 who helped by reviewing the 1.x PR

[mxnet-bot]

Hey @Kh4L , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

• To trigger all jobs: @mxnet-bot run ci [all]
• To trigger specific jobs: @mxnet-bot run ci [job1, job2]

---

CI supported jobs: [website, windows-gpu, unix-cpu, clang, windows-cpu, sanity, centos-cpu, unix-gpu, edge, centos-gpu, miscellaneous]

---

Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

[samskalicky]

LGTM

[Kh4L]

@samskalicky I updated the documentation to reflect the changes

[mseth10]

Thanks for the API cleanup @Kh4L

[waytrue17]

Thanks for the great work!

[Kh4L]

@mxnet-bot run ci [unix-cpu]

[mxnet-bot]

Jenkins CI successfully triggered : [unix-cpu]