Enable minimization of net e2e runtime

[marun]

• Stop running services tests that don't require multinode (and are therefore running as part of the core extended tests)
• Skip testing ovs and subnet plugin if NETWORKING_E2E_MINIMAL=1
• add test/extended/networking-minimal.sh wrapper to allow ci to run the minimal tests without requiring changes to vagrant-openshfit

cc: @openshift/networking @stevekuznetsov

[marun]

[test]

[marun]

I've set --env NETWORKING_E2E_MINIMAL=1 for test_pull_requests_origin_networking to see how much runtime is reduced by. Looking for feedback from the networking team that it's acceptable to only run the ovs tests post-merge or by explicitly invoking the extended job.

[stevekuznetsov]

Bash looks OK.

[openshift-bot]

Evaluated for origin test up to aa41cd8

[danwinship]

Looking for feedback from the networking team that it's acceptable to only run the ovs tests post-merge or by explicitly invoking the extended job.

What's the reasoning? "Changes to other peoples' code isn't likely to break this so we don't need to run it all the time"? That's probable true. If watches in general were broken then surely some other test would catch it, and if watches are working then the OVS test should really only fail if we broke something in pkg/sdn/plugin/.

(So actually, what would be awesome would be if we could make it run the full networking tests automatically if the patch contains any changes in pkg/sdn/, and run the short tests otherwise...)

[liggitt]

or code in the startup config->init->run paths that sets up sdn stuff

[marun]

@danwinship Yes, my reasoning is that testing stuff unlikely to break in response to non-networking changes shouldn't have to run on all PRs, only networking PRs. Plus, the ovs tests are really low level and should only break if connectivity tests (which will continue to run) are also failing.

As to dynamic invocation of jobs, maybe @stevekuznetsov could comment on that but afaik it's not an option today and we'll have to manually trigger.

[danwinship]

Plus, the ovs tests are really low level and should only break if connectivity tests (which will continue to run) are also failing.

I added them primarily to test when happens when you delete things, because that doesn't get tested by any other test (and has at least once broken catastrophically). But still, it's only likely to get broken by bugs in the plugin itself, not bugs elsewhere.

[openshift-bot]

continuous-integration/openshift-jenkins/test SUCCESS (https://ci.openshift.redhat.com/jenkins/job/test_pr_origin/10017/) (Base Commit: 6b0457c)

[danwinship]

[merge]

[openshift-bot]

continuous-integration/openshift-jenkins/merge SUCCESS (https://ci.openshift.redhat.com/jenkins/job/test_pr_origin/10017/) (Image: devenv-rhel7_5179)

[openshift-bot]

Evaluated for origin merge up to aa41cd8