filter hosts to query on zone wide storage by DaanHoogland · Pull Request #3733 · apache/cloudstack

DaanHoogland · 2019-12-02T15:25:52Z

Description

skip hosts to query for volume stats based on their hypervisor type.

Fixes: #3596
Fixes: #3093

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

Screenshots (if appropriate):

How Has This Been Tested?

As this command is send to the hypervisor by a worker thread, the only thing that needs to happen is for it to be installed and the stacktrace that was in the agent.log on the kvm hosts, to not appear anymore.

before

on the agent:

2019-12-02 14:47:52,059 WARN  [cloud.agent.Agent] (agentRequest-Handler-2:null) (logid:0221a2bf) Caught:
com.cloud.utils.exception.CloudRuntimeException: Could not fetch storage pool 16758e2d-e971-3f1e-be90-f1409ad449da from libvirt
        at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.getStoragePool(KVMStoragePoolManager.java:256)
        at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.getStoragePool(KVMStoragePoolManager.java:242)
        at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVolumeStatsCommandWrapper.getVolumeStat(LibvirtGetVolumeStatsCommandWrapper.java:62)
        at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVolumeStatsCommandWrapper.execute(LibvirtGetVolumeStatsCommandWrapper.java:52)
        at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVolumeStatsCommandWrapper.execute(LibvirtGetVolumeStatsCommandWrapper.java:40)
        at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
        at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1476)
        at com.cloud.agent.Agent.processRequest(Agent.java:640)
        at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1053)
        at com.cloud.utils.nio.Task.call(Task.java:83)
        at com.cloud.utils.nio.Task.call(Task.java:29)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

in the server logs:

Details from executing class com.cloud.agent.api.GetVolumeStatsCommand: com.c
loud.utils.exception.CloudRuntimeException: Could not fetch storage pool 16758e2d-e971-3f1e-be90-f1409ad449da from libvirt
        at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.getStoragePool(KVMStoragePoolManager.java:256)
        at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.getStoragePool(KVMStoragePoolManager.java:242)
        at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVolumeStatsCommandWrapper.getVolumeStat(LibvirtGetVolumeStatsCommandWrapper.java:62)
        at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVolumeStatsCommandWrapper.execute(LibvirtGetVolumeStatsCommandWrapper.java:52)
        at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVolumeStatsCommandWrapper.execute(LibvirtGetVolumeStatsCommandWrapper.java:40)
        at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
        at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1476)
        at com.cloud.agent.Agent.processRequest(Agent.java:640)
        at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1053)
        at com.cloud.utils.nio.Task.call(Task.java:83)
        at com.cloud.utils.nio.Task.call(Task.java:29)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

after applying patch (on server only):

2019-12-02 16:31:59,116 DEBUG [c.c.a.m.AgentManagerImpl] (StatsCollector-5:ctx-3794ee61) (logid:577754f7) Details from executing class com.cloud.agent.api.GetVolumeStatsCommand:

DaanHoogland · 2019-12-02T15:36:46Z

@blueorangutan package

blueorangutan · 2019-12-02T15:37:05Z

@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2019-12-02T15:55:43Z

Packaging result: ✖centos6 ✔centos7 ✔debian. JID-418

DaanHoogland · 2019-12-02T15:56:44Z

@blueorangutan package

blueorangutan · 2019-12-02T15:57:04Z

@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

nvazquez

Code LGTM

blueorangutan · 2019-12-02T16:15:15Z

Packaging result: ✖centos6 ✔centos7 ✔debian. JID-419

DaanHoogland · 2019-12-02T16:48:31Z

@bwsw maybe you want to review this. I think it might solve your #3093 ?

DaanHoogland · 2019-12-02T22:38:39Z

@blueorangutan test

blueorangutan · 2019-12-02T22:39:05Z

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2019-12-03T06:30:19Z

Trillian test result (tid-573)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 26783 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3733-t573-kvm-centos7.zip
Smoke tests completed. 77 look OK, 0 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File

weizhouapache

code LGTM

and 4.13 as well ?

wido

LGTM on code

DaanHoogland · 2019-12-05T14:30:41Z

will be backporting to 4.13 @weizhouapache

borisstoyanov · 2019-12-13T08:16:45Z

@blueorangutan package

blueorangutan · 2019-12-13T08:16:51Z

@borisstoyanov a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2019-12-13T08:38:51Z

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-463

borisstoyanov · 2019-12-13T12:26:52Z

@blueorangutan test

blueorangutan · 2019-12-13T12:27:52Z

@borisstoyanov a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

borisstoyanov

LGTM, can be merged if no regressions appear in tests

blueorangutan · 2019-12-13T23:09:55Z

Trillian test result (tid-619)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 36673 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3733-t619-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 75 look OK, 2 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_04_rvpc_internallb_haproxy_stats_on_all_interfaces	`Error`	201.17	test_internal_lb.py
test_05_rvpc_multi_tiers	`Failure`	417.23	test_vpc_redundant.py
test_05_rvpc_multi_tiers	`Error`	446.50	test_vpc_redundant.py

* create template from snapshot regression (partly reverted) (#3767) * Once again allow a VM to be on multiple networks from VPCs (#3754) to once again allow a VM to be on multiple networks from VPCs * convert protocal names to be found as labels (#3747) * convert protocal names to be found as labels * format * filter hosts to query on zone wide storage (#3733) * config: add isdynamic flag in configuration response (#3729) Co-authored-by: Wei Zhou <ustcweizhou@gmail.com>

* 4.13: Added zone check for attach iso (#3755) config: add isdynamic flag in configuration response (#3729) filter hosts to query on zone wide storage (#3733) convert protocal names to be found as labels (#3747) Once again allow a VM to be on multiple networks from VPCs (#3754) create template from snapshot regression (partly reverted) (#3767)

DaanHoogland requested a review from nvazquez December 2, 2019 15:54

nvazquez approved these changes Dec 2, 2019

View reviewed changes

DaanHoogland marked this pull request as ready for review December 2, 2019 19:06

DaanHoogland requested review from GabrielBrascher, weizhouapache and wido December 3, 2019 07:49

weizhouapache approved these changes Dec 3, 2019

View reviewed changes

wido approved these changes Dec 3, 2019

View reviewed changes

svenvogel added type:bug component:management-server labels Dec 3, 2019

filter hosts to query on zone wide - or local storage

2990eb3

DaanHoogland force-pushed the issue-3596 branch from a3b80b7 to 2990eb3 Compare December 5, 2019 15:17

DaanHoogland changed the base branch from master to 4.13 December 5, 2019 15:18

yadvr added this to the 4.13.1.0 milestone Dec 7, 2019

borisstoyanov approved these changes Dec 13, 2019

View reviewed changes

borisstoyanov added the status:ready-for-merge label Dec 13, 2019

DaanHoogland merged commit 3ac03c8 into apache:4.13 Dec 20, 2019

DaanHoogland deleted the issue-3596 branch December 20, 2019 13:12

ustcweizhou pushed a commit to ustcweizhou/cloudstack that referenced this pull request Feb 28, 2020

filter hosts to query on zone wide storage (apache#3733)

de9753b

DaanHoogland removed the status:ready-for-merge label Dec 4, 2024

Conversation

DaanHoogland commented Dec 2, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Screenshots (if appropriate):

How Has This Been Tested?

before

Uh oh!

DaanHoogland commented Dec 2, 2019

Uh oh!

blueorangutan commented Dec 2, 2019

Uh oh!

blueorangutan commented Dec 2, 2019

Uh oh!

DaanHoogland commented Dec 2, 2019

Uh oh!

blueorangutan commented Dec 2, 2019

Uh oh!

nvazquez left a comment

Choose a reason for hiding this comment

Uh oh!

blueorangutan commented Dec 2, 2019

Uh oh!

DaanHoogland commented Dec 2, 2019

Uh oh!

DaanHoogland commented Dec 2, 2019

Uh oh!

blueorangutan commented Dec 2, 2019

Uh oh!

blueorangutan commented Dec 3, 2019

Uh oh!

weizhouapache left a comment

Choose a reason for hiding this comment

Uh oh!

wido left a comment

Choose a reason for hiding this comment

Uh oh!

DaanHoogland commented Dec 5, 2019

Uh oh!

borisstoyanov commented Dec 13, 2019

Uh oh!

blueorangutan commented Dec 13, 2019

Uh oh!

blueorangutan commented Dec 13, 2019

Uh oh!

borisstoyanov commented Dec 13, 2019

Uh oh!

blueorangutan commented Dec 13, 2019

Uh oh!

borisstoyanov left a comment

Choose a reason for hiding this comment

Uh oh!

blueorangutan commented Dec 13, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

DaanHoogland commented Dec 2, 2019 •

edited

Loading