cks: use public links for templates and binaries iso for smoke tests

[shwstppr]

Description

Use public links for templates and binaries iso for CKS smoke tests

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)

Screenshots (if appropriate):

How Has This Been Tested?

[DaanHoogland]

@andrijapanicsb if we get this to work before release, i'd like it in so we have a clean run from trillian.

[DaanHoogland]

@shwstppr can you check those results? two platforms fail but the third passes the same test.

[DaanHoogland]

looking at the code i see a 1.16.0 mentioned but on the site it is missing. might that be the issue?

	Name	Last modified	Size	Description
	Parent Directory		-
	setup-1.11.4.iso	2020-01-14 11:52	1.0G
	setup-1.12.5.iso	2020-01-14 11:54	1.0G
	setup-1.13.2.iso	2020-01-14 11:55	927M
	setup-1.14.9.iso	2020-01-14 11:57	1.0G
	setup-1.15.0.iso	2020-01-14 12:00	1.0G
	setup-1.16.3.iso	2020-01-14 12:01	1.0G

[DaanHoogland]

I guess not, the error is in the other test file. Can you please look at this error as well and advice if this can independendly merged, @shwstppr ?

[shwstppr]

@DaanHoogland will look at the logs and will try to fix the issue in next hours or tomorrow morning

[shwstppr]

looking at the code i see a 1.16.0 mentioned but on the site it is missing. might that be the issue?
Name Last modified Size Description
Parent Directory -
setup-1.11.4.iso 2020-01-14 11:52 1.0G
setup-1.12.5.iso 2020-01-14 11:54 1.0G
setup-1.13.2.iso 2020-01-14 11:55 927M
setup-1.14.9.iso 2020-01-14 11:57 1.0G
setup-1.15.0.iso 2020-01-14 12:00 1.0G
setup-1.16.3.iso 2020-01-14 12:01 1.0G

@DaanHoogland yes 1.16.0 is missing and it is needed for testing upgrade on HA k8s cluster. I've internally asked for adding it at a public repo.

[blueorangutan]

Trillian test result (tid-1492)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 36895 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3992-t1492-xenserver-71.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Intermittent failure detected: /marvin/tests/smoke/test_scale_vm.py
Smoke tests completed. 81 look OK, 2 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_01_add_delete_kubernetes_supported_version	Error	1206.82	test_kubernetes_supported_versions.py
test_01_scale_vm	Failure	27.75	test_scale_vm.py

[blueorangutan]

Trillian test result (tid-1493)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 37150 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3992-t1493-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Smoke tests completed. 82 look OK, 1 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_01_add_delete_kubernetes_supported_version	Error	1207.17	test_kubernetes_supported_versions.py

[blueorangutan]

Trillian test result (tid-1494)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 45133 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3992-t1494-vmware-67u3.zip
Smoke tests completed. 83 look OK, 0 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File

[shwstppr]

@blueorangutan package

[blueorangutan]

@shwstppr a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result: ✔centos7 ✔debian. JID-1206

[shwstppr]

@blueorangutan test matrix

[blueorangutan]

@shwstppr a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware67, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

[blueorangutan]

Trillian test result (tid-1496)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 33488 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3992-t1496-xenserver-71.zip
Intermittent failure detected: /marvin/tests/smoke/test_scale_vm.py
Smoke tests completed. 82 look OK, 1 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_01_scale_vm	Failure	20.46	test_scale_vm.py

[blueorangutan]

Trillian test result (tid-1497)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 35859 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3992-t1497-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 83 look OK, 0 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File

[blueorangutan]

Trillian test result (tid-1498)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 50547 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3992-t1498-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vm_root_resize.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Intermittent failure detected: /marvin/tests/smoke/test_network.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Smoke tests completed. 82 look OK, 1 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_01_add_delete_kubernetes_supported_version	Error	1807.22	test_kubernetes_supported_versions.py

[shwstppr]

@blueorangutan test centos7 vmware-67u3

[blueorangutan]

@shwstppr a Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests

[shwstppr]

@PaulAngus @rhtyd @borisstoyanov @andrijapanicsb
Need your opinion on this. Since CKS tests need multiple k8s binaries ISOs (~1GB in size each) which makes tests to run for longer time for download (from the Internet) and install of these ISOs does it make sense to move CKS tests into component tests from smoke-tests?
Or any other way you can suggest to make this part of the tests more reliable, faster?
cc @DaanHoogland

[yadvr]

@shwstppr does it take more than 0.5-1 hour to run the test, I would keep it in smoketest/
You can host the isos locally on staging server which is closer to the trillian env; make the url configure URL specific to the Trillian env via its test_data: https://github.com/shapeblue/Trillian/blob/master/Ansible/roles/marvin/templates/test_data.py.j2
The test_data (default) is here: https://github.com/apache/cloudstack/blob/master/tools/marvin/marvin/config/test_data.py

[DaanHoogland]

@rhtyd I don't agree either a run of more than 0.5 hour or a download of multiple 1GB files would justify not running it on every test run.
adding the URLs to test_data does certainly make sense though.

[DaanHoogland]

@shwstppr could we spoof the ISOs with smaller files and put those in a local webserver somewhere?

[blueorangutan]

Trillian test result (tid-1499)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 38500 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3992-t1499-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Smoke tests completed. 81 look OK, 0 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File

[yadvr]

LGTM, merge this @DaanHoogland @andrijapanicsb ?

[andrijapanicsb]

@shwstppr can you please advise on the duration of these tests and can we also override the URLs the same way we do with i.e. systemVM templates or default centos templates, to be downloaded actually from our local web server - downloading from download.cloudstack is slow

[andrijapanicsb]

[root@ref-trl-902-k-M7-andrija-panic-mgmt1 ~]# wget http://download.cloudstack.org/cks/setup-1.14.9.iso
--2020-05-06 11:32:06-- http://download.cloudstack.org/cks/setup-1.14.9.iso
Resolving download.cloudstack.org (download.cloudstack.org)... 185.27.174.49, 2a00:f10:121:400:403:9cff:fe00:37f
Connecting to download.cloudstack.org (download.cloudstack.org)|185.27.174.49|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1077303296 (1.0G) [application/x-iso9660-image]
Saving to: ‘setup-1.14.9.iso’

100%[=========================================================================>] 1,077,303,296 8.07MB/s in 2m 8s

So for 5 x ISOs, that's 10min at least - could be worse I guess...

[shwstppr]

@andrijapanicsb I'll work for overriding URLs with local server URLs as Rohit suggested in next few days or over the weekend. Can be fixed in next release/after 4.14

[andrijapanicsb]

sure @shwstppr - keep in mind we override inside Trillian code, not in the public git code here!
thx