text:
CAMEL-20400: Support for Knative SinkBinding

Description

Target

[x] I checked that the commit is targeting the correct branch (note that Camel 3 uses camel-3.x, whereas Camel 4 uses the main branch)

Tracking

[x] If this is a large change, bug fix, or code improvement, I checked there is a JIRA issue filed for the change (usually before you start working on it).

Apache Camel coding standards and style

[x] I checked that each commit in the pull request has a meaningful subject line and body.

[x] I have run mvn clean install -DskipTests locally and I have committed all auto-generated changes

@squakez once merged, you should be able to use properties to configure sink binding
camel.component.knative.sink-binding.name = …
There is also a Producer Health Check that should mark the context as non ready till the K_SINK env var is made available to the process. Note that Producer Health Check must be explicitly enabled

repo: apache/camel


text:
CAMEL-20632- Fix documentation for Camel Langchain4j chat component

Description

Target

[ ] I checked that the commit is targeting the correct branch (note that Camel 3 uses camel-3.x, whereas Camel 4 uses the main branch)

Tracking

[ ] If this is a large change, bug fix, or code improvement, I checked there is a JIRA issue filed for the change (usually before you start working on it).

Apache Camel coding standards and style

[ ] I checked that each commit in the pull request has a meaningful subject line and body.

[ ] I have run mvn clean install -DskipTests locally and I have committed all auto-generated changes

I'll open a follow for the 4.5 branch, hopefully this will automatically fix the documentation

So it seems there's no need to follow up for 4.5 as it's a non LTS

source: gharchive/pull-request

license_type: permissive

license_source: github-api


text:
[Doc]Update HA doc

What changes were proposed in this pull request?
The param(celeborn.master.ha.node.id) is not required to set for master HA anymore.
Why are the changes needed?
remove the param from HA section
Does this PR introduce any user-facing change?
No
How was this patch tested?

Thanks. I'll make a Jira ticket and connect to it.

authors: FMX zhaohehuhu

repo: apache/celeborn


text:
CLIMATE-839 - Further improvements to DatasetLoader

@huikyole please review ASAP. New unit tests pass and I was able to run the examples (note I had to make a slight change to local.load_multiple_files().

Can one of the admins verify this patch?

I think this PR is ready to be merged.

authors: OCWJenkins agoodm huikyole

repo: apache/climate

license_source: bigquery


text:
Handle redirect to sub menu when click parent menu

Fixes #805
@rhtyd cc @svenvogel Please review it. Thanks.

@blueorangutan package

@davidjumani a Jenkins job has been kicked to build primate packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark:centos :heavy_check_mark:debian :heavy_check_mark:archive.
QA: http://primate-qa.cloudstack.cloud:8080/client/pr/876 (JID-3701)

I still see the same old behaviour @utchoang can you check the test/QA env.

@rhtyd Currently, it only activity for the tablet and mobile mode.

@utchoang can you do this for desktop as well?

@blueorangutan  package

@blueorangutan package

@davidjumani a Jenkins job has been kicked to build primate packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark:centos :heavy_check_mark:debian :heavy_check_mark:archive.
QA: http://primate-qa.cloudstack.cloud:8080/client/pr/876 (JID-3757)

Thanks @utchoang I tested does not seem to work, is there something I'm missing (I'm expecting the VM to be selected/open when I click compute)?

@rhtyd Sorry about my mistake. Please review it again. Thanks.

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build primate packages. I'll keep you posted as I make progress.

@blueorangutan package

@utchoang a Jenkins job has been kicked to build primate packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark:centos :heavy_check_mark:debian :heavy_check_mark:archive.
QA: http://primate-qa.cloudstack.cloud:8080/client/pr/876 (JID-3761)

Packaging result: :heavy_check_mark:centos :heavy_check_mark:debian :heavy_check_mark:archive.
QA: http://primate-qa.cloudstack.cloud:8080/client/pr/876 (JID-3762)

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build primate packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark:centos :heavy_check_mark:debian :heavy_check_mark:archive.
QA: http://primate-qa.cloudstack.cloud:8080/client/pr/876 (JID-3763)

Tested few screens for mobile and desktop - lgtm.

@utchoang It intermittently works on desktops. Just navigate from top to bottom and back up again and sometimes it does not open the first sub-menu. Could you fix it ?

@blueorangutan package

@utchoang a Jenkins job has been kicked to build primate packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark:centos :heavy_check_mark:debian :heavy_check_mark:archive.
QA: http://primate-qa.cloudstack.cloud:8080/client/pr/876 (JID-3765)

@blueorangutan package

@davidjumani a Jenkins job has been kicked to build primate packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark:centos :heavy_check_mark:debian :heavy_check_mark:archive.
QA: http://primate-qa.cloudstack.cloud:8080/client/pr/876 (JID-3768)

LGTM, @davidjumani are you lgtm on it for merging?

@utchoang Looks like navigation sometimes breaks now if anything other than the first sub-menu item is selected :(

@blueorangutan package

@utchoang a Jenkins job has been kicked to build primate packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark:centos :heavy_check_mark:debian :heavy_check_mark:archive.
QA: http://primate-qa.cloudstack.cloud:8080/client/pr/876 (JID-3771)

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build primate packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark:centos :heavy_check_mark:debian :heavy_check_mark:archive.
QA: http://primate-qa.cloudstack.cloud:8080/client/pr/876 (JID-3772)

LGTM - on latest testing on desktop and mobile; the only case it doesn't work is when you click on right on the nav title (not on the expand button but between the title/label and the right expand button, it expands than open the first child) - still better than what we've now.
@davidjumani can you test your cases?

@utchoang @rhtyd Works well on mobile and tablet. Clicking the icon / text while on desktop opens the first submenu, so users who'd just like to brows the navbar can click the blank space

authors: blueorangutan davidjumani rhtyd utchoang

repo: apache/cloudstack-primate


text:
IGNORE this PR

Ignore this PR/commit, this is for testing Trilllian, @BlueOrangutan

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-19

Test completed. 49 look ok, 4 have errors

Test
Result
Time (s)

test_isolate_network_password_server
Failure
90.686

test_04_rvpc_privategw_static_routes
Failure
157.803

test_03_vpc_privategw_restart_vpc_cleanup
Failure
96.899

test_02_vpc_privategw_static_routes
Failure
81.847

test_01_vpc_privategw_acl
Failure
51.507

test_router_dhcphosts
Error
823.283

test_01_vpc_remote_access_vpn
Error
0.074

ContextSuite context=TestRouterDHCPHosts>:teardown
Error
874.517

test_network_acl
Success
75.920

test_05_rvpc_multi_tiers
Success
443.256

test_04_rvpc_network_garbage_collector_nics
Success
792.190

test_04_rvpc_internallb_haproxy_stats_on_all_interfaces
Success
233.263

test_03_vpc_internallb_haproxy_stats_on_all_interfaces
Success
177.953

test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers
Success
469.464

test_03_RVR_Network_check_router_state
Success
252.395

test_02_redundant_VPC_default_routes
Success
493.688

test_02_isolate_network_FW_PF_default_routes_egress_false
Success
236.680

test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80
Success
515.308

test_02_VPC_default_routes
Success
284.319

test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
Success
269.897

test_01_vpc_site2site_vpn
Success
160.766

test_01_redundant_vpc_site2site_vpn
Success
230.337

test_01_isolate_network_FW_PF_default_routes_egress_true
Success
196.616

test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80
Success
424.778

test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
Success
1033.463

test_01_VPC_nics_after_destroy
Success
553.071

test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true
Success
257.057

TID-

Trillian smoketest result (#)
Hypervisor: kvm (x2)
Network type: Advanced
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/trillian-19-kvm-cs410.zip
Test completed. 49 look ok, 4 have errors

Test
Result
Time (s)

test_isolate_network_password_server
Failure
90.686

test_04_rvpc_privategw_static_routes
Failure
157.803

test_03_vpc_privategw_restart_vpc_cleanup
Failure
96.899

test_02_vpc_privategw_static_routes
Failure
81.847

test_01_vpc_privategw_acl
Failure
51.507

test_router_dhcphosts
Error
823.283

test_01_vpc_remote_access_vpn
Error
0.074

ContextSuite context=TestRouterDHCPHosts>:teardown
Error
874.517

test_network_acl
Success
75.920

test_05_rvpc_multi_tiers
Success
443.256

test_04_rvpc_network_garbage_collector_nics
Success
792.190

test_04_rvpc_internallb_haproxy_stats_on_all_interfaces
Success
233.263

test_03_vpc_internallb_haproxy_stats_on_all_interfaces
Success
177.953

test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers
Success
469.464

test_03_RVR_Network_check_router_state
Success
252.395

test_02_redundant_VPC_default_routes
Success
493.688

test_02_isolate_network_FW_PF_default_routes_egress_false
Success
236.680

test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80
Success
515.308

test_02_VPC_default_routes
Success
284.319

test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
Success
269.897

test_01_vpc_site2site_vpn
Success
160.766

test_01_redundant_vpc_site2site_vpn
Success
230.337

test_01_isolate_network_FW_PF_default_routes_egress_true
Success
196.616

test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80
Success
424.778

test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
Success
1033.463

test_01_VPC_nics_after_destroy
Success
553.071

test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true
Success
257.057

@blueorangutan test

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + vmware-55u3) has been kicked to run smoke tests against packages at http://packages.shapeblue.com/cloudstack/pr/1682

@blueorangutan test matrix

@rhtyd a Trillian-Jenkins matrix job (centos6 mgmt + xs56sp1, centos7 mgmt + vmware55u3, ubuntu mgmt + kvmcentos7) has been kicked to run smoke tests

authors: blueorangutan rhtyd

repo: apache/cloudstack


text:
default teardown methods with reversed() handling

Description

In this a small change to improve the marvin framework for resource cleanup and adding examples in the test-implementations.

Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[x] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (improves an existing feature and functionality)
[x] Cleanup (Code refactoring and cleanup, that may add test cases)

Screenshots (if appropriate):
How Has This Been Tested?

any thoughts @GabrielBrascher @borisstoyanov @rhtyd

also @vladimirpetrov

@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2179

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan test centos7 kvm-centos7 keepEnv

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Trillian test result (tid-2959)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 1276 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4289-t2959-kvm-centos7.zip
Smoke tests completed. 0 look OK, 1 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

@blueorangutan test centos7 kvm-centos7 keepEnv

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan package

@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2189

Trillian test result (tid-2966)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 58894 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4289-t2966-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_affinity_groups_projects.py
Intermittent failure detected: /marvin/tests/smoke/test_affinity_groups.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_virtio_scsi_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vms_with_varied_deploymentplanners.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vm_with_userdata.py
Intermittent failure detected: /marvin/tests/smoke/test_diagnostics.py
Intermittent failure detected: /marvin/tests/smoke/test_disk_offerings.py
Intermittent failure detected: /marvin/tests/smoke/test_dynamicroles.py
Intermittent failure detected: /marvin/tests/smoke/test_enable_role_based_users_in_projects.py
Intermittent failure detected: /marvin/tests/smoke/test_global_settings.py
Intermittent failure detected: /marvin/tests/smoke/test_host_annotations.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_simulator.py
Intermittent failure detected: /marvin/tests/smoke/test_human_readable_logs.py
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_iso.py
Intermittent failure detected: /marvin/tests/smoke/test_list_ids_parameter.py
Intermittent failure detected: /marvin/tests/smoke/test_login.py
Intermittent failure detected: /marvin/tests/smoke/test_metrics_api.py
Intermittent failure detected: /marvin/tests/smoke/test_multipleips_per_nic.py
Intermittent failure detected: /marvin/tests/smoke/test_network.py
Intermittent failure detected: /marvin/tests/smoke/test_nic.py
Intermittent failure detected: /marvin/tests/smoke/test_non_contigiousvlan.py
Intermittent failure detected: /marvin/tests/smoke/test_outofbandmanagement_nestedplugin.py
Intermittent failure detected: /marvin/tests/smoke/test_over_provisioning.py
Intermittent failure detected: /marvin/tests/smoke/test_password_server.py
Intermittent failure detected: /marvin/tests/smoke/test_primary_storage.py
Intermittent failure detected: /marvin/tests/smoke/test_pvlan.py
Intermittent failure detected: /marvin/tests/smoke/test_regions.py
Intermittent failure detected: /marvin/tests/smoke/test_routers.py
Intermittent failure detected: /marvin/tests/smoke/test_service_offerings.py
Intermittent failure detected: /marvin/tests/smoke/test_ssvm.py
Intermittent failure detected: /marvin/tests/smoke/test_staticroles.py
Intermittent failure detected: /marvin/tests/smoke/test_templates.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 49 look OK, 36 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

ContextSuite context=TestDynamicRoles>:teardown
Error
1.43
test_dynamicroles.py

test_DeployVmAntiAffinityGroup_in_project
Error
110.25
test_affinity_groups_projects.py

ContextSuite context=TestUpdateConfigWithScope>:teardown
Error
0.05
test_global_settings.py

test_DeployVmAffinityGroup
Error
108.33
test_affinity_groups.py

test_DeployVmAntiAffinityGroup
Error
72.56
test_affinity_groups.py

ContextSuite context=TestHostAnnotations>:teardown
Error
0.04
test_host_annotations.py

ContextSuite context=TestHostHA>:teardown
Error
0.01
test_hostha_simulator.py

test_02_verify_libvirt_after_restart
Error
0.02
test_deploy_virtio_scsi_vm.py

test_03_verify_libvirt_attach_disk
Error
0.01
test_deploy_virtio_scsi_vm.py

test_04_verify_guest_lspci
Error
810.72
test_deploy_virtio_scsi_vm.py

test_05_change_vm_ostype_restart
Error
0.02
test_deploy_virtio_scsi_vm.py

test_06_verify_guest_lspci_again
Error
748.13
test_deploy_virtio_scsi_vm.py

test_deployvm_userconcentrated
Error
0.04
test_deploy_vms_with_varied_deploymentplanners.py

test_deployvm_userdispersing
Error
0.04
test_deploy_vms_with_varied_deploymentplanners.py

ContextSuite context=TestCreateISOWithChecksum>:teardown
Error
6.29
test_iso.py

ContextSuite context=TestCreateIso>:teardown
Error
6.59
test_iso.py

test_deployvm_userdata
Error
74.27
test_deploy_vm_with_userdata.py

test_deployvm_userdata_post
Error
37.40
test_deploy_vm_with_userdata.py

test_02_ping_in_vr_failure
Error
0.01
test_diagnostics.py

test_07_arping_in_vr
Error
0.01
test_diagnostics.py

test_10_traceroute_in_vr
Error
0.01
test_diagnostics.py

test_13_retrieve_vr_default_files
Error
0.01
test_diagnostics.py

test_14_retrieve_vr_one_file
Error
0.01
test_diagnostics.py

ContextSuite context=TestCreateDiskOffering>:teardown
Error
0.02
test_disk_offerings.py

ContextSuite context=TestLogin>:teardown
Error
5.18
test_login.py

ContextSuite context=TestRoleBasedUsersInProjects>:teardown
Error
4.90
test_enable_role_based_users_in_projects.py

test_01_disableHumanReadableLogs
Error
63.17
test_human_readable_logs.py

test_02_enableHumanReadableLogs
Error
123.23
test_human_readable_logs.py

ContextSuite context=TestHumanReadableLogs>:teardown
Error
123.23
test_human_readable_logs.py

test_01_list_volumes
Error
0.02
test_list_ids_parameter.py

test_02_list_templates
Error
0.03
test_list_ids_parameter.py

test_03_list_snapshots
Error
0.03
test_list_ids_parameter.py

ContextSuite context=TestMetrics>:teardown
Error
0.15
test_metrics_api.py

ContextSuite context=TestDeployVM>:teardown
Error
102.06
test_multipleips_per_nic.py

ContextSuite context=TestDeleteAccount>:teardown
Error
175.32
test_network.py

test_public_ip_admin_account
Error
32.17
test_network.py

test_public_ip_user_account
Error
2.17
test_network.py

ContextSuite context=TestRebootRouter>:teardown
Error
363.24
test_network.py

ContextSuite context=TestReleaseIP>:teardown
Error
130.37
test_network.py

ContextSuite context=TestNic>:teardown
Error
64.08
test_nic.py

ContextSuite context=TestUpdatePhysicalNetwork>:teardown
Error
3.23
test_non_contigiousvlan.py

ContextSuite context=TestOutOfBandManagement>:teardown
Error
2.17
test_outofbandmanagement_nestedplugin.py

ContextSuite context=TestUpdateOverProvision>:teardown
Error
0.10
test_over_provisioning.py

test_isolate_network_password_server
Error
18.11
test_password_server.py

ContextSuite context=TestPrimaryStorageServices>:teardown
Error
31.68
test_primary_storage.py

test_create_pvlan_network
Error
1.15
test_pvlan.py

ContextSuite context=TestPVLAN>:teardown
Error
1.15
test_pvlan.py

ContextSuite context=TestRegions>:teardown
Error
0.06
test_regions.py

test_02_router_internal_adv
Error
0.01
test_routers.py

test_03_restart_network_cleanup
Error
0.01
test_routers.py

test_04_restart_network_wo_cleanup
Error
0.01
test_routers.py

test_05_router_basic
Error
0.01
test_routers.py

test_06_router_advanced
Error
0.01
test_routers.py

test_07_stop_router
Error
0.01
test_routers.py

test_08_start_router
Error
0.01
test_routers.py

test_09_reboot_router
Error
0.01
test_routers.py

ContextSuite context=TestCreateServiceOffering>:teardown
Error
0.06
test_service_offerings.py

ContextSuite context=TestSSVMs>:teardown
Error
0.02
test_ssvm.py

ContextSuite context=TestStaticRoles>:teardown
Error
0.01
test_staticroles.py

ContextSuite context=TestCreateTemplateWithChecksum>:teardown
Error
15.17
test_templates.py

test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
Failure
403.83
test_vpc_redundant.py

test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers
Error
402.71
test_vpc_redundant.py

test_04_rvpc_network_garbage_collector_nics
Error
3844.41
test_vpc_redundant.py

test_01_redundant_vpc_site2site_vpn
Error
358.25
test_vpc_vpn.py

test_01_vpc_site2site_vpn_multiple_options
Error
778.38
test_vpc_vpn.py

test_01_vpc_site2site_vpn
Error
204.20
test_vpc_vpn.py

ContextSuite context=TestHostMaintenance>:teardown
Error
635.78
test_host_maintenance.py

test_hostha_kvm_host_fencing
Error
170.50
test_hostha_kvm.py

ContextSuite context=TestHAKVM>:teardown
Error
6.59
test_hostha_kvm.py

@blueorangutan package

@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2196

@blueorangutan test

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Trillian test result (tid-2980)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 41493 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4289-t2980-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_accounts.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_virtio_scsi_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vms_with_varied_deploymentplanners.py
Intermittent failure detected: /marvin/tests/smoke/test_diagnostics.py
Intermittent failure detected: /marvin/tests/smoke/test_routers.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 77 look OK, 8 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

ContextSuite context=TestNonRootAdminsPrivileges>:teardown
Error
3.18
test_accounts.py

ContextSuite context=TestRemoveUserFromAccount>:teardown
Error
102.55
test_accounts.py

test_02_verify_libvirt_after_restart
Error
0.02
test_deploy_virtio_scsi_vm.py

test_03_verify_libvirt_attach_disk
Error
0.02
test_deploy_virtio_scsi_vm.py

test_04_verify_guest_lspci
Error
810.68
test_deploy_virtio_scsi_vm.py

test_05_change_vm_ostype_restart
Error
0.02
test_deploy_virtio_scsi_vm.py

test_06_verify_guest_lspci_again
Error
756.37
test_deploy_virtio_scsi_vm.py

test_deployvm_userconcentrated
Error
0.06
test_deploy_vms_with_varied_deploymentplanners.py

test_deployvm_userdispersing
Error
0.05
test_deploy_vms_with_varied_deploymentplanners.py

test_02_ping_in_vr_failure
Error
0.02
test_diagnostics.py

test_07_arping_in_vr
Error
0.02
test_diagnostics.py

test_10_traceroute_in_vr
Error
0.02
test_diagnostics.py

test_13_retrieve_vr_default_files
Error
0.02
test_diagnostics.py

test_14_retrieve_vr_one_file
Error
0.02
test_diagnostics.py

test_02_router_internal_adv
Error
0.02
test_routers.py

test_03_restart_network_cleanup
Error
0.01
test_routers.py

test_04_restart_network_wo_cleanup
Error
0.01
test_routers.py

test_05_router_basic
Error
0.01
test_routers.py

test_06_router_advanced
Error
0.02
test_routers.py

test_07_stop_router
Error
0.01
test_routers.py

test_08_start_router
Error
0.01
test_routers.py

test_09_reboot_router
Error
0.02
test_routers.py

test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
Failure
310.13
test_vpc_redundant.py

test_01_redundant_vpc_site2site_vpn
Error
244.58
test_vpc_vpn.py

test_01_vpc_site2site_vpn_multiple_options
Error
618.65
test_vpc_vpn.py

test_01_vpc_site2site_vpn
Error
155.94
test_vpc_vpn.py

test_hostha_kvm_host_fencing
Error
172.41
test_hostha_kvm.py

Trillian test result (tid-2979)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 58570 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4289-t2979-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_accounts.py
Intermittent failure detected: /marvin/tests/smoke/test_affinity_groups_projects.py
Intermittent failure detected: /marvin/tests/smoke/test_affinity_groups.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_virtio_scsi_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vms_with_varied_deploymentplanners.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vm_with_userdata.py
Intermittent failure detected: /marvin/tests/smoke/test_diagnostics.py
Intermittent failure detected: /marvin/tests/smoke/test_disk_offerings.py
Intermittent failure detected: /marvin/tests/smoke/test_dynamicroles.py
Intermittent failure detected: /marvin/tests/smoke/test_enable_role_based_users_in_projects.py
Intermittent failure detected: /marvin/tests/smoke/test_global_settings.py
Intermittent failure detected: /marvin/tests/smoke/test_host_annotations.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_simulator.py
Intermittent failure detected: /marvin/tests/smoke/test_human_readable_logs.py
Intermittent failure detected: /marvin/tests/smoke/test_iso.py
Intermittent failure detected: /marvin/tests/smoke/test_list_ids_parameter.py
Intermittent failure detected: /marvin/tests/smoke/test_login.py
Intermittent failure detected: /marvin/tests/smoke/test_metrics_api.py
Intermittent failure detected: /marvin/tests/smoke/test_multipleips_per_nic.py
Intermittent failure detected: /marvin/tests/smoke/test_network.py
Intermittent failure detected: /marvin/tests/smoke/test_nic.py
Intermittent failure detected: /marvin/tests/smoke/test_non_contigiousvlan.py
Intermittent failure detected: /marvin/tests/smoke/test_outofbandmanagement_nestedplugin.py
Intermittent failure detected: /marvin/tests/smoke/test_over_provisioning.py
Intermittent failure detected: /marvin/tests/smoke/test_password_server.py
Intermittent failure detected: /marvin/tests/smoke/test_primary_storage.py
Intermittent failure detected: /marvin/tests/smoke/test_pvlan.py
Intermittent failure detected: /marvin/tests/smoke/test_regions.py
Intermittent failure detected: /marvin/tests/smoke/test_routers.py
Intermittent failure detected: /marvin/tests/smoke/test_service_offerings.py
Intermittent failure detected: /marvin/tests/smoke/test_ssvm.py
Intermittent failure detected: /marvin/tests/smoke/test_staticroles.py
Intermittent failure detected: /marvin/tests/smoke/test_templates.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 48 look OK, 37 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

ContextSuite context=TestDynamicRoles>:teardown
Error
1.65
test_dynamicroles.py

ContextSuite context=TestAccounts>:teardown
Error
1.80
test_accounts.py

ContextSuite context=TestAddVmToSubDomain>:teardown
Error
0.06
test_accounts.py

ContextSuite context=TestDomainForceRemove>:teardown
Error
176.88
test_accounts.py

test_move_user_incorrect_account_id
Error
1.36
test_accounts.py

test_move_user_incorrect_account_name
Error
1.38
test_accounts.py

test_move_user_to_accountID
Error
1.42
test_accounts.py

test_move_user_to_account_name
Error
1.39
test_accounts.py

test_move_user_to_different_domain
Error
1.90
test_accounts.py

ContextSuite context=TestMoveUser>:teardown
Error
1.90
test_accounts.py

ContextSuite context=TestNonRootAdminsPrivileges>:teardown
Error
3.12
test_accounts.py

ContextSuite context=TestRemoveUserFromAccount>:teardown
Error
133.20
test_accounts.py

ContextSuite context=TestServiceOfferingHierarchy>:teardown
Error
0.05
test_accounts.py

ContextSuite context=TestServiceOfferingSiblings>:teardown
Error
0.06
test_accounts.py

ContextSuite context=TestTemplateHierarchy>:teardown
Error
0.06
test_accounts.py

ContextSuite context=TestUserAPIKeys>:teardown
Error
0.73
test_accounts.py

ContextSuite context=TestUserDetails>:teardown
Error
1.59
test_accounts.py

ContextSuite context=TestUserLogin>:teardown
Error
1.97
test_accounts.py

ContextSuite context=TestUpdateConfigWithScope>:teardown
Error
0.07
test_global_settings.py

test_DeployVmAntiAffinityGroup_in_project
Error
108.61
test_affinity_groups_projects.py

test_DeployVmAffinityGroup
Error
111.86
test_affinity_groups.py

test_DeployVmAntiAffinityGroup
Error
71.69
test_affinity_groups.py

ContextSuite context=TestHostAnnotations>:teardown
Error
0.06
test_host_annotations.py

ContextSuite context=TestHostHA>:teardown
Error
0.02
test_hostha_simulator.py

test_02_verify_libvirt_after_restart
Error
0.02
test_deploy_virtio_scsi_vm.py

test_03_verify_libvirt_attach_disk
Error
0.02
test_deploy_virtio_scsi_vm.py

test_04_verify_guest_lspci
Error
810.73
test_deploy_virtio_scsi_vm.py

test_05_change_vm_ostype_restart
Error
0.02
test_deploy_virtio_scsi_vm.py

test_06_verify_guest_lspci_again
Error
747.21
test_deploy_virtio_scsi_vm.py

test_deployvm_userconcentrated
Error
0.06
test_deploy_vms_with_varied_deploymentplanners.py

test_deployvm_userdispersing
Error
0.05
test_deploy_vms_with_varied_deploymentplanners.py

ContextSuite context=TestCreateISOWithChecksum>:teardown
Error
6.21
test_iso.py

ContextSuite context=TestCreateIso>:teardown
Error
6.71
test_iso.py

test_deployvm_userdata
Error
72.43
test_deploy_vm_with_userdata.py

test_deployvm_userdata_post
Error
36.45
test_deploy_vm_with_userdata.py

test_02_ping_in_vr_failure
Error
0.02
test_diagnostics.py

test_07_arping_in_vr
Error
0.02
test_diagnostics.py

test_10_traceroute_in_vr
Error
0.02
test_diagnostics.py

test_13_retrieve_vr_default_files
Error
0.01
test_diagnostics.py

test_14_retrieve_vr_one_file
Error
0.02
test_diagnostics.py

ContextSuite context=TestCreateDiskOffering>:teardown
Error
0.03
test_disk_offerings.py

ContextSuite context=TestLogin>:teardown
Error
6.17
test_login.py

ContextSuite context=TestRoleBasedUsersInProjects>:teardown
Error
5.07
test_enable_role_based_users_in_projects.py

test_01_disableHumanReadableLogs
Error
63.55
test_human_readable_logs.py

test_02_enableHumanReadableLogs
Error
123.57
test_human_readable_logs.py

ContextSuite context=TestHumanReadableLogs>:teardown
Error
123.57
test_human_readable_logs.py

test_01_list_volumes
Error
0.04
test_list_ids_parameter.py

test_02_list_templates
Error
0.04
test_list_ids_parameter.py

test_03_list_snapshots
Error
0.05
test_list_ids_parameter.py

ContextSuite context=TestMetrics>:teardown
Error
0.23
test_metrics_api.py

ContextSuite context=TestDeployVM>:teardown
Error
102.59
test_multipleips_per_nic.py

ContextSuite context=TestDeleteAccount>:teardown
Error
175.04
test_network.py

test_public_ip_admin_account
Error
32.26
test_network.py

test_public_ip_user_account
Error
2.23
test_network.py

ContextSuite context=TestRebootRouter>:teardown
Error
364.60
test_network.py

ContextSuite context=TestReleaseIP>:teardown
Error
133.08
test_network.py

ContextSuite context=TestNic>:teardown
Error
64.40
test_nic.py

ContextSuite context=TestUpdatePhysicalNetwork>:teardown
Error
3.29
test_non_contigiousvlan.py

ContextSuite context=TestOutOfBandManagement>:teardown
Error
1.22
test_outofbandmanagement_nestedplugin.py

ContextSuite context=TestUpdateOverProvision>:teardown
Error
0.11
test_over_provisioning.py

test_isolate_network_password_server
Error
17.66
test_password_server.py

ContextSuite context=TestPrimaryStorageServices>:teardown
Error
31.80
test_primary_storage.py

test_create_pvlan_network
Error
1.20
test_pvlan.py

ContextSuite context=TestPVLAN>:teardown
Error
1.21
test_pvlan.py

ContextSuite context=TestRegions>:teardown
Error
0.11
test_regions.py

test_02_router_internal_adv
Error
0.02
test_routers.py

test_03_restart_network_cleanup
Error
0.02
test_routers.py

test_04_restart_network_wo_cleanup
Error
0.02
test_routers.py

test_05_router_basic
Error
0.01
test_routers.py

test_06_router_advanced
Error
0.02
test_routers.py

test_07_stop_router
Error
0.01
test_routers.py

test_08_start_router
Error
0.02
test_routers.py

test_09_reboot_router
Error
0.02
test_routers.py

ContextSuite context=TestCreateServiceOffering>:teardown
Error
0.03
test_service_offerings.py

ContextSuite context=TestSSVMs>:teardown
Error
0.02
test_ssvm.py

ContextSuite context=TestStaticRoles>:teardown
Error
0.02
test_staticroles.py

ContextSuite context=TestCreateTemplateWithChecksum>:teardown
Error
15.22
test_templates.py

test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
Failure
407.95
test_vpc_redundant.py

test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers
Error
333.42
test_vpc_redundant.py

test_04_rvpc_network_garbage_collector_nics
Error
3849.39
test_vpc_redundant.py

test_01_redundant_vpc_site2site_vpn
Error
361.43
test_vpc_vpn.py

test_01_vpc_site2site_vpn_multiple_options
Error
805.37
test_vpc_vpn.py

test_01_vpc_site2site_vpn
Error
204.10
test_vpc_vpn.py

ContextSuite context=TestHostMaintenance>:teardown
Error
822.50
test_host_maintenance.py

test_hostha_kvm_host_fencing
Error
99.52
test_hostha_kvm.py

ContextSuite context=TestHAKVM>:teardown
Error
4.73
test_hostha_kvm.py

@blueorangutan package

@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2204

@blueorangutan test

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan test

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Trillian test result (tid-3005)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 42493 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4289-t3005-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_accounts.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_virtio_scsi_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vms_with_varied_deploymentplanners.py
Intermittent failure detected: /marvin/tests/smoke/test_diagnostics.py
Intermittent failure detected: /marvin/tests/smoke/test_routers.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 78 look OK, 7 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

ContextSuite context=TestNonRootAdminsPrivileges>:teardown
Error
3.74
test_accounts.py

ContextSuite context=TestRemoveUserFromAccount>:teardown
Error
105.61
test_accounts.py

test_02_verify_libvirt_after_restart
Error
0.03
test_deploy_virtio_scsi_vm.py

test_03_verify_libvirt_attach_disk
Error
0.03
test_deploy_virtio_scsi_vm.py

test_04_verify_guest_lspci
Error
810.82
test_deploy_virtio_scsi_vm.py

test_05_change_vm_ostype_restart
Error
0.02
test_deploy_virtio_scsi_vm.py

test_06_verify_guest_lspci_again
Error
752.01
test_deploy_virtio_scsi_vm.py

test_deployvm_userconcentrated
Error
0.08
test_deploy_vms_with_varied_deploymentplanners.py

test_deployvm_userdispersing
Error
0.07
test_deploy_vms_with_varied_deploymentplanners.py

test_02_ping_in_vr_failure
Error
0.03
test_diagnostics.py

test_07_arping_in_vr
Error
0.02
test_diagnostics.py

test_10_traceroute_in_vr
Error
0.02
test_diagnostics.py

test_13_retrieve_vr_default_files
Error
0.02
test_diagnostics.py

test_14_retrieve_vr_one_file
Error
0.02
test_diagnostics.py

test_02_router_internal_adv
Error
0.03
test_routers.py

test_03_restart_network_cleanup
Error
0.02
test_routers.py

test_04_restart_network_wo_cleanup
Error
0.02
test_routers.py

test_05_router_basic
Error
0.02
test_routers.py

test_06_router_advanced
Error
0.02
test_routers.py

test_07_stop_router
Error
0.02
test_routers.py

test_08_start_router
Error
0.02
test_routers.py

test_09_reboot_router
Error
0.02
test_routers.py

test_01_redundant_vpc_site2site_vpn
Error
248.18
test_vpc_vpn.py

test_01_vpc_site2site_vpn_multiple_options
Error
608.84
test_vpc_vpn.py

test_01_vpc_site2site_vpn
Error
177.22
test_vpc_vpn.py

test_hostha_kvm_host_fencing
Error
170.48
test_hostha_kvm.py

@blueorangutan package

@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2210

Trillian test result (tid-3023)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 44504 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4289-t3023-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_deploy_virtio_scsi_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vms_with_varied_deploymentplanners.py
Intermittent failure detected: /marvin/tests/smoke/test_diagnostics.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Intermittent failure detected: /marvin/tests/smoke/test_routers.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 77 look OK, 7 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_02_verify_libvirt_after_restart
Error
0.03
test_deploy_virtio_scsi_vm.py

test_03_verify_libvirt_attach_disk
Error
0.03
test_deploy_virtio_scsi_vm.py

test_04_verify_guest_lspci
Error
810.72
test_deploy_virtio_scsi_vm.py

test_05_change_vm_ostype_restart
Error
0.02
test_deploy_virtio_scsi_vm.py

test_06_verify_guest_lspci_again
Error
751.93
test_deploy_virtio_scsi_vm.py

test_deployvm_userconcentrated
Error
0.08
test_deploy_vms_with_varied_deploymentplanners.py

test_deployvm_userdispersing
Error
0.08
test_deploy_vms_with_varied_deploymentplanners.py

test_02_ping_in_vr_failure
Error
0.03
test_diagnostics.py

test_07_arping_in_vr
Error
0.03
test_diagnostics.py

test_10_traceroute_in_vr
Error
0.02
test_diagnostics.py

test_13_retrieve_vr_default_files
Error
0.03
test_diagnostics.py

test_14_retrieve_vr_one_file
Error
0.03
test_diagnostics.py

test_02_router_internal_adv
Error
0.03
test_routers.py

test_03_restart_network_cleanup
Error
0.03
test_routers.py

test_04_restart_network_wo_cleanup
Error
0.03
test_routers.py

test_05_router_basic
Error
0.03
test_routers.py

test_06_router_advanced
Error
0.02
test_routers.py

test_07_stop_router
Error
0.03
test_routers.py

test_08_start_router
Error
0.03
test_routers.py

test_09_reboot_router
Error
0.03
test_routers.py

test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
Failure
428.00
test_vpc_redundant.py

test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers
Failure
422.40
test_vpc_redundant.py

test_01_redundant_vpc_site2site_vpn
Error
380.33
test_vpc_vpn.py

test_01_vpc_site2site_vpn_multiple_options
Error
656.34
test_vpc_vpn.py

test_01_vpc_site2site_vpn
Error
250.76
test_vpc_vpn.py

test_hostha_kvm_host_fencing
Error
175.34
test_hostha_kvm.py

@blueorangutan package

@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2282

@blueorangutan test

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan package

@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2286

@blueorangutan test

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Trillian test result (tid-3084)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 39110 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4289-t3084-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_deploy_virtio_scsi_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vms_with_varied_deploymentplanners.py
Intermittent failure detected: /marvin/tests/smoke/test_diagnostics.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_routers.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_router_nics.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 81 look OK, 5 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_02_verify_libvirt_after_restart
Error
0.03
test_deploy_virtio_scsi_vm.py

test_03_verify_libvirt_attach_disk
Error
0.03
test_deploy_virtio_scsi_vm.py

test_04_verify_guest_lspci
Error
810.77
test_deploy_virtio_scsi_vm.py

test_05_change_vm_ostype_restart
Error
0.03
test_deploy_virtio_scsi_vm.py

test_06_verify_guest_lspci_again
Error
752.70
test_deploy_virtio_scsi_vm.py

test_deployvm_userconcentrated
Error
0.08
test_deploy_vms_with_varied_deploymentplanners.py

test_deployvm_userdispersing
Error
0.07
test_deploy_vms_with_varied_deploymentplanners.py

test_02_ping_in_vr_failure
Error
0.03
test_diagnostics.py

test_07_arping_in_vr
Error
0.03
test_diagnostics.py

test_10_traceroute_in_vr
Error
0.02
test_diagnostics.py

test_13_retrieve_vr_default_files
Error
0.02
test_diagnostics.py

test_14_retrieve_vr_one_file
Error
0.02
test_diagnostics.py

test_02_router_internal_adv
Error
0.02
test_routers.py

test_03_restart_network_cleanup
Error
0.02
test_routers.py

test_04_restart_network_wo_cleanup
Error
0.02
test_routers.py

test_05_router_basic
Error
0.02
test_routers.py

test_06_router_advanced
Error
0.02
test_routers.py

test_07_stop_router
Error
0.02
test_routers.py

test_08_start_router
Error
0.03
test_routers.py

test_09_reboot_router
Error
0.02
test_routers.py

test_01_redundant_vpc_site2site_vpn
Error
215.83
test_vpc_vpn.py

test_01_vpc_site2site_vpn_multiple_options
Error
624.12
test_vpc_vpn.py

test_01_vpc_site2site_vpn
Error
175.38
test_vpc_vpn.py

Trillian test result (tid-3090)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 34035 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4289-t3090-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_deploy_virtio_scsi_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Smoke tests completed. 84 look OK, 2 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_02_verify_libvirt_after_restart
Error
0.03
test_deploy_virtio_scsi_vm.py

test_03_verify_libvirt_attach_disk
Error
0.03
test_deploy_virtio_scsi_vm.py

test_04_verify_guest_lspci
Error
810.74
test_deploy_virtio_scsi_vm.py

test_05_change_vm_ostype_restart
Error
0.03
test_deploy_virtio_scsi_vm.py

test_06_verify_guest_lspci_again
Error
760.75
test_deploy_virtio_scsi_vm.py

test_01_redundant_vpc_site2site_vpn
Error
0.18
test_vpc_vpn.py

test_01_vpc_site2site_vpn_multiple_options
Error
0.14
test_vpc_vpn.py

test_01_vpc_site2site_vpn
Error
0.17
test_vpc_vpn.py

@blueorangutan package

@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2288

@blueorangutan test centos7 kvm-centos7 keepEnv

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Trillian test result (tid-3093)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 37107 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4289-t3093-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_deploy_virtio_scsi_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Smoke tests completed. 84 look OK, 2 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_02_verify_libvirt_after_restart
Error
0.03
test_deploy_virtio_scsi_vm.py

test_03_verify_libvirt_attach_disk
Error
0.02
test_deploy_virtio_scsi_vm.py

test_04_verify_guest_lspci
Error
810.74
test_deploy_virtio_scsi_vm.py

test_05_change_vm_ostype_restart
Error
0.02
test_deploy_virtio_scsi_vm.py

test_06_verify_guest_lspci_again
Error
756.38
test_deploy_virtio_scsi_vm.py

test_01_redundant_vpc_site2site_vpn
Error
209.36
test_vpc_vpn.py

test_01_vpc_site2site_vpn_multiple_options
Error
595.60
test_vpc_vpn.py

test_01_vpc_site2site_vpn
Error
161.93
test_vpc_vpn.py

ContextSuite context=TestVpcSite2SiteVpn>:teardown
Error
161.96
test_vpc_vpn.py

@blueorangutan package

@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2344

@blueorangutan test

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Trillian test result (tid-3135)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 39482 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4289-t3135-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_deploy_virtio_scsi_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_volumes.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Smoke tests completed. 83 look OK, 3 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_02_verify_libvirt_after_restart
Error
0.03
test_deploy_virtio_scsi_vm.py

test_03_verify_libvirt_attach_disk
Error
0.03
test_deploy_virtio_scsi_vm.py

test_04_verify_guest_lspci
Error
810.66
test_deploy_virtio_scsi_vm.py

test_05_change_vm_ostype_restart
Error
0.02
test_deploy_virtio_scsi_vm.py

test_06_verify_guest_lspci_again
Error
756.34
test_deploy_virtio_scsi_vm.py

test_01_isolate_network_FW_PF_default_routes_egress_true
Failure
58.06
test_routers_network_ops.py

test_01_vpc_site2site_vpn_multiple_options
Error
673.95
test_vpc_vpn.py

ContextSuite context=TestVpcSite2SiteVpn>:teardown
Error
255.71
test_vpc_vpn.py

Trillian test result (tid-3137)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 36849 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4289-t3137-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Smoke tests completed. 84 look OK, 2 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_07_deploy_kubernetes_ha_cluster
Failure
3618.69
test_kubernetes_clusters.py

test_08_deploy_and_upgrade_kubernetes_ha_cluster
Failure
0.08
test_kubernetes_clusters.py

test_09_delete_kubernetes_ha_cluster
Failure
0.08
test_kubernetes_clusters.py

ContextSuite context=TestKubernetesCluster>:teardown
Error
72.08
test_kubernetes_clusters.py

test_01_vpc_site2site_vpn_multiple_options
Error
613.62
test_vpc_vpn.py

ContextSuite context=TestVpcSite2SiteVpn>:teardown
Error
199.79
test_vpc_vpn.py

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2349

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2350

@PaulAngus not sure if it was you or a script of yours, but you removed the label bug here. It should be marked that way.

Trillian test result (tid-3139)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 37803 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4289-t3139-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Smoke tests completed. 85 look OK, 1 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_01_vpc_remote_access_vpn
Error
0.01
test_vpc_vpn.py

@blueorangutan package

@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2366

@blueorangutan test

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Trillian test result (tid-3157)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 33120 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4289-t3157-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Smoke tests completed. 85 look OK, 1 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_01_vpc_remote_access_vpn
Error
90.83
test_vpc_vpn.py

@blueorangutan package

@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✖centos7 ✔centos8 ✔debian. JID-2373

@blueorangutan package

@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2377

@blueorangutan test

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@borisstoyanov @vladimirpetrov can you review please?

Trillian test result (tid-3162)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 33390 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4289-t3162-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Smoke tests completed. 86 look OK, 0 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

authors: DaanHoogland blueorangutan


text:
LANG-341: Add number to byte array methods to org.apache.commons.lang3.Conversion

LANG-341: Add number to byte array methods to org.apache.commons.lang3.Conversion

Add toPrimitive(byte[]) and toByteArray(primitive) functions
Add hexToPrimitive(String) and toHex(primitive) functions
Add bitString functions
Add invertBitOrder() functions

Coverage decreased (-0.05%) to 94.239% when pulling d55aa3ff4e2953d372b97023ce75af04ffb8ae1e on yurelle:LANG-341_AddNumToByteArrayMethods into e56ac1c37d236c9a0e4e7b86bd7d1b7201ac177d on apache:master.

Crap. I used a primitive integer constant that's only in java 8. I'll fix.

Coverage decreased (-0.04%) to 94.246% when pulling 41103d1fc59c79d2dc1bbd3871131342b3d1dabd on yurelle:LANG-341_AddNumToByteArrayMethods into e56ac1c37d236c9a0e4e7b86bd7d1b7201ac177d on apache:master.

I got a little carried away; this PR is 4,500 lines. I'm sure this is probably going to take quite a while to validate and get accepted into the code base. I'm just glad I could get the ball rolling. If someone could check on my big/little endian implementations, I'd appreciate it. I can never keep those straight. I went off of these references:
https://www.cs.umd.edu/class/sum2003/cmsc311/Notes/Data/endian.html
https://en.wikipedia.org/wiki/Endianness
https://en.wikipedia.org/wiki/Bit_numbering
I'm not sure why it's saying that test coverage went down. As far as I can tell, I've got tests for all my new functions.
One thing to note: in all my new functions that had a pre-existing alternate implementation, I added a chunk in the unit test to compare my results with the existing implementation. However, I had some trouble getting proper data out of the pre-existing hexToPrimitive() functions. The test code to do this validation is in the JUnit functions, but it's commented out. I'm probably just not understanding the existing implementation properly.

Might be worth splitting this into smaller PRs that can be more easily reviewed?
The LANG-341 stuff was pretty much ready to go last time it was reviewed, so splitting that out would help get it into the code base sooner.
Some of the other methods are new (to me at least), and might warrant individual conversation - do we want/need these methods, etc. (Not saying we don't, but easier to have a coherent discussion about a smaller scope of change).

Avoiding a mixture of white-space and code changes is also welcome :-)

Needs conflicts resolved...

Ok, I fixed the merge conflicts and all the new build style tests that have been added since I first made this PR back in 2016. And the build works for Java 8, but it is dying on the Java 16 build. The errors are complaining about stuff that I didn't touch, so I don't think this is my doing.
Also, back when I wrote all this code, I wasn't allowed to use stuff that was added in Java 8 (specifically the BYTES static property that was added to the Primitive object classes). So, I manually created equivalent integer constants in the Conversion class. However, judging by the versions in the automated builds, it appears that java 8 is now the min supported version. If that is the case, then I can update this code to use the Java 8 BYTES fields, and remove the integer constants. Can you confirm if I'm allowed to use stuff that was introduced in Java 8?
Also, @dmjones mentioned a desire to have the PR broken up into chunks to make review easier. I can do that if you want, but I need to know how small you want those chunks to be. My original commit message split the changes into 4 groups of functions:

primitive to byte[] and back
hex to primitive and back
bit strings
invert bit order

Would these 4 PR's be sufficient? or do you want it more fine grained?
As for the whitespace, Intellij did it automatically; I'm not sure how to undo it.
-Yurelle
P.S. Sorry for leaving this in limbo for so long, but I've had depression for several years, and with a full-time job, I haven't had the energy to do much else. But I've had some time recently, and I'd like to finally get this finished, if yall still want it.

We know the build fails on Java 16, there is a separate thread about that
on the dev ML. It looks like a problem in our data parser or possibly an
issue in the JDK, hard to say, but we do need help figuring it out.
Gary
On Sun, Mar 28, 2021, 18:16 yurelle @.***> wrote:

Ok, I fixed the merge conflicts and all the new build style tests that
have been added since I first made this PR back in 2016. And the build
works for Java 8, but it is dying on the Java 16 build. The errors are
complaining about stuff that I didn't touch, so I don't think this is my
doing.
Also, back when I wrote all this code, I wasn't allowed to use stuff that
was added in Java 8 (specifically the BYTES static property that was added
to the Primitive object classes). So, I manually created equivalent integer
constants in the Conversion class. However, judging by the versions in the
automated builds, it appears that java 8 is now the min supported version.
If that is the case, then I can update this code to use the Java 8 BYTES
fields, and remove the integer constants. Can you confirm if I'm allowed to
use stuff that was introduced in Java 8?
Also, @dmjones https://github.com/dmjones mentioned a desire to have
the PR broken up into chunks to make review easier. I can do that if you
want, but I need to know how small you want those chunks to be. My original
commit message split the changes into 4 groups of functions:

primitive to byte[] and back
hex to primitive and back
bit strings
invert bit order

Would these 4 PR's be sufficient? or do you want it more fine grained?
As for the whitespace, Intellij did it automatically; I'm not sure how to
undo it.
-Yurelle
P.S. Sorry for leaving this in limbo for so long, but I've had depression
for several years, and with a full-time job, I haven't had the energy to do
much else. But I've had some time recently, and I'd like to finally get
this finished, if yall still want it.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/apache/commons-lang/pull/219#issuecomment-808968232,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJB6NZQ7CR5WU62E45545DTF6TETANCNFSM4CXUBPHQ
.

@yurelle
throw new IllegalArgumentException("Input data is longer than size of 'double' primitive ("+SIZE+" bytes)");

tips:
next time when you wanna bake some codes from a template for all primitive types, suggest use boolean, as boolean is the less troublesome one in them...
if cannot use boolean then can use double
do never use int or long as template...that will cause the longer->doubler thing.

I never thought about using boolean; that's a good idea.  Thanks.

authors: XenoAmess coveralls dmjones500 garydgregory yurelle

repo: apache/commons-lang


text:
ci(nightly): correct actions/checkout version tag

Platforms affected
n/a
Motivation and Context

Fix incorrect actions/checkout version tag.
Description

Changed actions/checkout@3 to actions/checkout@v3
Testing

n/a
Checklist

[ ] I've run the tests to see all new and existing tests pass
[ ] I added automated test coverage as appropriate for this change
[ ] Commit is prefixed with (platform) if this change only applies to one platform (e.g. (android))
[ ] If this Pull Request resolves an issue, I linked to the issue in the text above (and used the correct keyword to close issues using keywords)
[ ] I've updated the documentation if necessary

Codecov Report

Merging #304 (ff1f033) into master (76020ea) will decrease coverage by 1.83%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #304      +/-   ##
==========================================
- Coverage   58.73%   56.91%   -1.83%
==========================================
  Files           8        8
  Lines         492      492
==========================================
- Hits          289      280       -9
- Misses        203      212       +9

Impacted Files
Coverage Δ

src/repoutil.js
68.91% <ø> (ø)

... and 1 file with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

authors: codecov-commenter erisu

repo: apache/cordova-coho


text:
Add HealthPlanet, Ohalog and Undog Apps to Showcase.

This pull request adds three apps to the showcase. If you like, I can split it into three PRs.

Hi @argelius. Thanks for the PR! We're currently in the middle of a major merge in the repo, and I will revisit this PR next week. Please stay tuned!

authors: argelius dblotsky

repo: apache/cordova-docs


text:
CB-10730 Move plugin handlers tests into platform's repo

JIRA: https://issues.apache.org/jira/browse/CB-10730

Is there also a PR for removing these from cordova-lib?

Is there also a PR for removing these from cordova-lib?

No, because the code these tests are testing is still in cordova-lib. I think we could remove the tests along with the code as a part of PlatformApi polyfill deprecation (see CB-10644 and corresponding mailing list discussion )

authors: dblotsky vladimir-kotikov

repo: apache/cordova-windows


text:
Use weatherreport_check_nodes_connected:check/1

Overview
The arity 0 version does not exist.
Fixes dialyzer warning:
{warn_callgraph,{"src/weatherreport_node.erl",126},
                {call_to_missing,[weatherreport_check_nodes_connected,check,
                                  0]}}

Testing recommendations
make check

Related Issues or Pull Requests

Checklist

[x] Code is written and works correctly
[ ] Changes are covered by tests
[ ] Any new configurable parameters are documented in rel/overlay/etc/default.ini
[ ] A PR for documentation changes has been made in https://github.com/apache/couchdb-documentation

+1
Was wondering if we could wait until after we tag 3.2.1 to merge? Not big deal if we don't but in case we get a last minute -1, it would be nice to know we could still avoid branching.

Definitely will wait.

3.2.1 was tagged. Thank you for waiting

authors: jaydoane nickva

repo: apache/couchdb


text:
RAT-322: be able to scan hidden directories

This is an alternative to #160

This looks good to me, but I have not built and tested it.  I trust that the build works and there seems to be a test so I approve this change.

@Claudenw I'm checking the build failure on Windows (Jenkins). I tested locally without problem. I'm doing a new pass and I close the "old" PR on the same topic.

OK, I understand: on Windows, the HiddenFileFilter doesn't actually consider file hidden, so the tests are failing on Windows. I'm checking this part.

Extracted from java.io.File#isHidden() javadoc:
Tests whether the file named by this abstract pathname is a hidden file. The exact definition of hidden is system-dependent. On UNIX systems, a file is considered to be hidden if its name begins with a period character ('.'). On Microsoft Windows systems, a file is considered to be hidden if it has been marked as such in the filesystem.

So, for the tests, I have to bypass if on Windows. I'm doing that.

In order to be backward compatible (especially to work as before on Windows), I created NameBasedHiddenFileFilter that only check if file name contains .  (and not the hidden attribute on Windows).
User can still define its own filter if needed.

Thanks for the contribution @jbonofre
I'm still curious to get to know more details about why you want to scan in hidden directories? Can you provide more info about that.

@Claudenw could we just merge this change or would this introduce problems with other/existing PRs? Thanks

@jbonofre feel free to add a changelog entry (or I can do this upon merging your changes; sorry for my many questions in the code).

Thanks for the contribution @jbonofre
I'm still curious to get to know more details about why you want to scan in hidden directories? Can you provide more info about that.

@ottlinger at ASF, we should have license header in all files included in project release source distributions (which is the minimum a project has to provide for each release). Some projects include hidden directories/iles in their source distributions (for instance, Apache Iceberg includes hidden directories like .palantir, .baseline, etc). Those directories/files can't be excluded else it means that it would not be possible to build the project from the source distribution.
To be "clean", these directories/files have to include ASF/license headers and so they should be considered by RAT.
As workaround, RAT gradle|maven plugins use a "custom" walker to scan hidden directories/files. So I think it makes sense to improve our default walker to support this behavior and so having the same support in CLI.
That's basically the background :)

I did rebase and added an entry in the changes.
As suggested by @ottlinger I'm updating the tests to use assertj.

I started to use assertj in ReportConfigurationTest.
I'm fixing the build.
@ottlinger do you want me to use assertj everywhere ? Maybe it makes more sense to have another PR dedicated to that. Thought ?

I pushed the update to define assertj in dependenciesManagement and define where it's required.
@ottlinger please let me know if you want me to refactore all tests using assertj in this PR or in separated one. Thanks !

@jbonofre go ahead to use assertj more often in a different PR. Thanks for your contribution.

@ottlinger I propose to merge this PR (as it introduces assertj-core dependency). I will create another PR to use assertj everywhere.

authors: Claudenw jbonofre ottlinger

repo: apache/creadur-rat


text:
fix: Remove skip.surefire.tests mvn property

Which issue does this PR close?
Follow-up to #404
Closes #.
Rationale for this change
The skip.surefire.tests property introduced in #404 causes -DskipTests to not work for maven-surefire-plugin.
To skip only java tests we just add -Dtest=none.
What changes are included in this PR?

Remove skip.surefire.tests and set failIfNoSpecifiedTests to false for maven-surefire-plugin.
Update doc

How are these changes tested?

test  ./mvnw install -DskipTests -pl common
before this pr (-DskipTests does not take effect):
[INFO] --- surefire:3.1.0:test (default-test) @ comet-common-spark3.4_2.12 ---
[INFO] Using auto detected provider org.apache.maven.surefire.junit4.JUnit4Provider
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.comet.parquet.TestColumnReader
24/07/31 10:04:11 INFO core/src/lib.rs: Comet native library initialized
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.298 s - in org.apache.comet.parquet.TestColumnReader
[INFO] Running org.apache.comet.parquet.TestFileReader
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.347 s - in org.apache.comet.parquet.TestFileReader
[INFO] Running org.apache.comet.parquet.TestCometInputFile
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in org.apache.comet.parquet.TestCometInputFile
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0

after this pr (-DskipTests takes effect):
[INFO] --- surefire:3.1.0:test (default-test) @ comet-common-spark3.4_2.12 ---
[INFO] Tests are skipped.

test ./mvnw test -Dtest=none -Dsuites="org.apache.comet.CometCastSuite valid"
after this pr (works well and only executes the specified suites):
[INFO] --- surefire:3.1.0:test (default-test) @ comet-common-spark3.4_2.12 ---
[INFO]
[INFO] --- jacoco:0.8.11:report (report) @ comet-common-spark3.4_2.12 ---
......
[INFO] --- scalatest:2.0.2:test (test) @ comet-spark-spark3.4_2.12 ---
Run starting. Expected test count is: 2
CometCastSuite:
- all valid cast combinations covered (127 milliseconds)
24/07/31 10:12:19 INFO core/src/lib.rs: Comet native library initialized
- cast BinaryType to StringType - valid UTF-8 inputs (12 seconds, 452 milliseconds)
Run completed in 17 seconds, 535 milliseconds.
Total number of tests run: 2
Suites: completed 1, aborted 0
Tests: succeeded 2, failed 0, canceled 0, ignored 0, pending 0
All tests passed.

cc @edmondop

I am not sure what's the problem, can you see the original comment from @andygrove on the merged Pr?

I am not sure what's the problem, can you see the original comment from @andygrove on the merged Pr?

I updated How are these changes tested? to show how to reproduce this issue.

I was also very confused why -DskipTests wasn't skipping the tests

Thanks @wForget

authors: Kimahriman edmondop viirya wForget

repo: apache/datafusion-comet


text:
perf: Remove redundant copying of batches after FilterExec

Which issue does this PR close?

Closes https://github.com/apache/datafusion-comet/issues/757
Rationale for this change

We were often creating a filtered batch in FilterExec and then making a copy of that batch in CopyExec, resulting in redundant copying of data in the case where FilterExec had already created a new batch with the filtered data. It was necessary to have the CopyExec because FilterExec can pass through input batches in the case where the predicate evaluates to true for all rows and this is not safe in Comet because we re-use arrays in the ScanExec.
This PR introduces a customized version of FilterExec that uses Arrow's take_record_batch kernel to always create new batches, even in the case where the predicate evaluates to true for all rows. This removes the need to wrap FilterExec in a CopyExec.
This reduces our TPC-DS time by ~25 seconds.

The majority of queries are faster with this change:

What changes are included in this PR?

Copy DataFusion's FilterExec into Comet and make a one line change so that FilterExec never passes through input batches -- I will suggest some changes upstream so that we can customize rather than duplicate
Stop wrapping FilterExec in CopyExec

How are these changes tested?

Existing tests
I ran TPC-DS without error

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 33.95%. Comparing base (380f03d) to head (9458cfe).
Report is 1 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main     #835      +/-   ##
============================================
+ Coverage     33.83%   33.95%   +0.12%
- Complexity      870      880      +10
============================================
  Files           112      112
  Lines         42970    42993      +23
  Branches       9466     9473       +7
============================================
+ Hits          14538    14598      +60
+ Misses        25446    25404      -42
- Partials       2986     2991       +5

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@mbutrovich you may be interested in reviewing this PR

Thanks @Dandandan @viirya for the feedback so far. I have now added criterion benchmarks before I start addressing the feedback. Here are the current results.
filter/arrow_filter_record_batch - few
                        time:   [15.546 µs 15.601 µs 15.682 µs]
Found 9 outliers among 100 measurements (9.00%)
  3 (3.00%) high mild
  6 (6.00%) high severe
filter/arrow_filter_record_batch - many
                        time:   [74.935 µs 75.129 µs 75.324 µs]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild
filter/arrow_filter_record_batch - all
                        time:   [396.57 ns 397.80 ns 399.05 ns]
filter/comet_filter - few
                        time:   [21.772 µs 22.012 µs 22.337 µs]
Found 14 outliers among 100 measurements (14.00%)
  2 (2.00%) high mild
  12 (12.00%) high severe
Benchmarking filter/comet_filter - many: Warming up for 500.00 ms
Warning: Unable to complete 100 samples in 500.0ms. You may wish to increase target time to 762.3ms, enable flat sampling, or reduce sample count to 50.
filter/comet_filter - many
                        time:   [146.44 µs 147.25 µs 148.01 µs]
Found 12 outliers among 100 measurements (12.00%)
  12 (12.00%) low mild
Benchmarking filter/comet_filter - all: Warming up for 500.00 ms
Warning: Unable to complete 100 samples in 500.0ms. You may wish to increase target time to 895.1ms, enable flat sampling, or reduce sample count to 50.
filter/comet_filter - all
                        time:   [177.25 µs 183.04 µs 190.27 µs]
Found 13 outliers among 100 measurements (13.00%)
  2 (2.00%) high mild
  11 (11.00%) high severe

After addressing the first round of feedback, we now have:
pub fn comet_filter_record_batch(
    record_batch: &RecordBatch,
    predicate: &BooleanArray,
) -> std::result::Result<RecordBatch, ArrowError> {
    if predicate.true_count() == record_batch.num_rows() {
        // special case where we just make an exact copy
        let arrays: Vec<ArrayRef> = record_batch
            .columns()
            .iter()
            .map(|array| {
                let capacity = array.len();
                let data = array.to_data();
                let mut mutable = MutableArrayData::new(vec![&data], false, capacity);
                mutable.extend(0, 0, capacity);
                make_array(mutable.freeze())
            })
            .collect();
        let options = RecordBatchOptions::new().with_row_count(Some(record_batch.num_rows()));
        RecordBatch::try_new_with_options(record_batch.schema().clone(), arrays, &options)
    } else {
        filter_record_batch(record_batch, predicate)
    }
}

New benchmark results:
filter/comet_filter - few
                        time:   [14.650 µs 14.727 µs 14.831 µs]
                        change: [-36.702% -35.128% -33.681%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe
filter/comet_filter - many
                        time:   [75.962 µs 76.172 µs 76.381 µs]
                        change: [-48.681% -48.501% -48.303%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe
filter/comet_filter - all
                        time:   [34.497 µs 34.628 µs 34.764 µs]
                        change: [-80.854% -80.527% -80.256%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  6 (6.00%) low mild
  1 (1.00%) high mild
  1 (1.00%) high severe

This certainly looks a lot better. I am running TPC-DS again to make sure this really is always copying. I had tried an approach like this in the past but ran into data corruption issues.

The results are even better now :tada:

I created a PR against DataFusion to add a flag to FilterExec so that we can switch back to using the upstream version, assumign this PR gets accepted.

Thanks for the reviews @viirya @Dandandan @mbutrovich

authors: andygrove codecov-commenter


text:
feat: Implement ANSI support for Round

Which issue does this PR close?
Closes #466 .
Rationale for this change
Improves compatibility with spark
What changes are included in this PR?
ANSI support for Drop by adding input checks when ANSI mode enabled.
How are these changes tested?
val df = Seq(Int.MaxValue, Int.MinValue).toDF("a")
df.write.parquet("/tmp/int.parquet")
spark.read.parquet("/tmp/int.parquet").createTempView("t")

scala> spark.conf.set("spark.comet.ansi.enabled", true)

scala> spark.conf.set("spark.sql.ansi.enabled", true)

scala> spark.conf.set("spark.comet.enabled", true)

scala> spark.sql("select a, round(a,-1) from t").show

Ensure the above raises:
org.apache.spark.SparkArithmeticException: [ARITHMETIC_OVERFLOW] Overflow. If necessary set "spark.sql.ansi.enabled" to "false" to bypass this error.

I've just created a test that reproduces the error so far. I plan to keep working on this but I am having problems understanding the codebase and where the Round implementation is coming from. This is my first PR to the project so I am taking this as a learning exercise:
I've been investigating both:

https://github.com/apache/datafusion-comet/pull/471
https://github.com/apache/datafusion-comet/pull/500

And from my understanding I probably have to create a new native/core/src/execution/datafusion/expressions/round.rs to modify:
https://github.com/apache/datafusion-comet/blob/main/spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scala#L1892-L1895
To something like:
diff --git a/spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scala b/spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scala
index 51b32b7..890703f 100644
--- a/spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scala
+++ b/spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scala
@@ -1889,7 +1889,7 @@ object QueryPlanSerde extends Logging with ShimQueryPlanSerde with CometExprShim
           val optExpr = scalarExprToProto("pow", leftExpr, rightExpr)
           optExprWithInfo(optExpr, expr, left, right)

-        case r: Round =>
+        case r: Round(failOnError) =>

And from my understanding I might also have to add the Drop definition to core/src/execution/proto/expr.proto.
I might be quite lost around here so I would appreciate if the above sounds right or I am completely lost here.

Thanks for picking this up @raulcd. I'll be happy to help you navigate this.
So Spark defines Round as:
case class Round(
    child: Expression,
    scale: Expression,
    override val ansiEnabled: Boolean = SQLConf.get.ansiEnabled)

in QueryPlanSerde, we have case r: Round, which is fine, but we currently ignore the value in r.ansiEnabled.
When serializing to protobuf, we use the generic ScalarFunction:
message ScalarFunc {
  string func = 1;
  repeated Expr args = 2;
  DataType return_type = 3;
}

Perhaps we should add an extra field here for fail_on_error?

Perhaps we should add an extra field here for fail_on_error?

That's interesting, probably some of the other ScalarFunc  might need to send the fail_on_error flag too in order to match ANSI mode so that seems like a good idea to me.
I've tried to do it on this commit: https://github.com/apache/datafusion-comet/pull/989/commits/6d0b46c88452a90a380b6f78da6fa2f0045b17ef
As per where to really check for the Overflow Error I guess we could do something like the stab I took on the following commit: https://github.com/apache/datafusion-comet/pull/989/commits/113ec6d8ffb38e3599f6bab03050f135f4c691cf
I'll have to check how to fix this error for Spark 3.3 too:
Error:  /__w/datafusion-comet/datafusion-comet/spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scala:1930: value ansiEnabled is not a member of org.apache.spark.sql.catalyst.expressions.Round

authors: andygrove raulcd


text:
Datafusion fails on simple DISTINCT ON query

Describe the bug
Simple SELECT DISTINCT ON query triggers failure in datafusion Internal error: Failed due to a difference in schemas
To Reproduce
Open datafusion-cli and run the commands to get the error:
> create view test as values (1,1),(1,2),(2,0),(3,1),(3,2);
0 row(s) fetched.
Elapsed 0.001 seconds.

> select * from test;
+---------+---------+
| column1 | column2 |
+---------+---------+
| 1       | 1       |
| 1       | 2       |
| 2       | 0       |
| 3       | 1       |
| 3       | 2       |
+---------+---------+
5 row(s) fetched.
Elapsed 0.001 seconds.

> select distinct on (column1) * from test;
Optimizer rule 'replace_distinct_aggregate' failed
caused by
replace_distinct_aggregate
caused by
Internal error: Failed due to a difference in schemas, original schema: DFSchema { inner: Schema { fields: [Field { name: "column1", data_type: Int64, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }, Field { name: "column2", data_type: Int64, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }], metadata: {} }, field_qualifiers: [Some(Bare { table: "test" }), Some(Bare { table: "test" })], functional_dependencies: FunctionalDependencies { deps: [] } }, new schema: DFSchema { inner: Schema { fields: [Field { name: "column1", data_type: Null, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }], metadata: {} }, field_qualifiers: [Some(Bare { table: "test" })], functional_dependencies: FunctionalDependencies { deps: [] } }.
This was likely caused by a bug in DataFusion's code and we would welcome that you file an bug report in our issue tracker

Expected behavior
proper result should be returned
Additional context
Seen in
$ datafusion-cli -V
datafusion-cli 42.0.0

Thanks for reporting. I can reproduce the issue, gonna try to fix it!

I couldn't reproduce the issue in DataFusion CLI v42.1.0. Perhaps you could try testing it again? Let me know if you still encounter the issue. Thank you! @qrilka

It isn't even on crates.io - is that expected? Will try installing from sources later

I tested it on the latest main branch, and everything is working fine. The latest PR was merged last month showed in git blame. I think it's really weird; perhaps the error message pointed to the wrong cause.

Yeah, installed from main and looks to be fine, thanks!

source: gharchive/issue

repo: apache/datafusion


text:
[Bug] [dolphinscheduler-common]  JSONUtils tool class time zone problem reporting error

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened
report error  in mvn test，but not occur error when use Junit to run JSONUtilsTets.java file。
    @Test
    public void dateToString() {
        TimeZone.setDefault(TimeZone.getTimeZone("Asia/Shanghai"));
        String time = "2022-02-22 13:38:24";
        Date date = DateUtils.stringToDate(time);
        String json = JSONUtils.toJsonString(date);
        Assert.assertEquals(json, "\"" + time + "\"");

        String errorFormatTime = "Tue Feb 22 03:50:00 UTC 2022";
        Assert.assertNull(DateUtils.stringToDate(errorFormatTime));
    }

    @Test
    public void stringToDate() {
        TimeZone.setDefault(TimeZone.getTimeZone("Asia/Shanghai"));
        String json = "\"2022-02-22 13:38:24\"";
        Date date = JSONUtils.parseObject(json, Date.class);
        Assert.assertEquals(date, DateUtils.stringToDate("2022-02-22 13:38:24"));
    }

The dateToString and stringToDate methods get the following errors when running mvn test
[ERROR] Tests run: 19, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 0.118 s <<< FAILURE! - in org.apache.dolphinscheduler.common.utils.JSONUtilsTest
Tests run: 19, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 0.118 s <<< FAILURE! - in org.apache.dolphinscheduler.common.utils.JSONUtilsTest
stringToDate(org.apache.dolphinscheduler.common.utils.JSONUtilsTest)  Time elapsed: 0.014 s  <<< FAILURE!

java.lang.AssertionError: expected:<Tue Feb 22 21:38:24 CST 2022> but was:<Tue Feb 22 13:38:24 CST 2022>
	at org.apache.dolphinscheduler.common.utils.JSONUtilsTest.stringToDate(JSONUtilsTest.java:280)
[ERROR] dateToString(org.apache.dolphinscheduler.common.utils.JSONUtilsTest)  Time elapsed: 0.003 s  <<< FAILURE!
dateToString(org.apache.dolphinscheduler.common.utils.JSONUtilsTest)  Time elapsed: 0.003 s  <<< FAILURE!

org.junit.ComparisonFailure: expected:<"2022-02-22 [05]:38:24"> but was:<"2022-02-22 [13]:38:24">
	at org.apache.dolphinscheduler.common.utils.JSONUtilsTest.dateToString(JSONUtilsTest.java:269)

What you expected to happen
use case passed
How to reproduce
OS Version : Any
JDK Versin : Oracle JDK11
Run mvn test -e  in the terminal or press the button in IDEA
Anything else
I wrote a demo to show how to reproduce it easily without waiting a long time for mvn test to run. I will analyze why this problem occurs and submit a pull request.
import org.apache.dolphinscheduler.common.utils.JSONUtils;
import org.apache.dolphinscheduler.common.utils.JSONUtilsTest;
import java.util.TimeZone;
public class TimeZoneTest {
    public static void main(String[] args) {
        TimeZone.setDefault(TimeZone.getTimeZone("UTC"));
        JSONUtils.createArrayNode();// This step is to load class and initialize JSONUtils's static code block
        new JSONUtilsTest().stringToDate();
        new JSONUtilsTest().dateToString();
    }
}

The above one gives my demo and some source code screenshots. The reason for the bug is among them:

when using junit to execute use cases, each use case is isolated, and the JSONUtils class will not be shared among multiple use cases. In this case, TimeZone.setDefault(TimeZone.getTimeZone("Asia/Shanghai")) will not fail.Because the JSONUtils class is reinitialized once per use case

In the case of using mvn test, which is described in my demo, when the default time zone of the TimeZone class is set in other places through TimeZone.setDefault, the initialization of JSONUtils is carried out. At this time, JSONUtils final and static descriptors are modified parts The default time zone information in TimeZone will be loaded, and setting the TimeZone time zone via TimeZone.setDefault(TimeZone.getTimeZone("Asia/Shanghai")) in the future will not have any effect on JSONUtils.

上面个给出了我的demo和部分源码截图，bug出现的原因就在其中：

当使用junit执行用例时，每个用例都是隔离的，此时多个用例之间不会共享JSONUtils类，这种情况下就不会出现TimeZone.setDefault(TimeZone.getTimeZone("Asia/Shanghai"))失效。因为每个用例都会重新初始化一次JSONUtils类
而在使用mvn test的情况中，也就是我demo中描述的那样，当在其他地方通过TimeZone.setDefault设置过TimeZone类默认时区后再进行JSONUtils的初始化，此时JSONUtils final 和static描述符修饰的部分就会加载TimeZone中默认的时区信息，而在以后再通过TimeZone.setDefault(TimeZone.getTimeZone("Asia/Shanghai"))设置TimeZone时区并不会对JSONUtils产生任何影响。

Version
dev
Are you willing to submit PR?

[X] Yes I am willing to submit a PR!
#10284

Code of Conduct

[X] I agree to follow this project's Code of Conduct

你好，来件已收到，我会及时回复！

repo: apache/dolphinscheduler


text:
[Bug] [dolphinscheduler-api] Upload file  failed  because the file over 50M

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened
我在测试3.0.0-beta-2这个版本的时候，上传jar包，超过50M报错，但配置上传大小100M;
When I was testing version 3.0.0-beta-2, I uploaded the jar package and reported an error when it exceeded 50m, but the upload size was configured to be 100m
What you expected to happen
It should upload normally and successfully
How to reproduce
使用3.0.0-beta2版本，正常部署成功后，通过资源中心上传超过50M的文件
Using version 3.0.0-beta2, upload files over 50m through the resource center after normal deployment
Anything else
No response
Version
3.0.0-beta-2
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Thank you for your feedback, we have received your issue, Please wait patiently for a reply.

In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
If you haven't received a reply for a long time, you can join our slack and send your question to channel #troubleshooting

version :3.0.0-beta2
Upload jar file failed  when it exceeded 50M.  See the above picture for the error log

Close this issue, since duplicated with 10340

Hi @taoshengyijiua, Please close this issue, since duplicated with 10340

close due to this

authors: taoshengyijiua yangjf2019 zhongjiajie


text:
[Feature][Task] K8s namespace auth manager

Purpose of the pull request
Namespaces are managed with permissions that are created by the administrator and authorized to be used by ordinary users
Brief change log

Add unauth-namespace,authed-namespace,available-list to K8sNamespaceController
Add K8sNamespaceUserMapper to dao
Update some code to k8s api,service

Verify this pull request

This change added tests and can be verified as follows:

Added some tests to K8sNamespaceControllerTest.
Added some tests to UsersControllerTest.
Added some tests to K8SNamespaceServiceTest.

close #9252

Codecov Report

Merging #9303 (19bba99) into dev (9d68b17) will decrease coverage by 0.25%.
The diff coverage is 27.92%.

:exclamation: Current head 19bba99 differs from pull request most recent head a7945ea. Consider uploading reports for the commit a7945ea to get more accurate results

@@             Coverage Diff              @@
##                dev    #9303      +/-   ##
============================================
- Coverage     40.25%   39.99%   -0.26%
+ Complexity     4390     4389       -1
============================================
  Files           820      823       +3
  Lines         32834    33027     +193
  Branches       3650     3673      +23
============================================
- Hits          13216    13208       -8
- Misses        18382    18583     +201
  Partials       1236     1236

Impacted Files
Coverage Δ

...lphinscheduler/api/controller/UsersController.java
59.25% <0.00%> (-2.28%)
:arrow_down:

...ache/dolphinscheduler/dao/entity/K8sNamespace.java
0.00% <0.00%> (ø)

.../dolphinscheduler/dao/entity/K8sNamespaceUser.java
0.00% <0.00%> (ø)

...rver/master/processor/queue/TaskExecuteThread.java
0.00% <0.00%> (ø)

.../master/processor/queue/TaskExecuteThreadPool.java
2.17% <2.17%> (ø)

...heduler/api/controller/K8sNamespaceController.java
70.00% <63.63%> (-1.43%)
:arrow_down:

...erver/master/processor/queue/TaskEventService.java
59.18% <66.66%> (-8.71%)
:arrow_down:

...uler/api/service/impl/K8SNamespaceServiceImpl.java
73.91% <71.69%> (ø)

...inscheduler/api/service/impl/UsersServiceImpl.java
80.81% <86.36%> (+0.27%)
:arrow_up:

.../org/apache/dolphinscheduler/api/enums/Status.java
100.00% <100.00%> (ø)

... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9d68b17...a7945ea. Read the comment docs.

@songjianet please take a look of UI part, thanks.

BTW, @qianli2022 please update the doc too, DS had put the doc part into code repository, so it should update the code and doc  synchronously.

There are two problems here:

We will not perform non-bug functional maintenance on the old ui, we will gradually eliminate the old ui
For the new UI changes, you added an interface method, but it is not actually used, I don't know why

PTAL again @songjianet the next-ui had added.

The changes of the old ui can be reverted, because we will not do any non-bug processing on the old ui.

authors: caishunfeng codecov-commenter qianli2022 songjianet


text:
[Bug] compile error

Search before asking

[X] I had searched in the issues and found no similar issues.

Version
commit ce7fa5d6d9454d3b89e75859690fb216ca68167b (HEAD -> master, origin/master, origin/HEAD)
Author: renzhimin7 1240388654@qq.com
Date:   Mon Nov 22 14:47:00 2021 +0800
[typo] Update multi-tenant.md (#7162)

A double quote is missing

What's Wrong?
compilation using docker image failed.
docker image: apache/incubator-doris:build-env-1.4.1
sh build.sh --clean --be --fe --ui
we got error:
ninja: error: '/var/local/thirdparty/installed/lib64/libbreakpad_client.a', needed by 'src/service/palo_be', missing and no known rule to make it

What You Expected?
compilation success
How to Reproduce?
docker pull apache/incubator-doris:build-env-1.4.1
git clone git@github.com:minghong/talent-plan.git
docker run -it -v ~/.m2:/root/.m2 -v ~/work/incubator-doris/:/root/incubator-doris-DORIS-x.x.x-release/ apache/incubator-doris:build-env-1.4.1
inside docker container:
[root@7eec959b89ab incubator-doris-DORIS-x.x.x-release]# sh build.sh --clean --be --fe --ui
We got following error message
-- The ASM compiler identification is GNU
-- Found assembler: /opt/rh/devtoolset-10/root/usr/bin/gcc
-- Looking for sys/random.h
-- Looking for sys/random.h - not found
-- Some symbols from glibc will be replaced for compatibility
-- Configuring done
-- Generating done
-- Build files have been written to: /root/incubator-doris-DORIS-x.x.x-release/be/build_Release
[0/2] Re-checking globbed directories...
ninja: error: '/var/local/thirdparty/installed/lib64/libbreakpad_client.a', needed by 'src/service/palo_be', missing and no known rule to make it
Anything Else?
No response
Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Remove thirdparty/installed/ and create a empty dir thirdparty/installed/.

It may caused by image.
apache/incubator-doris:***  -> apache/doris:***

If you compile by docker, It may caused by image. apache/incubator-doris:*** -> apache/doris:***

Which image can I use to build, apache/doris:build-env-for-1.0.0 or other tags?

authors: englefly jackwener m1iktea

repo: apache/doris


text:
[Pipeline] Fix PipScannerContext::can_finish return wrong status

Proposed changes
Issue Number: close #xxx
Problem summary
Now in ScannerContext::push_back_scanner_and_reschedule, _num_running_scanners-- is before _num_scheduling_ctx++.
InPipScannerContext::can_finish, we check _num_running_scanners == 0 && _num_scheduling_ctx == 0 without obtaining _transfer_lock.
In follow case, PipScannerContext::can_finish will return wrong result.

_num_running_scanners--
Check _num_running_scanners == 0 && _num_scheduling_ctx == 0` return true.
_num_scheduling_ctx++

So, we can set _num_running_scanners-- after _num_scheduling_ctx++.
Describe your changes.
Checklist(Required)

Does it affect the original behavior:

[ ] Yes
[ ] No
[ ] I don't know

Has unit tests been added:

[ ] Yes
[ ] No
[ ] No Need

Has document been added or modified:

[ ] Yes
[ ] No
[ ] No Need

Does it need to update dependencies:

[ ] Yes
[ ] No

Are there any changes that cannot be rolled back:

[ ] Yes (If Yes, please explain WHY)
[ ] No

Further comments
If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 34.83 seconds
load time: 634 seconds
storage size: 17122433872 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221221170643_clickbench_pr_66606.html


text:
PipelineX Support data gen operator

Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall


text:
fix should do read lock on table being insert when analyze

Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

run buildall

run buildall


text:
fix Set the flag of short circuit only when hash join sink finished (#30977)

Proposed changes
pick #30977

Further comments
If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall


text:
opt For a mark join without other conjuncts, stop probing after matching one row

Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall


text:
opt Modify thedefault JEMALLOC_CONF and support flush Jemalloc tcache

Modify thedefault JEMALLOC_CONF, reduce muzzy_decay_ms and dirty_decay_ms, and modify the parameters of the regular dump heap profile for better understanding.
support flush Jemalloc tcache, when process memory exceed limit and tcache is larger than 1G, the limit of tcache is the number of pages, not the total number of page bytes.

run buildall

run buildall

run buildall

run buildall


text:
feature add system table catalog_meta_cache_statistics

Add new system table catalog_meta_cache_statistics
mysql> select * from information_schema.catalog_meta_cache_statistics;
+--------------+-----------------------------+----------------------+---------------------+
| CATALOG_NAME | CACHE_NAME                  | METRIC_NAME          | METRIC_VALUE        |
+--------------+-----------------------------+----------------------+---------------------+
| hms          | hive_file_cache             | eviction_count       | 0                   |
| hms          | hive_file_cache             | hit_ratio            | 0.6                 |
| hms          | hive_file_cache             | average_load_penalty | 7.60805755E7        |
| hms          | hive_file_cache             | estimated_size       | 2                   |
| hms          | hive_file_cache             | hit_count            | 3                   |
| hms          | hive_file_cache             | read_count           | 5                   |
| hms          | hive_partition_cache        | eviction_count       | 0                   |
| hms          | hive_partition_cache        | hit_ratio            | 1.0                 |
| hms          | hive_partition_cache        | average_load_penalty | 0.0                 |
| hms          | hive_partition_cache        | estimated_size       | 0                   |
| hms          | hive_partition_cache        | hit_count            | 0                   |
| hms          | hive_partition_cache        | read_count           | 0                   |
| hms          | hive_partition_values_cache | eviction_count       | 0                   |
| hms          | hive_partition_values_cache | hit_ratio            | 1.0                 |
| hms          | hive_partition_values_cache | average_load_penalty | 0.0                 |
| hms          | hive_partition_values_cache | estimated_size       | 0                   |
| hms          | hive_partition_values_cache | hit_count            | 0                   |
| hms          | hive_partition_values_cache | read_count           | 0                   |
| hms          | hudi_partition_cache        | eviction_count       | 0                   |
| hms          | hudi_partition_cache        | hit_ratio            | 1.0                 |
| hms          | hudi_partition_cache        | average_load_penalty | 0.0                 |
| hms          | hudi_partition_cache        | estimated_size       | 0                   |
| hms          | hudi_partition_cache        | hit_count            | 0                   |
| hms          | hudi_partition_cache        | read_count           | 0                   |
| iceberg      | iceberg_table_cache         | eviction_count       | 0                   |
| iceberg      | iceberg_table_cache         | hit_ratio            | 0.07407407407407407 |
| iceberg      | iceberg_table_cache         | average_load_penalty | 1.7263263424E8      |
| iceberg      | iceberg_table_cache         | estimated_size       | 25                  |
| iceberg      | iceberg_table_cache         | hit_count            | 2                   |
| iceberg      | iceberg_table_cache         | read_count           | 27                  |
| iceberg      | iceberg_snapshot_cache      | eviction_count       | 0                   |
| iceberg      | iceberg_snapshot_cache      | hit_ratio            | 1.0                 |
| iceberg      | iceberg_snapshot_cache      | average_load_penalty | 0.0                 |
| iceberg      | iceberg_snapshot_cache      | estimated_size       | 0                   |
| iceberg      | iceberg_snapshot_cache      | hit_count            | 0                   |
| iceberg      | iceberg_snapshot_cache      | read_count           | 0                   |
+--------------+-----------------------------+----------------------+---------------------+

run buildall

run buildall

run buildall


text:
enhancement support BE http action: hash_path and clear

Proposed changes
Add a http action which is useful when you debug.
API
GET /api/file_cache

request parameter
request parameter1

param
type
desc
require

op
string
the value must be list_cache, other value you can refer to #40831 #37484
yes

value
string
the segment file name
yes

request parameter2

param
type
desc
require

op
string
the value must be clear, other value you can refer to #40831 #37484
yes

value
string
the segment file name
yes

sync
bool
clean local cache in sync
no

response
response1
if success

param
type
desc

array
return the segment file cache in local path

if fail

param
type
desc

array
empty array

response2
if success

param
type
desc

status
string

msg
string

example
case 1
curl  '172.100.0.4:8040/api/file_cache?op=list_cache&value=0200000000000001bf42c14374fff491ffb7c89a1a65c5bb_0.dat'

return
["/opt/doris/be/file_cache/c6a/c6a599f453f67f0949f80ad9990fa3dd/0"]

case 2
curl '127.0.0.1:8040/api/file_cache?op=clear&sync=true&value=0200000000000001284b68fea3dcfe8a83e65cd88426b081_0.dat'

return
{
    "status": "OK",
    "msg": "OK"
}

run buidlall

run buildall

list what has been cleared, a file list with size

authors: gavinchou yagagagaga