text:
Fixed client processing time

Description
Fixed client processing time
Issues Resolved
Fixed an issue raised with client processing time addition.
https://github.com/opensearch-project/opensearch-benchmark/pull/450#issuecomment-1936836969
Testing
[Describe how this change was tested]
I tested it in most of the cases where OpenSearch call fails.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@IanHoang, @gkamat Please take a look. This PR fixes https://github.com/opensearch-project/opensearch-benchmark/pull/450#issuecomment-1936836969

sample benchmark result when there is an error in force-merge
 opensearch-benchmark % opensearch-benchmark execute-test --distribution-version=2.11.1 --workload=geonames --test-mode --kill-running-processes

   ____                  _____                      __       ____                  __                         __
  / __ \____  ___  ____ / ___/___  ____ ___________/ /_     / __ )___  ____  _____/ /_  ____ ___  ____ ______/ /__
 / / / / __ \/ _ \/ __ \\__ \/ _ \/ __ `/ ___/ ___/ __ \   / __  / _ \/ __ \/ ___/ __ \/ __ `__ \/ __ `/ ___/ //_/
/ /_/ / /_/ /  __/ / / /__/ /  __/ /_/ / /  / /__/ / / /  / /_/ /  __/ / / / /__/ / / / / / / / / /_/ / /  / ,<
\____/ .___/\___/_/ /_/____/\___/\__,_/_/   \___/_/ /_/  /_____/\___/_/ /_/\___/_/ /_/_/ /_/ /_/\__,_/_/  /_/|_|
    /_/

[INFO] Preparing for test execution ...
[INFO] Executing test with workload [geonames], test_procedure [append-no-conflicts] and provision_config_instance ['defaults'] with version [2.11.1].

[WARNING] merges_total_time is 37 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
[WARNING] indexing_total_time is 1631 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
[WARNING] refresh_total_time is 2419 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
[WARNING] flush_total_time is 1190 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
Running delete-index                                                           [100% done]
Running create-index                                                           [100% done]
Running check-cluster-health                                                   [100% done]
Running index-append                                                           [100% done]
Running refresh-after-index                                                    [100% done]
[ERROR] no handler found for uri [/geonames/_forcemerges] and method [POST] ({'error': 'no handler found for uri [/geonames/_forcemerges] and method [POST]'})
Running force-merge                                                            [100% done]
Running refresh-after-force-merge                                              [100% done]
Running wait-until-merges-finish                                               [100% done]
Running index-stats                                                            [100% done]
Running node-stats                                                             [100% done]
Running default                                                                [100% done]
Running term                                                                   [100% done]
Running phrase                                                                 [100% done]
Running country_agg_uncached                                                   [100% done]
Running country_agg_cached                                                     [100% done]
Running scroll                                                                 [100% done]
Running expression                                                             [100% done]
Running painless_static                                                        [100% done]
Running painless_dynamic                                                       [100% done]
Running decay_geo_gauss_function_score                                         [100% done]

@saimedhi For future reference, please provide clearer context in PR description for others who might not have full-context on what this PR fixes. Could you provide the following as well:

summary results for the test ran above
benchmark-metrics-* document that is associated with the force merge operation

benchmark-metrics-* document
{
    "took": 2,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 324,
            "relation": "eq"
        },
        "max_score": 1.0,
        "hits": [
            {
                "_index": "benchmark-metrics-2024-02",
                "_id": "KfFMn40BK0osdem3zqiS",
                "_score": 1.0,
                "_source": {
                    "@timestamp": 1707774626873,
                    "relative-time-ms": 5.7683749999988265,
                    "test-execution-id": "b1c6c994-b57b-4afa-8298-278a4bdbc7c3",
                    "test-execution-timestamp": "20240212T214955Z",
                    "environment": "local",
                    "workload": "geonames",
                    "test_procedure": "append-no-conflicts",
                    "provision-config-instance": "defaults",
                    "name": "service_time",
                    "value": 4.766000000000048,
                    "unit": "ms",
                    "sample-type": "warmup",
                    "meta": {
                        "plugins": [
                            "opensearch-alerting",
                            "opensearch-anomaly-detection",
                            "opensearch-asynchronous-search",
                            "opensearch-cross-cluster-replication",
                            "opensearch-custom-codecs",
                            "opensearch-geospatial",
                            "opensearch-index-management",
                            "opensearch-job-scheduler",
                            "opensearch-knn",
                            "opensearch-ml",
                            "opensearch-neural-search",
                            "opensearch-notifications",
                            "opensearch-notifications-core",
                            "opensearch-observability",
                            "opensearch-performance-analyzer",
                            "opensearch-reports-scheduler",
                            "opensearch-security",
                            "opensearch-security-analytics",
                            "opensearch-sql"
                        ],
                        "attribute_shard_indexing_pressure_enabled": "true",
                        "source_revision": "6b1986e964d440be9137eba1413015c31c5a7752",
                        "distribution_version": "2.11.1",
                        "distribution_flavor": "oss",
                        "success": true
                    },
                    "task": "asc_sort_with_after_geonameid",
                    "operation": "asc_sort_with_after_geonameid",
                    "operation-type": "search"
                }
            },
            {
                "_index": "benchmark-metrics-2024-02",
                "_id": "LvFMn40BK0osdem3zqiS",
                "_score": 1.0,
                "_source": {
                    "@timestamp": 1707774626878,
                    "relative-time-ms": 10.92083300000013,
                    "test-execution-id": "b1c6c994-b57b-4afa-8298-278a4bdbc7c3",
                    "test-execution-timestamp": "20240212T214955Z",
                    "environment": "local",
                    "workload": "geonames",
                    "test_procedure": "append-no-conflicts",
                    "provision-config-instance": "defaults",
                    "name": "client_processing_time",
                    "value": 0.22079100000027552,
                    "unit": "ms",
                    "sample-type": "normal",
                    "meta": {
                        "plugins": [
                            "opensearch-alerting",
                            "opensearch-anomaly-detection",
                            "opensearch-asynchronous-search",
                            "opensearch-cross-cluster-replication",
                            "opensearch-custom-codecs",
                            "opensearch-geospatial",
                            "opensearch-index-management",
                            "opensearch-job-scheduler",
                            "opensearch-knn",
                            "opensearch-ml",
                            "opensearch-neural-search",
                            "opensearch-notifications",
                            "opensearch-notifications-core",
                            "opensearch-observability",
                            "opensearch-performance-analyzer",
                            "opensearch-reports-scheduler",
                            "opensearch-security",
                            "opensearch-security-analytics",
                            "opensearch-sql"
                        ],
                        "attribute_shard_indexing_pressure_enabled": "true",
                        "source_revision": "6b1986e964d440be9137eba1413015c31c5a7752",
                        "distribution_version": "2.11.1",
                        "distribution_flavor": "oss",
                        "success": true
                    },
                    "task": "asc_sort_with_after_geonameid",
                    "operation": "asc_sort_with_after_geonameid",
                    "operation-type": "search"
                }
            },
            {
                "_index": "benchmark-metrics-2024-02",
                "_id": "5f9Mn40BBuuZzv8czmnm",
                "_score": 1.0,
                "_source": {
                    "@timestamp": 1707774627449,
                    "relative-time-ms": 586.0262919999997,
                    "test-execution-id": "b1c6c994-b57b-4afa-8298-278a4bdbc7c3",
                    "test-execution-timestamp": "20240212T214955Z",
                    "environment": "local",
                    "workload": "geonames",
                    "test_procedure": "append-no-conflicts",
                    "provision-config-instance": "defaults",
                    "name": "node_young_gen_gc_count",
                    "value": 1,
                    "unit": null,
                    "sample-type": "normal",
                    "meta": {
                        "plugins": [
                            "opensearch-alerting",
                            "opensearch-anomaly-detection",
                            "opensearch-asynchronous-search",
                            "opensearch-cross-cluster-replication",
                            "opensearch-custom-codecs",
                            "opensearch-geospatial",
                            "opensearch-index-management",
                            "opensearch-job-scheduler",
                            "opensearch-knn",
                            "opensearch-ml",
                            "opensearch-neural-search",
                            "opensearch-notifications",
                            "opensearch-notifications-core",
                            "opensearch-observability",
                            "opensearch-performance-analyzer",
                            "opensearch-reports-scheduler",
                            "opensearch-security",
                            "opensearch-security-analytics",
                            "opensearch-sql"
                        ],
                        "attribute_shard_indexing_pressure_enabled": "true",
                        "source_revision": "6b1986e964d440be9137eba1413015c31c5a7752",
                        "distribution_version": "2.11.1",
                        "distribution_flavor": "oss",
                        "node_name": "benchmark-node-0",
                        "host_name": "127.0.0.1",
                        "os_name": "Mac OS X",
                        "os_version": "14.2.1",
                        "cpu_logical_cores": 10,
                        "jvm_vendor": "Oracle Corporation",
                        "jvm_version": "17.0.9"
                    }
                }
            },
            {
                "_index": "benchmark-metrics-2024-02",
                "_id": "6P9Mn40BBuuZzv8czmnm",
                "_score": 1.0,
                "_source": {
                    "name": "jvm_memory_pool_stats",
                    "young": {
                        "peak_usage": 642777088,
                        "unit": "byte"
                    },
                    "old": {
                        "peak_usage": 113386496,
                        "unit": "byte"
                    },
                    "survivor": {
                        "peak_usage": 56098816,
                        "unit": "byte"
                    },
                    "@timestamp": 1707774627449,
                    "relative-time-ms": 586.0837500000002,
                    "test-execution-id": "b1c6c994-b57b-4afa-8298-278a4bdbc7c3",
                    "test-execution-timestamp": "20240212T214955Z",
                    "environment": "local",
                    "workload": "geonames",
                    "test_procedure": "append-no-conflicts",
                    "provision-config-instance": "defaults",
                    "meta": {
                        "plugins": [
                            "opensearch-alerting",
                            "opensearch-anomaly-detection",
                            "opensearch-asynchronous-search",
                            "opensearch-cross-cluster-replication",
                            "opensearch-custom-codecs",
                            "opensearch-geospatial",
                            "opensearch-index-management",
                            "opensearch-job-scheduler",
                            "opensearch-knn",
                            "opensearch-ml",
                            "opensearch-neural-search",
                            "opensearch-notifications",
                            "opensearch-notifications-core",
                            "opensearch-observability",
                            "opensearch-performance-analyzer",
                            "opensearch-reports-scheduler",
                            "opensearch-security",
                            "opensearch-security-analytics",
                            "opensearch-sql"
                        ],
                        "attribute_shard_indexing_pressure_enabled": "true",
                        "source_revision": "6b1986e964d440be9137eba1413015c31c5a7752",
                        "distribution_version": "2.11.1",
                        "distribution_flavor": "oss",
                        "node_name": "benchmark-node-0",
                        "host_name": "127.0.0.1",
                        "os_name": "Mac OS X",
                        "os_version": "14.2.1",
                        "cpu_logical_cores": 10,
                        "jvm_vendor": "Oracle Corporation",
                        "jvm_version": "17.0.9"
                    }
                }
            },
            {
                "_index": "benchmark-metrics-2024-02",
                "_id": "6_9Mn40BBuuZzv8czmnm",
                "_score": 1.0,
                "_source": {
                    "@timestamp": 1707774627449,
                    "relative-time-ms": 586.1085840000015,
                    "test-execution-id": "b1c6c994-b57b-4afa-8298-278a4bdbc7c3",
                    "test-execution-timestamp": "20240212T214955Z",
                    "environment": "local",
                    "workload": "geonames",
                    "test_procedure": "append-no-conflicts",
                    "provision-config-instance": "defaults",
                    "name": "node_total_old_gen_gc_time",
                    "value": 0,
                    "unit": "ms",
                    "sample-type": "normal",
                    "meta": {
                        "plugins": [
                            "opensearch-alerting",
                            "opensearch-anomaly-detection",
                            "opensearch-asynchronous-search",
                            "opensearch-cross-cluster-replication",
                            "opensearch-custom-codecs",
                            "opensearch-geospatial",
                            "opensearch-index-management",
                            "opensearch-job-scheduler",
                            "opensearch-knn",
                            "opensearch-ml",
                            "opensearch-neural-search",
                            "opensearch-notifications",
                            "opensearch-notifications-core",
                            "opensearch-observability",
                            "opensearch-performance-analyzer",
                            "opensearch-reports-scheduler",
                            "opensearch-security",
                            "opensearch-security-analytics",
                            "opensearch-sql"
                        ],
                        "attribute_shard_indexing_pressure_enabled": "true",
                        "source_revision": "6b1986e964d440be9137eba1413015c31c5a7752",
                        "distribution_version": "2.11.1",
                        "distribution_flavor": "oss"
                    }
                }
            },
            {
                "_index": "benchmark-metrics-2024-02",
                "_id": "7v9Mn40BBuuZzv8czmnm",
                "_score": 1.0,
                "_source": {
                    "@timestamp": 1707774627462,
                    "relative-time-ms": 599.1520840000035,
                    "test-execution-id": "b1c6c994-b57b-4afa-8298-278a4bdbc7c3",
                    "test-execution-timestamp": "20240212T214955Z",
                    "environment": "local",
                    "workload": "geonames",
                    "test_procedure": "append-no-conflicts",
                    "provision-config-instance": "defaults",
                    "name": "segments_memory_in_bytes",
                    "value": 0,
                    "unit": "byte",
                    "sample-type": "normal",
                    "meta": {
                        "plugins": [
                            "opensearch-alerting",
                            "opensearch-anomaly-detection",
                            "opensearch-asynchronous-search",
                            "opensearch-cross-cluster-replication",
                            "opensearch-custom-codecs",
                            "opensearch-geospatial",
                            "opensearch-index-management",
                            "opensearch-job-scheduler",
                            "opensearch-knn",
                            "opensearch-ml",
                            "opensearch-neural-search",
                            "opensearch-notifications",
                            "opensearch-notifications-core",
                            "opensearch-observability",
                            "opensearch-performance-analyzer",
                            "opensearch-reports-scheduler",
                            "opensearch-security",
                            "opensearch-security-analytics",
                            "opensearch-sql"
                        ],
                        "attribute_shard_indexing_pressure_enabled": "true",
                        "source_revision": "6b1986e964d440be9137eba1413015c31c5a7752",
                        "distribution_version": "2.11.1",
                        "distribution_flavor": "oss"
                    }
                }
            },
            {
                "_index": "benchmark-metrics-2024-02",
                "_id": "_v9Mn40BBuuZzv8czmnm",
                "_score": 1.0,
                "_source": {
                    "@timestamp": 1707774627462,
                    "relative-time-ms": 599.5308749999992,
                    "test-execution-id": "b1c6c994-b57b-4afa-8298-278a4bdbc7c3",
                    "test-execution-timestamp": "20240212T214955Z",
                    "environment": "local",
                    "workload": "geonames",
                    "test_procedure": "append-no-conflicts",
                    "provision-config-instance": "defaults",
                    "name": "translog_size_in_bytes",
                    "value": 344490,
                    "unit": "byte",
                    "sample-type": "normal",
                    "meta": {
                        "plugins": [
                            "opensearch-alerting",
                            "opensearch-anomaly-detection",
                            "opensearch-asynchronous-search",
                            "opensearch-cross-cluster-replication",
                            "opensearch-custom-codecs",
                            "opensearch-geospatial",
                            "opensearch-index-management",
                            "opensearch-job-scheduler",
                            "opensearch-knn",
                            "opensearch-ml",
                            "opensearch-neural-search",
                            "opensearch-notifications",
                            "opensearch-notifications-core",
                            "opensearch-observability",
                            "opensearch-performance-analyzer",
                            "opensearch-reports-scheduler",
                            "opensearch-security",
                            "opensearch-security-analytics",
                            "opensearch-sql"
                        ],
                        "attribute_shard_indexing_pressure_enabled": "true",
                        "source_revision": "6b1986e964d440be9137eba1413015c31c5a7752",
                        "distribution_version": "2.11.1",
                        "distribution_flavor": "oss"
                    }
                }
            },
            {
                "_index": "benchmark-metrics-2024-02",
                "_id": "UP9Mn40BBuuZzv8cjWlA",
                "_score": 1.0,
                "_source": {
                    "@timestamp": 1707774610140,
                    "relative-time-ms": 10.907416000000225,
                    "test-execution-id": "b1c6c994-b57b-4afa-8298-278a4bdbc7c3",
                    "test-execution-timestamp": "20240212T214955Z",
                    "environment": "local",
                    "workload": "geonames",
                    "test_procedure": "append-no-conflicts",
                    "provision-config-instance": "defaults",
                    "name": "processing_time",
                    "value": 69.75395899999981,
                    "unit": "ms",
                    "sample-type": "normal",
                    "meta": {
                        "plugins": [
                            "opensearch-alerting",
                            "opensearch-anomaly-detection",
                            "opensearch-asynchronous-search",
                            "opensearch-cross-cluster-replication",
                            "opensearch-custom-codecs",
                            "opensearch-geospatial",
                            "opensearch-index-management",
                            "opensearch-job-scheduler",
                            "opensearch-knn",
                            "opensearch-ml",
                            "opensearch-neural-search",
                            "opensearch-notifications",
                            "opensearch-notifications-core",
                            "opensearch-observability",
                            "opensearch-performance-analyzer",
                            "opensearch-reports-scheduler",
                            "opensearch-security",
                            "opensearch-security-analytics",
                            "opensearch-sql"
                        ],
                        "attribute_shard_indexing_pressure_enabled": "true",
                        "source_revision": "6b1986e964d440be9137eba1413015c31c5a7752",
                        "distribution_version": "2.11.1",
                        "distribution_flavor": "oss",
                        "index": "geonames",
                        "took": 64,
                        "success": true,
                        "success-count": 125,
                        "error-count": 0
                    },
                    "task": "index-append",
                    "operation": "index-append",
                    "operation-type": "bulk"
                }
            },
            {
                "_index": "benchmark-metrics-2024-02",
                "_id": "W_9Mn40BBuuZzv8cjWlA",
                "_score": 1.0,
                "_source": {
                    "@timestamp": 1707774610158,
                    "relative-time-ms": 27.35629199999856,
                    "test-execution-id": "b1c6c994-b57b-4afa-8298-278a4bdbc7c3",
                    "test-execution-timestamp": "20240212T214955Z",
                    "environment": "local",
                    "workload": "geonames",
                    "test_procedure": "append-no-conflicts",
                    "provision-config-instance": "defaults",
                    "name": "client_processing_time",
                    "value": 1.8018749999999528,
                    "unit": "ms",
                    "sample-type": "normal",
                    "meta": {
                        "plugins": [
                            "opensearch-alerting",
                            "opensearch-anomaly-detection",
                            "opensearch-asynchronous-search",
                            "opensearch-cross-cluster-replication",
                            "opensearch-custom-codecs",
                            "opensearch-geospatial",
                            "opensearch-index-management",
                            "opensearch-job-scheduler",
                            "opensearch-knn",
                            "opensearch-ml",
                            "opensearch-neural-search",
                            "opensearch-notifications",
                            "opensearch-notifications-core",
                            "opensearch-observability",
                            "opensearch-performance-analyzer",
                            "opensearch-reports-scheduler",
                            "opensearch-security",
                            "opensearch-security-analytics",
                            "opensearch-sql"
                        ],
                        "attribute_shard_indexing_pressure_enabled": "true",
                        "source_revision": "6b1986e964d440be9137eba1413015c31c5a7752",
                        "distribution_version": "2.11.1",
                        "distribution_flavor": "oss",
                        "index": "geonames",
                        "took": 76,
                        "success": true,
                        "success-count": 125,
                        "error-count": 0
                    },
                    "task": "index-append",
                    "operation": "index-append",
                    "operation-type": "bulk"
                }
            },
            {
                "_index": "benchmark-metrics-2024-02",
                "_id": "Xf9Mn40BBuuZzv8cjWlA",
                "_score": 1.0,
                "_source": {
                    "@timestamp": 1707774610158,
                    "relative-time-ms": 26.233624999999705,
                    "test-execution-id": "b1c6c994-b57b-4afa-8298-278a4bdbc7c3",
                    "test-execution-timestamp": "20240212T214955Z",
                    "environment": "local",
                    "workload": "geonames",
                    "test_procedure": "append-no-conflicts",
                    "provision-config-instance": "defaults",
                    "name": "latency",
                    "value": 82.27633300000115,
                    "unit": "ms",
                    "sample-type": "normal",
                    "meta": {
                        "plugins": [
                            "opensearch-alerting",
                            "opensearch-anomaly-detection",
                            "opensearch-asynchronous-search",
                            "opensearch-cross-cluster-replication",
                            "opensearch-custom-codecs",
                            "opensearch-geospatial",
                            "opensearch-index-management",
                            "opensearch-job-scheduler",
                            "opensearch-knn",
                            "opensearch-ml",
                            "opensearch-neural-search",
                            "opensearch-notifications",
                            "opensearch-notifications-core",
                            "opensearch-observability",
                            "opensearch-performance-analyzer",
                            "opensearch-reports-scheduler",
                            "opensearch-security",
                            "opensearch-security-analytics",
                            "opensearch-sql"
                        ],
                        "attribute_shard_indexing_pressure_enabled": "true",
                        "source_revision": "6b1986e964d440be9137eba1413015c31c5a7752",
                        "distribution_version": "2.11.1",
                        "distribution_flavor": "oss",
                        "index": "geonames",
                        "took": 76,
                        "success": true,
                        "success-count": 125,
                        "error-count": 0
                    },
                    "task": "index-append",
                    "operation": "index-append",
                    "operation-type": "bulk"
                }
            }
        ]
    }
}

benchmark-metrics-* document
{
    "took": 2,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 324,
            "relation": "eq"
        },
        "max_score": 1.0,
        "hits": [
            {
                "_index": "benchmark-metrics-2024-02",
                "_id": "KfFMn40BK0osdem3zqiS",
                "_score": 1.0,
                "_source": {
                    "@timestamp": 1707774626873,
                    "relative-time-ms": 5.7683749999988265,
                    "test-execution-id": "b1c6c994-b57b-4afa-8298-278a4bdbc7c3",
                    "test-execution-timestamp": "20240212T214955Z",
                    "environment": "local",
                    "workload": "geonames",
                    "test_procedure": "append-no-conflicts",
                    "provision-config-instance": "defaults",
                    "name": "service_time",
                    "value": 4.766000000000048,
                    "unit": "ms",
                    "sample-type": "warmup",
                    "meta": {
                        "plugins": [
                            "opensearch-alerting",
                            "opensearch-anomaly-detection",
                            "opensearch-asynchronous-search",
                            "opensearch-cross-cluster-replication",
                            "opensearch-custom-codecs",
                            "opensearch-geospatial",
                            "opensearch-index-management",
                            "opensearch-job-scheduler",
                            "opensearch-knn",
                            "opensearch-ml",
                            "opensearch-neural-search",
                            "opensearch-notifications",
                            "opensearch-notifications-core",
                            "opensearch-observability",
                            "opensearch-performance-analyzer",
                            "opensearch-reports-scheduler",
                            "opensearch-security",
                            "opensearch-security-analytics",
                            "opensearch-sql"
                        ],
                        "attribute_shard_indexing_pressure_enabled": "true",
                        "source_revision": "6b1986e964d440be9137eba1413015c31c5a7752",
                        "distribution_version": "2.11.1",
                        "distribution_flavor": "oss",
                        "success": true
                    },
                    "task": "asc_sort_with_after_geonameid",
                    "operation": "asc_sort_with_after_geonameid",
                    "operation-type": "search"
                }
            },
            {
                "_index": "benchmark-metrics-2024-02",
                "_id": "LvFMn40BK0osdem3zqiS",
                "_score": 1.0,
                "_source": {
                    "@timestamp": 1707774626878,
                    "relative-time-ms": 10.92083300000013,
                    "test-execution-id": "b1c6c994-b57b-4afa-8298-278a4bdbc7c3",
                    "test-execution-timestamp": "20240212T214955Z",
                    "environment": "local",
                    "workload": "geonames",
                    "test_procedure": "append-no-conflicts",
                    "provision-config-instance": "defaults",
                    "name": "client_processing_time",
                    "value": 0.22079100000027552,
                    "unit": "ms",
                    "sample-type": "normal",
                    "meta": {
                        "plugins": [
                            "opensearch-alerting",
                            "opensearch-anomaly-detection",
                            "opensearch-asynchronous-search",
                            "opensearch-cross-cluster-replication",
                            "opensearch-custom-codecs",
                            "opensearch-geospatial",
                            "opensearch-index-management",
                            "opensearch-job-scheduler",
                            "opensearch-knn",
                            "opensearch-ml",
                            "opensearch-neural-search",
                            "opensearch-notifications",
                            "opensearch-notifications-core",
                            "opensearch-observability",
                            "opensearch-performance-analyzer",
                            "opensearch-reports-scheduler",
                            "opensearch-security",
                            "opensearch-security-analytics",
                            "opensearch-sql"
                        ],
                        "attribute_shard_indexing_pressure_enabled": "true",
                        "source_revision": "6b1986e964d440be9137eba1413015c31c5a7752",
                        "distribution_version": "2.11.1",
                        "distribution_flavor": "oss",
                        "success": true
                    },
                    "task": "asc_sort_with_after_geonameid",
                    "operation": "asc_sort_with_after_geonameid",
                    "operation-type": "search"
                }
            },
            {
                "_index": "benchmark-metrics-2024-02",
                "_id": "5f9Mn40BBuuZzv8czmnm",
                "_score": 1.0,
                "_source": {
                    "@timestamp": 1707774627449,
                    "relative-time-ms": 586.0262919999997,
                    "test-execution-id": "b1c6c994-b57b-4afa-8298-278a4bdbc7c3",
                    "test-execution-timestamp": "20240212T214955Z",
                    "environment": "local",
                    "workload": "geonames",
                    "test_procedure": "append-no-conflicts",
                    "provision-config-instance": "defaults",
                    "name": "node_young_gen_gc_count",
                    "value": 1,
                    "unit": null,
                    "sample-type": "normal",
                    "meta": {
                        "plugins": [
                            "opensearch-alerting",
                            "opensearch-anomaly-detection",
                            "opensearch-asynchronous-search",
                            "opensearch-cross-cluster-replication",
                            "opensearch-custom-codecs",
                            "opensearch-geospatial",
                            "opensearch-index-management",
                            "opensearch-job-scheduler",
                            "opensearch-knn",
                            "opensearch-ml",
                            "opensearch-neural-search",
                            "opensearch-notifications",
                            "opensearch-notifications-core",
                            "opensearch-observability",
                            "opensearch-performance-analyzer",
                            "opensearch-reports-scheduler",
                            "opensearch-security",
                            "opensearch-security-analytics",
                            "opensearch-sql"
                        ],
                        "attribute_shard_indexing_pressure_enabled": "true",
                        "source_revision": "6b1986e964d440be9137eba1413015c31c5a7752",
                        "distribution_version": "2.11.1",
                        "distribution_flavor": "oss",
                        "node_name": "benchmark-node-0",
                        "host_name": "127.0.0.1",
                        "os_name": "Mac OS X",
                        "os_version": "14.2.1",
                        "cpu_logical_cores": 10,
                        "jvm_vendor": "Oracle Corporation",
                        "jvm_version": "17.0.9"
                    }
                }
            },
            {
                "_index": "benchmark-metrics-2024-02",
                "_id": "6P9Mn40BBuuZzv8czmnm",
                "_score": 1.0,
                "_source": {
                    "name": "jvm_memory_pool_stats",
                    "young": {
                        "peak_usage": 642777088,
                        "unit": "byte"
                    },
                    "old": {
                        "peak_usage": 113386496,
                        "unit": "byte"
                    },
                    "survivor": {
                        "peak_usage": 56098816,
                        "unit": "byte"
                    },
                    "@timestamp": 1707774627449,
                    "relative-time-ms": 586.0837500000002,
                    "test-execution-id": "b1c6c994-b57b-4afa-8298-278a4bdbc7c3",
                    "test-execution-timestamp": "20240212T214955Z",
                    "environment": "local",
                    "workload": "geonames",
                    "test_procedure": "append-no-conflicts",
                    "provision-config-instance": "defaults",
                    "meta": {
                        "plugins": [
                            "opensearch-alerting",
                            "opensearch-anomaly-detection",
                            "opensearch-asynchronous-search",
                            "opensearch-cross-cluster-replication",
                            "opensearch-custom-codecs",
                            "opensearch-geospatial",
                            "opensearch-index-management",
                            "opensearch-job-scheduler",
                            "opensearch-knn",
                            "opensearch-ml",
                            "opensearch-neural-search",
                            "opensearch-notifications",
                            "opensearch-notifications-core",
                            "opensearch-observability",
                            "opensearch-performance-analyzer",
                            "opensearch-reports-scheduler",
                            "opensearch-security",
                            "opensearch-security-analytics",
                            "opensearch-sql"
                        ],
                        "attribute_shard_indexing_pressure_enabled": "true",
                        "source_revision": "6b1986e964d440be9137eba1413015c31c5a7752",
                        "distribution_version": "2.11.1",
                        "distribution_flavor": "oss",
                        "node_name": "benchmark-node-0",
                        "host_name": "127.0.0.1",
                        "os_name": "Mac OS X",
                        "os_version": "14.2.1",
                        "cpu_logical_cores": 10,
                        "jvm_vendor": "Oracle Corporation",
                        "jvm_version": "17.0.9"
                    }
                }
            },
            {
                "_index": "benchmark-metrics-2024-02",
                "_id": "6_9Mn40BBuuZzv8czmnm",
                "_score": 1.0,
                "_source": {
                    "@timestamp": 1707774627449,
                    "relative-time-ms": 586.1085840000015,
                    "test-execution-id": "b1c6c994-b57b-4afa-8298-278a4bdbc7c3",
                    "test-execution-timestamp": "20240212T214955Z",
                    "environment": "local",
                    "workload": "geonames",
                    "test_procedure": "append-no-conflicts",
                    "provision-config-instance": "defaults",
                    "name": "node_total_old_gen_gc_time",
                    "value": 0,
                    "unit": "ms",
                    "sample-type": "normal",
                    "meta": {
                        "plugins": [
                            "opensearch-alerting",
                            "opensearch-anomaly-detection",
                            "opensearch-asynchronous-search",
                            "opensearch-cross-cluster-replication",
                            "opensearch-custom-codecs",
                            "opensearch-geospatial",
                            "opensearch-index-management",
                            "opensearch-job-scheduler",
                            "opensearch-knn",
                            "opensearch-ml",
                            "opensearch-neural-search",
                            "opensearch-notifications",
                            "opensearch-notifications-core",
                            "opensearch-observability",
                            "opensearch-performance-analyzer",
                            "opensearch-reports-scheduler",
                            "opensearch-security",
                            "opensearch-security-analytics",
                            "opensearch-sql"
                        ],
                        "attribute_shard_indexing_pressure_enabled": "true",
                        "source_revision": "6b1986e964d440be9137eba1413015c31c5a7752",
                        "distribution_version": "2.11.1",
                        "distribution_flavor": "oss"
                    }
                }
            },
            {
                "_index": "benchmark-metrics-2024-02",
                "_id": "7v9Mn40BBuuZzv8czmnm",
                "_score": 1.0,
                "_source": {
                    "@timestamp": 1707774627462,
                    "relative-time-ms": 599.1520840000035,
                    "test-execution-id": "b1c6c994-b57b-4afa-8298-278a4bdbc7c3",
                    "test-execution-timestamp": "20240212T214955Z",
                    "environment": "local",
                    "workload": "geonames",
                    "test_procedure": "append-no-conflicts",
                    "provision-config-instance": "defaults",
                    "name": "segments_memory_in_bytes",
                    "value": 0,
                    "unit": "byte",
                    "sample-type": "normal",
                    "meta": {
                        "plugins": [
                            "opensearch-alerting",
                            "opensearch-anomaly-detection",
                            "opensearch-asynchronous-search",
                            "opensearch-cross-cluster-replication",
                            "opensearch-custom-codecs",
                            "opensearch-geospatial",
                            "opensearch-index-management",
                            "opensearch-job-scheduler",
                            "opensearch-knn",
                            "opensearch-ml",
                            "opensearch-neural-search",
                            "opensearch-notifications",
                            "opensearch-notifications-core",
                            "opensearch-observability",
                            "opensearch-performance-analyzer",
                            "opensearch-reports-scheduler",
                            "opensearch-security",
                            "opensearch-security-analytics",
                            "opensearch-sql"
                        ],
                        "attribute_shard_indexing_pressure_enabled": "true",
                        "source_revision": "6b1986e964d440be9137eba1413015c31c5a7752",
                        "distribution_version": "2.11.1",
                        "distribution_flavor": "oss"
                    }
                }
            },
            {
                "_index": "benchmark-metrics-2024-02",
                "_id": "_v9Mn40BBuuZzv8czmnm",
                "_score": 1.0,
                "_source": {
                    "@timestamp": 1707774627462,
                    "relative-time-ms": 599.5308749999992,
                    "test-execution-id": "b1c6c994-b57b-4afa-8298-278a4bdbc7c3",
                    "test-execution-timestamp": "20240212T214955Z",
                    "environment": "local",
                    "workload": "geonames",
                    "test_procedure": "append-no-conflicts",
                    "provision-config-instance": "defaults",
                    "name": "translog_size_in_bytes",
                    "value": 344490,
                    "unit": "byte",
                    "sample-type": "normal",
                    "meta": {
                        "plugins": [
                            "opensearch-alerting",
                            "opensearch-anomaly-detection",
                            "opensearch-asynchronous-search",
                            "opensearch-cross-cluster-replication",
                            "opensearch-custom-codecs",
                            "opensearch-geospatial",
                            "opensearch-index-management",
                            "opensearch-job-scheduler",
                            "opensearch-knn",
                            "opensearch-ml",
                            "opensearch-neural-search",
                            "opensearch-notifications",
                            "opensearch-notifications-core",
                            "opensearch-observability",
                            "opensearch-performance-analyzer",
                            "opensearch-reports-scheduler",
                            "opensearch-security",
                            "opensearch-security-analytics",
                            "opensearch-sql"
                        ],
                        "attribute_shard_indexing_pressure_enabled": "true",
                        "source_revision": "6b1986e964d440be9137eba1413015c31c5a7752",
                        "distribution_version": "2.11.1",
                        "distribution_flavor": "oss"
                    }
                }
            },
            {
                "_index": "benchmark-metrics-2024-02",
                "_id": "UP9Mn40BBuuZzv8cjWlA",
                "_score": 1.0,
                "_source": {
                    "@timestamp": 1707774610140,
                    "relative-time-ms": 10.907416000000225,
                    "test-execution-id": "b1c6c994-b57b-4afa-8298-278a4bdbc7c3",
                    "test-execution-timestamp": "20240212T214955Z",
                    "environment": "local",
                    "workload": "geonames",
                    "test_procedure": "append-no-conflicts",
                    "provision-config-instance": "defaults",
                    "name": "processing_time",
                    "value": 69.75395899999981,
                    "unit": "ms",
                    "sample-type": "normal",
                    "meta": {
                        "plugins": [
                            "opensearch-alerting",
                            "opensearch-anomaly-detection",
                            "opensearch-asynchronous-search",
                            "opensearch-cross-cluster-replication",
                            "opensearch-custom-codecs",
                            "opensearch-geospatial",
                            "opensearch-index-management",
                            "opensearch-job-scheduler",
                            "opensearch-knn",
                            "opensearch-ml",
                            "opensearch-neural-search",
                            "opensearch-notifications",
                            "opensearch-notifications-core",
                            "opensearch-observability",
                            "opensearch-performance-analyzer",
                            "opensearch-reports-scheduler",
                            "opensearch-security",
                            "opensearch-security-analytics",
                            "opensearch-sql"
                        ],
                        "attribute_shard_indexing_pressure_enabled": "true",
                        "source_revision": "6b1986e964d440be9137eba1413015c31c5a7752",
                        "distribution_version": "2.11.1",
                        "distribution_flavor": "oss",
                        "index": "geonames",
                        "took": 64,
                        "success": true,
                        "success-count": 125,
                        "error-count": 0
                    },
                    "task": "index-append",
                    "operation": "index-append",
                    "operation-type": "bulk"
                }
            },
            {
                "_index": "benchmark-metrics-2024-02",
                "_id": "W_9Mn40BBuuZzv8cjWlA",
                "_score": 1.0,
                "_source": {
                    "@timestamp": 1707774610158,
                    "relative-time-ms": 27.35629199999856,
                    "test-execution-id": "b1c6c994-b57b-4afa-8298-278a4bdbc7c3",
                    "test-execution-timestamp": "20240212T214955Z",
                    "environment": "local",
                    "workload": "geonames",
                    "test_procedure": "append-no-conflicts",
                    "provision-config-instance": "defaults",
                    "name": "client_processing_time",
                    "value": 1.8018749999999528,
                    "unit": "ms",
                    "sample-type": "normal",
                    "meta": {
                        "plugins": [
                            "opensearch-alerting",
                            "opensearch-anomaly-detection",
                            "opensearch-asynchronous-search",
                            "opensearch-cross-cluster-replication",
                            "opensearch-custom-codecs",
                            "opensearch-geospatial",
                            "opensearch-index-management",
                            "opensearch-job-scheduler",
                            "opensearch-knn",
                            "opensearch-ml",
                            "opensearch-neural-search",
                            "opensearch-notifications",
                            "opensearch-notifications-core",
                            "opensearch-observability",
                            "opensearch-performance-analyzer",
                            "opensearch-reports-scheduler",
                            "opensearch-security",
                            "opensearch-security-analytics",
                            "opensearch-sql"
                        ],
                        "attribute_shard_indexing_pressure_enabled": "true",
                        "source_revision": "6b1986e964d440be9137eba1413015c31c5a7752",
                        "distribution_version": "2.11.1",
                        "distribution_flavor": "oss",
                        "index": "geonames",
                        "took": 76,
                        "success": true,
                        "success-count": 125,
                        "error-count": 0
                    },
                    "task": "index-append",
                    "operation": "index-append",
                    "operation-type": "bulk"
                }
            },
            {
                "_index": "benchmark-metrics-2024-02",
                "_id": "Xf9Mn40BBuuZzv8cjWlA",
                "_score": 1.0,
                "_source": {
                    "@timestamp": 1707774610158,
                    "relative-time-ms": 26.233624999999705,
                    "test-execution-id": "b1c6c994-b57b-4afa-8298-278a4bdbc7c3",
                    "test-execution-timestamp": "20240212T214955Z",
                    "environment": "local",
                    "workload": "geonames",
                    "test_procedure": "append-no-conflicts",
                    "provision-config-instance": "defaults",
                    "name": "latency",
                    "value": 82.27633300000115,
                    "unit": "ms",
                    "sample-type": "normal",
                    "meta": {
                        "plugins": [
                            "opensearch-alerting",
                            "opensearch-anomaly-detection",
                            "opensearch-asynchronous-search",
                            "opensearch-cross-cluster-replication",
                            "opensearch-custom-codecs",
                            "opensearch-geospatial",
                            "opensearch-index-management",
                            "opensearch-job-scheduler",
                            "opensearch-knn",
                            "opensearch-ml",
                            "opensearch-neural-search",
                            "opensearch-notifications",
                            "opensearch-notifications-core",
                            "opensearch-observability",
                            "opensearch-performance-analyzer",
                            "opensearch-reports-scheduler",
                            "opensearch-security",
                            "opensearch-security-analytics",
                            "opensearch-sql"
                        ],
                        "attribute_shard_indexing_pressure_enabled": "true",
                        "source_revision": "6b1986e964d440be9137eba1413015c31c5a7752",
                        "distribution_version": "2.11.1",
                        "distribution_flavor": "oss",
                        "index": "geonames",
                        "took": 76,
                        "success": true,
                        "success-count": 125,
                        "error-count": 0
                    },
                    "task": "index-append",
                    "operation": "index-append",
                    "operation-type": "bulk"
                }
            }
        ]
    }
}

Do you have a sample document for any force-merges? You can filter the dashboards to look for operation-type = force-merge

authors: IanHoang saimedhi

repo: opensearch-project/opensearch-benchmark


text:
Change the issue title for release to be 'proposed'

The current langauge makes the autocut release tickets seem like a
commitment for a release, whereas they are a best guess effort
that a release could happen.  E.g. there might ne a patch release in
flight and then don't release that in favor of a minor release that
also includes the contents as the patch release.
Check List

[X] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #1591 (acd44f0) into main (6c54533) will decrease coverage by 0.15%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##               main    #1591      +/-   ##
============================================
- Coverage     94.63%   94.48%   -0.16%
  Complexity       13       13
============================================
  Files           153      153
  Lines          3281     3281
  Branches         21       21
============================================
- Hits           3105     3100       -5
- Misses          173      178       +5
  Partials          3        3

Impacted Files
Coverage Δ

src/system/temporary_directory.py
83.87% <0.00%> (-12.91%)
:arrow_down:

src/system/os.py
92.30% <0.00%> (-7.70%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 484f6bc...acd44f0. Read the comment docs.

Can you please point me to a comment where someone is confused about that?

@bbarani can you provide background on this?

I am afraid this will re-auto-cut all old version tickets

Yes, if updating the title is the right path I could spend more time looking into how we can prevent recuts.  Making an update to the issue creation GHA could be a way.

Yes, if updating the title is the right path I could spend more time looking into how we can prevent recuts. Making an update to the issue creation GHA could be a way.

FYI I had forked the original GHA given its lack of maintenance, https://github.com/dblock/create-a-github-issue

@bbarani I haven't seen any updates, feel free to reopen this PR or one like it if you feel this change is needed

source: gharchive/pull-request

authors: codecov-commenter dblock peternied

repo: opensearch-project/opensearch-build

license_type: permissive

license_source: github-api


text:
[FEATURE] Add support for changing storage size of seed node

Is your feature request related to a problem?
The EBS volume size of seed node can't be configured and is hard-coded to 50GB:
https://github.com/opensearch-project/opensearch-cluster-cdk/blob/d3b12a8774835f1a6fdf41d2ea00b8e9fb7f8b8e/lib/infra/infra-stack.ts#L213
When deploying a cluster with no cluster manager node, the seed node will have data role:
https://github.com/opensearch-project/opensearch-cluster-cdk/blob/d3b12a8774835f1a6fdf41d2ea00b8e9fb7f8b8e/lib/opensearch-config/node-config.ts#L25
It means data will be stored evenly on the "seed" node and other data nodes. Because the storage size of other data nodes can be configured and is 100GB by default:
https://github.com/opensearch-project/opensearch-cluster-cdk/blob/d3b12a8774835f1a6fdf41d2ea00b8e9fb7f8b8e/lib/infra/infra-stack.ts#L130
The storage size of the seed node will be a bottleneck.
What solution would you like?
Make the EBS volume size of the seed node configurable.
What alternatives have you considered?
n/a
Do you have any additional context?
n/a

@rishabh6788 , pls have a review and comment.

@tlfeng you can use --context dataNodeStorage to set the storage size on data node. Please let me know if that helps your case.

source: gharchive/issue

authors: jordarlu rishabh6788 tlfeng


text:
[FEATURE] Pass extensions.yml file to ExtensionsRunner

Is your feature request related to a problem?
Currently, SDK relies on a test extenions.yml file to get the metadata of the extensions. Create a workflow to read the extensions.yml of the extensions and pass it to ExtensionSettings.java
What solution would you like?
A clear and concise description of what you want to happen.
What alternatives have you considered?
A clear and concise description of any alternative solutions or features you've considered.
Do you have any additional context?
Add any other context or screenshots about the feature request here.
Once the location of extension.yml is finalized we have to update Developer guide.

We will be passing one single object of the extension to SDK from OpenSearch. It will be covered here: https://github.com/opensearch-project/opensearch-sdk-java/issues/93. Closing tis in favor of that.

repo: opensearch-project/opensearch-sdk-java


text:
[BUG] [CCI] Hover effect of a first options menu item

Whenever panel options menu is opened, the first menu item has a colored background and underlined text, even when not hovering on the item:

Also hovering on another item only undelines the text, but does not change the background color:

Proposal:

To apply colored background and underlined text to menu item on hover.
Fix first menu item colored background and underlined text bug.

@olimeme & @ananzh : This is expected behavior within the EuiContextMenu / OuiContextMenu component for accessibility. This highlight lets the user know where the current keyboard focus is. There are several other components that also have a keyboard focus indicator on page load or other interactions.
Closing as this is not a bug and we will not be changing this.

authors: KrooshalUX olimeme

repo: opensearch-project/oui


text:
[Research] Research the usage of package yo in OUI

Research the usage of package yo in OUI and determine if we can get rid of the dependency.
Reason: yo depends on request which is deprecated and causing #630

According to the documentation, yo is the Yeoman command line utility allowing the creation of projects utilizing scaffolding templates (referred to as generators). The project has two generators, one to generate a document template (yo-doc), and the other to generate a component template (yo-component).
In version yo@4.3.1, insight was removed from the dependencies. Recently https://github.com/opensearch-project/oui/pull/599 was merged, which just raised the yo version and resolved this issue.


text:
Support popover vs. modal differentiation

Current guidelines state that popovers can be used when not in the context of a flyout, but should be modals when embedded within a flyout. Ideally we have pre-made components that are commonly popovers (e.g., filter list as seen on Discover page), and provide out-of-the-box support of that to render within a flyout, such that the popovers are converted to modals.

cc @xeniatup

The pattern guidance that was given in regards to the Quick-Create flow in an OuiFlyout for the Anywhere project was to not utilize OuiPopover specifically for sub-flows within forms in OuiFlyout. It is more appropriate for a sub-flow to be handled within OuiModal. The issue that we ran into was that in the current full-page experience, the subflow is expressed as a popover.
There are legitimate other uses of OuiPopover within OuiFlyout - for example, if OuiFlyout contains OuiBasicTable and OuiFiilterGroup to filter the table - OuiFilterGroups base component is OuiPopover. Supporting this automatic conversion would have unintended consequences.
As we work towards publishing design system pattern guidance (which is different than component library documentation), this can be clarified even further.
Given that clarification, @ohltyler , do you see us needing to support this conversion?

There are legitimate other uses of OuiPopover within OuiFlyout - for example, if OuiFlyout contains OuiBasicTable and OuiFiilterGroup to filter the table - OuiFilterGroups base component is OuiPopover. Supporting this conversion would have unintended consequences.

For this, could we have a lever to have it forced as a popover regardless what component it is under in? This would allow us to have a bit cleaner code and not duplicate many things. Especially since for any flows similar to the Anywhere project will essentially have to duplicate the code for the two different flows when there should really be no duplication needed there to support both flows.

@lezzago I would like to review the specific use case with you and @xeniatup - I have a hunch that the popover in the full flow that is providing this issue with maintenance may be a pattern that needs to be updated as well.
Let's take a look at that before we proceed further on this.

authors: KrooshalUX lezzago ohltyler


text:
Prepare for 1.9.0 release

Check List

[ ] New functionality includes testing.
[ ] New functionality has been documented.
[ ] All tests pass

[ ] yarn lint
[ ] yarn test-unit

[X] Update CHANGELOG.md
[X] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

OSD failures are expected.


text:
add completion count into shard level and task level SBP RCA

Related PRs

https://github.com/opensearch-project/performance-analyzer/pull/593
https://github.com/opensearch-project/performance-analyzer-commons/pull/53

github issue: https://github.com/opensearch-project/performance-analyzer/issues/587
Changing logic to take completionCount into account at shard and task level to accurately calculate the SBP RCAs.
2023-11-10 06:44:40.893 [PA:Reader] [9449wUCJQp2jb7FYh7CmAw-task-0-] INFO  org.opensearch.performanceanalyzer.plugins.PublisherEventsLogger - Action: [SearchBackPressureAction] published by decision maker publisher. action summary : {"thresholdName":"heap_usage","searchbpDimension":"shard","direction":"decrease","stepSizeInPercentage":5.0,"coolOffPeriodInMillis":86400000,"canUpdate":true}

Is your feature request related to a problem? Please provide an existing Issue # , or describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you are proposing
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
Check List

[ ] Backport Labels added.
[ ] New functionality includes testing.

[ ] All tests pass

[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[X] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

There are some build errors related to syntax. Please fix and try to build locally


text:
Add husky to pre-lint any commits, remove unused imports

Description
Adds a rule to lint any commits prior to pushing. This should fix cases where work is pushed up but not linted, failing CI. TODO: should we add running tests here?
Category
[Enhancement, New feature, Bug fix, Test fix, Refactoring, Maintenance, Documentation]
Why these changes are required?
What is the old behavior before changes and new behavior after changes?
Linting was not done automatically and developer cycles were needed to remember to manually run that command
Issues Resolved
[List any issues this PR will resolve (Is this a backport? If so, please add backport PR # and/or commits #)]
Fix: https://github.com/opensearch-project/security-dashboards-plugin/issues/1805
Testing
[Please provide details of testing done: unit testing, integration testing and manual testing]
Check List

[ ] New functionality includes testing
[ ] New functionality has been documented
[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@cwperks this address this comment as well: https://github.com/opensearch-project/security-dashboards-plugin/pull/1824#discussion_r1522123256

Thank you @derek-ho! This makes the commit process consistent with OSD core.
OSD Core has an issue filed around unused imports. If the issue is fixed in core, would those rules transitively apply to the security-dashboards-plugin?

This PR is already making unused imports error the linting process by adding the "unused-imports" plugin to the eslintrc.js file - I didn't manually remove those imports, it was automatically done as part of the pre-commit hook.


text:
[BUG] SeedHostsResolver failed to resolve host | OpenSSLTest test failures

What is the bug?
Both these tests have started to fail on main and 2.x branches

org.opensearch.security.ssl.OpenSSLTest.testNodeClientSSL
org.opensearch.security.ssl.OpenSSLTest.testNodeClientSSLwithOpenSslTLSv13

I believe this is the new error that is occuring
[2023-03-30T15:13:53,991][WARN ][org.opensearch.discovery.SeedHostsResolver] failed to resolve host [null:0]
  1> java.net.UnknownHostException: null
  1> 	at java.net.InetAddress$CachedAddresses.get(InetAddress.java:797) ~[?:?]
  1> 	at java.net.InetAddress.getAllByName0(InetAddress.java:1524) ~[?:?]
  1> 	at java.net.InetAddress.getAllByName(InetAddress.java:1382) ~[?:?]
  1> 	at java.net.InetAddress.getAllByName(InetAddress.java:1306) ~[?:?]
  1> 	at org.opensearch.transport.TcpTransport.parse(TcpTransport.java:615) ~[opensearch-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
  1> 	at org.opensearch.transport.TcpTransport.addressesFromString(TcpTransport.java:557) ~[opensearch-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
  1> 	at org.opensearch.transport.TransportService.addressesFromString(TransportService.java:1051) ~[opensearch-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
  1> 	at org.opensearch.discovery.SeedHostsResolver.lambda$resolveHostsLists$0(SeedHostsResolver.java:182) ~[opensearch-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
  1> 	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]

How can one reproduce the bug?
It reproduces 100% when run on GitHub Action, but does not reproduce locally for me, investigating
Do you have any additional context?
See failure log

Main https://github.com/opensearch-project/security/actions/runs/4565952723/jobs/8057787009
2.x https://github.com/opensearch-project/security/actions/runs/4566573970/jobs/8059266808?pr=2608

The failing tests are:

OpenSSLTest.testNodeClientSSLwithOpenSslTLSv13
OpenSSLTest.testNodeClientSSL

In both instances, the test create a three node ssl only cluster with 1 manager and 2 data nodes and in the test case they try adding a 4th node to the cluster, but the 4th node cannot discover the cluster manager.
These tests only run on JDK < 12.

Based on the log, it seems like we couldn't setup the openSSL connection correctly. For this error:
[ERROR][org.opensearch.security.ssl.http.netty.SecuritySSLNettyHttpServerTransport] Exception during establishing a SSL connection: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 474554202f5f6f70656e64697374726f2f5f73656375726974792f73736c696e666f3f70726574747920485454502f312e310d0a486f73743a203132372e302e302e313a31303237340d0a436f6e6e656374696f6e3a204b6565702d416c6976650d0a557365722d4167656e743a204170616368652d48747470436c69656e742f342e352e313320284a6176612f31312e302e3138290d0a4163636570742d456e636f64696e673a20677a69702c6465666c6174650d0a0d0a

It shows like there is a communication mismatch. I decoded the record, and it gives a http request:
GET /_opendistro/_security/sslinfo?pretty HTTP/1.1
Host: 127.0.0.1:8628
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.13 (Java/11.0.18)
Accept-Encoding: gzip,deflate

I think by enabling the openSSL, we need a https request instead.

Looks like root cause was:

https://github.com/opensearch-project/OpenSearch/pull/6331

I've updated comments on how I think we can get unblock https://github.com/opensearch-project/security/pull/2598#issuecomment-1491025215
Impacted PRs:

https://github.com/opensearch-project/security/pull/2598
https://github.com/opensearch-project/security/pull/2608
https://github.com/opensearch-project/security/pull/2613

On behalf of https://github.com/opensearch-project/OpenSearch/pull/6331, We have fixed the failure of OpenSSLTest by cherry pick @peternied's PR (#2062 ) . Because instead of using legacy settings, we have switched to use setting with new terminology for adding the 4th node in the above test. However, I do think that the change in core should also support legacy settings until the last release version of 2.x line. I will follow up with the core team to see if we/they need to do something for supporting these legacy terminologies in 2.x branch.
We can close this issue for now, once we merge #2598. And I will create a separate issue to track that if needed. Thanks for the support team!

authors: RyanL1997 cwperks peternied

repo: opensearch-project/security


text:
[BUG] Error while configuring JWT Authentication

I am trying to configure JWT Authentication for my Opensearch docker container which is on V2.7.0
How can one reproduce the bug?
Steps to reproduce the behavior:

Config file updated as follows:

jwt_auth_domain:
  http_enabled: true
  transport_enabled: true
  order: 0
  http_authenticator:
    type: jwt
    challenge: false
    config:
      signing_key: "VGhpc0lzQVRlc3ROYXRUb0JlVXNlZEFudGh3YXJl"
      jwt_header: "Authorization"
      jwt_url_parameter: null
      subject_key: "sub"
      roles_key: "roles"
      jwt_clock_skew_tolerance_seconds: 20
  authentication_backend:
    type: noop

What is the expected behavior?
[opensearch@59901e2344d3 tools]$ ./securityadmin.sh -cd ../securityconfig/ -icl -nhnv -cacert ../../../config/root-ca.pem -cert ../../../config/kirk.pem -key ../../../config/kirk-key.pem

Security Admin v7
Will connect to localhost:9200 ... done
Connected as "CN=kirk,OU=client,O=client,L=test,C=de"
OpenSearch Version: 2.7.0
Contacting opensearch cluster 'opensearch' and wait for YELLOW clusterstate ...
Clustername: opensearch-cluster
Clusterstate: YELLOW
Number of nodes: 1
Number of data nodes: 1
.opendistro_security index already exists, so we do not need to create one.
Populate config from /usr/share/opensearch/plugins/opensearch-security/securityconfig
ERR: Seems ../securityconfig/config.yml is not in OpenSearch Security 7 format: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "http_enabled" (class org.opensearch.security.securityconf.impl.v7.ConfigV7), not marked as ignorable (one known property: "dynamic"])
at [Source: (String)"{"_meta":{"type":"config","config_version":2},"config":{"dynamic":{"http":{"anonymous_auth_enabled":false,"xff":{"enabled":false,"internalProxies":"192\.168\.0\.10|192\.168\.0\.11"}},"authc":null}},"jwt_auth_domain":{"http_enabled":true,"transport_enabled":true,"order":0,"http_authenticator":{"type":"jwt","challenge":false,"config":{"signing_key":"VGhpc0lzQVRlc3ROYXRUb0JlVXNlZEFudGh3YXJl","jwt_header":"Authorization","jwt_url_parameter":null,"subject_key":"sub","roles_key":"roles","jwt_clo"[truncated 816 chars]; line: 1, column: 243] (through reference chain: org.opensearch.security.securityconf.impl.SecurityDynamicConfiguration["jwt_auth_domain"]->org.opensearch.security.securityconf.impl.v7.ConfigV7["http_enabled"])
ERR: Seems ../securityconfig/roles.yml is not in OpenSearch Security 7 format: java.io.FileNotFoundException: ../securityconfig/roles.yml (No such file or directory)
ERR: Seems ../securityconfig/roles_mapping.yml is not in OpenSearch Security 7 format: java.io.FileNotFoundException: ../securityconfig/roles_mapping.yml (No such file or directory)
ERR: Seems ../securityconfig/internal_users.yml is not in OpenSearch Security 7 format: java.io.FileNotFoundException: ../securityconfig/internal_users.yml (No such file or directory)
ERR: Seems ../securityconfig/action_groups.yml is not in OpenSearch Security 7 format: java.io.FileNotFoundException: ../securityconfig/action_groups.yml (No such file or directory)
ERR: Seems ../securityconfig/tenants.yml is not in OpenSearch Security 7 format: java.io.FileNotFoundException: ../securityconfig/tenants.yml (No such file or directory)
ERR: Seems ../securityconfig/nodes_dn.yml is not in OpenSearch Security 7 format: java.io.FileNotFoundException: ../securityconfig/nodes_dn.yml (No such file or directory)
ERR: Seems ../securityconfig/whitelist.yml is not in OpenSearch Security 7 format: java.io.FileNotFoundException: ../securityconfig/whitelist.yml (No such file or directory)

What is your host/environment?

OS: Ubuntu 20.04.5 LTS
Version 20.04.5
Plugins

Do you have any screenshots?
If applicable, add screenshots to help explain your problem.
Do you have any additional context?
Add any other context about the problem.

ERR: Seems ../securityconfig/config.yml is not in OpenSearch Security 7 format: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "http_enabled" (class org.opensearch.security.securityconf.impl.v7.ConfigV7), not marked as ignorable (one known property: "dynamic"])

Could you please attach your complete config.yml here (redacted for privacy if needed of course)? It seems like there might be some sort of miss in where fields are and where they are expected.

These issues are best discussed in the forums, so that other users experiencing similar issues can search through them as a knowledge base. I've taken the liberty to re-create this issue over there, please post your config.yaml there instead. Thanks!

authors: davidlago gvsrini


text:
Using jackson and jackson_databind version defined in OpenSearch (#1169)

Signed-off-by: Peng Huo penghuo@gmail.com
(cherry picked from commit 5073215f813bad5fdad0d5d37a20c7cb100296fd)
Check List

[ ] New functionality includes testing.

[ ] All tests pass, including unit test, integration test and doctest

[ ] New functionality has been documented.

[ ] New functionality has javadoc added
[ ] New functionality has user manual doc added

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #1178 (9d6ed58) into 2.2 (cf01179) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##                2.2    #1178   +/-   ##
=========================================
  Coverage     97.76%   97.76%
  Complexity     2880     2880
=========================================
  Files           276      276
  Lines          7077     7077
  Branches        447      447
=========================================
  Hits           6919     6919
  Misses          157      157
  Partials          1        1

Flag
Coverage Δ

sql-engine
97.76% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Ignore workbench test fail. It fail for a while https://github.com/opensearch-project/sql/commits/2.2.

authors: codecov-commenter penghuo

repo: opensearch-project/sql


text:
Add cpms shortname to ControlPlaneMachineSet

@damdo and I have been testing the ControlPlaneMachineSet a lot lately and typing controlplanemachineset every time is long. While I appreciate that adding shortnames can lead to ambiguity and we normally try to promote using fully groupified names, I'm expecting that cpms is a relatively unique short name that we are unlikely to duplicate in or around the near future, or at least, before I sort the upstream kubectl warning for ambiguous resources.
It would be nice to have this shortname for convenience sake

Had a discussion with @deads2k in slack. Short names, because of their nature of being condensed, have a high likelihood of clashing with others. Therefore we reserve them for objects that are going to be frequently interacted with by namespace admins. As this is a configuration API and won't be used frequently by most users, it doesn't warrant a short name.
/close

repo: openshift/api


text:
OSD-12962 Scaffolding for new VpcEndpointAcceptance controller

To help break apart this effort from a PR review standpoint, this PR is just for adding a new API for VpcEndpointAcceptance and does no work toward implementing it (no-op from a controller functionality perspective, just laying down a new API)

I realized these two functions: DefaultAVOLogger and DefaultAVORateLimiter need to be shared between the VpcEndpoint and VpcEndpointAcceptance controllers, so I pulled them out.
A bunch of the stuff in the config folder doesn't apply to boilerplated operators
Comment placeholders for the planned implementation (actual implementation to come in upcoming PRs) and commented out where it would normally be added to the controller-managed in main.go to be extra sure there are no negative effects on existing deployments of AVO.

	// Poll AWS for VPCE's in pendingAcceptance based on vpceAcceptance.spec.serviceIds

	// Setup a map of AWS Account IDs:Namespaces
	// List account.aws.managed.openshift.io's .spec.awsAccountId

	// Connect to the cluster in that namespace to see if it's a valid request

	// If valid, accept the VPCE connection

/lgtm

authors: mjlshen tonytheleg

repo: openshift/aws-vpce-operator


text:
updating client secret format

Due to the updated secret format from the CCO:
stringData:
  credentials: |-
    [default]
    type = access_key
    access_key_id: xxx
    access_key_secret: xxxx
kind: Secret
metadata:
  name: alibabacloud-credentials
  namespace: openshift-machine-api
type: Opaque

I have added some code to parse the secret from the INI format.
cc @elmiko @JoelSpeed

I will attempt to add a test for this function tomorrow.

cc @menglingwei

@JoelSpeed Thanks for the hints. I was in the middle of refactoring and adding tests and included some of your feedback.

cc @joelddiaz

@JoelSpeed @elmiko Would it be appropriate to write the secret to a temporary file, call os.SetEnv to the temporary file, and then use the Alibaba provider code?

Sounds a bit nasty tbh. That said, IIRC, AWS does something similar (https://github.com/openshift/cluster-api-provider-aws/pull/374/files), so perhaps we can get away with it. The only issue is that we must make sure we don't leak the files and fill up the disk

agreed with @JoelSpeed 's comment. if this is the most convenient way to get around this issue, then yeah making a file on disk seems reasonable. as Joel mentioned, we just have to be very careful about monitoring our writes.

Some kind of lock to make sure only this one function can set/unset the Alibaba env var, create the file (defer() cleaning up the file and unsetting the env var), and going through the SDK codepath should work.

@kwoodson i'm confused about this and #17, are both needed?

@elmiko no, these are two separate approaches. In this one, we manage the parsing of the INI file. in #17 we are doing the environment variable and temporary file parsing so that we can take advantage of Alibaba's provider library. I wanted to see which you and @JoelSpeed preferred.

I prefer 17, there's a few nits about positioning of defers, but otherwise LGTM on 17

thanks Kenny, that helps. i'm +1 with Joel, we can probably close this #17 is fine with everyone.

Closing in favor of #17

authors: JoelSpeed elmiko joelddiaz kwoodson

repo: openshift/cluster-api-provider-alibaba


text:
use the CVO to create the operator config

Stealing https://github.com/openshift/cluster-kube-apiserver-operator/pull/396 to add a default value for managementState so as to please the CRD validations

/lgtm
/retest


text:
refactor: to make find-builds rpm using brew API calls instead of com…

This PR did:

change using ThreadPool to invoke "brew list-tagged {tag} {latest} --type=image --quiet".format(tag=tag, latest=latest_option) indivitually to use koji/brew API call which is faster
move get_brew_build from brew.py to errata.py since I think this is more like a erratum API call.
remove  BrewTaggedRPMBuilds BrewTaggedImageBuilds etc entirely and also some of the unit test

still need to test from-diff and builds
[dev@a20e27f6cd18 elliott]$ cd /workspaces/elliott ; env PYTHONIOENCODING=UTF-8 PYTHONUNBUFFERED=1 /usr/bin/python3 /home/dev/.vscode-server/extensions/ms-python.python-2020.1.58038/pythonFiles/ptvsd_launcher.py --default --client --host localhost --port 34693 /workspaces/elliott/elliott -g openshift-3.11 find-builds -k rpm --json ./test
2020-01-17 12:13:37,628 INFO Data clone directory already exists, checking commit sha
2020-01-17 12:13:39,318 INFO https://github.com/openshift/ocp-build-data.git is already cloned and latest
2020-01-17 12:13:39,895 INFO Using branch from group.yml: rhaos-3.11-rhel-7
Generating list of rpms: Hold on a moment, fetching Brew builds
Gathering additional information rhaos-3.11-rhel-7: Brew buildinfo is required to continue
[**********************]
Gathering additional information rhaos-3.11-rhel-8: Brew buildinfo is required to continue
[]
[**********************]
The following 22 builds may be attached to an advisory:
 atomic-enterprise-service-catalog-3.11.165-1.git.1.18ddb6b.el7
 atomic-openshift-3.11.165-1.git.0.90cc173.el7
 atomic-openshift-cluster-autoscaler-3.11.165-1.git.1.2d1655b.el7
 atomic-openshift-descheduler-3.11.165-1.git.1.c0e4e57.el7
 atomic-openshift-dockerregistry-3.11.165-1.git.1.a7b8c4c.el7
 atomic-openshift-metrics-server-3.11.165-1.git.1.fb26703.el7
 atomic-openshift-node-problem-detector-3.11.165-1.git.1.5f77cf4.el7
 atomic-openshift-service-idler-3.11.165-1.git.1.09b7115.el7
 atomic-openshift-web-console-3.11.165-1.git.1.e2478bf.el7
 golang-github-openshift-oauth-proxy-3.11.165-1.git.1.ba3088b.el7
 golang-github-prometheus-alertmanager-3.11.165-1.git.1.6714094.el7
 golang-github-prometheus-node_exporter-3.11.165-1.git.1.eee22b2.el7
 golang-github-prometheus-prometheus-3.11.165-1.git.1.d9f256f.el7
 jenkins-2-plugins-3.11.1579107288-1.el7
 jenkins-2.204.1.1579106295-1.el7
 kibana-5.6.16-2.el7
 openshift-ansible-3.11.165-1.git.0.2b41335.el7
 openshift-enterprise-autoheal-3.11.165-1.git.1.18d1d97.el7
 openshift-enterprise-cluster-capacity-3.11.165-1.git.1.b7f7790.el7
 openshift-kuryr-3.11.165-1.git.1.1df6328.el7
 openshift-monitor-project-lifecycle-3.11.51-2.git.59.7b59e29.el7
 openshift-monitor-sample-app-3.11.51-1.git.5.f6d0188.el7
[dev@a20e27f6cd18 elliott]$ git st
On branch refactor
Changes not staged for commit:
[dev@a20e27f6cd18 elliott]$ ls
elliott           Jenkinsfile  OWNERS                requirements.txt     tests
elliottlib        LICENSE      README.md             rh_elliott.egg-info  tox.ini
functional_tests  MANIFEST.in  requirements-dev.txt  setup.py
[dev@a20e27f6cd18 elliott]$ git st
On branch refactor
[dev@a20e27f6cd18 elliott]$ ls
elliott           Jenkinsfile  OWNERS                requirements.txt     tests
elliottlib        LICENSE      README.md             rh_elliott.egg-info  tox.ini
functional_tests  MANIFEST.in  requirements-dev.txt  setup.py
[dev@a20e27f6cd18 elliott]$ cd /workspaces/elliott ; env PYTHONIOENCODING=UTF-8 PYTHONUNBUFFERED=1 /us
r/bin/python3 /home/dev/.vscode-server/extensions/ms-python.python-2020.1.58038/pythonFiles/ptvsd_laun
cher.py --default --client --host localhost --port 37097 /workspaces/elliott/elliott -g openshift-4.2
find-builds -k image
2020-01-17 12:14:57,949 INFO Data clone directory already exists, checking commit sha
2020-01-17 12:14:57,973 INFO Cloning config data from https://github.com/openshift/ocp-build-data.git
2020-01-17 12:15:01,954 INFO Using branch from group.yml: rhaos-4.2-rhel-7
Generating list of images: Hold on a moment, fetching Brew buildinfo
[*****************************************************************************************************
************************************************]
Generating build metadata: Fetching data for 144 builds
[*****************************************************************************************************
*******************************************]
Generating build metadata: Fetching data for 149 builds
[*****************************************************************************************************
************************************************]
[*****************************************************************************************************************************************************]
No builds needed to be attached.

@vfreex rebased, PTAL

PTAL

repo: openshift/elliott


text:
OTA-1211: api/v1beta1/hostedcluster_types: Add spec.updateService

XCMSTRAT-513 includes OTA-1185 pitching local update services in managed regions. However, HyperShift had (until this pull) been forcing ClusterVersion spec.upstream empty.  This pull is part of making the upstream update service configurable on HyperShift, so those clusters can hear about and assess known issues with updates in environments where the default https://api.openshift.com/api/upgrades_info update service is not accessible, or in which an alternative update service was desired for testing or policy reasons. This includes OTA-1185, mentioned above, but would also include any other instance of disconnected/restricted-network use. The alternative for folks who want HyperShift updates in places where api.openshift.com is inaccessible are:

Don't use an update service and manage that aspect manually.  But the update service declares multiple new releases each week, as well as delivering information about known risks/issues for local clusters to evalute their exposure.  That's a lot of information to manage manually, if folks decide not to plug into the existing update service tooling.
Run a local update service, and fiddle with DNS and X.509 certs so that packets aimed at api.openshift.com get routed to your local service . This one requires less long-term effort than manually replacing the entire update service system, but it requires your clusters to trust an X.509 Certificate Authority that is willing to sign certificates for your local service saying "yup, that one is definitely api.openshift.com".

The implementation is similar to e43807690c (#1954), where I added update-related status properties piped up via CVO -> ClusterVersion .status -> HostedControlPlane .status -> HostedCluster .status.  This commit adds a new spec property that is piped down via:

HostedCluster (management cluster) spec.updateService.
HostedControlPlane (management cluster) spec.UpdateService.
Cluster-version operator Deployment --update-service command line option (management cluster), and also ClusterVersion spec.upstream.
Cluster-version operator container (management cluster).

The CVO's new --update-service option is from openshift/cluster-version-operator#1035, and we're using that pathway instead of ClusterVersion spec, because we don't want the update service URI to come via the the customer-accessible hosted API. It's ok for the channel (which is used as a query parameter in the update-service GET requests) to continue to flow in via ClusterVersion spec.  I'm still populating ClusterVersion's spec.upstream in this commit for discoverability, although because --update-service is being used, the CVO will ignore the ClusterVersion spec.upstream value.
When the HyperShift operator sets this in HostedControlPlane for an older cluster version, the old HostedControlPlane controller (launched from the hosted release payload) will not recognize or propagate the new property.  But that's not a terrible thing, and issues with fetching update recommendations from the default update service will still be reported in the RetrievedUpdates condition with a message mentioning the update service URI if there are problems.  Although it doesn't look like we capture that condition currently:
$ grep -r 'ConditionType = "ClusterVersion' api/hypershift/v1beta1/
api/hypershift/v1beta1/hostedcluster_conditions.go:     ClusterVersionSucceeding ConditionType = "ClusterVersionSucceeding"
api/hypershift/v1beta1/hostedcluster_conditions.go:     ClusterVersionUpgradeable ConditionType = "ClusterVersionUpgradeable"
api/hypershift/v1beta1/hostedcluster_conditions.go:     ClusterVersionFailing ConditionType = "ClusterVersionFailing"
api/hypershift/v1beta1/hostedcluster_conditions.go:     ClusterVersionProgressing ConditionType = "ClusterVersionProgressing"
api/hypershift/v1beta1/hostedcluster_conditions.go:     ClusterVersionAvailable ConditionType = "ClusterVersionAvailable"
api/hypershift/v1beta1/hostedcluster_conditions.go:     ClusterVersionReleaseAccepted ConditionType = "ClusterVersionReleaseAccepted"

We could opt to collect it in follow-up work if issues occur frequently enough that dropping down to the ClusterVersion level to debug becomes tedious.
If, in the future, we grow an option to give the hosted CVO kubeconfig access to both the management and hosted Kubernetes APIs, we could drop --update-service and have the hosted CVO reach out and read this configuration off HostedControlPlane or HostedCluster directly.
I'm bumping v1alpha1 as described in 4af5ffa8e9 (#1954).  The recent 1ae3a3606b (#3527) suggests that the "v1alpha1 is a superset of v1beta1" policy remains current.
Fixes OTA-1211

[x] Subject and description added to both, commit and PR.
[x] Relevant issues have been referenced.
[ ] This change includes docs.
[ ] This change includes unit tests.

openshift/cluster-version-operator#1035 needs to land support for --upstream before the HostedControlPlane controller starts setting it via this pull.
/hold

PR pre-merge tested and passed, so
/label qe-approved

openshift/cluster-version-operator#1035 merged and I've updated here with 7d39a443d5aa367636c481bd560869c1fc44ae79 -> de4bcbe35e190cad3bb4be79de9da6fdc0f9bccd to use the --update-service flag it settled on.

I can launch clusters with this new HostedControlPlane controller via Cluster Bot and launch 4.16.0-0.ci,openshift/hypershift#3576 hypershift-hosted, but without building and launching my own HyperShift operator and updating the HostedCluster CRDs, I'm not sure how to test this pre-merge.

/hold cancel

/approve
/lgtm

authors: shellyyang1989 sjenning wking

repo: openshift/hypershift


text:
Revert "NO-JIRA: test/e2e: temporarily skip NTO machineconfig test"

Reverts openshift/hypershift#3683
Break has been reverted
https://amd64.ocp.releases.ci.openshift.org/#4.16.0-0.ci

/lgtm

authors: csrwng sjenning


text:
MON-1609: Bump kube-state-metrics to v2.0.0

Bump kube-state-metrics to v2.0.0.
/cc @paulfantom

/retest

/lgtm

/lgtm

/retitle Bug 1924446: Bump kube-state-metrics to v2.0.0

authors: dgrisonnet paulfantom simonpasquier

repo: openshift/kube-state-metrics


text:
cert-syncer: Fix events for not-found resources

When emitting an event concerning the absence of a resource, use the namespace and name that were used to try to get the resource, rather than the namespace and name from the result of trying to get the resource (which will both be blank since the resource did not exist).
Remove a superfluous err argument in the Warningf call made when removing the file for a non-existent secret.  The err argument has no corresponding placeholder in the format string; the extra argument, combined with using the empty result's namespace and name, resulted in messages like the following:
Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found)

As the error is always a "not found" error, it can be omitted.
Follow-up to #546.

pkg/operator/staticpod/certsyncpod/certsync_controller.go (sync): Omit err from "Removed file for secret" event.  Use the correct namespace and name for events concerning not-found resources.

/assign @soltysh

/lgtm

Rebased.

authors: Miciah wking

repo: openshift/library-go


text:
OLS-83: Added missing type hints into DocsSummarizer

Description
OLS-83: Added missing type hints into DocsSummarizer
Type of change

[x] Refactor
[ ] New feature
[ ] Bug fix
[ ] CVE fix
[ ] Optimization
[ ] Documentation Update
[ ] Configuration Update
[ ] Bump-up dependent library

Related Tickets & Documents

Related Issue #OLS-83

Checklist before requesting a review

[x] I have performed a self-review of my code.
[x] PR has passed all pre-merge test jobs.
[ ] If it is a core feature, I have added thorough tests.

/lgtm

/approve
/lgtm

authors: bparees onmete tisnik

repo: openshift/lightspeed-service


text:
OLS-1215: Refactored user feedback tests

Description
OLS-1215: Refactored user feedback tests
Type of change

[x] Refactor
[ ] New feature
[ ] Bug fix
[ ] CVE fix
[ ] Optimization
[ ] Documentation Update
[ ] Configuration Update
[ ] Bump-up dependent library
[ ] Bump-up library or tool used for development (does not change the final image)
[ ] CI configuration change
[ ] Konflux configuration change

Related Tickets & Documents

Related Issue #OLS-1215

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 73.43%. Comparing base (ad570c2) to head (fff59af).
Report is 12 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1889      +/-   ##
==========================================
- Coverage   73.46%   73.43%   -0.04%
==========================================
  Files          61       61
  Lines        2755     2755
==========================================
- Hits         2024     2023       -1
- Misses        731      732       +1

see 1 file with indirect coverage changes

/test e2e-ols-cluster-4-15

/lgtm

/approve

/override "Red Hat Konflux / lightspeed-service-on-pull-request"

/override "ci/prow/e2e-ols-cluster-4-16"
/override "ci/prow/e2e-ols-cluster-4-17"

authors: JoaoFula codecov-commenter tisnik