text:
Create Method dialog cannot create private properties in public properties folder.

When using the create method dialog, there is an issue when creating private properties.
If the properties folder is set to public, then if you try to create a private property using this dialog, it doesn't get created as private. I think the properties folder is set to public if you create a public property with this dialog? Not sure.
If I change the property folder access scope to "Not Specified" then it works fine. Private properties can be created with the dialog. I suggest that the properties folder be set to "Not Specified" always.

You are right the tool creates the "properties" folder with Public access scope, I'll change it to "Not Specified"

source: gharchive/issue

authors: MichaelAivaliotis MikaelHolmstrom

repo: opengds/OpenGDS

license: MIT

license_type: permissive

license_source: github-api


text:
Feature/use wfi cycles for mhpmcounter11

This PR replaces the functionality behind performance counter 11 from Cycles waiting for multiply to complete to Cycles waiting in WFI instruction

@szbieg seems this is now conflicting with the main branch after merging the dv changes. Could you rebase to main and update the PR?

the commits of this PR have been rebased and can now be merged to main

source: gharchive/pull-request


text:
add kurise-rollout imagepullsecrets support

Note on DCO:
If the DCO action in the integration test fails, one or more of your commits are not signed off. Please click on the Details link next to the DCO action for instructions on how to resolve this.
Checklist:

[ ] I have bumped the chart version according to versioning
[ ] I have updated the chart changelog with all the changes that come with this pull request according to changelog.
[ ] Any new values are backwards compatible and/or have sensible default.
[ ] I have signed off all my commits as required by DCO.

Changes are automatically published when merged to master. They are not published on branches.

/lgtm

/approve

authors: jingliming zmberg

repo: openkruise/charts


text:
Fix test to createShadowVolume property

Should fix https://github.com/openlayers/ol3-cesium/issues/329.

LGTM

when cesium is compiled, isn't createShadowVolume changed  to a smaller name?

Thx for the quick fix.

Thanks for the review.

authors: fredj gberaudo oterral

license_source: bigquery


text:
Remove displayProjection setting

displayProjection should be only set on individual controls, not on the map object. It's a bit too confusing that it doesn't do what it says, and it's only used in three controls: repeating something three times is not a very big deal.
I can roll a patch if others agree.

Could you explain in more detail your proposal. Now displayProjection property can be set inside controls, but if the map has a displayProjection and the control doesn't, then value of displayProjection for controls gets from map.displayProjection.

authors: drnextgis tmcw

repo: openlayers/openlayers


text:
GroundOverlay KMLs are not supported

As far as I can tell, GroundOverlays in KMLs are not supported. I found a pull request https://github.com/ericfong/openlayers/commit/888bacf65d2b6a551662ff4aa7896806e9465461, which is for OpenLayers 2, and which was never merged into the project. I feel like GroundOverlays are not an uncommon feature of KMLs and thus an implementation in OpenLayers would be extremely pertinent.
It seems like it would be similar to parsing Placemarks, which are supported, except the image is scaled to a particular extent on the map. I'll try to make a pull request, but does anybody have a desire for this feature?

Want to back this issue? Place a bounty on it! We accept bounties via Bountysource.

Google Earth Web Plugin is not a solution as it is being permanently deprecated. +1 for adding this feature to openlayers.

Yeah, the deprecation notice was issued in December. Although, I feel like Google may have something similar up their sleeve (based on WebGL, maybe).

If this is developped, it could be great to propose an enhancement on the gdal package to go to the latest version of ol3.
https://github.com/OSGeo/gdal/blob/11ab6f3132d025e9ccdee9d4dda817786cead5b3/gdal/swig/python/scripts/gdal2tiles.py#L1995

Was there any progress with support of GroundOverlay feature on Openlayers?

I would like to see this issue reopened. I am hoping that GroundOverlay support can be added.

authors: eric-g-97477 olegsv ponceta stevenpetryk tamarmot


text:
Retry Image Src Loading

Hey guys, thanks for all of your hard work on this and responsiveness to issues and questions. I think this is basically a feature request for a feature that I think was in OL2 and I think it's probably not too hard to implement something, but I'm just not that familiar with your large code base.
Anyways, I was wondering if you could add the ability to retry loading an image source multiple times if failing to load. Currently we are using ol.source.ImageStatic to add geo-rectified PNGs to the map, but sometimes they are not created yet when added. So we have manually check for them and wait for them to be created ourselves prior to adding the source or obviously we could reload the src at that point, either way it's a pain.
So basically I think it would be an awesome feature if we could pass in something like retry the source 3 times on failure (like 404 or something like that) and maybe wait 5 or 10 seconds in between attempts. Ideally I think both of those numbers would be configurable options you pass in and would be 0/disabled by default.
Any thoughts? Thanks again guys.

I think retries (number, defaults to 2) and retryDelay (number, defaults to 0) would make for nice source options.

Could also make sense to have retries default to 0 (I've got a couple use cases where some 4xx is expected).

I could see either way I guess. A default of 0 would work fine for me. A lot of tiles and images are pre-generated or generated on the fly in most cases I would think and that's true for me. I just have one case where it's generated on demand that this would be very useful.

I like the idea of adding a retries option, but so far my solution to similar problems has just been writing a custom tileLoadFunction. Here's a contrived example of timing out the request after 1s:
tileLoadFunction(imageTile, url)
    {
        var img = imageTile.getImage();

        var coords = imageTile.getTileCoord();
        coords = {x: coords[1], y: coords[2], z: coords[0]};
        coords.y = Math.pow(2, coords.z) + coords.y;

        var handle = setTimeout(() =>
            {
                console.warn('Tile timed out:', url);
                img.alt = "Tile timed out while loading";
                img.src = this.options.placeholder || "";
            }, 1000);

        $http.get('/some/special/image'), (error, response) =>
            {
                clearTimeout(handle);

                if(error)
                {
                    img.alt = JSON.stringify(error);
                    img.src = "";

                    return console.error("Error while getting tile!", error);
                } // end if

                // Convert our binary data to a blob
                var blob = new Blob([ response.data ], { type: response.contentType });
                img.src = window.URL.createObjectURL(blob);
            });
    } // end tileLoadFunction

It shouldn't be hard to turn my example into something with retry support. (And the code can probably be cleaned up a little.)

authors: Morgul sbowler tschaub


text:
Add/data features

Closes #9

TODO: Expand test coverage for the new endpoint

[ ]: all features
[ ]: error codes
[ ]: processing errors (55, 56, 59)

Left some notes in #118 in what is currently not tested (or even implemented?). But as integration tests pass against the PHP responses, I will call it done for now.

repo: openml/server-api


text:
怎么通过命令行访问Kubespider 进行直接下载

Ask your question here:
怎么通过调用命令行的方式，将下载连接 发送给 Kubespider，然后直接下载？

这个只能用curl，参考 https://github.com/opennaslab/kubespider/tree/main/docs/zh/user_guide/api_docs

我学习下，看看能不能通过手机触发

如果是iphone，可以：https://github.com/opennaslab/kubespider/blob/main/docs/zh/user_guide/tiktok_download_provider/README.md#友情提示
如果是安卓，可以使用telegrama发生消息触发

使用的是ios。计划用快捷指令

repo: opennaslab/kubespider


text:
OpenNEM %s requires Python 3.10 or greater

Hi, my virtual environment is python3.10.5，and when i execute 'alembic upgrade head' ,then print error 'requires python 3.10 or greater'. Someone say this is a bug for python 3.10. What can I do to  fix this problem?

Hi before I look at this and test with 3.10.5 you can make a quick fix by editing out these lines in opennem/__init__.py
https://github.com/opennem/opennem/blob/master/opennem/__init__.py#L10-L12
when you run python -V does it return the 3.10.5 version?

yes, i run python -V is 3.10.5

Hi before I look at this and test with 3.10.5 you can make a quick fix by editing out these lines in opennem/__init__.py
https://github.com/opennem/opennem/blob/master/opennem/__init__.py#L10-L12
and when i annotating code sys.version_info < (3, 10)，it's print error  'Could not read project meta. Likely virtualenv not activated'

but virtualenv has activated.

I think it might help if you send me just a history of all the commands you ran from beginning to end - we might be missing some minor detail here

I think it might help if you send me just a history of all the commands you ran from beginning to end - we might be missing some minor detail here
I have solve the problem of python3.10.  sys.path.append(my_path). And modified the code is not compatible with python 3.10, such as :
① AttributeError: module 're' has no attribute 'Match' --> change Match to match
② TypeError: unsupported operand type(s) for |: 'type' and 'NoneType' --> change str|None to str=None
③ ModuleNotFoundError: No module named 'sentry_sdk' --> need install sentry_sdk
④ ModuleNotFoundError: No module named 'zoneinfo' --> zoneinfo New in version 3.9, No matching distribution found for zoneinfo in 3.10
⑤ ModuleNotFoundError: No module named 'rich' -->  need install rich
⑥ ModuleNotFoundError: No module named 'geojson' --> need install geojson
⑦ ModuleNotFoundError: No module named 'geoalchemy2' --> need install geoalchemy2
⑧ ModuleNotFoundError: No module named 'validators' --> need install validators
when i solved those problem , i get error like this:

repo: opennem/opennem


text:
ODS-10 not dettected

File with settings does not return ODS-10
Testing D:\ODS1\TestCorpusV2\ODS-10\WithSettings.ods
APP-1: [INFO] Validating D:\ODS1\TestCorpusV2\ODS-10\WithSettings.ods.
APP-4: [INFO] Validation report for D:\ODS1\TestCorpusV2\ODS-10\WithSettings.ods.
DOC-3: mimetype [INFO] OpenDocument MIMETYPE application/vnd.oasis.opendocument.spreadsheet detected
VALID, no errors, no warnings and 1 info message found.

ODS_10 is now sub-document detection, will close this and open a new issue.

authors: carlwilson dewhattens

repo: openpreserve/odf-validator


text:
Added files for plot_qpe.py and plot_vad.py in pyart example gallery

I added MeteoSwiss files for the pyart example gallery.
See https://github.com/ARM-DOE/pyart/pull/1317 for more details

Great! Thanks @wolfidan !

repo: openradar/open-radar-data


text:
Add community basics

A new open repository should have:

License (MIT for code), Open Government 3.0 for data OR an explicit statement that the repo is not for public consumption.
Code of Conduct (see ORJ repo).
Github templates (see ORJ's .github) folder.

added

authors: Jamie-Atkinson arnau

repo: openregister/buyers-list-matching


text:
Provide a way to easily find correct versions alignments

What problem are you trying to solve?
As explained on Slack, https://rewriteoss.slack.com/archives/C01AB6L98TC/p1687767750280069:
We have an artifact with dependency on various OpenRewrite modules, this also reference Maven and Gradle plugins version in a property file (to be use to run the yaml/java recipes  in that artifact). Then I want to use that artifact from either the Maven or Gradle build plugin using the version of the plugins specified in the properties. but we need to know which versions of the plugins are compatible with the modules used in your artifact
Describe the solution you'd like
Have the Gradle/Maven Rewrite plugins versions aligned with the OpenRewrite core version (at least up to the minor).
Have you considered any alternatives or workarounds?
Yes but there is not a lot of other solutions
Additional context
Link are provided in the description
Are you interested in contributing this feature to OpenRewrite
Sure

Moved this issue to rewrite-docs, as we will likely end up documenting the compatible versions rather than aligning them.
Unless this earlier suggestion can help

we already capture the rewrite.version with each Maven plugin release for instance; could that be of some use here @ia3andy ?

@timtebeek is there a similar capture for gradle?

Also, using this solution is fixing our immediate problem (if we have the same for Gradle). Still I think having it in the doc to be consumed in a glimpse by users is a must have.

is there a similar capture for gradle?

I think this part does something similar for the Gradle plugin, although it's my first time seeing it. Hope it helps!

@timtebeek  Can't find the actual version on GH or in the zip, can you?

is there a similar capture for gradle?

I think this part does something similar for the Gradle plugin, although it's my first time seeing it. Hope it helps!

@timtebeek Can't find the actual version on GH or in the zip, can you?

I don't have the details there, but it looks like we might have defined the step, but are not yet referencing it anywhere. 🤔 @sambsnyd any idea why we have that generateVersionsProperties task defined but potentially unused?
For context: Andy is looking for a way to determine, from a given Maven/Gradle plugin version, what's the compatible OpenRewrite core version.

is there a similar capture for gradle?

I think this part does something similar for the Gradle plugin, although it's my first time seeing it. Hope it helps!

@timtebeek Can't find the actual version on GH or in the zip, can you?

I don't have the details there, but it looks like we might have defined the step, but are not yet referencing it anywhere. 🤔 @sambsnyd any idea why we have that generateVersionsProperties task defined but potentially unused?
For context: Andy is looking for a way to determine, from a given Maven/Gradle plugin version, what's the compatible OpenRewrite core version.

and I guess anyone using the plugins will need this info if they don't want to end up with breakages..

repo: openrewrite/rewrite-docs


text:
Data Picker Fails to Add Data Attribute for Creating Sampled Value Control Blocks

As a user of OpenSCD, I want to create Sampled Value Control Blocks. All steps work fine except for the final step, which involves adding Data Attributes. After clicking on the LDevice, we expect to see a list of LNodes. Then, by clicking on a LNode, a list of the supported DOs should appear. Finally, by clicking on each DO, all possible Data Attributes should be listed, and the user should be able to select them.
Currently, it is only possible to see a list of LNodes supported by each LDevice. From this point, the process does not proceed further, and the system does not display the subsequent levels, including the Data Attributes. Please see the image below:

The correct way to select multiple Data Attributes is as follows:

@JakobVogelsang can you give a status update of your investigation?

@Sander3003 I just remembered that the issue was related to the SCL file. The reason the data picker does not display further information, such as Do and DA, is that the required DO (Data Object) is not defined or supported for the IED and LDevice.

@DavoodSooran thanks, we can close the issue.

repo: openscd/open-scd


text:
rename some directories

What this PR does / why we need it:
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
fixes #<173>
Special notes for your reviewer:
Release note:

LGTM

LGTM

authors: kumarashit sfzeng xxwjj

repo: opensds/multi-cloud


text:
[BUG] Docker stack not running correctly when mounting config folder

I am trying to launch a stack with docker :
This is the docker-compose.yml
version: "3"

services:
  os01:
    image: opensearchproject/opensearch:1.1.0
    environment:
      OPENSEARCH_JAVA_OPTS: "-Xms1024m -Xmx1024m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      node.name: os01
      plugins.security.ssl.transport.pemkey_filepath: certificates/os01/os01.key # relative path
      plugins.security.ssl.transport.pemcert_filepath: certificates/os01/os01.pem
      plugins.security.ssl.http.pemkey_filepath: certificates/os01/os01.key
      plugins.security.ssl.http.pemcert_filepath: certificates/os01/os01.pem
      DISABLE_INSTALL_DEMO_CONFIG: "true"
      JAVA_HOME: /usr/share/opensearch/jdk
      bootstrap.memory_lock: "true" # along with the memlock settings below, disables swapping
      network.host: "0.0.0.0"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - ./opensearch.yml:/usr/share/opensearch/config/opensearch.yml
      - ./config.yml:/usr/share/opensearch/plugins/opensearch-security/securityconfig/config.yml
      - ./certs:/usr/share/opensearch/config/certificates
      - os-data1:/usr/share/opensearch/data
    ports:
      - 9200:9200

  dashboards:
    image: opensearchproject/opensearch-dashboards:1.1.0
    ports:
      - 5601:5601
    volumes:
      - ./certs:/usr/share/opensearch-dashboards/config/certificates:rw
      - ./opensearch-dashboards.yml:/usr/share/opensearch-dashboards/config/opensearch_dashboards.yml
    environment:
      DISABLE_INSTALL_DEMO_CONFIG: "true"

volumes:
  os-data1:

As soon as you launch that, you get this error
opensearch1-os01-1  | Disabling execution of install_demo_configuration.sh for OpenSearch Security Plugin
opensearch1-os01-1  | Enabling OpenSearch Security Plugin
opensearch1-os01-1  | sed: cannot rename /usr/share/opensearch/config/sed6FI3Gg: Device or resource busy
opensearch1-os01-1  | Exception in thread "main" org.opensearch.bootstrap.BootstrapException: java.nio.file.AccessDeniedException: /usr/share/opensearch/config/certificates/ca
opensearch1-os01-1  | Likely root cause: java.nio.file.AccessDeniedException: /usr/share/opensearch/config/certificates/ca
opensearch1-os01-1  | 	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
opensearch1-os01-1  | 	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
opensearch1-os01-1  | 	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
opensearch1-os01-1  | 	at java.base/sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:431)
opensearch1-os01-1  | 	at java.base/java.nio.file.Files.newDirectoryStream(Files.java:476)
opensearch1-os01-1  | 	at java.base/java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:300)
opensearch1-os01-1  | 	at java.base/java.nio.file.FileTreeWalker.next(FileTreeWalker.java:373)
opensearch1-os01-1  | 	at java.base/java.nio.file.Files.walkFileTree(Files.java:2840)
opensearch1-os01-1  | 	at org.opensearch.common.logging.LogConfigurator.configure(LogConfigurator.java:233)
opensearch1-os01-1  | 	at org.opensearch.common.logging.LogConfigurator.configure(LogConfigurator.java:142)
opensearch1-os01-1  | 	at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:367)
opensearch1-os01-1  | 	at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:182)
opensearch1-os01-1  | 	at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:173)
opensearch1-os01-1  | 	at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:99)
opensearch1-os01-1  | 	at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:140)
opensearch1-os01-1  | 	at org.opensearch.cli.Command.main(Command.java:103)
opensearch1-os01-1  | 	at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:139)
opensearch1-os01-1  | 	at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:105)
opensearch1-os01-1  | For complete error details, refer to the log at /usr/share/opensearch/logs/os-cluster.log
opensearch1-os01-1  | Killing performance analyzer process 15
opensearch1-os01-1  | OpenSearch exited with code 1
opensearch1-os01-1  | Performance analyzer exited with code 143

Any idea would be very helpful.
If you want to reproduce you will need to have the certs installed etc ...., just follow the steps in there : https://github.com/flavienbwk/opensearch-docker-compose
which is using the exact same docker-compose as I posted here.

This causes the same issue for swarm mode in Docker where opensearch.yml is mounted as a 'config' and thus is read-only. #1579 is the same problem.
The sed command attempts to modify the stock opensearch.yml to reflect features requested by environment variables. This is not appropriate (or in these cases is not possible) when someone has specified a custom opensearch.yml file.

Fixed in https://github.com/opensearch-project/opensearch-build/pull/1130

authors: manodupont peterzhuamazon scratchings

repo: opensearch-project/OpenSearch


text:
[BUG] The build fails with java.lang.OutOfMemoryError: Java heap space

Describe the bug
The java.lang.OutOfMemoryError: Java heap space  is back
Could not read standard output of command '/usr/lib/jvm/temurin-17-jdk-amd64/bin/java'.
java.lang.OutOfMemoryError: Java heap space
	at java.****/java.util.Arrays.copyOf(Arrays.java:3537)
	at java.****/java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:100)
	at java.****/java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:130)
	at org.gradle.process.internal.streams.ExecOutputHandleRunner.forwardContent(ExecOutputHandleRunner.java:67)
	at org.gradle.process.internal.streams.ExecOutputHandleRunner.run(ExecOutputHandleRunner.java:53)
	at org.gradle.internal.operations.CurrentBuildOperationPreservingRunnable.run(CurrentBuildOperationPreservingRunnable.java:42)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:49)
	at java.****/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.****/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.****/java.lang.Thread.run(Thread.java:833)

To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Plugins
Please list all plugins currently enabled.
Screenshots
If applicable, add screenshots to help explain your problem.
Host/Environment (please complete the following information):

CI

Additional context
https://build.ci.opensearch.org/job/gradle-check/16255/.

@reta  could you please add some details? Thanks

There aren't much - the builds fail, you could see the error in the description, everything else has to be investigated sadly ....

Is there any way we can enable -XX:+HeapDumpOnOutOfMemoryError on the builds to get more information in the logs?

It is there [1] but it is not available as Jenkins artifact (as it supposed to [2]), so the dump is not accessible
[1] https://github.com/opensearch-project/OpenSearch/issues/3973#issuecomment-1227886974
[2] https://github.com/opensearch-project/opensearch-build/pull/2515

authors: minalsha reta


text:
[Searchable Snapshot] Add Relevant Error handling for Restore API with remote_snapshot.

Description
This PR adds appropriate error handling in restore API path when user attempts to set index.store.type to remote_snapshot directly instead of using "storage_type": "remote_snapshot"
Related Issues
Resolves #11658

Check List

[x] New functionality includes testing.

[x] All tests pass

[x] New functionality has been documented.

[x] New functionality has javadoc added

[x] Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
[x] Commits are signed per the DCO using --signoff
[x] Commit changes are listed out in CHANGELOG.md file (See: Changelog)
[x] Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@Rishikesh1159 Looks like you have conflicts and failures. Can you please iterate to green?

@Rishikesh1159 Looks like you have conflicts and failures. Can you please iterate to green?

Done

testMultiGetWithNetworkDisruption_FailOpenEnabled

https://github.com/opensearch-project/OpenSearch/issues/10755

❌ Gradle check result for 7b511b4: FAILURE
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

https://github.com/opensearch-project/OpenSearch/issues/11875


text:
Add recovery chunk size setting

Description
Making recovery chunk size setting dynamic. This will be required to increase the recovery chunk size while non-remote to remote-store migration is happening. Default value is 512 KB.
Related Issues
Resolves [#14001]

Check List

[x] New functionality includes testing.

[x] All tests pass

[x] New functionality has been documented.

[x] New functionality has javadoc added

[ ] ~API changes companion pull request created.~
[ ] ~Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)~
[x] Commits are signed per the DCO using --signoff
[x] Commit changes are listed out in CHANGELOG.md file (See: Changelog)
[ ] ~Public documentation issue/PR created~

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Do we need an IT to verify that this setting works as expected?

Hi Ashish, there are existing ITs which decrease and increase the chunk_size (TruncatedRecoveryIT)

[org.opensearch.action.admin.cluster.node.tasks.ResourceAwareTasksTests.testTaskResourceTrackingDuringTaskCancellation](https://build.ci.opensearch.org/job/gradle-check/40297/testReport/junit/org.opensearch.action.admin.cluster.node.tasks/ResourceAwareTasksTests/testTaskResourceTrackingDuringTaskCancellation/)
    [org.opensearch.common.util.concurrent.QueueResizableOpenSearchThreadPoolExecutorTests.testResizeQueueDown](https://build.ci.opensearch.org/job/gradle-check/40297/testReport/junit/org.opensearch.common.util.concurrent/QueueResizableOpenSearchThreadPoolExecutorTests/testResizeQueueDown/)
    [org.opensearch.common.util.concurrent.QueueResizableOpenSearchThreadPoolExecutorTests.classMethod](https://build.ci.opensearch.org/job/gradle-check/40297/testReport/junit/org.opensearch.common.util.concurrent/QueueResizableOpenSearchThreadPoolExecutorTests/classMethod/)
    [org.opensearch.common.util.concurrent.QueueResizableOpenSearchThreadPoolExecutorTests.classMethod](https://build.ci.opensearch.org/job/gradle-check/40297/testReport/junit/org.opensearch.common.util.concurrent/QueueResizableOpenSearchThreadPoolExecutorTests/classMethod_2/)

Lots of flaky tests failing the build .

[org.opensearch.indices.store.IndicesStoreIntegrationIT.testShardActiveElseWhere {p0={"cluster.remote_store.state.enabled":"true"}}](https://build.ci.opensearch.org/job/gradle-check/40468/testReport/junit/org.opensearch.indices.store/IndicesStoreIntegrationIT/testShardActiveElseWhere__p0___cluster_remote_store_state_enabled___true___/)
[org.opensearch.search.SearchWeightedRoutingIT.testSearchWithWRRShardRouting](https://build.ci.opensearch.org/job/gradle-check/40468/testReport/junit/org.opensearch.search/SearchWeightedRoutingIT/testSearchWithWRRShardRouting/)
[org.opensearch.search.aggregations.bucket.MinDocCountIT.testStringScriptCountAsc {p0={"search.concurrent_segment_search.enabled":"false"}}](https://build.ci.opensearch.org/job/gradle-check/40468/testReport/junit/org.opensearch.search.aggregations.bucket/MinDocCountIT/testStringScriptCountAsc__p0___search_concurrent_segment_search_enabled___false___/)

3 flaky tests, retrying the build .

authors: astute-decipher gbbafna


text:
Star Tree Min and Max Value Aggregators

Description
The PR introduces the support for Min and Max Value Aggregators for Star Tree.
Built on top of: #14512
Related Issues
Resolves #14624
Check List

[ ] Functionality includes testing.
[ ] API changes companion pull request created, if applicable.
[ ] Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Please mark classes as non public if they are not needed beyond the scope of their package factory.


text:
[RW Separation] Add polling segment replication for search replicas

Description
Add segment replication polling for search replicas. This PR introduces segrep polling for search replicas at the existing refresh interval.  If the feature flag is enabled an async task from IndexService will trigger replication.
Related Issues
Related https://github.com/opensearch-project/OpenSearch/issues/4577
Check List

[x] Functionality includes testing.
[x] API changes companion pull request created, if applicable.
[x] Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

❌ Gradle check result for 173fcd9: FAILURE
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

https://github.com/opensearch-project/OpenSearch/issues/14327

❌ Gradle check result for 173fcd9:
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

https://github.com/opensearch-project/OpenSearch/issues/15692


text:
Limit RW separation to remote store enabled clusters and update recovery flow

Description
This PR includes multiple changes to search replica recovery to further decouple these shards from primaries.

Change to recover as empty store instead of peer. This will run a store recovery that syncs segments from remote store directly and eliminate any primary communication.
Remove search replicas from the in-sync allocation ID set and update routing table to exclude them from allAllocationIds.  This ensures primaries aren't tracking or validating the routing table for any search replica's presence.
Simplify RW separation by limiting to only remote store enabled clusters.  There are versions of the above changes that are still possible with primary based node-node replication but they require additional public api changes and I don't think we have the need at this time.

Related Issues
Resolves https://github.com/opensearch-project/OpenSearch/issues/15952
Check List

[x] Functionality includes testing.
[ ] API changes companion pull request created, if applicable.
[ ] Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Thanks @mch2 for the changes. On high level changes look good.
I have couple of clarifications:

Search-replicas are not tracked in ReplicationGroup, how are we ensuring search-replicas are not lagging behind? Is there separate tracking for search-replicas?
When primary turn red, would search-replica continue to serve, they don't know which copy to follow from remote anymore. It may break certain invariant in the code where search-replicas are still assigned without any active primary and would require code changes to handle this case.
Test node drops or full cluster restarts scenarios to ensure search-replicas are started automatically (without recovering from scratch) and no changes are needed in AsyncShardFetch logic.

Also just noticed, indices with auto-expand replicas shouldn't support search-replicas otherwise total copies can get more than no. of data nodes as well.

Thanks for taking a look @shwetathareja, you bring up great points.

Search-replicas are not tracked in ReplicationGroup, how are we ensuring search-replicas are not lagging behind? Is there separate tracking for search-replicas?

Not yet, reguar segrep has a check to fail lagging replicas based on their replication checkpoint (basically segment infos version + a timer) compared to the primary's active set of segments. We could implement something similar to fetch and compare only among the search replica group and remove any outlier.  I am thinking to start that we rely on a failure mechanism local to the shards that fail if a single download event takes too long without any progress.  Will add a separate task for this.

When primary turn red, would search-replica continue to serve, they don't know which copy to follow from remote anymore. It may break certain invariant in the code where search-replicas are still assigned without any active primary and would require code changes to handle this case.

SR would need to continue syncing up to the latest set uploaded by the dropped primary. Will add a test specifically for red cluster outside of the scale to zero case and see if we need any special handling.

Test node drops or full cluster restarts scenarios to ensure search-replicas are started automatically (without recovering from scratch) and no changes are needed in AsyncShardFetch logic.

ack will add some qa tests with this PR.

Also just noticed, indices with auto-expand replicas shouldn't support search-replicas otherwise total copies can get more than no. of data nodes as well.

Yeah, I will add a task to handle this.  We had brought this up before a while back, I was thinking to support this for search replicas within their own set of nodes if they exist and otherwise reject it, but I think blocking it outright makes sense to start.

When primary turn red, would search-replica continue to serve, they don't know which copy to follow from remote anymore. It may break certain invariant in the code where search-replicas are still assigned without any active primary and would require code changes to handle this case.

@shwetathareja, got some time to work on this.  I've added a test for this case to ensure a searcher is still searchable post primary failure.  First when there is no writer replica and the cluster turns red and second when there is a writer-replica and turns yellow & the new primary writes to the store.  This works without issue, however in the first case we should still be able to restore only writers if required and assign the primary to a new node, restore fails in this case because an open index with same name already exists in the cluster. , taking a look there.


text:
Adding precommit support for Windows

Signed-off-by: Sarat Vemulapalli vemulapallisarat@gmail.com
Description
Taking a stab at windows precommit in GHA
Issues Resolved
Closes https://github.com/opensearch-project/OpenSearch/issues/1448
Check List

[ ] New functionality includes testing.

[ ] All tests pass

[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[x] Commits are signed per the DCO using --signoff
[ ] Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #4676 (9465f53) into main (7b7ca98) will decrease coverage by 0.01%.
The diff coverage is n/a.

:exclamation: Current head 9465f53 differs from pull request most recent head 86b75cf. Consider uploading reports for the commit 86b75cf to get more accurate results

@@             Coverage Diff              @@
##               main    #4676      +/-   ##
============================================
- Coverage     70.59%   70.58%   -0.02%
+ Complexity    57619    57503     -116
============================================
  Files          4654     4654
  Lines        276978   276978
  Branches      40525    40525
============================================
- Hits         195542   195502      -40
+ Misses        65094    65041      -53
- Partials      16342    16435      +93

Impacted Files
Coverage Δ

.../java/org/opensearch/node/NodeClosedException.java
50.00% <0.00%> (-50.00%)
:arrow_down:

...ch/transport/ReceiveTimeoutTransportException.java
50.00% <0.00%> (-50.00%)
:arrow_down:

...cluster/coordination/PendingClusterStateStats.java
20.00% <0.00%> (-48.00%)
:arrow_down:

...adcast/BroadcastShardOperationFailedException.java
55.55% <0.00%> (-44.45%)
:arrow_down:

...rch/client/transport/NoNodeAvailableException.java
28.57% <0.00%> (-42.86%)
:arrow_down:

...h/action/ingest/SimulateDocumentVerboseResult.java
60.71% <0.00%> (-39.29%)
:arrow_down:

...java/org/opensearch/threadpool/ThreadPoolInfo.java
56.25% <0.00%> (-37.50%)
:arrow_down:

.../aggregations/support/values/ScriptLongValues.java
34.88% <0.00%> (-34.89%)
:arrow_down:

...opensearch/index/reindex/BulkByScrollResponse.java
48.38% <0.00%> (-29.04%)
:arrow_down:

...opensearch/snapshots/SnapshotMissingException.java
28.57% <0.00%> (-28.58%)
:arrow_down:

... and 485 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Expected: <9.4.0>
     but: was <9.3.0>

Will wait for your PR to be merged. @reta

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/3790/
CommitID: b37a0bb

[org.opensearch.cluster.routing.allocation.decider.DiskThresholdDeciderIT.testRestoreSnapshotAllocationDoesNotExceedWatermark](https://build.ci.opensearch.org/job/gradle-check/3790/testReport/junit/org.opensearch.cluster.routing.allocation.decider/DiskThresholdDeciderIT/testRestoreSnapshotAllocationDoesNotExceedWatermark/)
[org.opensearch.cluster.routing.allocation.decider.DiskThresholdDeciderIT.testRestoreSnapshotAllocationDoesNotExceedWatermark](https://build.ci.opensearch.org/job/gradle-check/3790/testReport/junit/org.opensearch.cluster.routing.allocation.decider/DiskThresholdDeciderIT/testRestoreSnapshotAllocationDoesNotExceedWatermark_2/)
[org.opensearch.cluster.routing.allocation.decider.DiskThresholdDeciderIT.testRestoreSnapshotAllocationDoesNotExceedWatermark](https://build.ci.opensearch.org/job/gradle-check/3790/testReport/junit/org.opensearch.cluster.routing.allocation.decider/DiskThresholdDeciderIT/testRestoreSnapshotAllocationDoesNotExceedWatermark_3/)
[org.opensearch.cluster.routing.allocation.decider.DiskThresholdDeciderIT.testRestoreSnapshotAllocationDoesNotExceedWatermark](https://build.ci.opensearch.org/job/gradle-check/3790/testReport/junit/org.opensearch.cluster.routing.allocation.decider/DiskThresholdDeciderIT/testRestoreSnapshotAllocationDoesNotExceedWatermark_4/)

authors: codecov-commenter saratvemulapalli


text:
[Backport] [1.x] Bumps Jackson from 2.4.1 to 2.4.2

Backport of https://github.com/opensearch-project/OpenSearch/pull/6129 to 1.x

I think we should wait for #6138 to avoid further conflicts.

authors: kotwanikunal reta


text:
Adding release notes for 2.0.0-rc1

Signed-off-by: Bharathwaj G bharath78910@gmail.com
Description
Adding release notes for 2.0.0-rc1
Issues Resolved
[List any issues this PR will resolve]
Check List

[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

:exclamation: No coverage uploaded for pull request base (2.0@d57c851). Click here to learn what that means.
The diff coverage is n/a.

:exclamation: Current head 823025b differs from pull request most recent head d501b42. Consider uploading reports for the commit d501b42 to get more accurate results

@@          Coverage Diff           @@
##             2.0     #138   +/-   ##
======================================
  Coverage       ?   83.36%
  Complexity     ?      553
======================================
  Files          ?       61
  Lines          ?     2050
  Branches       ?      146
======================================
  Hits           ?     1709
  Misses         ?      233
  Partials       ?      108

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d57c851...d501b42. Read the comment docs.

authors: bharath-techie codecov-commenter

repo: opensearch-project/asynchronous-search


text:
Release notes for 2.4

Description
Describe what this change achieves.
Issues Resolved
List any issues this PR will resolve, e.g. Closes [...].
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@d0794cd). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #197   +/-   ##
=======================================
  Coverage        ?   83.12%
  Complexity      ?      553
=======================================
  Files           ?       61
  Lines           ?     2050
  Branches        ?      146
=======================================
  Hits            ?     1704
  Misses          ?      238
  Partials        ?      108

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

authors: Arpit-Bandejiya codecov-commenter


text:
Fix import for IOUtils

Signed-off-by: Monu Singh msnghgw@amazon.com
Description
Fix imports
Issues Resolved
[List any issues this PR will resolve]
Check List

[ ] New functionality includes testing.

[ ] All tests pass

[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Duplicated


text:
remove hardcoded width for generate ppl button

Description
better to not hard code width. if font is large text gets truncated
Issues Resolved
[List any issues this PR will resolve]
Check List

[ ] New functionality includes testing.

[x] All tests pass, including unit test, integration test and doctest

[ ] New functionality has been documented.

[ ] New functionality has javadoc added
[ ] New functionality has user manual doc added

[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@joshuali925 I'm on a fix of the bootstrap failure.


text:
Update puppeteer flags

Signed-off-by: Rupal Mahajan maharup@amazon.com
Description
Added pipe option.
Issues Resolved
[List any issues this PR will resolve]
Check List

[ ] New functionality includes testing.

[ ] All tests pass, including unit test, integration test and doctest

[ ] New functionality has been documented.

[ ] New functionality has javadoc added
[ ] New functionality has user manual doc added

[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #460 (73af23c) into main (b0c5643) will increase coverage by 2.58%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##               main     #460      +/-   ##
============================================
+ Coverage     74.27%   76.85%   +2.58%
============================================
  Files            94       30      -64
  Lines          3984     1923    -2061
  Branches        641      412     -229
============================================
- Hits           2959     1478    -1481
+ Misses          891      442     -449
+ Partials        134        3     -131

Flag
Coverage Δ

dashboards-reports
76.85% <ø> (ø)

reports-scheduler
?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...r/routes/utils/visual_report/visualReportHelper.ts
88.37% <ø> (ø)

...ortsscheduler/model/OnDemandReportCreateRequest.kt

...reportsscheduler/action/GetReportInstanceAction.kt

...ortsscheduler/action/OnDemandReportCreateAction.kt

...nsearch/reportsscheduler/util/SecureIndexClient.kt

...opensearch/reportsscheduler/model/SearchResults.kt

...uler/resthandler/RestResponseToXContentListener.kt

...eportsscheduler/model/GetReportInstanceResponse.kt

...scheduler/resthandler/ReportInstanceRestHandler.kt

...tsscheduler/model/UpdateReportDefinitionRequest.kt

... and 55 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

authors: codecov-commenter rupal-bq

repo: opensearch-project/dashboards-reports


text:
[FEATURE][DocBot] README revamp

Is your feature request related to a problem?
Since starting DocBot we have made a lot of progress however our README.md file has not kept up. We need to describe what the DocBot project does succinctly and give an overview of it's parts.
What solution would you like?
Updated README.md file

Hi David, I'm happy to take this issue on. Can you assign it to me, please?
Thanks!

Aaron

authors: afrodidact dtaivpp

repo: opensearch-project/demos


text:
[DOC] [ml-commons] Add GPU support and prebuilt model to doc

What do you want to do?

[x] Request a change to existing documentation
[ ] Add new documentation
[ ] Report a technical problem with the documentation
[ ] Other

Tell us about your request. Provide a summary of the request and all versions that are affected.

We plan to support running NLP model on GPU ML node. In 2.5, we plan to support NVIDIA GPU and AWS Inferentia. We need to explain how to prepare GPU ML node in our doc.

We have built some popular text embedding models and user can use these models easily by specifying model name and version only. No effort needed to trace model and prepare model parameters. We should list these models in a table and add to ml-commons doc.

What other resources are available? Provide links to related issues, POCs, steps for testing, etc.

@ylwu-amzn: I'm already tracking against #2241. I'll close that one since this encompasses the new pre-built models as well.

GPU support PR https://github.com/opensearch-project/documentation-website/pull/2384
We are going to push prebuilt-model support to 2.6. Feel free to close this one. We can create a new issue for prebuilt model support in 2.6 release.

authors: Naarcha-AWS ylwu-amzn


text:
Accepts https as a property to set securityEnabled flag

Issue #, if available:

The error:

Build file '/local/home/dchanp/codebase/index-management/build.gradle' line: 441

* What went wrong:
Execution failed for task ':integTest'.
> `cluster{::integTest}` failed to wait for cluster health yellow after 40 SECONDS
    Unexpected end of file from server

Upon investigation I discovered that the integration tests triggered by opensearch-build do not pass security as an inline argument, instead they pass https (code-line). This cause the index-management tests in RC builds run with-security to fail with Not a valid SSL/TLS record: NotSslRecordException  which translates to mismatch between http vs https. The protocol is set here conditionally to https via securityEnabled flag via https://github.com/opensearch-project/index-management/blob/bc10724d39ced73e9f05ab94a554033d338784ce/build.gradle#L514

The culprit:
https://github.com/opensearch-project/index-management/blob/bc10724d39ced73e9f05ab94a554033d338784ce/build.gradle#L331

This PR adds an additional check to toggle securityEnabled flag by checking if https was passed.

The fix:

def securityEnabled = System.getProperty("security", "false") == "true" || System.getProperty("https", "false") == "true"

CheckList:

[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@bowenlan-amzn @r1walz mind reviewing this one?

repo: opensearch-project/index-management


text:
[Child Issue] Exploring Distributed Workload Generation

This is a subtask for Distributed Workload Generation Analysis and Scale Testing. For more details on this subtask, see this meta issue.
We're aware that OSB has limitations, but we don't know exactly under what circumstances they occur. For example, it's known that OSB can test single node and small multi node (3,4,5 etc.) OpenSearch clusters. However, when OSB tests a really large cluster (unclear on how large but let's just say 20 nodes), it tends to fail. To combat this, OSB has a feature called Distributed Workload Generation (DWG) which basically allows users to get over that hurdle of testing large 20 node clusters.
Questions

When using DWG against timeseries workloads, does OSB break up corpora by big chunks and assigns each client a chunk to ingest or do we use many clients and ingest sequentially (i.e. if client finishes first, it picks up the next line of the corpus)?
How many nodes can we use for DWG? Is it only max 3 nodes?
What parameters do we need to ensure that users include when using DWG?
Does running DWG on a single node cluster or small multi node clusters perform better than just using standard OSB (1 node) ?

Test Plan
Each test setup should be run 5 times to get consistent data.

[ ] 1. Run OSB without DWG: Run a single load driver with big5 workload against a large cluster (15+ node cluster). This will be our baseline to understand how OSB distributes the chunks of corpora with a single load driver.
[ ] 2. Run OSB with DWG: Run with two load drivers and one coordinator with big5 workload against a large cluster (15+ node cluster).
[ ] 3. How many DWG nodes can OSB run with:  Run with 3 load drivers, then 5 load drivers, and then try 10 load drivers against a large cluster (15+ node cluster).

Closing this for now. This is something we might look at in the future once we have a better understanding of scale testing. https://github.com/opensearch-project/opensearch-benchmark/issues/555

Reopening issue, since there is an intention to look into this in more detail after the scale-up investigation has concluded.

authors: IanHoang gkamat

repo: opensearch-project/opensearch-benchmark


text:
[Meta] OpenSearch Release Improvements

Coming from https://github.com/opensearch-project/opensearch-build/issues/3676 A Central release workflow to support OpenSearch public release and https://github.com/opensearch-project/opensearch-build/issues/2080,  following are the issues that can help improve the OpenSearch Release process. Once the following issues are achieved it would be easy to achieve the long term goal of one-click release or having an automated release tool that can release the OpenSearch bundle.
Please consider the following suggestions as areas for improvement in the OpenSearch release, aiming to enhance the efficiency and reduce the time and effort required for a release manager.
For more details about the OpenSearch release process, refer the release process guide RELEASE_PROCESS_OPENSEARCH.md
Enhancements

[x] Circuit Breaker for AUTOCUT issues
There are lot of AUTOCUT issues created and closed because of flaky tests, have a mechanism to reduce the number of AUTOCUT issues created.
Related Issue: https://github.com/opensearch-project/opensearch-build/issues/4171

[ ] Central workflow to execute pre-release promotion jobs.
Related issue: https://github.com/opensearch-project/opensearch-build/issues/4136
To have one single workflow to execute all the pre-release and all promotion jobs.

[ ] Proper Labelling for  AUTOCUT issues.
Have proper labels added to the created AUTOCUT issues, example like if the failure is per distribution add the distribution label to the created issues. Later it would be easy to filter the issues per label.

[ ] Consolidated Release notes automation
The process to gather the consolidated release notes and raise a PR is now done manually by a release manager, need an automation to generate and raise a PR.
Related Issue: https://github.com/opensearch-project/opensearch-build/issues/4009

[ ] Jenkins upgrade Cycle.
Upgrade the Jenkins, come up with a quarterly upgrade path based on released LTS version
Related Issue: https://github.com/opensearch-project/opensearch-ci/issues/333

[ ] Split Public Jenkins into multiple masters.
Related Issue: https://github.com/opensearch-project/opensearch-ci/issues/373

[ ] Incremental builds
Currently we are building all components sequentially and we have to rebuild all components to accommodate even small commit changes. This incremental build process will allow to rebuild only specific updated artifacts and save the unnecessary time on rebuilding other unchanged plugins. It call reduce the distribution bundle generating time from 1 hour to 10 minitues ideally.
Related Issue: https://github.com/opensearch-project/opensearch-build/issues/3922

[ ] Refactor usage of existing index.json used in the latest URL.
We plan to store the latest of each distribution type separately in different index.json files and retrieve the latest from the corresponding index.json that is applicable based on the distribution type specified in the url.
Related issue: https://github.com/opensearch-project/opensearch-build/issues/4151

[ ] OpenSearch Build repo versioning framework (Resolves the need of branching).
Related Issue: https://github.com/opensearch-project/opensearch-build/issues/2490

[ ] OpenSearch Playground with daily build artifact.
Related Issue: https://github.com/opensearch-project/opensearch-devops/issues/129

[ ] Integrate Validation mechanism with every OS and OSD build.
Related Issue: https://github.com/opensearch-project/opensearch-build/issues/3441

[ ] Mechanism to auto create the release branch for the plugin repos.
Related Issue: https://github.com/opensearch-project/opensearch-build/issues/4005

[ ] Introduce placeholder in manifests to allow skip the component or use the placeholder component for any other use-case.
Related Issue: https://github.com/opensearch-project/opensearch-build/issues/3707

[ ] Open source the signing mechanism
Related Issues: https://github.com/opensearch-project/opensearch-build/issues/118,
https://github.com/opensearch-project/opensearch-build/issues/3468

[ ] Once the RC is finalized, we run several tests on demand, having a central workflow that takes input of the RC numbers, runs all the test jobs sequentially and post the results in different mediums. This should include the benchmark tests as well.

[ ] One central workflow to trigger all promotion job for releasing. Have a fall back mechanisms in place.

[ ]  Option to choose types of promotion in case of any job failure. To is to make sure the promotion jobs are only incremental from failure.

Bugs

[ ] Merge the Jenkins Create/Close Github libraries to central library
Today we have multiple libraries for creating and closing the github issue which is adding some confusion and more effort to manage the libraries. Need to merge the logic into one library and use it for all Github actions.
closeBuildSuccessGithubIssue closeGithubIssue createBuildFailureGithubIssue createGithubIssue

[ ] S3 Upload fails randomly
Related Issues:
https://github.com/opensearch-project/opensearch-build/issues/3097
https://github.com/opensearch-project/opensearch-build-libraries/issues/319

[ ] Version Increment bug with OSD.
Related Issue: https://github.com/opensearch-project/opensearch-build/issues/3856

[ ] Validate POM Files before pushing to maven repo (nexus).
Related Issue:
https://github.com/opensearch-project/opensearch-build/issues/2928

[ ] Distribution build failure AUTOCUT issues do not provide detail information
Related Issue: https://github.com/opensearch-project/opensearch-build/issues/4155

[ ] Smaller Docker Images of OpenSearch and OpenSearch Dashboards.
Related Issue: https://github.com/opensearch-project/opensearch-build/issues/3853

[ ] RPM/DEB permissions update.
Related Issue: https://github.com/opensearch-project/opensearch-build/issues/3815

[ ] Move the logic that is used for build which is now part of groovy libs to python.
Related Issue: https://github.com/opensearch-project/opensearch-build/issues/2609

Documentation

[ ] GitHub Wiki for OpenSearch Build repo.
Related Issue: https://github.com/opensearch-project/opensearch-build/issues/3936

[ ] Release templates
As a release manager every time one has to type and update the release issue, it would be easy if we have any templates (like RC is ready, Integ tests failing, starting the release process, pending version increment PR’s etc) to use them to update the release issue. An automation later that should auto tag the repo owners/maintainers.

[Triage] We will be looking into this issue for enhancement.

authors: prudhvigodithi zelinh

repo: opensearch-project/opensearch-build


text:
[Feature/Extension] OBO Authenticator should inspect the issuer field and make sure it matches the current cluster

Description
OBO Authenticator should inspect the issuer field and make sure it matches the current cluster.

Closing this one due to the merge of #3117.

repo: opensearch-project/security


text:
[Backport 2.x] Integration tests framework

Description
Test framework that allows for declaring within the test class all
related users, roles, and other security primitives.  Emphasis on
hamcrest Matchers for straight forward to read, author, and
troubleshoot test cases.
Special Note
This is a giant pull request - I would highly recommend using the github.dev [view] for reviewing and providing feedback.  I'll also provide comments for the initial reviewers where I call out any deviations I've taken from what is in main
Issues Resolved

Resolves https://github.com/opensearch-project/security/issues/3230

Check List

[X] New functionality includes testing
[ ] New functionality has been documented
[X] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Tests with failures  - tracking burndown:

[ ] org.opensearch.security.DefaultConfigurationTests.shouldLoadDefaultConfiguration
[ ] org.opensearch.security.PointInTimeOperationTest.listPitSegmentsCreatedWithIndexAlias_positive
[ ] org.opensearch.security.PointInTimeOperationTest.listPitSegments_negative
[ ] org.opensearch.security.PointInTimeOperationTest.listPitSegments_positive
[ ] org.opensearch.security.PointInTimeOperationTest.listPitSegmentsCreatedWithIndexAlias_negative
[ ] org.opensearch.security.api.DashboardsInfoTest.testDashboardsInfoValidationMessage
[ ] org.opensearch.security.api.DashboardsInfoWithSettingsTest.testDashboardsInfoValidationMessageWithCustomMessage
[ ] org.opensearch.security.rest.AuthZinRestLayerTests.testAccessDeniedForUserWithNoPermissions
[ ] org.opensearch.security.rest.AuthZinRestLayerTests.testShouldAllowAtRestAndTransport
[ ] org.opensearch.security.rest.AuthZinRestLayerTests.testBackwardsCompatibility
[ ] org.opensearch.security.rest.AuthZinRestLayerTests.testShouldAllowAtRestAndBlockAtTransport
[ ] org.opensearch.security.rest.AuthZinRestLayerTests.testShouldBlockAccessToEndpointForWhichUserHasNoPermission
[ ] org.opensearch.security.rest.WhoAmITests.testWhoAmIWithGetPermissionsLegacy
[ ] org.opensearch.security.rest.WhoAmITests.testWhoAmIWithoutGetPermissions
[ ] org.opensearch.security.rest.WhoAmITests.testWhoAmIWithGetPermissions
[ ] org.opensearch.security.rest.WhoAmITests.testAuditLogSimilarityWithTransportLayer

Note; Moving this to draft while I'm making sure these tests are stable, I've added a bunch of debug only changes that should be reverted before merging, I'll pull this out of draft when its mergeable - but feel free to review it as it stands

With CrossCluster tests disabled additional tests to look into:

[ ] org.opensearch.security.SearchOperationTest.shouldUpdateTemplate_positive
[ ] org.opensearch.security.SearchOperationTest.shouldCreateIndexTemplate_positive

... waiting on additional results, will edit this comment as they come in from the latest [run]

OK - I'm pretty sure this is ready to merge, or near as ready as I'm going to try to push it, I've made a number of annotations to ignore flaky tests, we should revisit these and I'll add several tracking issues, along with disabling windows due to the network trouble.
Proof that this test collection is stable - 10x runs on JDK 11 / 17

https://github.com/opensearch-project/security/actions/runs/6305277168/job/17118384937?pr=3388

@RyanL1997 @DarshitChanpura could I get another review?

@DarshitChanpura & @RyanL1997 see [1] for the fast follow up items.

[1] https://github.com/opensearch-project/security/pull/3427


text:
Refactor AWSSigV4 auth to support different AWSCredentialProviders

Description
Refactored sigv4auth support to use credential Providers.
Check List

[ ] New functionality includes testing.

[ ] All tests pass, including unit test, integration test and doctest

[ ] New functionality has been documented.

[ ] New functionality has javadoc added
[ ] New functionality has user manual doc added

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #1389 (dfe1b83) into 2.x (a32bcbe) will decrease coverage by 0.10%.
The diff coverage is n/a.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@             Coverage Diff              @@
##                2.x    #1389      +/-   ##
============================================
- Coverage     98.38%   98.28%   -0.10%
+ Complexity     3698     3513     -185
============================================
  Files           345      316      -29
  Lines          9113     8531     -582
  Branches        585      535      -50
============================================
- Hits           8966     8385     -581
+ Misses          142      141       -1
  Partials          5        5

Flag
Coverage Δ

sql-engine
98.28% <ø> (-0.10%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...etheus/authinterceptors/AwsSigningInterceptor.java

...l/prometheus/storage/PrometheusStorageFactory.java

...h/sql/prometheus/storage/PrometheusMetricScan.java

...rage/querybuilder/TimeRangeParametersResolver.java

...metheus/storage/PrometheusMetricDefaultSchema.java

.../planner/logical/rules/MergeFilterAndRelation.java

...ql/prometheus/storage/PrometheusStorageEngine.java

...us/storage/querybuilder/StepParameterResolver.java

...rage/querybuilder/SeriesSelectionQueryBuilder.java

...thinterceptors/BasicAuthenticationInterceptor.java

... and 19 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

authors: codecov-commenter vamsi-amazon

repo: opensearch-project/sql


text:
[DOCS] Add terraform example for using opensearch_channel_configuration

There's currently no usage example for resource opensearch_channel_configuration. I would be useful to include it on the OpenSearch terraform module documentation.
resource "opensearch_channel_configuration" "sns_channel_configuration" {
  body = <<EOF
{
  "id": "sample-sns-id",
  "name": "sample-name",
  "config": {
    "name": "Sample Sns Channel",
    "description": "Sample chime description",
    "config_type": "sns",
    "is_enabled": true,
    "sns": {
      "topic_arn": "arn:aws:sns:us-east-1:123456789012:MyTopic",
      "role_arn": "arn:aws:iam::123456789012:role/MyRole"
    }
  }
}
EOF
}

Hi @prudhvigodithi , can u assign this to me?

The docs are missing here:
https://registry.terraform.io/providers/opensearch-project/opensearch/latest/docs/resources/channel_configuration
Need to run this cli to add the docs:
https://github.com/hashicorp/terraform-plugin-docs

The example above only partially worked for me... after applying the plan wanted to make further changes each time, here is what worked in the end:
resource "opensearch_channel_configuration" "name_here_slack" {
  body = <<EOF
{
  "config_id": "name-here",
  "config": {
    "name": "Name Here",
    "description" : "Description here",
    "config_type" : "slack",
    "is_enabled" : true,
    "slack": {
      "url": "https://slack.webhook.url.com/ddsfsdfsddsffsd"
    }
  }
}
EOF
}

The difference being that "name" isn't needed at the top level and id should be config_id. see the API docs for full reference https://opensearch.org/docs/latest/observing-your-data/notifications/api/

authors: SkollRyu joesome-git martinrw peterzhuamazon


text:
Use absolute URLs for links to the spec

I just submitted a PR to fix this on the v2.12 branch. This is to do the same on master, and so when we next cut a release, we have a simple change to make (swap master for v...). Thanks!

@shalako @duglin @vaikas-google @pmorie @angarg12
Same again, this time on master!

Relative links won't work? I'm wondering if we can continue the having the CF docs link to each other, or if we have to start linking out to the OSBAPI docs.

So this change will make the CF docs link out to the repository. We could
consider just linking directly out to the spec.md from the side menu and
not doing any syncing at all.
On Mon, 19 Jun 2017 at 19:12, Shannon Coen notifications@github.com wrote:

Relative links won't work? I'm wondering if we can continue the having the
CF docs link to each other, or if we have to start linking out to the
OSBAPI docs.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/openservicebrokerapi/servicebroker/pull/238#issuecomment-309521578,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AG7UzKbd3da-QXeVCTslfj8xoGjfURsyks5sFroXgaJpZM4N-lXm
.

I'm confused as to why relative links don't work.

This change will break any other fork/branch, right?

Good point @duglin, didn't think about that. This isn't an acceptable solution. Closing these PRs, will research another way!

I'm not 100% sure what issue this was trying to resolve, but I get the sense it was related to other issues we've seen where the CF docs would be better served by having some kind of pre-processor in place to tweak things to deal with CF specific needs.  Is that correct?

Yep, that is correct!
On Mon, 19 Jun 2017 at 19:25, Doug Davis notifications@github.com wrote:

I'm not 100% sure what issue this was trying to resolve, but I get the
sense it was related to other issues we've seen where the CF docs would be
better served by having some kind of pre-processor in place to tweak things
to deal with CF specific needs. Is that correct?
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/openservicebrokerapi/servicebroker/pull/238#issuecomment-309525164,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AG7UzILufcPfYekU9OVC8y5HuE3ifs4Dks5sFr0ugaJpZM4N-lXm
.

authors: duglin mattmcneeney shalako

repo: openservicebrokerapi/servicebroker


text:
OSM CLI emits validation error when using --osm-namespace option

While trying to run the following command
./osm install --osm-namespace osm-aaa --mesh-name osm-aaa

Got this error:
Error: rendered manifests contain a resource that already exists. Unable to continue with install: ClusterRole "osm-aaa" in namespace "" exists and cannot be imported into the current release: invalid ownership metadata; annotation validation error: key "meta.helm.sh/release-namespace" must equal "osm-aaa": current value is "osm-system-aaa"

Version ($ osm version)
Version: dev; Commit: f5499fbc19f86bc15937f58f8f748174eeb4145d; Date: 2020-10-27-14:23

Affected area (please mark with X where applicable):

Install                    [X]
SMI Traffic Access Policy  [ ]
SMI Traffic Specs Policy   [ ]
SMI Traffic Split Policy   [ ]
Permissive Traffic Policy  [ ]
Ingress                    [ ]
Egress                     [ ]
Envoy Control Plane        [ ]
CLI Tool                   [ ]
Metrics                    [ ]
Certificate Management     [ ]
Sidecar Injection          [ ]
Logging                    [ ]
Debugging                  [ ]
Tests                      [ ]
CI System                  [ ]

Expected behavior:
Steps to reproduce the bug (as precisely as possible):
How was OSM installed?:
Anything else we need to know?:
Environment:

OSM version (use osm version):
Kubernetes version (use kubectl version):
Size of cluster (number of worker nodes in the cluster):
Others:

My guess is that this issue arises when osm is installed with one namespace, the controller gets cleaned up but some resources are leaked, then those resources cause a conflict when installing with a new namespace.
It would be good if we could modify osm mesh uninstall to be able to clean up these resources even if there is no osm-controller present.

Attempted to reproducer the issue based on the scenario explained above but was unable to. I verified with @draychev too and he is no longer seeing the issue as well.

Attempted to reproducer the issue based on the scenario explained above but was unable to. I verified with @draychev too and he is no longer seeing the issue as well.

authors: draychev ksubrmnn snehachhabria

repo: openservicemesh/osm


text:
messaging: introduce message broker

Description:
Introduces message broker functionality to
handle messaging in OSM control plane. The broker
will handle proxy update events and k8s events
required by other components within the control plane.
The broker comprises of a workqueue to process events
and pub-sub instances to implement the messaging
channels. The message broker will replace the global
pub-sub functionality.
Testing done:

100% unit test coverage
Verified fixes for #4167 with the use of
message broker

Required by #4167

Affected area:

Functional Area

New Functionality
[X]

Control Plane
[X]

Please answer the following questions with yes/no.

Does this change contain code from or inspired by another project? no

Did you notify the maintainers and provide attribution?

Is this a breaking change? no

Codecov Report

Merging #4210 (d204402) into main (a84e325) will increase coverage by 0.17%.
The diff coverage is 100.00%.

:exclamation: Current head d204402 differs from pull request most recent head 9de645b. Consider uploading reports for the commit 9de645b to get more accurate results

@@            Coverage Diff             @@
##             main    #4210      +/-   ##
==========================================
+ Coverage   69.93%   70.11%   +0.17%
==========================================
  Files         212      214       +2
  Lines       11634    11704      +70
==========================================
+ Hits         8136     8206      +70
  Misses       3446     3446
  Partials       52       52

Flag
Coverage Δ

unittests
70.11% <100.00%> (+0.17%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

pkg/messaging/broker.go
100.00% <100.00%> (ø)

pkg/messaging/workqueue.go
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a84e325...9de645b. Read the comment docs.

authors: codecov-commenter shashankram


text:
injector: rename iptables chains for clarity

Description:
Renames the iptables chains and prefixes the
custom chains with OSM_ to indicate these
chains are owned by OSM.
It renames the custom chains as follows:
PROXY_INBOUND -> OSM_PROXY_INBOUND
PROXY_OUTPUT -> OSM_PROXY_OUTBOUND
PROXY_IN_REDIRECT -> OSM_PROXY_IN_REDIRECT
PROXY_REDIRECT -> OSM_PROXY_OUT_REDIRECT

Testing done:

CI

Affected area:

Functional Area

Sidecar Injection
[X]

Please answer the following questions with yes/no.

Does this change contain code from or inspired by another project? no

Did you notify the maintainers and provide attribution?

Is this a breaking change? no

Codecov Report

Merging #4379 (4222fb2) into main (332b52b) will increase coverage by 0.07%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #4379      +/-   ##
==========================================
+ Coverage   69.21%   69.29%   +0.07%
==========================================
  Files         212      212
  Lines       14450    14455       +5
==========================================
+ Hits        10002    10016      +14
+ Misses       4396     4387       -9
  Partials       52       52

Flag
Coverage Δ

unittests
69.29% <100.00%> (+0.07%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

pkg/catalog/fake.go
33.33% <ø> (+0.18%)
:arrow_up:

pkg/k8s/mock_controller_generated.go
0.00% <ø> (ø)

pkg/k8s/types.go
100.00% <ø> (ø)

pkg/envoy/cds/response.go
82.35% <100.00%> (ø)

pkg/envoy/lds/response.go
76.00% <100.00%> (ø)

pkg/injector/init_container.go
100.00% <100.00%> (ø)

pkg/injector/iptables.go
100.00% <100.00%> (ø)

pkg/k8s/client.go
93.01% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6d99e8a...4222fb2. Read the comment docs.


text:
Art-tools bump

Bump art-tools to incorporate

https://github.com/openshift-eng/art-tools/pull/41
https://github.com/openshift-eng/art-tools/pull/45
https://github.com/openshift-eng/art-tools/pull/46
https://github.com/openshift-eng/art-tools/pull/49
https://github.com/openshift-eng/art-tools/pull/24

/lgtm

authors: ashwindasr thegreyd

repo: openshift-eng/aos-cd-jobs


text:
Revert "Clarify access for the admin user"

Reverts openshift-evangelists/vagrant-origin#18

Jorge  (PM for this is on vacation this week) and will review this as soon as he returns!
thanks, Gareth

@dmueller2001 no problem. Thanks for this setup, it's super handy for experimenting locally with OpenShift.

I close this PR, as the one done by @garethr was fine, and was already merged.
@garethr Just FYI. For now, there is no identity provider set up, that means that user admin will be with role cluster-admin, any other user you type (non existing user) will be regular user. Also, password is anything, as it will not check it.
We'll probably change this to use htpasswd for user identity in the future.
Thanks for your comments, and if you find any other issue, feel free to contribute or report.

authors: dmueller2001 garethr jorgemoralespou

repo: openshift-evangelists/vagrant-origin


text:
Add terminationMessagePolicy: FallbackToLogsOnError to fix observability-termination-policy tnf test

build-depends: https://github.com/dci-labs/dallas-pipelines/pull/965

SUCCESS https://www.distributed-ci.io/jobs/6ecd3c0e-dda4-4063-94b1-4d4379247381/jobStates
from Add terminationMessagePolicy: FallbackToLogsOnError to fix observability-termination-policy tnf test #30

There are still two containers failing in this test, I'll recheck and rerun the job:
 "NonCompliantObjectsOut": [
    {
      "ObjectType": "Container",
      "ObjectFieldsKeys": [
        "Reason For Non Compliance",
        "Namespace",
        "Pod Name",
        "Container Name"
      ],
      "ObjectFieldsValues": [
        "TerminationMessagePolicy is not FallbackToLogsOnError",
        "example-cnf",
        "testpmd-app-77bb9c5c85-pkj8p",
        "testpmd"
      ]
    },
    {
      "ObjectType": "Container",
      "ObjectFieldsKeys": [
        "Reason For Non Compliance",
        "Namespace",
        "Pod Name",
        "Container Name"
      ],
      "ObjectFieldsValues": [
        "TerminationMessagePolicy is not FallbackToLogsOnError",
        "example-cnf",
        "testpmd-app-77bb9c5c85-t6j5r",
        "testpmd"
      ]
    }
  ]

In the current job that is being executed, which is about to finish, observability-termination-policy test is passing, you can see it here: https://www.distributed-ci.io/jobs/359b6279-5ef8-40ae-9240-136581e9b9b9/tests
The change is ready for review.

repo: openshift-kni/example-cnf


text:
[KNI][release-4.14] snyk: update ignore list

The linter is now complaining about an old dep,
for the time being (and likely for the mid term)
we will ignore the warning. cobra is an indirect dep anyway.
Signed-off-by: Francesco Romani fromani@redhat.com
(cherry picked from commit 68523f6f46381645d937296b4abf644ed7bfbddf)

/lgtm

authors: ffromani shajmakh

repo: openshift-kni/scheduler-plugins


text:
adding backup procedure

backup procedure
All CRs needed for the backup procedure have been added.

/ok-to-test

repo: openshift-kni/telco-reference


text:
log was calling function without parens

Found this while fixing another branch.

Coverage remained the same at 40.97% when pulling 2e5bada788732d1de9822f6c124c75fb070bf3b5 on fix-test-failure into 0050a51a8cd7f635615d9b1a5d08916aa5451d5e on master.

authors: coveralls jmrodri

repo: openshift/ansible-service-broker


text:
OCPBUGS-19358: Allow GIT_LFS_SKIP_SMUDGE environment variable

Since introducing the 'git-lfs' package into the build container, we have had customers that have an issue with the git-lfs automatically being run and would like the option to disable this functionality via the 'GIT_LFS_SKIP_SMUDGE' environment variable.

/jira refresh

/retest

/assign @adambkaplan

/retest

/skip

/retest

/retest

/assign @spadgett

/assign @bparees

/unassign @spadgett

/approve

/cherry-pick release-4.14

authors: bparees coreydaley

repo: openshift/api


text:
Contributing guide

/assign @gabemontero
fyi @openshift/sig-developer-experience @nalind

/retest
ping @coreydaley @gabemontero

@coreydaley @gabemontero @bparees updated based on feedback & squashed

LGTM

I'll defer on adding the label to allow @bparees and @coreydaley to chime in

lgtm

A few small nits, then I think we can give it the old lgtm

/lgtm

authors: adambkaplan bparees coreydaley gabemontero

repo: openshift/builder


text:
Add MovePrivateIP and its OpenStack implementation

Up until now when an IP is failed over from one node to another, CloudPrivateIPConfigController will call ReleasePrivateIP with the node being failed over and AssignPrivateIP with the new node.
This is far from ideal for OpenStack platform. In this case in addition to adding the IP to the allowedAddressPairs on the node's port, we also create a port in Neutron to act as a reservation of that IP, so that it won't be taken by another node being created. With traditional model on every failover the reservation port would get removed, making a race condition "stealing" the IP possible. It's also problematic if user assigned a Neutron floating IP to that reservation port to have a predictible SNAT address for EgressIP traffic going outside the cluster. On failover such association would be lost and manual intervention would be needed to assign FIP to new reservation port.
This commit introduces an optional CloudProviderWithMoveIntf interface that has MovePrivateIP operation to be called on IP fail over. This operation is supposed to move the IP assignment from one node to another, but doesn't require deleting cloud resources if not necessary. The required logic is added to CloudPrivateIPConfigController to call MovePrivateIP on IP fail over if plugin implements the CloudProviderWithMoveIntf interface.
An implementation of MovePrivateIP is added to the OpenStack plugin, that will remove the IP from old node's allowedAddressPairs, add the IP to the new node's allowedAddressPairs, but won't touch the Neutron port that serves as a reservation of that IP.

I think I addressed all of the comments. I still have an issue with ovn-kubernetes, where CloudPrivateIPConfig is getting deleted after a successful move operation. I'm not sure if this isn't an ovn-kubernetes bug and I'll be digging that.

I fixed ovn-kubernetes issue here: https://github.com/ovn-org/ovn-kubernetes/pull/3403. Please take a look.

/retest

@kyrtapz, @andreaskaris: https://github.com/ovn-org/ovn-kubernetes/pull/3403 is merged, so I think we can reconsider this one.

/lgtm

authors: dulek kyrtapz


text:
test/e2e/destroycluster: fix dropped errors

This fixes two dropped err variables in test/e2e/destroycluster.

/lgtm

/ok-to-test

authors: alrs dgoodwin staebler

repo: openshift/hive


text:
update library-go

The version of github.com/openshift/library-go that Hive is using has a dependency on bitbucket.com/ww/goautoneg, which is dead. The latest version of library-go uses github.com/munnerz/goautoneg instead.
This is intended to fix problems when running test verfiy, which tries to download the goautoneg module.
Generating deepcopy funcs
F0918 16:15:47.256381   19839 main.go:82] Error: Failed making a parser: unable to add directory "github.com/openshift/hive/pkg/apis/hive/v1": unable to import "github.com/openshift/hive/pkg/apis/hive/v1": go/build: importGo github.com/openshift/hive/pkg/apis/hive/v1: exit status 1
go: github.com/openshift/installer@v0.9.0-master.0.20200817183917-ab4e753ff30b requires
	github.com/openshift/library-go@v0.0.0-20200324092245-db2a8546af81 requires
	bitbucket.org/ww/goautoneg@v0.0.0-20120707110453-75cd24fc2f2c: reading https://api.bitbucket.org/2.0/repositories/ww/goautoneg?fields=scm: 404 Not Found

/assign @dgoodwin
/cc @akhil-rane

This PR requires #1143 to merge first so that the build image uses go 1.15.

/lgtm
/retest

The vendoring has been fixed.

/test verify

/lgtm

authors: dgoodwin staebler


text:
bundle-gen: Validate image

Validate that the image indicated by --image-repo and --image-tag-override (both of which may be defaulted/computed) exists in the repo.

We only validate quay.io/* images.
You may bypass validation by specifying the new --skip-image-validation flag.

HIVE-2202

/assign @lleshchi

/hold pending discussion

/hold cancel

/lgtm


text:
OCPBUGS-10125: UPSTREAM: 121: Add missing Kubernetes deps

https://github.com/kubernetes-sigs/ibm-vpc-block-csi-driver/pull/121
Add k8s.io/dynamic-resource-allocation and k8s.io/pod-security-admission to go.mod to be able to resolve it deterministically. Even though these dependencies are not really used by the CSI driver.
"go list -mod=readonly -m all" failed with:
go: k8s.io/dynamic-resource-allocation@v0.0.0: invalid version: unknown revision v0.0.0 go: k8s.io/pod-security-admission@v0.0.0: invalid version: unknown revision v0.0.0
@openshift/storage

/jira refresh

repo: openshift/ibm-vpc-block-csi-driver


text:
Fix report creation for CRs with no diffs

Recent changes in the compare command, changed the
value of "diff output" for diff entries that don't contain
any diffs from "None" to  "" (empty string). This Change
was done only in the Json/Yaml Output, the change was
not updated in the report creator, and resulted in
creation of failure test cases for CRs that don't
include diffs. This commit fixes the issue, by updating
the report creator.
Also in separate commit:
Added a check for each test that ensures that the report creator is using  the latest format of the JSON output  of the compare command, ensuring the report creator is up to date with the command.
To support this case a testsutils package was created including test code that was located in the compare package, and now moved to the new package to allow usage in the report creator.

/lgtm
/approve

/lgtm
/approve

repo: openshift/kube-compare


text:
Replaces helm chart with an example set of YAML manifests.

Description
Replaces helm chart with an example set of YAML manifests.
Type of change

[X] Refactor
[ ] New feature
[ ] Bug fix
[ ] CVE fix
[ ] Optimization
[X] Documentation Update
[ ] Configuration Update
[ ] Bump-up dependent library

Related Tickets & Documents
N/A
Checklist before requesting a review

[X] I have performed a self-review of my code.
[ ] PR has passed all pre-merge test jobs.
[ ] If it is a core feature, I have added thorough tests.

Testing
I deployed this set of manifests into a cluster when I was testing insights operator integration, so it was enough to allow me to use the Gradio UI to validate that it was working.

Codecov Report
Attention: 4 lines in your changes are missing coverage. Please review.

Comparison is base (618eeab) 94.49% compared to head (0517e37) 94.49%.
Report is 6 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #376   +/-   ##
=======================================
  Coverage   94.49%   94.49%
=======================================
  Files          32       32
  Lines        1035     1035
=======================================
  Hits          978      978
  Misses         57       57

Files
Coverage Δ

ols/constants.py
100.00% <100.00%> (ø)

ols/src/query_helpers/docs_summarizer.py
91.80% <63.63%> (ø)

/approve
/lgtm
@kyoto give this a try next time you need to deploy OLS. Note that you'll have to uncomment the route definition if you want to use the route to create a TLS endpoint that the console can talk to.

authors: bparees codecov-commenter thoraxe

repo: openshift/lightspeed-service


text:
OCPCLOUD-2565: Featuregate implementation

This change:

Updates the MAO, to bring in new changes to the Machine Reconciler.
Reads in feature gates from flags, passing them to the reconciler.

Running the binary without passing the --feature-gates flag will result in the feature gates initialising as follows:
I0812 17:36:32.555187    1120 main.go:154] Initializing feature gates: AllAlpha=true|false (ALPHA - default=false), AllBeta=true|false (BETA - default=false), MachineAPIMigration=true|false (ALPHA - default=false)
I0812 17:36:32.555275    1120 main.go:159] FeatureGateMachineAPIMigration initialised: false

Note that any feature gates that are GA will not be printed in the help text, but can still be set by flags.

/retest

/retest

/retest

/retest
CI infra issues

/retest

The /e2e-openstack-nfv-intel job is failing due to sriov-network-operator not installing. This has been permafailing for a while, so we can ignore the failure.

/approve
/lgtm

authors: nrb theobarberbany


text:
Don't getProviderSecret when noDefaultBackupLocation flag set

Closes #420
Signed-off-by: Tiger Kaovilai tkaovila@redhat.com

/retest

/retest

/retest

repo: openshift/oadp-operator


text:
Adds noobaa.namespacestores.io to the exposed APIs

This commit is required for KNIP-1600 and exposes the namespace store CRD for UI to use.
Signed-off-by: Vineet Badrinath vbadrina@redhat.com

/test ocs-operator-ci

repo: openshift/ocs-operator


text:
[WIP] Fix empty label selector crash

This is a more minimal attempt at #618 to fix the segfault in newStorageClassDeviceSets when an empty labelselector is provided.
This is more minimal for easier backporting.
It also provides a test case as a separate patch that demonstrates the segfault without this patch.
The refactoring and enhancement of #618 could follow on top.

@jarrpa @umangachapagain  raise this as a PR to see the results of functional tests.

not needed any more


text:
unstable Windows unit tests

From last 20 runs, on 2 were successful
https://prow.ci.openshift.org/job-history/gs/origin-ci-test/pr-logs/directory/pull-ci-openshift-odo-main-psi-unit-test-windows

most of the failure are CI or test setup failures:
2021/07/23 06:19:10 : timed out
{"component":"entrypoint","error":"wrapped process failed: exit status 1","file":"prow/entrypoint/run.go:80","func":"k8s.io/test-infra/prow/entrypoint.Options.Run","level":"error","msg":"Error executing test process","severity":"error","time":"2021-07-23T06:19:10Z"}
error: failed to execute wrapped command: exit status 1
INFO[2021-07-23T06:19:17Z] Step unit-test-windows-unit-test-windows-steps failed after 20m20s.
INFO[2021-07-23T06:19:17Z] Ran for 23m1s
ERRO[2021-07-23T06:19:17Z] Some steps failed:
ERRO[2021-07-23T06:19:17Z]
  * could not run steps: step unit-test-windows failed: "unit-test-windows" test steps failed: "unit-test-windows" pod "unit-test-windows-unit-test-windows-steps" failed: the pod ci-op-tnrsnvjn/unit-test-windows-unit-test-windows-steps failed after 20m13s (failed containers: test): ContainerFailed one or more containers exited
  *

[ssh:Windows 10] bash: line 0: cd: amqp_ci_rcv_odo-windows-unit-pr-build_PR_4905/wtnIx/repo: No such file or directory

[ssh:Windows 10] Cloning into 'amqp_ci_rcv_odo-windows-unit-pr-build_PR_4918/uaWvX/repo'...
[ssh:Windows 10] error: RPC failed; curl 56 OpenSSL SSL_read: Connection reset by peer, errno 104
[ssh:Windows 10] error: 5393 bytes of body are still expected
[ssh:Windows 10] fetch-pack: unexpected disconnect while reading sideband packet
[ssh:Windows 10] fatal: early EOF
[ssh:Windows 10] fatal: fetch-pack: invalid index-pack output

[ssh:Windows 10] fatal: unable to access 'https://github.com/openshift/odo/': Could not resolve host: github.com

[ssh:Windows 10] !!!attempt failed due to executor error!!!
 - failed to wait for command completion failed to wait ssh command: <nil>

[ssh:Windows 10] fatal: not a git repository (or any parent up to mount point /cygdrive)

[ssh:Windows 10] fatal: destination path 'amqp_ci_rcv_odo-windows-unit-pr-build_PR_4905/uaBEd/repo' already exists and is not an empty directory.

2021/07/23 07:22:29 : timed out
{"component":"entrypoint","error":"wrapped process failed: exit status 1","file":"prow/entrypoint/run.go:80","func":"k8s.io/test-infra/prow/entrypoint.Options.Run","level":"error","msg":"Error executing test process","severity":"error","time":"2021-07-23T07:22:29Z"}
error: failed to execute wrapped command: exit status 1

ping @prietyc123

Majority of the tests fail because windows vms are not restarted in weeks they laggy and hence results in doing a timeout and ssh failure.

Majority of the tests fail because windows vms are not restarted in weeks they laggy and hence results in doing a timeout and ssh failure.

what we can do so the vm doesn't need restart?

When such problem occurs we are not able to login into the machine to check what are taking resources and why its slowing down,
the only solution at that moment is to restart and access the windows machine.
I was thinking to schedule the restart on weekend(preferably on sunday) once every 3 week to avoid this situation.

As discussed in Planning call, we need to figure why restarting is the only option.

Error due to laggy behaviour of windows vm

**2021/07/23 06:19:10 : timed out
{"component":"entrypoint","error":"wrapped process failed: exit status 1","file":"prow/entrypoint/run.go:80","func":"k8s.io/test-infra/prow/entrypoint.Options.Run","level":"error","msg":"Error executing test process","severity":"error","time":"2021-07-23T06:19:10Z"}
error: failed to execute wrapped command: exit status 1
INFO[2021-07-23T06:19:17Z] Step unit-test-windows-unit-test-windows-steps failed after 20m20s.
INFO[2021-07-23T06:19:17Z] Ran for 23m1s
ERRO[2021-07-23T06:19:17Z] Some steps failed:
ERRO[2021-07-23T06:19:17Z]
  * could not run steps: step unit-test-windows failed: "unit-test-windows" test steps failed: "unit-test-windows" pod "unit-test-windows-unit-test-windows-steps" failed: the pod ci-op-tnrsnvjn/unit-test-windows-unit-test-windows-steps failed after 20m13s (failed containers: test): ContainerFailed one or more containers exited  ***

2021/07/23 07:22:29 : timed out
{"component":"entrypoint","error":"wrapped process failed: exit status 1","file":"prow/entrypoint/run.go:80","func":"k8s.io/test-infra/prow/entrypoint.Options.Run","level":"error","msg":"Error executing test process","severity":"error","time":"2021-07-23T07:22:29Z"}
error: failed to execute wrapped command: exit status 1

Network issue from PSI side (when not able to reach out to github)

[ssh:Windows 10] fatal: unable to access 'https://github.com/openshift/odo/': Could not resolve host: github.com

Git related errors are from ci-firewall as cloning of repo creating a temp dir for test and some other steps are covered by ci-firewall.

[ssh:Windows 10] fatal: destination path 'amqp_ci_rcv_odo-windows-unit-pr-build_PR_4905/uaBEd/repo' already exists and is not an empty directory.

[ssh:Windows 10] fatal: not a git repository (or any parent up to mount point /cygdrive)

[ssh:Windows 10] !!!attempt failed due to executor error!!!
 - failed to wait for command completion failed to wait ssh command: <nil>

[ssh:Windows 10] bash: line 0: cd: amqp_ci_rcv_odo-windows-unit-pr-build_PR_4905/wtnIx/repo: No such file or directory

[ssh:Windows 10] Cloning into 'amqp_ci_rcv_odo-windows-unit-pr-build_PR_4918/uaWvX/repo'...
[ssh:Windows 10] error: RPC failed; curl 56 OpenSSL SSL_read: Connection reset by peer, errno 104
[ssh:Windows 10] error: 5393 bytes of body are still expected
[ssh:Windows 10] fetch-pack: unexpected disconnect while reading sideband packet
[ssh:Windows 10] fatal: early EOF
[ssh:Windows 10] fatal: fetch-pack: invalid index-pack output

Git related errors are from ci-firewall as cloning of repo creating a temp dir for test and some other steps are covered by ci-firewall.

This looks like issue in one of our scripts

Not sure @kadel, as the scripts starts with Setup script  creating an artifact dir and all the steps before that is covered  by ci-firewall.
AFAIR ci-firewall currently performs

removing amqp..PRXXXX dir if exist
creation of the amqp..PRXXXX/XXXX/repo dir
cloning of the repo
fetching and merging the PR
executing the Scripts (setup and run)

My observations:

No of failures are caused due to consecutive push of commits, which result in failure for windows as the test take some time to stop and the new test try to cleanup the old test which is in progress.
Thus fails with

  [rm -rf amqp_ci_rcv_odo-windows-unit-pr-build_PR_4975]
[ssh:Windows 10] !!!Attempt 1!!!
[ssh:Windows 10] rm: cannot remove 'amqp_ci_rcv_odo-windows-unit-pr-build_PR_4975/GLNYf/repo/vendor/github.com/kylelemons/godebug/pretty': Device or resource busy
[ssh:Windows 10] rm: cannot remove 'amqp_ci_rcv_odo-windows-unit-pr-build_PR_4975/GLNYf/repo/vendor/github.com/onsi/gomega/format': Device or resource busy
[ssh:Windows 10] rm: cannot remove 'amqp_ci_rcv_odo-windows-unit-pr-build_PR_4975/GLNYf/repo/vendor/golang.org/x/text/encoding/charmap': Device or resource busy
[ssh:Windows 10] rm: cannot remove 'amqp_ci_rcv_odo-windows-unit-pr-build_PR_4975/GLNYf/repo/vendor/golang.org/x/text/encoding/japanese/tables.go': Device or resource busy
[ssh:Windows 10] rm: cannot remove 'amqp_ci_rcv_odo-windows-unit-pr-build_PR_4975/GLNYf/repo/vendor/golang.org/x/text/encoding/korean': Device or resource busy
[ssh:Windows 10] Executing command [mkdir -p amqp_ci_rcv_odo-windows-unit-pr-build_PR_4975/GRvek/repo]
[ssh:Windows 10] !!!previous command failed or skipped, skipping!!!

solution: a skip the cleanup if we encounter Device or resource busy  error (cleanup will be done in the end)

when error occur while test is running (need more info on this)

[ssh:Windows 10] !!!attempt failed due to executor error!!!
 - failed to wait for command completion failed to wait ssh command: <nil>

10 min timeout for odo watch unit tests

[ssh:Windows 10] Pushing files...
[ssh:Windows 10] panic: test timed out after 10m0s

solution: increasing the priority of golang operation on windows may help reduce the no of occurance.

We are constantly monitoring the windows unit test CI for failures, and no actual failures are observed in the last few weeks.
So we have decided to close this issue for now.
/close

authors: anandrkskd dharmit kadel

repo: openshift/odo


text:
Initial cluster context command

PR introduces a new command. osdctl cluster context which takes in either an internal clusterID, external clusterID, or clustername and returns whether the cluster is fully supported or in limited support.
Moving forward we will be adding more functionalities to this command such as reporting on triggered alerts, Sevicelogs, and more.

@sam-nguyen7 Could this be folded into osdctl cluster health?

@fahlmant this is intended to be a quick-reference of state based on external tools.  We're planning to extend it over time.

authors: clcollins fahlmant sam-nguyen7

repo: openshift/osdctl


text:
Bump to v0.17.0

Diff between upstream and this PR:
https://github.com/prometheus/alertmanager/compare/v0.17.0...simonpasquier:aos-bump-0.17.0
Diff between upstream and the current master:
https://github.com/prometheus/alertmanager/compare/v0.16.2...openshift:39c2c111ea818cd16dbd11e31cc682cf2b4042d3

cc @mxinden

/test e2e-aws-upgrade

Do we still want to keep the .spec file updated?  It still says 0.16.2.

Right so AFAICT the spec and systemd files aren't needed anymore.
I've submitted separate PRs to get rid of them:
https://github.com/openshift/prometheus-alertmanager/pull/24
https://github.com/openshift/prometheus/pull/30
https://github.com/openshift/node_exporter/pull/30

/lgtm

authors: pgier simonpasquier

repo: openshift/prometheus-alertmanager