text:
City of Saint John's, Newfoundland

I've contacted the source for St. John's after noticing a very large number of duplicates in the file. The source file for Saint John's has an error in it - 'STREETNUM' is not the civic address number, it's actually a number designating the street itself. The civic number is the CIVIC field.
Cheers,
Joseph

The source file has been updated.

repo: openaddresses/openaddresses


text:
Azure content management exception triggered by charset encoding issues

Describe the bug
I'm getting exceptions triggered by content management filter at the Azure endpoint, i.e. an InvalidRequestError with fields

error_code: content_filter
error_message: The response was filtered due to the prompt triggering Azure OpenAI?s content management policy. Please modify your prompt and retry. To learn more about our content filtering policies please read our documentation: https://go.microsoft.com/fwlink/?linkid=2198766

The prompt sent is trivial, the only issue is that it's in Spanish and has non-ASCII characters. I'm using gpt-35-turbo and the ChatCompletion method. The prompt is just:
  [{'role': 'user', 'content': 'quiero un móvil con una buena cámara'}]

(meaning I want a mobile with a good camera). This triggers the content filter, which does not make much sense, given that the message is harmless. Furthermore, if I remove the accented character in cámara, i.e.
  [{'role': 'user', 'content': 'quiero un móvil con una buena camara'}]

... then the filter is not triggered, and I get a proper text response. Which makes less sense.
I've dug a bit in the code. The prompt seems to be serialized here:
            if params and not files:
                data = json.dumps(params).encode()
                headers["Content-Type"] = "application/json"

Interestingly, if I change the serialization to
            if params and not files:
                data = json.dumps(params, ensure_ascii=False).encode()
                headers["Content-Type"] = "application/json"

..then the content management filter is not triggered, even with an accented character, and I get a proper response. Which suggests some kind of unusual behavior at the endpoint, given that it behaves differently with escaped unicode characters than with regular UTF-8 text.
To Reproduce

Peepare a prompt `[{'role': 'user', 'content': 'quiero un móvil con una buena cámara'}]
Send to the ChatCompletion API, with a gpt-35-turbo model
Content Management filter is triggered

Code snippets
No response
OS
Ubuntu
Python version
Python 3.8.10
Library version
openai-python v0.27.7

@paulovn I don't have an update for you yet, but just letting you know that I passed your issue to the Azure folks who work on content filtering.

This does not look like a bug in the Python SDK, so I'm going to go ahead and close this issue.

source: gharchive/issue

authors: kristapratico paulovn rattrayalex

repo: openai/openai-python

license: mit

license_type: permissive

license_source: bigquery


text:
Url pathname always returns endpoint url

While using the/app/myshinyapp/url style and printing the following statements:
print("–start of clientData–")
print(session$clientData$url_protocol)
print(session$clientData$url_hostname)
print(session$clientData$url_port)
print(session$clientData$url_pathname)
print("–end of clientData–")

I got:
[1] “–start of clientData–”
[1] “http:”
[1] “myshinyapp.dbrs.local”
[1] “”
[1] “/endpoint/b7219443-8125-41d4-9894-b19b1602614c/”
[1] “–end of clientData–”

I would understand if if I was using the app_direct style but here it is a bit confusing, I think
session$clientData$url_pathname
should return app/myshinyapp/  instead, don’t you agree ?
Thank you

Hi @dylancis, the upcoming release will support this, showing /app_direct/<app-name> instead of /endpoint/<proxy-id>, regardless of whether the app is accessed via iframe or directly.

Hi @fmichielssen - this is awesome. Thanks. Do you mean also in the browser url itself ?

Also @fmichielssen , is there a way to set which one between /app/ and /app_direct/ to use by default on the landing page:

This is pointing to EU RMBS Insight Model but what bout if by default we would like EU RMBS Insight Model instead ?

Hi @fmichielssen we have not deployed 2.2.1 and while I now see /app_direct/ in the JS condole:

If I tried to amend the url once the app is lunched to use /app_direct/ instead of /app/ I get a 503 error code:

I used to be able to do this in earlier version - anything I do wrong maybe ?
Thank you, Dylan

Hi @dylancis Maybe the following is relevant:
https://support.openanalytics.eu/t/app-direct-functionality/1108

thanks @tverbeke got it a forward slash is needed at the end of the url such as /app_direct/modelappname/ to make it works.
Is there any reason why the ' id required here ? Also I looked at https://www.shinyproxy.io/configuration/ and it seems there is no way we can specify the redirect mode /app/ vs /app_direct/ via the ShinyProxy landing page.
It would be useful to have an option within the configuration YAML file such as:
 specs:
  - id: 01_hello
    container-cmd: ["R", "-e", "shinyproxy::run_01_hello()"]
    container-image: openanalytics/shinyproxy-demo
    app_url: app_direct (or app)
    access-groups: [scientists, mathematicians]

authors: dylancis fmichielssen tverbeke

repo: openanalytics/shinyproxy


text:
Bootstrap Version

I've been using some of newer features of bootstrap for changing the lists of applications into cards. Cards aren't available in bootstrap 3.4.1, so have had to update bootstrap on the java file.
I've done this by adding the following to pom.xml
		<dependency>
			<groupId>org.webjars</groupId>
			<artifactId>bootstrap</artifactId>
			<version>5.0.2</version>
		</dependency>

and changing lines 122 and 123 of BaseController.java to
		map.put("bootstrapCss", "/webjars/bootstrap/5.0.2/css/bootstrap.min.css");
		map.put("bootstrapJs", "/webjars/bootstrap/5.0.2/js/bootstrap.min.js");

from 3.4.1.
This also required me to add in the following to the html templates
	<!-- CSS only -->
	<link href="https://cdn.jsdelivr.net/npm/bootstrap@5.0.2/dist/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-EVSTQN3/azprG1Anm3QDgpJLIm9Nao0Yz1ztcQTwFspd3yD65VohhpuuCOmLASjC" crossorigin="anonymous">
	<!-- JavaScript Bundle with Popper -->
	<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.0.2/dist/js/bootstrap.bundle.min.js" integrity="sha384-MrcW6ZMFYlzcLA8Nl+NtUVF0sA7MsXsP1UyJoMp4YLEuNSfAP+JcXn/tWtIaxVXM" crossorigin="anonymous"></script>

Although I did this a long time ago so can't remember all the bits that are necessary.
I have 2 questions:
1 - Is this the best way of doing this or am I overcomplicating things as I have to rebuild the java file?
2 - Is there any desire to update SP to be using a newer version of Bootstrap OOTB?
Example card usage included.

Regarding question 1, I think it would be sufficient to replace the lines
<link rel="stylesheet" media="screen" th:href="@{${bootstrapCss}}"/>
<script th:src="@{${bootstrapJs}}"></script>

in the default template by the links to bootstrap 5 that you pasted without the need to modify the source.
(alternatively if you've modified the source, you don't need to include the links to css and js manually again since default template includes the lines above which in that case would already load bootstrap 5)

By source are you referring to the source .jar file?
I would have thought just changing those two bits that you say to be links to the style sheet and .js file would be sufficient, but I just seem to have a memory of it not working for some reason. I'm going to test using a stock SP .jar and changing the stylesheet and script tag and see if that works.

authors: mnazarov nik-humphries


text:
[Question] Is it possible to use openapi-backend to validate that the response of an API match an OpenAPI spec?

Hi,
I'm currently using jest-openapi to validate if the responses of my API match my OpenAPI spec. It does not seem to be supported anymore and I'm looking to replace the tool I'm using for this use case. Is it a use case that openapi-backend supports? I see that openapi-backend is used to mock an API by using an OpenAPI spec but instead of doing that, I would like to use my API code and validate the responses it provides against my OpenAPI spec.
Thank you!

Sure! Would be great actually to document how to use validateResponse in tests 👍
https://openapistack.co/docs/openapi-backend/response-validation/

Wow, thank you for that very quick response! I believe the part I was most struggling with was to understand how I attach my Next.js API endpoints as handlers. Would you already have an example of how to do this?

Here’s a full stack example of using openapi-backend with nextjs. Doesn’t include tests unfortunately https://github.com/anttiviljami/openapi-stack-nextjs-starter

Thank you very much, I'll give it a try and see where that leads me!

I took the time to run a few tests and have hit some issues. I don't know if I'm doing it the right way but so far it seems to generally work so I'm wondering if I'm hitting a bug with the OpenAPI validator used.
https://github.com/davidlag0/todo-nextjs/pull/663/files
The test is code is in openapi2.test.js.
And the issue I've been trying to understand, and I'm no OpenAPI expert, is why the test fails with this error message when I use content with application/json in the spec:
> jest --testPathPattern='integration.openapi2' --runInBand

  console.log
    res: {"error":"No Task Found"}{"status":502,"err":[{"instancePath":"","schemaPath":"#/oneOf/0/type","keyword":"type","params":{"type":"array"},"message":"must be array"},{"instancePath":"","schemaPath":"#/oneOf","keyword":"oneOf","params":{"passingSchemas":null},"message":"must match exactly one schema in oneOf"}]}

      at Object.log (__tests__/integration/openapi2.test.js:82:13)

  console.log
    res headers: { 'content-type': 'application/json' }

      at Object.log (__tests__/integration/openapi2.test.js:83:13)

 FAIL  __tests__/integration/openapi2.test.js
  Tests to satisfy OpenAPI spec with OpenAPIBackend
    ✕ GET /api/tasks with empty task list (38 ms)

  ● Tests to satisfy OpenAPI spec with OpenAPIBackend › GET /api/tasks with empty task list

    expect(received).toBe(expected) // Object.is equality

    Expected: 404
    Received: 502

      83 |     console.log("res headers:", res._headers);
      84 |
    > 85 |     expect(res._getStatusCode()).toBe(404);
         |                                  ^
      86 |     expect(JSON.parse(res._getData())).toEqual(
      87 |       expect.objectContaining({
      88 |         error: "No Task Found",

      at Object.toBe (__tests__/integration/openapi2.test.js:85:34)

Test Suites: 1 failed, 1 total
Tests:       1 failed, 1 total
Snapshots:   0 total
Time:        1.215 s, estimated 2 s
Ran all test suites matching /integration.openapi2/i.

I removed the console.log lines in the code but you can see in the test response the error message and I don't understand what I need to change to get it to work.
And oddly enough, I don't even have to change the content of the response being tested (this response isn't being tested), which I find a bit strange.
Could you help me out with any pointer?
Thank you!

repo: openapistack/openapi-backend

license: MIT

license_source: github-api


text:
Code update : Reboot BMC during BMC activation in progress

[x] Upload BMC image
[x] Activation in progress and reset BMC
[x] Verify the failed BMC image

I think this is going to be the same as https://github.com/openbmc/openbmc-test-automation/issues/993. Activating the image is the same thing as a code update.

authors: charleshofer gkeishin

repo: openbmc/openbmc-test-automation


text:
Add update-bmc.py

Add this script to update BMC code with phosphor-software-manager.
Tested: Update BMC code on Witherspoon (ubifs) and Romulus (non-ubifs)
Signed-off-by: Lei YU mine260309@gmail.com

@mine260309 have you checked out @geissonator 's tools in this area? Particularly openbmc-sfw.
I'd like to start consolidating functionality we have rather than just continuing to grow everyone's random scripts. Getting all the random scripts in one place was phase 1 of this repo's intent, phase 2 is documentation, and phase 3 is consolidation.
I'm hoping we can skip to phase 3 in this instance and start reducing the bits and pieces we have lying around.

@amboar Indeed, we should make a python module for these common operations.

Ideally yes. But before that is it possible for you to just use openbmc-sfw over adding a new tool? If not can you hack openbmc-sfw to do what you need?

Checked the tool, it already covers what I want to do with update_auto option.
So we can drop this.

source: gharchive/pull-request

repo: openbmc/openbmc-tools


text:
Cursor does not change back to arrow when a annotation gets moved on the timeline

See screencast: When I activate an annotation and then move it on the the timeline the cursor gets shown as a cross. After having moved the annotation the cursor remains in this cross-shape. When I do such a "moving-an-annotation-movement" next to an annotation (click > move the mouse while having "pushed" the mouse button > release the button) the cursor returns to its arrow-shape.
Annotation Tool_Cursor ändert sich und bleibt so.mp4.zip

Done in 8cadb4eb305b279068fd2b5c72c228ed6e824238

authors: dagraf luniki

repo: opencast/annotation-tool


text:
Use @mui/material's Tooltip instead of title attributes.

Fixes https://github.com/opencast/opencast-admin-interface/issues/574

Hi @luniki, could you fix the merge conflicts?

Done

repo: opencast/opencast-admin-interface


text:
Endtime of segments fixed in the editor

fixes #2021
Added a check to the "segment update" function in the editor. If the endtime of a segment is larger than the total duration of the video, it will be set to the end of the video.

@marwyg Is this ready to go?  Not sure what to make of the self assignment :)

@gregorydlogan Sorry about the self assignment. I talked to @lkiesow and he explained the current procedure of reviewing to me. So I unassigned me, because I just made the fix. The commit still has to be reviewed.

@marwyg Do you have a test workflow for this?

@marwyg ping.

I tested it only via debugging. I will try to create a workflow for it.

In the studio-upload.xml there is a key named skipEditorPreview (https://github.com/opencast/opencast/blob/36594e0673c5f74cc5193c9a60163582cacfffe8/etc/workflows/studio-upload.xml#L109). If this is set to false, you can record a video with Opencast Studio (top right corner in Opencast) and it will be editable later in the event table in Opencast.
Directly after recording the video in Opencast Studio, there is the possibility to trim the video at the start and the end. When only the start was trimmed, the error occurred later in the editor in Opencast. It created a weird segment with NaN.

Oh, I can't find the loop because you updated the pull! This looks like a totally reasonable fix.

@marwyg Do you know if there is any scenario where the  duration would not be set on the video or default to 0?
Does it make sense to add a sanity check on  video.duration  to be safe? Something like...
            if (scope.video.duration && scope.video.duration > 0 && segment.end > scope.video.duration) {
              segment.end = scope.video.duration;
            }

authors: gregorydlogan karendolan marwyg

repo: opencast/opencast


text:
named the correct directory - Update workflow.md

adding the missing "s" from the directory
Your pull request should…

[x] have a concise title
[ ] close an accompanying issue if one exists
[ ] be against the correct branch
[ ] include migration scripts and documentation, if appropriate
[ ] pass automated tests
[ ] have a clean commit history
[ ] have proper commit messages (title and body) for all commits

Is the ICLA already signed? Does it matter for this small change?

Imo this fails into the realm of things which don't need an icla. But im Al's not a lawyer. @lkiesow you know most about this kind of thing?

Imo this fails into the realm of things which don't need an icla. But im Al's not a lawyer. @lkiesow you know most about this kind of thing?

Lars is on vacation right now. IMO one letter should be fine, and he's one of ours, so I promise we won't sue for the letter s. ;)
But I'm going to talk to Philip about the ICLA.

authors: KatrinIhler gregorydlogan mtneug pthien1


text:
Parsing for drop frame timecodes can be incorrect

Here's a simple project to show what I've been running into with F29_97_DF and F59_94_DF rates.
use vtc::{rates, Timecode};

fn main() {
    let tc_1 = Timecode::with_frames("01:00:34;01", rates::F29_97_DF).unwrap();

    assert_eq!("01:00:34;01", tc_1.timecode());
}

Running this program results in:
assertion `left == right` failed
  left: "01:00:34;01"
 right: "01:00:34;02"

And if I use a rate of F59_94_DF instead:
assertion `left == right` failed
  left: "01:00:34;01"
 right: "01:00:34;03"

I looked into this more and it looks like the problem lies in the drop_frame_adjustment function. I took my own stab at implementing that function based on the pseudocode from David Heidelberger and it seems to work correctly. Would you like me to submit a pull request?

Sure! Go ahead and make a PR!

repo: opencinemac/vtc-rs


text:
Deleting a collective with expenses crashes the host dashboard

A collective/event, xr-hackney-july-uprising-42829ev, was recently deleted with an APPROVED expense (id=8149). Deleting collectives is only blocked if an expense has been paid.
As a consequence, the host dashboard is crashing as it expect all expenses to have a collective attached. The crash occurs on this line: https://github.com/opencollective/opencollective-frontend/blob/d0721a8c15a4f03075ead849747ddd188abe163a/src/apps/expenses/components/Expense.js#L312
To ensure that no other expenses is affected:
SELECT * FROM "Expenses" e
INNER JOIN "Collectives" c ON c.id = e."CollectiveId"
WHERE c."deletedAt" IS NOT NULL
AND e."deletedAt" IS NULL

Slack report: https://opencollective.slack.com/archives/C6JTTA4SK/p1558886890003500
Freshdesk ticket: https://opencollective.freshdesk.com/a/tickets/534

Another outcome was that the funds in the event's balance just disappeared - were not paid out or replaced in the Collective's balance. The user contacted support here.

As a first quick action, we need to remove the the ability to delete events. Until done properly.

authors: Betree alanna znarf

repo: opencollective/opencollective


text:
Batch gift card contributions

It seems that most organizations running gift cards programs with a credit card attached expect that people will use them to contribute and that they'll get a single charge at the end of the month.
We could change the default behavior or add a checkbox to conditionally activate that gift card contributions are created in a pending state and only get processed at the end of the month, with a single credit card transaction per host.
Related:

#1916
https://github.com/opencollective/opencollective/issues/4206
https://github.com/opencollective/opencollective/issues/5164

This was suggested again at, https://opencollective.freshdesk.com/a/tickets/143611

authors: Betree SudharakaP


text:
Change to amount input instead of pre-defined amounts

I want to be able to type in a min and max amount myself instead of these pre-defined options:

Often I use these filters to find specific transactions or specific amount ranges which differ from these pre-set options and it would be very useful. This goes for everywhere this component is used to filter amounts (expenses, transactions, contributions, etc).
P3 high frequency, low impact

+1 The amounts are completely arbitrary and are not very useful.
There was an update to this from the design team  but It was burried in a full redesign of the transactions list. We should definitely try to tackle this in isolation.

+1 I usually end up changing the value directly in the URL to achieve that


text:
missing Capabilities message in tunnel.proto

The Capabilities message was defined in previous version as
message Capabilities {
bool handler = 1;  // Whether a handler is installed on the endpoint.
}
Should the Capabilities defined in gNMI be used in this proposal
Thanks
David

Capabilities message in gRPC tunnel had no correlation with the
Capabilities RPC in gNMI and the name conflict was coincidental. The
original intent of Capabilities message in this prototype tunnel library
was to allow for a single target to enumerate multiple protocols (e.g.
gNMI/gNOI and ssh). But that was subsumed in the update where the tunnel
target can enumerate a type specifically and register itself multiple times
for each protocol.  Thus Capabilities message was removed.
On Fri, Feb 5, 2021 at 4:18 PM dliu2223 notifications@github.com wrote:

The Capabilities message was defined in previous version as
message Capabilities {
bool handler = 1; // Whether a handler is installed on the endpoint.
}
Should the Capabilities defined in gNMI be used in this proposal
Thanks
David
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/openconfig/grpctunnel/issues/12, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEL4QLYUSDB4VKXJZ2APLOLS5ROBBANCNFSM4XFOJHZA
.

repo: openconfig/grpctunnel


text:
http 500 after install extension

Hi,
Followed steps:
1.fresh opencart 3.0.3 install (last version)
2. go to your OpenCart root folder, open a console and run the following commands step by step
3. git clone https://github.com/opencorero/opencore.git core
4. cd core/
5. composer update screenshot as:

php artisan key:generate

php artisan migrate:install

php artisan migrate screenshots as:

copy OpenCart extension files from core/opencart-module/2.x or 3.x/upload to you OpenCart root folder

Optional: in order to enable Developer & Example modules you need to access admin / user /

(4 permission given both permission and changes)

go to OpenCart admin panel / extensions / extensions / modules, find OpenCore module and install it

after install system gone. (http 500)
admin panel not accessible.
when i try to enter website main domain: following errors displayed:
Fatal error: Uncaught PDOException: SQLSTATE[42S02]: Base table or view not found: 1146 Table 'notebook_oc.opencore_routes' doesn't exist in /home/notebook/oc.notebookchipset.com/system/library/db/corepdo.php:205 Stack trace: #0 /home/notebook/oc.notebookchipset.com/system/library/db/corepdo.php(205): PDO->prepare('SELECT method, ...') #1 /home/notebook/storage/modification/system/library/db.php(51): DB\corePDO->query('SELECT method, ...') #2 /home/notebook/oc.xxxxx.com/core/support/Opencart/Startup.php(138): DB->query('SELECT method, ...') #3 /home/notebook/oc.xxxxx.com/core/support/Opencart/Startup.php(86): OpenCore\Support\Opencart\Startup->checkOpenCoreRoute() #4 /home/notebook/oc.xxxxx.com/catalog/controller/startup/opencore.php(46): OpenCore\Support\Opencart\Startup->executeIfRouteExists('common/home', NULL, false) #5 /home/notebook/storage/modification/system/engine/action.php(79): ControllerStartupOpencore->before_controller('common/home', NULL) #6 /home/notebook/oc.xxxxx.co in /home/notebook/oc.xxxxx.com/system/library/db/corepdo.php on line 205
phpmyadmin screenshot:

screenshot when i try to enter admin panel:

all steps replicated two times at live server(vps) can not able to install.
vps: centos 7, cwp panel lastest.
could me my mistake somewhere?
here log in core\storage\laravel 5-12-log.txt
[2020-05-12 01:02:24] production.ERROR: SQLSTATE[HY000] [1045] Access denied for user 'forge'@'localhost' (using password: NO) (SQL: select * from information_schema.tables where table_schema = forge and table_name = opencore_tasks and table_type = 'BASE TABLE')
ps note:
before click install (+ button) i just backup database. after drop database and install from backup everything works again and extension of course looks not installed.

Hi,
there's a problem with the documentation.
We are using the master branch for development and the latest version is not completed(indeed is missing the routes table).
Try using our latest release OpenCore v1.2.4, it should work fine without any errors.
Cheers!

Thank you. I will try write if any problem.

repo: opencorero/opencore


text:
Bail on build errors

--bail

This was just a test to see if build failed.

repo: opencredo/opencredo-react-boilerplate


text:
Add actions button in the task jobs list

Currently, the UI displays the Actions button on the task page

It works for the task, but there is no such button for the jobs list below.
However, there is such element for jobs in the jobs list on the Jobs tab:

The suggestion is to add more buttons to the job actions (e.g. clear annotations) and to add such actions button to the task jobs list. This will help to reduce the number of actions needed to do some things with the job (we will be able to skip job loading, page switches etc.).
Version: a7f7ff1 (2.4.1)

Added

authors: bsekachev zhiltsov-max

repo: opencv/cvat


text:
Import coco annotations with keypoints and boxes

My actions before raising this issue

[x] Read/searched the docs
[x] Searched past issues

I have images with two types of annotations (keypoints and bounding boxes) for each instance in each frame. I have a similar scenario as #5904  or #1421 .
When exporting everything works as expected and I get the outside bounding box in the annotation object. The issue comes when I import annotations to get the same/similar results.
Ideally, I would like to directly import the box and the keypoints as a group. If that is not possible, can I import them separately and then manually group them?
I have 2 categories: person, and person_bbox.
I have tried...

Importing as coco keypoints 1.0 the box in the annotation object: { ..., keypoints: [...], bbox: [...], ...]. --> This ignores the bbox data.
Importing two different annotations (coco and coco keypoint) --> This overwrites annotations.
Importing two annotation objects per instance (with different ids) one with category person and another one with person_bbox (this one with no keypoints. --> This throws and error when importing because keypoints are None type.

*Note: I am using cvat.ai online tool.
Expected Behaviour

The expected behavior is to import a keypoint dataset with different bounding box than the one defined by the keypoints. If not, be able to import Groups.
Current Behaviour

Possible Solution

Context

Your Environment

I am using cvat.ai online tool.
All my annotations are in coco format

Hi, I agree that it's inconvenient. You can use this workaround until it's implemented:

import os.path as osp
import sys
from glob import glob
from typing import Dict, Optional
import datumaro as dm
from datumaro.plugins.cvat_format.converter import _SubsetWriter, XmlAnnotationWriter
from unittest import mock

class RemapBboxLabels(dm.ItemTransform):
    def __init__(self, extractor: dm.IExtractor, new_label_id: int):
        super().__init__(extractor)
        self._new_label_id = new_label_id

    def transform_item(self, item: dm.DatasetItem) -> Optional[dm.DatasetItem]:
        updated_annotations = []
        for ann in item.annotations:
            if isinstance(ann, dm.Bbox):
                ann = ann.wrap(label=self._new_label_id)
            updated_annotations.append(ann)

        return self.wrap_item(item, annotations=updated_annotations)

class PatchedCvatSubsetWriter(_SubsetWriter):
    # CVAT will require 'outside' property on the skeleton points,
    # but it is missing in the datumaro export in CVAT format
    # Here we fix this by monkey-patching the export method.

    def _write_shape(self, shape, item):
        xml_writer = self._writer

        def patched_open_points(points: Dict):
            if isinstance(shape, dm.Points):
                points['outside'] = str(int(shape.visibility[0] == dm.Points.Visibility.absent))
                points['occluded'] = str(int(shape.visibility[0] == dm.Points.Visibility.hidden))

            XmlAnnotationWriter.open_points(xml_writer, points)

        with mock.patch.object(self._writer, 'open_points', patched_open_points):
            return super()._write_shape(shape, item)

def prepare_import_dataset(kp_dataset_dir: str, dst_dir: str):
    kp_dataset_annotation_filename = next(
        fn for fn in glob(osp.join(kp_dataset_dir, 'annotations', '*.json'))
        if 'person_keypoints' in osp.basename(fn)
    )
    bbox_dataset = dm.Dataset.import_from(kp_dataset_annotation_filename, 'coco_instances')
    kp_dataset = dm.Dataset.import_from(kp_dataset_annotation_filename, 'coco_person_keypoints')

    # Boxes need to have a separate label in CVAT,
    # but they will be parsed with the same label as skeletons,
    # since they are read from the same annotation. So, we just remap the labels.
    resulting_labels = kp_dataset.categories()[dm.AnnotationType.label]
    bbox_dataset.transform('project_labels', dst_labels=resulting_labels)
    bbox_label_id = resulting_labels.find('person_bbox')[0] # <<<< use your bbox label name here
    assert bbox_label_id is not None

    output_dataset = dm.Dataset.from_extractors(bbox_dataset, kp_dataset)
    output_dataset.transform(RemapBboxLabels, new_label_id=bbox_label_id)

    with mock.patch('datumaro.plugins.cvat_format.converter._SubsetWriter', PatchedCvatSubsetWriter):
        output_dataset.export(dst_dir, 'cvat', save_images=True)

if __name__ == '__main__':
    prepare_import_dataset(sys.argv[1], sys.argv[2])

You'll need to install Datumaro which is used in CVAT now:
python3 -m virtualenv venv
. ./venv/bin/activate
pip install 'datumaro[default]@git+https://github.com/cvat-ai/datumaro.git@ff83c00c2c1bc4b8fdfcc55067fcab0a9b5b6b11'

Convert your COCO dataset using the script above:
python ./convert.py <path/to/coco/dataset/dir> <output/dir>

And import the annotation file in CVAT task using the CVAT format.

Thanks a lot! converting the annotations to cvat 1.1 format and then importing those annotations to cvat worked.

Closing this issue

authors: twurdwz zhiltsov-max


text:
Tryfix: Not working from external IP

tryfix #126

close this due to inactivity

Hi,
Is there a solution to this problem ?
I have same problem through my 4G access.
I red #126 and test with a standalone server on 8080 without any problem.
I applied this patch, but still not able to access.
Any help ?
Should I reopen this ticket ?
Thanks,
BR

please create a new issue with the description of your problem and how to reproduce😎

repo: opendatacam/opendatacam


text:
[RHOAIENG-5305] - golang.org/x/net Allocation of Resources Without Li…

…mits or Throttling
chore: Fixes CVE-2023-45288

Please fix PR title

authors: spolti terrytangyuan

repo: opendatahub-io/odh-model-controller


text:
Expand list of approvers and set up auto-merge for org-management PRs

Right now, we require manual approval and merging from the authorized set of owners for this org. This introduces a bottleneck on adding new people to the org.
I believe we should:

Increase the number of people listed as approvers for PRs to this repo
Implement automation such that when a PR is approved by an authorized approver, the PR is automatically merged and org membership changes are applied.

@jkoehler-redhat @LaVLaS @VaishnaviHire what do you think of this proposal. If you approve, I can volunteer my team's time to make it happen.
cc @anishasthana

Increase the number of people listed as approvers for PRs to this repo
Implement automation such that when a PR is approved by an authorized approver, the PR is automatically merged and org membership changes are applied.
I agree with both points.  @accorvin I have some basic automation in development but if your team is will to create and own the automation then it would be greatly appreciated

@accorvin I think this is fine as long as all approvers know to approve aligned to the community guidelines: https://github.com/opendatahub-io/opendatahub-community/blob/main/community-membership.md

Since the previous updates have been blind invites with minimal context , I added templates to  outline the requirements for requesting organization membership for all new PRs.

So will we require an issue for every new membership PR? Or is just a PR following the template sufficient?

I can spend some time working on a github action automating it next week. I'm thinking for now we can start with the same OWNERS list  + myself and Alex. We can iterate over time to better tie in SIG leadership

For members of the RHODS team (including dev/qe/docs/program/sre/bu/etc), the issue template feels to heavy weight to me. We should be able to go straight to PR.

This is a good point we were discussing in the steering committe.  The original goal was to have a self governing community so the idea that there is a RHODS "team" controlling the community goes against the idea of community governance.  If we are auto-adding internal RHODS team members or RH OpenShift AI contributors then we just state that the membership requirements are you have to be an IBM/Red Hat employee or partners

This is handled by #48

Howdy, I'm new here :slightly_smiling_face:
I'm an admin of an org-management/Peribolos config for a different org here, and there might be some automation that I could propose in a PR here that you might like:

We do a Peribolos dry run on each PR automatically (typically takes about ~1min)
We do a Peribolos run (not dry) on each merge (similarly takes 60-90s)
We schedule that same Peribolos run daily, to flush out any manual changes and force people to remember that they need to make their changes here through the config :)

Currently I see that for this org, the Peribolos run appears to be triggered manually by the person who merges the PR, and it typically takes more than 20 minutes (and it seem to do much less / is a much smaller config than the other org that I'm talking about), so I think what I'm suggesting would be a positive improvement.
(I haven't looked into why the other one is so quick compared to this one, so I'm not sure)
Let me know what you think, thanks! :slightly_smiling_face:

The source for that image is in https://github.com/opendatahub-io/org-management/tree/main/.github/build.
@accorvin  recently created https://github.com/opendatahub-io/org-management/blob/main/.github/workflows/peribolos.yaml previously, so in theory any time a PR is merged in the membership is updated.
That being said, please raise PRs for anything you think you can improve. I'm happy to chat synchronously/asynchronously as well if you need someone to bounce ideas off

Thanks for the info @anishasthana, I'll try to find some time to look at it soon and see if there are improvements that I can suggest in a PR :+1:

@grdryn just chiming in to say that we strongly welcome any automation improvements you're willing to contribute to this repo!

authors: LaVLaS accorvin anishasthana grdryn

repo: opendatahub-io/org-management


text:
Add timestamps to log output

The log output should include the timestamp to make it easier to debug issues.

While we're at it, should add the possibility to add an (optional) tag as well so that the area from where the log message comes from can be identified easily.

repo: opendevstack/ods-pipeline


text:
PSI Library: Implement Constant (Transform) (Rust)

OpenAPI Definition

[x] Write definition
[ ] Write comments/docs

Rust Runtime

[x] Write runtime
[ ] Implement tests
[ ] Write comments/docs

Rust Validator

[x] Write validator
[ ] Implement tests
[ ] Write comments/docs

Python bindings

[x] Create bindings
[ ] Implement tests
[ ] Write docs

R bindings

[ ] Create bindings
[ ] Implement tests
[ ] Write docs

docs/tests moved to #62, #63

repo: opendifferentialprivacy/whitenoise-core


text:
chore(github-actions): bump up github action workflow step dependencies

Bump up github action workflow step dependencies

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (5f0b16d) 50.17% compared to head (7042f72) 50.17%.
Report is 8 commits behind head on develop.

Additional details and impacted files
@@           Coverage Diff            @@
##           develop     #173   +/-   ##
========================================
  Coverage    50.17%   50.17%
========================================
  Files           38       38
  Lines         2792     2792
========================================
  Hits          1401     1401
  Misses        1285     1285
  Partials       106      106

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

authors: Abhinandan-Purkait codecov-commenter

repo: openebs/dynamic-nfs-provisioner


text:
fix: Modify ratelimiter for controlled exponential backoff in lvmvolume controller

We have changed the rate limiter which was used in snapshot controller. This rate limiter re queues failed items after 5 secs for first 12 attempts. Then objects are re queued after 30 secs.
This is the new rate limiter being used:
rateLimiter := workqueue.NewItemFastSlowRateLimiter(5time.Second, 30time.Second, 12)
We used to use DefaultControllerRateLimiter() before this which would re queue failed items after 5ms delay which is really not required for Lvm localpv system. reconciliation failures are mostly misconfigurations on worker nodes where vol/snapshot being created doesn't have any network calls as far as node plugin implementation is concerned.
Checklist:

[X] Fixes #<194>
[X] PR Title follows the convention of  <type>(<scope>): <subject>

@abhilashshetty04 can you DCO signoff this PR?

@niladrih Commit is DCO signed off

repo: openebs/lvm-localpv


text:
fix: use relative url for submodule

Used for importing the control-plane to this repo. Also update commit of dependency.

Description

Use a relative URL for importing the control-plane submodule.
Motivation and Context

The top-level part of the URL is the same between repos, so need not be duplicated,
and can be inferred from the URL of the parent repo.

Regression

No

How Has This Been Tested?

Tested by building locally and via CI.
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have added unit tests to cover my changes.

bors try

bors merge

repo: openebs/mayastor-extensions


text:
feat(upgrade-job): use 'crds' subchart toggles to disable existing CRDs

Changes:

adds a CRDs API client
uses the crds subchart's enable toggles to disable CRDs which already exist in the cluster.

bors merge

bors cancel

bors merge

bors merge

authors: niladrih tiagolobocastro


text:
Use clippy and fmt rules from rust stable

refactor(fmt): use fmt style from rust stable

Signed-off-by: Tiago Castro <tiagolobocastro@gmail.com>

refactor(clippy): use rust stable and remove exceptions

Switch to rust-stable and remove most clippy exceptions from linter script.

Signed-off-by: Tiago Castro <tiagolobocastro@gmail.com>

bors try

bors cancel

bors try

bors merge

bors merge

repo: openebs/mayastor


text:
Fix golint issue in doc.go

File: doc.go
Line 1: warning: package comment should be of the form "Package types ..." (golint)

I will take this one, is that ok @Akash4927 ?

PR: https://github.com/openebs/maya/pull/580

Thanks for the PR @cpanato

repo: openebs/openebs


text:
add volume filesystem usage metrics

This patch add filesystem metrics (rawfile_volume_fs_used_bytes and rawfile_volume_fs_total_bytes) for rawfile volume. They represent the actual space usage from filesystem perspective:
# HELP rawfile_volume_fs_used_bytes Used space of volume filesystem
# TYPE rawfile_volume_fs_used_bytes gauge
rawfile_volume_fs_used_bytes{device="/dev/loop0",fstype="ext4",img="/data/pvc-c4ecfd41-9966-44fd-bd76-a88c34edc884/disk.img",loop="/dev/loop0",mountpoint="/var/lib/kubelet/plugins/kubernetes.io/csi/rawfile.csi.openebs.io/60498c10621c39b8674c272877221113fa5b1afc104a0826a8b4bb632f62e6b5/globalmount/mount",node="node-a",volume="pvc-c4ecfd41-9966-44fd-bd76-a88c34edc884"} 1.080041472e+09
# HELP rawfile_volume_fs_total_bytes Size of volume filesystem
# TYPE rawfile_volume_fs_total_bytes gauge
rawfile_volume_fs_total_bytes{device="/dev/loop0",fstype="ext4",img="/data/pvc-c4ecfd41-9966-44fd-bd76-a88c34edc884/disk.img",loop="/dev/loop0",mountpoint="/var/lib/kubelet/plugins/kubernetes.io/csi/rawfile.csi.openebs.io/60498c10621c39b8674c272877221113fa5b1afc104a0826a8b4bb632f62e6b5/globalmount/mount",node="node-a",volume="pvc-c4ecfd41-9966-44fd-bd76-a88c34edc884"} 2.046640128e+09

We used to actually expose those metrics, but apparently kubelet extracts the same metrics using CSI calls and exposes them as kubelet_volume_stats_capacity_bytes and kubelet_volume_stats_used_bytes, and that's why I dropped those metrics from the metrics endpoint. Will you please explain your usecase and why those metrics are not good enough?

thanks, kubelet_volume_stats_**  metrics are enough for us.

authors: mchtech semekh

repo: openebs/rawfile-localpv


text:
Verticals popup menu on CourseUnitView

Added verticals popup menu to CourseUnitView. It closed by feature key VERTICALS_MENU_ENABLED of UI_COMPONENTS category, to test it need to use config from Config PR #157.
This issue related to: #156
Sample with key
https://github.com/openedx/openedx-app-ios/assets/480059/3667b4c1-4ebc-4c11-8153-668bf6c2dfd8
Sample with disabled or without key
https://github.com/openedx/openedx-app-ios/assets/480059/cd3378b7-4f86-4006-8c83-0d4de5713f3d

@touchapp should we start the review of this PR or wait for:

product input on the linked issue i.e. #112?
and the scope and acceptance criteria to be added on the same issue?

authors: forgotvas miankhalid

repo: openedx/openedx-app-ios


text:
Organization scoped events

Description
This issue serves as a starting point for the discussion mentioned during the Open edX Events guide pull request, saying organization proprietary events might live in this repo as well.
Original comment: https://github.com/openedx/openedx-events/pull/80#discussion_r1146731541
### Tasks
- [ ] https://github.com/openedx/openedx-events/issues/357

@ormsbee: Without you bringing up this topic (and I lost where you originally mentioned this), I probably would have just created something like a 2u-events repo. Either way, I'd probably use 2u namespaced event types.
Are you still imagining that orgs should be able to add to this repo, and if so, can you add some of your thoughts here? Ultimately, this could turn in to an ADR for deciding for or against it.
Additionally, were you thinking of this only for events that are produced within an open source repo, or even if it may be produced from a private repo? Either way, we'll need to choose whether it makes sense (or is a problem) to make the private events public.
Note: There is a separate licensing issue documented https://github.com/openedx/openedx-events/issues/184, but I'm hopeful that that can quickly be resolved.

@robrap: I'm not sure whether this discussion happened elsewhere, but I created an issue documenting how to create organization-specific events for now. The docs will explain that if events are org or client-specific, then create their definitions in your repository. I've seen people do that, but I don't know if we're doing it to keep it as the standard. At least it's working for the time being.

Thanks @mariajgrimaldi. This was really started on behalf of @ormsbee, so I'm hoping he can respond or close the ticket.

Thanks folks. I totally forgot about this ticket, and I honestly don't remember the discussion that we had around this.
I think my take was that org-specific events should not live in openedx-events, but that it should be possible for orgs to import everything that they would need to create their own events from openedx-events (i.e. declare instances of OpenEdxPublicSignal in their own repos).
But honestly, I've been out of the loop of openedx-events for a while now, so I have no idea where conversations around that have gone in the last year.

Ok. I think I must have misunderstood you long ago, so I am going to close this ticket. Thank you all!

Thank you both! Still, I'll include Dave's comments in the documentation since they're helpful.

authors: mariajgrimaldi ormsbee robrap

repo: openedx/openedx-events


text:
[BD-46] docs: updated the description on the Installation and usage page

Description
Include a description of your changes here, along with a link to any relevant Jira tickets and/or Github issues.
Deploy Preview
Include a direct link to your changes in this PR's deploy preview here (e.g., a specific component page).
Merge Checklist

[ ] If your update includes visual changes, have they been reviewed by a designer? Send them a link to the Netlify deploy preview, if applicable.
[ ] Does your change adhere to the documented style conventions?
[ ] Do any prop types have missing descriptions in the Props API tables in the documentation site (check deploy preview)?
[ ] Were your changes tested using all available themes (see theme switcher in the header of the deploy preview, under the "Settings" icon)?
[ ] Were your changes tested in the example app?
[ ] Is there adequate test coverage for your changes?
[ ] Consider whether this change needs to reviewed/QA'ed for accessibility (a11y). If so, please add wittjeff and adamstankiewicz as reviewers on this PR.

Post-merge Checklist

[ ] Verify your changes were released to NPM at the expected version.
[ ] If you'd like, share your contribution in #show-and-tell.
[ ] 🎉 🙌 Celebrate! Thanks for your contribution.

Thanks for the pull request, @PKulkoRaccoonGang!
When this pull request is ready, tag your edX technical lead.

authors: PKulkoRaccoonGang openedx-webhooks

repo: openedx/paragon


text:
Update gha to use latest shared action and turnstyle

This change uses new options on the shared action and a new public action called Turnstyle.
New Options:
Resource groups - When switching to Grillo's k8s cluster, we started using resource groups
Working directories - Allows us room to grow as we are considering breaking the Dashboard into two Dockerfiles
Turnstyle: Forces actions to execute one at a time for all actions and jobs that use turnstyle.
After this PR is merged, I'll change the Github secrets with the new cluster information.

The linter issues are being fixed in another PR as they have nothing to do with this task.

repo: openeew/openeew-dashboard


text:
throwErrno symbols missing when Iex is linked before other libraries statically

I'm not sure anything can be done about this, but in case someone else has this problem, I'll file it. I won't be offended if its closed :)
We got symbol errors when OpenEXR tried to throw an exception based on an errno. We linked OpenEXR statically and the link order was "Half Iex ...[rest]". This caused a symbol error when it tried to call throwErrno. By moving Iex to the end of the link order, we fixed this issue.

Yeah, I'm pretty sure this is standard linker behavior, symbols are searched for left to right. But to be clear - this link order is in your application Makefile, not from OpenEXR's build files, right?

Yeah, it was the order in our CMake scripts. Probably nothing OpenEXR can do about it.

Closing the issue for now, feel free to re-open or file a new issue if you need further help.

authors: cary-ilm skurmedel

repo: openexr/openexr


text:
Fix specs

What's New?
This will disable the tests that fail on CI on the master branch today.
It will give us a more accurate picture of what's actually broke vs. what is a CI / config issues.
Once this is merged I will get back to the bundler upgrades.

Coverage increased (+0.1%) to 99.077% when pulling 12085ca08014be0046ca372622b2fdf3e1eb80df on RickCarlino:fix_specs into e9bca8654f81cfcfc9ae968c4488de02eed49779 on openfarmcc:master.

authors: RickCarlino coveralls

repo: openfarmcc/OpenFarm


text:
Remove references of geodata and dataset link relation type

They are currently not used but listed in the table.

Note: We will also separate the resources and link relation types in Section 6, and move the list of relation types to section 5 - conventions.

Actually those geodata and dataset link relation types are used for the Coverage Tiles conformance classes, since they are used in tilesets, so keeping them.
The link relation types were moved to a separate section in Section 5: Conventions.

repo: opengeospatial/ogcapi-coverages


text:
支持 navigation timing

W3C: https://w3c.github.io/navigation-timing/

https://www.w3.org/TR/navigation-timing-2/

repo: openkraken/kraken


text:
e.dragging is updated too late when starting dragging the map

In the pointermove callback event, we have the e.dragging var which is true if we are dragging the map, and false if not.
The thing is apparently that var is updated too late in the process.
Steps to reproduce the behavior:
Go to :  https://codepen.io/sylvainbeo/pen/Vwwdpvb
And follow the instructions in the JS code
This can be a problem, and this is a problem in my case. Because in the time the e.dragging is not updated, my code is executed, and i do a forEachFeatureAtPixel() which is a bit slow (i'm using vector tiles) resulting on a non fluid navigation on the map (there is an impression of the map stuck on the screen for half a second).
For infos, we have been debugging a bit OL with @elemoine, and it seems not so easy to fix (?)
I let Eric answer to give more information if he wants to.
To me, once the mouse is down, the pointermove event should know that we are possibily in a drag mode. And that's not the case i think. We wait until the user has really move the mouse.
Thank you.

We are using a tolerance for click events. As long as you are within that tolerance, your drag might end up being a click, so we cannot say for sure we're e.dragging.
You can configure your map with moveTolerance: 0´ to get e.dragging` set earlier, but that will make it almost impossible for the user to do a click.

We are using a tolerance for click events. As long as you are within that tolerance, your drag might end up being a click, so we cannot say for sure we're e.dragging.

I understand.
But at least, pressing the mouse button down should set the dragging param to true after a few millisecond, don't you think so ? When you press the button for dragging, you won't release it before a certain period of time.

authors: ahocevar sylvainbeo

repo: openlayers/openlayers


text:
ImageStatic source img url does not work

Could not load content for webpack://ol/src/extent.js (Fetch through target failed: Unsupported URL scheme; Fallback: HTTP error: status code 404, net::ERR_UNKNOWN_URL_SCHEME)
Not able to load image from local server with path like '/images/background/1.png'. This path working fine for featurues icons, but does not work for static Image.
Even I've tried server generated link which does working if I paste to the browser I get same error when Im trying to load as ImageStatic.
http://intranet/wcsfiles/icons/user.png
var projection = new ol.proj.Projection({
  code: 'xkcd-image',
  units: 'pixels',
  extent: [0, 0, 1024, 968],
});

var source = new ol.source.ImageStatic({
  attributions: '',
  url: '/images/background/1.png',
  projection: projection,
});

map.getLayers().forEach(function(layer) {
  if (layer instanceof ol.layer.Image) {
    layer.setSource(source);
  }
});

Same problem here. Have you fixed it?

Yes I've added
imageExtent: [0, 0, 1024, 968], after projection inside source variable.


text:
Vector.Redraw should set zoomChanged to true in moveTo

See #305
Except for Tile/Image (#311) all tests pass in FireFox 10, Chrome 17, Safari 5.1, IE8 and IE9.

As discussed on IRC, please merge.

authors: ahocevar elemoine


text:
webpack:"Cannot read property 'getExtent' of null"

I use webpack to pack my app.But in the production environment, there are always errors like "Cannot read property'get Extent'of null". What did I do wrong? I want to know the answer.By the way, there is no problem in the development environment.
Complete:
Uncaught TypeError: Cannot read property 'getExtent' of null
at qv (main.6f105a2a60e74063e468.js:1)
at new t (main.6f105a2a60e74063e468.js:1)
at t.baseGmapImg (main.6f105a2a60e74063e468.js:1)
at t.baseMapGoogle (main.6f105a2a60e74063e468.js:1)
at t.init (main.6f105a2a60e74063e468.js:1)
at new t (main.6f105a2a60e74063e468.js:1)
at main.6f105a2a60e74063e468.js:1
at Fo (main.6f105a2a60e74063e468.js:1)
at main.6f105a2a60e74063e468.js:1
at new e (main.6f105a2a60e74063e468.js:1)
ol -----> 5.3.0
webpack ----->4.25.1
angular----->6.1.10

This issue was fixed with #8893

@fredj Same issue here, I am using
"ol": "^5.3.0",

Help will be highly appreciated.

@navinna please try with ol version 5.3.1

@fredj Thank you! it is working after updating

authors: fredj navinna yang-small-fan


text:
Include IE fallback for PointerEvent constructor

Fixes #12226
Include IE fallback for PointerEvent constructor and target property

Closed in favour of #12330 as this only fixed IE and not other old browsers (e.g. old Android).


text:
Fix webgl PointsLayer not rendering anything

Attributes were used in the shader but not bound to a buffer, which made the rendering fail in some implementations.
This should fix #9014 (to be confirmed)
Example visible at https://498-4723248-gh.circle-artifacts.com/0/examples/heatmap-earthquakes.html

Works for me in Chrome 70 and Safari 12 on macOS 10.14.  Thanks for the continued work on this @jahow.

authors: jahow tschaub


text:
Report query: Overdue and lost

Report of all requests where the due date has passed and the state is not "Complete."
Report mockup: https://docs.google.com/spreadsheets/d/1KgfneTJZAQGow_87_Mra-xGqsAi9AmtWVnS0ph4VO4Y/edit#gid=0

At the June 9 meeting, we clarified that having requests with an "overdue" status is not required to begin working on this report. The query will select all requests where the due date has passed and the state is not "complete." At the moment, there are still no overdue requests in the test system, but I will continue to work on getting some in. I think there is bug interfering with this.

I would like to work on this issue

https://github.com/openlibraryenvironment/reshare-analytics/commit/c0efa8be727bca63b436cc3765f369a3c8d418a9

authors: kristenwilson riyaku11 scwiek


text:
Container version

Description
Make the openmethane package version available inside the container so that data files can be annotated with the version that was used to process them.
Checklist
Please confirm that this pull request has done the following:

[x] Tests added
[x] Documentation added (where applicable)
[x] Changelog item added to changelog/

Notes

@prayner the container will now have access to the OPENMETHANE_VERSION env var when running shell or python scripts. Where do you think the best place would be to embed the package version so that it flows through to our result data files?
This container is used for several jobs:

CMAQ fetching and preprocessing
Fetch and process TROPOMI data
Bias correction
fourdvar daily run
fourdvar monthly run

At a minimum, I'd say we should write the version:

fourdvar daily run
fourdvar monthly run

I believe both of these make use of the archive method, so this might be a logical place for it:    - possibly in the archive method? https://github.com/openmethane/openmethane/blob/d0b65742a5aa3421e7af29d13d2053f0661fe09d/src/fourdvar/datadef/abstract/_physical_abstract_data.py#L123-L127

I just checked that calculate_average_emissions in postproc copies
attributes from the dataset for the prior to the output dataset.
However the prior.nc file is produced by make_prior.py which doesn't
use PhysicalAbstractData so the environment variable will need to be
written explicitly into the attributes of the root group. It should
also get written as a key in the domain definition at the start of the
obs list produced by tropomi_methane_preprocess .

However the prior.nc file is produced by make_prior.py which doesn't use PhysicalAbstractData so the environment variable will need to be written explicitly into the attributes of the root group.

It looks as though the setup method in user_driver.py does load prior.ncf using the PhysicalData class and then writes it back to disk:
https://github.com/openmethane/openmethane/blob/d6d30aa745e10dc67e28dd2897c600fc85db3139/src/fourdvar/user_driver.py#L49-L51
Sure enough, I'm seeing the openmethane_prior attribute when I view it with ncdump after doing a local run.

It should also get written as a key in the domain definition at the start of the obs list produced by tropomi_methane_preprocess .

I'll look at adding this.

It should also get written as a key in the domain definition at the start of the obs list produced by tropomi_methane_preprocess .

I've added it to get_domain within obs_preprocess.model_space which I believe is where the domain definition is created.
    def get_domain(self):
        domain = deepcopy(self.gridmeta)
        domain["SDATE"] = self.sdate
        domain["EDATE"] = self.edate
        domain["openmethane_version"] = os.getenv('OPENMETHANE_VERSION')

authors: aethr prayner

repo: openmethane/openmethane


text:
Update Dell FC630 bios and idrac versions

Updates  the dell-update-fc630.yml playbook with the latest versions of the BIOS and iDRAC. These can be deployed with ansible-playbook -i <INVENTORY> hardware-dell/dell-update-fc630.yml -l <LIMIT-HOSTS> --ask-become-pass and will involve an automatic reboot.

Seems fine.

authors: joshmoore manics

repo: openmicroscopy/infrastructure


text:
Organizations designated as terrorist by Bahrain #159

fixes opensanctions/crawler-planning#159

I'm gonna close for now - the source doesn't cover the individuals, so I think it's best to hold off until we have a better source.

authors: jbothma nvmbrasserie

repo: opensanctions/opensanctions


text:
fcda-binding-list test fails

Describe the bug
fcda-binding-list test sometimes fails. It looks like a synchronization/timing issue.
To Reproduce
Steps to reproduce the behavior:

run npm run test:watch in open-scd
Check the results, it doesn't always happen which make it hard to reproduce

Expected behavior
Test will always succeed and always have the same behaviour.
Screenshots

@danyill has similar experiences
Possible solution: add an animation frame

repo: openscd/open-scd


text:
Nodeplugin pod falls “CrashLoopBackOff” status in second or later deployment when using helm

Is this a BUG REPORT or FEATURE REQUEST?:

/kind bug

What happened:
First deployment using helm(opensds-installer) works fine. But if nodeplugin pod is deleted once, next generated nodeplugin pod falls “CrashLoopBackOff” status. Then k8s node mounting “/dev/termination-log (deleted)”. After unmount it and deploy nodeplugin pod, nodeplugin pod works again.
What you expected to happen:
Nodeplugin pod start normally.
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Environment:

Hotpot(release/branch) version: Aruba
OS (e.g. from /etc/os-release):
Kernel (e.g. uname -a):
Install tools:
Others:

This issue would be solved by #75

authors: leonwanghui wisererik

repo: opensds/opensds-installer


text:
Acl creation is not creating backend rule properly

Is this a BUG REPORT or FEATURE REQUEST?:

Uncomment only one, leave it on its own line:
/kind bug
/kind feature

What happened:
When first time i tried to create acl with "read" permission and second time with read and write permissions for the same ip. But i can't find both the rules on backend
opensds db entry is created but actual rule i didn't find filesystem
"exportfs -v"  has only one entry i.e second rule
What you expected to happen:
When we create 2 different acl rule for the same IP. It should create both rules on backend file system
How to reproduce it (as minimally and precisely as possible):
First create file share
For that file share create 2  different acl rule for the same ip using cli/dashboard/curl command.
Anything else we need to know?:
Environment:

Hotpot(release/branch) version:
OS (e.g. from /etc/os-release):
Kernel (e.g. uname -a):
Install tools:
Others:

@PravinRanjan10, Please take a look at this issue

For same IP, two rules not possible. This is not a valid scenario. Always 2nd rule will be updated if user have give same ip. This issue is sort of validation in dashboard/API(I mean two entries of rule must not be there for same IP)

@PravinRanjan10 , It it is so then fix the issue in apiserver itself.

@Shruthi-1MN @PravinRanjan10 I'm going to add restrictions in dashboard

@jimccfun  what restriction you are going to add??

/assign @PravinRanjan10

It is fixed in PR #1019, so closing this issue.

it's not fixed, I reopen it

It is fixed by this PR #1143

repo: opensds/opensds


text:
Decouple attach dock and provision dock

What this PR does / why we need it:
Normally Provision Dock will trigger capability report but Attach Dock will not, so we need to decouple these two services to avoid some errors.
Which issue this PR fixes:
None
Special notes for your reviewer:
Release note:

Coverage increased (+1.9%) to 60.048% when pulling ae6d13aee79c68bf82e2d79784fe7a5b32752820 on leonwanghui:dock-updater into f394ae84aafaa3b449edec5d7bb1fc99ca1bd4b7 on opensds:development.

Coverage increased (+1.9%) to 60.048% when pulling ae6d13aee79c68bf82e2d79784fe7a5b32752820 on leonwanghui:dock-updater into f394ae84aafaa3b449edec5d7bb1fc99ca1bd4b7 on opensds:development.

Coverage increased (+1.9%) to 60.048% when pulling ae6d13aee79c68bf82e2d79784fe7a5b32752820 on leonwanghui:dock-updater into f394ae84aafaa3b449edec5d7bb1fc99ca1bd4b7 on opensds:development.

Coverage decreased (-0.4%) to 57.8% when pulling ae6d13aee79c68bf82e2d79784fe7a5b32752820 on leonwanghui:dock-updater into f394ae84aafaa3b449edec5d7bb1fc99ca1bd4b7 on opensds:development.

authors: coveralls leonwanghui


text:
Adding multiple overlays using XML, how to add onclick function to each overlay

I am trying to add multiple overlays grabbing information from an xml file. I have successfully added the overlays (as discussed in #1419), but I am unable to add onclick function to each of the overlays. I get an htmlcollection from the xml, so I think I cannot use ForEach. I added the mousetracker inside the for loop. The onclick function is added but only the last id is shown when clicked on any of the overlays. Here is my code. Any pointers appreciated.
	function getHSlist(xml) {
		var i;
		var xmlDoc = xml.responseXML;
		var x = xmlDoc.getElementsByTagName("hotspot");

		for (i = 0; i <x.length; i++) {

			var overlayid = x[i].getElementsByTagName("id")[0].childNodes[0].nodeValue;
			var overlayx = Number(x[i].getElementsByTagName("x")[0].childNodes[0].nodeValue);
			var overlayy = Number(x[i].getElementsByTagName("y")[0].childNodes[0].nodeValue);
			var overlaytitle = x[i].getElementsByTagName("title")[0].childNodes[0].nodeValue;
			var overlaysrc = x[i].getElementsByTagName("src")[0].childNodes[0].nodeValue;

			var elt = document.createElement("img");
			elt.id = overlayid;
			elt.className = "hotspot";
			elt.src = 'images/info-icon.png';
			elt.title = overlaytitle;
			elt.alt = overlaytitle;

			viewer.addOverlay({
				element: elt,
				location: new OpenSeadragon.Point(overlayx, overlayy),
				placement:	OpenSeadragon.Placement.CENTER,
				checkResize: false
			});

			var tracker = new OpenSeadragon.MouseTracker({

				element: overlayid,
				clickHandler: function(event) {
					alert (overlayid);
				}
			 });
		}
	}

Wow.. Just tried it and it works. Thanks. I was trying to find a solution all day and only just found that setAttribute worked .. not sure which way is better:
function getHSlist(xml) {
var i;
var xmlDoc = xml.responseXML;
var x = xmlDoc.getElementsByTagName("hotspot");
			for (i = 0; i <x.length; i++) {

				var overlayid = x[i].getElementsByTagName("id")[0].childNodes[0].nodeValue;
				var overlayx = Number(x[i].getElementsByTagName("x")[0].childNodes[0].nodeValue);
				var overlayy = Number(x[i].getElementsByTagName("y")[0].childNodes[0].nodeValue);
				var overlaytitle = x[i].getElementsByTagName("title")[0].childNodes[0].nodeValue;
				var overlaysrc = x[i].getElementsByTagName("src")[0].childNodes[0].nodeValue;

				var elt = document.createElement("img");
				elt.id = overlayid;
				elt.className = "hotspot";
				elt.src = 'images/info-icon.png';
				elt.title = overlaytitle;
				elt.alt = overlaytitle;

				viewer.addOverlay({
					element: elt,
					location: new OpenSeadragon.Point(overlayx, overlayy),
					placement:	OpenSeadragon.Placement.CENTER,
					checkResize: false
				});

				var funcStr = "alert ('" + overlayid + "')";
				elt.setAttribute("onclick", funcStr);

			}
		}

repo: openseadragon/openseadragon


text:
Load tiles from the closest level before tiles from "covered" layers

Hi,
In our project we are using OSD to load tiles from a remote server since they are created on the spot.
When a user zooms fast it seems like the tiles are loaded from the farthest level to the closest one by one, and the render time and load time of each causes a bad experience to the user.
Is there a way to make the viewer render the tiles that are in the FOV before (or instead of) loading the ones that will be covered by them?
Thanks.

Yes, use immediateRender: true in your options when creating the viewer.

I went over the properties a lot and somehow still missed it :)

authors: iangilman ibex-lior


text:
dzi url without extension

Greetings,
what a great library !
Back to the point,
I need to expose my generated dzi xml with a url like http://example.com/A123, so without .dzi extension. My files are in http://example.com/A123_files
The missing extension somewhat confuses tiles url building, and the browser try to retrieve urls like http://example.com/A1239/0_0.jpg instead of http://example.com/A123_files/9/0_0.jpg.
I tried to change the regex that builds tilesUrl in dzitilesource.js line 142 this way and it seems to work
options.tilesUrl = url.replace(/([^\/]+)(.(dzi|xml|js))|\?|\/?$/, '$1_files/');

Good point! I think you'll actually want to change the regex to ([^\/]+)(\.(dzi|xml|js))?(\?.*|$) though.
Would you be up for making a pull request with this?

Sure,
I've come with these cases:

Data url
Tiles root url

/path/my.dzi
/path/my_files/

/path/my
/path/my_files/

/my/
/my_files/

my.xml
my_files/

/p/foo.dzi?a=1&b=2
/p/foo_files/

If the cases are OK, the pull request implementing this is #835

this breaks the case:
/iipsrv?DeepZoom=/my.dzi

I would suggest changing regex to /([^\/]+?)(\.(dzi|xml|js))?(\?[^\/]*)?\/?$/
that seem to cover mentioned cases as well as mine

authors: altert gaetano-guerriero iangilman


text:
Replication IT for #12047

Description
Detailed reproduction of issue described in #12047 implemented as an IT.
The goal is to see if we can reproduce the issue – currently the issues is not reproducible.
This is WIP to drive the discussion further, do not merge it!
Related Issues
Relevant to #12047
Check List

~[ ] New functionality includes testing.~

~[ ] All tests pass~

~[ ] New functionality has been documented.~

~[ ] New functionality has javadoc added~

~[ ] Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)~
[x] Commits are signed per the DCO using --signoff
~[ ] Commit changes are listed out in CHANGELOG.md file (See: Changelog)~
~[ ] Public documentation issue/PR created~

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Can someone add a skip-changelog label please?

repo: opensearch-project/OpenSearch


text:
Optimized ClusterStatsIndices to precomute shard stats

Description
The current approach for generating cluster stats indices incurs significant overhead in large clusters with numerous shards distributed across nodes. This issue arises due to the following steps:

Fetching Shard-Level Stats: Each node fetches a map containing individual shard statistics.
Accumulating on Request Node: The node receiving the REST request (request node) gathers this information from all participating nodes.
Inefficient Data Structures: Creating hash-maps from the StreamInput and iterating through them for accumulation are computationally expensive.

This pull request (PR) addresses this efficiency bottleneck by setting up each node to pre-compute its own statistics prior to sending the response. This allows the requested node to only perform node-level accumulation instead of the shard level. This is done by returning a new NodeIndexShard object containing the accumulated node-level statistics used by the request node for final aggregation.
The optimisation hides behind a flag - useOptimizedClusterStatsResponse (Defaults to False). This is only enabled in RestClusterStatsAction.
Improvement statistics:
Tested on a 20k shards (empty) cluster with 500 attempts and parallelisation of 5.

Time taken for complete call
Flag Disabled
Flag Enabled

Average (s)
0.77155
0.20277

P90 (s)
1.04287
0.32985

Max (s)
1.50661
0.62719

Min (s)
0.26654
0.08716

Related Issues
Resolves #14714

Check List

[x] Functionality includes testing.
[ ] API changes companion pull request created, if applicable.
[ ] Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@Pranshu-S can you also share memory overhead on the node serving the request should have reduced after the optimization

Testing on a large cluster yielded the following results -
Current behavior

Optimisation results

Without optimisation
With optimisation

Allocations in Heap in 100 calls serially
321 GB
30.44 GB

CPU Samples in ClusterStatsResponse.init
68536
15007

CPU Samples in TransportClusterStatsAction.newNodeResponse
84359
15009

Total Heap Size acumulated over 100 calls
1.39TB
885.53 GB

❌ Gradle check result for aac6776: FAILURE
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Test Result (1 failure / +1)
org.opensearch.action.admin.indices.create.RemoteCloneIndexIT.testCreateCloneIndexFailure
Flaky - https://github.com/opensearch-project/OpenSearch/issues/14292

❌ Gradle check result for 79a7c19:
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Test Result (1 failure / ±0)
org.opensearch.remotestore.RemoteStoreStatsIT.testDownloadStatsCorrectnessSinglePrimaryMultipleReplicaShards
Flaky - https://github.com/opensearch-project/OpenSearch/issues/14310

❌ Gradle check result for 31fd6d8: FAILURE
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Test Result (2 failures / +1)
org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.classMethod
org.opensearch.action.admin.cluster.node.tasks.ResourceAwareTasksTests.testTaskResourceTrackingDuringTaskCancellation
Flaky - https://github.com/opensearch-project/OpenSearch/issues/14296, https://github.com/opensearch-project/OpenSearch/issues/14293

Maybe someone more familiar with this are (@shwetathareja or @Bukhtawar?) can please CR this?

Test Result (2 failures / +2)
org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
org.opensearch.remotestore.CreateRemoteIndexClusterDefaultDocRepIT.testDefaultRemoteStoreNoUserOverrideExceptReplicationTypeSegment
Flaky behaviour -
CreateRemoteIndexClusterDefaultDocRepIT.testDefaultRemoteStoreNoUserOverrideExceptReplicationTypeSegment
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=4922, name=Thread-71, state=RUNNABLE, group=TGRP-CreateRemoteIndexClusterDefaultDocRepIT]

Similar to tests in - https://github.com/opensearch-project/OpenSearch/issues/14381

org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
Flaky - https://github.com/opensearch-project/OpenSearch/issues/14289

authors: Pranshu-S dblock shwetathareja


text:
Adding workflow to auto delete backport merged branches from backport workflow

Signed-off-by: Vacha Shah vachshah@amazon.com
Description
Workflow to automatically delete the branches created by the backport action  of type backport/*.
I will backport this PR to release branches so that the auto delete runs on backport PRs created for those branches.
Issues Resolved
Part 2 of #1962
Check List

[X] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Is it possible to put both workflows for backports into backport.yml?

I kept them separate since the backport workflow will run on PRs on the main branch while the delete branch workflow needs to run on backport PRs created on the release branches. WDYT?

Is it possible to put both workflows for backports into backport.yml?

I kept them separate since the backport workflow will run on PRs on the main branch while the delete branch workflow needs to run on backport PRs created on the release branches. WDYT?

No strong feelings. My only argument to combine is that when other projects want to introduce the backport process they would need to copy 2 workflows instead of 1.

No strong feelings. My only argument to combine is that when other projects want to introduce the backport process they would need to copy 2 workflows instead of 1.

Thats true. This just came to my mind, combining them might not be possible since both the workflows have different on to define what kind of events trigger the workflow.

@VachaShah Feel free to merge if you're happy with this as is.

This works for new backport PRs. How about the ones which already exist?

For the ones that already exist, I will delete them from the PRs. We just updated the setting to allow deletes such branches.

Auto-backported this PR to 1.x #2065 and the backport branch is auto-deleted after merge.

authors: VachaShah dblock


text:
[DRAFT] Add PrimaryShardBatchAllocator to take allocation decisions for a batch of shards

Description
This PR is a continuation of the project https://github.com/opensearch-project/OpenSearch/issues/5098
As part of node drop and joins, we fetch the metadata of unassigned shards on per shard basis.
https://github.com/opensearch-project/OpenSearch/blob/c25c175c94efa4d90a1d98e3c7eac3e4aad6e41f/server/src/main/java/org/opensearch/gateway/AsyncShardFetch.java#L74
PrimaryShardAllocator can build allocation decision for a single unassigned shard. We need to add support for building decisions for a batch of shards, so adding new class for that.
Note :

UTs and ITs will be added in upcoming commits.
Gradle check will fail on compilation because this PR is dependent on multiple other PRs :
Not creating a helper class for common code as old PrimaryShardAllocator class will be deprecated in future.
Most of the code is copy pasted from PrimaryShardAllocator class, only changes are handling of multiple shards metadata and handling of single DiscoveryNode object in response for a batch of shards.
CHANGELOG will be updated only after last PR is raised(being merged) for this project.

#8218
#8356
#8742
#8776

Additional context
Please go through the discussion https://github.com/opensearch-project/OpenSearch/issues/5098 to understand the overall enhancement approach and check https://github.com/opensearch-project/OpenSearch/issues/8098 to see the sub tasks of overall project.
Related Issues
Resolves #8960
Check List

[x] New functionality includes testing.

[ ] All tests pass

[x] New functionality has been documented.

[x] New functionality has javadoc added

[x] Commits are signed per the DCO using --signoff
[x] Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

PR will be updated in few days.

This PR will be picked by @shwetathareja after other prerequisite PRs are done.

testConcurrentDecommissionAction

Flaky test : https://github.com/opensearch-project/OpenSearch/issues/12197

❌ Gradle check result for d372c54: null
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Flaky test testConnectAndExecuteRequest  https://github.com/opensearch-project/OpenSearch/issues/12338

❌ Gradle check result for d372c54: null
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Flaky test testDelayWithALargeAmountOfShards : https://github.com/opensearch-project/OpenSearch/issues/10558


text:
[BUG] The spotless should take care of unified imports ordering (benchmarks/ subprojects)

Description
Came out of https://github.com/opensearch-project/OpenSearch/pull/9082#pullrequestreview-1572289035, the spotless does not enforce any formatting on order of imports statements, as such those are very chaotic in the codebase.
Related Issues
Part of https://github.com/opensearch-project/OpenSearch/issues/9306, since there are ~3k files changed across all subprojects, sending smaller pull requests for each of the subproject.

Check List

[ ] New functionality includes testing.

[ ] All tests pass

[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[x] Commits are signed per the DCO using --signoff
[ ] Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@reta are these import changes done manually or did we add importOrder here to get the correct order of the imports?

@owaiskazi19 it is all automatic (please check the https://github.com/opensearch-project/OpenSearch/issues/9306 for the formatting rules), it is just too big as one shot change, so I am sending small pull requests and than would send the final one with formatting rules, thanks!

@owaiskazi19 it is all automatic (please check the #9306 for the formatting rules), it is just too big as one shot change, so I am sending small pull requests and than would send the final one with formatting rules, thanks!

I see. Reminded me of the same way we followed while applying spotless through the whole project.

I see. Reminded me of the same way we followed while applying spotless through the whole project.

Exactly, thanks @owaiskazi19 !


text:
Restricting logs permissions

Description
Currently, the permissions for opensearch logs are -rw-r-r-, which gives read access to anyone. This weak permission structure can lead to leakage of any sensitive information (if published) in the logs. This commit restricts read access for logs with -rw-r--- permission.
New permission structure (verified w/ linux and darwin distros)
-rw-r----- 1 ubuntu ubuntu  7016 Jul 14 19:23 opensearch.log
-rw-r----- 1 ubuntu ubuntu     0 Jul 14 19:23 opensearch_deprecation.json
-rw-r----- 1 ubuntu ubuntu     0 Jul 14 19:23 opensearch_deprecation.log
-rw-r----- 1 ubuntu ubuntu     0 Jul 14 19:23 opensearch_index_indexing_slowlog.json
-rw-r----- 1 ubuntu ubuntu     0 Jul 14 19:23 opensearch_index_indexing_slowlog.log
-rw-r----- 1 ubuntu ubuntu     0 Jul 14 19:23 opensearch_index_search_slowlog.json
-rw-r----- 1 ubuntu ubuntu     0 Jul 14 19:23 opensearch_index_search_slowlog.log
-rw-r----- 1 ubuntu ubuntu 11779 Jul 14 19:23 opensearch_server.json

Issues Resolved
#958
Check List

[ ] New functionality includes testing.

[ ] All tests pass

[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.
Signed-off-by: Himanshu Setia setiah@amazon.com

start gradle check


text:
[Segment Replication] Update version back to 2_10_0 in Segmentstats.

Description
With this PR we have temporarily changed the version from 2_10_0 to 3_0_0. As the PR is merged and backported we can now revert it back to 2_10_0
Related Issues
Resolves #[Issue number to be closed when this PR is merged]

Check List

[ ] New functionality includes testing.

[ ] All tests pass

[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[ ] Commits are signed per the DCO using --signoff
[ ] Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Merging with failing Gradle check since we have a deadlock here: the BWC tests are broken by 2 pull requests, and we need to get this one and https://github.com/opensearch-project/OpenSearch/pull/9722 to fix main.


text:
Incremented version to 1.3.4.

Issue #, if available:
Part of https://github.com/opensearch-project/opensearch-build/issues/2205
Description of changes:
Increment to 1.3.4
CheckList:
[ ] Commits are signed per the DCO using --signoff
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@opensearch-project/alerting-plugin Can we get another approval for this one? This is blocking our process for release 1.3.4

repo: opensearch-project/alerting


text:
[BUG] Missing geotile_grid aggregation documentation

What is the bug?
The documentation website does not cover the geotile_grid aggregation
How can one reproduce the bug?
Navigate to: https://opensearch.org/docs/latest/opensearch/bucket-agg/#geo_distance-geohash_grid
What is the expected behavior?
Documentation to include geotile_grid aggregation.
What is your host/environment?
Internet / Chrome
Do you have any screenshots?
I can provide if needed
Do you have any additional context?

@nknize: If you do have screenshots, they would be useful here.

Hi @nknize - the documentation for geotile_grid is here: https://opensearch.org/docs/latest/aggregations/bucket/geotile-grid/. Thanks so much! I am closing this issue.

authors: Naarcha-AWS hdhalter nknize


text:
[Backport 2.x] add sample templates for text to visualziation (#936)

Backport e259c12ebf40dcf8677343a1744643804f9efd02 from #936

DCO was failing on the backport because author and DCO signoff emails differed:
commit 8549d9a4dc63b9882d9befae07fd697067456876
Author: Yulong Ruan <ruanyu1@gmail.com>
Date:   Wed Nov 20 03:05:43 2024 +0800

    add sample templates for text to visualziation (#936)

    * add sample templates for text to visualziation

    Signed-off-by: Yulong Ruan <ruanyl@amazon.com>

Reset the author on the cherry-picked commit to match DCO, but in the future @ruanyl would be helpful if you use the same email you're committing with as your DCO signature.

repo: opensearch-project/flow-framework


text:
[BUG]: Table view toggle button gets enabled for Heatmap chart

What is the bug?
After removing stats from the PPL query when we select Heatmap chart from the chart selector the table view gets enabled and when we click on table view toggle button the page turns blank
How can one reproduce the bug?
Steps to reproduce the behavior:

Go to Observability/Event Analytics/Explorer page
Navigate to Visualizations page
Remove stats section from the PPL query and click on Refresh button
Select the Heatmap chart option from chart selector dropdown
Click on table view toggle button
Observe the issue

What is the expected behavior?
If we don't have stats in the PPL query then the table view toggle button should be disabled
Do you have any screenshots?

@anirudha @mengweieric
cc: @abasatwar @spattnaik

Bug has been fixed and tested.

repo: opensearch-project/observability


text:
Build incremental components through build workflow with loading previous build manifest

Description
This is the last PR to enable incremental build on the Python workflow.
The idea of this PR is to build those components that were detected by rebuild_component from build_incremental class.
Firstly we add a new parameter to the build_recorder class so that it could load existing build manifest if provided; otherwise generate a brand new manifest.
Then similar to how we build in run_build.py, here for incremental build, we are importing the previous build manifest and only build the list of components from buildIncremental.rebuild_plugins().
Issues Resolved
https://github.com/opensearch-project/opensearch-build/issues/4210
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

I'm looking to add additional tests to fix the cov.

PR is ready for review. Seems like some issues with Maven so that the groovy test failed.

This is pretty clean. See below, rename/simplify. Document this feature?

@dblock Just added some documentations. Please help review it as well. Thanks!

Just reviewed with @zelinh on this PR.
I have no issues with the implementation and is clear on the details.
Regarding the conflict between --component and --incremental, is it possible to go these two routes:

If both parameters present, error out the build and provide information in console, that user can only use either parameter, not both at the same time.
If both parameters present, output a warning msg in the console, notify user that --component will not take effect when --incremental is presented at the same time.

The 1st route needs more implementation, while the 2nd route requires one more logging.
Please let me know what you think.
Thanks.

After discussing again among @gaiksaya @zelinh and I, we have decided to make both --component and --incremental mutually exclusive by following the solution here:
https://github.com/opensearch-project/opensearch-build/blob/main/src/validation_workflow/validation_args.py#L114
Later, we can make both parameters work together, in cases where user want to use --incremental function to verify if there is a change within the component(s) defined by --component parameter.

Updated based on what @peterzhuamazon @gaiksaya and I discussed earlier.
I make --components and --incremental mutually exclusive for now. We can discuss about the future enhancement later.
If the rebuild list is empty, the whole build process will stop and skip building from everything.
Please help review it again. Thanks! @peterzhuamazon @gaiksaya @dblock

Almost there!
Please add test for skipping the entire build and ensure the graceful exit. Thanks!

Added new tests. I tried locally and it works.
[opensearch@c81bdd48ecdb opensearch-build]$ ./build.sh manifests/2.12.0/opensearch-2.12.0-inc.yml --incremental
Installing dependencies in . ...
Installing dependencies from Pipfile.lock (fcf5b2)...
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
Running ./src/run_build.py manifests/2.12.0/opensearch-2.12.0-inc.yml --incremental ...
2024-01-04 00:47:28 INFO     Loading tar/builds/opensearch/manifest.yml
2024-01-04 00:47:29 INFO     Updating ref for https://github.com/opensearch-project/OpenSearch.git from 923aa25367006b830a0b66aca82eb9c47dafba22 to 923aa25367006b830a0b66aca82eb9c47dafba22 (923aa25367006b830a0b66aca82eb9c47dafba22)
2024-01-04 00:47:29 INFO     Updating ref for https://github.com/opensearch-project/common-utils.git from 2.x to 8e2537b6c1cde03449f0818242a1a36089a69522 (refs/heads/2.x)
2024-01-04 00:47:29 INFO     Updating ref for https://github.com/opensearch-project/job-scheduler.git from 2.x to 482c90fccf59a3e02bc5cb60124580c164a9d7c3 (refs/heads/2.x)
2024-01-04 00:47:30 INFO     Updating ref for https://github.com/opensearch-project/geospatial.git from 2.x to 273fa8fe051d469b5a79ee37a1ce203738239ddb (refs/heads/2.x)
2024-01-04 00:47:30 INFO     Updating ref for https://github.com/opensearch-project/security.git from 2.x to 2d7d6702c8aab1fbb25ab660f3432062fa564e10 (refs/heads/2.x)
2024-01-04 00:47:30 INFO     No commit difference found between any components. Skipping the build

Issue created here as we discussed. https://github.com/opensearch-project/opensearch-build/issues/4320

authors: peterzhuamazon zelinh

repo: opensearch-project/opensearch-build


text:
[FEATURE] Hadoop Client

Is your feature request related to a problem?
When using Elastic Search in the past, some in the community had been used to connecting using a Hadoop client. The ask is to create a Hadoop client which will connect to OpenSearch.

Thanks @harshavamsi  and @wbeckler. Would like to know if we have any development on ETA in last two weeks?

@prashantsc the client is ongoing security review inside of Amazon. We are targeting 4/30/23 currently, but it could change depending on outcomes.

Thanks @harshavamsi  for the information.

@harshavamsi do you have updates on the security review ? We passed 4/30/23 so wondering if a new date is to be set.

@junhl Thanks for checking in. Happy to report that the security review is complete and we're now working towards a release. There is some ongoing discussion over at https://github.com/opensearch-project/opensearch-build/issues/3385 about the nuances of releasing this client. That is also where we will track the release. Stay tuned this week where most likely a release is going to happen. Thank for being patient!

Spark 3.x clusters can't connect to Opensearch using elasticsearch-spark libraries. Somehow the compatibility is broken and the connection fails with
org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot detect ES version
However Spark 2.x clusters don't have this problem. Blocker for us because we only have Spark 3.x clusters :(
Is this client coming anytime soon?

Hi @chaitujil the OpenSearch build team is working on getting a release out as soon as possible. In the meanwhile could you let me know if you're using Managed OpenSearch on AWS or if you're hosting your own service. Also, can I know what versions of OpenSearch you're targetting? Thanks.

We are targeting OS 1.1 cluster. Managed OpenSearch.

Hi folks, we published Snapshots here -- https://aws.oss.sonatype.org/content/repositories/snapshots/org/opensearch/client/. Do give it a try and let us know. We're going to publish an actual release early this week. Thanks!

@harshavamsi It looks like the opensearch-spark libs shared at the link above https://aws.oss.sonatype.org/content/repositories/snapshots/org/opensearch/client/ work with Amazon OpenSearch Service with OpenSearch engine v2.3. We want to connect to Amazon OpenSearch Service with Elasticsearch engine v7.10 in the AWS Glue Job (Type: Spark, Glue version - 3.0 or 4.0) . Can you please share the version of the libraries/jar files that we need to use for the same?

@harshavamsi It looks like the opensearch-spark libs shared at the link above https://aws.oss.sonatype.org/content/repositories/snapshots/org/opensearch/client/ work with Amazon OpenSearch Service with OpenSearch engine v2.3. We want to connect to Amazon OpenSearch Service with Elasticsearch engine v7.10 in the AWS Glue Job (Type: Spark, Glue version - 3.0 or 4.0) . Can you please share the version of the libraries/jar files that we need to use for the same?

Hi, we have a workaround that supports ES 7.x clusters. Have you given the connector a try? Does it fail?

Hi Harsha
We want to go away from the AWS Marketplace connector (and open source
ElasticSearch Hadoop library)  which has security issues such as it needs
wide open permissions/policies.

With the latest updates we are good with use case # 1.
We need to use  ElasticSearch Hadoop library if there is no other way
with opensearch hadoop library that uses an Assume role.

Regards,
Prashant
On Mon, May 15, 2023 at 6:46 PM Harsha Vamsi Kalluri <
@.***> wrote:

@harshavamsi https://github.com/harshavamsi It looks like the
opensearch-spark libs shared at the link above
https://aws.oss.sonatype.org/content/repositories/snapshots/org/opensearch/client/
work with Amazon OpenSearch Service with OpenSearch engine v2.3. We want to
connect to Amazon OpenSearch Service with Elasticsearch engine v7.10 in the
AWS Glue Job (Type: Spark, Glue version - 3.0 or 4.0) . Can you please
share the version of the libraries/jar files that we need to use for the
same?
Hi, we have a workaround that supports ES 7.x clusters. Have you given the
connector a try? Does it fail?
—
Reply to this email directly, view it on GitHub
https://github.com/opensearch-project/opensearch-hadoop/issues/134#issuecomment-1548846009,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIKKPACM57OITKQIMXMYB4LXGLL7BANCNFSM6AAAAAAVLDP6ZU
.
You are receiving this because you were mentioned.Message ID:
@.***>

Hi folks, we published Snapshots here -- https://aws.oss.sonatype.org/content/repositories/snapshots/org/opensearch/client/. Do give it a try and let us know. We're going to publish an actual release early this week. Thanks!

Is this release compatible with Open Search 1.2.x and Spark 3.2.x / Scala 2.12 ?

@harshavamsi It looks like the opensearch-spark libs shared at the link above https://aws.oss.sonatype.org/content/repositories/snapshots/org/opensearch/client/ work with Amazon OpenSearch Service with OpenSearch engine v2.3. We want to connect to Amazon OpenSearch Service with Elasticsearch engine v7.10 in the AWS Glue Job (Type: Spark, Glue version - 3.0 or 4.0) . Can you please share the version of the libraries/jar files that we need to use for the same?

Hi, we have a workaround that supports ES 7.x clusters. Have you given the connector a try? Does it fail?

@harshavamsi On referencing the jar file opensearch-spark-30_2.13-1.0.0-20230513.002822-1.jar in our AWS Glue Job (Type: Spark, Glue version - 3.0 or 4.0) we get the following error:
Scala signature package has wrong version expected: 5.0 found: 5.2 in package.class
Does this jar have to be compiled with java 8? Can you please help take a look at the issue?

Hi @harshavamsi , is there a place wherein I can get all the configuration/options supported by Open search through spark?
This helps me in my project to allow users to define these as per the requirement and need.
Using, OpenSearch 1.3.10 with Spark 3.3.1 with Scala 2.12.

@akshayjain3450 please refer to this file -- https://github.com/opensearch-project/opensearch-hadoop/blob/main/mr/src/main/java/org/opensearch/hadoop/cfg/ConfigurationOptions.java. It has all the configuration options that you can set within the client. Thanks!

Closing this issue as completed via #227

@harshavamsi I don't see workflow for scala 2.12 version. are not publishing any jar for Spark 3.x and Scala 2.12 compatable versions?
https://github.com/opensearch-project/opensearch-hadoop/tree/main/.github/workflows

@harshavamsi I don't see workflow for scala 2.12 version. are not publishing any jar for Spark 3.x and Scala 2.12 compatable versions?
https://github.com/opensearch-project/opensearch-hadoop/tree/main/.github/workflows

@venkatbrr The artifact you're looking for is published as  org.opensearch.client/opensearch-spark-30_2.12

@harshavamsi @Xtansia Can you please update the compatability section for 1.0.1 version? we are trying to use Hadoop client for AWS Opensearch managed cluster of 2.5 version.
https://github.com/opensearch-project/opensearch-hadoop/blob/main/COMPATIBILITY.md

authors: Xtansia akshayjain3450 brijos chaitujil harshavamsi indranilr junhl mgolatkar prashantsc venkatbrr

repo: opensearch-project/opensearch-hadoop