text:
Endre party-identifikator til noe mer fremoverkompatibelt

Introduksjon
I stedet for /org/ og /person/<norsk fnr/dnr> trenger vi noe som kan håndtere hva som kommer fremover , som potensielt kan være utenlandske organisasjoner og personer identifisert med ulike typer identikatorer (kan potensielt være en public key fra en wallet).
Beskrivelse
Det er uklart om det i det hele tatt eksisterer noen noenlunde etablert standard for å enkode identfikatorer av ulike typer. ISO6523 benyttes for europeiske organisasjonsnummer, men vil bare være én type identifikator som må kunne støttes. ISO6523 vanskeliggjør bruk av underenheter/indre enehter, fordi disse da skal uttrykkes sammen med juridisk enhet - informasjon vi ikke nødvendigvis har i alle tilfeller (Altinn skiller i liten grad på ulike typer 9-sifrede orgnr)
For fremoverkompabilitet må dette være et format som støtter ulike kodelister, noe tilsvarende consumer-claims i ID/Maskinporten-tokens og sender/receiver i SBD-en brukt i eFormidling.
Siden dette vil være en identifikator og potensielt fremmednøkkel, trenger vi også å lande en skalar enkoding av dette.
{ns}{identifikator-type}{separator}{identifikator}

{ns} vil være et Altinn-spesifikt navnerom for dette, siden dette skal brukes i kontekst av Altinn Autorisasjon
{identifikator-type} må være fra en kodeliste over kjente typer av identifikatorer
{separator} må være noe som utvetydig skiller type fra identifikator
{identifikator} må være verdien som identifiserer parten utfra reglene definert av {identifikator-type}
Identifier-type må være en av organization:identifier-{cc}, person:identifier-{cc}. Det legges til grunn at det kan komme andre identifier types her siden (f.eks. did)
{cc} må være en ISO 3166-1 alpha 2 landskode i lowercase, f.eks. no. I første omgang støttes bare no
Identifier er enhver alfanumeris verdi, og kan innholde .-_/.

Format i ABNF:
rule = ns identifier-type separator identifier
ns = "altinn:"
identifier-type = organization-with-country-code / person-with-country-code
organization-with-country-code = "organization:identifier-" country-code
person-with-country-code = "person:identifier-" country-code
;country-code = alpha alpha  ; ISO 3166-1 alpha-2 country codes
country-code = "no" ; i første omgang støttes kun norske fnr/dnr/orgnr
separator = "::"
identifier = alpha / digit *(alpha / digit / "." / ":" / "-" / "_" / "/")

I XACML må dette være "URI-reference", så vi strukturerer dette som urn-er. Eksempel-verdier:
urn:altinn:person:identifier-no::12345678901
urn:altinn:organization:identifier-no::912345678

### Oppgaver
- [x] Lande enkoding
- [x] Validere Create på SO-API
- [x] Validere search (Party-parameter)
- [x] Oppdatere dokumentasjon
- [ ] Legge inn e2e-test (hvis relevant)
- [x] Manuelt gjøre konvertering i database

Se også

https://github.com/Altinn/altinn-authorization/issues/657

@benedicteos

#390

#395

authors: elsand knuhau oskogstad

repo: digdir/dialogporten


text:
Reuse Election code

Fixes https://github.com/digital-asset/daml-finance/issues/661

I was wondering whether the right place for the election logic is Daml.Finance.Lifecycle or Daml.Finance.Claims

I actually tried putting it in Daml.Finance.Claims first, but it resulted in a build error. See commit c52e413 above. I think we would have to change the build order of the packages if we want it there.
By the way, I noticed that the ElectionEffect was already in Daml.Finance.Lifecycle.

Merging this now to benefit from the Reuse part. We can move it later on if we wish to do so.

source: gharchive/pull-request

repo: digital-asset/daml-finance

license_type: permissive

license_source: github-api


text:
CONTRIBUTION.mdとCONTRIBUTION.ja.mdの作成

関連するチケット

#7

概要
コントリビューション方法について、適切なドキュメントを作成する。
それぞれのドキュメントでは次のことを説明する。

pull requestを送る際には、issuesで議論をしてから送ること。
Git flowに従って、features/issue_** (**の部分は issue番号) でブランチ名を作ること

作成するファイル

[ ] CONTRIBUTION.md
[ ] CONTRIBUTION.ja.md

ありがとうございます。では英語版を作ってマージしておきます。

repo: digital-go-jp/abr-geocoder

license: MIT


text:
description란 생성

related issue: #950

일단 코멘트란 상단에 고정적으로 나오도록 했어요

+2


text:
add exitAfterExecute to avoid process.exit() in programmatically execute API.

add exitAfterExecute arg.

I added the option in version 3.0.1 which slightly changes the specs on the main esrun function

authors: Gin-Quin xingxiuyi

repo: digital-loukoum/esrun


text:
Use child logger instead of manually prefixing logger messages

See FIXMEs for child logger use or const logPrefix = ; usage.

Addressed by #105.

source: gharchive/issue

repo: digitalbazaar/bedrock


text:
Add verify()

addresses #5

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@d702330). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master       #6   +/-   ##
=========================================
  Coverage          ?   88.23%
=========================================
  Files             ?        2
  Lines             ?       34
  Branches          ?        0
=========================================
  Hits              ?       30
  Misses            ?        4
  Partials          ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d702330...dad4dbf. Read the comment docs.

@JSAssassin this needs a changelog entry.

authors: JSAssassin codecov-io mattcollier

repo: digitalbazaar/http-digest-header


text:
Sentinel-1 for water detection

Proposed changes
Include a brief description of the changes being proposed, and why they are necessary.
Closes issues (optional)

Closes Issue #000
Closes Issue #000

Checklist (replace [ ] with [x] to check off)

[x] Remove any unused Python packages from Load packages
[x] Remove any unused/empty code cells
[x] Remove any guidance cells (e.g. General advice)
[x] Ensure that all code cells follow the PEP8 standard for code. The jupyterlab_code_formatter tool can be used to format code cells to a consistent style: select each code cell, then click Edit and then one of the Apply X Formatter options (YAPF or Black are recommended).
[x] Include relevant tags in the final notebook cell and re-use tags if possible
[x] Clear all outputs, run notebook from start to finish, and save the notebook in the state where all cells have been sequentially evaluated

Thank you for reviewing this notebook and providing feedback.

All items raised by the reviewer except the last one addressed in the notebook. The last item needs more thinking about the approach in order to better compare two indices due to the spatial resolution differences.

@fangfy and @cbur24  I edited the notebook following a recent discussion yesterday and added a few contexts to the interpretation. Please check and advise.

@neginmoghaddam @fangfy I've made a number of changes to the S1 water detection notebook.  I ported more of the plots from the earlier radar water detection notebook as I thought the workflow 'flowed' a little better using these plots.  I've also, somewhat opinionatedly, removed parts of the notebook that I thought weren't adding a lot of information but extended the length of the notebook. For example, the 'mean filter' segment, and the facet plots that weren't showing much.  The S1 values have also been converted to dB as this makes for much nice histogram plots.
Also, maybe we should make a shoreline detection notebook using S1 now that we can match tide times with the S1 images?

Thanks, @cbur24 for the review and the proposed changes to the notebook. I am happy to keep the values in dB if it is something that @fangfy approves. We had a couple of discussion on not using the dB values. No doubt that the histogram plots are pretty now. On the side of the facet, my main objective was to show the backscatter signature in each polarisation for comparison- I added a few lines of interpretation about them but I can see that they were removed.
I think S1 for shoreline detection could be a good case study if you have any suggestion on the location.

converting to dB is good for thresholding, but we should apply lee filter first. And I think the threshold should be lower than -22 in this case? I suggest explicitly define threshold_vh because the threshold is very different for vv.

Is the removed facet plot the one showing change over time? maybe that can be added towards the end as to discuss

coastal and wetlands are highly dynamic, so cloud free assessment with radar is valuable to capture change in all seasons.
this method doesn't work for water area with standing vegetation. this will result in difference when comparing to optically mapped extent

Also, should this notebook be in frequently used code instead of real world example?

@fangfy my opinion is it should stay in Real_world_Examples as its quite an extensive notebook and covers more than just one or two analysis snippets.  The markdown at the top of the notebook also highlights a use case - water classification - which fits with the Real_world_examples theme.  I might be a good idea though to have a notebook in Frequently_used_code that shows loading S1 data, applying a lee filter, and converting to dB.  Oh, and should the lee filter function be put into the deafrica-tools.spatial.py script?

@neginmoghaddam in the Products used section at the top of the notebook, there is a reference to S2 that should be removed

@fangfy and @cbur24 requested changes are now applied including:

Adjusting the coordinates to the location that better represent wetland dynamics
Adding a few sentences for better interpretation of S1 TS
Removing the reference link to S2
Adding facet plots for each polarisation bands and ratio band using S1 DN values
Restructuring the notebook to reflect lee filtering step first followed by a step to convert DN values to dB values
Change the keyword threshold to threshold_vh and choose a lower value than -22 dB.
Please let me know should I missed anything.

A couple of things I noticed:

Not sure you need to do the RGB facet plot (ln[10]), as it doesn't add much and you're calculating vh/vv but without an explanation of why that's useful (and then its not used anywhere else in the notebook).  Also, when doing RGB plots, don't be afraid to use deafrica-tools.plotting.rgb(), its has all the functionality you've replicated with the verbose plotting code currently in the notebook.
The subtitle Visualise S1 in its two polarisation bands and RGB colour composite is misleading because there isn't a RGB plot in this section.
With the histogram analysis, the markdown says you're going to show the impact of lee-filtering on the histograms, but then you only plot the vh and vv filtered histograms, not the unfiltered histograms. I recommend restoring the plots from the previous iteration of the notebook.
Might be worth explaining the conversion from DN to dB. How it works and why its useful.

Made a few updates and will push once I can save the notebook....
Mainly:

Updated text to be consistent throughout can have explanations for each step
Cleaned up import section to only include used packages
Updated rgb plotting - scaled by median and use rgb()
Removed reference to explorer because data is available everywhere for Africa.

@fangfy the notebook is now ready to be merged to the repo. Thanks, @cbur24 and @fangfy for your valuable inputs.

authors: NMoghaddam cbur24 fangfy


text:
Show a list of not updated pages

Give content creators / editors the chance to spot outdated pages. Provide a list with pages that have not been updated in a selected time frame.

Covered by translation coverage.

authors: svenseeberg ulliholtgrave

repo: digitalfabrik/integreat-cms


text:
Update packages and fix pylint issues in the ini file

Short description

This PR updates our JS and python packages to their newest version.
We should keep in mind that our setup.cfg will be deprecated by some point and we should consider switching to a pyproject.toml setup (see Issue #1362 ).

I resolved all issues and created https://github.com/digitalfabrik/integreat-cms/issues/1362 as a reminder to keep track of the current development in Setup Tools.


text:
Use d-bus instead of RPC?

Does you plan provide support from digitalocean to this package or after some times  move to dbus based? (i think you know that libvirt devs write dbus interface to libvirt and in this case this generated wire binding can be relaced with dbus based)

We don't have any plans to do that at the moment, and we haven't investigated to see what would be gained. I'd be curious to see whether this simplified the go binding generator, but there's nothing urgent pushing us in this direction right now.

thanks!

authors: trapgate vtolstov

repo: digitalocean/go-libvirt


text:
DCIM Devicetype with special characters like Ä Ö Ü ß = crash

If im using special chars like ÖÄÜß in the model field, netbox cannot be used until special chars are removed with an sql query.

Should be fixed in 0d46a65.

repo: digitalocean/netbox

license_source: bigquery


text:
please allow overrides of requests.Session

I'd like to have the ability to override the default (requests) session object created in Api.__init__(), especially since the constructor immediately starts to make use of the requests Session object. Thing is, we run a custom setup requiring some non-standard attributes to be set to requests.Session, but there is no way to do by overriding Api.http_session before it's being used the first time in the constructor itself.
Current constructor is (relevant parts only)
class Api(object):
    def __init__(
        self,
        url,
        token=None,
        private_key=None,
        private_key_file=None,
        ssl_verify=True,
        threading=False,
    ):
...
        self.http_session = requests.Session()
...
        req = Request(
            base=base_url,
            token=token,
            private_key=private_key,
            ssl_verify=ssl_verify,
            http_session=self.http_session
        )
        if self.token and self.private_key:
            self.session_key = req.get_session_key()
...

Now thing is, Session has some attribtes that can be set, in my particular case a client cert I need to pass like so
s = requests.Session()
s.cert = ("cert", "key")

You do support ssl_verify which maps to requet's verify attribute. That too could be set through a session paramater. Therefore I suggest to rewrite Api.__init__() as follows
    def __init__(
        self,
        url,
        token=None,
        private_key=None,
        private_key_file=None,
        session=requests.Session(),
        threading=False,
    ):
        self.http_session = session

Then I as a user could do:
s = requests.Session()
s.cert = ("cert", "key")
s.verify = False
# whatever else one may want to tune and customize
nb = pynetbox.api(..., session=s)

If you remove verify_ssl this obviously breaks the API, but that could be easily worked around and kept for compatibility reasons, even without breaking existing code.

Oh and by the way, #189 deals with a similar problem, though that can be worked around otherwise. My suggestion would be a drive-by fix for #189 as well.

Yeah, #209 and #219 could probably benefit from it as well. Real blocker seems to be making the call to .get_session_key() key in init. I think that may make more sense in App when it's called for the secrets endpoint. That would at least let you set api.http_session universally after instantiation, but not opposed to letting it be passed as kwarg as well.

Absolutely. Session.auth allows basic authentication settings fixing #209, and Session.headers to add custom headers for #210, though that may need more care since you also set headers on a by-request basis, and I don't think requests.Session would merge them.

Done, opted not to make it a kwarg though, but we don't make any requests in Api's init now.

authors: daemonkeeper zachmoody

repo: digitalocean/pynetbox


text:
[DIG-117] [Feature]: Change NavLink [EASY]

Is your feature request related to a problem? Please describe.
Blogs in navbar
Describe the solution you'd like
replace /about in navbar to blogs.digitomize.com (We might have multiple navbar components for dashboard/ mobile/ desktop)
Alternatives
No response
Additional Context
NOTE: We won't assign this, first good PR will be merged
Are you interested in contributing?

[ ] I'd like to work on this issue

Code of Conduct

[X] I have used the search function to check if an issue already exists
[X] I agree to follow this project's Code of Conduct

DIG-117

Hey there @pranshugupta54 Can I be assigned to this issue?

assign me this issue.

repo: digitomize/digitomize


text:
Span N+1 Insight affected endpoints

The endpoints field is already sent from the server, but was never used.
The field endpoints[].endpointInfo.entrySpanCodeObjectId is NEW and should be used for navigation.
{
    "name": "N+1",
    "type": "SpaNPlusOne",
    "category": "Performance",
    "specifity": 2,
    "importance": 2,
    "span": {
        "name": "OwnerRepository.findByLastName",
        "displayName": "OwnerRepository.findByLastName",
        "instrumentationLibrary": "io.opentelemetry.spring-data-1.8",
        "spanCodeObjectId": "span:io.opentelemetry.spring-data-1.8$_$OwnerRepository.findByLastName",
        "methodCodeObjectId": "org.springframework.samples.petclinic.owner.OwnerRepository$_$findByLastName",
        "kind": "Internal",
        "codeObjectId": "org.springframework.samples.petclinic.owner.OwnerRepository$_$findByLastName"
    },
    "occurrences": 51,
    "traceId": "FB7C38EB2B0F55634F6A04C603A1B467",
    "clientSpanName": "select distinct o1_0.id,o1_0.address,o1_0.city,o1_0.first_name,o1_0.last_name,o1_0.telephone from owners o1_0 left join pets p1_0 on o1_0.id=p1_0.owner_id where o1_0.last_name like ? escape ? offset ? rows fetch first ? rows only",
    "clientSpanCodeObjectId": "span:io.opentelemetry.jdbc$_$9FA5EE4CBE7B619CC989600EB7B783",
    "duration": {
        "value": 10.0,
        "unit": "ms",
        "raw": 9998808.0
    },
    "endpoints": [
        {
            "endpointInfo": {
                "route": "HTTP GET /owners",
                "instrumentationLibrary": "io.opentelemetry.jdbc",
                "spanCodeObjectId": "span:io.opentelemetry.jdbc$_$9FA5EE4CBE7B619CC989600EB7B783",
                "entrySpanCodeObjectId": "span:io.opentelemetry.tomcat-10.0$_$HTTP GET /owners",
                "serviceName": "spring-petclinic"
            },
            "occurrences": 51
        }
    ],
    "scope": "Span",
    "spanInfo": {
        "name": "OwnerRepository.findByLastName",
        "displayName": "OwnerRepository.findByLastName",
        "instrumentationLibrary": "io.opentelemetry.spring-data-1.8",
        "spanCodeObjectId": "span:io.opentelemetry.spring-data-1.8$_$OwnerRepository.findByLastName",
        "methodCodeObjectId": "org.springframework.samples.petclinic.owner.OwnerRepository$_$findByLastName",
        "kind": "Internal",
        "codeObjectId": "org.springframework.samples.petclinic.owner.OwnerRepository$_$findByLastName"
    },
    "shortDisplayInfo": {
        "title": "",
        "targetDisplayName": "",
        "subtitle": "",
        "description": ""
    },
    "codeObjectId": "org.springframework.samples.petclinic.owner.OwnerRepository$_$findByLastName",
    "decorators": [
        {
            "title": "N+1",
            "description": "Supected NPlus One"
        }
    ],
    "environment": "ASAFC-LAPTOP[LOCAL]",
    "severity": 0.0,
    "isRecalculateEnabled": false,
    "prefixedCodeObjectId": "method:org.springframework.samples.petclinic.owner.OwnerRepository$_$findByLastName",
    "customStartTime": null,
    "actualStartTime": "2023-06-07T11:17:46.955556Z"
},

Related to https://github.com/digma-ai/digma-intellij-plugin/pull/831

repo: digma-ai/digma-ui


text:
Update langchain + langchain-openai packages

For #362. For anthropic tool support, I think we'll have to install langchain-anthropic

Have you been able to test this at all>

Have you been able to test this at all>

Yeah I tested it again. FYI: I see two of the anthorpic models doesn't work (on prod also). We can maybe tackle that with the anthropic tool change

authors: SmittieC snopoke

repo: dimagi/open-chat-studio


text:
Replace type alias with a 'use'.

Fixes #222

We should consider doing that for the other aliases, e.g. use na::Vector3 as Vector instead of type Vector<N> = Vector3<N> (can be in a separate PR).

See https://github.com/dimforge/parry/pull/232

the difference is more subtle and I'm not sure it's more consistent, it still redirects to an alias (which is alright, but has this big dislaimer "Because this is an alias, not all its methods are listed here. See the Point type too." ; but yes let's discuss it in the other PR.

This adds a warning when running cargo +nightly clippy:
warning: casting to the same type is unnecessary (`f32` -> `f32`)
    --> crates\parry3d\../../src\shape\shape.rs:1466:9
     |
1466 |         f32::MAX as Real
     |         ^^^^^^^^^^^^^^^^ help: try: `f32::MAX`
     |
     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_cast

Interestingly, this first one seems correct ^ ; but the following is more difficult to fix:
warning: casting float literal to `f32` is unnecessary
  --> crates\parry3d\../../src\query\sat\sat_cuboid_cuboid.rs:14:18
   |
14 |     let signum = (1.0 as Real).copysign(pos12.translation.vector.dot(axi...
   |                  ^^^^^^^^^^^^^ help: try: `1.0_f32`
   |
   = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_cast

Real::MAX was what I intended to do when I got to these.
For the other, you could try Real::from(1.0) and see if that warns ... or a quick trait like the One trait from num.

authors: Vrixyz waywardmonkeys

repo: dimforge/parry


text:
add support for opentofu for binary plans

At the moment trying to use tf-summarize with a binary plan that was generated by opentofu does not work.
It will raise the error (version refs will depend on your terraform version, latest opentofu version is 1.6.1)
bsh ❯ tf-summarize .tfplan
error when running 'terraform show -json .tfplan':
╷
│ Error: Couldn't show local plan
│
│ Plan read error: plan file was created by Terraform 1.6.1, but this is
│ 1.6.6; plan files cannot be transferred between different Terraform
│ versions
╵

Make sure you are running in terraform directory and terraform init is done

Workaround
Under the covers, tf-summarize just calls terraform show -json when presented with a binary plan; so just turn it into json via tofu.
bsh ❯ tofu plan -out .tfplan
bsh ❯ tofu show -json .tfplan >.tfplan.json
bsh ❯ tf-summarize .tfplan.json
+--------+----------+
| CHANGE | RESOURCE |
+--------+----------+
+--------+----------+

At this moment if you need to have a support of Terraform as well as OpenTofu (and Terragrunt :) ) in one tool you can use https://github.com/tofuutils/tenv which my team wrote some months ago. A lot of users switched to that tool to unify version management in the world of Terraform.
You're welcome to open any issues or contribute to tenv.

authors: kvendingoldo quotidian-ennui

repo: dineshba/tf-summarize


text:
Add copy_to_enumerator

See https://github.com/diogob/postgres-copy/issues/42

Thanks for the PR 👍
Could you add a spec for for this code?

Added spec and also added buffering support (since the primary use-case is web streaming, where it can easily offer a 100x speedup).

authors: DanielHeath diogob

repo: diogob/postgres-copy

license: mit


text:
Sync settings when users are fetched

Description of the PR
When users or partial users should be fetched, their settings should also. A similar pr should be made to the member gatway plugin.
Changes Proposed in this Pull Request (List new items in CHANGELOG.MD)

Semver Classification

[ ] This PR only includes documentation or non-code changes.
[ ] This PR fixes a bug and does not change the (intended) framework interface.
[x] This PR adds methods or properties to the framework interface.
[ ] This PR removes or renames methods or properties in the framework interface.

This is to Settings branch, and can be merged when @kyranet feels it's ready

repo: dirigeants/klasa


text:
Added Dassault Systèmes SE (3DS)

Summary
< A quick summary of the associated pull request >
Quality Assurance Checklist
< Confirmation of this pull request meeting the following requirements, prior to merge >

Review Items
Y/N

Site has a publicly known bug bounty or vulnerability disclosure program
y

Disclosure policy is publicly available
y

Public URL
y

Submission follows the Diodb schema
y

Your Twitter handle (Optional):
@sickcodes

Thanks @sickcodes!

repo: disclose/diodb


text:
Update APPLICATION_CURRENT endpoint

Update the url of APPLICATION_CURRENT endpoint to get new data not available in the old one.
New data:

role_connections_verification_url
approximate_guild_count
(And more, but undocumented at the moment)

See:

https://github.com/discord/discord-api-docs/pull/5971

Apparently both endpoint returns different values as per API spec:
https://www.postman.com/discord-api/workspace/discord/example/23484324-6a5eff22-2298-44f4-8814-0193a2bbf37e
https://www.postman.com/discord-api/workspace/discord/example/23484324-4bec98b0-4e32-4e15-a307-969329ae7bb3
So we will revert this change and add them as new endpoint instead

authors: SQKo jocafamaka

repo: discord-php/DiscordPHP-Http


text:
enable admins to configure the slack help text

The help text displayed when a user enters /discourse into slack is hardcoded as:
`/discourse [watch|follow|mute|help|status] [category|all]`
*watch* – notify this channel for new topics and new replies
*follow* – notify this channel for new topics
*mute* – stop notifying this channel
*status* – show current notification state and categories

in plugin.rb self.help
Discourse admins should be able to modify this default help text.
We don't refer to our discourse instance as "discourse" and so have renamed the slash command in slack to the name more familiar with our users. It'd be nice to reflect this in the help text (which has hardcoded the command to /discourse).

This request does not make sense; the tool is a Slack to Discourse
connector. The instance name does not change the fact that it is Discourse
underneath..
On Wed, Mar 15, 2017 at 4:13 AM stu-b-doo notifications@github.com wrote:

The help text displayed when a user enters /discourse into slack is
hardcoded as:
/discourse [watch|follow|mute|help|status] [category|all]
watch – notify this channel for new topics and new replies
follow – notify this channel for new topics
mute – stop notifying this channel
status – show current notification state and categories
in plugin.rb self.help
https://github.com/discourse/discourse-slack-official/blob/0c4e8611c19edc3e3247dfa31a927635fa775ecc/plugin.rb#L251
Discourse admins should be able to modify this default help text.
We don't refer to our discourse instance as "discourse" and so have
renamed the slash command in slack to the name more familiar with our
users. It'd be nice to reflect this in the help text (which has hardcoded
the command to /discourse).
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/discourse/discourse-slack-official/issues/21, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABc7Vbj4YdwGz5YyXY-j7nAdM98lwaUyks5rl8fpgaJpZM4MdzVJ
.

I would welcome this change.  On the slack side, when you set up the integration you can choose what the keyword is for the Slack command, and we have also chosen to match our site name, rather than discourse.
So to actually follow a category, you type something like the following:
/mycoolforum watch foo-category

and if you want help, you type:
/mycoolforum help

but then it responds with help saying:
/discourse [watch|follow|mute|help|status] [category|all]

which, if you type it, will do nothing....
So yeah, being able to make the help text match how you configure it in Slack makes total sense to me.

I suppose it is more defensible if there are multiple sites, you would need
to disambiguate among them. That makes sense, so I withdraw my objection.
On Wed, Mar 15, 2017 at 10:12 PM David McClure notifications@github.com
wrote:

I would welcome this change. On the slack side, when you set up the
integration you can choose what the keyword is for the Slack command, and
we have also chosen to match our site name, rather than discourse.
So to actually follow a category, you type something like the following:
/mycoolforum watch foo-category
and if you want help, you type:
/mycoolforum help
but then it responds with help saying:
/discourse [watch|follow|mute|help|status] [category|all]
which, if you type it, will do nothing....
So yeah, being able to make the help text match how you configure it in
Slack makes total sense to me.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/discourse/discourse-slack-official/issues/21#issuecomment-286959826,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABc7VW1RM8aETgQiFIbVJhlzV8h2R0W-ks5rmMSmgaJpZM4MdzVJ
.

This is already available in my PR which will be merged soon.

@coding-horror even if you only have one discourse site, as @mcwumbly  mentioned, our users are used to referring to discourse as mycoolforum . So our slack command reflects that: /mycoolforum.
But in the command's help it's assumed the slack command is called /discourse, adding minor confusion.
To be honest, the most of the users who know about the slack command are probably well aware theat mycoolforum is an instance of this other thing called discourse, so it's not that big a deal. More for completeness really 😃

@vinothkannans in your PR, the message still looks hardcoded in the yml. Is this file going to be modifyable by admins or would it need to be changed at source before the app is installed?

@stu-b-doo you can modify all the strings which are inside .yml file from the admin panel of your Discourse installation in the page /admin/customize/site_texts

so config/locales/server.en.yml would become available as a site text?  Sounds good!

Not just server.en.yml.
All .yml files inside /config/locales/ folder.

authors: coding-horror mcwumbly stu-b-doo vinothkannans

repo: discourse/discourse-slack-official


text:
[Feature Request] - Rpush instrumentation

My current company is using RPush and RPush has a plugin system. I found the following: https://github.com/rpush/rpush-plugin-statsd/blob/master/lib/rpush/plugin/statsd.rb
Any suggestions for how I can achieve something similar using prometheus_exporter?

Here you go https://github.com/equinux/rpush-plugin-prometheus-exporter

Well, that shows how to do it :)

authors: maxsz mhenrixon


text:
FIX for ruby 3 incompatibility in sidekiq middleware, fixes #229

See #229 for more details.
The test case might seem a bit weird, but I didn't find a more elegant way to reproduce Sidekiq's middleware initialization. So definitely open for suggestions.

Sidekiq middleware chain for reference: https://github.com/mperham/sidekiq/blob/main/lib/sidekiq/middleware/chain.rb

@mperham what do you think of the workaround? I guess param handling changes between 2.7 / 2.8 / 3.0 are a bit of a sore point?

Lgtm!

thanks @benzimmer  !

authors: SamSaffron benzimmer mperham


text:
Can not read filesystem in ISO9660 image with creation timestamp

When I create ISO-image with UltraISO program with any data and any image size, and set image creation (or/and modification) timestamp, this image can not be opened with this API. This image is well opened in Windows explorer, but cannot be read by API. Call to GetFilesystem(0) returns error "Unknown filesystem on partition 0". Creation timestamp can be set in UltraISO image properties in tab date/time. If timestamp is not set, it's all working.
ISO-image is opens with subsequent calls:
	if disk, err = diskfs.OpenWithMode(fpath, diskfs.ReadOnly); err != nil {
		return
	}
	if fs, err = disk.GetFilesystem(0); err != nil { // assuming it is the whole disk, so partition = 0
		return
	}

If you upload an image that fails to be read to somewhere it can be downloaded, we can look at it. I think there are size limits on GitHub issue attachments, but not sure.

example.zip
I put example ISO-image into zip-archive. There is 3 short text files nested into image for example. When I create this image, I set only creation filestamp and nothing any more.

The creation timestamp on this image appears to be corrupt. The ISO primary volume descriptor date is supposed to be in byte positions 813-829, as follows:

year: bytes 0-3
month: bytes 4-5
date: bytes 6-7
hour: bytes 8-9
minute: bytes 10-11
second: bytes 12-13
centisecond: bytes 14-15
offset from UTC: byte 16 (in quarter hours)

All of the above look fine, except for centisecond.
2021 04 20 15 47 16 \x000

They all are ASCII codes for characters, except for the bytes at 14-15, which are: [0,48]. 0 is a null character, nit a valid number. The ASCII code for the character 0 is 48.
It is possible I misunderstood the spec on that specific part - centiseconds - so checking it again.

Nope, it is pretty clear that the hundredths of a second are in the format strD here, or if you prefer, check the official ECMA spec, it is on page 28.

I looked in the hex editor image with all timestamp fields set, in all timestamps hundredths of a second is not set, and it's looks like zero-terminated string. Standard does not specify behavior if hundredths of a second is not set (or case when date is set, but time is not set). What is optimal resolution can be for this case I do not know, but I think that broken timestamp can not be a valid reason for image open failure. Those images are produced by popular UltraISO prorgam, so such cases can occurs very often.

I am pretty confident that UltraISO is incorrect. If this is 0, it should be 0x48 0x48, and not 0x00 0x00. Either way, it definitely is not a null-terminated string.
That having been said, there is no reason why this library couldn't handle those deviant cases, by checking if the hundredths of second is 0x00 and treating either of the fields as 0x48.
Go ahead and open a Pull Request for it.

authors: deitch schwarzlichtbezirk

repo: diskfs/go-diskfs


text:
[DISQ-33] Duplicate reads bug

This reproduces the problem (#33) and provides a fix. It also adds log debugging to provide diagnostic information about splits and BGZF chunk boundaries to help with debugging issues like this in the future.
The bug occurs when a BGZF block boundary coincides with a split boundary. Here's a log file for a run using a file that produced an errant count:
10:49:42.990 DEBUG BamSource - PathChunk for partition 294: PathChunk{path=file:/home/tom/bamaftersparksortsize.bam, span=9865016001:489-9898557440:65535}
10:49:43.239 DEBUG BamSource - PathChunk for partition 295: PathChunk{path=file:/home/tom/bamaftersparksortsize.bam, span=9898557440:190-9932111872:65535}

From the log we see that the spans for the two paths overlap, since the first ends at 9898557440:65535 which is after 9898557440:190, where the second starts. This results in duplicate reads in the two partitions. The first span should end at the beginning of the BGZF block where the next span starts, i.e. 9898557440:0.
The problem was observed since 9898557440 is 9440MB, which is a multiple of the default block size, 32MB for local files.  (It is not a multiple of the default block size for HDFS, which is 128MB, so the problem was not observed on HDFS unless the split size was set explicitly to 32MB.)

@jamesemery thanks for the review! I've addressed your comments and made a minor fix to the test that you spotted.

Thanks @jamesemery

repo: disq-bio/disq


text:
fix for https://github.com/nim-lang/Nim/pull/18818

fix to unblock https://github.com/nim-lang/Nim/pull/18818
this fixes several package failures and is a harmless change

Why would I want this?

authors: disruptek timotheecour

repo: disruptek/balls


text:
Pushing large layer to S3 backed registry results in HTTP 500

Sorry for opening another issue around this topic... I tried in Docker forum but I did not get any response there.
I am hosting a private docker repo on AWS EC2 and want to push a 100GB layer to it. The upload itself seems to work well, meaning the upload progress bar on the client goes up to 100% but then there are multiple retries and eventually the client exits with received unexpected HTTP status: 500 Internal Server Error.
There are other smaller layers within the same image that push without problems so this issue only applies to the single large layer. Thus, I suspect that it is not a configuration problem on the registry side.
Registry version: 2.6.2
docker client version: 17.12.1-ce
I am running the registry image as a swarm service with 1 replica. docker version on the swarm node:
Client:
 Version:       18.03.0-ce
 API version:   1.37
 Go version:    go1.9.4
 Git commit:    0520e24
 Built: Wed Mar 21 23:10:01 2018
 OS/Arch:       linux/amd64
 Experimental:  false
 Orchestrator:  swarm

Server:
 Engine:
  Version:      18.03.0-ce
  API version:  1.37 (minimum version 1.12)
  Go version:   go1.9.4
  Git commit:   0520e24
  Built:        Wed Mar 21 23:08:31 2018
  OS/Arch:      linux/amd64
  Experimental: false

The infrastructure between the client and the registry looks like this:
+--------+       +-------+     +-------+     +-------+      +----------+
| Client |  -->  | ELB 1 | --> | Nginx | --> | ELB 2 | -->  | Registry |
+--------+       +-------+     +-------+     +-------+      +----------+

Inspired by some other problem reports around similar issues, the idle timeout for both ELBs is set to 3600 seconds.
The nginx configuration follows the official recipe.
I do not see any relevant error messages in the registry logs.
The nginx logs show 5 pairs of the following lines:
2018/06/12 18:02:29 [warn] 8#8: *108325 a client request body is buffered to a temporary file /var/cache/nginx/client_temp/0000000056, client: 185.242.113.117, server: registry.*, request: "PATCH /v2/nominatim/blobs/uploads/3efb293b-0f9e-4cf2-9124-6b6ea878559b?_state=LNzokuLcihlUHRyt3qILWr4RPhSZQEcigU-63oF7E2J7Ik5hbWUiOiJub21pbmF0aW0iLCJVVUlEIjoiM2VmYjI5M2ItMGY5ZS00Y2YyLTkxMjQtNmI2ZWE4Nzg1NTliIiwiT2Zmc2V0IjowLCJTdGFydGVkQXQiOiIyMDE4LTA2LTEyVDE4OjAyOjI5LjAxNTg3OTA2NFoifQ%3D%3D HTTP/1.1", host: "registry.*"
2018/06/12 19:09:46 [crit] 8#8: *108325 pwrite() "/var/cache/nginx/client_temp/0000000056" failed (28: No space left on device), client: 185.242.113.117, server: registry.*, request: "PATCH /v2/nominatim/blobs/uploads/3efb293b-0f9e-4cf2-9124-6b6ea878559b?_state=LNzokuLcihlUHRyt3qILWr4RPhSZQEcigU-63oF7E2J7Ik5hbWUiOiJub21pbmF0aW0iLCJVVUlEIjoiM2VmYjI5M2ItMGY5ZS00Y2YyLTkxMjQtNmI2ZWE4Nzg1NTliIiwiT2Zmc2V0IjowLCJTdGFydGVkQXQiOiIyMDE4LTA2LTEyVDE4OjAyOjI5LjAxNTg3OTA2NFoifQ%3D%3D HTTP/1.1", host: "registry.*"

I suspect that these pairs correspond to the client's retry attempts. As you can see, there is approximately 1 hour between the warning and the error which corresponds to the idle timeout of 3600 seconds that is configured on my load balancers. But the error message indicates that it is not a timeout issue but a disk space issue. I checked the relevant nodes running nginx after the push failed - the free disk space was 17G.
So I am not getting smart out of this and would appreciate some help.

Btw I am using nginx 1.13.6.
According to http://nginx.org/en/docs/http/ngx_http_proxy_module.html proxy_request_buffering is enabled by default and When buffering is enabled, the entire request body is read from the client before sending the request to a proxied server. This would explain a lot.
Moreover it says When HTTP/1.1 chunked transfer encoding is used to send the original request body, the request body will be buffered regardless of the directive value unless HTTP/1.1 is enabled for proxying.
So I am now trying again with this configuration:
proxy_http_version 1.1;
proxy_request_buffering off;
chunked_transfer_encoding on;

With the above configuration changes nginx does not log any client request buffering error any more.
But now there is an error on the registry side that says upload resumed at wrong offest: 20971520000 != 29813393708.
registry_registry.1.ty5vu0u6ibnt@ip-172-31-17-47.eu-central-1.compute.internal    | 10.255.0.7 - - [29/Jun/2018:19:01:20 +0000] "PATCH /v2/nominatim/blobs/uploads/e6cd1694-9daa-4abc-8c97-e20d56fdc8d2?_state=-zn3L39pkNc41oYfFWYU6MNa1An3SvsDYf_peM-V7rl7Ik5hbWUiOiJub21pbmF0aW0iLCJVVUlEIjoiZTZjZDE2OTQtOWRhYS00YWJjLThjOTctZTIwZDU2ZmRjOGQyIiwiT2Zmc2V0IjowLCJTdGFydGVkQXQiOiIyMDE4LTA2LTI5VDE5OjAxOjE5LjU1Mzk5Nzg1MloifQ%3D%3D HTTP/1.1" 202 0 "" "docker/18.03.0-ce go/go1.9.4 git-commit/0520e24 kernel/4.4.0-31-generic os/linux arch/amd64 UpstreamClient(Docker-Client/18.03.1-ce \\(linux\\))"
registry_registry.1.ty5vu0u6ibnt@ip-172-31-17-47.eu-central-1.compute.internal    | time="2018-06-29T21:36:26Z" level=error msg="upload resumed at wrong offest: 20971520000 != 29813393708" go.version=go1.7.6 http.request.host=registry.ordami.com http.request.id=0ac2fb9e-8a99-4191-8cfb-2079689fa5ec http.request.method=PUT http.request.remoteaddr=185.242.113.117 http.request.uri="/v2/nominatim/blobs/uploads/e6cd1694-9daa-4abc-8c97-e20d56fdc8d2?_state=CPkczBflJxRaQdc2FoZFhu8yVl4Z6hpMUO01YHYTzL97Ik5hbWUiOiJub21pbmF0aW0iLCJVVUlEIjoiZTZjZDE2OTQtOWRhYS00YWJjLThjOTctZTIwZDU2ZmRjOGQyIiwiT2Zmc2V0IjoyOTgxMzM5MzcwOCwiU3RhcnRlZEF0IjoiMjAxOC0wNi0yOVQxOTowMToxOVoifQ%3D%3D&digest=sha256%3A5d1cc2855dffb1d4c3f775ee8f41a83f911410f54f10678238b8b63b02aad064" http.request.useragent="docker/18.03.0-ce go/go1.9.4 git-commit/0520e24 kernel/4.4.0-31-generic os/linux arch/amd64 UpstreamClient(Docker-Client/18.03.1-ce \\(linux\\))" instance.id=06d43f37-296c-4bad-9097-87a39ca0a5a2 vars.name=nominatim vars.uuid=e6cd1694-9daa-4abc-8c97-e20d56fdc8d2 version=v2.6.2
registry_registry.1.ty5vu0u6ibnt@ip-172-31-17-47.eu-central-1.compute.internal    | time="2018-06-29T21:36:26Z" level=error msg="response completed with error" err.code="blob upload invalid" err.message="blob upload invalid" go.version=go1.7.6 http.request.host=registry.ordami.com http.request.id=0ac2fb9e-8a99-4191-8cfb-2079689fa5ec http.request.method=PUT http.request.remoteaddr=185.242.113.117 http.request.uri="/v2/nominatim/blobs/uploads/e6cd1694-9daa-4abc-8c97-e20d56fdc8d2?_state=CPkczBflJxRaQdc2FoZFhu8yVl4Z6hpMUO01YHYTzL97Ik5hbWUiOiJub21pbmF0aW0iLCJVVUlEIjoiZTZjZDE2OTQtOWRhYS00YWJjLThjOTctZTIwZDU2ZmRjOGQyIiwiT2Zmc2V0IjoyOTgxMzM5MzcwOCwiU3RhcnRlZEF0IjoiMjAxOC0wNi0yOVQxOTowMToxOVoifQ%3D%3D&digest=sha256%3A5d1cc2855dffb1d4c3f775ee8f41a83f911410f54f10678238b8b63b02aad064" http.request.useragent="docker/18.03.0-ce go/go1.9.4 git-commit/0520e24 kernel/4.4.0-31-generic os/linux arch/amd64 UpstreamClient(Docker-Client/18.03.1-ce \\(linux\\))" http.response.contenttype="application/json; charset=utf-8" http.response.duration=112.456807ms http.response.status=404 http.response.written=76 instance.id=06d43f37-296c-4bad-9097-87a39ca0a5a2 vars.name=nominatim vars.uuid=e6cd1694-9daa-4abc-8c97-e20d56fdc8d2 version=v2.6.2

Thanks for the report @Mobe91
100GB for a single layer is... huge :-)

Does the docker client really push a single layer in one huge request?

Docker supports resumable push (using byte ranges).

proxy_request_buffering

Absolutely, this must be disabled
Here are my suggestions moving forward:

try to push directly to the registry (bypass your nginx proxy) and validate this works
honestly, we never tested layers that big, so, it's uncharted territory at this point :-)

Closing as stale. Feel free to reopen.

repo: distribution/distribution


text:
Inconsistent PIs for linktext generated from navtitle

Converted from SourceForge issue 2903326, submitted by stokecn
DITA-OT1.5 m23
I am using <topicgroup toc="no"> and child topicrefs to simulate keydefs.
In my keydef (topicrefs) file, I have:
<topicgroup toc="no">
<topicref keys="WWW_SF"  navtitle="navtitle text"  href="http://sourceforge.net" scope="external"/>
</topicgroup>

The keydefs (topicrefs) are imported into the project ditamap via mapref.  In my project ditamap I have this:
<reltable>
<relrow>
<relcell>
<topicref id="mapSFLink" keyref="WWW_SF"/>
</relcell>
<relcell>
<topicref href="testTopic.dita"/>
</relcell>
</relrow>
</reltable>

And in testTopic.dita, to test xref output, I have:
<xref id="aickeyWWWTrademark" keyref="WWW_SF"/>

When I build this project, the navtitle text is used as the link text for both xref and topic/link output. However, the PI assigned to the text derived from the navtitle is different for each type of link. The following is taken from stage1.xml using PDF2, but it's also true forthe  htmlhelp temp files.
When I build, I get the following for the xref:
<xref class="- topic/xref " href="http://sourceforge.net" id="testTopicWWWSFID" keyref="WWW_SF" scope="external" xtrc="xref:1"  xtrf="C:\DITA-OT1.5m23-112209\assembledContent\20091124134154622\miniTestAliases.dita">
Note the PI--><?ditaot usertext?>navtitle text</xref>

And I get the following for the reltable link:
<link class="- topic/link " href="http://sourceforge.net" mapclass="- map/topicref " role="friend" scope="external" xtrc="topicref:6" xtrf="C:\DITA-OT1.5m23-112209\assembledContent\20091124134154622\miniTestBookmap.ditamap">
<?ditaot usertext?>
<linktext class="- topic/linktext ">
Note the PI--><?ditaot gentext?>navtitle text</linktext>
</link>

Marking as obsolete. After 6 years of seeing this bumped from release to release, I'm not anticipating activity on it and not seeing a straightforward way to change the behavior.

authors: jelovirt robander

repo: dita-ot/dita-ot


text:
🛑 Cobi-MQTT - MQTT Port is down

In 200c3eb, Cobi-MQTT - MQTT Port ($COBI_MQTT) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Cobi-MQTT - MQTT Port is back up in 0dbdaac.

repo: diveliastudio/upptime


text:
analysis step awarness

f4d4887
Proposed feature
mothulity know each step of analysis from each other. This might be achieved with few steps:

Enclosing particular mothur commands, instead of keeping them in a plaing text in the template. The question is: to what extent the logic from templates should be removed?
Once the command is passed to mothulity, it should know how the files would be named after each step. If it's done - resolving #9 is straightforward.
mothulity already creates the hidden logfile. It can be used as the cache.

The simplest way of re-running the analysis could be done in a similar way to calling mothulity for running the analysis phase.

repo: dizak/mothulity


text:
Problem rendering date field

Hello,
I installed django BMF latest version 0.3.0.a.2 in a VM.
There is a problem in the date field of all forms that use it. For example in: Sales -> Open Quotations -> Add.
All the other fields render correctly but the date field doesn't process the content below so it shows in the form just like this:
Date
The snapshot below shows it better.
Maybe i am missing a python package in my virtual envirnment or my settings is also missing something. I have no idea.

Thanks anyway.
Helio

I got it. The version I was using was from pypi.
The file djangobmf/templatetags/djangobmf_forms.py shows on line 157:
return data
And it should be return format_html(data), as is in github.
Maybe someone should update pypi to the last version available.
Thanks
Helio

repo: django-bmf/django-bmf


text:
"Page not found" while opening an attachments in the ticket

Hi,
I am getting an "Page not found" error when I am trying to open an attachment. Do I have forget something, because I cant see the path in the URLS file

Thanks in advance!

Have you set your media_url and media_root in  your settings.py file?
https://docs.djangoproject.com/en/2.0/howto/static-files/

I just checked. No, I forget that. Does it have to be a special value or should I use the standard path (MEDIA_ROOT = '/media/')?

repo: django-helpdesk/django-helpdesk


text:
Switch to Django-defined colors for dark mode support

See #57 – in particular the dark mode support problem:

This PR addresses this by switching to Django’s custom properties, which will automagically switch to variants that are compatible with dark mode. I’ve kept the previous values as fallbacks, for backwards compatibility with older versions of Django. Note this syntax doesn’t work in IE11 – if you think that’s an issue, we can also add a fallback for that.
Here is the result:

I believe the first 3 colors are the same as before. The last one is a bit of a hack, as I don’t think Django has a "highlight" color, but their breadcrumb color is intended to work well with both light and dark text.

Metric name
Screenshot

Metric name
Screenshot

00-index.png

01-available-extensions.png

02-cache-hits.png

03-detailed-index-usage.png

04-index-size.png

05-index-usage.png

06-sequence-usage.png

07-table-size.png

authors: MarkusH thibaudcolas


text:
add none() method to SalesforceManager

Prior to this patch, if you ran the none method on a salesforce model, it returned a salesforce error.
i.e. Lead.objects.none() returned a salesforce error.
The salesforce error returned is:
salesforce.backend.base.SalesforceError
salesforce.backend.base.SalesforceError: SOQL statements cannot be empty or null
This permits you to use the none() method on salesforce models.
This is my first pull request to a public repo ever, so apologies in advance.

This looks good to me, should be able to merge soon.

It must work also inside the queryset. The module salesforce.backend.manager is not the right place where to fix this bug. If you try e.g.
Lead.objects.all().none()

you get the same error as before the fix.
What is your Django version and django-salesforce? I can not reproduce your error on the current public master branch. Django 1.4 and 1.5 works correctly. Django 1.6 - 1.8 raises other error EmptyResultSet

I'm running Django 1.7.8, python 3.4, and django-salesforce 0.6.
You are indeed correct that
Lead.objects.all().none()

returns the same error.  I will investigate further.

Do you get SalesforceError? This is the same EmptyResultSet on my computer. Can you reproduce it on the test project? Unpack the zip from pypi. Copy your database settings to salesforce/testrunner/example/local_settings.py  and run python manage.py shell the the base directory of the package.
what is the result of
Lead.objects.none().query.get_compiler('salesforce').as_sql()

I'm going to close the pull request and open an issue.  If we figure out that a patch is indeed needed, I can open a new pull request.

authors: chromakey hynekcer philchristensen

repo: django-salesforce/django-salesforce


text:
Refs #34233 -- Bumped minimum supported versions of Selenium to 4.8.0.

This bumps minimum supported versions of selenium to the first release to support Python 3.10.
ticket-34233
refs PR #17001

Thanks both for checking :+1:

repo: django/django


text:
Fixed #35727 -- Added HttpResponse.text property.

Trac ticket number

ticket-35727
Branch description
Added a text property to the HttpResponse class to simplify access to the response content as a decoded string.
Checklist

[x] This PR targets the main branch.
[x] The commit message is written in past tense, mentions the ticket number, and ends with a period.
[x] I have checked the "Has patch" ticket flag in the Trac system.
[x] I have added or updated relevant tests.
[x] I have added or updated relevant docs, including release notes if applicable.
[ ] I have attached screenshots in both light and dark modes for any UI changes.

Small bookkeeping note: you can update the PR title / commit message to "Fixed #..." because we'll be closing the ticket.
Also, that reminded me that as a new feature this needs a release note in 5.2.txt. Remember to uncheck "Needs documentation" on the ticket when that's done.

@saJaeHyukc have you had a look into using the property to simplify assertContains as suggested in the ticket? I guess we need to decide if a StreamingHttpResponse should have a text property (could use getvalue())

@sarahboyce As far as I know, using getvalue() could work, but in doing so, the memory efficiency, which is a key advantage of streaming, might be compromised. For large content, this could lead to performance issues, so I'm not sure if adding a text cached property is the best solution. I’d appreciate your feedback, especially if there are any mistakes or if you have additional thoughts on this.
Also, if there’s a positive consensus on this approach, should we submit it as a separate ticket or PR?

@sarahboyce As far as I know, using getvalue() could work, but in doing so, the memory efficiency, which is a key advantage of streaming, might be compromised. For large content, this could lead to performance issues, so I'm not sure if adding a text cached property is the best solution. I’d appreciate your feedback, especially if there are any mistakes or if you have additional thoughts on this.

We should probably document that it doesn't exist. StreamingHttpResponse specifically mentions that it doesn't have a content attribute:
* It has no ``content`` attribute. Instead, it has a
  :attr:`~StreamingHttpResponse.streaming_content` attribute. This can be used
  in middleware to wrap the response iterable, but should not be consumed.

So maybe we add a new bullet point or tweak this bullet to mention that it also doesn't have a text attribute
We should decide if we want a similar attribute error for text as for content:
    @property
    def content(self):
        raise AttributeError(
            "This %s instance has no `content` attribute. Use "
            "`streaming_content` instead." % self.__class__.__name__
        )

We should probably document that it doesn't exist. StreamingHttpResponse specifically mentions that it doesn't have a content attribute:
* It has no ``content`` attribute. Instead, it has a
  :attr:`~StreamingHttpResponse.streaming_content` attribute. This can be used
  in middleware to wrap the response iterable, but should not be consumed.

So maybe we add a new bullet point or tweak this bullet to mention that it also doesn't have a text attribute
We should decide if we want a similar attribute error for text as for content:
    @property
    def content(self):
        raise AttributeError(
            "This %s instance has no `content` attribute. Use "
            "`streaming_content` instead." % self.__class__.__name__
        )

In short, I think we need to either explicitly not add it or add it 👍

Thank you very much for the detailed explanation. I will refer to this and proceed with updating the documentation and codebase to apply the changes to the relevant PR. 😃

Thank you for the updates! Looks great 👍

have you had a look into using the property to simplify assertContains as suggested in the ticket?

Did you look into this?

Thank you for the updates! Looks great 👍

have you had a look into using the property to simplify assertContains as suggested in the ticket?

Did you look into this?

Currently reviewing that part, and I'll update it shortly after a quick check. Thank you for the reminder!

@sarahboyce, I noticed that content.decode() is used quite a bit in the existing test code. Perhaps it would be good to track and update these instances as well? What do you think?

@sarahboyce, I noticed that content.decode() is used quite a bit in the existing test code. Perhaps it would be good to track and update these instances as well? What do you think?
example
https://github.com/django/django/blob/5ed72087c450af1a5da138bdfa674a069cf3f09c/tests/sitemaps_tests/test_http.py#L32

Yes, I think you can a commit here updating these :+1:

Happy with the first and last commit so pulled them out to #18681 and will merged shortly
This can then be reduced to only the 2nd commit :+1:

authors: jacobtylerwalls saJaeHyukc sarahboyce


text:
caching?

Any plans for caching mechanisms?

I've actually been thinking about caching, but I haven't really planned to submit a PR to implement into the library.
One of the things I like about virgen-acl is that it's dirt simple and can be conformed to almost any use case (that I have anyway). If virgen-acl implements caching then it will

Be one more configuration step
Be an opinionated configuration step

If I write caching into virgen-acl for example I'm going to write a redis cache layer, which will probably suit most people, but it seems like A Bad Thing to turn away memcached die hards, you know? Or there are people who would be perfectly suited using an in-memory cache that I'm not particularly interested in writing because I wouldn't use it in my projects (I want TTL and I don't want to write TTL).
My plan was to implement my own caching in my .allow() callbacks, but if you don't need .allow() callbacks then you won't be able to cache (and it won't cache the role -> resource -> permission lookup).
That being said, though, now that you've got me thinking about it I do like the idea of caching the permission lookup and not even executing the .allow() callback if there's a cached value. My own use case allows me to implement this cache before I call query, but maybe it's worthing talking about building into the lib?
If you have suggestions, though, this is likely something I'm going to be exploring in the near future and if @djvirgen is on board it might be something worth implementing directly into the library. In the mean time the simplest way to implement caching would be something like
var role = 'guest', resource = 'blog', action='edit';
var result = aclCacheQuery(role, resource, action);

if (typeof cacheResult === 'undefined') {
  result = acl.query(role, resource, action);
}

It's some extra boilerplate to your .query() calls, but it'll work

I like the idea of caching support, but I agree with @spruce-bruce that an opinionated solution might make this code less portable. One thing we can do to get around that is to provide a cache interface that you could use to wrap your own custom caching solution, for example (in pseudocode):
interface AclCache {
  @return this - fluent inteface
  public function set(key, value, ttl);

  @return boolean|null
  public function get(key);
}

The idea being you could implement this interface and provide the instance to VirgenACL:
class MyCache {
  constructor(backend) {
    this.backend = backend;
  }

  set(key, value, ttl) {
    this.backend.store(key, value, ttl);
    return this;
  }

  get(key) {
    let value = this.backend.load(key);
    return (typeof value !== 'undefined') ? value : null;
  }
}

let myCache = new MyCache(customCacheBackend);
assert(typeof myCache.set == 'function');
assert(typeof myCache.get == 'function');
$acl->setCache(myCache);

When a cache is provided, VirgenACL can automatically write to it after successfully determining access, and fetch from it prior to checking for access. If the AclCache#get function returns null, we can assume that it was a cache miss.
We may need to think through this a bit more but overall what do you guys think of this approach?

That suits me quite well. The other thing I hadn't thought about is the case where people are using this on the frontend and won't ever care about non-memory caching. The interface solutions handles the frontend case perfectly in that you provide a different cache adapter for in your frontend acl instance than your backend.
👍  from me!
I can probably volunteer some time to implement as well, though not likely in the near term (and if not me then I might make one of my coworkers do it). Either way I think this is a good idea.

Awesome! If I get some free time soon, I'll work on it and update you all here before I get started. Thanks for the feedback!

Instead of calling from cache, and if cache doesn't exist, then call virgen, perhaps implement a layer on top that does the caching behind the scenes, so:
virgen = require('virgen-acl)(someCacheLayer);
// continue to use virgen as normal.

Instead of calling from cache, and if cache doesn't exist, then call virgen, perhaps implement a layer on top that does the caching behind the scenes, so:

This is essentially what @djvirgen proposed, and will work really nicely. I was just giving an example of how to implement caching on top of virgen-acl until caching gets implemented into the library, since there's not really a good way to know when anybody will be able to free up time for this addition.

I personally don't think there's a need for TTL, what I would personally do is, whenever something is added/removed from a role or whatever, invalidate the relevant cache keys

The use case for TTL (or some other manual invalidation) is for people (like myself) who are using callbacks in their .allow() calls to do extra logic on top of what virgen-acl provides.
I think TTL is important, but doesn't need to be implemented by virgen-acl, this could be left up to the individual cache adapters.

authors: KieronWiltshire djvirgen spruce-bruce

repo: djvirgen/virgen-acl


text:
Redmine 4.0 compatibility

Replaces the deprecated alias_method_chain to be compatible with the new Redmine 4.0 relase.

Thank you.

authors: cappuMUC ncreuschling

repo: dkd/plantuml


text:
#190 - Improve PMD analysis on CI server

Adds proposed PMD plugin config to demonstrate "analysis caching" works on CI server at UKP.

Note: Requires a change which moves the new PMD plugin section to the pmd profile (marked as TODO), once the CI server executes the PMD goal again via activation of the pmd profile.

@reckart For what I can see, it seems to use the analysis cache just fine. Randomly taken from the last PR run:

[INFO] Analysis cache loaded
[INFO] Analysis cache updated

So the aim of this PR is basically reached. Question: Did you previously activate the pmd profile? If so, could you re-enable it, so I can put the pmd config back into an overriden profile? Atm - for testing purposes - I added it to the standard buildsection. However, I think this should only be run ifpmd` profile is actively chosen, e.g. by CI servers.

@reckart @mawiesne should we update this branch so it can be merged ? It looks like things are ramping up for a release soon.

@tgalery Been to Vietnam for 2 weeks and had a tight schedule over there. Having returned now, I just updated this branch to proceed here.
From my perspective, @reckart has to reconfigure the PR- build job to conduct pmd analysis given the new pmd profile and its related configuration is correctly activated. Last info I have: He disabled Jenkins' pmd plugin (I guess?) for testing purposes, so pmd isn't run twice.
@reckart Could you check this and give this PR a review afterwards?

@tgalery afaik Jenkinses PMD plugin doesn't really run PMD, it just picks up the log files produced by PMD in order to generate a report.
I have added the "pmd" profile to the PR build now. The master build already had it. If PMD is run twice, then that can probably be resolved by a) just adding the additional config in the POM via <pluginsManagement>, but not adding another execution b) ensuring that the new PMD execution has the same execution ID as the one inherited from the DKPro Parent POM so that it overrides it.

@mawiesne do you have any idea where the plugin stores its cache? I am asking for two reasons

The Jenkins build is configured to purge all untracked files from the repo and to reset all modified files to the actual remote states before actually building. This is designed to avoid cross-build effects such as having the license checker stumble over heap dump files etc, but also thwarts benign cross-build effects such as the cache you are aiming at.
The build runs clean install meaning that all target folders are deleted.

So I wonder if the loaded cache line is shown always (i.e. even if there was nothing to load at all) or if the plugin actually stores its data outside the reach of Jenkins and Maven (which would in fact worry me a bit because build jobs are not supposed to do that).

authors: mawiesne reckart tgalery ukp-svc-jenkins

repo: dkpro/dkpro-jwpl


text:
PATCH: Respect CC environment variable in Makefile

The GNU C compiler does not by default come with every platform.
Hardcoding it makes the library extremely unportable. This checks
whether CC is declared in the environment. If so, it uses that. In any
other case it defaults to /usr/bin/cc which, on most Linux distros
defaults to GCC. On FreeBSD, which was the issue, it defaults to
clang.
Tested on:
[nico@triton ~]$ uname -apKU
FreeBSD triton.herrhotzenplotz.geek 13.0-CURRENT FreeBSD 13.0-CURRENT #0 r367174: Fri Oct 30 22:07:29 CET 2020     nico@triton.herrhotzenplotz.geek:/usr/obj/usr/src/amd64.amd64/sys/TRITON  amd64 amd64 1300124 1300124

@Geod24 yes, that'd be perfect!


text:
Explain why a Lazy Variadic Delegate is better than a Lazy Variadic Array

Inspired by @schveiguy's forum post.

Closing & reopening to restart the CI.

authors: ZombineDev ntrel


text:
Remove special case of __traits(parameters) in foreach

The trait now consistently returns the function parameters
See dlang/dmd#13798

Would it make more sense to remove the foreach example entirely, since it's no longer a special case?

Would it make more sense to remove the foreach example entirely, since it's no longer a special case?

Would it make more sense to remove the foreach example entirely, since it's no longer a special case?

I vote for keeping it because it confirms that it doesn't to this to the experienced user, and indicates that opApply exists to the uninitiated

I vote for keeping it because it confirms that it doesn't to this to the experienced user

Agreed. The spec explicitly states that the foreach body is passed as a delegate to opApply, so it's better to be explicit IMO.

authors: MoonlightSentinel maxhaton pbackus


text:
HDRI Feature

Is your feature request related to a problem? Please describe.
Would be nice with the HDRI Feature. Or if there is a way to compile my own wasm with this feature enabled.

Describe the solution you'd like
Have HDRI Feature
Describe alternatives you've considered
No response
Additional context
No response

Having a HDRI build requires a Q16 build of this project. That should be possible but requires a separate version of this library. It would also require 4 times as much memory so I don't think I will be adding this anytime soon. Once we have proper 64-bit memory in wasm I will probably pick this up again.

authors: dlemstra ultimateshadsform

repo: dlemstra/magick-wasm


text:
Added section for team description

Provide some detail on affiliations

Branch merged and deleted.


text:
Allow disjoing components

fixes #72

Should wait until #71 is reviewed and merged so we can see the CI passing

Looks good, it might be an idea to add a yaml file to examples with a few disjointed components.

Done with the latest commit.

authors: abbiemery callumforrester

repo: dls-controls/tickit


text:
import_schema_path yaml.constructor.ConstructorError

dlt version
0.4.12
Describe the problem
import_schema_path seems to be generating an incorrectly formatted schema/import/.yaml
I'm getting the following error:
Traceback (most recent call last):
  File "/workspaces/dbt-duckdb-deltalake/dlt_newtest/filesystem_pipeline copy 5.py", line 23, in <module>
    load_info = pipeline.run(sql_table_source, table_name='Hazards_dlt', write_disposition="append")
  File "/usr/local/lib/python3.9/site-packages/dlt/pipeline/pipeline.py", line 222, in _wrap
    step_info = f(self, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/dlt/pipeline/pipeline.py", line 267, in _wrap
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/dlt/pipeline/pipeline.py", line 683, in run
    self.normalize(loader_file_format=loader_file_format)
  File "/usr/local/lib/python3.9/site-packages/dlt/pipeline/pipeline.py", line 222, in _wrap
    step_info = f(self, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/dlt/pipeline/pipeline.py", line 182, in _wrap
    schema = self._schema_storage.load_schema(name)
  File "/usr/local/lib/python3.9/site-packages/dlt/common/storages/schema_storage.py", line 54, in load_schema
    return self._maybe_import_schema(name, storage_schema)
  File "/usr/local/lib/python3.9/site-packages/dlt/common/storages/schema_storage.py", line 107, in _maybe_import_schema
    imported_schema = self._load_import_schema(name)
  File "/usr/local/lib/python3.9/site-packages/dlt/common/storages/schema_storage.py", line 155, in _load_import_schema
    return self._parse_schema_str(
  File "/usr/local/lib/python3.9/site-packages/dlt/common/storages/schema_storage.py", line 207, in _parse_schema_str
    imported_schema = yaml.safe_load(schema_str)
  File "/usr/local/lib/python3.9/site-packages/yaml/__init__.py", line 125, in safe_load
    return load(stream, SafeLoader)
  File "/usr/local/lib/python3.9/site-packages/yaml/__init__.py", line 81, in load
    return loader.get_single_data()
  File "/usr/local/lib/python3.9/site-packages/yaml/constructor.py", line 51, in get_single_data
    return self.construct_document(node)
  File "/usr/local/lib/python3.9/site-packages/yaml/constructor.py", line 60, in construct_document
    for dummy in generator:
  File "/usr/local/lib/python3.9/site-packages/yaml/constructor.py", line 413, in construct_yaml_map
    value = self.construct_mapping(node)
  File "/usr/local/lib/python3.9/site-packages/yaml/constructor.py", line 218, in construct_mapping
    return super().construct_mapping(node, deep=deep)
  File "/usr/local/lib/python3.9/site-packages/yaml/constructor.py", line 143, in construct_mapping
    value = self.construct_object(value_node, deep=deep)
  File "/usr/local/lib/python3.9/site-packages/yaml/constructor.py", line 100, in construct_object
    data = constructor(self, node)
  File "/usr/local/lib/python3.9/site-packages/yaml/constructor.py", line 427, in construct_undefined
    raise ConstructorError(None, None,
yaml.constructor.ConstructorError: could not determine a constructor for the tag 'tag:yaml.org,2002:python/object/apply:sqlalchemy.sql.elements.quoted_name'
  in "<unicode string>", line 81, column 15:
        resource: !!python/object/apply:sqlalchemy ..

Part of the yaml generated is below:
     updated_by:
        data_type: text
        nullable: true
        precision: 50
      sys_start_time:
        data_type: timestamp
        nullable: true
      sys_end_time:
        data_type: timestamp
        nullable: true
    write_disposition: append
    **resource: !!python/object/apply:sqlalchemy.sql.elements.quoted_name**
    - Hazards
    - null
  _dlt_pipeline_state:
    columns:
      version:
        data_type: bigint
        nullable: false

Script:
 import dlt
 import sql_database

pipeline = dlt.pipeline(
    pipeline_name="sql_pipeline",
    destination='mssql',
    dataset_name="sink", #schema at destination
    progress="log",
    import_schema_path="schemas/import",
    export_schema_path="schemas/export",
    full_refresh=True
    )

sql_table_source = sql_database.sql_table(
    credentials = <my_connection_string>,
    table= "Hazards",
    schema = "sink",
    detect_precision_hints=True
)

load_info = pipeline.run(sql_table_source, table_name='Hazards_dlt', write_disposition="append")
print(load_info)

Expected behavior
No response
Steps to reproduce
error:
    import_schema_path="schemas/import",
    export_schema_path="schemas/export",

error:
    import_schema_path="schemas/import",
    #export_schema_path="schemas/export",

Works fine:
    #import_schema_path="schemas/import",
    export_schema_path="schemas/export"

Possibly related to:  https://github.com/dlt-hub/dlt/issues/575
Operating system
Linux
Runtime environment
Docker, Docker Compose
Python version
3.9
dlt data source
dlt init sql_database mssql
dlt destination
No response
Other deployment details
No response
Additional information
No response

@MarkrJames we'll investigate this. looks like an sqlalchemy identifier is leaking into the schema. (possibly an explicit cast to str is missing)
I'll move this to verified sources

authors: MarkrJames rudolfix

repo: dlt-hub/dlt


text:
Add supports R4ACv2

Check list:

xqsystem/login: 🟢
xqsystem/init_info: 🟢
misystem/status: 🟢
xqnetwork/mode: 🟢
xqsystem/vpn_status: 🟢
misystem/topo_graph: 🟢
xqsystem/check_rom_update: 🟢
xqnetwork/wan_info: 🟢
misystem/led: 🟢
xqnetwork/wifi_detail_all: 🟢
xqnetwork/wifi_diag_detail_all: 🟢
xqnetwork/avaliable_channels: 🟢
xqnetwork/wifi_connect_devices: 🟢
misystem/devicelist: 🟢
xqnetwork/wifiap_signal: 🟢
misystem/newstatus: 🟢
xqsystem/reboot: ⚪
xqsystem/upgrade_rom: ⚪
xqsystem/flash_permission: ⚪
xqnetwork/set_wifi: ⚪
xqnetwork/set_wifi_without_restart: ⚪

Might last points be optional

I mean, why some routers cannot be added at all if some features like following are unavailable. Let's disable specific features instead of disabling the router at all.
xqsystem/reboot: ⚪
xqsystem/upgrade_rom: ⚪
xqsystem/flash_permission: ⚪
xqnetwork/set_wifi: ⚪
xqnetwork/set_wifi_without_restart: ⚪

repo: dmamontov/hass-miwifi


text:
[BUGFIX] Fix Vocab with unknown_token remapped to != 0 via token_to_idx arg

Description
https://github.com/dmlc/gluon-nlp/pull/732 forgot to remap the default specified in DefaultLookupDict, but only remapped the idx_to_token / token_to_idx data structures.
Checklist
Essentials

[X] PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
[X] Changes are complete (i.e. I finished coding on this PR)
[X] All changes have test coverage
[X] Code is well-documented

Changes

[X] Fix Vocab with unknown_token remapped to != 0 via token_to_idx arg

Comments

Needs to be backported to 0.7

Job PR-862/1 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-862/1/index.html

Job PR-862/2 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-862/2/index.html

authors: leezu mli

repo: dmlc/gluon-nlp


text:
How to run these ipynb example?

There is no document how to run these examples?

updated README.md for how to run

repo: dmlc/mxnet-notebooks


text:
Performance of deep speech is bad

Environment info
Operating System: ubuntu 14.04
GPU: GTX 1080
Package used (Python/R/Scala/Julia): python
MXNet version: 0.9.5
Or if installed from source: git clone https://github.com/dmlc/mxnet.git ~/mxnet --recursive
Error Message:
I just follow the default configure script in the example of speech recognition with the corpus of librispeech (clean-360 subset).

After two epochs, the CER seems to be bad with CER 70%+

It works so slow(it takes a whole week for two epochs of clean-360 subset)

Minimum reproducible example
use the default.cfg to train the deep speech model.
Steps to reproduce

python main.py --python main.py --configfile default.cfg

What have you tried to solve it?

Is there any result/performance you have tested in librispeech?

@Soonhwan-Kwon

train only on clean-360 can be bad. I'm training on the whole(teain+teain_360+train_5000) dataset and got results cer 0.362836,0.207333,0.173531,0.156486 on epoch 0,1,2,3 for the test-clean and for the train (sample every 100 batch)0.5152, 0.2408,0.1883,0.1622.And it is slow for now because it doesn't use rnn fused cell or variable length approach, but we are working on for the improvement.

And there is deepspeech.cfg for real use and default.cfg is for just for the example.

@Soonhwan-Kwon , @piiswrong
Many thanks. I will try it again.

@Soonhwan-Kwon ， after the sorted epoch, the random epochs get no much improvement in my training as shown in the figure. Can you give me some advice, please?

I found baidu's flac to wav script has bug when it has short duration, it generates repeated sound file as  many 6 times or more. So I changed flac to wav converter from avconv to sox, and I found it generates identical outputs but fixes bug. I put fix of fiac_to_wav.sh on https://github.com/samsungsds-rnd/deepspeech.mxnet/commit/ee99375ed5082ec38d03fe0cf30c1c5284edea0c for now. But I am middle of testing because found the bug yesterday with my team member, it may takes time to confirm it doesn't makes any unintended effect on model.

@xinq2016 I pulled request but it is not permitted yet. Please see https://github.com/samsungsds-rnd/deepspeech.mxnet for now. And there is a bi-graphemes option to deal with label in bi-graphemes, and the last result of 15% accuracy was using that option.

This issue is closed due to lack of activity in the last 90 days. Feel free to ping me to reopen if this is still an active issue. Thanks!
Also, do please check out our forum (and Chinese version) for general "how-to" questions.

@Soonhwan-Kwon Could you do me a favor to give me guidance of drawing performance curve like you did?

@megaSpoon have a look at mxboard https://github.com/awslabs/mxboard

authors: Soonhwan-Kwon ThomasDelteil megaSpoon piiswrong szha xinq2016

repo: dmlc/mxnet


text:
[Relay][ONNX] channels issue.

Dear All,
I try run THIS example with patch bellow, by importing ONNX via relay and then using relay.build() thus avoiding relay.build_module.create_executor().
Encountering several issues this way:

The first issue is that channels attribute is None type in replace_conv2d:

  File "/usr/lib64/python3.7/site-packages/topi/x86/conv2d.py", line 297, in _alter_conv2d_layout
    out_channel = attrs.get_int("channels") if F == sym else attrs.get_int("channels").value
AttributeError: 'NoneType' object has no attribute 'value'

If i try add channels to ONNX importer:

--- /usr/lib64/python3.7/site-packages/tvm/relay/frontend/onnx.py.orig	2019-04-04 19:42:33.696385914 +0300
+++ /usr/lib64/python3.7/site-packages/tvm/relay/frontend/onnx.py	2019-04-04 19:42:46.763339757 +0300
@@ -154,6 +154,8 @@
     @classmethod
     def _impl_v1(cls, inputs, attr, params):
         # get number of channels
+        channels = infer_channels(inputs[1], True)
+        attr['channels'] = channels
         out = AttrCvt(op_name=dimension_picker('conv'),
                       transforms={
                           'kernel_shape': 'kernel_size',

then
%18 = fn (%v1: Tensor[(1, 1, 224, 224), float32], %v2: Tensor[(64, 1, 5, 5), float32], %v3: Tensor[(64,), float32], %v4: Tensor[(64, 64, 3, 3), float32], %v5: Tensor[(64,), float32], %v6: Tensor[(32, 64, 3, 3), float32], %v7: Tensor[(32,), float32], %v8: Tensor[(9, 32, 3, 3), float32], %v9: Tensor[(9,), float32]) {
  %0 = nn.conv2d(%v1, %v2, padding=[2, 2], channels=1, kernel_size=[5, 5]) // an internal invariant was violdated while typechecking your program [19:42:59] /home/cbalint/rpmbuild/BUILD/tvm/src/relay/pass/type_solver.cc:100: Check failed: resolved.defined(): Unable to unify parent types: TensorType([1, 1, 5, 5], float32) and TensorType([64, 1, 5, 5], float32)

I could ignore value by this way:

--- /usr/lib64/python3.7/site-packages/topi/x86/conv2d.py	2019-04-04 20:27:49.735863767 +0300
+++ /usr/lib64/python3.7/site-packages/topi/x86/conv2d.py.orig	2019-04-04 19:48:48.452073215 +0300
@@ -294,7 +294,7 @@
     batch_size, in_channel, height, width = get_const_tuple(data.shape)

     groups = attrs.get_int("groups")
-    out_channel = attrs.get_int("channels") #if F == sym else attrs.get_int("channels").value
+    out_channel = attrs.get_int("channels") if F == sym else attrs.get_int("channels").value
     padding = attrs.get_int_tuple("padding")
     strides = attrs.get_int_tuple("strides")
     dilation = attrs.get_int_tuple("dilation")

Now it works, but not for different target like 'opencl -device=mali'.

For  'opencl -device=mali', running on rk3399 itself, there is other error triggered related to undefined channels:

In `main`:
v0.0.1
%14 = fn (%v1: Tensor[(1, 1, 224, 224), float32]) -> Tensor[(1, 1, 672, 672), float32] {
  %0 = nn.conv2d(%v1, meta[relay.Constant][0] // , padding=[2, 2], kernel_size=[5, 5]) //
  %1 = add(%0, meta[relay.Constant][1] // ) //
  %2 = nn.relu(%1) //
  %3 = nn.contrib_conv2d_winograd_without_weight_transform(%2, meta[relay.Constant][2] // , meta[relay.attrs.Conv2DWinogradAttrs][0]) // an internal invariant was violdated while typechecking your program [20:33:11] /builddir/build/BUILD/tvm/src/relay/op/nn/convolution.cc:383: Check failed: param->kernel_size.defined() && param->channels.defined(): The kernel size and channels of a Conv must be set or infered by previous pass

Let us move to https://github.com/dmlc/tvm/issues/2941

repo: dmlc/tvm


text:
[RELAY][PASS] Fix expr subst and CombineParallelConv2D

The entry in the substituion map may refer to substituded nodes. We need to recursively visit the result after subsitution.
Benchmark of CombineParallelConv2D
Inception-V3 (float32, CUDA 1080ti, batch=1) :
Before: 6.267070 ms
With CombineParallelConv2D: 5.989514 ms
cc @masahi @tqchen

Is CombineParallelConv2D result obtained after AutoTVM tuning? I assume weight concat will introduce new workloads.

Yes I tuned the new workload

@vinx13 This is not necessarily the most efficient way to do it, as subst will always get expanded every time it get referred to, is it possible to memomize?

@tqchen Could you elaborate a bit? Since ExprMutator::VisitExpr does memorization.

Sorry, my mistake

authors: masahi tqchen vinx13


text:
model trained is different with tree_method "gpu_hist" between on single GPU and on multi-GPU?

I tested tree_method = 'gpu_hist' on single GPU and multi-GPU, and I found the predictions are different. According to my understanding, the hist method first converts continuous feature values into
k discrete bins, and then construct a histogram with k bins for each feature. Single GPU or multi-GPU just distribute and evaluate the gradients and hess in k bins, so the trained model should be the same when controlling the randomness. Is there any mistake in my understanding?.
Test code in R (xgboost 0.81.0.1):
single GPU:
library('xgboost')
# Simulate N x p random matrix with some binomial response dependent on pp columns
set.seed(111)
N <- 1000000
p <- 50
pp <- 25
X <- matrix(runif(N * p), ncol = p)
betas <- 2 * runif(pp) - 1
sel <- sort(sample(p, pp))
m <- X[, sel] %*% betas - 1 + rnorm(N)
y <- rbinom(N, 1, plogis(m))

tr <- sample.int(N, N * 0.75)
dtrain <- xgb.DMatrix(X[tr,], label = y[tr])
dtest <- xgb.DMatrix(X[-tr,], label = y[-tr])
wl <- list(train = dtrain, test = dtest)

param <- list(objective = 'reg:logistic', eval_metric = 'auc',
  subsample = 1, num_parallel_tree = 1, nthread = 1L, eta = 0.3, max_depth = 6,
  colsample_bytree = 1, max_bin = 64, seed = 2018, tree_method = 'gpu_hist', n_gpus = 1, gpu_id = 0,
  grow_policy = "lossguide", max_leaves = 1e3)
pt <- proc.time()
bst_gpu <- xgb.train(param, dtrain, watchlist = wl, nrounds = 20)
proc.time() - pt

test_pred <- predict(bst_gpu, newdata = X[-tr,])
head(test_pred)
[1] 0.4517163 0.2565421 0.5847197 0.2956713 0.7016808 0.4059679

multi-GPU:
library('xgboost')

# Simulate N x p random matrix with some binomial response dependent on pp columns
set.seed(111)
N <- 1000000
p <- 50
pp <- 25
X <- matrix(runif(N * p), ncol = p)
betas <- 2 * runif(pp) - 1
sel <- sort(sample(p, pp))
m <- X[, sel] %*% betas - 1 + rnorm(N)
y <- rbinom(N, 1, plogis(m))

tr <- sample.int(N, N * 0.75)
dtrain <- xgb.DMatrix(X[tr,], label = y[tr])
dtest <- xgb.DMatrix(X[-tr,], label = y[-tr])
wl <- list(train = dtrain, test = dtest)

param <- list(objective = 'reg:logistic', eval_metric = 'auc',
  subsample = 1, num_parallel_tree = 1, nthread = 1L, eta = 0.3, max_depth = 6,
  colsample_bytree = 1, max_bin = 64, seed = 2018, tree_method = 'gpu_hist', n_gpus = 2, gpu_id = 1,
  grow_policy = "lossguide", max_leaves = 1e3)
  # max_depth = 20, grow_policy = "lossguide", max_leaves = 1e3)
pt <- proc.time()
bst_gpu <- xgb.train(param, dtrain, watchlist = wl, nrounds = 20)
proc.time() - pt

test_pred1 <- predict(bst_gpu, newdata = X[-tr,])
head(test_pred1)
[1] 0.5315008 0.2953689 0.6302567 0.2885332 0.6240512 0.3615361

@joegaotao I think the two models are different to begin with. You should get text dump of each model and compare. The predictor itself should give the same answer for the same model, regardless of n_gpus set. Prediction is different because model trained is different.

@hcho3  I know the trained models  are different leading to the different prediction, but the seed, dataset and parameter are the same except the n_gpu. When specifying tree_method="gpu_exact", the model is the same, while tree_method="gpu_hist"with different gpu number is different. So I wonder what it is the difference in multi-GPU and single GPU when implementing gpu_hist algorithm?

@joegaotao GPU hist uses approximate quantile sketchs to produce candidate split points. The sketch algorithm is approximate and thus may produce different answers for different n_gpus. Do you see significant drop in AUC?
@RAMitchell @trivialfis Correct me if I'm wrong.

@hcho3 I use it in the regression mode with our customized evaluation metric, and I found the metric decayed significantly. The candidate split points is not  sketched by all data firstly?

@joegaotao Does your example script also show drop in metric?

@hcho3 @joegaotao Let me take a look.  I didn't try to look into this is because reason described in #3889.  If this sorted out, the results from single GPU and multi-gpu won't be the same in future anyway.

@hcho3 the example script is just a toy example, the different predictions seems to have no effect in such scenario.

@trivialfis I see. If each device propose splits based on its subset of the data, the parameter n_gpus will introduce some randomness for the data split strategy...

authors: hcho3 joegaotao trivialfis

repo: dmlc/xgboost


text:
the leaf info for multi-class model during dump booster is wrong

import xgboost as xgb
from sklearn.datasets import make_classification
x, y = make_classification(n_classes=3, n_informative=10)
model = xgb.XGBClassifier(n_estimators=1, max_depth=1).fit(x, y)
booster = model.get_booster()
print(booster.get_dump()[0])

outputs this
0:[f2<-5.2242012] yes=1,no=2,missing=1
	1:leaf=0.119999997
	2:leaf=-0.0128686372

here the leaf section doesn't mean anything towards prediction.
i did try booster.tree_to_dataframe() which gives similar makes no sense results

It's the weight of leaf.  The final prediction is softmax(sum of weights) .

I'm not exactly sure what weight it is. since the prediction would be a 3d vector. and the leaf only outputs 1 scalar.
here it is a 3 class classification

XGBoost employs 1 stack of trees for 1 class.  So for 3 classes there will be 3 stacks of trees.  Each tree leaf contains a scalar output called weight.  Please refer to the tutorial in our documents for an explanation of tree models.

interesting.. so it's different from the sklearn.tree.DecisionTreeClassifier. Thanks for the clarification.

a follow-up question, how does the stack behave when there is more than 1 estimator?
[estimator_1_class_1, estimator_1_class_2, estimator_1_class_3, estimator_2_class_1, estimator_2_class_2, estimator_2_class_3]
or
[estimator_1_class_1, estimator_2_class_1, estimator_1_class_2, estimator_2_class_2, estimator_1_class_3, estimator_2_class_3]

import xgboost as xgb
from sklearn.datasets import make_regression
import numpy as np
x, y = make_regression( n_informative=10)
y = np.vstack([y,y]).T
model = xgb.XGBRegressor(n_estimators=1, max_depth=1).fit(x, y)
assert model.predict(x).shape == y.shape, "prediction should have same shape as fitting"

AssertionError                            Traceback (most recent call last)
 in ()
5 y = np.vstack([y,y]).T
6 model = xgb.XGBRegressor(n_estimators=1, max_depth=1).fit(x, y)
----> 7 assert model.predict(x).shape == y.shape, "prediction should have same shape as fitting"
AssertionError: prediction should have same shape as fitting

@yupbank The trees are handled internally in XGBoost.  You don't have to do anything special for it.

authors: trivialfis yupbank


text:
Throw error when using unsupported objective with predict_proba.

This partially reverts https://github.com/dmlc/xgboost/pull/6817 on the use of softmax.  Looking into it more thoroughly, the softmax function can never appear in training due to automatic configuration in fit.  But the objective can still be invalid for predict_proba (like softmax) as illustrated in the test as an edge case.  Previously I let those functions pass and may return an incorrect result.  This PR reverts that part and throws a ValueError.  Other than that fixes in linked PR are still useful and kept.

Closing for now.  Handling this edge case might require more efforts.


text:
THIS PROJECT IS DEAD, any active fork ?

or so it looks by lack of feedback or any other action against any of the old contributions.
I file this ticket aiming to gather interest in an active fork...

I've been wondering the same thing. I'd like to move forward with its use, but concerned that noone is actively fixing code.

I tried sending all my PRs to the most recently modified fork known by github: https://github.com/renesugar/sqlines/pull/1 - that's my "exp" branch, containing a merge of all of my open PRs in this repo. Let's see if @renesugar is willing to take on maintainance or would you be willing to do that @djgreen ?

I'm not really a programmer. I'd recommend that you ask a collection of folks who've submitted PRs to see if they'd be willing to share that burden.

FYI: @renesugar had merged all my PRs into his fork

Finding it very strange that there are June and July 2018 binaries available on http://www.sqlines.com/download -- who's updating that site?

@djgreen, I'm still wondering the same thing... Have you learned anything else?

Nothing. Though I didn't investigate.

How can I get the head files and libs in linux which deps by sql-server, sybase, db2, postgresql

authors: dayuanyuan1989 djgreen strk szydek

repo: dmtolpeko/sqlines


text:
Commits not selectable

In small repos especially, there can be instances where a commit cannot be selected

Also, they sometimes get colored strangely, being green instead of blue

At least occurs when you have a repo with 1 commit, then outside of elegit make a new commit

repo: dmusican/Elegit


text:
Lexicon changed for data tier definition.

Client unit test test11: web.DBSClientWriter.insertDataset is broken. need to fix it.
dbsException-invalid-input2: '/unittest_web_primary_ds_name_6102/acq_era_6102-v6102/GEN-SIM-RAW_2' does not match regular expression ^/[a-zA-Z0-9-]{1,99}/[a-zA-Z0-9.-]{1,199}/[A-Z-]{1,50}$ key-value pair (dataset, /unittest_web_primary_ds_name_6102/acq_era_6102-v6102/GEN-SIM-RAW_2) cannot pass input checking

Same thing for ClientBlockWriter test

Done and committed to upmaster. This does not require a rebuilding because it is a unit test. Will include it in next release, then close the issue.

Done.

repo: dmwm/DBS


text:
Update buildreleases script to handle different branches

I also removed all the CVS handling, since it's dead anyways for quite a bit.
The building of RPM was also cleared.
@ticoann please review. I already tested with the tag for 1.0.14_crab branch :-D
I still have to update this documentation:
https://github.com/dmwm/WMCore/wiki/TaggingAndReleasing

Thanks Alan, Does this also update the tagging differences on branch as well?

It seems to update the differences based on the branch base, like:
https://github.com/dmwm/WMCore/commit/ba0a5f6ffc59520a1056b9a1dc68c85d6de09306
Is it what you mean?

Yes, Alan. Good work. :smile:

:-) I also updated the documentation in:
https://github.com/dmwm/WMCore/wiki/TaggingAndReleasing

Thanks Alan, stop working. :smile:

authors: amaltaro ticoann

repo: dmwm/WMCore


text:
Add creation details for files

Fixes #9042
What I found was that the block has "create_by" and "creation_date" fields:
https://cmsweb.cern.ch/dbs/prod/global/DBSReader/blocks?block_name=/WminusH_HToZZTo4L_M190_13TeV_powheg2-minlo-HWJ_JHUGenV7011_pythia8/RunIIFall17MiniAODv2-PU2017_12Apr2018_94X_mc2017_realistic_v14-v2/MINIAODSIM%239fef307a-6fe4-4dce-9df1-ca700e5dc7a9&detail=true
The file does not:
https://cmsweb.cern.ch/dbs/prod/global/DBSReader/files?block_name=/WminusH_HToZZTo4L_M190_13TeV_powheg2-minlo-HWJ_JHUGenV7011_pythia8/RunIIFall17MiniAODv2-PU2017_12Apr2018_94X_mc2017_realistic_v14-v2/MINIAODSIM%239fef307a-6fe4-4dce-9df1-ca700e5dc7a9&detail=true
This PR adds the "create_by" and "creation_date" to a file, similar to how it is added to the block.

Patch looks good, Erik. If you don't manage to get it tested by tomorrow morning, I'll then run a quick test.
@yuyiguo Yuyi, apparently WMAgent has never sent those 2 file parameters when inserting a block.
This patch fixes that, can you just confirm whether it makes sense and it's indeed expected by the DBSServer?

I won't get to testing this today. Give it a test tomorrow if you can.

This doesn't work! We're getting the following exception on the DBS3Upload component:
2019-05-19 09:38:12,380:139667854329600:ERROR:DBSUploadPoller:Error trying to process block /DYJetsToLL_Pt-50To100_TuneCUETP8M1_13TeV-amcatnloFXFX-pythia8/DMWM_Test-StepChain_MC_Agent122_Validation_Privv2-v20/LHE#5702bfe6-c764-4d5c-a2f6-d4742b6dffe1 through DBS. Error: HTTP Error 400: illegal variable name/number from input

which is likely coming from this DBSServer code:
https://github.com/dmwm/DBS/blob/master/Server/Python/src/dbs/web/DBSWriterModel.py#L278
which brings us to this suspicious code:
https://github.com/dmwm/DBS/blob/master/Server/Python/src/dbs/business/DBSBlockInsert.py#L216
pointing to an old issue (likely from the old SVN VCS).
From the DBSSErver logs, it looks like the DAO [1] doesn't support those 2 parameters and it always uses the DB schema value.
@yuyiguo can you confirm whether it's really not supported and thus we should stop working on it? Or is it something that needs to be fixed/supported on the DBS side?
[1]
 'insert all\n           when not exists (select * from cms_dbs3_int_global_owner.file_data_types where file_type = file_t) then\n                into cms_dbs3_int_global_owner.file_data_types(file_type_id, file_type) values( cms_dbs3_int_global_owner.seq_ft.nextval, file_t )\n           when 1 = 1 then\n                into cms_dbs3_int_global_owner.files (file_id, logical_file_name, is_file_valid, \n                        dataset_id, block_id, file_type_id, check_sum, event_count, file_size,\n                        adler32, md5, auto_cross_section,\n                        last_modification_date, last_modified_by)\n                values (:file_id, :logical_file_name, :is_file_valid, :dataset_id, :block_id, \n                        nvl(( select file_type_id from cms_dbs3_int_global_owner.file_data_types where file_type = file_t ), cms_dbs3_int_global_owner.seq_ft.nextval ),  \n                        :check_sum, :event_count, :file_size, \n                        :adler32, :md5, :auto_cross_section, \n                        :last_modification_date, :last_modified_by) \n         select :file_type file_t from dual               \n       ' {'check_sum': '776043786', 'adler32': '6c748173', 'block_id': 19067039, 'event_count': 3008, 'file_type': 'EDM', 'create_by': 'WMAgent', 'last_modified_by': 'wmagent@vocms0192.cern.ch', 'creation_date': 1558154593, 'file_id': 376512317, 'logical_file_name': '/store/backfill/1/Run2016H/HLTPhysicsIsolatedBunch/MINIAOD/ReReco_RunBlockWhite_Agent122_Validation_Privv2-v11/00000/7A618884-2679-E911-84DB-AC1F6BAC7F24.root', 'file_size': 151867119, 'last_modification_date': 1558224000.334906, 'dataset_id': 13835648, 'is_file_valid': 1, 'auto_cross_section': 0.0, 'md5': None}

Just checking on this PR @yuyiguo. Should we keep this open or close it out?

@yuyiguo can you please confirm whether this is data that we cannot / should not provide to DBS? Or is it a problem on the server side? Thanks

@yuyiguo did you manage to look into it? Please let us know if you need another test on going.

As discussed in:
https://github.com/dmwm/DBS/issues/615
there is nothing for us to do here and we should not be providing any value for those 2 fields at file level.
Sorry for wasting your time, Erik. Closing this issue.

authors: amaltaro goughes


text:
Don't share the same DBS object amongst multiple threads

Fixes #9221
Fixes #9253
Status
not-tested
Description
Instead of using a global cache of DBSReader, which are then shared between multiple threads, make sure each thread constructs its own DBSReader object (and then reuse it within the thread).
This PR also fixes how we evaluate whether the DBSReader object is pointing to the global instance or not (no need to make any calls to the server).
I wanted to get rid of this __dbses global variable:
https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/WorkQueue/WorkQueueUtils.py#L30
but it would require substantial changes to the start policy algorithms. Given that only those are now using this global var, and they are sequential, I decided to let them be.
Is it backward compatible (if not, which system it affects?)
yes
Related PRs
no
External dependencies / deployment changes
Needs to be validated on both levels: global and local workqueue.

does this mean that the dbsapi is not really thread safe @yuyiguo @vkuznet ?

Yes, Jean-Roch. See this comment
https://github.com/dmwm/DBS/pull/610#issuecomment-503639476
where Valentin describes the problem.
I let them comment it further though.
Don't you see DBS problems on the Unified side? Or it catches an exception and silently retry?

test this please

Tests look good in my VM as well. I double checked the jenkins job configuration and I still don't get why those Rucio tests don't get reported as unstable...

authors: amaltaro vlimant