text:
[Bug] config.get(config.field, default_value) not working properly?

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
for the code below in adapters,
{%- set enable_refresh = config.get('enable_refresh', false) -%}

When the field enable_refresh is not set by model config, a default value false is not used by config.get().
Instead, enable_refresh is set to none.
Expected Behavior
for the code below in adapters,
{%- set enable_refresh = config.get('enable_refresh', false) -%}

When the field enable_refresh is not set by model config, a default value false shall be used. enable_refresh shall be set to `false.
Interestingly, if I use the code below, it works as expected.
{%- set enable_refresh = config.get('enable_refresh', default=false) -%}

Steps To Reproduce
Background
I came across this unexpected behavior when I tried to upgrade BQ materialized_view in dbt-labs-experimental-features  to work with dbt 0.21.0
If I have a model materialized_view_A.
Not setting the enable_refresh field in the model config.
{{
  config(
    materialized = "materialized_view",
    partition_by = {
      "field" : "TIMESTAMP_TRUNC(stamp_hourly, DAY)",
      "data_type" : "date"
    },
  )
}}

in bigquery__create_materialized_view_as macro,
    {%- set enable_refresh = config.get('enable_refresh', false) -%}

if you log the value of enable_refresh, you will see it's set to none, which is unexpected.
Now, if you change the code to
    {%- set enable_refresh = config.get('enable_refresh', default=false) -%}

if you log the value of enable_refresh, you will see it's set to false, which is expected.
Relevant log output
No response
Environment
- OS: MacOS
- Python: 3.7
- dbt: 0.21.0

What database are you using dbt with?
bigquery
Additional Context
No response

@hui-zheng You're right! This is a duplicate of a much older issue, one we closed a while ago: https://github.com/dbt-labs/dbt-core/issues/2441.
@gshank It looks like you resolved that issue after v0.20, and I agree we got 90% of the way there, by making config behave much more like a dict, and config.get() a lot more like dict.get(). The subtle difference, raised again in this issue, is that you can provide the default value as the second positional argument for dict.get(). By contrast, default is still not the second positional argument for our config.get()—it's the third, with validator coming in second:
https://github.com/dbt-labs/dbt-core/blob/0722922c034b48c21c8fca8937c2f6b08688e8cd/core/dbt/context/providers.py#L328-L329
https://github.com/dbt-labs/dbt-core/blob/0722922c034b48c21c8fca8937c2f6b08688e8cd/core/dbt/context/providers.py#L372-L373
I think the resolution to this issue would be as simple as switching the argument order:
    def get(self, name, default=None, validator=None):
        return ''

    def get(self, name, default=None, validator=None):
        to_return = self._lookup(name, default)

That way, config.get('arg1', 'arg2') will be the same as config.get('arg1', default = 'arg2').
This would be a big breaking change. v1 is absolutely the right time to do it; v1 after we've already put out the first release candidate, not as much. Reasons I'm still open to this:

We're planning a longer release candidacy, and we're planning to put out a few more RCs before the final
Setting default is much more common than setting validator; 100x more common, if I had to guess. (The only place I can think where we use the validator arg is here in a dbt-spark macro, and even there we use it as a keyword argument, since it's a pretty uncommon thing.)

So, what do we think of making this change?

@hui-zheng As an unrelated point: Where did you see enable_refresh? I don't see that code anywhere in our adapter plugins; I think it's in the "experimental feature" for materialized views.

I think this is the right time to make this change.

The python dictionary get method only returns the default if the key doesn't exist. So if 'enable_refresh' is set to None in the config, that's what it will return, without regard to the default. I'm thinking that this should work how it does in Python, which it does right now.

@gshank If that's the case, I'd expect:
-- models/any_model.sql
select {{ config.get('made_up_nonexistent_key', 'default_value') }} as col_value

To compile to:
select 'default_value' as col_value

Instead of (current):
select None as col_value

It works in Python. I think Jinja does something to the arguments.

You're right, it's the 'get' method in the RuntimeConfigObject. I didn't realize that we were wrapping that there. What is the 'validator' used for?

My proposal is that we remove the 'validator' keyword argument from 'get' and 'require'. All it does is pass along and execute the passed in function with the value, which can be done in the template if that's wanted. No tests failed. Do you know of anybody who uses it?

Offhand, I know of exactly two places (really one place) where validator is used, in the dbt-spark plugin.
My vote might be to reverse the order of the keyword arguments, but I'd be okay with removing validator entirely if that's cleaner. We'd just need to update dbt-spark, which is fine.

We can leave it in place.

This was fixed with PR #4297. It didn't automatically close because I incorrectly put the ticket number as 4272

authors: gshank hui-zheng jtcohen6


text:
[CT-2833] Make metric dimension list available on the metric documentation page

Describe the feature
When I open the documentation of a metric, I would like to see the dimensions available for this metric.
Describe alternatives you've considered
When scanning through the open issues list, I found this issue about exposing the metric yaml in the documentation. If that was exposed, it would provide the same value.
Additional context
When moving to MetricFlow the available dimension list will be extended thanks to additional functionality. However, I'm sure 'dbt compile' actually has to know the available dimension list anyway, so I'm guessing it will not be that much more difficult to make it available in documentation. And considering the MetricFlow functionality, the available dimension list will be even more valuable than in current metrics.
Who will this benefit?
When looking for metrics, it is very important to know at which granularities these metrics are available. Let's say I want to calculate country level 'number of orders' metric. I find the 'number of orders' metric in dbt docs, but I still have no idea if I can actually use it to calculate the country level values.
Are you interested in contributing this feature?
Sure, but I have no idea where to start.

Hello! Thanks for opening up this issue.
We did some work for 1.6.1 to include semantic models in the docs site, would you want dimensions included in the semantic models page?

I think this would be pretty straightforward to add for the semantic models, but not possible (or would take significant effort) for metrics.
Candidly, our team isn't able to prioritize new feature development in dbt-docs, so this isn't something we'd be picking up ourselves.

would you want dimensions included in the semantic models page?

Absolutely!

Seems like you already have an idea of how this could be implemented. How can I support?

The code to add dimensions to the semantic model page would probably be quite similar to the code we already have for displaying entities:
https://github.com/dbt-labs/dbt-docs/blob/b8b0dc12f2be9145809414a7cb32b264575b8494/src/app/docs/semantic_model.html#L61-L85
To reiterate Grace above, we won't be able to prioritize this one ourselves - but if you're interested in having a go at it, we'd welcome a PR!

Hi
Linked a PR of a friend of mine that might be useful (or even can be merged). I did solve mine issue with simple visualization, but obviously other attributes might be visualized as well and expand/collapse logic might be added.
Hope it's helpful

source: gharchive/issue

authors: SOVALINUX graciegoheen jtcohen6 siljamardla

repo: dbt-labs/dbt-docs

license_type: permissive

license_source: github-api


text:
Update starrocks-setup.md

updating github URI

preview looks great! https://docs-getdbt-jd4g6h2bd-dbt-labs.vercel.app/docs/core/connect-data-platform/starrocks-setup

source: gharchive/pull-request


text:
Fix typo: a missing ; on the line 46

I think a ; should be placed at this position. On ST3 + CSS3 pkg, the bio id selector which is follow right after it, has not been highlighted correctly.

Thanks @xuansamdinh. It's not necessary as a termination character though.

authors: dbtek xuansamdinh

repo: dbtek/dbyll

license: mit

license_source: bigquery


text:
DCOS-13710 [preparation]: Fix the code that prettier can't process

Currently prettier can't prettify pieces of code where return statements have comments in it.
return (
  // Comment
  someLogic !== here
);

So I decided to fix those as a separate PR so that these changes could be tested separately.

Can one of the admins verify this patch?

@MatApple fixed!

Ping @orlandohohmeier @jfurrow 🙇

Thanks for the suggestion @jfurrow! Addressed all comments, please give it another shot
@MatApple @orlandohohmeier

I think that we should reduce the complexity of our shouldComponentUpdate methods as there way to many conditions but that is a different issue and not part of this PR. Great job! 💯 🥇

LGMT @nLight! I'll hold off on merging until @MatApple has had a chance to review the changes.

@MatApple can we get an approval?

I rebased this as it's following PR had conflicts and I am resolving those

Run integration tests!

run unit tests

authors: MatApple jfurrow mesosphere-ci mlunoe nLight orlandohohmeier

repo: dcos/dcos-ui


text:
DCOS-41250: implement basic soak tests for DC/OS UI

Testing

That's pretty hard, you can run the script against a cluster by setting the environment variables (according to the readme, therefore I won't give you a command).
Trade-offs

Decided to use puppeteer (which has a high level of abstraction) to move fast. We might need to move to sth that supports multiple browsers later, but first starting to make experiences with soaking in general is a good first step.
Decided to manually log in, partly because I think our test should fail if the login fails, also because I think adding some python utils to handle token generation won't help wiht maintainability

Dependencies

This PR needs to be merged before we can deploy it to soak.
Screenshots

Just crossed my mind. Having it in the OSS repo would mean we don't soak EE plugins, thus it won't be able to login, right?

@nLight you are right, we should move this 🤔 I'll make it work and move it then

Even with the special Jenkins node it doesn't work?

@nLight Nope, couldn't get it running on any node so far. @juliangieseke mentioned that Docker in Docker could also referr to being able to use a different dockerfile on these agents and not promise a docker deamon to be available. We briefly thought about installing docker on the mac mini and calling it a day, but this seems like a non-ideal solution.

Shall we move the code into EE repo and scope out the CI part?
So that before another SOAK test we would just manually build the image? (< Should be documented)

I bet we could just extract this step and setup a separate job that would run in docker-in-docker image. But I'd say let's scope that out

@nLight I tried out 4 different jenkins labels with docker in docker and neither of them worked. I can move it over and we scope it out, seems reasonable to me.

authors: DanielMSchmidt nLight


text:
fix(deps): update dependency semver to v7.3.2

This PR contains the following updates:

Package
Type
Update
Change

semver
dependencies
patch
7.3.0 -> 7.3.2

Release Notes

npm/node-semver
v7.3.2
Compare Source
v7.3.1
Compare Source

Renovate configuration
:date: Schedule: At any time (no schedule defined).
:vertical_traffic_light: Automerge: Disabled by config. Please merge this manually once you are satisfied.
:recycle: Rebasing: Whenever PR becomes conflicted, or if you modify the PR title to begin with "rebase!".
:no_bell: Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

Available now for Enterprise: Renovate Pro with real-time webhook handling and priority job queue.

:tada: This PR is included in version 5.0.21 :tada:
The release is available on:

v5.0.21
GitHub release

Your semantic-release bot :package::rocket:

authors: mesosphere-ci mesosphere-renovate


text:
Enabled all cgroup subsystems for DC/OS.

High-level description
This PR enabled all cgroup subsystems for DC/OS.
Corresponding DC/OS tickets (obligatory)

DCOS_OSS-3680 Enable auto cgroup in DC/OS.

Checklist for all PRs

[ ] Added a comprehensible changelog entry to CHANGES.md or explain why this is not a user-facing change:
[ ] Included a test which will fail if code is reverted but test is not. If there is no test please explain here:
[ ] Read the DC/OS contributing guidelines
[ ] Followed relevant code rules Rules for Packages and Systemd

Checklist for component/package updates:
If you are changing components or packages in DC/OS (e.g. you are bumping the sha or ref of anything underneath packages), then in addition to the above please also include:

[ ] Change log from the last version integrated (this should be a link to commits for easy verification and review): example
[ ] Test Results: [link to CI job test results for component]
[ ] Code Coverage (if available): [link to code coverage report]

PLEASE FILL IN THE TEMPLATE ABOVE / DO NOT REMOVE ANY SECTIONS ABOVE THIS LINE
Instructions and review process
What is the review process and when will my changes land?
All PRs require 2 approvals using GitHub's pull request reviews.
Reviewers should be:

Developers who understand the code being modified.
Developers responsible for code that interacts with or depends on the code being modified.

It is best to proactively ask for 2 reviews by @mentioning the candidate reviewers in the PR comments area. The responsibility is on the developer submitting the PR to follow-up with reviewers and make sure a PR is reviewed in a timely manner. Once a PR has 2 ship-it's, no red reviews, and all tests are green it will be included in the next train.

This repo has @mesosphere-mergebot integration. You can interact with the following commands.
@mesosphere-mergebot override-status
@mesosphere-mergebot label [Request For Comment |Work In Progress |Ship It |Ready For Review |Holding]
@mesosphere-mergebot merge-it
@mesosphere-mergebot bump-ee

PR creators can apply one of  [Ready For Review |Work In Progress]. Owners can apply any label.

@mesosphere-mergebot bump-ee

@mesosphere-mergebot label Ready For Review

repo: dcos/dcos


text:
dcos-test-utils: bump for better Marathon group cleanup

This bumps dcos-test-utils for a better cleanup of the Marathon state between tests in the DC/OS Integration test suite.
Specifically, this brings this patch: https://github.com/dcos/dcos-test-utils/pull/69/files
To be sure: https://github.com/dcos/dcos-test-utils/compare/713dea0...1286f4b86faadc75

@mesosphere-mergebot bump-ee

@mesosphere-mergebot bump-ee

@mesosphere-mergebot bump-ee

@mesosphere-mergebot label Ready For Review

@mesosphere-mergebot override-status teamcity/dcos/test/dcos-docker/static/group3 https://jira.mesosphere.com/browse/DCOS-45663

@mesosphere-mergebot override-status teamcity/dcos/test/dcos-docker/static/group3 https://jira.mesosphere.com/browse/DCOS-45663

@mesosphere-mergebot merge-it

Fast merge, to move along.

@mesosphere-mergebot bump-ee

Fast Merge approved :+1:

authors: gpaul jgehrcke rukletsov


text:
3dt bug fixes

increase exhibitor response timeout to 11sec.
If a master node goes down, a get request to exhibitor may take up to 10 sec.
(http://127.0.0.1:8181/exhibitor/v1/cluster/status).
https://github.com/dcos/3dt/pull/50

fix typo
https://github.com/dcos/3dt/pull/51

🚢

authors: cmaloney darkonie


text:
[master] Automated Bump DC/OS UI v2.91.2

High-level description
What features does this change enable? What bugs does this change fix?
Corresponding DC/OS tickets (obligatory)
These DC/OS JIRA ticket(s) must be updated (ideally closed) in the moment this PR lands:

DCOS_OSS- Foo the Bar so it stops Bazzing.

Related tickets (optional)
Other tickets related to this change:

DCOS_OSS- Foo the Bar so it stops Bazzing.

Checklist for all PRs

[ ] Added a comprehensible changelog entry to CHANGES.md or explain why this is not a user-facing change:
[ ] Included a test which will fail if code is reverted but test is not. If there is no test please explain here:
[ ] Read the DC/OS contributing guidelines
[ ] Followed relevant code rules Rules for Packages and Systemd

Checklist for component/package updates:
If you are changing components or packages in DC/OS (e.g. you are bumping the sha or ref of anything underneath packages), then in addition to the above please also include:

[ ] Change log from the last version integrated (this should be a link to commits for easy verification and review): example
[ ] Test Results: [link to CI job test results for component]
[ ] Code Coverage (if available): [link to code coverage report]

PLEASE FILL IN THE TEMPLATE ABOVE / DO NOT REMOVE ANY SECTIONS ABOVE THIS LINE
Instructions and review process
What is the review process and when will my changes land?
All PRs require 2 approvals using GitHub's pull request reviews.
Reviewers should be:

Developers who understand the code being modified.
Developers responsible for code that interacts with or depends on the code being modified.

It is best to proactively ask for 2 reviews by @mentioning the candidate reviewers in the PR comments area. The responsibility is on the developer submitting the PR to follow-up with reviewers and make sure a PR is reviewed in a timely manner. Once a PR has 2 ship-it's, no red reviews, and all tests are green it will be included in the next train.

@mesosphere-mergebot label [Ready for Review]

Superseeded by #5421


text:
exhibitor: bump

High-level description
What features does this change enable? What bugs does this change fix?
Corresponding DC/OS tickets (required)

D2IQ-ID JIRA title / short description.

Related tickets (optional)

D2IQ-ID JIRA title / short description.

@mesosphere-mergebot test

@mesosphere-mergebot test

@mesosphere-mergebot test all

@mesosphere-mergebot test all

closing and rebasing to master


text:
🛑 Danbs is down

In 045beb8, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 154 ms

Resolved: Danbs is back up in 1c07181 after 26 minutes.

repo: ddanbs/upptime

license: MIT


text:
🛑 Danbs is down

In f7a6b38, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 95 ms

Resolved: Danbs is back up in 10ef5b5 after 36 minutes.


text:
🛑 Danbs is down

In a8de767, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 3909 ms

Resolved: Danbs is back up in 5bc9814 after 35 minutes.


text:
🛑 Danbs is down

In cb33c34, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 2090 ms

Resolved: Danbs is back up in 5fa17f8 after 34 minutes.


text:
🛑 Danbs is down

In 6e221de, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 158 ms

Resolved: Danbs is back up in 206b43a after 19 minutes.


text:
🛑 Danbs is down

In ea84447, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 115 ms

Resolved: Danbs is back up in 5e0d594.


text:
🛑 Danbs is down

In fed28bd, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 158 ms

Resolved: Danbs is back up in 7ef1a2f.


text:
🛑 Danbs is down

In 4109168, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 88 ms

Resolved: Danbs is back up in 57620f7.


text:
🛑 Danbs is down

In d003ff7, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 93 ms

Resolved: Danbs is back up in 45301eb.


text:
🛑 Danbs is down

In 2c15cad, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 154 ms

Resolved: Danbs is back up in 1a797f2.


text:
:bug: Account for single-row to multi-row axis=0 apply

Fixes #22

Codecov Report

Merging #23 (9a795f9) into master (3232bfd) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##            master       #23   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            3         3
  Lines           80        80
  Branches        13        13
=========================================
  Hits            80        80

Impacted Files
Coverage Δ

src/mapply/mapply.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3232bfd...9a795f9. Read the comment docs.

authors: codecov-commenter ddelange

repo: ddelange/mapply


text:
Drop drupal version (9) from project name

I see people are avoiding the use of this addon because it says drupal9 on the project name. It only happens with people newer to DDEV.
With drupal9 being EOL, I would advise to rename the project and completely remove the core version from that and the description.

The actual recommended add-on, maintained by the maintainer of search_api_solr is the ddev-solr add-on. Interested in your thoughts on right path forward. I'm ok with rename, demotion...

Then maybe we should decommission this one, in favor of ddev-solr.
The thing is that ddev get --list displays an addon that says "drupal9" in it's name. That's confusing to people. I've been using it in D9 projects and continue to use it now. I had discussions with people asking me where is the D10 version.
And to be honest, I never thought of using ddev-solr, since there was a "more" drupal version.

The problem is that so many people use this one. And, frankly, it's a bit simpler.
Would you be willing to take on the maintenance of this add-on?
I'd be fine with any of these directions. I think this one should at least

Rename
Update the README  (and explain the difference, etc)

I admit that I've been using this one too since it appeared. I don't mind trying to maintain this one (time is a bit limited right now), but shouldn't we have just one?
If ddev-solr is the way to go, then maybe we should deprecate this?
Decisions... decisions... decisions..

This one is definitely simpler and smaller, but ddev-solr is "the recommended technique". As a non-solr-expert I have trouble following such a complex subject.
Let's at least deprecate as far as referring to the other one clearly in the README, and let's change the title. But maybe ddev-olddrupal-solr ? ddev drupal-oldsolr ?
I gave you maintainer privs on this. Let me know what I can do to help.

In that case, I would suggest ddev-drupal-simple-solr or ddev-drupal-solr.
I've been reading the other projects README. I understand there is place for a simpler approach just for Drupal (since Solr is so popular in Drupal).
I'll try to identify all the differences and document them here. If time allows I'll even try the other project to get a better picture of what it offers and how harder it is.
Thanks for the privileges, it's an honour.
If we can decide to ditch 9 from drupal9 then I guess it's a good first step. So I would need your help with that :)

Absolutely let's get rid of the '9'. Just let me know what the new name should be and I'll do it. ddev-drupal-simple-solr would be fine with me. I do want to honor @mkalkbrenner and his maintenance and effort that's gone into ddev-solr, and of course as the maintainer of Drupal's search_api_solr he should have lots of sway here. Maybe ddev-drupal-simpler-solr ?
I think redirects should work fine, but I'd rather not have multiple redirects, so we'll just figure out one time what the change should be. ddev-drupal-simplest-solr ?

I'll wait for @mkalkbrenner too. I won't be surprised if he suggests that ddev-solr is simpler, which seems to be the case since it has the possibility to auto-update the config judging by the README! So lets wait for him.
drupal-simple is simple, simplest might have people expecting they don't have anything to do, they'll still have to set the core etc.
The files in the addon are already ok from what I can tell. So, people using ddev get ddev/ddev-drupal9-solr would simply continue with ddev get ddev/ddev-drupal-solr (or what we name it).
In reality, it's just that 9 that confuses people.

No, let's not wait for him to chime in, he's busy and doesn't care for this one anyway :) ddev-simple-solr is OK with me if you like it.

In reality, it's just that 9 that confuses people.

The thing is, the other one is the one preferred for search_api_solr, and thus for Drupal. So naming this ddev-drupal-solr seems to defeat so many things.
How about renaming to ddev-drupal-solr and adding output in the install that mentions ddev-solr, and of course making strong recommendation in the README?

Thanks for asking me.
"my" version also covers solarium in the README. I think Solarium is used in 99% of the use-cases. But maybe not directly but hidden because of integrations like the TYPO3 or Wordpress plugins or the bundles for symfony or laravel. The truth is that any of these could directly use Solr in Cloud mode without any problem as the way of the connection is backward compatible. If they connect the classic way, they simply can't use the advanced cloud features.
I know many of the Solr hosting service providers. Especially those who are not focused on Drupal use Solr Cloud as backend.
So I think there should be only one Solr integration for Dev based on Solr Cloud. BTW, the current one starts only one node as this is sufficient for local development in most cases. So it doesn't require much more resources compared to a Solr instance running in classic mode.
The only real differences between cloud and classic are the way a config-set gets deployed and how a new "core" gets created. And that's the only real issue because the cloud mode is not covered in the READMEs of most frameworks.
So if we could offer a generic PHP script in the Solarium project and a ddev command to call it to deploy a config-set and to create a core leveraging the cloud APIs in the background, we'll have the simplest Solar integration for ddev that is possible. In other words, the task is to have a pure PHP equivalent to the existing Drupal Drush commands for deployment and to only offer Solr Cloud mode.

@mkalkbrenner would appreciate if you could also comment on

https://github.com/ddev/ddev/issues/5860

I'm sure you have dealt with many people's resistance, but it does seem to continue. I have hoped for your ddev-solr to work for everybody as well.

Let's move forward on this @bserem , thanks.

I made a PR and changed the description of the project to mention ddev-solr, so that it gets visible into the ddev get --list table.
The next step would be to rename the project @rfay
In the future we could consider deprecating this entirely, but for now I feel it's ok to point people to the better alternative and let them decide.

Project is renamed.

In the future we could consider deprecating this entirely, but for now I feel it's ok to point people to the better alternative and let them decide.

Consider stronger language when mentioning ddev-solr then, saying that "ddev-drupal-solr may be deprecated in the future" or something.

This was completed. I think there's probably still some cleanup to be done @bserem , thanks!

authors: bserem mkalkbrenner rfay


text:
Problem on MacOs with base64 option

Hi,
I've tried a
ddev get ddev/ddev-platformsh
And got on Macos
base64: illegal option -- w
That stops the installation.
Could you help me ?

Sorry, duplicate of https://github.com/ddev/ddev-platformsh/issues/93

repo: ddev/ddev-platformsh


text:
Implement automated tests using key platform templates

It is time to return to the automated tests, preferably trying to do something live.

Platform templates are at https://github.com/platformsh-templates
Spreadsheet showing usage - with PHP highlighted.

Here are Platform's template priorities

Another source for prioritization is https://docs.google.com/document/d/1qoGitKdzUCu89Ceb_5U-X8JhZBmOIf_lgNfHeDqflEI/edit
Tech Stacks:
Drupal (CMS)
Wordpress (CMS)
Magento (CMS)
Laravel (Framework)
Symfony (Framework)

@gilzow for our strategies coming up, could you suggest what the best way to work through maybe 4 of them could be?
It looks to me like we have decent tests for Drupal and wordpress-bedrock. The WP Bedrock has a database, so better tests, I'll add a database to Drupal 9.
We have a test for laravel, which means that it seemed to work when I tried it, but it doesn't have a database.
Does that mean that magento2ce and Symfony would be our next targets?

Some others we can/should test:
Pimcore
Typo3
As for priorities, I think you're right: symfony and magento should be next.

Sounds good. I'm scared of pimcore, as the only round I've had with it had all kinds of crazy stuff in it. TYPO3 should work out fine.

So priorities for coming weeks:

Magento2ce (may not be trivial)
Symfony (where is the unofficial template?)
TYPO3
Pimcore

The official template(s) for symfony is here: https://github.com/symfonycorp/platformsh-symfony-template
And then from the branches choose the one you want/need

I've been told we don't need to worry about Symfony as their cli ships with its own local dev solution. But I need to make sure @lolautruche agrees with this.

It is true that Symfony CLI provides an efficient local webserver and good support of Docker for services. Nevertheless, I think we should not neglect it

for Django template testing: https://github.com/wsvincent/djangox

The good news is that we already have testing for laravel, which is @chadwcarlson 's highest priority.

authors: gilzow lolautruche rfay


text:
mkcert not found on Arch

Is there an existing issue for this?

[X] I have searched the existing issues

Output of ddev debug test
Expand `ddev debug test` diagnostic information
[COPY-PASTE HERE THE OUTPUT OF `ddev debug test`]

Expected Behavior
The install guide for arch linux can be followed as-is.
Actual Behavior
The guide tells the user that they should run mkcert -install as a one-time initialization when installing on arch linux. The guide never tells the user to explicitly install mkcert and the ddev-bin package doesn't include mkcert as dependency:
paru -Si ddev-bin
Repository      : aur
Name            : ddev-bin
Version         : 1.21.6-1
Description     : DDEV: a local web development environment
URL             : https://github.com/ddev/ddev
AUR URL         : https://aur.archlinux.org/packages/ddev-bin
Groups          : None
Licenses        : Apache 2
Provides        : ddev
Depends On      : docker
Make Deps       : None
Check Deps      : None
Optional Deps   : bash-completion
Conflicts With  : ddev
Maintainer      : cweagans
Votes           : 11
Popularity      : 0.868024
First Submitted : Mon, 30 Apr 2018 19:42:17 +0000
Last Modified   : Fri, 10 Mar 2023 19:37:59 +0000
Out Of Date     : No

Steps To Reproduce
On a arch install without mkcert follow the install guide:

paru/yay/whatever ddev-bin
mkcert -install

Anything else?
No response

A docs PR would be welcome. You can just click the pencil icon in the upper right of https://ddev.readthedocs.io/en/latest/users/install/ddev-installation/ - or do a PR to add mkcert as a dependency of ddev-bin. Maybe the dependency got lost a year or two ago when we started using goreleaser for arch packages. https://github.com/ddev/ddev/blob/cd8d78a5d6244f4029774f5e134c4ef9cb64b6c4/.goreleaser.yml#L280-L297

repo: ddev/ddev


text:
/api/token/validate requires AllowAnonymous attribute

I may be missing something,
But I tried calling /api/token/generate
then taking that value to POST /api/token/validate
and I always get a 401.
Once I add [AllowAnonymous ] to ValidateToken, then I'll get a 200 and a result of
{
    "isValid": true
}

I've tried adding the token in the header as a Bearer token but it still fails to authenticate.
Ideas?

To authorize with generated token you need to send it with headers when calling a method.

You may insert token in swagger's authorize parameters: Authorize button -> Value input. Swagger will place token to headers.
Format:
bearer {your_token}

You may add token to headers when you make a http api calls from external app.
Format:
authorization: bearer {your_token}

Adding AllowAnonymous to ValidateToken disables authorization for method.
The ideas was to make token validation only when you was authorized, that's why ValidateToken method does not have AllowAnonymous attribute and need authorization like others methods.
But if you don't need this function (validate token) you may delete this method in api.
Thank you.

authors: bcbeatty ddiyteam


text:
Voice Recognition Service

Hello, I share this workaround for certain devices, in my case it happened with a Samnsung J1 Mini Prime.
It happened that the user had not selected a voice service and the recognition does not work, nor did he warn of such an error, he only listened and never finished.
Option 1 : RNAndroidSpeechRecognizerModule.java line 83
@ReactMethod
  public void isRecognitionAvailable(final Promise promise) {
    boolean available = SpeechRecognizer.isRecognitionAvailable(this.reactContext);
    Log.d(TAG, "isRecognitionAvailable: " + available);
    String s = android.provider.Settings.Secure.getString(reactContext.getContentResolver(), "voice_recognition_service");

    if(android.text.TextUtils.isEmpty(s))
      available=false;

    promise.resolve(available);
  }

In this way, notify the user that there is a lack of configuration on your device. But to not be indicating to this that you should install opt for this other solution
Option 2 : RNAndroidSpeechRecognizerModule.java line 110
private void doCreateSpeechRecognizer(final Promise promise) {
    String s = android.provider.Settings.Secure.getString(reactContext.getContentResolver(), "voice_recognition_service");

    if(android.text.TextUtils.isEmpty(s))
      this.speechRecognizer = SpeechRecognizer.createSpeechRecognizer(this.reactContext,ComponentName.unflattenFromString("com.google.android.googlequicksearchbox/com.google.android.voicesearch.serviceapi.GoogleRecognitionService"));
    else
      this.speechRecognizer = SpeechRecognizer.createSpeechRecognizer(this.reactContext);

    this.speechRecognizer.setRecognitionListener(new ListenerMapRecognitionListener(
      this.enabledEvents,
      this.reactContext.getJSModule(RCTNativeAppEventEmitter.class),
      this.eventPrefix
    ));
    promise.resolve(null);
  }

I hope this helps someone.
Regards

Thank you for that. Sounds like a bug in the Android SDK. Do you have an idea why the settings may not have been present and are you aware of other users having encountered the same?

The issue was very rare, SpeechRecognizer started but did not emit the corresponding error so it was frozen.
In my case I could try only with a Samsung Galaxy J1 Mini. I followed all the tutorials to verify that everything was correctly installed but still I could not fix it.
Then I found this fix on the internet that served me and it would be easier than walking configuring users.
regards


text:
Add collector for core RabbitMQ metrics

This was done by @dcorbacho in https://github.com/dcorbacho/prometheus_rabbitmq_exporter/commit/db4762d1b23070df83ca8a5dc850eff359f46798, I'm just submitting the PR
This is a follow-up from https://github.com/deadtrickster/prometheus_rabbitmq_exporter/issues/63

This collector puts the minimum amount of pressure on RabbitMQ since the
metrics are not aggregated. Metrics are read from ETS tables, they are
strictly node local, and the amount of data is minimal. We wouldn't even
need rabbitmq_management_agent for the majority of core metrics, meaning
that prometheus_rabbitmq_exporter wouldn't need to depend on
rabbitmq_management if it was configured to use just this collector.
The other benefit of core metrics is that they are constantly updated,
since every object writes metrics on its own interval (typically every
5s). When /api/metrics is requested, raw values from ETS are read &
returned - no aggregation, no delay. What you get for core metrics is
what happened at most 5s ago.
Our end-goal is to expose raw, per-node metrics. Cluster metrics are
just an aggregation of per-node metrics, which should be done in
Prometheus/Grafana, just-in-time. This would help visualise cluster
imbalances, such as a subset of nodes running hotter than others, as
well as off-load all metrics processing so that RabbitMQ can spend as
many reductions on message-related work. We are imagining the equivalent
of Linux's /proc subsystem, but for RabbitMQ + Erlang + Mnesia.
I would be very excited to enable a mode for this plugin that only
exposes node-local metrics and doesn't require rabbitmq_management. Is
that something that you would like to have a go at @deadtricker?
For completeness, I'm adding the metrics that this collector adds to the
/api/metrics output of a freshly booted node:
# TYPE rabbitmq_core_connection_recv_oct counter
# HELP rabbitmq_core_connection_recv_oct Count of octects received on the connection.
# TYPE rabbitmq_core_connection_send_oct counter
# HELP rabbitmq_core_connection_send_oct Count of octects sent on the connection.
# TYPE rabbitmq_core_connection_reductions counter
# HELP rabbitmq_core_connection_reductions Count of reductions that take place on the queue process.
# TYPE rabbitmq_core_queue_messages_ready gauge
# HELP rabbitmq_core_queue_messages_ready Number of messages ready to be delivered to clients.
# TYPE rabbitmq_core_queue_messages_unacknowledge gauge
# HELP rabbitmq_core_queue_messages_unacknowledge Number of messages delivered to clients but not yet acknowledged.
# TYPE rabbitmq_core_queue_messages gauge
# HELP rabbitmq_core_queue_messages Sum of ready and unacknowledged messages (queue depth).
# TYPE rabbitmq_core_queue_reductions counter
# HELP rabbitmq_core_queue_reductions Count of reductions that take place on the queue process.
# TYPE rabbitmq_core_channel_exchange_publish gauge
# HELP rabbitmq_core_channel_exchange_publish Count of messages published.
# TYPE rabbitmq_core_channel_exchange_confirm gauge
# HELP rabbitmq_core_channel_exchange_confirm Count of messages confirmed.
# TYPE rabbitmq_core_channel_exchange_return_unroutable gauge
# HELP rabbitmq_core_channel_exchange_return_unroutable Count of messages returned to publisher as unroutable.
# TYPE rabbitmq_core_channel_process_reductions counter
# HELP rabbitmq_core_channel_process_reductions Count of reductions that take place on the channel process.
# TYPE rabbitmq_core_queue_disk_reads gauge
# HELP rabbitmq_core_queue_disk_reads Total number of times messages have been read from disk by this queue.
# TYPE rabbitmq_core_queue_disk_writes gauge
# HELP rabbitmq_core_queue_disk_writes Total number of times messages have been written to disk by this queue.
# TYPE rabbitmq_core_node_io_read_count counter
# HELP rabbitmq_core_node_io_read_count Read operations since node start.
rabbitmq_core_node_io_read_count{node="rabbit@08c5bed93480"} 1
# TYPE rabbitmq_core_node_io_read_bytes gauge
# HELP rabbitmq_core_node_io_read_bytes Bytes read since node start.
rabbitmq_core_node_io_read_bytes{node="rabbit@08c5bed93480"} 1
# TYPE rabbitmq_core_node_io_read_time gauge
# HELP rabbitmq_core_node_io_read_time Total time of read operations.
rabbitmq_core_node_io_read_time{node="rabbit@08c5bed93480"} 29
# TYPE rabbitmq_core_node_io_write_count counter
# HELP rabbitmq_core_node_io_write_count Write operations since node start.
rabbitmq_core_node_io_write_count{node="rabbit@08c5bed93480"} 0
# TYPE rabbitmq_core_node_io_write_bytes gauge
# HELP rabbitmq_core_node_io_write_bytes Bytes written since node start.
rabbitmq_core_node_io_write_bytes{node="rabbit@08c5bed93480"} 0
# TYPE rabbitmq_core_node_io_write_time gauge
# HELP rabbitmq_core_node_io_write_time Total time of write operations.
rabbitmq_core_node_io_write_time{node="rabbit@08c5bed93480"} 0
# TYPE rabbitmq_core_node_io_sync_count counter
# HELP rabbitmq_core_node_io_sync_count Sync operations since node start.
rabbitmq_core_node_io_sync_count{node="rabbit@08c5bed93480"} 0
# TYPE rabbitmq_core_node_io_sync_time gauge
# HELP rabbitmq_core_node_io_sync_time Total time of sync operations.
rabbitmq_core_node_io_sync_time{node="rabbit@08c5bed93480"} 0
# TYPE rabbitmq_core_node_io_seek_count counter
# HELP rabbitmq_core_node_io_seek_count Seek operations since node start.
rabbitmq_core_node_io_seek_count{node="rabbit@08c5bed93480"} 0
# TYPE rabbitmq_core_node_io_seek_time gauge
# HELP rabbitmq_core_node_io_seek_time Total time of seek operations.
rabbitmq_core_node_io_seek_time{node="rabbit@08c5bed93480"} 0
# TYPE rabbitmq_core_node_io_reopen_count counter
# HELP rabbitmq_core_node_io_reopen_count Times files have been reopened by the file handle cache.
rabbitmq_core_node_io_reopen_count{node="rabbit@08c5bed93480"} 0
# TYPE rabbitmq_core_node_mnesia_ram_tx_count counter
# HELP rabbitmq_core_node_mnesia_ram_tx_count Mnesia transactions in RAM since node start.
rabbitmq_core_node_mnesia_ram_tx_count{node="rabbit@08c5bed93480"} 11
# TYPE rabbitmq_core_node_mnesia_disk_tx_count counter
# HELP rabbitmq_core_node_mnesia_disk_tx_count Mnesia transactions in disk since node start.
rabbitmq_core_node_mnesia_disk_tx_count{node="rabbit@08c5bed93480"} 12
# TYPE rabbitmq_core_node_msg_store_read_count counter
# HELP rabbitmq_core_node_msg_store_read_count Read operations in the message store since node start.
rabbitmq_core_node_msg_store_read_count{node="rabbit@08c5bed93480"} 0
# TYPE rabbitmq_core_node_msg_store_write_count counter
# HELP rabbitmq_core_node_msg_store_write_count Write operations in the message store since node start.
rabbitmq_core_node_msg_store_write_count{node="rabbit@08c5bed93480"} 0
# TYPE rabbitmq_core_queue_index_journal_write_count counter
# HELP rabbitmq_core_queue_index_journal_write_count Write operations in the queue index journal since node start.
rabbitmq_core_queue_index_journal_write_count{node="rabbit@08c5bed93480"} 0
# TYPE rabbitmq_core_queue_index_write_count counter
# HELP rabbitmq_core_queue_index_write_count Queue index write operations since node start.
# TYPE rabbitmq_core_queue_index_read_count counter
# HELP rabbitmq_core_queue_index_read_count Queue index read operations since node start.
# TYPE rabbitmq_core_queue_io_file_handle_open_attempt_count counter
# HELP rabbitmq_core_queue_io_file_handle_open_attempt_count File descriptor open attempts.
rabbitmq_core_queue_io_file_handle_open_attempt_count{node="rabbit@08c5bed93480"} 10
# TYPE rabbitmq_core_queue_io_file_handle_open_attempt_time gauge
# HELP rabbitmq_core_queue_io_file_handle_open_attempt_time Total time of file descriptor open attempts.
rabbitmq_core_queue_io_file_handle_open_attempt_time{node="rabbit@08c5bed93480"} 264
# TYPE rabbitmq_core_node_fd_used gauge
# HELP rabbitmq_core_node_fd_used File descriptors used.
rabbitmq_core_node_fd_used{node="rabbit@08c5bed93480"} 81
# TYPE rabbitmq_core_node_sockets_used gauge
# HELP rabbitmq_core_node_sockets_used Sockets used.
rabbitmq_core_node_sockets_used{node="rabbit@08c5bed93480"} 0
# TYPE rabbitmq_core_node_mem_used gauge
# HELP rabbitmq_core_node_mem_used Memory used in bytes.
rabbitmq_core_node_mem_used{node="rabbit@08c5bed93480"} 97021952
# TYPE rabbitmq_core_node_disk_free gauge
# HELP rabbitmq_core_node_disk_free Disk free in bytes.
rabbitmq_core_node_disk_free{node="rabbit@08c5bed93480"} 55178477568
# TYPE rabbitmq_core_node_proc_used gauge
# HELP rabbitmq_core_node_proc_used Erlang processes used.
rabbitmq_core_node_proc_used{node="rabbit@08c5bed93480"} 399
# TYPE rabbitmq_core_node_gc_num counter
# HELP rabbitmq_core_node_gc_num GC runs.
rabbitmq_core_node_gc_num{node="rabbit@08c5bed93480"} 5141
# TYPE rabbitmq_core_node_gc_bytes_reclaimed counter
# HELP rabbitmq_core_node_gc_bytes_reclaimed Bytes reclaimed by GC.
rabbitmq_core_node_gc_bytes_reclaimed{node="rabbit@08c5bed93480"} 379204736
# TYPE rabbitmq_core_node_context_switches counter
# HELP rabbitmq_core_node_context_switches Context switches since node start.
rabbitmq_core_node_context_switches{node="rabbit@08c5bed93480"} 190670
# TYPE rabbitmq_core_connection_created counter
# HELP rabbitmq_core_connection_created Connections created.
rabbitmq_core_connection_created{node="rabbit@08c5bed93480"} 0
# TYPE rabbitmq_core_connection_closed counter
# HELP rabbitmq_core_connection_closed Connections closed.
rabbitmq_core_connection_closed{node="rabbit@08c5bed93480"} 0
# TYPE rabbitmq_core_channel_created counter
# HELP rabbitmq_core_channel_created Channels created.
rabbitmq_core_channel_created{node="rabbit@08c5bed93480"} 0
# TYPE rabbitmq_core_channel_closed counter
# HELP rabbitmq_core_channel_closed Channels closed.
rabbitmq_core_channel_closed{node="rabbit@08c5bed93480"} 0
# TYPE rabbitmq_core_queue_declared counter
# HELP rabbitmq_core_queue_declared Queues declared.
rabbitmq_core_queue_declared{node="rabbit@08c5bed93480"} 0
# TYPE rabbitmq_core_queue_deleted counter
# HELP rabbitmq_core_queue_deleted Queues deleted.
rabbitmq_core_queue_deleted{node="rabbit@08c5bed93480"} 0
# TYPE rabbitmq_core_node_node_send_bytes counter
# HELP rabbitmq_core_node_node_send_bytes Count of bytes sent to node.
# TYPE rabbitmq_core_node_node_recv_bytes counter
# HELP rabbitmq_core_node_node_recv_bytes Count of bytes received from node.
# TYPE rabbitmq_core_channel_queue_get counter
# HELP rabbitmq_core_channel_queue_get Count of messages delivered in acknowledgement mode in response to basic.get.
# TYPE rabbitmq_core_channel_queue_get_no_ack counter
# HELP rabbitmq_core_channel_queue_get_no_ack Count of messages delivered in no-acknowledgement mode in response to basic.get.
# TYPE rabbitmq_core_channel_queue_deliver counter
# HELP rabbitmq_core_channel_queue_deliver Count of messages delivered in acknowledgement mode to consumers.
# TYPE rabbitmq_core_channel_queue_deliver_no_ack counter
# HELP rabbitmq_core_channel_queue_deliver_no_ack Count of messages delivered in no-acknowledgement mode to consumers.
# TYPE rabbitmq_core_channel_queue_redeliver counter
# HELP rabbitmq_core_channel_queue_redeliver Count of subset of delivered messages which had the redelivered flag set.
# TYPE rabbitmq_core_channel_queue_ack counter
# HELP rabbitmq_core_channel_queue_ack Count of messages acknowledged.
# TYPE rabbitmq_core_channel_queue_get_empty counter
# HELP rabbitmq_core_channel_queue_get_empty Count of basic.get operations on empty queues.
# TYPE rabbitmq_core_channel_consumer_count gauge
# HELP rabbitmq_core_channel_consumer_count Consumers count.
# TYPE rabbitmq_core_channel_messages_unacknowledged gauge
# HELP rabbitmq_core_channel_messages_unacknowledged Count of messages unacknowledged.
# TYPE rabbitmq_core_channel_messages_unconfirmed gauge
# HELP rabbitmq_core_channel_messages_unconfirmed Count of messages unconfirmed.
# TYPE rabbitmq_core_channel_messages_uncommited gauge
# HELP rabbitmq_core_channel_messages_uncommited Count of messages uncommited.
# TYPE rabbitmq_core_channel_messages_prefetch_count gauge
# HELP rabbitmq_core_channel_messages_prefetch_count Limit to the number of unacknowledged messages on every connection on a channel.
# TYPE rabbitmq_core_channel_messages_global_prefetch_count gauge
# HELP rabbitmq_core_channel_messages_global_prefetch_count Global limit to the number of unacknowledged messages shared between all connections on a channel.
# TYPE rabbitmq_core_connection_recv_count counter
# HELP rabbitmq_core_connection_recv_count Count of bytes received on the connection.
# TYPE rabbitmq_core_connection_send_count counter
# HELP rabbitmq_core_connection_send_count Count of bytes send on the connection.
# TYPE rabbitmq_core_node_fd_total gauge
# HELP rabbitmq_core_node_fd_total File descriptors available.
rabbitmq_core_node_fd_total{node="rabbit@08c5bed93480"} 1048576
# TYPE rabbitmq_core_node_sockets_total gauge
# HELP rabbitmq_core_node_sockets_total Sockets available.
rabbitmq_core_node_sockets_total{node="rabbit@08c5bed93480"} 943626
# TYPE rabbitmq_core_node_mem_limit gauge
# HELP rabbitmq_core_node_mem_limit Memory usage high watermark.
rabbitmq_core_node_mem_limit{node="rabbit@08c5bed93480"} 13475676160
# TYPE rabbitmq_core_node_disk_free_limit gauge
# HELP rabbitmq_core_node_disk_free_limit Free disk space low watermark.
rabbitmq_core_node_disk_free_limit{node="rabbit@08c5bed93480"} 50000000
# TYPE rabbitmq_core_node_proc_total gauge
# HELP rabbitmq_core_node_proc_total Erlang processes limit.
rabbitmq_core_node_proc_total{node="rabbit@08c5bed93480"} 1048576
# TYPE rabbitmq_core_node_uptime gauge
# HELP rabbitmq_core_node_uptime Time in milliseconds since node start.
rabbitmq_core_node_uptime{node="rabbit@08c5bed93480"} 24942
# TYPE rabbitmq_core_node_run_queue gauge
# HELP rabbitmq_core_node_run_queue Runtime run queue.
rabbitmq_core_node_run_queue{node="rabbit@08c5bed93480"} 1
# TYPE rabbitmq_core_node_processors gauge
# HELP rabbitmq_core_node_processors Logical processors.
rabbitmq_core_node_processors{node="rabbit@08c5bed93480"} 10
# TYPE rabbitmq_core_node_net_ticktime gauge
# HELP rabbitmq_core_node_net_ticktime Periodic tick interval between all pairs of nodes to maintain the connections and to detect disconnections.
rabbitmq_core_node_net_ticktime{node="rabbit@08c5bed93480"} 60
# TYPE rabbitmq_core_channel_queue_exchange_publish counter
# HELP rabbitmq_core_channel_queue_exchange_publish Count of messages published.
# TYPE rabbitmq_core_connections gauge
# HELP rabbitmq_core_connections RabbitMQ Connections count.
rabbitmq_core_connections 0
# TYPE rabbitmq_core_channels gauge
# HELP rabbitmq_core_channels RabbitMQ Channels count.
rabbitmq_core_channels 0
# TYPE rabbitmq_core_consumers gauge
# HELP rabbitmq_core_consumers RabbitMQ Consumers count.
rabbitmq_core_consumers 0
# TYPE rabbitmq_core_queues gauge
# HELP rabbitmq_core_queues RabbitMQ Queues count.

Build passes, check out all these new & beautiful rabbitmq_core_* metrics: https://travis-ci.org/deadtrickster/prometheus_rabbitmq_exporter/builds/499414875#L917-L1118
I'm thinking that it's time to cut a new version after this is merged, would you agree @deadtrickster ?

Yeah, +1 for the new version. Actually at the beginning I was reading ETS tables directly too for some metrics. Was really easy to break. Hopefully I was doing different.

Will merge and release tomorrow with a fresh head.

I would be very excited to enable a mode for this plugin that only
exposes node-local metrics and doesn't require rabbitmq_management. Is
that something that you would like to have a go at @deadtricker?

Yeah, all parts are there. Perhaps a configurable switch? Also need to agree on http interface replacement. This plugin mounts itself inside management api space. But if that configurable switch is on, metrics can be exposed via raw prometheus_httpd using  the same default /api/metrics.

Merged, will release soon.

authors: deadtrickster gerhard


text:
Multi-linguals: Use the Command ID, not the Command Name (as shown in the Command Palette)

Thanks for another useful plugin @deathau—and even more thanks that you didn’t forget us multi-lingual people who switch the Obsidian language often, by allowing us to use the Command ID instead of its name!
I’ve missed that a lot in the Buttons and Obsidian Leaflet plugins: Whenever one switches Obsidian’s language, the command names used aren’t the same anymore and all logic breaks. Not so when using the Command IDs.
Since there is unfortunately no official list of command ids (and plugins can add to the commands anyway), you can use my Obsidian Command and Hotkey List (new version) to show command ids and their currently assigned hotkeys within Obsidian. This requires the Dataview plugin to be installed and operational.
Here’s the link to the code, in the Obsidian Forums:
https://forum.obsidian.md/t/dataviewjs-snippet-showcase/17847/37

I just ran into this problem as well...
Cheers

repo: deathau/command-uri-obsidian

license: Unlicense


text:
Update scalafmt-core to 3.7.11

About this PR
📦 Updates org.scalameta:scalafmt-core from 3.4.3 to 3.7.11
📜 GitHub Release Notes - Version Diff
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scalameta", artifactId = "scalafmt-core" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "org.scalameta", artifactId = "scalafmt-core" }
}]

labels: library-update, early-semver-minor, semver-spec-minor, commit-count:1

Superseded by #247.

repo: debasishg/tradeio


text:
DBZ-5565 Formatting characters no longer render in published doc

DBZ-5565
Fixes bad formatting of names and examples in several of the connector properties table entries.
Prior to the update, instead of the correct formatting being applied in the published documentation, formatting characters were rendered.
Tested in local Antora build.

@roldanbob Applied, thanks

authors: jpechane roldanbob

repo: debezium/debezium


text:
Pippo integration with Security framework

I wanted to know if there are plans to integrate any security framework with Pippo. Possible candidates could be:

pac4j
Apache Shiro

Or... any other framework

I don't plan to add new module in the near future in Pippo. I wish to consolidate, test and document what already exists. In the same time, I am happy to help new initiatives. So, if you are interested in a security framework please create a project, share this project with us on Github and I am ready to contribute. I am curios to see a security framework integrated in Pippo, not something complicated/big.

A security framework should resolve two problems:

authentication
authorization

I prefer something light, extensible, easy to integrate in Pippo (maybe not complete from the features point of view). In the end, Pippo is the land of choice for people that write small and medium application (from complexity point of view) so in this context I prefer libraries in same note.

I am toying with the idea of building an integration for pac4j. Seems like a popular choice by other micro frameworks like Ratpack or Jooby.
So I have a few questions:

Any thoughts or preferences about pac4j?
Would you be willing to add it as part of Pippo (as in a pippo-security-parent and pippo-pac4j?

@codematix

Any thoughts or preferences about pac4j?

I don't know to much about pac4j so I cannot tell you to much.

Would you be willing to add it as part of Pippo (as in a pippo-security-parent and pippo-pac4j)?

In the first phase I prefer a new repo, something like pippo-pac4j. A new module in Pippo means supplementary code to maintain and we should concentrate our efforts on core.
I am curios to know how other Pippo users resolve the security aspect in their applications. Maybe we can choose/adopt something that it's already integrated with Pippo.

In the first phase I prefer a new repo, something like pippo-pac4j. A new module in Pippo means supplementary code to maintain and we should concentrate our efforts on core.
I am curios to know how other Pippo users resolve the security aspect in their applications. Maybe we can choose/adopt something that it's already integrated with Pippo.

Yes it would be great to hear from other Pippo users as to how they have managed to implement the security aspect into their applications and also what their preference would be for a first-class security integration with Pippo.
About my experience, I have been using Spring Framework and the Spring Security modules for my applications. It has served us well in the past by integrating with most authenication/authroization systems.
We are trying to build some new products in our current project and as part of that, we are closely evaluating Pippo. We are very pleased with what we see in Pippo and want to see if we can use a off-the-shelf security framework like pac4j/Shiro can be integrated with Pippo.
I would request the Pippo community to voice out their preference/thoughts of a security framework integration with Pippo.

Maybe it's useful http://www.pac4j.org/docs/how-to-implement-pac4j-for-a-new-framework.html.

@decebals Yes that is what I am using to build the integration along with referring to some other integrations.
Please consider contributing to my attempt at this - pippo-pac4j

@decebals I am curios to know how other Pippo users resolve the security aspect in their applications. Maybe we can choose/adopt something that it's already integrated with Pippo.

Below my simple contribution.
Authentication controllers (www and web services) I did not put in because they had a lot of specific logic, but they followed the basic flow of validating username and password. The controller to "log in" to web services (ie getting JWT) validates username and password, and if valid returns the token (JWT). The next requests for web services must be made through the token (JWT).
AuthenticationFilter: check if the user is authenticated at each request (www) (screen requests);
WebServiceAuthenticationFilter: check if JSON Web Token (JWT) is valid for web services requests;
RequireRole: annotation for Pippo to intercept access to the method and check the authorization;
RoleHandler: business logic to check authorization;
CustomRouteContext: to force controller methods to define some of the authorization annotations;
public class PippoApplication extends ControllerApplication {
	protected void onInit() {
		ANY("/p/.*", new AuthenticationFilter()); // Check if the user is authenticated at each request (www)
		ANY("/ws/p/.*", new WebServiceAuthenticationFilter()); // Check if (JWT) JSON Web Token is valid for web services requests
	}
}

/////

/**
 * '/p/.*'
*/
public class AuthenticationFilter implements RouteHandler {

    @Override
    public void handle(RouteContext routeContext) {
        Pair<Boolean, String> loggedAndValidAndStatus = SecurityHelper.loggedAndValid(routeContext);
        boolean loggedAndValid = loggedAndValidAndStatus.getFirst();
        if (loggedAndValid) {
            routeContext.next();
        } else {
            String status = loggedAndValidAndStatus.getSecond();
            routeContext.setSession(Constantes.KEY_SESSION_ORIGINAL_DESTINATION, routeContext.getRequest().getApplicationUriWithQuery());
            routeContext.flashWarning(status);
            routeContext.redirect(Constantes.PATH_LOGIN);
        }
    }

}

/////

/**
 * '/ws/p/.*'
 * Filter for web services to check if the JSON Web Token of the requester is valid.
 * <p>
 * The token must be passed in the request header in the attribute {@link JsonWebTokenHelper#ATTR_JSON_WEB_TOKEN}.
 * </p>
 */
public class WebServiceAuthenticationFilter implements RouteHandler {

    private static final Logger log = LoggerFactory.getLogger(WebServiceAuthenticationFilter.class);

    @Override
    public void handle(RouteContext routeContext) {
        String jwtString = JsonWebTokenHelper.getJsonWebToken(routeContext);
        try {
            if (StringUtils.isBlank(jwtString)) {
                throw new MyException(JsonWebTokenHelper.ATTR_JSON_WEB_TOKEN + " attribute in header");
            }

            final String username;
            try {
                username = JsonWebTokenHelper.getUsername(jwtString);
            } catch (Exception e) {
                throw new MyException(JsonWebTokenHelper.ATTR_JSON_WEB_TOKEN + " invalid token");
            }

            if (StringUtils.isBlank(username)) {
                throw new MyException(JsonWebTokenHelper.ATTR_JSON_WEB_TOKEN + " user not defined in the token");
            }

            UserService userService = GuiceInjector.get().getInstance(UserService.class);
            User user = userService.getByUsername(username);
            if (user == null) {
                throw new MyException(JsonWebTokenHelper.ATTR_JSON_WEB_TOKEN + " no user for the informed token");
            }

            String jwtBase64Key = user.getJwtBase64Key();
            if (StringUtils.isBlank(jwtBase64Key)) {
                throw new MyException(JsonWebTokenHelper.ATTR_JSON_WEB_TOKEN + " need to sign in to get the token");
            }

            JsonWebTokenHelper.validateJWT(jwtString, jwtBase64Key);

            SecurityHelper.setLoggedUser(routeContext, user);

            routeContext.next();
        } catch (MalformedJwtException e) { // string is not a valid JWS
            throw new MyException(JsonWebTokenHelper.ATTR_JSON_WEB_TOKEN + " in invalid format.");
        } catch (SignatureException e) { // JWS signature validation fails
            throw new MyException(JsonWebTokenHelper.ATTR_JSON_WEB_TOKEN + " with invalid signature.");
        } catch (ExpiredJwtException e) { // Se o token estiver expirado
            throw new MyException(JsonWebTokenHelper.ATTR_JSON_WEB_TOKEN + " expired.");
        } catch (MyException e) {
            throw e;
        } catch (Exception e) { // Other
            throw new MyException(JsonWebTokenHelper.ATTR_JSON_WEB_TOKEN + " impossible to validate", e);
        } finally {
            if (routeContext.hasSession()) {
                routeContext.invalidateSession();
            }
        }
    }

/////

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
@Interceptor(RoleHandler.class)
public @interface RequireRole {

    String roleName() default "";

    String description() default "";

}

/////

public class RoleHandler implements RouteHandler {

    private static final Logger log = LoggerFactory.getLogger(RoleHandler.class);

    @Override
    public void handle(RouteContext routeContext) {
        log.trace("RoleHandler.handle");

        Method method = PippoJarvisUtils.getMethod(routeContext.getRoute());

        log.debug("Checking authorization for '{}'", LangUtils.toString(method));

        RequireRole requireRole = method.getAnnotation(RequireRole.class);

        final String roleName = requireRole.roleName();
        User user = SecurityHelper.getUsuarioLogado(routeContext);

        UserService userService = GuiceInjector.get().getInstance(UserService.class);

        boolean authorized = userService.authorize(user, roleName);
        if (!authorized) {
            throw new MyException("You are not authorized to take the action: " + roleName, "/p/home", true);
        }

        // here the user can access the method
    }

}

/////

public class CustomRouteContext extends DefaultRouteContext {

    protected CustomRouteContext(Application application, Request request, Response response, List<RouteMatch> routeMatches) {
        super(application, request, response, routeMatches);
    }

    // Requires that all controller methods have at least one of the authorization annotations
    // TODO: The annoying thing to intercept here is that here are not only the routes of Controllers, but also of Filters and other
    @Override
    protected void handleRoute(Route route) {
        if (ControllerRouteHandler.class.isAssignableFrom(route.getRouteHandler().getClass())) {
            Method method = PippoJarvisUtils.getMethod(route);
            if (method.isAnnotationPresent(Public.class) || method.isAnnotationPresent(RequireRole.class)) {
                super.handleRoute(route);
            } else {
                throw new MyException("Need to apply security policy on " + LangUtils.toString(method) + ".", "/p/home", true);
            }
        } else {
            super.handleRoute(route);
        }
    }

}

/////

@AutoService(RouteContextFactory.class)
public class CustomRouteContextFactory extends DefaultRouteContextFactory {

    @Override
    public CustomRouteContext createRouteContext(Application application, Request request, Response response, List<RouteMatch> routeMatches) {
        return new CustomRouteContext(application, request, response, routeMatches);
    }

}

/////

public final class SecurityHelper {

    public static Pair<Boolean, String> loggedAndValid(RouteContext routeContext) {
        if (SecurityHelper.userLoggedIn(routeContext)) {
            String clientIpSession = getIpUsuarioLogado(routeContext);
            String clientIp = routeContext.getRequest().getClientIp();
            if (clientIp.equals(clientIpSession)) { // If IP has not changed
                return new Pair<Boolean, String>(true, "Authenticated and valid user.");
            } else { // If IP has changed, re-login
                return new Pair<Boolean, String>(false, "Your IP has changed. Authentication required again.");
            }
        } else { // If unauthenticated user
            return new Pair<Boolean, String>(false, "Authentication required.");
        }
    }

}

@mhagnumdw Your code looks good. The code is easy to understand and follow.
The part with CustomRouteContext is a little bit tricky. For example I don't understand the utility of this class.
Take an example:
if (method.isAnnotationPresent(Public.class) || method.isAnnotationPresent(RequireRole.class)) {
    ...
}

I see here a simplification, if the web method  is not annotated with @RequireRole then, by default, the web method is public (no security constraint). Again, I feel that CustomRouteContext is not needed.

if (method.isAnnotationPresent(Public.class) || method.isAnnotationPresent(RequireRole.class)) {
...
}
I see here a simplification, if the web method is not annotated with @RequireRole then, by default, the web method is public (no security constraint). Again, I feel that CustomRouteContext is not needed.

@decebals You're right. I'm in the process of migrating from Pippo 1.1.0 to 1.5.0 and this is a remnant code.

@codematix Thanks for your contribution!

authors: codematix decebals mhagnumdw

repo: decebals/pippo


text:
feat: Add support to urns in the item upsert

Add support for URNs when building the item.

Pull Request Test Coverage Report for Build 1740840283

1 of 1 (100.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.4%) to 86.837%

Totals

Change from base Build 1719428293:
-0.4%

Covered Lines:
281

Relevant Lines:
314

💛  - Coveralls

authors: LautaroPetaccio coveralls

repo: decentraland/builder-client


text:
Wrong message when no schedule listed

Fixed in another issue

repo: decentraland/events


text:
[QA] Plaza Functionality Disrupted After Returning from Distant Location

Explorer Alpha build version:
0.3.402-g7a23dc7e
Issue Description:
Upon returning to Genesis Plaza after jumping to any distant location, several functionalities become disrupted. The carousels (Best Rated, Events, Crowd) appear empty, there is no music playing, and the portal does not emit any sound. However, after visiting the Metadynelabs world and returning, Genesis Plaza starts working again.
Steps to reproduce:

Open Genesis Plaza.
Jump to any other location far away.
Ensure no disruptions occur.
Return to Genesis Plaza.
Jump to Metadynelabs.
Return to Genesis Plaza.
Verify that functionalities are restored.

Expected behavior:
Upon returning to Genesis Plaza from a distant location, all functionalities such as carousel content, music playback, and portal sound should resume as expected.
Current behavior:
After returning to Genesis Plaza from a distant location, the carousels are empty, there is no music playing, and the portal does not emit any sound.
https://github.com/decentraland/unity-explorer/assets/146741383/af80a807-9fc7-4c6f-a662-844be6692574
Platforms / OS:

[❌ ] Windows 11
[ ] Windows 10
[ ] Mac Sonoma
[ ] Mac (specify the version on Additional Notes)

Additional Notes:
Windows 11. 12th Gen Intel(R) Core(TM) i7-12650H   2.30 GHz

@Ludmilafantaniella can you validate this again? I am sure this is fixed.

repo: decentraland/unity-explorer


text:
[QA] Minigame | Cells Openable During Countdown in Memory Circus

Build version:

v0.40.0

Issue Description:
In the Memory Circus minigame (location: 50,-88), players are able to open the cells while the countdown is active. This allows interaction with the cells before the game officially starts, which affects the intended flow of gameplay. Cells should remain locked until the countdown finishes and the game begins.
STR:

Go to the Memory Circus minigame (50,-88)
Start the game and observe the countdown timer.
Attempt to open the cells during the countdown.

❌ Actual Result:
Players can open cells while the countdown is still in progress.
https://github.com/user-attachments/assets/03202355-e523-4c1a-bb32-e69de410de34
✔ Expected Result:
Cells should be locked and unclickable until the countdown ends and the game officially begins.
Repro Index:
10/10
Additional Notes:
Windows 11. 12th Gen Intel(R) Core(TM) i7-12650H 2.30 GHz. nvidia geforce RTX
Macbook M1 Pro Sonoma 14.5

I was not able to see any timer when being on memory circus while trying to repro this one on latest dev but I was able to see this issue with or without timer.
https://github.com/user-attachments/assets/6bda384c-29d4-4697-9377-7e0edf9439a0

authors: DafGreco Ludmilafantaniella


text:
Import and export

[ ] Ability to import CSVs
[ ] Ability to export CSVs
[ ] Ability to import JSONs
[ ] Ability to export JSONs

json exporting/importing is kinda done given that that's the main format of the api, so I'm gonna check those boxes. If you had a different idea for what this meant, though, lmk.

Going to also close this, as I don't see a way to generically support CSVs without a complicated GUI or interactive cli program. I think the better path is to just educate people how to post to the db and help them write their own csv parser. Very few projects have legacy CSVs anyway, so this isn't something I see much value in spending a lot of time on, which I feel it would take to do well

authors: EllipticEllipsis ethteck

repo: decompals/frogress


text:
Use alternate icon for mainnet build

get the latest full color logo from https://decred.org/brand/

@raedah I updated the icons for App Store. Can I share them here?

authors: denyszayets raedah

repo: decred/dcrandroid

license: ISC


text:
🛑 Studio Legale Fabiola Tombolini is down

In 6db070b, Studio Legale Fabiola Tombolini (https://fabiolatombolini.law) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Studio Legale Fabiola Tombolini is back up in 46f6116 after 18 minutes.

repo: dedotombo/upptime


text:
Cannot run Network Architecture Search, ends up in error

Unable to run notebooks 2,3 which have network architecture search.
For instance, for notebook 3, on Deephyper version 0.4.2, it ends up with error:
/usr/local/lib/python3.9/dist-packages/deephyper/evaluator/_evaluator.py:101: UserWarning: Applying nest-asyncio patch for IPython Shell!
  warnings.warn(
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
[<ipython-input-17-10be92266985>](https://localhost:8080/#) in <module>
      3
      4
----> 5 agebo_search = AgEBO(
      6     problem,
      7     run_base_trainer,

2 frames
[/usr/local/lib/python3.9/dist-packages/deephyper/skopt/learning/forest.py](https://localhost:8080/#) in __init__(self, n_estimators, criterion, max_depth, min_samples_split, min_samples_leaf, min_weight_fraction_leaf, max_features, max_leaf_nodes, min_impurity_decrease, bootstrap, oob_score, n_jobs, random_state, verbose, warm_start, ccp_alpha, max_samples, min_variance, splitter)
    232         splitter="random"
    233     ):
--> 234         super().__init__(
    235             base_estimator=DecisionTreeRegressor(splitter=splitter),
    236             n_estimators=n_estimators,

TypeError: __init__() missing 1 required positional argument: 'estimator'

for notebook 3, on Deephyper version 0.5.0, it ends up with error:
/usr/local/lib/python3.9/dist-packages/deephyper/evaluator/_evaluator.py:126: UserWarning: Applying nest-asyncio patch for IPython Shell!
  warnings.warn(
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[<ipython-input-17-10be92266985>](https://localhost:8080/#) in <module>
     10     random_state=42,
     11 )
---> 12 results = agebo_search.search(max_evals=30)

11 frames
[/usr/local/lib/python3.9/dist-packages/keras/optimizers/optimizer_experimental/optimizer.py](https://localhost:8080/#) in tf___update_step_xla(self, gradient, variable, key)
     26                 try:
     27                     do_return = True
---> 28                     retval_ = ag__.converted_call(ag__.ld(self)._update_step, (ag__.ld(gradient), ag__.ld(variable)), None, fscope)
     29                 except:
     30                     do_return = False

ValueError: in user code:

    File "/usr/local/lib/python3.9/dist-packages/keras/engine/training.py", line 1249, in train_function  *
        return step_function(self, iterator)
    File "/usr/local/lib/python3.9/dist-packages/keras/optimizers/optimizer_experimental/optimizer.py", line 208, in _update_step_xla  *
        return self._update_step(gradient, variable)
    File "/usr/local/lib/python3.9/dist-packages/keras/optimizers/optimizer_experimental/optimizer.py", line 224, in _update_step  **
        self.update_step(gradient, variable)
    File "/usr/local/lib/python3.9/dist-packages/keras/optimizers/optimizer_experimental/adam.py", line 200, in update_step
        variable.assign_sub((m * alpha) / (tf.sqrt(v) + self.epsilon))

    ValueError: None values not supported.

Found similar issue: https://github.com/deephyper/deephyper/issues/172
In short adding the epsilon=1e-7 to the problem.hyperparameters fixed the issue.


text:
feat: upgrade

upgrade from 1.7.0-8 to 1.7.2-3

/check obs

/approve

authors: Gui-Yue Zeno-sole hudeng-go

repo: deepin-community/apr


text:
Implement TdLambdaReturns for alpha_zero_torch

I've been using this locally while hacking with backgammon.  Lemme know if it's worth merging into the repo.  Logs showing the state values of a trajectory and the training values returned from TdLambdaReturns for different settings are pasted below.
Sorry this is such a large change - I can break it up if needed for review:

Changed the value stored in a Trajectory::State to match the MCTS value of the chosen action.  (Note TD(0) would train to this value)  Far as I could tell, Trajectory::State::value was only being used for outcome prediction/accuracy, and this usage remains valid after the change.
Set missing config settings by merging with defaults, so that saved config files continue to work when new settings are added.
Added a min_simulations setting to allow the MCTS value of a forced action more sims.  This required updates to all 3 implementations of alpha_zero, as well as pybind + julia.
Added config settings td_lambda and td_n_steps
Added a (default false) verbose setting to learner() for logging how TrainInputs are populated from Trajectory.
Considered adding TdLambdaReturns in a new file under algorithms, but it would have required some refactoring to make Trajectory visible outside of the AlphaZero namespace, so... left for possibly later.

Examples from tic_tac_toe for various settings:
td_lambda: 0.0
td_n_steps: 0
  StateIdx: 0  Value: 0.408  TrainTo: 0.408
  StateIdx: 1  Value: 0.312  TrainTo: 0.312
  StateIdx: 2  Value: 0.152  TrainTo: 0.152
  StateIdx: 3  Value: -0.430  TrainTo: -0.430
  StateIdx: 4  Value: -0.682  TrainTo: -0.682
  StateIdx: 5  Value: -1.000  TrainTo: -1.000
td_lambda: 1.0
td_n_steps: 0
  StateIdx: 0  Value: 0.408  TrainTo: -1.000
  StateIdx: 1  Value: 0.312  TrainTo: -1.000
  StateIdx: 2  Value: 0.152  TrainTo: -1.000
  StateIdx: 3  Value: -0.430  TrainTo: -1.000
  StateIdx: 4  Value: -0.682  TrainTo: -1.000
  StateIdx: 5  Value: -1.000  TrainTo: -1.000
td_lambda: 1.0
td_n_steps: 1
  StateIdx: 0  Value: 0.408  TrainTo: 0.312
  StateIdx: 1  Value: 0.312  TrainTo: 0.152
  StateIdx: 2  Value: 0.152  TrainTo: -0.430
  StateIdx: 3  Value: -0.430  TrainTo: -0.682
  StateIdx: 4  Value: -0.682  TrainTo: -1.000
  StateIdx: 5  Value: -1.000  TrainTo: -1.000
td_lambda: 1.0
td_n_steps: 3
  StateIdx: 0  Value: 0.408  TrainTo: -0.430
  StateIdx: 1  Value: 0.312  TrainTo: -0.682
  StateIdx: 2  Value: 0.152  TrainTo: -1.000
  StateIdx: 3  Value: -0.430  TrainTo: -1.000
  StateIdx: 4  Value: -0.682  TrainTo: -1.000
  StateIdx: 5  Value: -1.000  TrainTo: -1.000
td_lambda: 0.1
td_n_steps: 0
  StateIdx: 0  Value: 0.408  TrainTo: 0.396
  StateIdx: 1  Value: 0.312  TrainTo: 0.290
  StateIdx: 2  Value: 0.152  TrainTo: 0.091
  StateIdx: 3  Value: -0.430  TrainTo: -0.459
  StateIdx: 4  Value: -0.682  TrainTo: -0.714
  StateIdx: 5  Value: -1.000  TrainTo: -1.000
td_lambda: 0.1
td_n_steps: 1
  StateIdx: 0  Value: 0.408  TrainTo: 0.398
  StateIdx: 1  Value: 0.312  TrainTo: 0.296
  StateIdx: 2  Value: 0.152  TrainTo: 0.094
  StateIdx: 3  Value: -0.430  TrainTo: -0.456
  StateIdx: 4  Value: -0.682  TrainTo: -0.714
  StateIdx: 5  Value: -1.000  TrainTo: -1.000
td_lambda: 0.1
td_n_steps: 3
  StateIdx: 0  Value: 0.408  TrainTo: 0.396
  StateIdx: 1  Value: 0.312  TrainTo: 0.290
  StateIdx: 2  Value: 0.152  TrainTo: 0.091
  StateIdx: 3  Value: -0.430  TrainTo: -0.459
  StateIdx: 4  Value: -0.682  TrainTo: -0.714
  StateIdx: 5  Value: -1.000  TrainTo: -1.000
td_lambda: 0.5
td_n_steps: 0
  StateIdx: 0  Value: 0.408  TrainTo: 0.222
  StateIdx: 1  Value: 0.312  TrainTo: 0.035
  StateIdx: 2  Value: 0.152  TrainTo: -0.242
  StateIdx: 3  Value: -0.430  TrainTo: -0.636
  StateIdx: 4  Value: -0.682  TrainTo: -0.841
  StateIdx: 5  Value: -1.000  TrainTo: -1.000
td_lambda: 0.5
td_n_steps: 1
  StateIdx: 0  Value: 0.408  TrainTo: 0.360
  StateIdx: 1  Value: 0.312  TrainTo: 0.232
  StateIdx: 2  Value: 0.152  TrainTo: -0.139
  StateIdx: 3  Value: -0.430  TrainTo: -0.556
  StateIdx: 4  Value: -0.682  TrainTo: -0.841
  StateIdx: 5  Value: -1.000  TrainTo: -1.000
td_lambda: 0.5
td_n_steps: 3
  StateIdx: 0  Value: 0.408  TrainTo: 0.247
  StateIdx: 1  Value: 0.312  TrainTo: 0.055
  StateIdx: 2  Value: 0.152  TrainTo: -0.242
  StateIdx: 3  Value: -0.430  TrainTo: -0.636
  StateIdx: 4  Value: -0.682  TrainTo: -0.841
  StateIdx: 5  Value: -1.000  TrainTo: -1.000
td_lambda: 0.9
td_n_steps: 0
  StateIdx: 0  Value: 0.408  TrainTo: -0.585
  StateIdx: 1  Value: 0.312  TrainTo: -0.696
  StateIdx: 2  Value: 0.152  TrainTo: -0.808
  StateIdx: 3  Value: -0.430  TrainTo: -0.914
  StateIdx: 4  Value: -0.682  TrainTo: -0.968
  StateIdx: 5  Value: -1.000  TrainTo: -1.000
td_lambda: 0.9
td_n_steps: 1
  StateIdx: 0  Value: 0.408  TrainTo: 0.321
  StateIdx: 1  Value: 0.312  TrainTo: 0.168
  StateIdx: 2  Value: 0.152  TrainTo: -0.372
  StateIdx: 3  Value: -0.430  TrainTo: -0.657
  StateIdx: 4  Value: -0.682  TrainTo: -0.968
  StateIdx: 5  Value: -1.000  TrainTo: -1.000
td_lambda: 0.9
td_n_steps: 3
  StateIdx: 0  Value: 0.408  TrainTo: -0.233
  StateIdx: 1  Value: 0.312  TrainTo: -0.487
  StateIdx: 2  Value: 0.152  TrainTo: -0.808
  StateIdx: 3  Value: -0.430  TrainTo: -0.914
  StateIdx: 4  Value: -0.682  TrainTo: -0.968
  StateIdx: 5  Value: -1.000  TrainTo: -1.000
'''

Ok, sounds good, and thanks for the kind words.  @lanctot: I emailed you some questions about OpenSpiel back in May, and you mentioned in reply something like this might help with backgammon... I finally got some time to work on it, and am happy to share.

Just an update on my end: Took a quick look-through but will look at it more thoroughly on the weekend and publish the review.

authors: christianjans mattrek


text:
Multiple optimizers using optax

Sorry if this already exists in optax as a feature, but how would you go about making a multi-optimizer (similar to Flax optim) that could use different learning rates for different parts of a network?
Specifically, I'm have a full model in Haiku with one learning rate for most of the parameters, but different learning rates for specific subsets. I can partition the params appropriately and create a separate optimizer for each subset, but ideally I'd like to maintain the simplicity of a single set of params and optimizer_state. Is there a common approach to this?

One possible way to do this is by combining chain and masked. Here's an example using Haiku where you apply different optimizers to the weights vs biases:
import jax
import jax.numpy as jnp

import optax
import haiku as hk

def forward_fn(x):
  return hk.Sequential([
    hk.Linear(10),
    hk.Linear(10),
  ])(x)
forward = hk.transform(forward_fn)

params = forward.init(jax.random.PRNGKey(0), jnp.ones([1, 1]))

# a pytree with the same structure as params with boolean values at the leaves
is_bias_mask = hk.data_structures.map(lambda mname, name, val: name != 'b', params)
not_bias_mask = jax.tree_map(lambda x: not x, is_bias_mask)

optimizer = optax.chain(
    # only applies to leaves where is_bias_mask is True:
    optax.masked(optax.sgd(learning_rate=0.1), is_bias_mask),
    # only applies to leaves where not_bias_mask is True:
    optax.masked(optax.adam(learning_rate=0.01), not_bias_mask))

state = optimizer.init(params)
...

Once #60 is merged, it will be possible to use masked without knowing the structure of the params pytree in advance:
from functools import partial

# mask_fn returns a mask pytree given the parameters pytree as input
is_bias_mask_fn = partial(hk.data_structures.map, lambda mname, name, val: name != 'b')
not_bias_mask_fn = partial(hk.data_structures.map, lambda mname, name, val: name == 'b')

optimizer = optax.chain(
    optax.masked(optax.sgd(learning_rate=0.1), is_bias_mask_fn),
    optax.masked(optax.adam(learning_rate=0.01), not_bias_mask_fn))

Does that work for your use case?

Yeah, that works well for my case, I can create masks for each param group, then do a chain entry for each. I appreciate the help, thank you! Any idea when #60 might be merged? It already looks like I might be able to just copy it to my local repo for now

Great! #60 was merged a few hours ago, so you should be able to use it now 😊.

As of #132, this is much simpler: https://optax.readthedocs.io/en/latest/api.html?highlight=multi#optax.multi_transform

repo: deepmind/optax


text:
Tensorflow 2.4.0 support

What problem are we trying to solve?:
Deeppavlov currently supports tensorflow version 1.15.2 for running various models.
The code which is built using these models shows security vulnerabilities in whitesource (https://www.whitesourcesoftware.com/) scans. These vulnerabilities are due to outdated version of tensorflow.

How can we solve it?:
We can have support for latest tensorflow version 2.4.0 which has patches for all the security vulnerabilities found in older versions.

Are there other issues that block this solution?:
No.

Hi! Idea is great, and it is on our list. However, given the wide number of available models, we plan to re-factor the library based on the model's popularity, and then we'll work on migration to 2.0. :)
If you want to contribute let us know!

All tensorflow-based models will be removed/replaces with pytorch-based ones in 1.0.0. 1.0.0rc0 is already available.

authors: IgnatovFedor danielkornev rupamojha

repo: deepmipt/DeepPavlov


text:
Add documentation for using own components and metrics

adding to the library and registering with registry rebuild
using class in config without registering
using metadata.imports

Implemented in #461

authors: nikolay-bushkov yoptar


text:
Compiling error on Ubuntu 16.04

Hi. I got this error when compiling.
qt/qt_DeepOnion_qt-bitcoin.o: In function main': /home/**/deeponion/src/qt/bitcoin.cpp:129: undefined reference to qInitResources_bitcoin()'
collect2: error: ld returned 1 exit status
Makefile:2244: recipe for target 'qt/DeepOnion-qt' failed
make[1]: *** [qt/DeepOnion-qt] Error 1
make[1]: Leaving directory '/home/**/deeponion/src'
Makefile:542: recipe for target 'all-recursive' failed
make: *** [all-recursive] Error 1
I tried this
https://github.com/deeponion/deeponion/issues/75
but still got the same error.
I think this problem arose after a commitment after 1.6.3 (Not sure the exact commitment, first 1.6.3 release compiles fine).
1.7 is important so could you please help me out? Thank you!

It's a known problem with the QT build tools. Please use this command to work around the issue.
touch src/qt/bitcoin.qrc

That works. The key is running make again after touch. Thank you.

@zxhaxdr I think that I know where the problem is.
Do the following, after cloning the repository, do qmake before doing ./autogen.sh
Tell me if it works.

@nezero @zxhaxdr I have done the following, after cloning the repo, I have done qmake before doing  \autogen.sh
This has worker for me.

authors: aitorp6 nezero zxhaxdr

repo: deeponion/deeponion


text:
Update repositories.toml

ℹ Repository information
https://github.com/kaushikc44/codevim/issues
The repository has:

[x] At least three issues with the good first issue label.https://github.com/kaushikc44/codevim/issues
[ ] At least 10 contributors. None cause I made it individual
[x] Detailed setup instructions for the project.https://github.com/kaushikc44/codevim/blob/master/README.md
[x] CONTRIBUTING.md https://github.com/kaushikc44/codevim/blob/master/DEVELOPER.md
[x] Actively maintained. yes I keep committing to it.

Hey @kaushikc44  ! I tried to contribute to this repo as well but I haven't seen any activity in the last few months. I decided to fork it to keep it alive. Please see https://github.com/lucavallin/gfi and feel free to open a pull request there. At this time I am setting up the hosting and cleaning up the code. If you're up for it, I could use some help to convert the app from Nuxt.js & Python to Next.js & Typescript.

Feel free to contribute, will commit it have lately been busy with work.

@kaushikc44 Thanks, great to hear! I don't really use discord but if you open an issue/discussion on my repo I'll be happy to have a chat there!

Hey @kaushikc44 - Thanks for creating the PR.
To ensure quality and standard, we require at least 10 contributors to add your repository to Good First Issue.
You can check the criteria here.

Oh I see, sorry about that.

authors: kaushikc44 lucavallin withshubh

repo: deepsourcelabs/good-first-issue


text:
test(sanity-check): add script for a semi-manual sanity-test.

at release, update script to find latest version

PR for dsh-core #175

added sanity-test-steps to readme.  i don't have a sense of how a readme is supposed to be.  this good?  pushed to branch.


text:
The CI pipeline should run pre-commit run -a to verify a clean codebase

As a maintainer of Zarf, I want the CI pipeline to run pre-commit run -a and fail if any of the checks come back "dirty", so that we can continue to keep a clean codebase by requiring linting and formatting rules.

@jeff-mccoy What's the impact here?  Would the codebase as it sits now pass this?  If so, let's push this up the backlog seems like a quick win.

We have linting in the pipeline now, closing

authors: JasonvanBrackel RothAndrew jeff-mccoy

repo: defenseunicorns/zarf


text:
feat: exposing gitea image vars for custom init pkg.

Description
We want to expose the image values for gitea for users to override just as they can do for zarf-registry and zarf-agent.
...
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Other (security config, docs update, etc)

Checklist before merging

[ ] Test, docs, adr added or updated as needed
[ ] Contributor Guide Steps followed

There are also broken links: https://app.netlify.com/sites/zarf-docs/deploys/652da9fd09c7d000081dc655#L141


text:
968/9 variables improvements

Description
This introduces the ability to load package variables from files and refines the workflow to reduce confusion.
Related Issue
Fixes #968, #969
Type of change

[X] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist before merging

[X] Tests have been added/updated as necessary (add the needs-tests label)
[X] Documentation has been updated as necessary (add the needs-docs label)
[X] An ADR has been written as necessary (add the needs-adr label) [ 1 2 3 ]
[X] (Optional) Changes have been linted locally with golangci-lint. (NOTE: We haven't turned on lint checks in the pipeline yet so linting may be hard if it shows a lot of lint errors in places that weren't touched by changes. Thus, linting is optional right now.)

@Madeline-UX from talking with @jeff-mccoy last night I think this PR has a need to validate with users (potentially just from the k8s slack) to gather input on what they expect.  With @jeff-mccoy, @RothAndrew and @YrrepNoj  being devs there can be a bias with the different ways that instantiating a "variable" can be done with different languages acting differently (go and bash will initialize a string as an empty string, javascript, java, c, and others will initialize as null or undefined, and python will force you to set it to None or an empty string).

Specifically:
variables:
  - name: "NOTHING"
    # default: ""   <- currently under the hood
    # prompt: false <- currently under the hood
  - name: "NO_PROMPT"
    default: "no-prompt"
    # prompt: false <- currently under the hood
  - name: "NO_DEFAULT"
    # default: ""   <- currently under the hood
    prompt: true
  - name: "BOTH"
    default: "both"
    prompt: true

zarf package deploy package.zst --confirm --set NOTHING=hello --set NO_PROMPT=world --set NO_DEFAULT=today --set BOTH=tomorrow

zarf package deploy package.zst --confirm --set NOTHING=hello --set NO_DEFAULT=today

zarf package deploy package.zst --confirm --set NOTHING=hello

zarf package deploy package.zst --confirm

zarf package deploy package.zst

@Racer159 - I agree that more validation is needed. I am working off of the assumption that we can use the UI to get feedback from users from different skill levels.  I am working to integrate deploy time variables into the UI and have already been getting feedback on the feature (mostly questions at this time.) I would like some help with creating a more realistic prototype @RothAndrew that shows all the different options we are proposing here and see if that makes sense to users.

Specifically:
variables:
  - name: "NOTHING"
  - name: "NO_PROMPT"
    default: "no-prompt"
  - name: "NO_DEFAULT"
    prompt: true
  - name: "BOTH"
    default: "both"
    prompt: true

zarf package deploy package.zst --confirm --set NOTHING=hello --set NO_PROMPT=world --set NO_DEFAULT=today --set BOTH=tomorrow

zarf package deploy package.zst --confirm --set NOTHING=hello --set NO_DEFAULT=today

zarf package deploy package.zst --confirm --set NOTHING=hello

zarf package deploy package.zst --confirm

zarf package deploy package.zst

@Racer159 That looks like what the package definition would be - but what would the user see on deployment in the CLI?

Possible questions we could ask:

If you create a variable called "FOO" that does not have a default, and is not set to prompt the user for a value, what should happen?

The system will automatically assign "" (an empty value / empty string) as the value of FOO and continue processing
The system will stop processing and return an error asking the user to set a value for FOO using <command> --set "FOO=YourValueHere"
Something else?

If you create a variable called "BAR" that has a default and is not set to prompt the user for a value, what should happen?

It uses the default unless the user has used <command> --set "BAR=YourValueHere". If --set was used it uses that as the value instead.
Something else?

If you create a variable called "BAZ" that does not have a default and is set to prompt the user for a value, what should happen if the user uses --confirm, which overrides interactive prompts?

The system will automatically assign "" (an empty value / empty string) as the value of BAZ and continue processing
The system will stop processing and return an error asking the user to set a value for BAZ using <command> --set "BAZ=YourValueHere"
Something else?

If you create a variable called "QUX" that has a default and is set to prompt the user for a value, what should happen if the user uses --confirm, which overrides interactive prompts?

?

@RothAndrew This is helpful. I think to get the feedback we are looking for having common or easily understood examples will be helpful. The users I have discussed variables with so far do not understand what they are being asked at a general level but the context of (DNS, Ports, etc) helps. So is there a real/common example for each of those use cases outlined above?

@RothAndrew @Racer159 @jeff-mccoy  Here are some of the mock up's I did for variables in the UI. Don't think I captured all the different ways variables work but trying to explore how we would present all the different types of variables to a user. Desired outcome is that the user completes all variables correctly and is able to successfully deploy as they intended without having to edit or redo the deployment due to a variable being incorrectly implemented.

The top is the package definition, and then the bottom are the commands the user runs.

The top is the package definition, and then the bottom are the commands the user runs.

zarf package deploy package.zst --confirm --set NOTHING=hello --set NO_PROMPT=world --set NO_DEFAULT=today --set BOTH=tomorrow
zarf package deploy package.zst --confirm --set NOTHING=hello --set NO_DEFAULT=today
zarf package deploy package.zst --confirm --set NOTHING=hello
zarf package deploy package.zst --confirm
zarf package deploy package.zst

So I don't see how this invites user interaction. - It also does not follow the current Zarf model to my knowledge for promoting user input.

The top is the package definition, and then the bottom are the commands the user runs.

zarf package deploy package.zst --confirm --set NOTHING=hello --set NO_PROMPT=world --set NO_DEFAULT=today --set BOTH=tomorrow
zarf package deploy package.zst --confirm --set NOTHING=hello --set NO_DEFAULT=today
zarf package deploy package.zst --confirm --set NOTHING=hello
zarf package deploy package.zst --confirm
zarf package deploy package.zst
* So I don't see how this invites user interaction. - It also does not follow the current Zarf model to my knowledge for promoting user input.

The --confirm flag will just execute without needing any interaction, the last command example without a --confirm flag would currently prompt for the following (... is for truncated output):
...
Deploy this Zarf package? (y/N)
...
Please provide a value for 'NO_DEFAULT':
Please provide a value for 'BOTH' (both):
...

Can I get a quick SWAG on when this might merge? DI2-ME updated to the latest version of Zarf but this is what we really need to start the transition to making it so users don't have to fork the repo to use the software factory package

I'll poke around this in the next couple of days but think we might go in a slightly different direction with viper.

Cool, yeah I don't really care how it's done, just want the capability.
Are we talking a week? a month? longer? Trying to figure out when to anticipate when the big push / refactor for DI2-ME might be able to be undertaken.

@jeff-mccoy @YrrepNoj @RothAndrew @Madeline-UX reset this PR now that the Viper PR has been merged.

@jeff-mccoy @Racer159 I am working to find users to validate this. I have a few who I think will be a good fit.

I'd like to be one of the users pls

Closing this ancient wonder as stale until we can circle back to it at some future point.

authors: Madeline-UX Racer159 RothAndrew jeff-mccoy