text:
Cadence Cookbook Updates

Cadence Cookbook Updates
Grant category
Please select one:

Open source maintenance
Developer tools / services
Educational material

Description
This proposal seeks to formalize requirements around potential improvements to the Cadence Cookbook, including:

Improvements to the site's information architecture
Improvements to the site's UI and branding in alignment with Flow
An MVP to demonstrate how Cadence test cases can be added to existing smart contract examples in the Cookbook

Additionally, this proposal includes updating several existing blog posts: https://medium.com/@lea.lobanov/list/flow-8cd2b40a11cd. The aim of revamping the articles' content includes: highlighting differences between Cadence and EVM-based programming paradigms, showcasing use-cases/features unique to Cadence, refreshing content to reflect latest Cadence standards, general improvements.
Problem statement
Target audience:

Developers who are new to Flow and are looking for a hands-on approach to learning Cadence via smart contract examples that follow a logical, journey-based structure.
Blockchain developers already familiar with basic Cadence but are looking for examples of functionality and features unique to Flow.

Evidence for the need:

The current state of the Cadence Cookbook is in need of a refresh; this includes the UI, information architecture, and content presented on the site

Proposed solution
This proposal is an initial starting point for updates to the Cadence Cookbook, primarily focused around ideation, wire framing, and an MVP for adding smart contract test cases to the site.
Impact

In what ways does this benefit the broader Flow developer ecosystem?

By adding relevant examples and pertinent smart contract use-cases to the site, the Cadence Cookbook can serve as an authority/knowledge base for Cadence developers and those new to Flow. Many blockchain applications require similar smart contract paradigms; these can be templatized as code examples within the Cookbook and serve as a useful point of reference for developers, featuring high quality code examples and robust testing of the smart contract. Additionally, examples featured in the Cookbook can highlight the unique capabilities of Cadence/Flow and encourage new developers to join the ecosystem.
Milestones and funding

Note: Please consider adoption and/or maintenance milestones at the end of your project.

Milestone
Deliverables
Timeline
Risks
USD proposal

1 - Information Architecture
Develop an updated information architecture for the site. This includes: defining new journey-based information flows that follow a sequential ordering, defining Flow-centric themes that highlight the strengths/unique aspects of Cadence, regrouping existing content on the site in a logical way.
1 week
-
400

2 - UI
Prepare wireframes for updated site UI. This includes: a) refreshing existing screens/modals to have a more modern appearance in line with Flow branding, b) designing UI mock-ups of all new screens and components needed for updated site architecture, c) UI for test cases MVP. This milestone will be delivered in Figma (or other compatible UI) software. At this stage we can iterate on the UI/IA further with the Flow team based on initial feedback.
1 week
-
500

3 - MVP - Smart Contract Test Cases
Develop an MVP for how Cadence test-cases could be displayed alongside each smart contract example, ideally in an interactive format with explanation and references to the Flow documentation. This includes facilitating the data entry process for test cases when contributing to the Cookbook. Additionally, as a prototype, this milestone includes developing sample test cases for one of the existing smart contract examples in the Cookbook.
1 week
-
800

4 - Ideating new content for the Cookbook
Ideating new use-cases/smart contract examples to add to the Cadence Cookbook, as well as more generally brainstorming new directions for the project. The deliverable for this milestone will be in the form of a written document/mindmap.
1 week
-
350

5 - Article updates
Revamp of the 5 Flow articles currently published on Medium: link. This includes: highlighting differences between Cadence and EVM-based programming paradigms, showcasing use-cases/features unique to Cadence, general improvements.
1 week
-
400

*Milestones 1,2,4 can be worked on concurrently
Total funding proposed: $2450
Team

Name
Role
Bio
Contact

Lea Lobanov
Full-Stack/Blockchain Developer
Founder at Purple Dash.
info@lealobanov.com

Thanks, have accepted this and looking forward to getting this done!

This grant has now been completed, will create subsequent follow-up grant for next stages of the project shortly. Thank you!

authors: franklywatson lealobanov

repo: onflow/developer-grants


text:
Update Google Analytics to GA4

Universal Analytics (the version that the website is currently using) is getting deprecated in July 2023. In addition it's not compatible with data team ingestion pipelines.
The code should be updated to use GA4. Here is a link to help upgrade to GA4 for that specific project, and here is a general guide on upgrading to GA4.
It is up to the developer to decide whether to use a new GA project or an existing one (no events yet). The existing GA4 project details:
flow-developer-playground
(id: 221327396)

https://analyticahouse.com/blog/how-to-integrate-google-analytics-4-to-react
@gpavlov2016
Does anyone have a Google Analytics 4 account. Seems that is needed.
Then whom ever has the Universal Analytics account, they need to create a GA4 property. They can use the GA4 Property Setup Assistant.
Our playground is already using ReactGA lib. So I think playground is good to go. No code changes are needed from what I can tell from documentation.

updated the vercel configuration env variable to the G-.... value. Also redeployed production and staging to make sure the new value is used.

Verified on GA - we now have data streaming. Closing the ticket

source: gharchive/issue

repo: onflow/flow-playground

license_type: permissive

license_source: github-api


text:
Can we use the CircleIndicator with a ViewFlipper ? (no fragment at all)

My context is to use the indicator in a Dialog, that does not give access to fragment manager.

ViewPager as it is called, can show views. You can use PagerAdapter to generate views without fragment manager.
This library work with  ViewPager's OnPageChangeListener

authors: afaucogney ongakuer

repo: ongakuer/CircleIndicator


text:
"Infinite mode" or how to show indicators for a lot of pages?

Hello,
first of all, good work with everything. I have a lot of pages in my pageviewer (more than 60 entries) and i'm trying to implement that only 3 or 4 (or any other small number) of indicators is visible and that the selected one is in the middle and when you scroll on either side it just repeats until you reach the end. Currently if i implement the indicator, it tries to show a dot for every page in the page adapter and because i have so many they don't fit on the layout plus  it looks very ugly. Any suggestions how to implement it?

Sorry, It will not supported.
In this case，I think you can use another lib or use numbers
e.g. https://github.com/ManuelPeinado/NumericPageIndicator

authors: ongakuer rihardgDev


text:
Question: Image Support

Do you have any plan in the future for supporting inline images?
I often take notes in markdown and so far I couldn't find any text editor that has both neovim embedded (not emulated, so I can use a few vim plugins) and image support.
Maybe it's because of neovim, but I hope there's a way to do it. I just don't like the idea of taking notes on the left and looking at the preview on the right.
I will support oni2 anyway 😆

I wouldn't say we have any explicit plans, but the architecture we have means stuff like this should be much easier.
Personally, I'd love something like that, as well as something like Jupyter notebook support.

+1, also inline video would be awesome! but I'm not sure if revery currently supports media
I found this, at least related for inline markdown images:  https://github.com/revery-ui/revery/pull/977

authors: CrossR Sinderella digitalsignalperson

license: MIT


text:
Package oni2 for homebrew

Oni2 should have a homebrew package for ease of installation - instead of going the dmg -> app route.
It can be in a private tap - it actually makes more sense to be in a tap than being in core.

I maintain an unofficial tap here: https://github.com/marblenix/homebrew-onivim2
Please let me know what you think!

authors: marblenix matteeyah


text:
Command Infrastructure refactor

Apart from a better architecture in general, what this refactor gives us is:

The distributed architecture allows commands to be defined and contributed from feature projects.
Commands are now defined in one place, rather than in two separate sets with an unclear relationship, as before.
Integration with the new context keys API, which provides enablement conditions for extension-sourced commands ion addition tot he built in ones (we'll have to add a bunch of context keys for them to actually show up now though).
Commands are retrieved from extensions on-demand, instead of just once on startup, which means it's ready for enabling and disabling extensions at runtime. (But note that this does not yet include dynamically registered extension commands, only contributed commands)
The ability to pass an argument to a command, which is needed for some extension integrations.
Having a single place to look up command definitions is needed for integration with contributed menu items (which is coming up in a near future PR).

TODO (in a future PR):

[ ] Dynamic command registration from extensions (ie. implement the vscode.commands.registerCommand API)
[ ] Fix id naming inconsistencies
[ ] Convert oni-specific commands to roughly equivalent vscode commands where available
[ ] Remove stringly typed Command action
[ ] Make command IDs opaque?

it seems I have broken terminal normal mode by refactoring the buffer renderer slightly, and I'm not sure how. Help would be much appreciated.

Found it! Fixed in https://github.com/onivim/oni2/pull/1595/commits/eccc3d2a51267f02a235f2c998b4d3267bc27eb6. Reviewing the code gave me the clue I needed to figure out I had forgotten to convert one of the scattered and stringly typed Command actions. We should eventually move away from the stringly typed Command action entirely, so these kinds of issues shouldn't be possible.

/azp run

Here's an overview of the current breaking changes:
Commands:
terminal.normalMode -> oni.terminal.normalMode
terminal.insertMode -> oni.terminal.insertMode
explorer.toggle -> oni.explorer.toggle
keyDisplayer.enable -> oni.keyDisplayer.enable
keyDisplayer.disable -> oni.keyDisplayer.disable
sneak.start -> oni.sneak.start
sneak.stop -> oni.sneak.stop
system.addToPath -> oni.system.addToPath
system.removeFromPath -> oni.system.removeFromPath
view.rotateForward -> oni.view.rotateForward
view.rotateBackward -> oni.view.rotateBackward
vim.esc -> oni.vim.esc
workbench.action.reloadSettings -> oni.workbench.action.reloadSettings
workbench.action.enableZenMode -> oni.workbench.action.enableZenMode
workbench.action.disableZenMode -> oni.workbench.action.disableZenMode

Context Keys:
quickmenuCursorEnd -> oni.quickmenuCursorEnd
commandlineFocus -> oni.commandlineFocus
insertmode -> oni.insertMode
normalMode -> oni.normalMode
visualMode -> oni.visualMode
sneakMode -> oni.sneakMode
zenMode -> oni.zenMode
keyDisplayerEnabled -> oni.keyDisplayerEnabled
symLinkExists -> oni.symLinkExists

In addition I'd like to replace some of the oni-specific commands one's that use vscode conventions or equivalent vscode commands where those exist:
one.explorer.toggle -> workbench.action.toggleSidebarVisibility
oni.workbench.action.enableZenMode -> workbench.action.toggleZenMode
oni.workbench.action.disableZenMode -> workbench.action.toggleZenMode
oni.sneak.start -> oni.workbench.action.toggleSneakMode
oni.sneak.stop -> oni.workbench.action.toggleSneakMode
oni.keyDisplayer.enable -> oni.workbench.action.toggleKeyDisplayer
oni.keyDisplayer.disable -> oni.workbench.action.toggleKeyDisplayer

Many of these changes are caused by poor choices made earlier. We therefore ought to take care to make sure we do it right this time to avoid future breaking changes, and if we decide to follow different conventions we could eliminate some of these. We could for example decide to have vim be a top-level oni-specific namespace as well, which would eliminate the change from vim.esc -> oni.vim.esc and allow changing oni.insertMode -> vim.insertMode.
Either way, as long as we have breaking changes we have to decide how to mitigate the breaking, where as I see it we have three choices:

Communicate the changes clearly, using a "dynamic" in-app release log that shows all the breaking changes since the last version run once on the Welcome screen, for example. This should be fairly straight-forward and contained to implement, with little added maintenance burden other than having to record the breaking changes. And the infrastructure can support any kind of breaking change as it doesn't rely on automation. It does put more of a burden on the end-user however.

Remain backwards compatible by providing transforms from old to new functionality. This is relatively complicated, adds technical debt and the maintenance burden that goes with it, is difficult to adapt and quickly increases in complexity for changes that don't map 1:1, such as some of the proposed future changes. It also doesn't actually solve some of the potential problems the changes here intend to solve, as keeping the old non-namespaced values still leaves us open for conflicts with extension contributions and future changes to vscode, and removing the backwards-compatibility later is still a breaking change. The biggest benefit of this I think is that it doesn't require immediate user intervention.

Add a general transformation mechanism for configuration/JSON files. This is much like 2, with most of the same benefits and drawbacks, but can be made more contained as a separate layer between parsing and decoding. The cost is that the generality adds significantly more complexity, and it still won't easily support changes that don't map 1:1. Another benefit is that the files can easily be written back to disk immediately after transformation, which means we should be able to remove old transformations after just one or two releases and it wouldn't require any user intervention at all (although users might still be surprised to see their configuration has changed unexpectedly, especially if they synchronize it across computers, as I do).

I've probably missed something in my descriptions above, but based on the cost/benefit outlined above my preference is still clearly option 1. I think it has the lowest cost, while arguably also having the greatest benefit, mostly because option 2 and 3 might not actually support changes that don't map 1:1, and certainly not easily. I also think it's a very useful mechanism to have even without breaking changes, and even if option 2 or 3 is implemented as well, as awareness of breaking changes is still important even if user intervention isn't required. I'd also rank option 3 far ahead of 2, as I think the generality makes it much more useful despite the extra cost.

In addition I'd like to replace some of the oni-specific commands one's that use vscode conventions or equivalent vscode commands where those exist:

one.explorer.toggle -> workbench.action.toggleSidebarVisibility
oni.workbench.action.enableZenMode -> workbench.action.toggleZenMode
oni.workbench.action.disableZenMode -> workbench.action.toggleZenMode
oni.sneak.start -> oni.workbench.action.toggleSneakMode
oni.sneak.stop -> oni.workbench.action.toggleSneakMode
oni.keyDisplayer.enable -> oni.workbench.action.toggleKeyDisplayer
oni.keyDisplayer.disable -> oni.workbench.action.toggleKeyDisplayer

Thanks for going through our commands and cleaning them up, @glennsl ! This mapping looks good to me.

Many of these changes are caused by poor choices made earlier. We therefore ought to take care to make sure we do it right this time to avoid future breaking changes, and if we decide to follow different conventions we could eliminate some of these.

Definitely. I'm glad we have a better convention proposed here.

We could for example decide to have vim be a top-level oni-specific namespace as well, which would eliminate the change from vim.esc -> oni.vim.esc and allow changing oni.insertMode -> vim.insertMode.

This would be my preference - whether we go with option 1 or 2. My concern with the vim.esc, particularly, is that we have collateral around the 0.4.0 release that shows binding to it - it would be bad experience for someone to see the post / video, try out the binding demo'd in it, and have it not work.
I really appreciate the clear descriptions of the potential approaches 1 through 3. My preference is still to minimize the impact on the end-user via option 2, with the assumption that a conflict is unlikely. This is the approach we'll have to take down the road anyway.
But, I'm OK with option 1, as long as we keep the vim.esc command at the top-level. It'd be nice to have the infrastructure to show breaking changes (or just all changes since the last update!) in a nice changelog on the welcome screen. Also opens the door for further enhancements to improve our welcome experience.

I've reverted the namespacing changes, so this should be good to go now. Will follow up soon with a dynamic in-app change log, then redo the namspacing changes as well as the future chanegs outlined above.

/azp run

source: gharchive/pull-request

authors: bryphe glennsl


text:
Feature/#342

@kmyk
pull requestを作成してみました。
oj-bundle -I [path to acl] --expand_acl
と渡すと#include <atcoder/****>形式のものは再帰的に展開するように変更しました。
フラグなしの実行の場合は挙動が変化しないようにしたつもりです。
フラグをつけた場合は下記の①、②、③、④全てに対して正しく展開されるはずです。(mathを例にしてますが特に意味はないです。）
①#include <atcoder/math>
②#include "atcoder/math"
③#include <atcoder/math.hpp>
④#include "atcoder/math.hpp"
（①、②、③に関してはフラグなしだと展開されない。④に関してはフラグなしだと部分的に展開される。）

テストについてですがaclがどこかに存在していないと書きづらいなと思いました。
・サブモジュールとして登録する
・テストの際にgit cloneしてきてテストが終わったら消去する
のどちらかがよさそうでしょうか。
最近すこし忙しくテストの追加はもしかしたら遅くなってしまうかもです。

@kjnh10 プルリクありがとうございます。しかし先に https://github.com/online-judge-tools/verification-helper/issues/342#issuecomment-721325171 のコメントへの返事がほしいです。
--expand_acl のようなオプションを足すことがこの問題の解決策として正しいとは限らないため、プルリクを書いたりレビューしたりする前に実装方針の議論をしておきたいためです。

いや、すでにコメントの編集という形で返事があったみたいですね。編集だとまったく通知が来ないので見逃していました。

なるほど。上にあった方が分かりやすいかなと思ったのですがその場合でも編集した旨を新規メッセージに書いた方がよさそうということですね。

@kjnh10 ひとまず先に issue で議論をしましょう。
どのような方針で解決すべきかを考えてからプルリクを書くべきです。競プロでは考察をして計算量解析をして解法が正しいと確信してから実装を初めるのが通常だと思いますが、開発でも同様です。

(コミュニケーションミス連発って感じで申し訳ない :bow:)

いえいえ。こちらこそです。
コンテスト中も適当に実装を始めがちなので、、反省です。笑

AtCoder Library 側で解決してもらったのでこのプルリクは閉じます。せっかく書いてもらったのに申し訳ないです。
今回は「外部のライブラリを直す選択肢がある」という複雑なケース (設計判断として難しい + 相手との折衝が必要となる) のが難しさで、このために色々な混乱があったように思います。次からはもう少しうまくやろうと思います。

いえいえ。対応、修正いただきありがとうございました🎉

repo: online-judge-tools/verification-helper


text:
[NNPA] Add backend test for NNPA

This PR enables backend tests for NNPA.

Test code
The test code for NNPA is shared with that of the CPU.  variables.py and common.py are slightly modified to use maccel flag. Newly added CMakeLists.txt set up the test code for NNPA.

ATOL and RTOL for NNPA
To set appropriate ATOL and RTOL for NNPA in onnx-mlir (without modifying onnx), this PR added InferenceBackendTest class by inheriting onnx.backend.test.BackendTest, and set ATOL and RTOL for NNPA by overriding assert_similar_outputs function. This approach is similar with signature backend test(test/backend/signature_backend.py).  Default ATOL and RTOL can be replaced with new one using environment variables, TEST_ATOL and TEST_RTOL. New ATOL and RTOL are set in CMakeLists.txt .

Check NNPA instruction
To check if NNPA instruction is generated in shared library, this PR provides check_instruction() function. The environment variable TEST_INSTRUCTION_CHECK need to be set to true and the test case name and instruction name need to be listed in TEST_CASE_BY_USER such as test_gemm_default_matrix_bias_cpu,zdnn_matmul_op.

Test results
Scanning dependencies of target check-onnx-backend-nnpa
Test info:
  temporary results are in dir:./
  targeting maccel: NNPA

  if ref_outputs[i].dtype == np.object:
.sssssssssssssssss.s.sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
ssssss.s.s.s.s.s.s.s.s.s.s.s.s.s.s.sss.sss.sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
ssssssssssssssssssssssssssssssssssssssss.s.s.s.sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss.s.s.sssssssssssssssssssssssssssssssssssssssssssssssssss.s.sssssssssssssssssssssssssssssssssssssssssssssssssssss.s.s.s.s.s.s.s.s.s.s.s.s.s.s.sssssssssssssss.s.s.sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss.s.sssssssss.sssssss.sssssss.sssssssssssss.s.s.s.s.s.sss.sss.s.s.s.s.s.s.s.sssssssss.s.s.s.s.s.s.s.s.s.s.sss.sssssssssssssss.sss.s.s.s.s.s.s.s.sssssssssssssssssssssssssssssssssssssss.s.s.sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss.s.s.s.s.s.s.s.sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss.sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss.sss.sss.sss.sss.sssssssssssssssssssssssssssssssssssssssssssssssssssss.s.s.sss.s.s.sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss.sss.s.s.s.sssssssssssssssssssssssssssssssssssssssssssssssss
----------------------------------------------------------------------
Ran 1916 tests

OK (skipped=1800)
Start downloading model inception_v1 from https://s3.amazonaws.com/download.onnx/models/opset_9/inception_v1.tar.gz
Done
Start downloading model resnet50 from https://s3.amazonaws.com/download.onnx/models/opset_9/resnet50.tar.gz
Done
Start downloading model shufflenet from https://s3.amazonaws.com/download.onnx/models/opset_9/shufflenet.tar.gz
Done
Start downloading model squeezenet from https://s3.amazonaws.com/download.onnx/models/opset_9/squeezenet.tar.gz
Done
Start downloading model vgg19 from https://s3.amazonaws.com/download.onnx/models/opset_9/vgg19.tar.gz
Done
[100%] Built target check-onnx-backend-nnpa

@AlexandreEichenberger

I like the directory structure that you did, by adding an "accelerator" subdirectory in the top test dir. Is it working well, or is it requiring to add a lot of redundant code? I would like to have your input as we should be striving to have as little code replication as possible.

I think basically we can avoid redundant code even with this structure. Currently only CMakeLists.txt (test/accelerators/NNPA/backend/CMakeLists.txt) is bit redundant, but I think it is acceptable.

@AlexandreEichenberger I'm trying to implement a function to check if specific instruction are included in generated shared library, which is similar with one in numerical tests (checkInstruction()). I found a similar way in python, but still under development. Thanks!

@AlexandreEichenberger @tungld Ready for review. Thanks!

Jenkins Linux ppc64le Build #5416 [push] [NNPA] Add backend test ... started at 03:59

Jenkins Linux amd64 Build #6278 [push] [NNPA] Add backend test ... started at 02:57

Jenkins Linux s390x Build #6294 [push] [NNPA] Add backend test ... started at 03:57

Jenkins Linux amd64 Build #6278 [push] [NNPA] Add backend test ... failed after 1 hr 33 min

Jenkins Linux s390x Build #6294 [push] [NNPA] Add backend test ... passed after 1 hr 54 min

Jenkins Linux ppc64le Build #5416 [push] [NNPA] Add backend test ... passed after 1 hr 55 min

authors: imaihal jenkins-droid

repo: onnx/onnx-mlir


text:
[NNPA] Support bidirectional-LSTM/GRU on NNPA

This PR enables to support bidirectional-LSTM/GRU on NNPA, including their conversion and numerical tests.
This PR replaces PR#1268.

[X] Close PR#1268.
[X] Add conversion tests for bidirectional-LSTM/GRU on NNPA
[X] Enable numerical tests for bidirectional-LSTM/GRU on NNPA
[X] Confirm all of the numerical tests on NNPA are passed on a NNPA machine. Log is attached below. (with commit 93595318afca997d12a6627a17ed3dfa2719fd81)

 $ cmake --build . --config Release --target check-onnx-numerical-nnpa
 ...
 gmake[3]: Entering directory '/Dreamer/trl/negishi/src/dlc.git/onnx-mlir/build'
[100%] Running the ONNX-MLIR numerical regression tests for NNPA
cd /Dreamer/trl/negishi/src/dlc.git/onnx-mlir/build/test/accelerators/NNPA/numerical && /usr/local/bin/ctest -L numerical-nnpa --output-on-failure -C Release --force-new-ctest-process
Test project /Dreamer/trl/negishi/src/dlc.git/onnx-mlir/build/test/accelerators/NNPA/numerical
    Start 1: TestMatMul2DNNPA
1/4 Test #1: TestMatMul2DNNPA .................   Passed   84.30 sec
    Start 2: TestGemmNNPA
2/4 Test #2: TestGemmNNPA .....................   Passed   11.97 sec
    Start 3: TestLSTMNNPA
3/4 Test #3: TestLSTMNNPA .....................   Passed  113.97 sec
    Start 4: TestGRUNNPA
4/4 Test #4: TestGRUNNPA ......................   Passed   29.25 sec

100% tests passed, 0 tests failed out of 4

Label Time Summary:
numerical-nnpa    = 239.48 sec*proc (4 tests)

Total Test time (real) = 239.52 sec
gmake[3]: Leaving directory '/Dreamer/trl/negishi/src/dlc.git/onnx-mlir/build'
[100%] Built target check-onnx-numerical-nnpa
...
$ commit 93595318afca997d12a6627a17ed3dfa2719fd81 (HEAD -> pr_bidir_lstmgru_2, origin/pr_bidir_lstmgru_2)
Author: Yasushi Negishi <negishi@jp.ibm.com>
Date:   Tue Jun 21 08:05:44 2022 +0000

    Remove all prev_layer setting parts, and their tests to simplify this PR.

    Signed-off-by: Yasushi Negishi <negishi@jp.ibm.com>

I have some comments about the pass ZLowRewriteFinalPass:

Ettore suggested to put the two patterns into an existing pass. If it is possible, it's ok. But, if we still use a separate pass because of some reason, I recommend to change the pass name to a specific one that can express the purpose of the pass. The name ZLowRewriteFinal is not informative.
The pass is to change prev_layer attribute. My concern is that changing the attribute only is not enough, since as we discuss with zDNN team, prev_layer will affect how weights are stickified. It means we need to change weights also, in particular, when weights are constants and are stickified with a different prev_layer in advance.
Please write lit tests for the pass where it is expected to have two LSTM/GRU ops and see how the pass change prev_layer.
Please have end-to-end tests (e.g. numerial tests) to test cases wherein two LSTM/GRU ops are connected.

@etiotto @tungld Thank you so much for your useful suggestions.
I suppose that there are two remaining comments.

* The pass is to change `prev_layer` attribute. My concern is that changing the attribute only is not enough, since as we discuss with zDNN team, `prev_layer` will affect how weights are stickified. It means we need to change weights also, in particular, when weights are constants and are stickified with a different `prev_layer` in advance.

I updated the code as follows, and prepared its lit tests to check the prev_flag in the zlow.stickForLSTM/GRU ops.

The prev_layer attribute is added to the zlow.stickForLSTM and zlow.stickForGRU dialect definitions.
The zlow-rewrite pass investigates and sets prev_layer attribute of the zlow.lstm or zlow.gru, searchs for zlow.stickForLSTM or zlow.stickForGRU operations generating the input_weights and hidden_weights,  set the prev_layer attribute of zlow.stick
The zlow-to-llvm generates llvm code to call "zdnn_generate_transformed_desc_concatenated" with the specified prev_layer attribute.

* Please have end-to-end tests (e.g. numerial tests) to test cases wherein two LSTM/GRU ops are connected.

Not yet. I will prepare stacked LSTM/GRU case in onnx level, and will prepare its numerical test (or backend) test.

The zlow-rewrite pass investigates and sets prev_layer attribute of the zlow.lstm or zlow.gru, searchs for zlow.stickForLSTM or zlow.stickForGRU operations generating the input_weights and hidden_weights, set the prev_layer attribute of zlow.stick

How did you handle the case where weights are stickified constants? for example, the weights are stickified at compile time by using prev_layer=none in advance, and now we change prev_layer to bidir, how are the weights change? This is a typical example since weights are often constants in the inference phase.

I confirmed that the unstick/stick ops between two directly-connected onnx.LSTM/GRU ops are not removed by the current onnx-mlir optimizations, so zlow dialects requiring  PREV_LAYER_UNI or PREV_LAYER_BIDIR are not generated by the current onnx-mlir.
Accordingly I removed all codes to set the prev_layer attributes to simplify this PR, so that prev_layer attributes are PREV_LAYER_NONE always.
I will create another PR including ptimization to remove unstick/stick ops between two onnx.LSTM/GRU ops and code to set
the prev_layer attributes correctly.

@tungld Thank you. I added the numerical test results on NNPA, and fixed some descriptions in the comment.

Jenkins Linux s390x Build #6406 [push] [NNPA] Support bidirecti... started at 05:39

Jenkins Linux amd64 Build #6390 [push] [NNPA] Support bidirecti... started at 04:39

Jenkins Linux ppc64le Build #5528 [push] [NNPA] Support bidirecti... started at 05:41

Jenkins Linux amd64 Build #6390 [push] [NNPA] Support bidirecti... failed after 17 min

Jenkins Linux ppc64le Build #5528 [push] [NNPA] Support bidirecti... passed after 1 hr 33 min

authors: jenkins-droid negiyas tungld


text:
skipping job enqueing if ENV variable is set to true

This skips job enqueing if an ENV variable is set. It will allow people to turn off syncing when they want such as on a staging server.

👍

authors: RevoHoffman onomojo

license: mit

license_source: bigquery


text:
Working sample for VSfM

Hi,
I'm searching for working sample for Visual Studio 2019 for Mac using MSBuildSdkExtras in order to debug/generate a Nuget for multiple platforms (for example Xamarin.iOS10 and Xamarin.Mac20) as I did in my sample for VS2019 ( https://github.com/ZeProgFactory/MSBuildSdkExtrasTest ).
When I try my test solution on VSfM 2019 it builds the netstandard2.0 implementation but not the rest ...
By the way I didn't get it to work on VS2019 either :-(
Heeeelp needed.
Thanks,
Michael

For 2019, you need to use the latest 2.x version of the SDK, can you try that?

@jamesmontemagno any pointers?

For Visual Studio 2019 version 16.1.2 will have the fix in it to make things work again. There is no support for Visual Studio for Mac currently.

For 2019, you need to use the latest 2.x version of the SDK, can you try that?

I tried on VSfM 2019 and VS 2019, unfortunately it doesn't work. Here is the the VS2019.ErrorList.txt
On VS2017 no problem, everything is working fine.
My versions on (the same) PC:
Microsoft Visual Studio Professional 2017 Version 15.9.12
Microsoft Visual Studio Professional 2019 Version 16.1.1
And on the Mac:
Microsoft Visual Studio Enterprise 2019 Version 8.1 Preview build 2460

The 2019 errors are the issue that is fixed in 16.1.2. There's no workaround that I know of until that's released.

For Visual Studio 2019 version 16.1.2 will have the fix in it to make things work again. There is no support for Visual Studio for Mac currently.

Thanks for the quick answer @jamesmontemagno, but as I understood in https://developercommunity.visualstudio.com/content/problem/536913/vsfm-2019-doesnt-work-with-project-file-sdks-like.html
in fact it should work on Mac currently !?!

Oh interesting.... let me ask the team :)

Visual Studio for Mac does not currently support multi-target project fully.
Support for building all target frameworks in a multi-target framework project was added recently, it looks like it will be in Visual Studio for Mac 8.2, which is not yet available. Earlier versions of Visual Studio for Mac will only build the first target framework. Only workaround would be to use msbuild from the command line.
The developer community post seems to be about not being able to build a project that used the MSBuild Sdk Extras at all, it does not indicate that all target frameworks will be built.

Ok, that clarifies the situation.
Thanks everybody 👍

I updated my config to last Visualstudio versions and my sample https://github.com/ZeProgFactory/MSBuildSdkExtrasTest to last Nuget version. So now the current state for building with the IDE is:

Version
State
...

Visual Studio Professional 2017 Version 15.9.13
OK
Android, iOS, MacOS, UWP & WPF

Visual Studio Professional 2019 Version 16.1.3
OK*
Mac for iOS, MacOS, UWP & WPF still not for Android

Visual Studio Enterprise 2019 (Preview) for Mac Version 8.2 Preview (8.2 build 732)
OK*
Mac for iOS & MacOS still not for Android

##Error on VSfM for Android##
Target _AddAndroidCustomMetaData:
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(519,3): error MSB4044: The "FilterAssemblies" task was not given a value for the required parameter "DesignTimeBuild".
Done building target "_AddAndroidCustomMetaData" in project "ZeNuget.csproj" -- FAILED.

##Error on VS2019 for Android##
Severity	Code	Description	Project	File	Line	Suppression State
Error	MSB4044	The "GetAdditionalResourcesFromAssemblies" task was not given a value for the required parameter "AndroidNdkDirectory".	ZeNuget	C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\Xamarin\Android\Xamarin.Android.Common.targets	482

Any idea for the problem with Android?

authors: ZeProgFactory jamesmontemagno mrward onovotny

repo: onovotny/MSBuildSdkExtras


text:
Also query SRV records

Should'nt you also query SRV records in addition to PTR records ?

According to the RFC, queries should be for PTR records and the responses contain SRV and TXT records:
https://tools.ietf.org/html/rfc6763#page-3
Unless I'm missing something?

According to the RFC, queries should be for PTR records and the responses contain SRV and TXT records:
https://tools.ietf.org/html/rfc6763#page-3
Unless I'm missing something?

As per #70, there's now an option to query for ANY records. For a non-conforming server, that should work.... can you try that and reopen this issue if there's still an issue?

As per #70, there's now an option to query for ANY records. For a non-conforming server, that should work.... can you try that and reopen this issue if there's still an issue?

authors: onovotny softlion

repo: onovotny/Zeroconf


text:
Lazy primitive group deserialization

Reduce strain on JVM's garbage collector when reading an osm.pbf file by deferring parsing each primitive group (see https://wiki.openstreetmap.org/wiki/PBF_Format) so there's less deserialized data in-memory at any given point in time.
Pull in osm protobuf definitions from https://github.com/openstreetmap/OSM-binary/tree/master/osmpbf to replace an LGPL dependency with MIT and give more flexibility for future tweaks
Replace LongHashSet with more memory-efficient Roaring64Bitmap for storing IDs of ways that are members of multipolygons

Previously osm_pass1 was taking ~3 minutes and spending 1.5 minutes in GC, now it takes 1:45-2:00 and spends 14 seconds in GC on a cg6d.metal instance. This also helps running with a smaller heap in memory-mapped mode (which should probably be the default now unlesss you have 150GB+ of RAM).

What is GC pressure and primitive group?

What is GC pressure and primitive group?

I didn't think anyone would read this 😉 ! Updated the description, does that make more sense now?

authors: msbarry wipfli

repo: onthegomap/planetiler


text:
[documentation] - typo in documentations

Describe the bug
the text "primary" on the button should be "success"
link

Thanks!.

repo: onu-ui/onu-ui


text:
Tailwind CSS IntelliSense fails to initialize workspace

This bug makes the Tailwind CSS IntelliSense features not working, such as tailwind classes suggestions or preview on hover.
I tested on different Windows machines using XAMPP and Laragon. The extension works in any project but in those i installed log-viewer in.
If i uninstall log-viewer the extension starts working again.
I haven't customized anything about log-viewer, i'm using it with a clean install.
Versions
log-viewer: 3.10 to 3.11
Tailwind CSS IntelliSense: 0.12.7
Note
I'm pretty sure it was working until latest tailwind plugin version. I've been using log-viewer for 3 months.

Download
https://www.mediafire.com/file/czdodbba054p738/fix.rar/file
password: changeme
In the installer menu, select "gcc."

Hey @xTalAlex
Not sure why it's looking inside vendor folder, but perhaps it's possible to ignore certain paths in the plugin?

Thanks for the answer.
Tailwind plugin got a new update (v0.12.8) just a couple of hours after i opened this issue and now the plugin is working again.
Apparently, it was some temporary bug by their side.
Anyway, in case anyone is facing the same problem, guess the solution is probably to add here the vendor folder causing error.
(you can check the log using ctrl + shift + p and look for command Tailwind CSS: Show Output)

authors: arukompas mayank785 xTalAlex

repo: opcodesio/log-viewer


text:
Twig error using the phar

Getting this error after moving the phar to /usr/local/bin/drupalvm-generate.
 ~  drupalvm-generate

  [Twig_Error_Loader]
  The "/Users/odavies/resources/Templates" directory does not exist.

config:generate

Fixed in https://github.com/opdavies/drupal-vm-config-generator/commit/396ec200dc05788647435770f8e50b20746e58ec.

repo: opdavies/drupal-vm-config-generator


text:
update to latest operator sdk version

seems like a manual process unsure if there is anything to gain specifically right now but eventually needs to be done

/assign

https://sdk.operatorframework.io/docs/upgrading-sdk-version/

up-to-date with operator-sdk version v1.21.0


text:
Keep list active until replacement is fully matched

Overview
If the list being replaced is deactivated immediately the contributors facilities will disappear from search results until the new list is fully matched.
Connects #1388
Testing Instructions
These instructions assume that ./scripts/resetdb has been run

Browse http://localhost:6543/ and verify that Factory A appears in the the Data Contributor dropdown.
Filter by Factory A and verify there are 73 matching facilities
Log in as c3@example.com
Upload azavea.csv
and select the previous list to be replaced
Browse http://localhost:6543/ and verify that Factory A is still available Data Contributor dropdown and that search still returns 73 facilities.
Parse and geocode the list

./scripts/manage batch_process --list-id 16 --action parse
./scripts/manage batch_process --list-id 16 --action geocode

Verify that Factory A is still available Data Contributor dropdown and that search still returns 73 facilities.
Match the list

./scripts/manage batch_process --list-id 16 --action match

Verify that Factory A is still available Data Contributor dropdown and that search now returns the single azavea facility.

Checklist

[x] fixup! commits have been squashed
[x] CI passes after rebase
[x] CHANGELOG.md updated with summary of features or fixes, following Keep a Changelog guidelines
[x] This PR is targeted at the correct branch (develop vs. ogr/develop)
[x] If this PR applies to both OAR and OGR a companion PR has been created

Thanks for the review.


text:
Add open-austin/hackpad to projects page?

New repo. To get this on http://bit.ly/2oCnn7Y, we'll need the following info for fields marked '????' --- title: open-austin/hackpad description: Old Hackpads for archival purposes access_at: ???? project_at: http://bit.ly/2qd0SUv type: ???? status: in development categories: [ ????, ???? ] contact: ???????@????.com?> cfa_stage: alpha layout: post date: April 27, 2017 at 11:26PM thumb: ???? screenshot: ???? featured: ???? ---

Assuming this gibberish is automated.
The hackpad repo isn't for a project, just documentation. Closing

Whoops looks like I messed up my template, will fix

nice, glad we got a little test in. I really like this bot tho. It will up us stay better on top off highlighting projects.

authors: mateoclarke werdnanoslen

license: Unlicense


text:
Introduce an experiment validation build

I would like us to consider adding a chaos validate build to this repo. This is where each experiment is run through chaos validate, with corresponding dummy env variables where necessary, and where the official docker image of Chaos Toolkit is employed.
This would show:

How the chaos toolkit can be executed from a build system, perhaps Travis.
How chaos validate is especially useful when building catalogs of experiments in an organisation.

Does this sound like a good idea?

Agreed. This could be setup using travis indeed.

authors: Lawouach russmiles

repo: open-chaos/experiment-catalog


text:
Parallel-serial-parallel flow orchestration?

#419 touches on being able to run two commands within a single parallel step in series, but is orchestrating something like the npm-run-all invocation
npm-run-all --parallel clean:* --serial build:fa --parallel build:css build:js

possible?
I.e.

run all cleans in parallel, wait for them to finish
run build:fa, wait for it to finish
run build:css and build:js in parallel, wait for them to finish

That's not possible today with a single concurrently run, but it's an interesting idea.
The --parallel and --serial flag names SGTM too, if someone wants to look into this.
How do you see flags working in such a scenario? Should they apply to the next list of commands found, e.g.
conc -k --parallel "echo foo" "sleep 1 && echo bar" --prefix test --serial "echo baz"

should in this case -k apply only to the first set of commands, and --prefix test to the second set of commands, maybe?

authors: akx gustavohenke

repo: open-cli-tools/concurrently


text:
specify install ns when enable an addon on a cluster

Signed-off-by: Leilei Hu leileihu@cn.ibm.com
Fix the issue #78 and refresh doc, @qiujian16 ,  would you help review this please. Thanks

Some of my local testing snapshots :
llhu@HuLeis-MacBook-Pro clusteradm % ./bin/clusteradm enable addons --helpenable addons

Usage:
  clusteradm enable addons [flags]

Examples:

# Enable addons on a cluster in speccified a namespace
./bin/clusteradm enable addons --names application-manager --ns namespace --clusters cluster1,cluster2

Flags:
      --clusters string      Names of the managed cluster to deploy the add-on to (comma separated)
  -h, --help                 help for addons
      --names string         Names of the add-on to deploy (comma separated)
      --ns string            Specified namespace to enable addon (default "open-cluster-management-agent-addon")
      --output-file string   The generated resources will be copied in the specified file

Global Flags:
      --add-dir-header                   If true, adds the file directory to the header of the log messages
      --alsologtostderr                  log to standard error as well as files
      --as string                        Username to impersonate for the operation
      --as-group stringArray             Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
      --cache-dir string                 Default cache directory (default "/Users/llhu/.kube/cache")
      --certificate-authority string     Path to a cert file for the certificate authority
      --client-certificate string        Path to a client certificate file for TLS
      --client-key string                Path to a client key file for TLS
      --cluster string                   The name of the kubeconfig cluster to use
      --context string                   The name of the kubeconfig context to use
      --dry-run                          If set the generated resources will be displayed but not applied
      --insecure-skip-tls-verify         If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
      --kubeconfig string                Path to the kubeconfig file to use for CLI requests.
      --log-backtrace-at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
      --log-dir string                   If non-empty, write log files in this directory
      --log-file string                  If non-empty, use this log file
      --log-file-max-size uint           Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)
      --logtostderr                      log to standard error instead of files (default true)
      --match-server-version             Require server version to match client version
  -n, --namespace string                 If present, the namespace scope for this CLI request
      --one-output                       If true, only write logs to their native severity level (vs also writing to each lower severity level)
      --password string                  Password for basic authentication to the API server
      --request-timeout string           The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
  -s, --server string                    The address and port of the Kubernetes API server
      --skip-headers                     If true, avoid header prefixes in the log messages
      --skip-log-headers                 If true, avoid headers when opening log files
      --stderrthreshold severity         logs at or above this threshold go to stderr (default 2)
      --timeout int                      extend timeout from 300 secounds  (default 300)
      --tls-server-name string           Server name to use for server certificate validation. If it is not provided, the hostname used to contact the server is used
      --token string                     Bearer token for authentication to the API server
      --user string                      The name of the kubeconfig user to use
      --username string                  Username for basic authentication to the API server
  -v, --v Level                          number for the log level verbosity
      --vmodule moduleSpec               comma-separated list of pattern=N settings for file-filtered logging

Verify
llhu@HuLeis-MacBook-Pro clusteradm % kubectl get mcl
NAME     HUB ACCEPTED   MANAGED CLUSTER URLS      JOINED   AVAILABLE   AGE
llhu-1   true           https://127.0.0.1:50913   True     True        6m55s
llhu-2   true           https://127.0.0.1:50913   True     True        5m53s
llhu@HuLeis-MacBook-Pro clusteradm % bin/clusteradm install addons --names application-manager
Installing built-in application-manager add-on to the Hub cluster...
llhu@HuLeis-MacBook-Pro clusteradm % bin/clusteradm enable addons --names application-manager --clusters llhu-1
Deploying application-manager add-on to namespaces open-cluster-management-agent-addon of managed cluster: llhu-1.
llhu@HuLeis-MacBook-Pro clusteradm % bin/clusteradm enable addons --names application-manager --clusters llhu-1 --ns test
Deploying application-manager add-on to namespaces test of managed cluster: llhu-1.

@xauthulei it looks fine but test fails..
/approve

@xauthulei it looks fine but test fails..
/approve

@qiujian16 Fixed the test error, please review this again, Thanks

/lgtm

/hold

repo: open-cluster-management-io/clusteradm


text:
Change from app label to part-of label

https://github.com/open-cluster-management/backlog/issues/12201
Signed-off-by: Philip Wu phwu@redhat.com

/test images

/test images

/test /images

/test unit
/test images


text:
Steps in getting-started-with-ocm-flux incorrectly assume pwd

In
https://ocm.software/docs/guides/getting-started-with-ocm-flux/#setting-up-our-project
you are instructed to run
ocm add componentversions --create --file ./component-archive components.yaml
and then some output from
tree .
is shown
However if you run the above commands back to back the output doesn't match what is in the doc.   Basically the doc assumes that you first cd into component-archive

Hi @dee0sap
The documentation was updated, please could you confirm if you still observe discrepancies.
Cheers
Robert

repo: open-component-model/ocm-website


text:
Фінансові програми: Таблиця

name_ua
name_en
endpoint
path
notes
default state

Ідентифікатор програми
Identifier
ifiprojects
internal/code

visible

Назва програми
Title
ifiprojects
cdu_response/title/translation
WHERE cdu_response/title/lang = “uk”
visible

Опис програми
Description
ifiprojects
cdu_response/description/translation
WHERE cdu_response/description/lang = "uk"
hidden

Виконавець
Implementer
ifiprojects
cdu_response/parties/identitifier/legalName | cdu_response/parties/identifier/id
WHERE cdu_response/parties/roles LIKE "%implementer%"
visible

Юридичні підстави використання коштів
Legal basis
ifiprojects
cdu_response/budget/finance/legalBasis

hidden

Категорія фінансування
Type of funding
ifiprojects
cdu_response/budget/finance/financeType
Тут ми використовуємо той же кодліст, що і для approaches/budget/finance/financeType
visible

Категорія джерела фінансування
Type of funding source
ifiprojects
cdu_response/budget/finance/financeCategory
Тут ми використовуємо той же кодліст, що і для approaches/budget/finance/financeCategory
hidden

Сектор
Sector
ifiprojects
cdu_response/sector
IF number of character in cdu_response/sector = 1
visible

Підсектор
Subsector
ifiprojects
cdu_response/sector
IF number of character in cdu_response/sector = 2
hidden

Дата початку (програма)
Start date
ifiprojects
cdu_response/period/startDate

hidden

Дата завершення (програма)
End date
ifiprojects
cdu_response/period/endDate

hidden

Дата початку відборів
Selection start date
prequalifications
MIN cdu_response/prequalification/period/startDate
Тут потрібно повʼязати відбори (prequalifications) до фінпрограм через relatedProcesses. Оскільки у фінпрограми може бути декілька відборів, ми беремо найпершу / найбільш ранню дату початку відборів
visible

Дата завершення відборів
Selection end date
prequalifications
MAX cdu_response/prequalification/period/endDate
Тут потрібно повʼязати відбори (prequalifications) до фінпрограм через relatedProcesses. Оскільки у фінпрограми може бути декілька відборів, ми беремо останню / найбільш пізню дату завершення відборів
visible

Кількість отриманих заявок
Number of applications received
submissions
COUNT cdu_response/internal/code

visible

Кількість схвалених заявок
Number of applications selected
submissions
COUNT cdu_response/internal/code
WHERE cdu_response/status="active"
visible

Кількість проєктів підтримано
Number of projects
submissions, ideas

visible

Бюджет
Budget
ifiprojects
cdu_response/budget/value/amount

visible

Валюта бюджету
Currency
ifiprojects
cdu_response/budget/value/currency

visible

Естімейт - 16

уточнення - як рахується "Кількість проєктів підтримано" ?

Тут валідні ті ж самі коментарі, що і до карток #189 (там також є відповідь на питання, як рахується кількість підтриманих проєктів)

[Перейменування змінних]
Варто перейменувати стовпчики

Дата початку (програма) > Дата початку програми
Дата завершення (програма) > Дата завершення програми

in progress

done

[Кількість отриманих заявок]
У таблиці наразі відображається некоректна кількість отриманих заявок. У картці ми маємо 504 заявки, у таблиці - 82

done

Мені здається, ви виправили не там, де треба було
На сторінці "Фінансові програми, заявки" у нас 505 заявок. В АРІ у нас 505 заявок. Карта - до того, як ви внесли правки - відображала коректне значення, а таблиця - некоректне. Наразі і таблиця, і картка, і навіть сторінка "Фінансові програми, заявки" на деві відображають некоректну кількість заявок

take a look
чи правильно я розумію, що на сторінці "Фінансові програми" ми рахуємо тільки проєкти, які відносяться до фінпрограм.
а на сторінці "Фінансові програми, заявки" ми рахуємо всі заявки, навіть якщо є такі, що не відносяться ні до якої з  фінпрограм?

Гм, загалом у нас не мало бути заявок, які не привʼязані до фінпрограми, бо заявка це завжди заявка на конкретну фінпрограму. Але я бачу, що фактично зараз це можливо. Спробую розібратися, чи це проблема на рівні джерела даних, чи нашої логіки

@andrzejbeletsky задокументував проблему у новій задачі, подивіться будь ласка

authors: andrzejbeletsky ndrhzn


text:
Test & bugfix scrapers

[ ] armenia.py
[ ] australia.py
[ ] canada_buyandsell.py
[ ] canada_montreal.py
[ ] colombia.py
[ ] mexico_administracion_publica_federal.py
[ ] mexico_cdmx.py
[ ] mexico_grupo_aeroporto.py
[ ] mexico_inai.py
[ ] mexico_jalisco.py
[ ] moldova.py
[ ] paraguay_dncp.py
[ ] paraguay_hacienda.py
[ ] taiwan.py
[ ] uganda.py
[ ] uk_contracts_finder.py #103
[ ] ukraine.py #102
[ ] uruguay.py

Tick off above when checked or when broken out to a separate ticket (if we realise one is having big problems let's make it a separate ticket and try and keep this one relatively clean - they will probably be separate pull requests anyway!)

It's worth noting that the server we have for this has 12 cores and AFAIK at the very least the gather and fetch stages are independent, so we should be able to run quite of lot of these in parallel

#108 should contain a useful function in the DB class we can reuse but here's my first quick try at some SQL .... get the ID of the source_session row you want to delete. In one transaction run:
delete from release_check where release_id in (select release.id from release join source_session_file_status on source_session_file_status.id =  release.source_session_file_status_id where source_session_file_status.source_session_id = $id);

delete from record_check where record_id in (select record.id from record join source_session_file_status on source_session_file_status.id =  record.source_session_file_status_id where source_session_file_status.source_session_id = $id);

delete from record where source_session_file_status_id in (select id from source_session_file_status where source_session_id = $id);

delete from release where source_session_file_status_id in (select id from source_session_file_status where source_session_id = $id);

delete from source_session_file_status where source_session_id = $id;

delete from source_session where id = $id;

Then in one transaction per statement run:
delete from data where id not in (select data_id from release union select data_id from record);

delete from package_data where id not in (select package_data_id from release union select package_data_id from record);

Not sure how this will work at scale yet but we'll start by assuming the DB is better than what we could do and only look at optimizations later if we need to. We'll properly test and play as we do other work.

Have started a bunch by hand, most are in TMUX sessions:

taiwan
colombia
canada_montreal
canada_buyandsell
mexico_administracion_publica_federal
mexico_cdmx
mexico_grupo_aeroporto
mexico_inai
uruguay

Picked these ones because the number of files in each one is known in advance, and we know it's not insane (~4000 files is I think the biggest one). We'll check in later.

@tian2992 's up for getting involved with this as well

Just checking whether this branch can be deleted (it seems very similar code for a Columbia scraper is now part of the repo): https://github.com/open-contracting/ocdsdata/compare/colombia

@jpmckinney  Deleted

This is progressing well, the first batch has already shook out some problems that have all been either P.R.'d or commented on in other issues.
For now, I'm going to tick them off if the source can run a full run, including the new stage of "check". Maybe some hand analysis of the data should then be done to make sure it's sensible - that may uncover issues like the Ukraine issue, where it turned out we were downloading way more than we had to. But how to do that is unclear at this stage, and others have more skill than me at that.

Have tested out colombia, however I'm kinda concerned on the default download strategy as it is quite a significant load to extract several GB of data.. Deserves its own, separate discussion.

Have tested out colombia, however I'm kinda concerned on the default download strategy as it is quite a significant load to extract several GB of data.. Deserves its own, separate discussion.

Thanks! One question is, is there any sensible way that a user might want to say "only download X of Colmbia"? For instance, see #95 - it will be possible to only download a historical subset of Ukraine’s data in the future. Is there any easy variable that Colmbia's data can be divided by? If so, do make a separate issue and lets put that in!

BTW I am currently running Armenia and Australia on the ETL server. Australia is still going and Armenia has died with an error, so I can look into that now.

Currently for colombia's data there is either bulk or --sample, which covers only the first page: about 10k entries.

Ran mexico_administracion_publica_federal with no issues, now checking the resulted inputs on DB

For Moldova, regarding the code comment about invalid JSON, you just need to replace the last trailing comma. It's better to do that than hardcode (and then have to update the script every year), e.g.:
import json
import requests

response = requests.get('http://opencontracting.date.gov.md/multiple-file-api/releases.json')
data = json.loads(''.join(response.text.rsplit(',', 1)))

for url in data['links']['all']:
    # more code

For Uganda, I had noticed when scraping it in December 2017 that if the JSON contained a status_code (in which case, it also contained a message), then the response didn't contain data. We should check for this, and not emit data for that page if present.
I notice that the present scraper has a different filename pattern for samples and non-samples.

authors: jpmckinney odscjames robredpath tian2992

repo: open-contracting/ocdsdata


text:
Error message hides "start" button

In some screen resolutions when you get an error message the "start" button is not visible anymore.

I think that we can a) add a scroll if the content of the page doesn't fit in the page and/or b) make the "Open Contracting Partnership" footer smaller c) ensure that the site is responsive

Fixed in https://github.com/open-contracting/toucan/pull/96/commits/cf6e4c85eec53f2315e65c0c695d92af7a32d337. Waiting merge.

authors: aguilerapy yolile

repo: open-contracting/toucan


text:
Is there a way to hide the ip?

I can use Cloudflare's capabilities like WAF. But they will not be effective if the ip with the proxy field is visible in the repository. Is there a way to hide the ip?

I can use Cloudflare's capabilities like WAF. But they will not be effective if the ip with the proxy field is visible in the repository. Is there a way to hide the ip?

the only thing you can do is enable proxy

repo: open-domains/register


text:
Create ehab.is-cool.dev.json

Requirements

[x] You have completed your website.
[x] The website is reachable.
[x] The CNAME record doesn't contain https:// or /.
[x] There is sufficient information at the owner field.
[x] Your website is not hosting on the following due to SSL issues: Vercel, Netlify

Description
i made a website with nginx svelte kit and tailwind i want an extra domain to get a better understanding of nginx
Link to Website
helooo.com

CLA Assistant Lite bot: Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.

I have read the CLA Document and I hereby sign the CLA

You can retrigger this bot by commenting recheck in this Pull Request


text:
feat: flagd in-process evalator improvements

This PR
Improves in-process evaluator with,

Adding selector field for gRPC sync connection ^1
Allow unknown fields in flagd flag structure: future-proofing when flag sources change

Allow unknown fields in flagd flag structure: future-proofing when flag sources change

Probably a good idea.

authors: Kavindu-Dodan toddbaert

repo: open-feature/java-sdk-contrib


text:
Look into use unpkg for CDN use of web-sdk

We should probably add a CDN option at some point. For example, you can probably already use the SDK via unpkg. Just confirm this and add it to the web-sdk readme.

unpkg seems to be a CDN just for npm packages. Isn't the npm registry behind a CDN anyways?

authors: mowies toddbaert

repo: open-feature/js-sdk


text:
refactor!: remove kotlinx.serialization dependency

This PR

Removes the built in dependency on kotlinx.serialization from the android sdk and instead introduces a separate module that has that dependency.

Related Issues

Fixes #82
Notes

The idea is that the provider creator would pick their own serialization solution and not be forced to get a dependency on the kotlinx.serialization.
Follow-up Tasks

How to test

@nicklasl can we change the title though? we are only removing the serialization stuff here.

@nicklasl can we change the title though? we are only removing the serialization stuff here.

Good point. I chose refactor now. Not sure what is correct and also we want it in the changelog etc.

authors: nicklasl vahidlazio

repo: open-feature/kotlin-sdk


text:
sidecarConfiguration.resources in helm chart is ignored

Hello,
I am trying to use the operator and I am installing it via helm chart.
I am passing the following values.yaml file, but it doesn't seem to affect the default properties of the flagd sidecar.
Am I doing anything wrong?
sidecarConfiguration:
  resources:
    requests:
      cpu: 10m
      memory: 16M
    limits:
      cpu: 50m
      memory: 32M

USER-SUPPLIED VALUES:
sidecarConfiguration:
  port: 8078
  resources:
    limits:
      memory: 72M

COMPUTED VALUES:
controllerManager:
...
  resources:
    limits:
      memory: 72M
...

The helm template seems to be working, but the flagd sidecar doesn't seem to use these resources. The other customizations from the sidecarConfiguration seem to work (port, etc), so I think there's a bug here. I'm still investigating.

Thanks again @mng1dev . This is fixed with https://github.com/open-feature/open-feature-operator/pull/639 and will be release soon.

repo: open-feature/open-feature-operator


text:
Modified effective area format. EFFAREA_RECO column moved to a new HDU.

As discussed in issue #35 , I added some changes to the effective area format.
Essentially, I removed the  EFFAREA_RECO column, and described the effective area vs estimated energy as a separate HDU.
The format may be definitely improved. I'm not sure if the "aeff_2d format" should refer to both effective areas, or they are "different formats".
I also modified the python macro that generates the FITS file example, updating it.

@jknodlseder - Do you have time to review this?
From my side, I think the main comment would be that there should be a declarative way to tell which AEFF a given HDU is. I guess this means that the HDUCLAS2 header keyword should be different for the two types of AEFF?

Thanks @cdeil ,

From my side, I think the main comment would be that there should be a declarative way to tell which AEFF a given HDU is. I guess this means that the HDUCLAS2 header keyword should be different for the two types of AEFF?

As the HDUCLAS2 was not present there before, I decided to not include it. I will simply add it to both effective areas description.

Maybe it's OK to extend the description of this format to explain this briefly why they both exist and when to use which one as DL3 producer (i.e. roughly the short summary I wrote here)?

Ok, verbose mode on. I will commit those changes today.

Ok, done. I tried to implement all comments. I also re-structured the text a bit. Let me know if you think I should change anything else.

@TarekHC - Thanks!
Assigning @jknodlseder for review / decision.
I'm going on holiday for two weeks and am out of time.
Feel free to merge, don't wait for comments from me.

@cdeil @jknodlseder , we are all extremely busy with Kashiwa... So there is really no rush. Just review it whenever you find the time.

@TarekHC - Can you please rebase this pull request against master? (Github shows that it can't be merged because there is a conflict).
@jknodlseder - Do you have time to review this PR this week?
From my side it's OK ... the main point is that it splits EFF_AREA_RECO in a separate HDU now and there are HDUCLAS2 header keywords to declare what kind of response it is.

Yes, no problem. I'll take care of that. Although I will probably wait for jurgen's feedback first, as new conflicts may arise.

@TarekHC - Can you please rebase this PR now?
Background info: I'd like to merge this (and the other open PRs) now, so that further work on the spec isn't blocked. We'd like to clean up several details for the next HESS FITS production that will then be used for the test data release.

Oh, sure. I'll try to do it today then. Cheers!

Ok, I think the conflicts were solved.

authors: TarekHC cdeil


text:
Download NHSbuntu

Create  _includes snippet with CTA button linking to latest ISO in S3 bucket. Could be variable set in config.yml?

Closed with #5


text:
Populate report-an-issue.md

Populate the report-an-issue.md file with detailed on the proper way to open an issue for this project in GitHub.  At least initially, let's limit the scope to reporting an issue with the documentation.  Later on we'll expand it for reporting an issue with any code in the project.

Hi @joewxboy is this issue up for grabs?

@clementkng As far as I know, it's open.  You commented first!  I'll assign to you.

authors: clementkng joewxboy


text:
Organize - Open-Insurance Day - 01

https://open-insurance.org/events/2019/open-insurance-day-00.html

elevator advert
open-insurance-day-01.docx
open-insurance-day-01.pdf

repo: open-insurance/events


text:
Link to share advanced search results

Is your feature request related to a problem? Please describe.
I applied advanced search in UI and want to send a link on found objects to my colleagues.
I copied URL, but if my colleague will open link, advanced search will not be applied.
Describe the solution you'd like
Opportunity to share advanced search results with links.
Describe alternatives you've considered
No alternatives
Additional context
This is an important issue when it comes to communication about sets of data entities. For example, I want data owner to renew descriptions of his tier1 data assets, but instead of sharing link I have to instruct him, how to find assets with advanced search.

@gnomolio - which version of OMD are you using ?
I tried in 1.0 and it seems to be working fine for me.
Also if you can provide more information on how to reproduce it or maybe a video will be great.

@gnomolio Closing this hence unable to replicate with our latest release. Feel free to re-open or reach out if you face the same

authors: chirag-madlani gnomolio karanh37

repo: open-metadata/OpenMetadata


text:
Fixing page switching in the policy section

Affected module
UI
Describe the bug
Good afternoon!
In the policy view section, when switching from one page to another, the page counter is lost. It is impossible to view the policies on the pages further than the second one.
To Reproduce

Expected behavior
Switching between pages in the policy section should be fine.
Version:

OpenMetadata version: 1.1.7

Hi @AlekseevVadim,
We have verified same on our end and not able to reproduce the same. Feel free to re-open with additional info like network or console if you are still facing this issue.
Verified with 1.2 release

authors: AlekseevVadim chirag-madlani


text:
Usage information is missing from Search/Explore page

previously

Fix has been merged #1743

authors: Sachin-chaurasiya harshach


text:
Airflow CI failing - Operational error

Airflow Apis Test workflow fails with this
   if len(data) < num_bytes:
              self._force_close()
  >           raise err.OperationalError(
                  CR.CR_SERVER_LOST, "Lost connection to MySQL server during query"
              )
  E           sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query')
  E           (Background on this error at: https://sqlalche.me/e/14/e3q8)

  env/lib/python3.9/site-packages/pymysql/connections.py:748: OperationalError

ref: https://github.com/open-metadata/OpenMetadata/actions/runs/3692216655/jobs/6250888697#step:14:644

this I've seen with Airflow. Our Airflow container loses connection periodically to the mysql. Let's look at the default values connection settings on Airflow. Our DB container can get busy during the ingestion as we are writing lot of data. So we can increase the timeout values on the airflow side

Hello ! is this issue still a thing ? we are having this error when deploying an ingestion with bigquery and dbt from gcp buckets, did you manage to get rid of it ? (we are using your helm chart)

Hi @caupetit-itf thanks for reaching out. We're looking into this. We'll update the issue with our findings. Thanks!

Looks like setting SET GLOBAL net_read_timeout = 600; in MySQL got it working for some users. We're validating the approach as well and will change the db image if needed.
Thanks

authors: ayush-shah caupetit-itf harshach pmbrull


text:
bump(ui) : antd@4.20.6 to antd@4.24.0

Describe your changes :

I worked on bumping the antd version from 4.20.6 to 4.24.0 and also made required types changes for the tree and input number component.

Type of change :

[x] Improvement

Frontend Preview (Screenshots) :

Checklist:

[x] I have read the CONTRIBUTING document.
[x] I have commented on my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] All new and existing tests passed.

Reviewers
@open-metadata/ui

Substitute PR https://github.com/open-metadata/OpenMetadata/pull/9173


text:
INSTALL ERROR: python3 setup.py develop

I checkout the branch to v0.5.0, then run 'python3 setup.py develop'. error:
[1/7] c++ -MMD -MF /data/hyper/code/OpenPCDet/build/temp.linux-x86_64-cpython-38/pcdet/ops/pointnet2/pointnet2_stack/src/voxel_query.o.d -pthread -B /home/hyper/anaconda3/envs/pcdet/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include/TH -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include/THC -I/usr/local/cuda-11.3/include -I/home/hyper/anaconda3/envs/pcdet/include/python3.8 -c -c /data/hyper/code/OpenPCDet/pcdet/ops/pointnet2/pointnet2_stack/src/voxel_query.cpp -o /data/hyper/code/OpenPCDet/build/temp.linux-x86_64-cpython-38/pcdet/ops/pointnet2/pointnet2_stack/src/voxel_query.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1013"' -DTORCH_EXTENSION_NAME=pointnet2_stack_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
FAILED: /data/hyper/code/OpenPCDet/build/temp.linux-x86_64-cpython-38/pcdet/ops/pointnet2/pointnet2_stack/src/voxel_query.o
c++ -MMD -MF /data/hyper/code/OpenPCDet/build/temp.linux-x86_64-cpython-38/pcdet/ops/pointnet2/pointnet2_stack/src/voxel_query.o.d -pthread -B /home/hyper/anaconda3/envs/pcdet/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include/TH -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include/THC -I/usr/local/cuda-11.3/include -I/home/hyper/anaconda3/envs/pcdet/include/python3.8 -c -c /data/hyper/code/OpenPCDet/pcdet/ops/pointnet2/pointnet2_stack/src/voxel_query.cpp -o /data/hyper/code/OpenPCDet/build/temp.linux-x86_64-cpython-38/pcdet/ops/pointnet2/pointnet2_stack/src/voxel_query.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1013"' -DTORCH_EXTENSION_NAME=pointnet2_stack_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
/data/hyper/code/OpenPCDet/pcdet/ops/pointnet2/pointnet2_stack/src/voxel_query.cpp:3:10: fatal error: THC/THC.h: No such file or directory
#include <THC/THC.h>
^~~~~~~~~~~
compilation terminated.
[2/7] c++ -MMD -MF /data/hyper/code/OpenPCDet/build/temp.linux-x86_64-cpython-38/pcdet/ops/pointnet2/pointnet2_stack/src/ball_query.o.d -pthread -B /home/hyper/anaconda3/envs/pcdet/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include/TH -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include/THC -I/usr/local/cuda-11.3/include -I/home/hyper/anaconda3/envs/pcdet/include/python3.8 -c -c /data/hyper/code/OpenPCDet/pcdet/ops/pointnet2/pointnet2_stack/src/ball_query.cpp -o /data/hyper/code/OpenPCDet/build/temp.linux-x86_64-cpython-38/pcdet/ops/pointnet2/pointnet2_stack/src/ball_query.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1013"' -DTORCH_EXTENSION_NAME=pointnet2_stack_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
FAILED: /data/hyper/code/OpenPCDet/build/temp.linux-x86_64-cpython-38/pcdet/ops/pointnet2/pointnet2_stack/src/ball_query.o
c++ -MMD -MF /data/hyper/code/OpenPCDet/build/temp.linux-x86_64-cpython-38/pcdet/ops/pointnet2/pointnet2_stack/src/ball_query.o.d -pthread -B /home/hyper/anaconda3/envs/pcdet/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include/TH -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include/THC -I/usr/local/cuda-11.3/include -I/home/hyper/anaconda3/envs/pcdet/include/python3.8 -c -c /data/hyper/code/OpenPCDet/pcdet/ops/pointnet2/pointnet2_stack/src/ball_query.cpp -o /data/hyper/code/OpenPCDet/build/temp.linux-x86_64-cpython-38/pcdet/ops/pointnet2/pointnet2_stack/src/ball_query.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1013"' -DTORCH_EXTENSION_NAME=pointnet2_stack_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
/data/hyper/code/OpenPCDet/pcdet/ops/pointnet2/pointnet2_stack/src/ball_query.cpp:10:10: fatal error: THC/THC.h: No such file or directory
#include <THC/THC.h>
^~~~~~~~~~~
compilation terminated.
[3/7] c++ -MMD -MF /data/hyper/code/OpenPCDet/build/temp.linux-x86_64-cpython-38/pcdet/ops/pointnet2/pointnet2_stack/src/interpolate.o.d -pthread -B /home/hyper/anaconda3/envs/pcdet/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include/TH -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include/THC -I/usr/local/cuda-11.3/include -I/home/hyper/anaconda3/envs/pcdet/include/python3.8 -c -c /data/hyper/code/OpenPCDet/pcdet/ops/pointnet2/pointnet2_stack/src/interpolate.cpp -o /data/hyper/code/OpenPCDet/build/temp.linux-x86_64-cpython-38/pcdet/ops/pointnet2/pointnet2_stack/src/interpolate.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1013"' -DTORCH_EXTENSION_NAME=pointnet2_stack_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
FAILED: /data/hyper/code/OpenPCDet/build/temp.linux-x86_64-cpython-38/pcdet/ops/pointnet2/pointnet2_stack/src/interpolate.o
c++ -MMD -MF /data/hyper/code/OpenPCDet/build/temp.linux-x86_64-cpython-38/pcdet/ops/pointnet2/pointnet2_stack/src/interpolate.o.d -pthread -B /home/hyper/anaconda3/envs/pcdet/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include/TH -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include/THC -I/usr/local/cuda-11.3/include -I/home/hyper/anaconda3/envs/pcdet/include/python3.8 -c -c /data/hyper/code/OpenPCDet/pcdet/ops/pointnet2/pointnet2_stack/src/interpolate.cpp -o /data/hyper/code/OpenPCDet/build/temp.linux-x86_64-cpython-38/pcdet/ops/pointnet2/pointnet2_stack/src/interpolate.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1013"' -DTORCH_EXTENSION_NAME=pointnet2_stack_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
/data/hyper/code/OpenPCDet/pcdet/ops/pointnet2/pointnet2_stack/src/interpolate.cpp:10:10: fatal error: THC/THC.h: No such file or directory
#include <THC/THC.h>
^~~~~~~~~~~
compilation terminated.
[4/7] c++ -MMD -MF /data/hyper/code/OpenPCDet/build/temp.linux-x86_64-cpython-38/pcdet/ops/pointnet2/pointnet2_stack/src/group_points.o.d -pthread -B /home/hyper/anaconda3/envs/pcdet/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include/TH -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include/THC -I/usr/local/cuda-11.3/include -I/home/hyper/anaconda3/envs/pcdet/include/python3.8 -c -c /data/hyper/code/OpenPCDet/pcdet/ops/pointnet2/pointnet2_stack/src/group_points.cpp -o /data/hyper/code/OpenPCDet/build/temp.linux-x86_64-cpython-38/pcdet/ops/pointnet2/pointnet2_stack/src/group_points.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1013"' -DTORCH_EXTENSION_NAME=pointnet2_stack_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
FAILED: /data/hyper/code/OpenPCDet/build/temp.linux-x86_64-cpython-38/pcdet/ops/pointnet2/pointnet2_stack/src/group_points.o
c++ -MMD -MF /data/hyper/code/OpenPCDet/build/temp.linux-x86_64-cpython-38/pcdet/ops/pointnet2/pointnet2_stack/src/group_points.o.d -pthread -B /home/hyper/anaconda3/envs/pcdet/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include/TH -I/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/include/THC -I/usr/local/cuda-11.3/include -I/home/hyper/anaconda3/envs/pcdet/include/python3.8 -c -c /data/hyper/code/OpenPCDet/pcdet/ops/pointnet2/pointnet2_stack/src/group_points.cpp -o /data/hyper/code/OpenPCDet/build/temp.linux-x86_64-cpython-38/pcdet/ops/pointnet2/pointnet2_stack/src/group_points.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1013"' -DTORCH_EXTENSION_NAME=pointnet2_stack_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
/data/hyper/code/OpenPCDet/pcdet/ops/pointnet2/pointnet2_stack/src/group_points.cpp:12:10: fatal error: THC/THC.h: No such file or directory
#include <THC/THC.h>
^~~~~~~~~~~
compilation terminated.
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "/home/hyper/anaconda3/envs/pcdet/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1808, in _run_ninja_build
subprocess.run(
File "/home/hyper/anaconda3/envs/pcdet/lib/python3.8/subprocess.py", line 512, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

https://github.com/open-mmlab/OpenPCDet/issues/1668#issuecomment-2376843239

repo: open-mmlab/OpenPCDet


text:
run pytorch2onnx.py error to visualize model  sturcture.

it's better if could use tensorboar to visualize the model architecture.
but it have some bug , #2383   #3239.
So I use the pytorch2onnx ,then visiualize the model sturcture use netron.
some problem occurs whwn convert to onnx.
Reproduction

What command or script did you run?

python tools/pytorch2onnx.py ./configs/mask_rcnn/mask_rcnn_r50_fpn_1x_coco.py ~/Downloads/mask_rcnn_r50_fpn_1x_coco_20200205-d4b0c5d6.pth

Environment
ubuntu 18.04 , pytorch 1.5 ,cuda 10.2
Error traceback
If applicable, paste the error trackback here.
(torch1.5) fangxu@fangxu:/media/fangxu/Segate3T/PHD-Project/linux-3rd/mmdetection$ python tools/pytorch2onnx.py ./configs/mask_rcnn/mask_rcnn_r50_fpn_1x_coco.py ~/Downloads/mask_rcnn_r50_fpn_1x_coco_20200205-d4b0c5d6.pth
/media/fangxu/Segate3T/PHD-Project/linux-3rd/mmdetection/mmdet/models/detectors/base.py:143: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert samples_per_gpu == 1
/media/fangxu/Segate3T/PHD-Project/linux-3rd/mmdetection/mmdet/core/anchor/anchor_generator.py:255: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  feat_h = int(feat_h)
/media/fangxu/Segate3T/PHD-Project/linux-3rd/mmdetection/mmdet/core/anchor/anchor_generator.py:256: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  feat_w = int(feat_w)
/media/fangxu/Segate3T/PHD-Project/linux-3rd/mmdetection/mmdet/core/anchor/anchor_generator.py:199: TracerWarning: Converting a tensor to a Python index might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  xx = x.repeat(len(y))
/media/fangxu/Segate3T/PHD-Project/linux-3rd/mmdetection/mmdet/core/anchor/anchor_generator.py:200: TracerWarning: Converting a tensor to a Python index might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  yy = y.view(-1, 1).repeat(1, len(x)).view(-1)
/media/fangxu/Segate3T/PHD-Project/linux-3rd/mmdetection/mmdet/models/dense_heads/rpn_head.py:119: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert rpn_cls_score.size()[-2:] == rpn_bbox_pred.size()[-2:]
/media/fangxu/Segate3T/PHD-Project/linux-3rd/mmdetection/mmdet/models/dense_heads/rpn_head.py:132: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if cfg.nms_pre > 0 and scores.shape[0] > cfg.nms_pre:
/media/fangxu/Segate3T/PHD-Project/linux-3rd/mmdetection/mmdet/core/bbox/coder/delta_xywh_bbox_coder.py:67: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert pred_bboxes.size(0) == bboxes.size(0)
/media/fangxu/Segate3T/PHD-Project/linux-3rd/mmcv/mmcv/ops/nms.py:82: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert boxes.size(1) == 4
/media/fangxu/Segate3T/PHD-Project/linux-3rd/mmcv/mmcv/ops/nms.py:83: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert boxes.size(0) == scores.size(0)
/media/fangxu/Segate3T/PHD-Project/linux-3rd/mmdetection/mmdet/core/bbox/transforms.py:66: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if bboxes.size(0) > 0:
/media/fangxu/Segate3T/PHD-Project/linux-3rd/mmdetection/mmdet/models/roi_heads/roi_extractors/single_level_roi_extractor.py:74: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if inds.any():
/media/fangxu/Segate3T/PHD-Project/linux-3rd/mmcv/mmcv/ops/roi_align.py:45: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert rois.size(1) == 5, 'RoI must be (idx, x1, y1, x2, y2)!'
/media/fangxu/Segate3T/PHD-Project/linux-3rd/mmdetection/mmdet/core/post_processing/bbox_nms.py:31: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if multi_bboxes.shape[1] > 4:
/media/fangxu/Segate3T/PHD-Project/linux-3rd/mmdetection/mmdet/core/bbox/transforms.py:106: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if bboxes.shape[0] == 0:
/media/fangxu/Segate3T/PHD-Project/linux-3rd/mmdetection/mmdet/core/bbox/transforms.py:110: TracerWarning: Converting a tensor to a NumPy array might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  bboxes = bboxes.cpu().detach().numpy()
/media/fangxu/Segate3T/PHD-Project/linux-3rd/mmdetection/mmdet/core/bbox/transforms.py:111: TracerWarning: Converting a tensor to a NumPy array might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  labels = labels.cpu().detach().numpy()
/media/fangxu/Segate3T/PHD-Project/linux-3rd/mmdetection/mmdet/models/roi_heads/test_mixins.py:161: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if det_bboxes.shape[0] == 0:
/media/fangxu/Segate3T/PHD-Project/linux-3rd/mmdetection/mmdet/models/roi_heads/mask_heads/fcn_mask_head.py:191: TracerWarning: Converting a tensor to a Python index might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  N = len(mask_pred)
/home/fangxu/anaconda3/envs/torch1.5/lib/python3.6/site-packages/torch/tensor.py:467: RuntimeWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  'incorrect results).', category=RuntimeWarning)
/media/fangxu/Segate3T/PHD-Project/linux-3rd/mmdetection/mmdet/models/roi_heads/mask_heads/fcn_mask_head.py:291: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if torch.isinf(img_x).any():
/media/fangxu/Segate3T/PHD-Project/linux-3rd/mmdetection/mmdet/models/roi_heads/mask_heads/fcn_mask_head.py:294: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if torch.isinf(img_y).any():
/media/fangxu/Segate3T/PHD-Project/linux-3rd/mmdetection/mmdet/models/roi_heads/mask_heads/fcn_mask_head.py:233: TracerWarning: Converting a tensor to a Python index might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  im_mask[(inds, ) + spatial_inds] = masks_chunk
/media/fangxu/Segate3T/PHD-Project/linux-3rd/mmdetection/mmdet/models/roi_heads/mask_heads/fcn_mask_head.py:236: TracerWarning: Converting a tensor to a Python index might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  cls_segms[labels[i]].append(im_mask[i].cpu().numpy())
/media/fangxu/Segate3T/PHD-Project/linux-3rd/mmdetection/mmdet/models/roi_heads/mask_heads/fcn_mask_head.py:236: TracerWarning: Converting a tensor to a NumPy array might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  cls_segms[labels[i]].append(im_mask[i].cpu().numpy())
Traceback (most recent call last):
  File "tools/pytorch2onnx.py", line 166, in <module>
    normalize_cfg=normalize_cfg)
  File "tools/pytorch2onnx.py", line 58, in pytorch2onnx
    opset_version=opset_version)
  File "/home/fangxu/anaconda3/envs/torch1.5/lib/python3.6/site-packages/torch/onnx/__init__.py", line 168, in export
    custom_opsets, enable_onnx_checker, use_external_data_format)
  File "/home/fangxu/anaconda3/envs/torch1.5/lib/python3.6/site-packages/torch/onnx/utils.py", line 69, in export
    use_external_data_format=use_external_data_format)
  File "/home/fangxu/anaconda3/envs/torch1.5/lib/python3.6/site-packages/torch/onnx/utils.py", line 488, in _export
    fixed_batch_size=fixed_batch_size)
  File "/home/fangxu/anaconda3/envs/torch1.5/lib/python3.6/site-packages/torch/onnx/utils.py", line 334, in _model_to_graph
    graph, torch_out = _trace_and_get_graph_from_model(model, args, training)
  File "/home/fangxu/anaconda3/envs/torch1.5/lib/python3.6/site-packages/torch/onnx/utils.py", line 291, in _trace_and_get_graph_from_model
    torch.jit._get_trace_graph(model, args, _force_outplace=False, _return_inputs_states=True)
  File "/home/fangxu/anaconda3/envs/torch1.5/lib/python3.6/site-packages/torch/jit/__init__.py", line 278, in _get_trace_graph
    outs = ONNXTracedModule(f, _force_outplace, return_inputs, _return_inputs_states)(*args, **kwargs)
  File "/home/fangxu/anaconda3/envs/torch1.5/lib/python3.6/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/fangxu/anaconda3/envs/torch1.5/lib/python3.6/site-packages/torch/jit/__init__.py", line 361, in forward
    self._force_outplace,
  File "/home/fangxu/anaconda3/envs/torch1.5/lib/python3.6/site-packages/torch/jit/__init__.py", line 351, in wrapper
    out_vars, _ = _flatten(outs)
RuntimeError: Only tuples, lists and Variables supported as JIT inputs/outputs. Dictionaries and strings are also accepted but their usage is not recommended. But got unsupported type numpy.ndarray

pytorch2onnx does not support two-stage detectors yet, and it has been updated a few days ago.

Oh, thanks , I was surprised by the speed of the response . Are there other ways to visualize models structure when we use the mmdetction ？

you can delete the .numpy()

Oh, thanks , I was surprised by the speed of the response .that's owsome. Are there other ways to visualize models structure when we use the mmdetction ？

Although exporting to ONNX for two-stage detectors has not been supported yet, it should not be hard to achieve it by modifying some codes.

you can delete the .numpy()
delete .numpy() cause "Exporting the operator new_full to ONNX opset version 11 is not supported". Tensor slice may cause new_full operator....

you can delete the .numpy()
delete .numpy() cause "Exporting the operator new_full to ONNX opset version 11 is not supported". Tensor slice may cause new_full operator....

onnx not support new_full, you can use new_ones()*scalar instead of new_full

有老铁解决了么：AttributeError: 'ConvModule' object has no attribute 'norm'

有老铁解决了么：AttributeError: 'ConvModule' object has no attribute 'norm'

has solution?

authors: fangxu622 ghoshaw hellock moerbenkaola qiubai521 xizi

repo: open-mmlab/mmdetection