text:
add test vectors for metafeed managment messages

This adds a new JSON file similar to the simple one but using actual metafeed managment messages, so metafeed/add and metafeed/tombstone.
KeyPairSeed and EncodedData are byte arrays encoded as hex.
The KeyPairSeed are the 32bytes that created the authors keypair and should thus lead to determinisitc data when re-creating these messages (together with the zero timestamps). With the KeyPairSeed and the nonces in the Metadata section, people should be able to recreate all the posts.
HighlevelContent is an expression of what should be found in the content. Slight deviation here is Seq:1. In that case go's json encoder used base64 to represent a byte array (or buffer in JS).
Here is a pretty printed version of the checked in file:
{
  "Description": "A bunch of metafeed related messages with double signed posts. Two feeds get added and one of them gets revoked/tombstoned",
  "KeyPairSeed": "7365633073656330736563307365633073656330736563307365633073656330",
  "Author": "@TuD7hkArcXo7xIQP4R9zAbjfRKNlmHrDoTXsDgVJ9wQ=.bbfeed-v1",
  "Entries": [
    {
      "EncodedData": "6c6c33343a00024ee0fb86402b717a3bc4840fe11f7301b8df44a365987ac3a135ec0e0549f70469316533343a010200000000000000000000000000000000000000000000000000000000000000006930656c6431303a66656564666f726d6174393a0600636c617373696331313a66656564707572706f736531343a06006d61696e2064656661756c74383a6d6574616665656433343a00024ee0fb86402b717a3bc4840fe11f7301b8df44a365987ac3a135ec0e0549f704353a6e6f6e636533323a2323232323232323232323232323232323232323232323232323232323232323373a7375626665656433343a00005d6429badb2f0d9fd92ef3401cf46c4cc1bfd359de699d1cdedc0330e4f9a7e0373a74616e676c657364383a6d6574616665656464383a70726576696f7573323a0602343a726f6f74323a06026565343a7479706531343a06006d657461666565642f6164646536363a0400f722fc3c2a3c6fbe551f6b510a7fb7d1b818e015f546cb79c6dc1774709e9f0ec4fae7d17abfa9ed5f115d22a77e4783859e6e81e6d85f4f646d1886cd9db40d656536363a04003f5ae4afcede7f7df453d705bbe54a3c3b4d99bfeab66d9d3740f946b36feac58005d3822bda084f5c5c641f5240ec3e00db3af2bf8834e4d77d4828c35a960265",
      "Key": "%hyMujD3zNMqwMm8QRru16u59RYRCsf/nUh+khfs/9Gk=.bbmsg-v1",
      "Sequence": 1,
      "Previous": "%AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=.bbmsg-v1",
      "Timestamp": 0,
      "HighlevelContent": [
        {
          "type": "metafeed/add",
          "feedformat": "classic",
          "feedpurpose": "main default",
          "subfeed": "@XWQputsvDZ/ZLvNAHPRsTMG/01neaZ0c3twDMOT5p+A=.ed25519",
          "metafeed": "@TuD7hkArcXo7xIQP4R9zAbjfRKNlmHrDoTXsDgVJ9wQ=.bbfeed-v1",
          "nonce": "IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyM=",
          "tangles": {
            "metafeed": {
              "root": null,
              "previous": null
            }
          }
        },
        {
          "Name": "subfeed signature",
          "HexString": "0400f722fc3c2a3c6fbe551f6b510a7fb7d1b818e015f546cb79c6dc1774709e9f0ec4fae7d17abfa9ed5f115d22a77e4783859e6e81e6d85f4f646d1886cd9db40d"
        }
      ]
    },
    {
      "EncodedData": "6c6c33343a00024ee0fb86402b717a3bc4840fe11f7301b8df44a365987ac3a135ec0e0549f70469326533343a010287232e8c3df334cab0326f1046bbb5eaee7d458442b1ffe7521fa485fb3ff4696930656c6431303a66656564666f726d6174393a0600636c617373696331313a66656564707572706f736531343a06006d61696e2064656661756c74383a6d6574616665656433343a00024ee0fb86402b717a3bc4840fe11f7301b8df44a365987ac3a135ec0e0549f704353a6e6f6e636533323a2323232323232323232323232323232323232323232323232323232323232323373a7375626665656433343a00005d6429badb2f0d9fd92ef3401cf46c4cc1bfd359de699d1cdedc0330e4f9a7e0373a74616e676c657364383a6d6574616665656464383a70726576696f7573323a0602343a726f6f74323a06026565343a7479706531343a06006d657461666565642f6164646536363a0400f722fc3c2a3c6fbe551f6b510a7fb7d1b818e015f546cb79c6dc1774709e9f0ec4fae7d17abfa9ed5f115d22a77e4783859e6e81e6d85f4f646d1886cd9db40d656536363a040019fd80c347e3d2c8be8c16c4b23a9e466230752c593d62471d7df0e3209a51560c99e0e983c7975260faaede65cae13c88a6a65089c3bb4bea186f1cfbfb760865",
      "Key": "%IYlK2/xauMXhrpz6ciAxXyxXQ221DoISUh3U9SY17w4=.bbmsg-v1",
      "Sequence": 2,
      "Previous": "%hyMujD3zNMqwMm8QRru16u59RYRCsf/nUh+khfs/9Gk=.bbmsg-v1",
      "Timestamp": 0,
      "HighlevelContent": [
        {
          "type": "metafeed/add",
          "feedformat": "gabby-grove-v1",
          "feedpurpose": "experimental",
          "subfeed": "@xQrDhN1BG0ozoCeH+So6X6MMI5pTmr0hz8jHlU/BUXA=.ggfeed-v1",
          "metafeed": "@TuD7hkArcXo7xIQP4R9zAbjfRKNlmHrDoTXsDgVJ9wQ=.bbfeed-v1",
          "nonce": "QkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkI=",
          "tangles": {
            "metafeed": {
              "root": null,
              "previous": null
            }
          }
        },
        {
          "Name": "subfeed2 signature",
          "HexString": "0400e263b158c5f2b49c1b0adf604086f26577c91c1db97d8cb56c5dffd9da14f0a713c640331ec170c63691278ffb34bfe03eb7b448218ae8549a7faaf5b399e90c"
        }
      ]
    },
    {
      "EncodedData": "6c6c33343a00024ee0fb86402b717a3bc4840fe11f7301b8df44a365987ac3a135ec0e0549f70469336533343a010221894adbfc5ab8c5e1ae9cfa7220315f2c57436db50e8212521dd4f52635ef0e6930656c64373a7375626665656433343a00005d6429badb2f0d9fd92ef3401cf46c4cc1bfd359de699d1cdedc0330e4f9a7e0373a74616e676c657364383a6d6574616665656464383a70726576696f75736c33343a010287232e8c3df334cab0326f1046bbb5eaee7d458442b1ffe7521fa485fb3ff46965343a726f6f7433343a010287232e8c3df334cab0326f1046bbb5eaee7d458442b1ffe7521fa485fb3ff4696565343a7479706532303a06006d657461666565642f746f6d6273746f6e656536363a0400dfe8b973d88123725e4f27b6e6e0e7a81e6b52d1ecdbad6d3a5ce9d40ea282f56ac45f4bd42feead6cf9e476b1185949db7093af6ceeeb870030152aa37a830f656536363a040025203d3b82720bef2feac3c32cfddf7db746918675c1d8c1fce04b2aaa18e985b3446033b8dfd9f7192455bdcda4f85e7e18c70ef3d99e40908a4d3938be290c65",
      "Key": "%/1SC1QQmjUs1yyIuOgqm5OPLvez2+pgQyXhTmfKNllQ=.bbmsg-v1",
      "Sequence": 3,
      "Previous": "%IYlK2/xauMXhrpz6ciAxXyxXQ221DoISUh3U9SY17w4=.bbmsg-v1",
      "Timestamp": 0,
      "HighlevelContent": [
        {
          "type": "metafeed/tombstone",
          "subfeed": "@XWQputsvDZ/ZLvNAHPRsTMG/01neaZ0c3twDMOT5p+A=.ed25519",
          "tangles": {
            "metafeed": {
              "root": "%hyMujD3zNMqwMm8QRru16u59RYRCsf/nUh+khfs/9Gk=.bbmsg-v1",
              "previous": [
                "%hyMujD3zNMqwMm8QRru16u59RYRCsf/nUh+khfs/9Gk=.bbmsg-v1"
              ]
            }
          }
        },
        {
          "Name": "subfeed1 signature",
          "HexString": "0400dfe8b973d88123725e4f27b6e6e0e7a81e6b52d1ecdbad6d3a5ce9d40ea282f56ac45f4bd42feead6cf9e476b1185949db7093af6ceeeb870030152aa37a830f"
        }
      ]
    }
  ],
  "Metadata": [
    {
      "Name": "subfeed1 nonce",
      "HexString": "2323232323232323232323232323232323232323232323232323232323232323"
    },
    {
      "Name": "subfeed author",
      "Feed": "@XWQputsvDZ/ZLvNAHPRsTMG/01neaZ0c3twDMOT5p+A=.ed25519"
    },
    {
      "Name": "subfeed2 nonce",
      "HexString": "4242424242424242424242424242424242424242424242424242424242424242"
    },
    {
      "Name": "subfeed2 author",
      "Feed": "@xQrDhN1BG0ozoCeH+So6X6MMI5pTmr0hz8jHlU/BUXA=.ggfeed-v1"
    }
  ]
}

Here is a json-schema description of the data:
{
  "$schema": "https://json-schema.org/draft/2019-09/schema",
  "$id": "https://github.com/ssb-ngi-pointer/ssb-meta-feed-spec#vectors",
  "type": "object",
  "properties": {
    "Description": {
      "description": "Freeform description of the contents of the vector file",
      "type": "string"
    },
    "KeyPairSeed": {
      "description": "the seed used as input for the ed25519 keypair generation to create the author",
      "type": "string",
      "pattern": "^[0-9a-f]+$"
    },
    "Author": {
      "description": "the authors public key, as an ssb-ref",
      "type": "string",
      "pattern": "^@[0-9a-zA-Z]+=.bbfeed-v1$"
    },
    "Metadata": {
      "description": "addtional metadata needed to create the Entries",
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "Name": {
            "type": "string",
            "description": "what this piece of metadata contains"
          },
          "HexString": {
            "type": "string",
            "description": "a string of hexadecimal characters, representing some binary data",
            "pattern": "^[0-9a-f]+$"
          },
          "Feed": {
            "description": "addtional feed references/public key, as ssb-ref",
            "type": "string",
            "pattern": "^@[0-9a-zA-Z]+"
          }
        }
      }
    },
    "Entries": {
      "description": "the entries on a the example feed",
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "EncodedData": {
            "type": "string",
            "description": "the bencoded entry as a string of hexadecimal characters, representing binary data",
            "pattern": "^[0-9a-f]+$"
          },
          "Sequence": {
            "type": "integer",
            "description": "the sequence number of that entry"
          },
          "Previous": {
            "description": "the previous message hash, as a ssb-ref (its encoded as TFK when bencoded)",
            "type": "string",
            "pattern": "^%[0-9a-zA-Z+/]+=.bbmsg-v1$"
          },
          "Timestamp": {
            "type": "integer"
          },
          "HighlevelContent": {
            "description": "this contains a JSON version of the bencoded data, for a highlevel comparision. They all follow the same schema tho. The 1st object in the array is the content portion. The 2nd object is containing the signature bytes, created with the subfeeds keypair"
          }
        }
      }
    }
  }
}

repo: ssb-ngi-pointer/go-metafeed


text:
Feature/inspector generic names

Systems/Components with generic type parameters correctly display the full type name in the inspector.

Systems/Components implementing ICustomDisplayName show in the inspector with the user-provided name.

This is what generic names currently look like.

@sschmid Please merge this PR. It's really awesome feature.

source: gharchive/pull-request

authors: jeffvella optimisez

repo: sschmid/Entitas-CSharp

license: mit

license_type: permissive

license_source: bigquery


text:
Update CITATION.cff - include iain maitland

:)

👍 Needs to also go in https://github.com/ssec-jhu/base-template/blob/main/.zenodo.json.

👍 Needs to also go in https://github.com/ssec-jhu/base-template/blob/main/.zenodo.json.

Ok NVM, this shouldn't go in here...

@imaitland FYI the PR author/asignee is normally responsible for merging their own PRs. Looks like this one fell by the way side so I merged it for you.

authors: imaitland jamienoss

repo: ssec-jhu/base-template

license_source: github-api


text:
JFXHighlighter throws exception

I'm trying to use JFXHighlighter but it instantly throws an exception as soon as my app starts.
Here's the exception thrown: https://hasteb.in/guyisepa.rb
Caused from this:
            this.textLayoutMethod = Text.class.getDeclaredMethod("getTextLayout");             this.textLayoutMethod.setAccessible(true);
My project is running on java 11, I'm using jfoenix 9.0.6.
It's obvious java's new module security features are tampering with this. What must I do to fix it?

Ok, for anyone wondering from doing a google search, I'm going to add the error messages here and with the solution so google can find them:
The solution:
--add-opens javafx.graphics/javafx.scene=ALL-UNNAMED --add-opens javafx.graphics/javafx.scene.text=ALL-UNNAMED
The keywords of the issue:
java.lang.reflect.InaccessibleObjectException: Unable to make private com.sun.javafx.scene.text.TextLayout javafx.scene.text.Text.getTextLayout() accessible: module javafx.graphics does not "opens javafx.scene.text" to unnamed module @2357d90a
Thank you. :)

@SwissCheese5
i'm facing this issue
Unrecognized option: --add-exports javafx.graphics/com.sun.javafx.geom=ALL-UNNAMED
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
Could not create the Java Virtual Machine.

source: gharchive/issue

repo: sshahine/JFoenix

license: MIT


text:
Brick breakout game

I want to add the brick-breaking game, using React JS and HTML Canvas for the board, the player will also get 5 lives if the ball got missed, and the score will be counted when the ball hits the bricks, and the ball is controlled by the paddle.

I want to work on this game, please assign me this issue

Assigned this issue to you. Kindly complete the work in 10 days and go through the readme and contributing.md and other guidelines for successful merge. All the best to your work Ganesh... !!!

Sure, Thank You!!

PR request #137 was Merged, and the issue is done.

repo: ssitvit/Games-and-Go


text:
Propagate the original MinIO exception in case of connection errors.

In my local environment, I am always seeing the following MinIO error while listing objects:
    raise ServiceXException(msg)
servicex.utils.ServiceXException: (ServiceXException(...), 'Unable to reach Minio at localhost:9000/2ca35b92-ddc0-4f73-bdd4-ccad718f1dfb?location=. Max retries exceeded.')

I confirmed that there was no connection reset happening. It turned out that the original MinIO exception was not being propagated in this case and that made debugging the problem very difficult. With the change in this PR, I now see this trace:
...
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:997)
...
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='localhost', port=9000): Max retries exceeded with url: /2ca35b92-ddc0-4f73-bdd4-ccad718f1dfb?location= (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:997)')))
...
    raise ServiceXException(msg) from e
servicex.utils.ServiceXException: (ServiceXException(...), 'Unable to reach Minio at localhost:9000/2ca35b92-ddc0-4f73-bdd4-ccad718f1dfb?location=. Max retries exceeded.')

It is now clear why the problem is happening.

Ah, this is awesome. Thanks for submitting that fix!!

@BenGalewsky - I'll leave you to merge this when you think is the right time! Thanks so much @draghuram !!

No problem. It certainly helped me fix the issue (set objectstore.publicClientUseTLS: false) and move on with the transformation process

Thank you so much, @draghuram  - actually we are in the process of a major rewrite of this client library. Take a look at the service_client branch. A pre-release has been deployed as version 3.0.0-alpha.2 and documentation is published as github pages from here: https://ssl-hep.github.io/ServiceX_frontend/

Good to know, thanks.

authors: BenGalewsky draghuram gordonwatts


text:
NextJS Warmer Fails to Deploy

When I set warm: 5 on my nextjs component I get the following error on deploy:
VisibleError: Could not find handler file "index.handler" for function "WebWarmer"

If you've already posted your issue on Discord, make sure to leave a link to it here.

Fixed in v0.0.146

authors: fwang garretcharp

repo: sst/ion


text:
clarify where to run the live command from

After spending time researching is that I realized VS Code requires it to be executed from within their terminal windows.

Thanks!

If anyone else is interested, I figured a hacky way of running things from outside vscode. I hope it helps!

authors: jayair zvictor


text:
Enhancement: Add Node 20 support

Add support for updated CDK & Node 20

tried standardizing how runtimes are configured between constructs
left Node 18 as the default runtime where applicable so that Node 20 is opt-in

Awesome stuff! Just wanted to check if this PR is going to have any potential backward incompatibility issues because CDK changed it's cloud assembly schema version from 34 to 35 in v2.106.0.
https://github.com/aws/aws-cdk/commits/c6471f2f603e2570e8781d6e1ae7d8252aec0f79/packages/aws-cdk-lib/cloud-assembly-schema/schema/cloud-assembly.version.json
https://github.com/aws/aws-cdk/compare/v2.105.0...v2.110.0?diff=unified#diff-8ea0b1084baf243d6a1181bade0e37fd6cbf1b346bfbd0d97e51df6d0cec6d0a

Thanks! I'll pass this along to the team. I think it's easier to do the CDK updates on our side.

Thanks @nowlena!
I moved the runtime changes in this commit — https://github.com/sst/sst/commit/eb0e725ebcc87671a24000bfe36aea887b8a93d8
Will be in the next release.

authors: fwang jayair nowlena vvashisht

repo: sst/sst


text:
review world-templates documentation for clarity and structure

Briana to leave notes here regarding the layman-friendliness of said documentation.

no descriptions for defaults and locale - even if these sections should not be edited, having that called out is helpful for recognizing which areas of the code can be safely changed
~if technically possible, potentially rearrange the config files and the documentation so that templates comes before start (sorted chronologically)~ discussed offline, currently sorted alphabetically to improve searchability
for starting actor/room/etc, what happens if there is only one possible entity to choose from? presumably, that is the entity that will be chosen, but having this called out is helpful for the sake of documenting expected behavior
"Template Types" is somewhat confusing; is there an alternative name for this section that would be more descriptive/distinctive?
missing description of flags, slots, mods

for slots, include explanation of how to create a slot that starts empty but that can be used

for stats, which variable are built in (e.g. health, stamina) and what does the user need to do to create additional variables (e.g. hunger for Hansel & Gretel, croquet score for Alice & Wonderland, etc.)

repo: ssube/textual-engine


text:
fix encode UTF16 symbols with quotes

Hi,
This PR fix issue with encoding symbols.
thks

Codecov Report

Merging #25 into master will not change coverage.
The diff coverage is 100%.

@@           Coverage Diff           @@
##           master      #25   +/-   ##
=======================================
  Coverage   90.21%   90.21%
=======================================
  Files          11       11
  Lines         991      991
=======================================
  Hits          894      894
  Misses         97       97

Impacted Files
Coverage Δ

lib/src/postgresql_codec.dart
83.07% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a65e856...171e2a0. Read the comment docs.

Looks like a good addition. Two small items:

Can you add a test that executes an insert/select query using UTF16 characters? This would be a test that is similar to "UTF8 strings in query" in query_test.dart.

What is the intention behind adding the additional escape characters to the existing UTF8 tests? Seem like good checks, but just want to make sure I'm not missing something.

authors: andrewst codecov-io joeconwaystk

repo: stablekernel/postgresql-dart


text:
Added EDC Connector

The image builds and works for me.

note: as of today we do not want to merge this.
The image is built though, it's in the 'sandbox'

@lfrancke Any particular reason you commented on this now?

I received a notification. Probably because you merged main

authors: fhennig lfrancke

repo: stackabletech/docker-images


text:
prompt is too long: 200156 tokens > 200000 maximum

Describe the bug
prompt is too long: 200156 tokens > 200000 maximum

Link to the Bolt URL that caused the error
https://bolt.new/~/sb1-pesz3g
Steps to reproduce
it just shows up in chat. I updated, rebooted, cleared the cache, nothing helps. It just stopped working and gives an error.
Expected behavior
chat should work
Screen Recording / Screenshot
No response
Platform
Browser name = Chrome
Full version = 132.0.0.0
Major version = 132
navigator.appName = Netscape
navigator.userAgent = Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/132.0.0.0 Safari/537.36
performance.memory = {
  "totalJSHeapSize": 62230805,
  "usedJSHeapSize": 55071329,
  "jsHeapSizeLimit": 4294705152
}
Username = Habartru
Chat ID = f3cfb0336eba

Additional context
No response

Appreciate the feedback! This is a known issue with the AI model’s context window size and is being tracked here: #1322.

I wrote earlier and I want to share my thoughts about exceeding the context window limit again.  my project went beyond the limits of what was possible in bolt at the stage when I had already spent $ 300 on it. moreover, he was already at the stage when there were little things left. but faced with the problem of the system's failure to continue working with the project files, I began to look for ways to fix it. for a week I wrote letters, created topics on github and received the same answers as everyone else. a week later, the developers recommended using the ignore file to exclude it from the context. but the thing is that this will only work if you assemble a modular project from different components that are minimally connected to each other, but if your project works as one system, then you are doomed to failure. The AI simply forgets about how the project should look as a whole, and begins to create code, disrupting performance and connections with already created parts of the project placed in ignore. while I was coming to this understanding, I spent another $200 creating forks, trying to get at least what was there. Now I have a reasonable question for the developers, of course with full respect for their work and understanding of the complexity of the pioneers, - tell me here and in other branches I have met people like myself who unknowingly, due to the lack of any warnings, started creating projects and stopped because of these problems, losing time and money and remaining in fact, with a one-on-one problem. is there any plan on your part to compensate for the loss of users, in terms of refunds, or accrual of tokens to the balance, or will you continue to ignore it? don't you think that at this stage you will get more negativity than a well-deserved HYPE? did you know about the limitation of the context window when launching bolt, and if you did, why didn't you warn users that it is not recommended to create sites that are more than banal landing pages?  someone in your team has obviously given up on this bolt. And returning to my problem, I want to ask another and main question, how do I complete my project, are you ready to help me with this, or do I have to experiment again, for my money, in some way that is still incomprehensible to me?  This is a post without negativity and some kind of commercialism, I just want to get answers to questions, like others here.  And as a possible piece of advice, maybe you should think about delineating projects from the inside? for example, you could keep a counter for a context window, and when you reach close to the limit, start a new window, but with the ability to transfer information about the project from the first one. that is, in fact, a multi-agent communicating system. this will not be necessary for everyone, but only for those who really do something voluminous, and bring you the most money. I hope you understand what I mean.  and another point, from the practice of observing the use of bolts. is it possible to prescribe a condition in the system script under which error messages from the user will be deleted from the context window, especially those that the bolt fixes on the machine? it seems that they are not required in the context at all, but they take up a very large amount of context.   I'm sorry that I write so much, and maybe not everything is on the case, but there are a lot of questions, but there are no answers yet, and I don't see the point in carrying money back into the bolt, having unfinished even one project.... Thanks for attention.

authors: Habartru endocytosis


text:
HAving issues getting anything to run or render on my macs

Describe the bug
Apps don't launch, usually an error with installign initial files or dependencies
Link to the Bolt URL that caused the error
https://bolt.new/~/sb1-kabdbg
Steps to reproduce
load and try to launch.
Expected behavior
app would launch and render in preview
Screen Recording / Screenshot

Platform
Browser name = Chrome
Full version = 131.0.0.0
Major version = 131
navigator.appName = Netscape
navigator.userAgent = Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36 Edg/131.0.0.0
performance.memory = {
  "totalJSHeapSize": 77495827,
  "usedJSHeapSize": 76042967,
  "jsHeapSizeLimit": 4294705152
}
Username = uelsimon
Chat ID = 1b567cfc6ac3

Additional context
ive tried on multiuple browsers..and on multiple machines.
i havent had it work properly in weeks.  is it something going on with the browser or somehting?

Appreciate the feedback! We are aware of this issue and are tracking it here.

authors: endocytosis uelsimon


text:
Fixes for Barbican with Vault plugin

Currently gives the error below, as the default field is not being applied to the varriable.
TASK [Write barbican Approle ID to file if requested] *************************************************************************
Monday 30 October 2023  09:57:48 +0000 (0:00:00.075)       0:00:07.517 ********
fatal: [controller01 -> localhost]: FAILED! =>
  msg: |-
    The conditional check 'stackhpc_write_barbican_role_id_to_file | bool | default(false)' failed. The error was: error while evaluating conditional (stackhpc_write_barbican_role_id_to_file | bool | default(false)): 'stackhpc_write_barbican_role_id_to_file' is undefined

    The error appears to be in '/var/lib/home/stackhpc/mattc/src/kayobe-config/etc/kayobe/ansible/vault-deploy-barbican.yml': line 80, column 7, but may
    be elsewhere in the file depending on the exact syntax problem.

    The offending line appears to be:

        - name: Write barbican Approle ID to file if requested
          ^ here

Also included another fix for the SSL with Barbican Vault Plugin. Needed to specify the cert path

I've also proposed another fix. The conditional check for the Barbican AppRole secret ID being present in vault would not work. Firstly, because the "when" statement was matching on it being present, when it should have been when it is not present.
However, the module hashivault_approle_role_secret_list also does not return a useful output. It lists the vault-uuids for the secrets, rather than the actual approle id which we would want to match for.
I've instead changed to use a different module, which can check for the presence of the secret specifically.

repo: stackhpc/stackhpc-kayobe-config


text:
Parsing a stored selector fails if the selector is using commas

I'm unsure of the root case, but parsing this selector:
repository.properties['github/primary_language'] in ['TypeScript', 'Go']

from the database fails. We do have a hand-rolled Scan method that converts a JOIN with an ARRAY into a Go type which breaks.

CC @danbarr who reported the issue.
Marking as P0 because after such selector has been added minder just throws errors attempting to manipulate that profile.

repo: stacklok/minder


text:
DO NOT REVIEW Dashrews/network flow improvements no copy -- RESEARCH

Description
A detailed explanation of the changes in your PR.
Feel free to remove this section if it is overkill for your PR, and the title of your PR is sufficiently descriptive.
Checklist

[ ] Investigated and inspected CI test results
[ ] Unit test and regression tests added
[ ] Evaluated and added CHANGELOG entry if required
[ ] Determined and documented upgrade steps
[ ] Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)

If any of these don't apply, please comment below.
Testing Performed
TODO(replace-me)
Use this space to explain how you tested your PR, or, if you didn't test it, why
you did not do so. Valid reasons include, for example, "CI is sufficient",
"No testable changes". Feel free to attach JSON snippets, curl commands,
screenshots.
In addition to reviewing your code, reviewers must also review your testing
instructions and make sure they are sufficient.

/test gke-postgres-scale-tests

/test gke-postgres-scale-tests

/test gke-postgres-scale-tests

/test gke-postgres-scale-tests

repo: stackrox/stackrox


text:
ROX-17963: CI: Fix central channel handling

Description
#6590 changed the test<->central channel state to use ThreadLocal variables. The intent being that each concurrent test thread would not have to be concerned with other tests altering its central connection. However a side effect of this change is that when threads complete and the ThreadLocal vars are GC'd the gRPC package produces a nasty error message:
    Jun 30, 2023 9:29:07 AM io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference cleanQueue
    SEVERE: *~*~*~ Channel ManagedChannelImpl{logId=16, target=localhost:8000} was not shutdown properly!!! ~*~*~*
        Make sure to call shutdown()/shutdownNow() and wait until awaitTermination() returns true.
    java.lang.RuntimeException: ManagedChannel allocation site
    	at io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference.<init>(ManagedChannelOrphanWrapper.java:94)
    	at io.grpc.internal.ManagedChannelOrphanWrapper.<init>(ManagedChannelOrphanWrapper.java:52)
... snip very long stack trace ...

It also turns out that because of the use of org.junit.rules.Timeout, each spock test feature is executed in a separate thread. And so along with #6590, a new channel is now created for each test feature.
This PR reverts most of #6590 and switches back to using global state to manage the test<->central channel. Changes to the channel are protected with synchronized blocks in anticipation of future concurrent test work. Note: with this change, tests that test the central connection (e.g. AuthServiceTest, ClientCertAuthTest) will not be parallel-able.
ref: https://redhat-internal.slack.com/archives/CLUNQEEMA/p1688074475205819
cc @rukletsov
Checklist

[ ] Investigated and inspected CI test results

Testing Performed
CI is sufficient.

/test gke-upgrade-tests

/test gke-upgrade-tests

/fyi @janisz @tommartensen a cleanup relating to #6590


text:
Reflect MUI changes regarding Date and Time pickers moved to MUI X from lab

Closes #324

Thanks @KOliver94 going to start reviewing this today

Hi @cliedeman,
Have you managed to review it? Let me know if you need any change.
Thanks!

Gentle bump on this. @cliedeman

waiting on this for an upgrade too

Hi @cliedeman,
This PR is open for more than 1.5 months. I would like to help to keep this package maintained so can you give some feedback if any change is needed or merge it?
Thanks!

Sorry for the delay. This is released now

authors: KOliver94 chrishawn cliedeman ramspeedy

repo: stackworx/formik-mui


text:
broadcast-stream not found (windows)

version 1.2.3 crashes app on start
reverting to 1.1.1 and it works fine.
I have no idea what is trying to call broadcast-stream ... it's not a listed dependancy in the stack (that's probably the problem?)

The latest version of ssb-lan uses dgram-broadcast as a replacement for broadcast-stream because the latter is now in an npm junkyard (no one can publish to it).

authors: mixmix staltz

repo: staltz/ssb-lan


text:
Stanc3 fails to catch int arg to inv_sqrt; C++ compiler throws error

model which contains expression:
inv_sqrt(size(node_cts)));

fails with C++ compiler warning, requires this rewrite:
inv_sqrt(1.0*size(node_cts)));  // promote to real

C++ compiler output:
INFO:cmdstanpy:compiling stan program, exe file: /Users/mitzi/github/stan-dev/example-models/knitr/car-iar-poisson/update_2021_02/bym2_islands
INFO:cmdstanpy:compiler options: stanc_options=None, cpp_options=None
ERROR:cmdstanpy:file /Users/mitzi/github/stan-dev/example-models/knitr/car-iar-poisson/update_2021_02/bym2_islands.stan, exception ERROR
In file included from /Users/mitzi/github/stan-dev/example-models/knitr/car-iar-poisson/update_2021_02/bym2_islands.hpp:1:
In file included from stan/src/stan/model/model_header.hpp:4:
In file included from /Users/mitzi/.cmdstan/cmdstan-2.26.1/stan/lib/stan_math/stan/math.hpp:19:
In file included from /Users/mitzi/.cmdstan/cmdstan-2.26.1/stan/lib/stan_math/stan/math/rev.hpp:10:
In file included from /Users/mitzi/.cmdstan/cmdstan-2.26.1/stan/lib/stan_math/stan/math/rev/fun.hpp:7:
In file included from /Users/mitzi/.cmdstan/cmdstan-2.26.1/stan/lib/stan_math/stan/math/prim/fun.hpp:142:
/Users/mitzi/.cmdstan/cmdstan-2.26.1/stan/lib/stan_math/stan/math/prim/fun/inv_sqrt.hpp:43:10: error: implicit instantiation of undefined template 'stan::math::apply_scalar_unary<stan::math::inv_sqrt_fun, unsigned long, void>'
  return apply_scalar_unary<inv_sqrt_fun, Container>::apply(x);
         ^
/Users/mitzi/github/stan-dev/example-models/knitr/car-iar-poisson/update_2021_02/bym2_islands.hpp:239:21: note: in instantiation of function template specialization 'stan::math::inv_sqrt<unsigned long, nullptr, nullptr>' requested here
                    inv_sqrt(size(node_cts))));
                    ^
/Users/mitzi/.cmdstan/cmdstan-2.26.1/stan/lib/stan_math/stan/math/prim/functor/apply_scalar_unary.hpp:39:8: note: template is declared here
struct apply_scalar_unary;
       ^
1 error generated.
make: *** [/Users/mitzi/github/stan-dev/example-models/knitr/car-iar-poisson/update_2021_02/bym2_islands] Error 1
ERROR:cmdstanpy:model compilation failed

Transferring to Math.
https://github.com/stan-dev/math/blob/develop/stan/math/prim/fun/inv_sqrt.hpp is missing and implementation for inv_sqrt(int).

I tried to tackle this; it seems like inv_sqrt(int) does work, but inv_sqrt(unsigned long) does not. This is also true for similar functions, like sin and sqrt.

Whether it errors or not probably also depending on the exact compiler. I am guessing the compiler used in the top comment is on macOS, so its using clang++.

This was due to the templating in apply_scalar_unary, have opened #2469 with a fix

authors: adamhaber andrjohns mitzimorris rok-cesnovar

repo: stan-dev/math


text:
Update to OpenCL 2.0

Summary
This updates the OpenCL headers that allow for up to version 2.2 support. We use the flags -DCL_HPP_TARGET_OPENCL_VERSION=200 so that our code base uses OpenCL 2.0 by default.
There are a few code changes to update us from some deprecated functions.

to_size_t was removed and instead we use cl::array<size_t, 3>
We no longer need to pass a pair to the OpenCL program to compile kernels (ex here)
cl::make_kernel was deprecated so we now use KernelFunctor when constructing kernels.
The OpenCL header is now just a single file and so I removed the CL folder in the dependency library folder. Now we just do #include <cl.hpp>

Tests
All basic opencl tests pass
Side Effects
We are not using any of the OpenCL 2.0 kernel features and so this should have no side effects for the kernels.
Checklist

[x] Math issue #1276

[x] Copyright holder: (fill in copyright holder information)

Steve Bronder
The Khronos Group Inc.

[x] the basic tests are passing

unit tests pass (to run, use: ./runTests.py test/unit)
header checks pass, (make test-headers)
docs build, (make doxygen)
code passes the built in C++ standards checks (make cpplint)

[x] the code is written in idiomatic C++ and changes are documented in the doxygen

[x] the new changes are tested

Can we somehow check what OpenCL version is supported on the test Mac machine?

Ughhh I totally forgot about metal and Mojave. When I get home I think I can set CL_HPP_MINIMUM_OPENCL_VERSION=120 to let this compile with 1.2 and 2.0

I tried to google this awhile a go, but had not luck finding an anwser. On pre-Mojave OSx with an AMD/NVIDIA GPU, do the GPU OpenCL runtimes that come with the drivers support OpenCL 1.2.x or 2.0?

I'm pretty sure it's only 1.2 (mby even 1.1) that are supported on the pre-installed drivers before Mojave

Gonna close this until I write the dependency wiki

authors: SteveBronder rok-cesnovar


text:
Feature/issue 1221 Use OpenCL in rev/multiply

Summary
This PR adds the use of existing OpenCL kernels in rev/multiply. With this PR we will close issue #1221
The input size break point for when to switch from the CPU to the GPU is the same as for the prim/multiply. Same as in mdivide_left_tri, this could be set lower as the break point in prim is set for cases of A*B where we need to transfer 2 input matrices and the result matrix. In the rev/multiply there are also cases where we perform A*B and A*C where we transfer all three input matrices and both results. There is therefore less data transfers per multiplication here, but I think we can reuse the same limit here for the time being as.
This PR adds the same 2 constructors as the rev/mdivide_left_tri #1303
Tests
There are a few added tests. They compare the values and adjoints of OpenCL and CPU multiply() for all combinations of input in rev/multiply: (var, double), (double, var) and (var, var).
Side Effects
/
Checklist

[x] Math issue #1221

[x] Copyright holder: Rok Češnovar, University of Ljubljana

[x] the basic tests are passing

[x] the code is written in idiomatic C++ and changes are documented in the doxygen

[x] the new changes are tested

Thanks Steve for the changes and the reviews for both PRs! I know that you are pretty busy, so its really appreciated.
I dont have any new PRs scheduled until after Stancon so I wont bother you with any new reviews for a week or two :). We also now have everything that we promised to deliver in develop before Stancon, which I think is nice :)
I am going to make an issue for separate varis, we should move them in the /opencl/rev folder, so that the OpenCL code in /prim and /rev etc is minimal. I would move all current opencl varis at once so that we are consistent (not have cholesky opencl vari in /rev other in /opencl).

Develop is failing after this merge, which is odd. I am looking into this.
@seantalts @serban-nicusor-toptal are there any differences between the run of "Full unit with GPU" in the PR and after merge?

The "Full unit with GPU" was run with "OPENCL_DEVICE_ID=0", while all other recent develop tests were run with "OPENCL_DEVICE_ID=1". Does anyone have any info on the 2 devices and how to we determine which one is selected?


text:
Always-run tests part 2 will now run only on master and develop

Summary
Issue #1713
Tests
Side Effects
No
Checklist

[ ] Math issue #(issue number)

[ ] Copyright holder: (fill in copyright holder information)
The copyright holder is typically you or your assignee, such as a university or company. By submitting this pull request, the copyright holder is agreeing to the license the submitted work under the following licenses:
- Code: BSD 3-clause (https://opensource.org/licenses/BSD-3-Clause)
- Documentation: CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/)

[ ] the basic tests are passing

unit tests pass (to run, use: ./runTests.py test/unit)
header checks pass, (make test-headers)
dependencies checks pass, (make test-math-dependencies)
docs build, (make doxygen)
code passes the built in C++ standards checks (make cpplint)

[ ] the code is written in idiomatic C++ and changes are documented in the doxygen

[ ] the new changes are tested

Hey @rok-cesnovar, I hope I understood correctly the issue in #1713

Hey,
this isnt what I meant. Sorry if my issue wasnt clear.
Example:

I create a local branch something_develop
I push the branch to the repo and dont create a PR

This would start tests on Jenkins, even without opening a PR. I think its because the branch has develop in the name.

Understood, thanks for the quick answer. Will look into it now!

authors: rok-cesnovar serban-nicusor-toptal


text:
Redesign loggers

Summary:
From @bob-carpenter

Our loggers should just take these, as in:
logger.info() << "Gradient evaluation took " << ...;
Or we should have a polyadic info function that could look like:
logger.info("Gradient evaluation took ", deltaT, " seconds");

Description:
Our loggers were introduced as a stepping stone to getting us to a better design.
We can use this issue to discuss different designs for the loggers.
Additional Information:
cc @sakrejda
Original comment from: https://github.com/stan-dev/stan/pull/2570#r201154784
Current Version:
v2.17.1

The current implementations are:
https://github.com/stan-dev/stan/blob/develop/src/stan/callbacks/logger.hpp
https://github.com/stan-dev/stan/blob/develop/src/stan/callbacks/stream_logger.hpp

authors: mitzimorris syclik

repo: stan-dev/stan


text:
Memory corruption when using stan_model class in more than one thread

(This is a bug report with a test case in the PR)
The stan_model class is not threadsafe. Given two separate instances of a given stan_model, you can't call, say, stan::model::log_prob_grad with them in parallel in different threads. I suspect that there's some unsafe manipulation of the class variables at fault.
The test attached leads to a segfault.
gdb tells me that the problem is memory corruption, "corrupted double-linked list" in stan_math/stan/math/rev/core/grad.hpp:
li#1  0x0000555555560bd3 in stan::math::grad (vi=0x55555583dfa0) at lib/stan_math/stan/math/rev/core/grad.hpp:44
44              (*it)->chain();
(gdb) list
39            vi->init_dependent();
40            it_t begin = ChainableStack::var_stack_.rbegin();
41            it_t end = empty_nested()
42              ? ChainableStack::var_stack_.rend() : begin + nested_size();
43            for (it_t it = begin; it < end; ++it) {
44              (*it)->chain();
45            }
46          }
47
48        }

The test attached to the PR makes the problem clear, I hope. You should be able to sample using two separate model instances -- but instead you get a segfault.
Being able to sample in parallel in system threads (not processes) is particularly important for PyStan on Windows.
My sense is that this should be easy to fix. Perhaps a variable needs to be an instance variable instead of a class variable? Any ideas?
Intended Effect
Segfault
How to Verify
make test/unit/services/sample/hmc_nuts_diag_e && test/unit/services/sample/hmc_nuts_diag_e

Reviewer Suggestions
@bob-carpenter @syclik

The problem is autodiff, not the model class.
What you need to do to work in parallel is to make the singletons for autodiff into thread local variables.  There's a performance penalty, but then they'll be kept separate across threads and the model class will become threadsafe.
We'll be looking at that more with C++11 threading and the desire to parallelize ODE solvers.

Thanks! Glad to know that you understand what's going on.
Is this tracked elsewhere? This blocks at least one PyStan feature.
If it's not tracked elsewhere, should I perhaps close this and open an issue "Autodiff is not threadsafe"?

It should go in the math library.  The fix is to declare the variables as thread local, but that might not do what some users intend, as it doesn't let you compute one piece of a likelihood in one thread, it just lets you run multiple models in parallel.
I think the slowdown is around 10--20% but we should try and profile again.

Great. Is this as easy as adding thread_local somewhere (I see that
that's a C++11 keyword)? In the short term I could maintain a simple
patched version. (The problem is a bit of a show-stopper for the new
"backend" for PyStan 3.)
In the meantime, I'll close this and write up an issue for stan-math.
On 05/08/2017 08:06 PM, Bob Carpenter wrote:

It should go in the math library. The fix is to declare the variables as
thread local, but that might not do what some users intend, as it
doesn't let you compute one piece of a likelihood in one thread, it just
lets you run multiple models in parallel.
I think the slowdown is around 10--20% but we should try and profile again.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/stan-dev/stan/pull/2306#issuecomment-300024490, or
mute the thread
https://github.com/notifications/unsubscribe-auth/AABN7iZaIbIrxaBDFR1mYOO0YqsGhHY7ks5r3631gaJpZM4NUl2g.

See https://github.com/stan-dev/math/issues/551

authors: ariddell bob-carpenter


text:
List externally-defined value sets used in IG on value set page

Currently only one of the ODH value sets ("YesNoVS") is listed and rendered in the IG Value Sets tab.  All of the value sets should be represented and linked to here.

This has been done some time ago. The value set page in the IG has both the local and externally-defined value sets.

repo: standardhealth/shr-cli


text:
Models v3

This PR updates the models to match capabilities in the v3 grammar and updates the importer and existing exporters minimally to still work with the updated model.  What I mean by minimally is that I got it so the current tests would still pass and the end-to-end translation still works with the v2 grammar.  So, while the models are v3, everything else is still v2, but done in such a way to still work.
Next steps will be to update the importer and exporters to utilize the full capabilities of the v3 models.

This change was distributed to all team members and reviewed by at least one (who provided verbal approval) -- so I'm merging it.


text:
Duplicate subelement name of ClinicallyRelevantTime in Observation

In Observation, two subelements are named ClinicallyRelevantTime.

I don't see this. What branch are you on?

master branch.
EntryElement: 	Observation
Concept:		MTH#C1554188
Based on:		Action
Description:	"An Observation represents evidence, both subjective and objective. Observation includes any information about a subject that results from an act of observing, measuring, or evaluation. The focus of an observation can include the subject's behavior, physiological state, health state, functional status, environment, exposures to substances, etc. An Observation contains information about the act of observing or measuring, and the result of the observation. The method of observing can vary widely, from questioning, physical examination, formal assessment vehicles, laboratory tests, imaging procedures, etc. Patient-reported information is also considered an observation, where subject and observer are the same individual."
Value:			0..1 (Quantity or CodeableConcept could be from ObservationResultVS or string or boolean or Range or Ratio or time or dateTime or TimePeriod)
// TODO: Support additional value types: SampledData or Attachment
//----------- Define the observation to be made
1..1			SpecificType   // inherited from Action, here required
0..*			Category from ObservationCategoryVS if covered
0..1			BodySite  // TODO: this should be 0..* but that doesn't map to FHIR -- This should be part of the 'Focus' of the observation
// TODO: 0..* AdditionalBodySite - observations can involve multiple body sites
//----------- Why this observation is taking place
0..1			Reason
0..1			AssociatedStudy
//----------- Execution of the observation action
1..1			Status from http://hl7.org/fhir/ValueSet/observation-status
NonOccurrenceModifier.Reason from ObservationNotMadeReasonVS
0..1			Method
//----------- Capture the result and its significance
//				The Value
0..1			AssertionNegationModifier
0..*			ObservationQualifier
0..1			DataAbsentReason
0..1			ClinicallyRelevantTime  // may not be the same as the OccurrenceTime
0..*			ReferenceRange
0..1			Interpretation
0..1			Comment
//----------- Related and Sub-Observations
0..1			PanelMembers  // separate observations occurring in a panel
0..1			ClinicallyRelevantTime
0..*			Participant 	// TODO: add to metadata?
0..*			Setting
0..*			Location

authors: gregquinn2001 jafeltra markkramerus


text:
Add parameter-dataset.tsv sample links in README.md

When trying to train a model, I have met some problems, that can be explained in README.md which will help future users and save their time.

Add parameter-dataset.tsv sample links in README.md
Add GENIE_PARSER_PATH setting in README.md
Fix split-train-eval command, remove -i and []

Fix it, thanks!

repo: stanford-oval/genie-toolkit


text:
Update the pruning size and depth default values

For generate and generate-dialogs.
Fixes an issue with the basic tutorial, which doesn't pass the command-line arguments and runs out of memory very quickly.

Uh. I checked, I use 9 for multiwoz, and 8 for thingpedia. Maybe 8 should be the default?

as long as it runs on 8gb ram machine, it's fine

authors: gcampax rayslxu


text:
持币列表表显示的持币数和地址详情页的持币数对不上

如图所示，持币列表页，显示的持币数是：205,541，但是进入详情页之后，显示的持币数为 0
https://stcscan.io/main/address/0xe11b3a647fe805ce14914432c5995972

@mzcaptainyang Holder列表数据需要统计来自链上的数据，而地址的详情页数据直接取自链上，所以Holder列表数据会有些延迟，现在已经更正

repo: starcoinorg/starcoin-explorer


text:
Removed duplicate text

On the home page, there is a text saying you can log in with discord to save your data across devices.
Below that, there is another text saying the exact same thing which is shown if you're not logged in.
This PR removes the second text which is only shown if the user isn't logged in.

ahh good catch - thank you!

authors: OfficialCRUGG jacc


text:
Memory Error during training

Hello,
I have encountered some strange behavior when training with large 3D images.
I have a pipeline to train a 3D stardist and I use a Dataloader using keras.Sequence to load the 3D on the fly.
The training patch_size is fixed to (60, 128, 128) and all the 3D I use are bigger than that. I am also always using a batch size of 1.
If I train with a set of relatively small images (around 100x200x200) everything runs just fine.
However if my training set contains large images (400x500x1200) I get the error copied bellow.
I'm wondering what could be causing this?
I have 64Go of RAM, so one image loaded in RAM can't reasonnably overload the memory.
I have 12Go of memory in my GPU, is the image loaded to GPU before or after being cropped to the train_patch_size?
Can it be that if I have that the number of cells in a mask does not scale well in memory?

---------------------------------------------------------------------------
MemoryError                               Traceback (most recent call last)
<ipython-input-22-54ffb4970400> in <module>
      3             augmenter=augmenter_fn(ZOOM_RATIO, ZOOM_PROB),
      4             epochs=200, #100,
----> 5             seed=42)

/work/.cache/poetry/kc-segmentation-DJpFP61h-py3.7/lib/python3.7/site-packages/stardist/models/model3d.py in train(self, X, Y, validation_data, classes, augmenter, seed, epochs, steps_per_epoch, workers)
    554                       callbacks=self.callbacks, verbose=1,
    555                       # set validation batchsize to training batchsize (only works in tf 2.x)
--> 556                       **(dict(validation_batch_size = self.config.train_batch_size) if _tf_version_at_least("2.2.0") else {}))
    557         self._training_finished()
    558

/work/.cache/poetry/kc-segmentation-DJpFP61h-py3.7/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py in _method_wrapper(self, *args, **kwargs)
     64   def _method_wrapper(self, *args, **kwargs):
     65     if not self._in_multi_worker_mode():  # pylint: disable=protected-access
---> 66       return method(self, *args, **kwargs)
     67
     68     # Running inside `run_distribute_coordinator` already.

/work/.cache/poetry/kc-segmentation-DJpFP61h-py3.7/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing)
    813           workers=workers,
    814           use_multiprocessing=use_multiprocessing,
--> 815           model=self)
    816
    817       # Container that configures and calls `tf.keras.Callback`s.

/work/.cache/poetry/kc-segmentation-DJpFP61h-py3.7/lib/python3.7/site-packages/tensorflow/python/keras/engine/data_adapter.py in __init__(self, x, y, sample_weight, batch_size, steps_per_epoch, initial_epoch, epochs, shuffle, class_weight, max_queue_size, workers, use_multiprocessing, model)
   1110         use_multiprocessing=use_multiprocessing,
   1111         distribution_strategy=ds_context.get_strategy(),
-> 1112         model=model)
   1113
   1114     strategy = ds_context.get_strategy()

/work/.cache/poetry/kc-segmentation-DJpFP61h-py3.7/lib/python3.7/site-packages/tensorflow/python/keras/engine/data_adapter.py in __init__(self, x, y, sample_weights, workers, use_multiprocessing, max_queue_size, model, **kwargs)
    770     # Since we have to know the dtype of the python generator when we build the
    771     # dataset, we have to look at a batch to infer the structure.
--> 772     peek, x = self._peek_and_restore(x)
    773     assert_not_namedtuple(peek)
    774     peek = self._standardize_batch(peek)

/work/.cache/poetry/kc-segmentation-DJpFP61h-py3.7/lib/python3.7/site-packages/tensorflow/python/keras/engine/data_adapter.py in _peek_and_restore(x)
    828   @staticmethod
    829   def _peek_and_restore(x):
--> 830     peek = next(x)
    831     return peek, itertools.chain([peek], x)
    832

/work/.cache/poetry/kc-segmentation-DJpFP61h-py3.7/lib/python3.7/site-packages/csbdeep/internals/train.py in __iter__(self)
    112         # print(f"### __iter__", flush=True)
    113         for i in range(len(self)):
--> 114             yield self[i]
    115
    116     def batch(self, i):

/work/.cache/poetry/kc-segmentation-DJpFP61h-py3.7/lib/python3.7/site-packages/stardist/models/model3d.py in __getitem__(self, i)
     57         arrays = [sample_patches((self.Y[k],) + self.channels_as_tuple(self.X[k]),
     58                                  patch_size=self.patch_size, n_samples=1,
---> 59                                  valid_inds=self.get_valid_inds(k)) for k in idx]
     60
     61         if self.n_channel is None:

/work/.cache/poetry/kc-segmentation-DJpFP61h-py3.7/lib/python3.7/site-packages/stardist/models/model3d.py in <listcomp>(.0)
     57         arrays = [sample_patches((self.Y[k],) + self.channels_as_tuple(self.X[k]),
     58                                  patch_size=self.patch_size, n_samples=1,
---> 59                                  valid_inds=self.get_valid_inds(k)) for k in idx]
     60
     61         if self.n_channel is None:

/work/.cache/poetry/kc-segmentation-DJpFP61h-py3.7/lib/python3.7/site-packages/stardist/models/base.py in get_valid_inds(self, k, foreground_prob)
    197         else:
    198             patch_filter = (lambda y,p: self.max_filter(y, self.maxfilter_patch_size) > 0) if foreground_only else None
--> 199             inds = get_valid_inds((self.Y[k],)+self.channels_as_tuple(self.X[k]), self.patch_size, patch_filter=patch_filter)
    200             if self.sample_ind_cache:
    201                 with self.lock:

/work/.cache/poetry/kc-segmentation-DJpFP61h-py3.7/lib/python3.7/site-packages/stardist/sample_patches.py in get_valid_inds(datas, patch_size, patch_filter)
     43         patch_mask = np.ones(datas[0].shape,dtype=np.bool)
     44     else:
---> 45         patch_mask = patch_filter(datas[0], patch_size)
     46
     47     # get the valid indices

/work/.cache/poetry/kc-segmentation-DJpFP61h-py3.7/lib/python3.7/site-packages/stardist/models/base.py in <lambda>(y, p)
    196             inds = _ind_cache[k]
    197         else:
--> 198             patch_filter = (lambda y,p: self.max_filter(y, self.maxfilter_patch_size) > 0) if foreground_only else None
    199             inds = get_valid_inds((self.Y[k],)+self.channels_as_tuple(self.X[k]), self.patch_size, patch_filter=patch_filter)
    200             if self.sample_ind_cache:

/work/.cache/poetry/kc-segmentation-DJpFP61h-py3.7/lib/python3.7/site-packages/stardist/models/base.py in <lambda>(y, patch_size)
    175         if self.use_gpu:
    176             from gputools import max_filter
--> 177             self.max_filter = lambda y, patch_size: max_filter(y.astype(np.float32), patch_size)
    178         else:
    179             from scipy.ndimage.filters import maximum_filter

/work/.cache/poetry/kc-segmentation-DJpFP61h-py3.7/lib/python3.7/site-packages/gputools/convolve/generic_separable_filters.py in max_filter(data, size, res_g, sub_blocks)
    137         _filt = make_filter(_generic_filter_gpu_3d(FUNC = "(val>res?val:res)", DEFAULT = "-INFINITY"))
    138
--> 139     return _filt(data = data, size = size, res_g = res_g, sub_blocks=sub_blocks)
    140
    141

/work/.cache/poetry/kc-segmentation-DJpFP61h-py3.7/lib/python3.7/site-packages/gputools/convolve/generic_separable_filters.py in _filter(data, size, res_g, sub_blocks)
     86         if isinstance(data, np.ndarray):
     87             if sub_blocks is None or set(sub_blocks) == {1}:
---> 88                 return _filter_numpy(data, size)
     89             else:
     90                 # cut the image into tile and operate on every of them

/work/.cache/poetry/kc-segmentation-DJpFP61h-py3.7/lib/python3.7/site-packages/gputools/convolve/generic_separable_filters.py in _filter_numpy(data, size)
     78             data = data.astype(np.float32)
     79         data_g = OCLArray.from_array(data)
---> 80         return filter_gpu(data_g = data_g, size=size).get()
     81
     82     def _filter(data, size=10, res_g=None, sub_blocks=(1, 1, 1)):

/work/.cache/poetry/kc-segmentation-DJpFP61h-py3.7/lib/python3.7/site-packages/gputools/convolve/generic_separable_filters.py in _filt(data_g, size, res_g)
     65
     66         prog.run_kernel("filter_3_x", data_g.shape[::-1], None, data_g.data, res_g.data)
---> 67         prog.run_kernel("filter_3_y", data_g.shape[::-1], None, res_g.data, tmp_g.data)
     68         prog.run_kernel("filter_3_z", data_g.shape[::-1], None, tmp_g.data, res_g.data)
     69         return res_g

/work/.cache/poetry/kc-segmentation-DJpFP61h-py3.7/lib/python3.7/site-packages/gputools/core/oclprogram.py in run_kernel(self, name, global_size, local_size, *args, **kwargs)
     44             self._kernel_dict[name] = getattr(self,name)
     45
---> 46         self._kernel_dict[name](self._dev.queue,global_size, local_size,*args,**kwargs)
     47
     48

/work/.cache/poetry/kc-segmentation-DJpFP61h-py3.7/lib/python3.7/site-packages/pyopencl/__init__.py in kernel_call(self, queue, global_size, local_size, *args, **kwargs)
    886         # Note: This is only used for the generic __call__, before
    887         # kernel_set_scalar_arg_dtypes is called.
--> 888         return self._enqueue(self, queue, global_size, local_size, *args, **kwargs)
    889
    890     def kernel_capture_call(self, filename, queue, global_size, local_size,

<generated code> in enqueue_knl_filter_3_y(self, queue, global_size, local_size, arg0, arg1, global_offset, g_times_l, allow_empty_ndrange, wait_for)

MemoryError: clEnqueueNDRangeKernel failed: MEM_OBJECT_ALLOCATION_FAILURE

The error indicates, that there is no GPU memory left for the OpenCL helper computations (enabled with use_gpu=True), which happens if tensorflow grabs all the GPU memory at the start.
You can limit tensorflows to just use a fraction of the total memory, which should fix this
from csbdeep.utils.tf import limit_gpu_memory
limit_gpu_memory(.8, total_memory=12000) # or smaller if memory errors persist

Alternatively, you could switch OpenCL off with use_gpu=False (tensorflow will still run on GPU)

Hello again,
Sorry to ask for help again but hopefully this will also serve other people.
So the error I mentioned above was coming from the GPU, but now I think there might be a leakage in CPU memory.
I created a minimal code to try to reproduce what I see (code below).
If you run the code and look at the RAM of your computer, you should see (at least I see) that memory is constantly increasing epoch over epoch. On this example code is particularly visible and the end of an epoch, but when I run my real code, I get memory error  in the middle of an epoch (so the leak is not only coming from validation or end of an epoch).
Could you please try my code and see if you can reproduce the leakage?
Note that I'm using a dataloader using keras.Sequence which might be the reason for the leakage (as that's the only custom part I'm using).
# import libraries
import tensorflow as tf
import numpy as np

import os
from stardist.models import Config3D, StarDist3D

os.environ['CUDA_VISIBLE_DEVICES']= "0"# '1,0'

from tensorflow.keras.utils import Sequence
import stardist

from csbdeep.utils.tf import limit_gpu_memory
# adjust as necessary: limit GPU memory to be used by TensorFlow to leave some to OpenCL-based computations
limit_gpu_memory(0.7, total_memory=12000,)

# some custom functions to show the memory error

def calculate_extents(lbl, func=np.median):
    """
    Intermediate function : https://github.com/stardist/stardist/issues/57
    Aggregate bounding box sizes of objects in label images. """

    import numpy as np
    from collections.abc import Iterable
    from csbdeep.utils import _raise
    from skimage.measure import regionprops

    if (isinstance(lbl,np.ndarray) and lbl.ndim==4) or (not isinstance(lbl,np.ndarray) and  isinstance(lbl,Iterable)):
        return func(np.stack([calculate_extents(_lbl,func) for _lbl in lbl], axis=0), axis=0)

    n = lbl.ndim
    n in (2,3) or _raise(ValueError("label image should be 2- or 3-dimensional (or pass a list of these)"))

    regs = regionprops(lbl)
    if len(regs) == 0:
        return np.zeros(n)
    else:
        extents = np.array([np.array(r.bbox[n:])-np.array(r.bbox[:n]) for r in regs])
        return func(extents, axis=0)

def random_image(shape):
    """
    just a random imaga
    """
    return np.random.randint(0, 255, shape)

def diag_mask(shape, n_cells=3, cell_size=2, cell_dist=5):
    """
    Labels with cells on the diagonal
    """
    mask = np.zeros(shape, dtype=np.int16)
    for cell_id in range(1, n_cells+1):
        center_pos = cell_id*cell_dist
        mask[center_pos-cell_size:center_pos+cell_size,
             center_pos-cell_size:center_pos+cell_size,
             center_pos-cell_size:center_pos+cell_size,
             ] = cell_id
    return mask

class DataLoader(Sequence):
    """
    Dataloader to simulate reading from disk
    """
    def __init__(self, shapes_list, mode):
        self.x, self.mode = shapes_list, mode

        #retro compatibility with numpy
        self.ndim=3
        if mode == "Y":
            self.dtype = np.int
        else:
            self.dtype = np.float

    def __len__(self):
        return len(self.x)

    def __getitem__(self, idx):

        shape = self.x[idx]
        if self.mode == "X":
            return random_image(shape)
        else:
            return diag_mask(shape)

# Just a random bunch of 3D syntetic data
SHAPES_DIFF = [(100, 100, 100), (110, 190, 90), (100, 100, 110), (110, 100, 100), (100, 110, 100)]*20
X_DIFF = DataLoader(SHAPES_DIFF, mode="X")
Y_DIFF = DataLoader(SHAPES_DIFF, mode="Y")

extents = calculate_extents(Y_DIFF)
anisotropy = tuple(np.max(extents) / extents)
print('empirical anisotropy of labeled objects = %s' % str(anisotropy))

from stardist import Rays_GoldenSpiral
n_rays = 96

rays = Rays_GoldenSpiral(n_rays,
                         anisotropy=anisotropy
                        )

# Predict on subsampled grid for increased efficiency and larger field of view
grid = tuple(1 if a > 1.5 else 2 for a in anisotropy)
n_channel = 1
conf = Config3D (
    rays             = rays,
    grid             = grid,
    anisotropy       = anisotropy,
    use_gpu          = True,
    n_channel_in     = n_channel,
    train_patch_size = (64,64,64),
    train_batch_size = 1,
    train_epochs = 100,
    train_reduce_lr={'factor': 0.5, 'patience': 10, 'min_delta': 0},
    train_tensorboard=False
)
print(conf)
vars(conf)

model_name = f'any_name'
model = StarDist3D(conf, name=model_name, basedir="stardist_models")

model.train(X_DIFF, Y_DIFF,
            validation_data=(X_DIFF, Y_DIFF),
            epochs=10, #100,
            seed=42)

Thank you in advance for your precious help!

Hi @Optimox ,
I cannot reproduce this - although memory is increasing until epoch ~9 (which might be due to keras internal caching), it stabilizes after that and remains constant:

I slight adapted your code to log the memory consumption: https://gist.github.com/maweigert/98af7d72078ce3701dc186cc47b0c6a8 Could you run it and post the created file memory.log?

Hello @maweigert,
Thanks for your fast answer.
I just ran your code and my graph does not look like yours at all:

Here is the log file itself:
memory.log
My stardist version is 0.7.1. I noticed this warning message at the beginning of the training :
Error: Could not find scikit-tensor which is needed separable approximations...
pip install scikit-tensor-py3

I'll try to install it and rerun everything.

I just ran your code and my graph does not look like yours at all:

This is very strange. What OS are you on?

I'll try to install it and rerun everything.

No need to do that, the error message is from gputools, indicates that some components are missing, which are however not needed for stardist, and can be safely ignored.

I'm running everything with Linux on Ubuntu 20.4

I have no idea what the problem is, but trying to disable the internal data generator cache is worth a shot:
conf = Config3D (
    ...
    train_sample_cache = False
)

So I struggled quite a bit (A LOT ^^) on this.
I switched from cuda 10.1 cudnn7 to cuda11.0 cudnn8, after this I had to add these lines to make things work:
gpu_devices = tf.config.list_physical_devices('GPU')
for device in gpu_devices:
    tf.config.experimental.set_memory_growth(device, True)

With this setup up I have a far nicer graph for RAM consumption:

However when I switch to real data my RAM ran out of memory.
Switching to train_sample_cache = False now solves the problem, thanks @uschmidt83!
So I suspect that I had a problem with GPU drivers and CUDA that created a leak somewhere. But even with that you need to turn off caching otherwise the entire training set will still end up in your memory, defying the purpose of the DataLoader I'm using.
Also I now see this warning when launching the training:
WARNING:tensorflow | AutoGraph could not transform <function _gcd_import at 0x7fa74fb2db00> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Unable to locate the source code of <function _gcd_import at 0x7fa74fb2db00>. Note that functions defined in certain environments, like the interactive Python shell do not expose their source code. If that is the case, you should to define them in a .py source file. If you are certain the code is graph-compatible, wrap the call using @tf.autograph.do_not_convert. Original error: could not get source code
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert

Have you ever faced this? Should I be worried?

I switched from cuda 10.1 cudnn7 to cuda11.0 cudnn8, after this I had to add these lines to make things work:

Ok, good to now.  Thanks for reporting this! So the issue can be closed?

Also I now see this warning when launching the training:

Don't think that is an issue, I would just ignore it :)

authors: Optimox maweigert uschmidt83

repo: stardist/stardist


text:
Add global contract cache test for CachedState

The global cache update action will be tested in a subsequent PR (it needs some extra changes in pyo3).

This change is

Codecov Report

Merging #805 (9928181) into main (72c7b4f) will increase coverage by 0.05%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main     #805      +/-   ##
==========================================
+ Coverage   66.03%   66.08%   +0.05%
==========================================
  Files          40       40
  Lines        5352     5352
  Branches     5352     5352
==========================================
+ Hits         3534     3537       +3
+ Misses       1479     1476       -3
  Partials      339      339

see 1 file with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

authors: codecov-commenter giladchase

repo: starkware-libs/blockifier


text:
Project loading should happen asynchronously

Primarily, calling scarb metadata should be an asynchronous process that does not block project model updates like it does currently.
Requirements

This is a good opportunity to drop the detect_crates_for method and replace it with a proper abstraction. See the description for old issue #6124 for more clues.
Implement this in such a way, that when doing #6132 we won't have to change a lot here. Primarily, this means that this system should escape async world as early as possible. Use spawn_blocking like we attempt to do in with_db method.
Avoid calling project refreshing process (scarb metadata, cairo_project.toml parsing) if another instance is already running (but if one is running: always schedule at most 1 extra run to pick any changes that could happen while the previous run was running).
Gracefully deal with projects which loading fails. In such cases, avoid clearing relevant salsa inputs to preserve as much cached queries as possible. For example, a syntax error in the manifest file (this will make scarb metadata fail) shouldn't cause wiping of all analysis state. Check out what RA does here.
Handle detached files well (i.e. one without matching Scarb.toml/cairo_project.toml well). Do the same as Rust Analyzer does.

Gracefully deal with projects which loading fails. In such cases, avoid clearing relevant salsa inputs to preserve as much cached queries as possible. For example, a syntax error in the manifest file (this will make scarb metadata fail) shouldn't cause wiping of all analysis state

@mkaput  this is no longer the case, no? We just try to init unmanaged corelib in case metadata fails now

This is more about recovering from breakage when metadata did succeed already. Take a following scenario as an example:

You open OpenZeppelin/cairo-contracts
It analyses successfully.
You open Scarb.toml and make a syntax error.
scarb metadata will now refresh and it will fail

I don't want to lose all the analysis information due to this problem because it is easily recoverable: if we know something about the code already, and this information is reliable, let's persist that information instead of wiping it.

repo: starkware-libs/cairo


text:
Test secure field circle gen

This change is

#452  👈
dev

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @shaharsamocha7 and the rest of your teammates on  Graphite

repo: starkware-libs/stwo


text:
Documentation/Meta: API stability policy

Following the discussion in #848, this issue seeks to track the process of implementing the discussed policy.
TODO

[ ] Craft a document describing the specifics of our API stability/deprecation policy
[ ] Update deprecation utilities to the new requirements
[ ] Create a developer guideline for deprecations
[ ] Integrate the deprecation process with current tooling (i.e. linting stage, automated notes in documentation)
[ ] Integrate the changelog into the docs, include "migraton guide" for breaking changes

Hey @provinzkraut did you updated all the info in documentation.
If not can you help me
I am learning how to write techincal documentation it will a good training for me.

No, this isn't done yet, hence this issue still being open.
If you want to give it a shot, go ahead. If you need help, I recommend reaching out on our discord. I'm still away at the moment for Christmas, but I'll be able to review it / give input once I'm back.

Completed by #1131

authors: Kd-Here provinzkraut

repo: starlite-api/starlite


text:
Perks do not add/remove correctly

Save editor doesn't properly catch all perks that are on the character when loading the file, and allows you to add perks that already exist on the character. This leads to funny issues of having double the same perk, but it also means you can't remove the original.

Just another picture example

Strange. Please post the save file that this happened with and I'll take a look

Mauve (H) - 14Hrs 49Mins, 29 Days - Casstech Z14, Kalas (2).zip

So far I've tried all I can think of to replicate the issue but no luck.
Some context: The editor will first prioritize perks already owned by the character, and only then add whatever else is not owned to the selectable pool. This is to avoid this exact problem and as far as I've seen and re-tested it's doing its job.
I've also seen that this only happened to perks and not status effects, even though they both use the same system described above.
Is this the save after editing it for the first time? If not, can you also post that one and any others in between?

but it also means you can't remove the original.

As a workaround you can remove the dupes by unselecting one of the dupes and saving it like that. If you want to see it visually gone: unselecting the dupe, switching characters and then switching back to PC should remove the dupe from the list.