text:
MudFormComponent: Fix validation handling (#9215)

Description
Fixes validation handling in MudFormComponent:

Prioritize validation context of the more specific input component over the more general one from the surrounding form.
Skip 'internal' validation when there is an EditContext because all generated validation messages will get overwritten by OnValidationStateChanged once the validation handler of the EditContext is updating the validation state. This also prevents 'double validation' and possible issues with custom validators that might get executed twice and as a bonus there is a minimal performance gain as unneeded code parts are skipped.

Fixes #9215
How Has This Been Tested?
All existing unit tests have passed and manual testing of the forms section in MudBlazor.Docs.Server has been done.
Type of Changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation (fix or improvement to the website or code docs)

Checklist

[x] The PR is submitted to the correct branch (dev).
[x] My code follows the code style of this project.
[ ] I've added relevant tests. -> not needed, existing tests already cover form validation

Before I merge it, do we need any notes on little behavior change for migration guide or not or we just consider it as bug fix?

I'd argue this is a bug fix. However I suggested a clarification note in the docs of EditForm https://github.com/MudBlazor/MudBlazor/issues/9215#issuecomment-2226798752

Thanks!

Will this fix an issue we're seeing with Mud input components that have Required=true within an EditForm (not MudForm) with a custom validator?  Anything with required=true pops up a validation error rather than running our own fluent validator which does more conditional validation (and could skip required fields in certain instances)?

Will this fix an issue we're seeing

I would say the best thing is to try it out. It's part of MudBlazor starting from version 7.2.0.

Hmm, didn't realise it was already in release, in which case, it does not.  I'll need to raise a bug.

Mud input components that have Required=true within an EditForm (not MudForm) with a custom validator? Anything with required=true pops up a validation error rather than running our own fluent validator which does more conditional validation (and could skip required fields in certain instances)?

I don't think it will, based on my knowledge. When you set Require, it adds a required attribute to the underlying native HTML inputs. When used in conjunction with a form, the default HTML browser validation will be triggered, and this validation will always dominate, as in this case: https://github.com/MudBlazor/MudBlazor/issues/9409#issuecomment-2229173674. Unfortunately, I don't know if this validation can be suppressed.

Hmm, didn't realise it was already in release, in which case, it does not. I'll need to raise a bug.

Not sure if it's even possible to fix tho.

I think you can try to add the novalidate attribute to the EditForm

I think you can try to add the novalidate attribute to the EditForm

OK, I guess with v7, MudBlazor adds the required HTML tag where previously it didn't (and just appended * to the label to denote required).  Looks like adding the novalidate overrides the native handing of required.  Thanks for the help!

authors: SKSniperSK ScarletKuro elylv henon


text:
Fix net8.0 Templates

Fixes https://github.com/MudBlazor/Templates/issues/453
Fixes https://github.com/MudBlazor/Templates/issues/437
Use the new <MudThemeingProvider /> to avoid <MudPopoverProvider /> issues.
Some fixes to namespaces
Added install template, create all projects and build to pipeline
Other minor fixes

On Hold for https://github.com/MudBlazor/MudBlazor/pull/8102

Maybe someone want to test these.
You can download the repo and run the build script which will give you a directory of projects to test.
@Garderoben

I can confirm that the build script builds the test projects fine on windows

Works well, tried InteractiveAuto.
This has nothing to do with this PR, but I noticed that for small screen sizes the Menu disappears entirely.

Thats due to using MudDrawer.
I guess I should use MudGrid for the layout when the project is not Interative at root?

Thanks for looking :-)

In InteractivityAuto_AllInteractive it shows the burger menu which eventually works after a delay of 10 seconds or so

I notice that in InteractivityNone there is no .sln file, just a csproj file.

I notice that in InteractivityNone there is no .sln file, just a csproj file.

I believe that's because you only need a solution file for multi project solution.
So if webassembly (client porject) is not required it creates just a csproj
https://github.com/MudBlazor/Templates/blob/1ffe033ab440a8228c8c93abe6dfb330cec84f1f/src/mudblazor/.template.config/template.json#L39

@henon Thanks for your continued review.  I found the Home page description had the providers as tags in the description text which obviously caused some major issues.  I've added a fix if you would like to retest.  You will need to run the build script again after pull.

@henon Maybe one last pass through the templates.
Be aware interactive auto does not work correctly at present.  The web socket for BSS does not open until the wasm files have been downloaded.  This manifests itself in the fact that the counter page takes a long time to become interactive.
Should be fixed by MS in 8.0.2

I stumbled upon an exception on clicking the weather nav link in the InteractivityServer example.
System.InvalidOperationException: A component of type 'InteractivityServer.Components.Pages.Weather' has render mode 'InteractiveAutoRenderMode', but the required endpoints are not mapped on the server. When calling 'MapRazorComponents', add a call to 'AddInteractiveWebAssemblyRenderMode'. For example, 'builder.MapRazorComponents<...>.AddInteractiveWebAssemblyRenderMode()'
   at Microsoft.AspNetCore.Components.Endpoints.SSRRenderModeBoundary.AssertRenderModeIsConfigured[TRequiredMode](Type componentType, IComponentRenderMode specifiedMode, IComponentRenderMode[] configuredModes, String expectedCall)
   at Microsoft.AspNetCore.Components.Endpoints.SSRRenderModeBoundary.AssertRenderModeIsConfigured(HttpContext httpContext, Type componentType, IComponentRenderMode renderMode)
   at Microsoft.AspNetCore.Components.Endpoints.SSRRenderModeBoundary..ctor(HttpContext httpContext, Type componentType, IComponentRenderMode renderMode)
   at Microsoft.AspNetCore.Components.Endpoints.EndpointHtmlRenderer.ResolveComponentForRenderMode(Type componentType, Nullable`1 parentComponentId, IComponentActivator componentActivator, IComponentRenderMode renderMode)
   at Microsoft.AspNetCore.Components.ComponentFactory.InstantiateComponent(IServiceProvider serviceProvider, Type componentType, IComponentRenderMode callerSpecifiedRenderMode, Nullable`1 parentComponentId)
   at Microsoft.AspNetCore.Components.RenderTree.Renderer.InstantiateChildComponentOnFrame(RenderTreeFrame[] frames, Int32 frameIndex, Int32 parentComponentId)
   at Microsoft.AspNetCore.Components.RenderTree.RenderTreeDiffBuilder.InitializeNewComponentFrame(DiffContext& diffContext, Int32 frameIndex)
   at Microsoft.AspNetCore.Components.RenderTree.RenderTreeDiffBuilder.InitializeNewSubtree(DiffContext& diffContext, Int32 frameIndex)
   at Microsoft.AspNetCore.Components.RenderTree.RenderTreeDiffBuilder.InsertNewFrame(DiffContext& diffContext, Int32 newFrameIndex)
   at Microsoft.AspNetCore.Components.RenderTree.RenderTreeDiffBuilder.AppendDiffEntriesForRange(DiffContext& diffContext, Int32 oldStartIndex, Int32 oldEndIndexExcl, Int32 newStartIndex, Int32 newEndIndexExcl)
   at Microsoft.AspNetCore.Components.RenderTree.RenderTreeDiffBuilder.ComputeDiff(Renderer renderer, RenderBatchBuilder batchBuilder, Int32 componentId, ArrayRange`1 oldTree, ArrayRange`1 newTree)
   at Microsoft.AspNetCore.Components.Rendering.ComponentState.RenderIntoBatch(RenderBatchBuilder batchBuilder, RenderFragment renderFragment, Exception& renderFragmentException)
   at Microsoft.AspNetCore.Components.RenderTree.Renderer.ProcessRenderQueue()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Components.RenderTree.Renderer.ProcessRenderQueue()
   at Microsoft.AspNetCore.Components.RenderTree.Renderer.AddToRenderQueue(Int32 componentId, RenderFragment renderFragment)
   at Microsoft.AspNetCore.Components.ComponentBase.StateHasChanged()
   at Microsoft.AspNetCore.Components.ComponentBase.CallOnParametersSetAsync()
   at Microsoft.AspNetCore.Components.ComponentBase.RunInitAndSetParametersAsync()
   at Microsoft.AspNetCore.Components.Rendering.ComponentState.SetDirectParameters(ParameterView parameters)
   at Microsoft.AspNetCore.Components.RenderTree.Renderer.RenderRootComponentAsync(Int32 componentId, ParameterView initialParameters)
   at Microsoft.AspNetCore.Components.HtmlRendering.Infrastructure.StaticHtmlRenderer.BeginRenderingComponent(IComponent component, ParameterView initialParameters)
   at Microsoft.AspNetCore.Components.Endpoints.EndpointHtmlRenderer.RenderEndpointComponent(HttpContext httpContext, Type rootComponentType, ParameterView parameters, Boolean waitForQuiescence)
   at Microsoft.AspNetCore.Components.Endpoints.RazorComponentEndpointInvoker.RenderComponentCore(HttpContext context)
   at Microsoft.AspNetCore.Components.Endpoints.RazorComponentEndpointInvoker.RenderComponentCore(HttpContext context)
   at Microsoft.AspNetCore.Components.Rendering.RendererSynchronizationContext.<>c.<<InvokeAsync>b__10_0>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddlewareImpl.Invoke(HttpContext context)

The weather page in InteractivityServer_AllInteractive works

On the other hand, if you choose the "Per page/component" Interactivity Location,
it is necessary to include the
<MudPopoverProvider />
<MudDialogProvider />
<MudSnackbarProvider />

component on every interactive page.

This sentence should be fixed.

On the other hand, if you choose the "Per page/component" Interactivity Location,
it is necessary to include the
<MudPopoverProvider />
<MudDialogProvider />
<MudSnackbarProvider />

component on every interactive page.

This sentence should be fixed.

Plural for components spacing?

Added fi

I stumbled upon an exception on clicking the weather nav link in the InteractivityServer example.
System.InvalidOperationException: A component of type 'InteractivityServer.Components.Pages.Weather' has render mode 'InteractiveAutoRenderMode', but the required endpoints are not mapped on the server. When calling 'MapRazorComponents', add a call to 'AddInteractiveWebAssemblyRenderMode'. For example, 'builder.MapRazorComponents<...>.AddInteractiveWebAssemblyRenderMode()'
   at Microsoft.AspNetCore.Components.Endpoints.SSRRenderModeBoundary.AssertRenderModeIsConfigured[TRequiredMode](Type componentType, IComponentRenderMode specifiedMode, IComponentRenderMode[] configuredModes, String expectedCall)
   at Microsoft.AspNetCore.Components.Endpoints.SSRRenderModeBoundary.AssertRenderModeIsConfigured(HttpContext httpContext, Type componentType, IComponentRenderMode renderMode)
   at Microsoft.AspNetCore.Components.Endpoints.SSRRenderModeBoundary..ctor(HttpContext httpContext, Type componentType, IComponentRenderMode renderMode)
   at Microsoft.AspNetCore.Components.Endpoints.EndpointHtmlRenderer.ResolveComponentForRenderMode(Type componentType, Nullable`1 parentComponentId, IComponentActivator componentActivator, IComponentRenderMode renderMode)
   at Microsoft.AspNetCore.Components.ComponentFactory.InstantiateComponent(IServiceProvider serviceProvider, Type componentType, IComponentRenderMode callerSpecifiedRenderMode, Nullable`1 parentComponentId)
   at Microsoft.AspNetCore.Components.RenderTree.Renderer.InstantiateChildComponentOnFrame(RenderTreeFrame[] frames, Int32 frameIndex, Int32 parentComponentId)
   at Microsoft.AspNetCore.Components.RenderTree.RenderTreeDiffBuilder.InitializeNewComponentFrame(DiffContext& diffContext, Int32 frameIndex)
   at Microsoft.AspNetCore.Components.RenderTree.RenderTreeDiffBuilder.InitializeNewSubtree(DiffContext& diffContext, Int32 frameIndex)
   at Microsoft.AspNetCore.Components.RenderTree.RenderTreeDiffBuilder.InsertNewFrame(DiffContext& diffContext, Int32 newFrameIndex)
   at Microsoft.AspNetCore.Components.RenderTree.RenderTreeDiffBuilder.AppendDiffEntriesForRange(DiffContext& diffContext, Int32 oldStartIndex, Int32 oldEndIndexExcl, Int32 newStartIndex, Int32 newEndIndexExcl)
   at Microsoft.AspNetCore.Components.RenderTree.RenderTreeDiffBuilder.ComputeDiff(Renderer renderer, RenderBatchBuilder batchBuilder, Int32 componentId, ArrayRange`1 oldTree, ArrayRange`1 newTree)
   at Microsoft.AspNetCore.Components.Rendering.ComponentState.RenderIntoBatch(RenderBatchBuilder batchBuilder, RenderFragment renderFragment, Exception& renderFragmentException)
   at Microsoft.AspNetCore.Components.RenderTree.Renderer.ProcessRenderQueue()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Components.RenderTree.Renderer.ProcessRenderQueue()
   at Microsoft.AspNetCore.Components.RenderTree.Renderer.AddToRenderQueue(Int32 componentId, RenderFragment renderFragment)
   at Microsoft.AspNetCore.Components.ComponentBase.StateHasChanged()
   at Microsoft.AspNetCore.Components.ComponentBase.CallOnParametersSetAsync()
   at Microsoft.AspNetCore.Components.ComponentBase.RunInitAndSetParametersAsync()
   at Microsoft.AspNetCore.Components.Rendering.ComponentState.SetDirectParameters(ParameterView parameters)
   at Microsoft.AspNetCore.Components.RenderTree.Renderer.RenderRootComponentAsync(Int32 componentId, ParameterView initialParameters)
   at Microsoft.AspNetCore.Components.HtmlRendering.Infrastructure.StaticHtmlRenderer.BeginRenderingComponent(IComponent component, ParameterView initialParameters)
   at Microsoft.AspNetCore.Components.Endpoints.EndpointHtmlRenderer.RenderEndpointComponent(HttpContext httpContext, Type rootComponentType, ParameterView parameters, Boolean waitForQuiescence)
   at Microsoft.AspNetCore.Components.Endpoints.RazorComponentEndpointInvoker.RenderComponentCore(HttpContext context)
   at Microsoft.AspNetCore.Components.Endpoints.RazorComponentEndpointInvoker.RenderComponentCore(HttpContext context)
   at Microsoft.AspNetCore.Components.Rendering.RendererSynchronizationContext.<>c.<<InvokeAsync>b__10_0>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddlewareImpl.Invoke(HttpContext context)

Should be fixed now

Btw, this is the first time I check any of the Auth examples. The InteractivityAuto_UseMain looks like any other example, where is the auth?

Btw, this is the first time I check any of the Auth examples. The InteractivityAuto_UseMain looks like any other example, where is the auth?

There is no Auth for that example.  The examples with Auth have it in the project name.

Can we merge this or there are still some concerns left?

Not from my side.

Any new about this topic?

@mikes-gh should we merge it?

We can merge its not perfect but better than current.

@Garderoben can we push the nuget ASAP?
As I understand only you can do it. @mikes-gh would be nice to do a github pipeline.

@Garderoben can we push the nuget ASAP? As I understand only you can do it. @mikes-gh would be nice to do a github pipeline.

mike is working on it, i can do a release now

source: gharchive/pull-request

authors: Garderoben ScarletKuro ashpetrosyan97 henon mikes-gh

repo: MudBlazor/Templates

license: MIT

license_type: permissive

license_source: github-api


text:
feat(datahub-client): add java file emitter

Checklist

[ ] The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
[ ] Links to related issues (if applicable)
[ ] Tests for the changes have been added/updated (if applicable)
[ ] Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
[ ] For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

merged

repo: MugdhaHardikar-GSLab/datahub


text:
I cannot create a world

When i use /mvcreate to create a world, it displays Failed. and i can't import that world.
command: /mvcreate create normal -t flat
Console Log: https://paste.ubuntu.com/p/JSG76dGcbZ/

Interestingly enough, I have been seeing more and more of these reports, so I'm beginning to think something's wrong.
Here are some things to try:

Make sure your server software is up to date, be it Spigot or Paper. CraftBukkit is generally not recommended these days and isn't a great choice.
Make sure you're Multiverse Core and its subplugins are up to date. The latest versions of each plugin are available at here.
Try deleting the generated world's folder (in this case, the create folder) and removing the entry from Multiverse Core's worlds.yml file. Then try to create it again.

Thank you for your answer, but that didn't help. It displays Failed. again.
Console log: https://paste.ubuntu.com/p/2dbF9qGkTz/

I'm using the latest version Multiverse-Core 4.1.0 and the server version is Spigot 1.15.2

I've been trying to reproduce this but I've been having some trouble. Can you please provide the following:

Which host you are using (or if you're self hosting)
Which version of Java your server runs on
The output of /version
The output of /mv version

The server is running on my computer (Windows 10).
Java version: Click here
/version output: Click here
/mv version output: Click here

I can use /mvcreate to create some worlds before, but now I can't.

You can't create any world?

Yes.

I'm a Chinese and I'm not good at English.

I used /mvcreate nether2 nether just now and here is the output in the console.

That's okay. Are there any errors on startup or any errors on shutdown?

Let me have a look.

Here's the output of Multiverse-Core.

I don't know if that's an error.

There aren't any errors when the server is shutting down.

Okay. I have some more questions:

Is your system's language English? If not, which language?
Do you have Administrator access and sufficient privileges to the folder containing the server?

My system language is Chinese.
I'm the Administrator user of my computer.

Can you try with Paper and provide the log if you get a similar error?

Paper?

Yes, PaperMC, a fork of Spigot.
Available here.

I'll try that later. Thank you.

I'm downloading PaperMC, but I still want to use Spigot.
Can you fix those errors in Spigot?
Thank you very much!

We'll fix the issue if it's an issue with Spigot. I just want to see if it also happens with Paper.

OK.

There aren't any errors when I use /mvcreate in Paper. I hope that never will be wrong.

Thanks for giving it a try. Can you upload your server.properties, bukkit.yml, and spigot.yml files?

Here's the server.properties.

spigot.yml
bukkit.yml

In your server.properties, make sure to set gamemode, then try to create a world!

Yeah! I can create world now! Thank you very much.

Multiverse-Core is really a good plugin.

Awesome! Thanks for your kind remark. I'll see if something can be done to prevent this in the future.

But I have another question:
When using /reload, the config of Multiverse-Core will not save, so I must use /stop to restart.
Can you fix this error?

I will look into it, although, using /reload is generally not recommended.

I know that, but /reload is faster than /stop, so I like to use /reload

source: gharchive/issue

repo: Multiverse/Multiverse-Core


text:
Add support for configuring extra bridges

We have a requirement to pass multiple network interfaces through to the kubernetes nodes.  I've implemented this by adding a list of extraBridges for each instance that is used to create additional network_interface sections in the libvirt_domain and populate the cloud_init_network_static.tpl file to configure the new network interfaces in the VM.
If you're happy to proceed I'll create a pull request.  Can any of the issues I've created today be grouped into a single pull request or do you want one for each issue?

Hi,
thanks again for opening all these issues.
If you could open separate pull requests, I would be very grateful as we can then discuss each issue separately further.

authors: MusicDin seumasdunlop

repo: MusicDin/kubitect


text:
Want to add a PR template for this repo

Please assign me under GSSOC'24

please clarify what do you want and how to solve this this issue also explain PR template in detail with an example

I would like to work on this issue. Please assign it to me.

assigned both @AnitSarkar123 and @suhanipaliwal who will first raise pr and first serve

Please add the label of appropriate level as per gssoc guidelines on this issue.

Please add the label of appropriate level as per gssoc guidelines on this issue.

Please add the label of appropriate level as per gssoc guidelines on this issue.

authors: AnitSarkar123 suhanipaliwal techmannih


text:
希望增加消息转发至HTTP服务器的功能

issue规范

MacOS版本 : 无
微信版本 : 无
小助手版本 : 无
BUG描述 : 否
操作场景 :  现在的小助手有转发文本消息到其它用户的功能，希望能够支持用户提供自定义的HTTP接口，小助手将文本消息提交到这个HTTP接口，方便有一定开发能力的用户根据微信的消息做一些自定义的业务，非常感谢！

@MustangYM ， 谢谢

@iscaswang 哈喽，消息发送至HTTP服务器的这个需求你有找到其他的解决方案吗？我也在找，如果有其他好的方案可以分享下，谢谢啦

没有哦，源代码作者不再公布了，而且这个issue作者也没看，不知道telegram上的消息他们会不会看，我现在都登录不了TG了。不过我用之前提供的老版本源代码2.6左右的改了个，马马虎虎可以把文本消息发到我的服务器上，你不是ios开发者吗？你可以用老的代码参考下，然后适配新的微信版本，搞个自己想要的功能啊，我不是搞ios开发的，不太熟ios和mac程序插件开发的，所以只能改改老的版本

repo: MustangYM/WeChatExtension-ForMac


text:
可以在Alfred的workflow里面选择会话之后添加输入法切换为中文的功能嘛～

每次切换好麻烦的说～

每次切换好麻烦的说～

暂时没这打算

authors: MustangYM MyLullaby


text:
检测僵尸粉，对方有提示

如果好友中有企业微信好友，会生成企业微信群聊，对方会有群聊提示。
而且如果要退出检测僵尸粉的功能后，会自动退出群聊，但好友还在群中，非常尴尬。😂

@iwangjin 企业微信这个我确实没想到，我会尽快修复这个问题

我。。。香菇

我今天刚刚使用完这个功能，微信好友中也有企业微信好友，并没有这个现象

如果好友中有企业微信好友，会生成企业微信群聊，对方会有群聊提示。
而且如果要退出检测僵尸粉的功能后，会自动退出群聊，但好友还在群中，非常尴尬。😂

同尴尬

已下架此功能

authors: MustangYM ParfoisMeng iwangjin shadow198 supergithuber


text:
企业微信更新到3.0.28.2286 安装小助手后，打开闪退，无法运行

没有提示错误，打开就闪退

+1 ，出现相同的问题。不过看这个仓库已经8个多月没有更新了，作者已经忘了吧。。。

有好心人接着更新吗？

@KevinLex @fengxuway @Smi1e96 [已修复]（https://github.com/MustangYM/WeChatICU-ForMac/releases/tag/v1.0.1）

repo: MustangYM/WeChatICU-ForMac


text:
arm

系统支持arm吗

在，1

repo: MyMonsterCat/RapidOcr-Java


text:
How to access the client secret from a genesyscloud_oauth_client resource

Hi,
How does one get the client secret from a client credentials OAuth resource created with genesyscloud_oauth_client
It seems like a standard use-case but I can't find it documented anywhere. The secret seems to be exposed on the API at https://developer.genesys.cloud/devapps/api-explorer#get-api-v2-oauth-clients--clientId-
Many thanks

Hi,
Terraform is used to create and manage resources, it is not intended to be used to extract read only data, in this case the client secret, from the created oauth client (or at least our provider is not). Cx as Code will manage your oauth clients but if you need to get the oauth secrets you will need to find the oauth client created by terraform in the UI or call GET /api/v2/oauth/clients/{clientId}.
If you need to get the id of oauth client you created you can use this command to read the resource's state
terraform state show genesyscloud_oauth_client.<resource_name>

I would also recommend installing our CLI tool, if you don't already have it. It can call our API's and is a helpful tool when using Cx as Code for situations like this.
Regards,
Declan

Thanks @dginty4 but I thought it was pretty standard practice to expose properties of terraform resources to be used in properties of other resources.

As far as I know the only properties we expose are those used to manage the resource, we don't expose read-only attributes. What is your use case for this? I'm not aware of any resources that reference the client secret.

@tarlingovo  is correct.  The secret is exposed through this API endpoint:

I can see a use case for exposing the secret through Terraform.  We're building a set of scripts to create several resources common to many of our customers.  We're running these scripts via automated means, such as Azure DevOps pipelines and Terraform CLI.  The resources include an OAuth client used for dedicated cloud functions, and the client uses client-credentials.  I can get the id of this newly created client, but I can't access the secret from the Terraform block.  That means I'll have to run a separate API command after the Terraform script to get the OAuth credentials and store them in my password repository.
If the genesyscloud_oauth_client exposed the secret so that I could define it in an output, then I would not need a second step.  As a reminder, Terraform supports a sensitive flag to prevent the value from being logged:
output "oauth_client_secret" {
  description = "The Azure cloud-based resources OAuth Client Secret"
  value       = genesyscloud_oauth_client.app_cloud_functions.secret
  sensitive = true
}

I don't see much risk in exposing this property in the Terraform provider when it's already exposed in the API.  Thanks for your consideration.


text:
Invalid JSON String: "{}[]{}[]{}[]{}[]{}"

What is the output link of /viaversion dump?
http://hastebin.com/ugemedalak.json
Are you using any additional software like BungeeCord? If so, what software and version? (Please
BungeeCord (latest) with ServerListPlus (latest), SkinsRestorer (Shevchik's version) and NuVotifier (latest)
How does this error happen? login? Using an item?
Randomly from what I saw so far.
Is there an error in the console? Use pastebin.com. Is there a kick message?
Just this:
[01:50:14 INFO]: Invalid JSON String: "{}[]{}[]{}[]{}[]{}" Please report this issue to the ViaVersion Github: us.myles.viaversion.libs.gson.stream.MalformedJsonException: Use JsonReader.setLenient(true) to accept malformed JSON at line 1 column 4 path $
[01:50:14 INFO]: Invalid JSON String: "{}[]{}[]{}[]{}[]{}" Please report this issue to the ViaVersion Github: us.myles.viaversion.libs.gson.stream.MalformedJsonException: Use JsonReader.setLenient(true) to accept malformed JSON at line 1 column 4 path $

[01:58:54 INFO]: ViaVersion 0.9.10-SNAPSHOT

You're using a plugin that's sending this data. I'd suggest you work out what it is and remove it, we can't help if you're using a plugin sending bad data.

Any tips on how can I identify that plugin? The issue is somewhat random...

Remove plugins? Honestly can't really help other than the above.
Ensure they're up to date, ish?
The best idea is to try and work out when it happens or if there's anything useful before that error in the console like a player typing a command.

repo: MylesIsCool/ViaVersion

license: mit

license_source: bigquery


text:
Update problem statement

tl;dr

[x] Adds a missing comma
[x] Adds a few educational resources

Thank you for the contribution @amorriscode, looks good! :+1: Merging this

authors: amorriscode villesundell

repo: MystenLabs/awesome-move


text:
docs-sdk: fixed typo in README splitCoins example

Description
There was a typo in the Transfer SUI section of the README of the SDK.
The error prevented the example code to work properly, resulting in the following error:
StructError: At path: amounts -- Expected an array value, but received: [object Object]

The error was reported by several users on Discord, e.g. https://discord.com/channels/916379725201563759/955861929346355290/1098875792873762857

Reply
On Sat, May 6, 2023, 10:19 AM coccoinomane @.***> wrote:

Description
There was a typo in the Transfer SUI
https://github.com/coccoinomane/sui/blob/1090c2d487c574b7f5b58c8775a3b0060c4dee19/sdk/typescript/README.md#transfer-sui
section of the README of the SDK.
The error prevented the example code to work properly, resulting in the
following error:
StructError: At path: amounts -- Expected an array value, but received: [object Object]
The error was reported by several users on Discord, e.g.
https://discord.com/channels/916379725201563759/955861929346355290/1098875792873762857
You can view, comment on, or merge this pull request online at:
https://github.com/MystenLabs/sui/pull/11781
Commit Summary

1090c2d
https://github.com/MystenLabs/sui/pull/11781/commits/1090c2d487c574b7f5b58c8775a3b0060c4dee19
docs-sdk: fixed README typo in splitCoins example

File Changes
(1 file https://github.com/MystenLabs/sui/pull/11781/files)

M sdk/typescript/README.md
https://github.com/MystenLabs/sui/pull/11781/files#diff-9be9f99d130daf9a9aea3153e83306403823122bffa83285f1545b5048f04dda
(2)

Patch Links:

https://github.com/MystenLabs/sui/pull/11781.patch
https://github.com/MystenLabs/sui/pull/11781.diff

—
Reply to this email directly, view it on GitHub
https://github.com/MystenLabs/sui/pull/11781, or unsubscribe
https://github.com/notifications/unsubscribe-auth/A7BX2D7YY5XYCI4S4QCU663XEXNNRANCNFSM6AAAAAAXX34RZU
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

authors: coccoinomane petvaizAkhtar

repo: MystenLabs/sui


text:
[GraphQL/Cursor] Coin pagination

Description
Use opaque cursors and the pagination framework for querying coin objects. Introduces a new ExactTypeFilter that represents a type filter that requires an exact match (not a prefix).
Test Plan
New E2E test for coins:
sui-graphql-e2e-tests$ cargo nextest \
  -j 1 --features pg_integration     \
  -- objects/coin.move

Stack

#15661
#15695
#15698
#15699
#15701
#15710
#15711
#15712
#15713
#15721

Staging for land here!


text:
🛑 OARC ADSB is down

In 38bff3c, OARC ADSB (https://adsb.oarc.uk/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: OARC ADSB is back up in 1086e9d after 8 minutes.

repo: MysterAitch/oarc-monitor


text:
Dig conformance testing

TODO

[x] failing spec: handle out-of-bounds indexes into Structs
[x] broken spec: probe for range support before testing ranges on String#[] and Integer#[] (unsupported in 2.6.x)

There's more work to be done here, but merging for now as 1) we're green, and 2) I'd like to work on other features for a bit


text:
Add more details about options to create_merge_request doc

To avoid issues like the https://github.com/NARKOZ/gitlab/issues/323 this MR aims to improve the doc of create_merge_request command.
#hacktoberfest

@fabioperrella Nice!
@NARKOZ What do you mean by Hacktoberfest? Can this be merged? :D

@timofonic -> https://hacktoberfest.digitalocean.com/

Thank you.

authors: NARKOZ fabioperrella timofonic

repo: NARKOZ/gitlab


text:
[CHORE] Migrate from Jest to Node test runner

Removes Jest test runner and replaces its use with the built in Node test runner.
Faster, simpler, cleaner.
Absolutely drop in - just import describe, it, test, beforeEach from node:test and import expect from expect
Resolves #33

@dyst5422 the ticket mentioned node 20, but I noticed the project is using node 18 still. Do we need to move to node 20 as well?

No, they stabilized the test runner in a minor version of node 18 - I can run it with 18.16, though it may work with earlier minor versions

Cool, I will give it a try now.

repo: NASA-AMMOS/aerie-ts-user-code-runner


text:
Update README.md

test workflow

workflows run with branch protection and trigger

repo: NASA-AMMOS/aerie


text:
Test not working

Dear developers,
I used aMeta January-February 2023 and it was working perfectly. Now I need to run it again but when I run the test is not working.
./runtest.sh -j 30
This looks like the first test run... Installing bioconda packages...
No validator found for JSON Schema version identifier 'http://json-schema.org/draft/2020-12/schema#'
Defaulting to validator for JSON Schema version 'https://json-schema.org/draft/2020-12/schema'
Note that schema file may not be validated correctly.
Excluding samples 'foobar' from analysis
Restricting analysis to samples 'foo','bar'
No validator found for JSON Schema version identifier 'http://json-schema.org/draft/2020-12/schema#'
Defaulting to validator for JSON Schema version 'https://json-schema.org/draft/2020-12/schema'
Note that schema file may not be validated correctly.
Changing directory from /home/user/aMeta/.test to /home/user/aMeta/workflow
Changing directory back to /home/user/aMeta/.test
Building DAG of jobs...
Creating conda environment ../workflow/envs/krona.yaml...
Downloading and installing remote packages.
Cleaning up conda package tarballs and package cache.
Environment for /home/user/aMeta/workflow/rules/../envs/krona.yaml created (location: .snakemake/conda/01d22bc3188d41766b4b53f4d700f1c9_)
Creating conda environment ../workflow/envs/multiqc.yaml...
Downloading and installing remote packages.
Cleaning up conda package tarballs and package cache.
Environment for /home/user/aMeta/workflow/rules/../envs/multiqc.yaml created (location: .snakemake/conda/73ec0bf71681323174e5c299cdb30232_)
Creating conda environment ../workflow/envs/r.yaml...
Downloading and installing remote packages.
Cleaning up conda package tarballs and package cache.
Environment for /home/user/aMeta/workflow/rules/../envs/r.yaml created (location: .snakemake/conda/af3138f89c22b5caa67dfec4b96a8933_)
Creating conda environment ../workflow/envs/bowtie2.yaml...
Downloading and installing remote packages.
Cleaning up conda package tarballs and package cache.
Environment for /home/user/aMeta/workflow/rules/../envs/bowtie2.yaml created (location: .snakemake/conda/c42776cb10655143deab9a0ad9c60817_)
Creating conda environment ../workflow/envs/malt.yaml...
Downloading and installing remote packages.
Cleaning up conda package tarballs and package cache.
Environment for /home/user/aMeta/workflow/rules/../envs/malt.yaml created (location: .snakemake/conda/2b924bbb596d214e9d92d63009fdc853_)
Creating conda environment ../workflow/envs/cutadapt.yaml...
Downloading and installing remote packages.
Cleaning up conda package tarballs and package cache.
Environment for /home/user/aMeta/workflow/rules/../envs/cutadapt.yaml created (location: .snakemake/conda/066d8bc8ace1569fd902f9fc765557d8_)
Creating conda environment ../workflow/envs/mapdamage.yaml...
Downloading and installing remote packages.
The last bit (mapdamage) is taking hours. I don't remember the test run to take this long when I ran it last year. Could this be related to this problem? : https://github.com/NBISweden/aMeta/issues/164

Thanks for reporting @mudymudy! The test does not seem to be crushing from the output you posted above. Are you running the test on a node / laptop with at least 10-16 GB of RAM? Due to some latest modifications, mapdamage might take a bit more RAM than previously. Also, if the installation of the mapdamage environment takes hours I would suspect problems with internet connections because I regularly run the test in different clusters and so far has not discovered that it hangs on the installation step. However, I suspect that if you are running the test on an login node that has less than 10-16 GB or unstable internet connection, this might be an explanation. Please try to run the test either on your laptop or a compute node on an HPC with more RAM and let us know if that helped. Otherwise, we will need to take a closer look and inspect the issue in details. Thanks for your valuable feedback!

Thank you for your reply.
I killed the test run because it was frozen in that step, and decided to just run aMeta with just 1 sample. Before running it I checked the RAM:
$ free -hg
total        used        free      shared  buff/cache   available
Mem:           1.0Ti       7.4Gi        26Gi        11Mi       972Gi       992Gi
Swap:          1.0Ti       3.4Gi       1.0Ti
So I don't think RAM is a problem. After running for a bit it failed here:
[bam_index_build2] fail to open the BAM file.
Waiting at most 5 seconds for missing files.
MissingOutputException in rule Bowtie2_Alignment in file /user/home/aMeta/workflow/rules/align.smk, line 27:
Job 5  completed successfully, but some output files are missing. Missing files after 5 seconds. This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait:
results/BOWTIE2/file1.concat/AlignedToBowtie2DB.bam
results/BOWTIE2/file1.concat/AlignedToBowtie2DB.bam.bai
I tried with incrementing the latency but the same happened. After a minute every file in the BOWTIE2 results folder was deleted. Not sure why.
My conda versions:
$ mamba -V
mamba 1.5.7
conda 24.5.0
Any help would be appreciated!! Thank you again.

Dear @mudymudy, thanks! I advise you not to proceed with the real sample run until the test run completed successfully. If you have not set aMeta properly up there can be a number of issues with very misleading error messages that do not point at the right cause of the problem. Are you a member of the SPAAM community and using their slack channel? If not, please drop me a line here nikolay.oskolkov@scilifelab.se and we could schedule a zoom where I can assist you with installation. I would first make sure that the testrun works.

Hi Nikolay,
Thank you for all your help, it is very much appreciated! After doing what you suggested:
rm -rf aMeta
conda remove -n aMeta --all
Then install it again but this time adding this line:

snakemake=6.3.0

in:
aMeta/workflow/envs/environment.yaml
and finally replacing conda with mamba frontend in:
/aMeta/.test/runtesh.sh
Made aMeta test run work!

Excellent @mudymudy, this is good to know. We should probably make mamba default frontend in the tesrun script @percyfal. Strangely enough it is still conda by default while I thought we switched to mamba long time ago.

@LeandroRitter as of conda version >= 23.10.0 the solver is libmamba by default, therefore there should be no need to change to mamba. As the documentation of the runtest script states, if you have a conda version that is lower than 23.10.0 you can run the test with the command CONDA_FRONTEND=mamba ./runtest.sh -j 1

@percyfal true, it is written in the runtest.sh but in the case of @mudymudy there was likely an old conda used, therefore environment installation was taking too long, and the info about conda vs. mamba was somehow missed. So I wonder whether we should replace this line in testrun.sh:
CONDA_FRONTEND="${CONDA_FRONTEND:=conda}"
with this line
CONDA_FRONTEND=mamba
to avoid this issues in the future? In the case of @mudymudy, specifying mamba explicitly was the solution, while it took us quite some time to figure this out :))

Addressed in #172. Closing for now as this seems related to the conda frontend.

authors: LeandroRitter mudymudy percyfal

repo: NBISweden/aMeta


text:
The computed CAPE value is not correct

There was typo in the variable name for computed cape value: cape_val VS. cape_val4. Deleted cape_val4.

John will install a development version of the MET master_v8.1 branch on Theia and ask Perry to test the derivation of CAPE there.  Tara and Jonathan are the NCAR scientists working on this and Howard is the engineer fixing it.

The GitHub/NCAR/MET master_v8.1 branch is compiled on theia here:
/scratch4/BMC/dtc/MET/MET_development/MET-master_v8.1/met/bin
Run commands like this:
module use /contrib/modulefiles
module load intel anaconda/anaconda2-4.4.0 met/8.1
/scratch4/BMC/dtc/MET/MET_development/MET-master_v8.1/met/bin/pc2nc

Howard merged the bugfix branch into develop.  And Julie compiled a tarball built from the develop branch on WCOSS (tide) for Perry to test:
module use /global/noscrub/Julie.Prestopnik/modulefiles
module load met/8.1_dev

Perry corrected the CAPE output in the VSDB log files and we should do a new comparison of CAPE values between VSDB and PB2NC to see if they are closer.
VSDB CAPE values from log file:
/stmpp2/Perry.Shafran/tmpnwprd/verf_gridtobs_nam_00/00/gto.nam00.out
MET CAPE values from PB2NC:
/meso/noscrub/Perry.Shafran/metplus_test/nam/conus_sfc/prepbufr.nam.2019091000.nc
Also need to compare the PBL values to see how they match up.

On 9/20/2019 Perry confirmed that the CAPE values computed by PB2NC match the VSDB values very well.
There are 2 main source of difference:
(1) VSDB uses all input vertical levels in the CAPE computation, while MET only uses the levels which meet the specified quality marker threshold.  This leads to a difference, but setting the PB2NC quality marker threshold to the maximum value of 15 eliminates it.  And having the ability in MET to filter by quality marker is preferable anyway.
(2) VSDB computes CAPE rounded to the nearest integer, while MET computes CAPE with a decimal component.  And the higher precision in MET is preferable anyway.

authors: JohnHalleyGotway hsoh-u

repo: NCAR/MET


text:
TF Widget fails to display histogram

To reproduce:

Follow steps reported in issue #917  to have a wireframe rendered.
Go to "Appearance" tab. Notice there's nothing displayed on the TF widget, as the screenshot shows.

@shaomeng Hey sam, I'm currently unable to reproduce this one.  Screenshot is below.  Let me know if you still have a problem moving forward.

authors: sgpearse shaomeng

repo: NCAR/VAPOR


text:
Add support for HiveContext

Currently only SQLContext is supported. Since HiveContext offers the superset of operations, and is already available on the classpath, it seems appropriate to expose this to users to be able to interact with datasets in the Hive metastore.

Some (most?) Hadoop distributions do not have the build of Spark jar with Hive
http://spark.apache.org/docs/latest/sql-programming-guide.html#hive-tables
"Spark SQL also supports reading and writing data stored in Apache Hive. However, since Hive has a large number of dependencies, it is not included in the default Spark assembly. Hive support is enabled by adding the -Phive and -Phive-thriftserver flags to Spark’s build. This command builds a new assembly jar that includes Hive"
Would that be a problem?

As it turns out, it is already on the classpath. Also, Hive is already supported it seems

Example:
import org.apache.spark.sql.hive.HiveContext

val hc = new HiveContext(z.sc)
val rdd = hc.sql(""SELECT 1 as x").cache()
val q = z.sqlContext.applySchema(rdd,rdd.schema)
q.registerTempTable("t")

%sql
SELECT * from t

Cool.
This requires spark-hive in the classpath right?
I don't have it there but I think it's cool. They (Michael Armbrust) say HiveContext is the superset and we should all use it..

I created small improvements at #306.
The idea is basically let user configure what kind of context (HiveContext vs SQLContext) %sql is going to use.
@btiernay suggested way still uses SQLContext when it queries with %sql.
Will it help this issue? or is there better way to handle it?

I like what you've done. Much better. Mine was just showing how it was possible but very awkward.  Thanks!

I guess this issue is nicely handled by #306 , let's close this one.

Its working fine in Zapplien , I have been writing in Scala IDE, when I tried loading or creating hive tables using HiveContext, it's creating or searching locally in metastore_db , wven though I placed   hive-site.xml in Spark conf folder and even in my class path , How to make my application to pick from hive-site.xml metastore
I tried:
objHiveContext.setConf("hive.metastore.warehouse.dir", "hdfs://ServerIP:HiveportNum/apps/hive/warehouse")
Can anyone help on this???

@sudhir9p This github repository is no longer in use. see https://github.com/NFLabs/zeppelin/blob/master/README.md
If you're writing spark application in your ide, you can try include hive-site.xml in your application's classpath.

authors: Leemoonsoo btiernay felixcheung sudhir9p swkimme

repo: NFLabs/zeppelin


text:
nz-checkbox-group not rendering inside  tag

Reproduction link
https://stackblitz.com/edit/ng-zorro-antd-start-uespdt?file=src/app/app.component.html
Steps to reproduce
Define nz-checkbox-group in a simple html form tag.
What is expected?
Check box group to appear as normal.
What is actually happening?
No checkbox is created. They are missing in the DOM.

Environment
Info

ng-zorro-antd
7.5.0

Browser
Chrome 74, Firefox 67

@anantoghosh I think you'd better check the console and you will see your app is totally error

Oh, ok @vthinkxie , sorry for the trouble. Thank you!

authors: anantoghosh vthinkxie

repo: NG-ZORRO/ng-zorro-antd


text:
release(0.5.0-rc.1): release 0.5.0-rc.1 pre release

PR Checklist
Please check if your PR fulfills the following requirements:

[x] The commit message follows our guidelines: https://github.com/NG-ZORRO/ng-zorro-antd/blob/master/CONTRIBUTING.md#commit
[x] Tests for the changes have been added (for bug fixes / features)
[x] Docs have been added / updated (for bug fixes / features)

PR Type
What kind of change does this PR introduce?

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[ ] Documentation content changes
[ ] Application (the showcase website) / infrastructure changes
[ ] Other... Please describe:

What is the current behavior?

Issue Number: N/A
What is the new behavior?
Does this PR introduce a breaking change?
[ ] Yes
[ ] No

Other information

Coverage remained the same at 52.233% when pulling 23ab4f991786c561a48352c3b5b5fa92602fa8ec on vthinkxie:release-0.5.0-rc.1 into d4a6aa198de4178a5974c21f26c19421225ea193 on NG-ZORRO:master.

Coverage remained the same at 52.233% when pulling ecc6d6376933c0da4a05adca4dea46c2daa27092 on vthinkxie:release-0.5.0-rc.1 into d4a6aa198de4178a5974c21f26c19421225ea193 on NG-ZORRO:master.

authors: coveralls vthinkxie


text:
Figurer og figurtekst er foreløpig produsert i en tabell

dette er meldt til Quarto-teamet, som mener det krever en større endring av hvordan ting gjøres for at man skal unngå dette (fordi delene regnes som floats, eller noe sånt).

Samme gjelder tabeller. Tabell i en tabell...

https://ardata-fr.github.io/officeverse/officedown-for-word.html#automated-formatting er en mulighet

repo: NIFU-NO/nifudocx


text:
Release 1.4.0 on NITRC

Do you guys plan to upload the newest releases on NITRC (1.4.0)? The last downloadable version is still 1.3.2 (2 years old), so no update since @laurapascal worked on it: https://www.nitrc.org/projects/shapepopviewer/

We are still debating if we are going to maintain the NITRC version since ShapePopulationViewer is accessible through Slicer.
@laurapascal what do you think? Should we create executables for linux/max/windows?

Some functionalities are removed in Slicer extension correct? Like PDF export.
The only users I know use SPV directly, opening Slicer before is often slow and only keeps an instance of Slicer open which most of the time is not needed.
Just my 2c.

Considering what you said @agirault , we will work on it.
Thank for your feedback.

@ninatubau @juanprietob  Should this be closed ?

authors: agirault jcfr juanprietob laurapascal

repo: NIRALUser/ShapePopulationViewer


text:
Timer resets to 25 minute, after choosing any type of break

Describe your issue here.
Steps to reproduce
Tell us how to reproduce this issue.
Expected behaviour
The timer should start from the remaining time left before choosing break.
Actual behaviour
The timer starts from 25 minutes itself.
Would you like to work on the issue?
Yes

Interested in taking up this issue? Leave a comment expressing your interest.
Each issue expects a 24hr turnaround time by default. In case where a student doesn't come back with an update after 24 hours have elapsed, the issue is up for grabs by anyone.

I don't understand what the issue is.

When we click session complete (the tick image), we are asked to choose for a break. After we choose any type of break, and ends the break, the time of the app resets to 25 minutes, instead of time left to complete the session.

That's actually how it is supposed to work.
The complete button is to be only used when you've finished a task early, I.e. Before the timer ticks 0. Once you complete the session you can take a break and come back to start a new 25 minute session.

Then, there must be a button to pause the timer in between. This would actually help to find the real time taken to complete the task.

Pausing allows for distractions.

So, whats the main motto of the app "wether to complete the task within 25 minutes" or  "25 minutes of continue non-destruction work "?

https://en.m.wikipedia.org/wiki/Pomodoro_Technique


text:
Add possibility to override configs

This commit implements a method to use the additional_config folder to override specific configurations.

I didn't get it to work the way I wanted (in the order that it will be overwritten). Did you manage to keep the functionality as mentioned? I think the plugin might still be interesting, as we can also extend it to read directly from repositories.

It seems that this plugin would still be interesting with the possibility to extend it to read directly from repositories.
I have two comments regarding the expected behaviour of this PR:

In manipulate_search_path no warning is given if the additional_path is not a directory. Perhaps logging something something simple, such as "Only using standard ahcore config folder for Hydra configurations.", would be good to avoid any confusion.
The env.load_env function starts searching recursively from ahcore/utils folder instead of the working directory. It will find the .env folder if it is in a parent folder but not when placed in the tools directory.

@BPdeRooij I believe I have addressed both concerns.

dotenv.load_dotenv expects an explicit file name. It does not start searching from working_dir. For this we could use dotenv.find_dotenv(usecwd=True) and give this as dotenv_path

Alright, but won’t the cwd be the folder where you’ve executed the command? Can you edit this in the PR?On Oct 30, 2023, at 10:01 AM, Bart @.***> wrote:
dotenv.load_dotenv expects an explicit file name. It does not start searching from working_dir. For this we could use dotenv.find_dotenv(usecwd=True) and give this as dotenv_path
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

authors: BPdeRooij jonasteuwen

repo: NKI-AI/ahcore


text:
Python divisibility rule

Using divisibility rule to determine text to be printed. Mods and Divs are not used in this implementation.

I forgot the Kotlin commit is still in this branch.

license: Unlicense


text:
#23、#24里的采样逻辑，有点疑问

这里提到了负例的采样方法，好像有点矛盾，到底是采样batch内其他query的正例还是负例，做为当前query的随机负例？

对于只有正例：
1.batch内随机负例是其他query的正样本。
对于有正例也有负例：
1.batch内随机负例是其他query的所有负样本。

repo: NLPJCL/RAG-Retrieval


text:
v5 causes Lintian warnings about changelog errors

From: https://github.com/NLnetLabs/routinator/actions/runs/3518581343/jobs/5897989371#step:18:67
W: routinator: syntax-error-in-debian-changelog line 3 "badly formatted trailer line"
W: routinator: syntax-error-in-debian-changelog line 3 "found eof where expected more change data or trailer"

Fixed in the v5.0.1 release.

repo: NLnetLabs/ploutos


text:
Extend outform with openbgpd format

It would be cool if routinator's -f option allows openbgpd format so that users of the software can immediately plug the output into the openbgpd daemon.
The openbgpd format is as following:
roa-set {
    57.0.0.0/8 source-as 2647
    100.128.0.0/9 maxlen 24 source-as 21928
    35.0.0.0/10 source-as 237
    78.192.0.0/10 maxlen 11 source-as 12322
    79.192.0.0/10 source-as 3320
    8.128.0.0/10 source-as 37963
    ... etc ...
}

What is of note is that the maxlen argument is optional if the prefixlen and the MaxLength are the same value.

Oh - one more note, ipv4 and ipv6 can just be mixed in the roa-set

repo: NLnetLabs/routinator


text:
Add API to support meaningful comments

OpenDNSSEC (maybe others too) uses comments to store signing state in the zone file itself. Therefore, it makes sense to expand the API and support returning these comments. Pending further information, the following two examples were provided to start planning the feature.
;;Key: locator ad06eeab0b0906e14ce2e27fb70d0425 algorithm 8 flags 256 publish 1 ksk 0 zsk 1 rfc5011 0
;;
example.com. 300 IN SOA ns1.example.com. postmaster.example.com.

example.com. 300 IN DNSKEY 256 3 8 AwEAAaVhvyRrg2vmOgiolwfvRHekdHGPUXt6OzNNz1iAMrwUx6S9yQ0L0IyUX0n2rcHIw5qgtoW0ZSkOMBT/1mS5CDUlvh3iba4BplYlEQ/XGYHxYoi0lYpsFiMgaDg9nOU03AkvjTFr1/0b8qXqgNWxR0EB+8s517T/BnDXo7S3ReLR ;{id = 46921 (zsk), size = 1024b}

Apparently Unbound uses similar tricks for trust anchors. If you run unbound-anchor -a root.key you'll get output like this:
; autotrust trust anchor file
;;id: . 1
;;last_queried: 1683198206 ;;Thu May  4 13:03:26 2023
;;last_success: 1683198206 ;;Thu May  4 13:03:26 2023
;;next_probe_time: 1683240986 ;;Fri May  5 00:56:26 2023
;;query_failed: 0
;;query_interval: 43200
;;retry_time: 8640
.	86400	IN	DNSKEY	257 3 8 AwEAAaz/tAm8yTn4Mfeh5eyI96WSVexTBAvkMgJzkKTOiW1vkIbzxeF3+/4RgWOq7HrxRixHlFlExOLAJr5emLvN7SWXgnLh4+B5xQlNVz8Og8kvArMtNROxVQuCaSnIDdD5LKyWbRd2n9WGe2R8PzgCmr3EgVLrjyBxWezF0jLHwVN8efS3rCj/EWgvIWgb9tarpVUDK/b58Da+sqqls3eNbuv7pr+eoZG+SrDK6nWeL3c6H5Apxz7LjVc1uTIdsIXxuOLYA4/ilBmSVIzuDWfdRUfhHdY6+cn8HFRm+2hM8AnXGXws9555KrUB5qihylGa8subX2Nn6UwNR1AkUTV74bU= ;{id = 20326 (ksk), size = 2048b} ;;state=2 [  VALID  ] ;;count=0 ;;lastchange=1683198206 ;;Thu May  4 13:03:26 2023

An example of an initial bootstrap file is:
example.com.    10800   IN      DS      55582 5 1 66e829ebd8145e6f030b840ae63b77273c9575e1 ; xiniv-mopov-rakoc-galuk-zibeb-ricob-penuf-rutad-lyzan-hetuv-caxox

repo: NLnetLabs/simdzone


text:
Fix path in ConfigureNLog

Described at
https://github.com/NLog/NLog/issues/641#issuecomment-180731577

Ignore Travis, it's not working yet with DNU


text:
Set User-Agent header for WebServiceTarget

WebServiceTarget with 'User-Agent' header doesn't work.
NLog version: 4.7.10
var webTarget = LogManager.Configuration.FindTargetByName<WebServiceTarget>("targetName");
webTarget.Headers.Add(new MethodCallParameter("User-Agent", Layout.FromString("Custom user agent")));

There is an exception in the internal log
Error WebService Target[logRemote]: Error has been raised. Exception: System.ArgumentException: The 'User-Agent' header must be modified using the appropriate property or method.
Parameter name: name
   at System.Net.WebHeaderCollection.ThrowOnRestrictedHeader(String headerName)
   at System.Net.WebHeaderCollection.Set(String name, String value)
   at NLog.Targets.WebServiceTarget.DoInvoke(Object[] parameters, AsyncLogEventInfo logEvent)
   at NLog.Targets.MethodCallTargetBase.Write(AsyncLogEventInfo logEvent)
   at NLog.Targets.Target.WriteAsyncThreadSafe(AsyncLogEventInfo logEvent)
   at NLog.Targets.Target.WriteAsyncLogEvent(AsyncLogEventInfo logEvent)

As I see, WebServiceTarget should have 'UserAgent' property that will be set to appropriate HttpWebRequest's property.

Yes UserAgent-Header is special and needs to be assigned through HttpRequest.UserAgent Property
Created possible solution here: #4475

Thanks, it would be nice, if you add this fix to v4.7.11

The NLog 4.7-branch is mostly for bug-fixes. But I guess if @304NotModified is in the mood for executing the whole release-exercise then I guess it could be done.

Sure no problem for creating a 4.7.11

Created #4489 for NLog 4

Think I prefer to have UserAgent as Layout property instead of mixing with headers. It will also be easier explain in documentation

From: Julian Verdurmen @.>
Sent: Thursday, July 8, 2021 21:01
To: NLog/NLog @.>
Cc: Rolf Kristensen @.>; Comment @.>
Subject: Re: [NLog/NLog] Set User-Agent header for WebServiceTarget (#4474)
Created #4489https://github.com/NLog/NLog/pull/4489 for NLog 4
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/NLog/NLog/issues/4474#issuecomment-876742843, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACXZ7HC5TXN3C3VQCOPBLS3TWYG2LANCNFSM47GLICMQ.

NLog v4.7.11 has been released - https://www.nuget.org/packages/NLog/4.7.11

authors: 304NotModified avdynut snakefoot

repo: NLog/NLog


text:
Replaced question.md issue-template with contact_links (fix bom)

Followup to #4912

We could only test it after merge? Or maybe at a fork?
I see the indents on https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/configuring-issue-templates-for-your-repository#configuring-the-template-chooser, so not sure if this would fix things

Yes testing it now. there is some BOM-issue

now it works

Yes indeed! Nice!


text:
Add support for byte order mark suffix to unicode transformation encodings.

Add support for byte order marks to FileTarget (Issue #26).

Adds "-bom" and "-no-bom" suffix for UTF8, UTF16, and UTF32 encodings.
Adds support for writing the Encoding's preamble to the FileTarget class.
Removes "IncludeBOM" property from WebServiceTarget as it is no longer needed.
Adds DefaultByteOrderMarkAttribute which when applied to a property will determine whether a BOM is included when specifying a UTF encoding that does not specifically indicate the inclusion or exclusion of the BOM.

When specifying UTF-8 encoding, the default state will be to exclude the BOM. UTF-16 and UTF-32 encodings, by default include the BOM. The DefaultByteOrderMarkAttribute can be used to override this on a per-property basis. The FileTarget.Encoding property is an example of this. The FileTarget's Encoding property is attributed to ensure the BOM is excluded by default, this ensure that existing configurations with UTF-16 or UTF-32 encodings will not begin including a BOM automagically.

Thanks for the PR. But this is a breaking change  and we need to discuss this first.
Some things:

we have tested before with "UTF-8-bom" encoding name, but this was leading to backwardscomp problems: what is UTF-8? BOM or no BOM? In FileTarget and WebserviceTarget, it had different behavior
BOM is required for UTF16 and UTF32 AFAIK. (because it uses multiple bytes, and order does matter)

I understand completely. I just wanted to get the code up here so you guys could decide if you wanted it or not. I have attempted to address the concerns that you raised in your comment.

The code understands a number of different names for the various UTF encodings, rather than list them here, I will just refer you to the code which is in EncodingHelper.cs. As far as I am aware, all of the names supported by the framework are covered.
Each name has three possible states: "name", "name-bom", and "name-no-bom". For example, UTF-32BE may be referred to as "utf-32be", "utf-32be-bom" and "utf-32be-no-bom". If the name is suffixed with "-bom" or "-no-bom" then the resulting encoding will always follow that suffix. For example, "utf-16-bom" will always return an Encoding instance that will return an array of bytes from Encoding.GetPreamble(). This leaves the encoding name that does not contain a suffix, for example: "utf-16le", "utf-32" and "utf-8". The default for a non-suffixed encoding name depends on the encoding, UTF-8 defaults to no BOM (which follows the Unicode spec that states it is not recommended, but differs from Microsoft's default stance which includes it). UTF-16 and UTF-32 (regardless of BE/LE) default to including a BOM, which passes the Unicode spec which states the BOM "may" be included.
As you point out, FileTarget and WebServiceTarget have different behavior as to whether or not a BOM is included. FileTarget did not include a BOM, regardless of the selected encoding. Prior to this point, FileTarget did not contain any code to write the BOM. Where as WebSeriviceTarget would include the BOM, as it ultimately used a StreamWriter which respected the contents of Encoding.GetPreamble(). This is why I included the DefaultByteOrderMarkAttribute. This attribute can be applied to Property members and indicates to the configuration system whether encoding names without a suffix should or should not include a BOM. Without this attribute, the code falls back to the rules defined in the previous bullet point: UTF-8 has no BOM, UTF-16 and UTF-32 have a BOM (unless the encoding name specifies a BOM or no-BOM specifically).

So, without changes, a FileTarget configured with any of the UTF encodings will continue to operate as it did before this commit is applied.
A WebServiceTarget configured for UTF-8 will not include a BOM, but will include a BOM for UTF-16 and UTF-32. This differs from pre-4.0 code where UTF-8, UTF-16 and UTF-32 would all include a BOM when used with a WebServiceTarget.
I believe the only breaking change involved is that of IncludeBOM being removed from the WebServiceTarget. I'm sure with a little bit of thought, I could come up with a way to include it while still allowing for the newly suffixed encoding names.

With this change, how do we handle the if a user uses UTF8Encoding in the API? Is it with or without BOM?
I was wrong with the required BOM with UTF-16  / UTF-32.

Where the precise type of the data stream is known (e.g. Unicode big-endian or Unicode little-endian), the BOM should not be used.

(from: http://unicode.org/faq/utf_bom.html)

Programmatic configuration will always follow the Encoding instance's preamble configuration. This does have the side effect that existing programmatic configuration will probably include a BOM where it previously did not.
I believe this is proper result for the following reasons:

If the caller specifies an encoding configured to include a preamble, then the code should output the preamble in the appropriate locations.
The fact that FileTarget is not including a preamble when the encoding specifies a preamble is a bug.

Basically, my view is that non-programmatic configuration should default to following the appropriate protocol for what is being configured (ie: text files have a BOM for UTF-16/UTF-32, but not UTF-8). Where as programmatic configuration should do exactly as the caller specifies (even if he or she doesn't realize what is being specified).
With that stated, my code isn't quite following those guidelines. I need to modify the DefaultByteOrderMarkAttribute to allow different defaults for each of the different transform encodings.

I've updated the code to adjust the defaults, the results are now:
For FileTarget non-programmatic configuration will default to a BOM for UTF-16 and UTF-32 encodings and default to no-BOM for UTF-8 encodings, if not specified.
Programmatic configuration will continue to follow the encoding's preamble setting.

I was thinking about this change:

The 'IncludeBOM' solution was chosen,inter alia, because it would have the same solution from the config as from the API. WIth this solution it hasn't?
We just changed to 'IncludeBOM' in 4.0 with a breaking change. The next breaking change will be at 5.0, but I think 5.0 will take at least a year.

Concluding, I can understand you don't find the 'IncludeBOM' an optimal solution, but we're looking for a backwards-compatible change for the moment.

OK, before writing off this change completely, consider the following:
The specific problem that I'm attempting to solve is the FileTarget not supporting encoding preambles. My solution burrowed it's way into other targets, specifically the WebServiceTarget.
Outstanding problems are:

Programmatic configuration operates differently than non-programmatic configuration.
Removal of the "IncludeBOM" property is a breaking change

Solutions:

Change non-programmatic configuration to follow Encoding.UTF8, Encoding.Unicode and Encoding.UTF32 defaults. This ensures that the programmatic and non-programmatic configuration will result in the same output.
Restore the IncludeBOM property, making it a Nullable<bool> type. If specified (ie, non-null) it will forcibly override the current encoding's (after determining if it's UTF8, UTF16 or UTF32) preamble setting. If the IncludeBOM property is not configured, the WebServiceTarget target would follow the encoding's preamble setting.

Issues that would arise from these changes:

FileTarget will begin including BOM's for UTF-8 files by, where as currently it does not. This would bring FileTarget into alignment with Microsoft practices, but differ from Unicode suggested practices.

Do you believe this would meet the requirements?

I'm sorry but I'm a bit lost. What are the differences compared to the 4.0 release?

Closing this one due to inactivity.


text:
Initial command line interface

I created an initial command line interface (CLI) for MM. As part of this I also started modernizing the setup/build config.

Here's an example of the output, for my idealized example:

Let me know any questions/comments/suggestions regarding behavior and features. Looking for people to try it out. @rschwant @dwfncar @Duseong

The one where it succeeded (after I added typer to the conda env) was using Sphinx v4.4.0, but then with v4.5.0 it started failing.

Now the CRN URLs in the docs are not working.

I just released Typer 0.4.1 that should handle the Typer issue. :rocket: :nerd_face:

@rschwant can you check the package metadata in setup.cfg? I think the auto-job submission I can try in another PR. I want to get this one in sooner so I can start updating the install docs.

Thanks @rschwant

Related to https://github.com/NOAA-CSL/MELODIES-MONET/issues/117

authors: tiangolo zmoon

repo: NOAA-CSL/MELODIES-MONET


text:
Create documentation

Create documentation
Some ideas:
Quickstart

source setup.sh
create_env.py
supported platforms

Setting up on a new system

Finding compiler/modules

Finding common external packages

spack external find

Spack Mirrors/Caches

Manually Configuring Compilers/versions

Workflow

Installing a new package/version
Uninstalling packages
Modules

Nearly done! Next week I'll have the PR ready.

Progress towards this was made in https://github.com/NOAA-EMC/spack-stack/pull/148 - once it's finalized for the first release, we can close this issue.

authors: climbfuji kgerheiser

repo: NOAA-EMC/spack-stack


text:
Add gh-action to run unit tests marked slow at some regular interval

Yeah good catch, I was not aware of that either. That sounds like something a pytest --run-slow should have caught. I need to create a new gh-action that runs our slow tests every Sunday if something new has been added since the last time the action ran.
Originally posted by @aaraney in https://github.com/NOAA-OWP/hydrotools/issues/90#issuecomment-861621966

Has this been resolved? It looks like slow tests have been running.

Yes this has been resolved. Sorry, I should have closed this a long time ago.

repo: NOAA-OWP/hydrotools


text:
Running CFE fails with error related with 'ice_fraction_schaake'

Hi all, a newbie here.
I followed this tutorial, successfully built NexGen and successfully run the first example of simulation (entitled Running CFE but the realization runs a PET, not CFE).
In my learning pathway, I tried to run a simplistic model with CFE for a system with 4 catchments and 3 nexus in an "Y" topological shape. Relevant files are attached.
Current behavior
When I run ngen, after Running timestep 0, I get some warnings regarding issues with unit conversion and it finally breaks with:
(...)
terminate called after throwing an instance of 'std::runtime_error'
  what():  Cannot get forcing value for unrecognized parameter name 'ice_fraction_schaake'.
Aborted (core dumped)

From the CFE repository, the ice_fraction_schaake  is provided by the module SFT, which could be disabled with parameter sft_coupled=false. I included this parameter in the config file (attached cfe_uniform.input ), tried to use Xinanjiang partitioning scheme, and kept getting the same output.
Expected behavior
Or ngen to complete the run gracefully not using SFT, or to get an error complaining about the lack of ice_fraction_xinan when using the Xinanjiang partitioning scheme.
Questions

Shouldn't the ice_fraction_[schaake/xinan] only be a parameter when sft_coupled=true ?

Is there an place (like a forum or so) for questions and discussions like this (that are more related with usage/how-to than development)?

cfe_uniform.txt
forcings_uniform.csv
realization.txt

@adlzanchetta
Thanks for filing this issue, it is a good reminded that the wiki tutorial has gotten out of date with some of our recent developments.
With the addition of ice fraction to CFE, the old examples do not work correctly.  We do have a module that can be used to provide constant values for these required inputs, but it does require setting up a Multi BMI Realization as you can see in this example.  You can remove the fortran noaa-owp-modular part of this realization and use just the C++ Sloth module and the CFE module and that should allow you to run CFE in your test setup.

repo: NOAA-OWP/ngen


text:
Might need another release (0.4.2)!

Hi,
After installing the package using pip or poetry, we realize that for instance in the match_result.py file, the path_to_dataframe() function is not available in the latest release 0.4.1.
It would be very practical to have another release for this package to be installable easily with package managers like pip or poetry (to include the latest commit).
Thank you very much for your kind support and consideration. Keep up the excellent work.
Best regards,
Amin Sajadi

Amin - Thanks for bringing this to our attention and apologies for the oversight in not releasing those minor changes! I just pushed a new release 0.4.2 that should include all of those changes. Let us know if you have any trouble with it.

Closing this since we released 0.4.2.

authors: amin-sajadi nreinicke

repo: NREL/mappymatch


text:
NVIDIA's model store is a dead link

The link to NVIDIA's model store seems to be dead from this page:
https://github.com/NVIDIA/DIGITS/blob/master/docs/ModelStore.md
export DIGITS_MODEL_STORE_URL='http://developer.download.nvidia.com/compute/machine-learning/modelstore/4.5.0'
I'm not getting any pretrained model in DIGITS with this environment variable set, and opening that URL in a browser gives a 404.

You're right - we haven't actually set up the official model store yet. We've hit some delays getting the documentation and licensing for each model just right.
/cc @IsaacYangSLA

What is the solution?

Until we post the official model store, connecting DIGITS to that URL will do nothing.
For now, you're welcome to follow these instructions to set up your own store:
https://github.com/NVIDIA/DIGITS/blob/digits-5.0/docs/ModelStore.md#create-your-own-model-store-server

The model store is now live. For now, it contains trained versions of:

LeNet (MNIST)
AlexNet (ImageNet)
GoogLeNet (ImageNet)

We will release more models as they get approved - hopefully a DetectNet (KITTI) will be coming soon!

Thank you for youh notice.I will try soon. 发自网易邮箱大师 在2017年01月25日 02:16，Luke Yeager 写道： The model store is now live. For now, it contains trained versions of: LeNet (MNIST) AlexNet (ImageNet) GoogLeNet (ImageNet) We will release more models as they get approved - hopefully a DetectNet (KITTI) will be coming soon! — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

Is this offline again? The links to the models on the store page (http://developer.download.nvidia.com/compute/machine-learning/modelstore/5.0) are broken.

They aren't broken. The website just isn't set up for browsing. If you connect with DIGITS 5.0 it should all show up OK. e.g. this link loads fine, but this one doesn't. DIGITS doesn't need to access the second link in order to download the model.

I apologize if I missed it somewhere, but how to connect the models link with DIGITS 5.0 ? Thank you.

@mrasyid this doc should explain everything you'd want to know:
https://github.com/NVIDIA/DIGITS/blob/digits-5.0/docs/ModelStore.md

it seems the model store is offline again.

Okay, updating to the latest repository solved the issue

With DIGITS 5.0, when I try to access the model store, the following error is generated:
[WARNING] Skip http://developer.download.nvidia.com/compute/machine-learning/modelstore/5.0/ due to error HTTPConnectionPool(host='developer.download.nvidia.com', port=80): Max retries exceeded with url: /compute/machine-learning/modelstore/5.0/master.json (Caused by <class 'socket.gaierror'>: [Errno -2] Name or service not known)
And the following message is displayed in the web interface: 'No models in Model Store'.
I am simply interested in accessing the official NVIDIA model store, and therefore, I have not configured my own.
Any suggestions on how to proceed would be great.

@fkhalid check your docker outbound connections settings. Is the problem solved nowadays?

@wlmx It was indeed a docker network issue. As soon as I started the container with '--net=host', the store became accessible.
Many thanks for your help.

authors: TanLingxiao VanitarNordic adnphllps dbolkensteyn fkhalid lukeyeager mrasyid wlmx

repo: NVIDIA/DIGITS


text:
Testing data db

According to this line:
https://github.com/NVIDIA/DIGITS/blob/master/digits/dataset/images/classification/views.py#L384
It should be possible to add testing database in the gallery of image too. Is it just adding it to the GET parameter ?

Hello @tonytan4ever, exploring the test DB is supported if you created a test DB (by e.g. setting the % for testing field to something greater than 0):

Thanks a lot. What is the url you use to generate the graph ? Can you show it to me ?

The graph is shown on the dataset page.

It doesn't seem like there is an issue there. Closing. Thanks for the feedback!


text:
[REVIEW] Use DaskDataset as dataset for unittests

Move unittest fixtures from testing a GPUDatasetIterator to a DaskDataset,
and start using the dask workflow for most of the ops unittests.
This also changes the baseworkflow to accept a GPUDatasetIterator or a
DaskDataset object and moves the dask client fixture to be available for
the entire project

rerun tests

run tests

run tests

rerun tests

rerun tests

repo: NVIDIA/NVTabular


text:
Jp itn 20240221

What does this PR do ?
Add a one line overview of what this PR aims to accomplish.
PR for Japanese itn instead of #101
Before your PR is "Ready for review"
Pre checks:

[ *] Have you signed your commits? Use git commit -s to sign.
[ *] Do all unittests finish successfully before sending PR?

pytest or (if your machine does not have GPU) pytest --cpu from the root folder (given you marked your test cases accordingly @pytest.mark.run_only_on('CPU')).
Sparrowhawk tests bash tools/text_processing_deployment/export_grammars.sh --MODE=test ...

[ *] If you are adding a new feature: Have you added test cases for both pytest and Sparrowhawk here.
[* ] Have you added __init__.py for every folder and subfolder, including data folder which has .TSV files?
[ *] Have you followed codeQL results and removed unused variables and imports (report is at the bottom of the PR in github review box) ?
[ *] Have you added the correct license header Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. to all newly added Python files?
[ ] If you copied nemo_text_processing/text_normalization/en/graph_utils.py your header's second line should be Copyright 2015 and onwards Google, Inc.. See an example here.
[ *] Remove import guards (try import: ... except: ...) if not already done.
[ ] If you added a new language or a new feature please update the NeMo documentation (lives in different repo).
[ *] Have you added your language support to tools/text_processing_deployment/pynini_export.py.

PR Type:

[ *] New Feature
[ *] Bugfix
[ ] Documentation
[ ] Test

If you haven't finished some of the above items you can still open "Draft" PR.

The updated files from previous round of review.

Jenkinsfile (to add test for Japanese and date update)
tests/nemo_text_processing/ja/data_inverse_text_normalization/test_cases_fraction.txt (changed the text format, changed 1 3/4 to 1荷3/4 and 1と3/4)
tools/text_processing_deployment/pynini_export.py (added ITN Pocess fst for space issue)
nemo_text_processing/inverse_text_normalization/ja/verbalizers/post_processing.py (to resolve space issue)
updates later received but also already merged to the main including the files under tools/text_processing_deployment

Updating according to XInchao's comment.

updated file nemo_text_processing/inverse_text_normalization/ja/verbalizers/date.py at line 36 for comments
updated file nemo_text_processing/inverse_text_normalization/ja/taggers/date.py at line 37 and from line 89 to line 108

repo: NVIDIA/NeMo-text-processing


text:
Llama 3 70B FP8 engine build failed with FMHA

System Info
AWS p5 (4 x 80GB H100 GPUs)
TensorRT-LLM v0.11.0
Who can help?
@byshiue @Tracin
Information

[X] The official example scripts
[ ] My own modified scripts

Tasks

[X] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[ ] My own task or dataset (give details below)

Reproduction
python ./quantize.py --model_dir ./Meta-Llama-3-70B-Instruct --dtype bfloat16 --output_dir ./Meta-Llama-3-70B-Instruct_fp8 --calib_size 1024 --calib_dataset /home/triton-server/calibration --tp_size 4 --qformat fp8
trtllm-build --checkpoint_dir ./Meta-Llama-3-70B-Instruct_fp8 --output_dir ./Meta-Llama-3-70B-Instruct_fp8_engine_fmha --gemm_plugin auto --workers 1 --use_paged_context_fmha enable --use_fp8_context_fmha enable --max_batch_size 16
Expected behavior
Engine created succesfully.
actual behavior
Engine build fails with
[08/18/2024-23:36:42] [TRT] [W] Detected layernorm nodes in FP16.
[08/18/2024-23:36:42] [TRT] [W] Running layernorm after self-attention in FP16 may cause overflow. Exporting the model to the latest available ONNX opset (later than opset 17) to use the INormalizationLayer, or forcing layernorm layers to run in FP32 precision can help with preserving accuracy.
[08/18/2024-23:36:42] [TRT] [W] [RemoveDeadLayers] Input Tensor position_ids is unused or used only at compile-time, but is not being removed.
[08/18/2024-23:36:42] [TRT] [I] Global timing cache in use. Profiling results in this builder pass will be stored.
terminate called after throwing an instance of 'tensorrt_llm::common::TllmException'
  what():  [TensorRT-LLM][ERROR] Assertion failed: getIdx() should not be used with entry 16
 (/workspace/tensorrt_llm/cpp/tensorrt_llm/plugins/gptAttentionPlugin/gptAttentionPlugin.cpp:127)
1       0x7fc56bf865ce /usr/local/lib/python3.10/dist-packages/tensorrt_llm/libs/libnvinfer_plugin_tensorrt_llm.so(+0x7f5ce) [0x7fc56bf865ce]
2       0x7fc56bf86df0 /usr/local/lib/python3.10/dist-packages/tensorrt_llm/libs/libnvinfer_plugin_tensorrt_llm.so(+0x7fdf0) [0x7fc56bf86df0]
3       0x7fc56c020d2e tensorrt_llm::plugins::GPTAttentionPlugin::supportsFormatCombination(int, nvinfer1::PluginTensorDesc const*, int, int) + 1118
4       0x7fc986625a14 /usr/local/lib/python3.10/dist-packages/tensorrt_libs/libnvinfer.so.10(+0xafda14) [0x7fc986625a14]
5       0x7fc9868d3d33 /usr/local/lib/python3.10/dist-packages/tensorrt_libs/libnvinfer.so.10(+0xdabd33) [0x7fc9868d3d33]
6       0x7fc98665ef2d /usr/local/lib/python3.10/dist-packages/tensorrt_libs/libnvinfer.so.10(+0xb36f2d) [0x7fc98665ef2d]
7       0x7fc986939abe /usr/local/lib/python3.10/dist-packages/tensorrt_libs/libnvinfer.so.10(+0xe11abe) [0x7fc986939abe]
8       0x7fc9867e116f /usr/local/lib/python3.10/dist-packages/tensorrt_libs/libnvinfer.so.10(+0xcb916f) [0x7fc9867e116f]
9       0x7fc9867e9e0c /usr/local/lib/python3.10/dist-packages/tensorrt_libs/libnvinfer.so.10(+0xcc1e0c) [0x7fc9867e9e0c]
10      0x7fc986926c19 /usr/local/lib/python3.10/dist-packages/tensorrt_libs/libnvinfer.so.10(+0xdfec19) [0x7fc986926c19]
11      0x7fc98692e21c /usr/local/lib/python3.10/dist-packages/tensorrt_libs/libnvinfer.so.10(+0xe0621c) [0x7fc98692e21c]
12      0x7fc986930328 /usr/local/lib/python3.10/dist-packages/tensorrt_libs/libnvinfer.so.10(+0xe08328) [0x7fc986930328]
13      0x7fc98657f2ac /usr/local/lib/python3.10/dist-packages/tensorrt_libs/libnvinfer.so.10(+0xa572ac) [0x7fc98657f2ac]
14      0x7fc986584501 /usr/local/lib/python3.10/dist-packages/tensorrt_libs/libnvinfer.so.10(+0xa5c501) [0x7fc986584501]
15      0x7fc986584f0b /usr/local/lib/python3.10/dist-packages/tensorrt_libs/libnvinfer.so.10(+0xa5cf0b) [0x7fc986584f0b]
16      0x7fc92bca7458 /usr/local/lib/python3.10/dist-packages/tensorrt_bindings/tensorrt.so(+0xa7458) [0x7fc92bca7458]
17      0x7fc92bc458f3 /usr/local/lib/python3.10/dist-packages/tensorrt_bindings/tensorrt.so(+0x458f3) [0x7fc92bc458f3]
18      0x5578df496c9e /usr/bin/python3(+0x15ac9e) [0x5578df496c9e]
19      0x5578df48d3cb _PyObject_MakeTpCall + 603
20      0x5578df4a53eb /usr/bin/python3(+0x1693eb) [0x5578df4a53eb]
21      0x5578df48559a _PyEval_EvalFrameDefault + 25674
22      0x5578df49759c _PyFunction_Vectorcall + 124
23      0x5578df481a9d _PyEval_EvalFrameDefault + 10573
24      0x5578df49759c _PyFunction_Vectorcall + 124
25      0x5578df47f96e _PyEval_EvalFrameDefault + 2078
26      0x5578df49759c _PyFunction_Vectorcall + 124
27      0x5578df47f827 _PyEval_EvalFrameDefault + 1751
28      0x5578df49759c _PyFunction_Vectorcall + 124
29      0x5578df4a5db2 PyObject_Call + 290
30      0x5578df481a9d _PyEval_EvalFrameDefault + 10573
31      0x5578df49759c _PyFunction_Vectorcall + 124
32      0x5578df4a5db2 PyObject_Call + 290
33      0x5578df481a9d _PyEval_EvalFrameDefault + 10573
34      0x5578df49759c _PyFunction_Vectorcall + 124
35      0x5578df4a5db2 PyObject_Call + 290
36      0x5578df481a9d _PyEval_EvalFrameDefault + 10573
37      0x5578df49759c _PyFunction_Vectorcall + 124
38      0x5578df47f827 _PyEval_EvalFrameDefault + 1751
39      0x5578df47bf96 /usr/bin/python3(+0x13ff96) [0x5578df47bf96]
40      0x5578df571c66 PyEval_EvalCode + 134
41      0x5578df59cb38 /usr/bin/python3(+0x260b38) [0x5578df59cb38]
42      0x5578df5963fb /usr/bin/python3(+0x25a3fb) [0x5578df5963fb]
43      0x5578df59c885 /usr/bin/python3(+0x260885) [0x5578df59c885]
44      0x5578df59bd68 _PyRun_SimpleFileObject + 424
45      0x5578df59b9b3 _PyRun_AnyFileObject + 67
46      0x5578df58e45e Py_RunMain + 702
47      0x5578df564a3d Py_BytesMain + 45
48      0x7fc9ab1e4d90 /usr/lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fc9ab1e4d90]
49      0x7fc9ab1e4e40 __libc_start_main + 128
50      0x5578df564935 _start + 37

additional notes
The engine build runs fine when I don't include --use_paged_context_fmha enable --use_fp8_context_fmha enable on running trtllm-build.

Hi, @ayush1399  it seems a version mismatched issue, could you:

update to the latest commit
install the latest pypi
clean and rebuild trtllm
rebuild the engine

repo: NVIDIA/TensorRT-LLM


text:
Fix warning pynvml message

There is a warning message that refers to 'psutil', when it should refer to 'pynvml'.

Hi @miguelusque Thanks for your contribution to TRT-LLM.
We've fixed this typo internally.
Sorry for the late response.

authors: miguelusque nv-guomingz


text:
add building workflow for TE/Jax

This is the first commit for TE/Jax.
The build process for TE/PyTorch and TE/Jax components is different. In the building step, the TE/Jax only depends on CPP STL and CUDA, so we simply add needed building commands in CMakeList.txt. Unlike PyTorch, Jax has nothing like torch.utils.cpp_extension.BuildExtension and torch.utils.cpp_extension.CUDAExtension.
To ensure that users can build only TE/PyTorch, only TE/Jax, or both, I created FrameworkBuilder to manage framework-specific stuff. For example, if Jax users want to build TE/Jax, then PyTorch should not be required in their environment.
The user can set up an environment variable - FRAMEWORK to select framework, such as:
FRAMEWORK=jax pip install .
FRAMEWORK=pytorch pip install .
FRAMEWORK=all pip install .

The default value is all
The other Jax modules will be submitted after this PR is merged.

Addition, TE/Jax need pybind11 to build Python-CPP binding, so we need
apt install pybind11-dev
pip install pybind11

Both of above packages are needed

@timmoon10 ,
The CI triggers build failed:
https://github.com/NVIDIA/TransformerEngine/actions/runs/3901104182/jobs/6662679011#step:4:85
As I mentioned, TE/Jax needs pybind11
We have to install pybind11-dev in the CI container.
$ apt install pybind11-dev

Once we iron out the style and build issues, I think this is okay to merge.
I'm not thrilled with how we're refactoring layer norm and RMSNorm to use the same helper function. In principle they are independent operations that just happen to look similar. But in practice I don't expect them to undergo significant API changes in the future, and this is an internal implementation detail anyways.

I see the linter found some style issues.

It should be done

/te_ci

/te-ci

/te-ci

The scope of this PR has grown quite a bit. I'm still working through everything, but I don't see any glaring issues yet.
/te-ci

/te-ci

The scope of this PR has grown quite a bit. I'm still working through everything, but I don't see any glaring issues yet.

Sorry for inconvenience.
We tried to refactor the code and separate them into smaller pieces.
But the schedule pression doesn't allow us to refactor further

@timmoon10 ,
I just rebased to main, and #62 breaks the building process for TE/JAX again.
Looking at it...

@timmoon10 , I just rebased to main, and #62 breaks the building process for TE/JAX again. Looking at it...

The problem is here setup.py#L317. It requires installing flash-attn and flash-attn requires installing PyTorch. However, usually, JAX users don't have PyTorch in their environment.

It should be fixed by 2321cc3

/te-ci

The test failures are unrelated to this PR (see https://github.com/NVIDIA/TransformerEngine/pull/67#issuecomment-1430626755), so I think this is ready after addressing @ksivaman's concerns. I don't have full confidence that I got everything and in the future I would try to keep each PR with a clearly defined scope. It's much faster and safer to review multiple simple PRs than a single monstrous one.

The test failures are unrelated to this PR (see #67 (comment)), so I think this is ready after addressing @ksivaman's concerns. I don't have full confidence that I got everything and in the future I would try to keep each PR with a clearly defined scope. It's much faster and safer to review multiple simple PRs than a single massive one.

@timmoon10 ,
I understand it is difficult to review so much files at once.
The next PR will include less files. Hope it can be easier for code review.
I appreciate your help very much.

/te-ci

@ksivaman and @timmoon10 ,
I have merged the main

@ksivaman and I think this is ready to merge, but we'd like to discuss with @ptrendx tomorrow. Barring any issues, we'll merge it by California afternoon.

repo: NVIDIA/TransformerEngine


text:
Create devcontainer.json

"ghcr.io/wxw-matt/devcontainer-features/script_runner:0": {}

@AlexiPyt can you provide more details why you would like to include this file ?

authors: AlexiPyt angudadevops

repo: NVIDIA/cloud-native-stack


text:
Missing NVLINK bandwidth metrics in dcgm-exporter

What is the version?
3.3.3-3.3.0, 3.3.5-3.4.1
What happened?
I can not see profiling metrics like is in the documentation, NVLink Bandwidth,  PROF_NVLINK_TX_BYTES, PROF_NVLINK_RX_BYTES, but I can see for example DCGM_FI_PROF_SM_ACTIVE metric.
When I tried to run dcgmi on that nodes, I can see the proper metrics:
dcgmi profile -l -i 0
+----------------+----------+------------------------------------------------------+
| Group.Subgroup | Field ID | Field Tag                                            |
+----------------+----------+------------------------------------------------------+
| A.0            | 1001     | gr_engine_active                                     |
| A.0            | 1002     | sm_active                                            |
| A.0            | 1003     | sm_occupancy                                         |
| A.0            | 1004     | tensor_active                                        |
| A.0            | 1005     | dram_active                                          |
| A.0            | 1006     | fp64_active                                          |
| A.0            | 1007     | fp32_active                                          |
| A.0            | 1008     | fp16_active                                          |
| A.0            | 1009     | pcie_tx_bytes                                        |
| A.0            | 1010     | pcie_rx_bytes                                        |
| A.0            | 1011     | nvlink_tx_bytes                                      |
| A.0            | 1012     | nvlink_rx_bytes                                      |
| A.0            | 1013     | tensor_imma_active                                   |
| A.0            | 1014     | tensor_hmma_active                                   |
| A.0            | 1015     | tensor_dfma_active                                   |
| A.0            | 1016     | integer_active                                       |
| A.0            | 1017     | nvdec0_active                                        |
| A.0            | 1018     | nvdec1_active                                        |
| A.0            | 1019     | nvdec2_active                                        |
| A.0            | 1020     | nvdec3_active                                        |
| A.0            | 1021     | nvdec4_active                                        |
| A.0            | 1022     | nvdec5_active                                        |
| A.0            | 1023     | nvdec6_active                                        |
| A.0            | 1024     | nvdec7_active                                        |
| A.0            | 1025     | nvjpg0_active                                        |
| A.0            | 1026     | nvjpg1_active                                        |
| A.0            | 1027     | nvjpg2_active                                        |
| A.0            | 1028     | nvjpg3_active                                        |
| A.0            | 1029     | nvjpg4_active                                        |
| A.0            | 1030     | nvjpg5_active                                        |
| A.0            | 1031     | nvjpg6_active                                        |
| A.0            | 1032     | nvjpg7_active                                        |
| A.0            | 1033     | nvofa0_active                                        |
| B.0            | 1040     | nvlink_l0_tx_bytes                                   |
| B.0            | 1041     | nvlink_l0_rx_bytes                                   |
| B.0            | 1042     | nvlink_l1_tx_bytes                                   |
| B.0            | 1043     | nvlink_l1_rx_bytes                                   |
| B.0            | 1044     | nvlink_l2_tx_bytes                                   |
| B.0            | 1045     | nvlink_l2_rx_bytes                                   |
| B.0            | 1046     | nvlink_l3_tx_bytes                                   |
| B.0            | 1047     | nvlink_l3_rx_bytes                                   |
| B.0            | 1048     | nvlink_l4_tx_bytes                                   |
| B.0            | 1049     | nvlink_l4_rx_bytes                                   |
| B.0            | 1050     | nvlink_l5_tx_bytes                                   |
| B.0            | 1051     | nvlink_l5_rx_bytes                                   |
| B.0            | 1052     | nvlink_l6_tx_bytes                                   |
| B.0            | 1053     | nvlink_l6_rx_bytes                                   |
| B.0            | 1054     | nvlink_l7_tx_bytes                                   |
| B.0            | 1055     | nvlink_l7_rx_bytes                                   |
| B.0            | 1056     | nvlink_l8_tx_bytes                                   |
| B.0            | 1057     | nvlink_l8_rx_bytes                                   |
| B.0            | 1058     | nvlink_l9_tx_bytes                                   |
| B.0            | 1059     | nvlink_l9_rx_bytes                                   |
| B.0            | 1060     | nvlink_l10_tx_bytes                                  |
| B.0            | 1061     | nvlink_l10_rx_bytes                                  |
| B.0            | 1062     | nvlink_l11_tx_bytes                                  |
| B.0            | 1063     | nvlink_l11_rx_bytes                                  |
| B.0            | 1064     | nvlink_l12_tx_bytes                                  |
| B.0            | 1065     | nvlink_l12_rx_bytes                                  |
| B.0            | 1066     | nvlink_l13_tx_bytes                                  |
| B.0            | 1067     | nvlink_l13_rx_bytes                                  |
| B.0            | 1068     | nvlink_l14_tx_bytes                                  |
| B.0            | 1069     | nvlink_l14_rx_bytes                                  |
| B.0            | 1070     | nvlink_l15_tx_bytes                                  |
| B.0            | 1071     | nvlink_l15_rx_bytes                                  |
| B.0            | 1072     | nvlink_l16_tx_bytes                                  |
| B.0            | 1073     | nvlink_l16_rx_bytes                                  |
| B.0            | 1074     | nvlink_l17_tx_bytes                                  |
| B.0            | 1075     | nvlink_l17_rx_bytes                                  |
+----------------+----------+------------------------------------------------------+

Topology:
nvidia-smi topo -m
	GPU0	GPU1	GPU2	GPU3	GPU4	GPU5	GPU6	GPU7	CPU Affinity	NUMA Affinity	GPU NUMA ID
GPU0	 X 	NV12	PXB	PXB	SYS	SYS	SYS	SYS	0-31,65-95	0		N/A
GPU1	NV12	 X 	PXB	PXB	SYS	SYS	SYS	SYS	0-31,65-95	0		N/A
GPU2	PXB	PXB	 X 	NV12	SYS	SYS	SYS	SYS	0-31,65-95	0		N/A
GPU3	PXB	PXB	NV12	 X 	SYS	SYS	SYS	SYS	0-31,65-95	0		N/A
GPU4	SYS	SYS	SYS	SYS	 X 	NV12	PXB	PXB	32-63,96-127	1		N/A
GPU5	SYS	SYS	SYS	SYS	NV12	 X 	PXB	PXB	32-63,96-127	1		N/A
GPU6	SYS	SYS	SYS	SYS	PXB	PXB	 X 	NV12	32-63,96-127	1		N/A
GPU7	SYS	SYS	SYS	SYS	PXB	PXB	NV12	 X 	32-63,96-127	1		N/A

I can see even the actual values from NVLINKs connections when I am using dcgmi dmon, for example:
dcgmi dmon -e 1061,1062,1063,1064,1065,1066,1067

Also when I am looking for NVLINK metrics that are exported from dcgm-exporter I can not see metrics mentioned above, I have configuration for dcgm-exporter with fully enabled NVLINK metrics:
DCGM_FI_DEV_NVLINK_CRC_FLIT_ERROR_COUNT_TOTAL, counter, Total number of NVLink flow-control CRC errors.
DCGM_FI_DEV_NVLINK_CRC_DATA_ERROR_COUNT_TOTAL, counter, Total number of NVLink data CRC errors.
DCGM_FI_DEV_NVLINK_REPLAY_ERROR_COUNT_TOTAL,   counter, Total number of NVLink retries.
DCGM_FI_DEV_NVLINK_RECOVERY_ERROR_COUNT_TOTAL, counter, Total number of NVLink recovery errors.
DCGM_FI_DEV_NVLINK_BANDWIDTH_TOTAL,            counter, Total number of NVLink bandwidth counters for all lanes.
DCGM_FI_DEV_NVLINK_BANDWIDTH_L0,               counter, The number of bytes of active NVLink rx or tx data including both header and payload.

This is label output from some NVLINK metrics mentioned above: {DCGM_FI_DRIVER_VERSION="545.23.08", DCGM_FI_NVML_VERSION="12.545.23.08}
What did you expect to happen?
dcgm-exporter should expose metrics about NVLINK connections
What is the GPU model?
NVIDIA H100 PCIe
What is the environment?
dcgm-exporter is running as a docker container or in K8s as a pod on baremetal
How did you deploy the dcgm-exporter and what is the configuration?
I am using plain manifest for dcgm-exporter, I am not using Helm or GPU Operator
How to reproduce the issue?
Anything else we need to know?
If you want to I can provide docker or k8s manifest implementation
No response

Can you attach the logs from the pod?

│ time="2024-04-25T06:28:39Z" level=info msg="Starting dcgm-exporter"                                                                                │
│ time="2024-04-25T06:28:40Z" level=info msg="DCGM successfully initialized!"                                                                                │
│ time="2024-04-25T06:28:40Z" level=info msg="Collecting DCP Metrics"                                                                                │
│ time="2024-04-25T06:28:40Z" level=info msg="No configmap data specified, falling back to metric file /etc/dcgm-exporter/default-counters.csv"                                  │
│ time="2024-04-25T06:28:40Z" level=info msg="Initializing system entities of type: GPU"                                                                                │
│ time="2024-04-25T06:28:52Z" level=info msg="Initializing system entities of type: NvSwitch"                                                                                │
│ time="2024-04-25T06:28:52Z" level=info msg="Not collecting switch metrics: no switches to monitor"                                                                             │
│ time="2024-04-25T06:28:52Z" level=info msg="Initializing system entities of type: NvLink"                                                                                │
│ time="2024-04-25T06:28:52Z" level=info msg="Not collecting link metrics: no switches to monitor"                                                                               │
│ time="2024-04-25T06:28:52Z" level=info msg="Initializing system entities of type: CPU"                                                                                │
│ time="2024-04-25T06:28:52Z" level=info msg="Not collecting cpu metrics: Error retrieving DCGM MIG hierarchy: This request is serviced by a module of DCGM that is not currentl │
│ y loaded"                                                                                │
│ time="2024-04-25T06:28:52Z" level=info msg="Initializing system entities of type: CPU Core"                                                                                │
│ time="2024-04-25T06:28:52Z" level=info msg="Not collecting cpu core metrics: Error retrieving DCGM MIG hierarchy: This request is serviced by a module of DCGM that is not cur │
│ rently loaded"                                                                                │
│ time="2024-04-25T06:28:52Z" level=info msg="Kubernetes metrics collection enabled!"                                                                                │
│ time="2024-04-25T06:28:52Z" level=info msg="Pipeline starting"                                                                                │
│ time="2024-04-25T06:28:52Z" level=info msg="Starting webserver"                                                                                │
│ level=info ts=2024-04-25T06:28:52.263Z caller=tls_config.go:313 msg="Listening on" address=:9400                                                                               │
│ level=info ts=2024-04-25T06:28:52.263Z caller=tls_config.go:316 msg="TLS is disabled." http2=false address=:9400

If you are running dcgm-exporter inside a container, it's possible the switches and links are not mounted. See https://github.com/NVIDIA/dcgm-exporter/issues/169#issuecomment-1604771610 But also note you may run into this outstanding issue related to switch/link metrics: https://github.com/NVIDIA/DCGM/issues/155

We have not mounted switch and links, you mean SXM architecture I suppose, I am using NVIDIA H100 PCIe so the system does not know about the NVLINK

I encountered the same problem. Do you have resolved it ？

no @xuchenhui-5

repo: NVIDIA/dcgm-exporter