text:
Cannot read property uids of undefined

Storage Explorer Version: 1.9.0
Build Number: 20190701.8
Platform/OS: Windows 10
Architecture: ia32
Regression From:  Never tried before
Bug Description
Fails with an error when trying to add an account with SAS URL. Error Message is "Error occurred while adding new connection: TypeError:Cannot read property 'uids' of undefined"
Steps to Reproduce

Right click one normal account -> Select 'Get Shared Access Signature...'
Set a specific expiry time and permissions on SAS dialog -> 'Create'.
Copy the generated SAS URL -> Paste it to 'Use a shared access signature SAS URI' dialog.
Check the result.
Click Next and Then Click Connect.

Expected Experience
There is no error.
Account is added the content is visible in Storage Explorer
Actual Experience
Gives an Error
"Error occurred while adding new connection: TypeError:Cannot read property 'uids' of undefined"

Additional Context
Error Details:
{
  "name": "TypeError",
  "message": "Cannot read property 'uids' of undefined",
  "__zone_symbol__error": {
    "originalStack": "Error: Cannot read property 'uids' of undefined\n    at AppInsightsAsyncCorrelatedErrorWrapper.ZoneAwareError (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\zone.js\\dist\\zone-node.js:811:33)\n    at new AppInsightsAsyncCorrelatedErrorWrapper (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\applicationinsights\\out\\AutoCollection\\CorrelationContextManager.js:172:18)\n    at Function.ExceptionSerializer.deserialize (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\se-exception-serializer\\dist\\ExceptionSerializer.js:49:28)\n    at CloudExplorerProxyMarshaler.finalizeOperation (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\CloudExplorerProxyMarshaler.js:76:91)\n    at Object.finalizeOperation (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\CloudExplorerProxyMarshaler.js:99:76)\n    at Function.<anonymous> (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\StandardMarshaler.js:28:63)\n    at step (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:136:27)\n    at Object.next (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:117:57)\n    at C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:110:75\n    at new ZoneAwarePromise (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\zone.js\\dist\\zone-node.js:551:29)\n    at Object.__awaiter (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:106:16)\n    at JsonPortMarshaler.standardMarshaler.(anonymous function) [as finalizeOperation] (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\StandardMarshaler.js:22:32)\n    at JsonPortMarshaler.onMessage (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\messaging\\marshaler.js:101:41)\n    at Port.EventTarget.dispatchEvent (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\common\\event.js:69:30)\n    at Port.processMessage (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\messaging\\communication.js:163:18)",
    "zoneAwareStack": "Error: Cannot read property 'uids' of undefined\n    at AppInsightsAsyncCorrelatedErrorWrapper.ZoneAwareError (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\zone.js\\dist\\zone-node.js:811:33)\n    at new AppInsightsAsyncCorrelatedErrorWrapper (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\applicationinsights\\out\\AutoCollection\\CorrelationContextManager.js:172:18)\n    at Function.ExceptionSerializer.deserialize (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\se-exception-serializer\\dist\\ExceptionSerializer.js:49:28)\n    at CloudExplorerProxyMarshaler.finalizeOperation (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\CloudExplorerProxyMarshaler.js:76:91)\n    at Object.finalizeOperation (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\CloudExplorerProxyMarshaler.js:99:76)\n    at Function.<anonymous> (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\StandardMarshaler.js:28:63)\n    at step (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:136:27)\n    at Object.next (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:117:57)\n    at C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:110:75\n    at new ZoneAwarePromise (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\zone.js\\dist\\zone-node.js:551:29)\n    at Object.__awaiter (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:106:16)\n    at JsonPortMarshaler.standardMarshaler.(anonymous function) [as finalizeOperation] (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\StandardMarshaler.js:22:32)\n    at JsonPortMarshaler.onMessage (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\messaging\\marshaler.js:101:41)\n    at Port.EventTarget.dispatchEvent (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\common\\event.js:69:30)\n    at Port.processMessage (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\messaging\\communication.js:163:18)"
  },
  "stack": "TypeError: Cannot read property 'uids' of undefined\n    at StorageExplorerProvider.<anonymous> (file:///C:/Program%20Files%20(x86)/Microsoft%20Azure%20Storage%20Explorer/resources/app/out/CloudHub/js/Providers/StorageExplorer/StorageExplorerProvider.js:282:51)\n    at step (file:///C:/Program%20Files%20(x86)/Microsoft%20Azure%20Storage%20Explorer/resources/app/out/CloudHub/js/Providers/StorageExplorer/StorageExplorerProvider.js:47:23)\n    at Object.next (file:///C:/Program%20Files%20(x86)/Microsoft%20Azure%20Storage%20Explorer/resources/app/out/CloudHub/js/Providers/StorageExplorer/StorageExplorerProvider.js:28:53)\n    at fulfilled (file:///C:/Program%20Files%20(x86)/Microsoft%20Azure%20Storage%20Explorer/resources/app/out/CloudHub/js/Providers/StorageExplorer/StorageExplorerProvider.js:19:58)\n    at <anonymous>",
  "originalStack": "Error: Cannot read property 'uids' of undefined\n    at AppInsightsAsyncCorrelatedErrorWrapper.ZoneAwareError (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\zone.js\\dist\\zone-node.js:811:33)\n    at new AppInsightsAsyncCorrelatedErrorWrapper (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\applicationinsights\\out\\AutoCollection\\CorrelationContextManager.js:172:18)\n    at Function.ExceptionSerializer.deserialize (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\se-exception-serializer\\dist\\ExceptionSerializer.js:49:28)\n    at CloudExplorerProxyMarshaler.finalizeOperation (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\CloudExplorerProxyMarshaler.js:76:91)\n    at Object.finalizeOperation (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\CloudExplorerProxyMarshaler.js:99:76)\n    at Function.<anonymous> (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\StandardMarshaler.js:28:63)\n    at step (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:136:27)\n    at Object.next (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:117:57)\n    at C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:110:75\n    at new ZoneAwarePromise (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\zone.js\\dist\\zone-node.js:551:29)\n    at Object.__awaiter (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:106:16)\n    at JsonPortMarshaler.standardMarshaler.(anonymous function) [as finalizeOperation] (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\StandardMarshaler.js:22:32)\n    at JsonPortMarshaler.onMessage (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\messaging\\marshaler.js:101:41)\n    at Port.EventTarget.dispatchEvent (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\common\\event.js:69:30)\n    at Port.processMessage (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\messaging\\communication.js:163:18)",
  "zoneAwareStack": "Error: Cannot read property 'uids' of undefined\n    at AppInsightsAsyncCorrelatedErrorWrapper.ZoneAwareError (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\zone.js\\dist\\zone-node.js:811:33)\n    at new AppInsightsAsyncCorrelatedErrorWrapper (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\applicationinsights\\out\\AutoCollection\\CorrelationContextManager.js:172:18)\n    at Function.ExceptionSerializer.deserialize (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\se-exception-serializer\\dist\\ExceptionSerializer.js:49:28)\n    at CloudExplorerProxyMarshaler.finalizeOperation (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\CloudExplorerProxyMarshaler.js:76:91)\n    at Object.finalizeOperation (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\CloudExplorerProxyMarshaler.js:99:76)\n    at Function.<anonymous> (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\StandardMarshaler.js:28:63)\n    at step (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:136:27)\n    at Object.next (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:117:57)\n    at C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:110:75\n    at new ZoneAwarePromise (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\zone.js\\dist\\zone-node.js:551:29)\n    at Object.__awaiter (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:106:16)\n    at JsonPortMarshaler.standardMarshaler.(anonymous function) [as finalizeOperation] (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\StandardMarshaler.js:22:32)\n    at JsonPortMarshaler.onMessage (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\messaging\\marshaler.js:101:41)\n    at Port.EventTarget.dispatchEvent (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\common\\event.js:69:30)\n    at Port.processMessage (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\messaging\\communication.js:163:18)",
  "__zone_symbol__stack": "TypeError: Cannot read property 'uids' of undefined\n    at StorageExplorerProvider.<anonymous> (file:///C:/Program%20Files%20(x86)/Microsoft%20Azure%20Storage%20Explorer/resources/app/out/CloudHub/js/Providers/StorageExplorer/StorageExplorerProvider.js:282:51)\n    at step (file:///C:/Program%20Files%20(x86)/Microsoft%20Azure%20Storage%20Explorer/resources/app/out/CloudHub/js/Providers/StorageExplorer/StorageExplorerProvider.js:47:23)\n    at Object.next (file:///C:/Program%20Files%20(x86)/Microsoft%20Azure%20Storage%20Explorer/resources/app/out/CloudHub/js/Providers/StorageExplorer/StorageExplorerProvider.js:28:53)\n    at fulfilled (file:///C:/Program%20Files%20(x86)/Microsoft%20Azure%20Storage%20Explorer/resources/app/out/CloudHub/js/Providers/StorageExplorer/StorageExplorerProvider.js:19:58)\n    at <anonymous>",
  "__zone_symbol__name": "TypeError",
  "__zone_symbol__message": "Cannot read property 'uids' of undefined",
  "__zone_symbol__originalStack": "Error: Cannot read property 'uids' of undefined\n    at AppInsightsAsyncCorrelatedErrorWrapper.ZoneAwareError (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\zone.js\\dist\\zone-node.js:811:33)\n    at new AppInsightsAsyncCorrelatedErrorWrapper (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\applicationinsights\\out\\AutoCollection\\CorrelationContextManager.js:172:18)\n    at Function.ExceptionSerializer.deserialize (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\se-exception-serializer\\dist\\ExceptionSerializer.js:49:28)\n    at CloudExplorerProxyMarshaler.finalizeOperation (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\CloudExplorerProxyMarshaler.js:76:91)\n    at Object.finalizeOperation (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\CloudExplorerProxyMarshaler.js:99:76)\n    at Function.<anonymous> (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\StandardMarshaler.js:28:63)\n    at step (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:136:27)\n    at Object.next (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:117:57)\n    at C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:110:75\n    at new ZoneAwarePromise (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\zone.js\\dist\\zone-node.js:551:29)\n    at Object.__awaiter (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:106:16)\n    at JsonPortMarshaler.standardMarshaler.(anonymous function) [as finalizeOperation] (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\StandardMarshaler.js:22:32)\n    at JsonPortMarshaler.onMessage (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\messaging\\marshaler.js:101:41)\n    at Port.EventTarget.dispatchEvent (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\common\\event.js:69:30)\n    at Port.processMessage (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\messaging\\communication.js:163:18)",
  "__zone_symbol__zoneAwareStack": "Error: Cannot read property 'uids' of undefined\n    at AppInsightsAsyncCorrelatedErrorWrapper.ZoneAwareError (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\zone.js\\dist\\zone-node.js:811:33)\n    at new AppInsightsAsyncCorrelatedErrorWrapper (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\applicationinsights\\out\\AutoCollection\\CorrelationContextManager.js:172:18)\n    at Function.ExceptionSerializer.deserialize (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\se-exception-serializer\\dist\\ExceptionSerializer.js:49:28)\n    at CloudExplorerProxyMarshaler.finalizeOperation (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\CloudExplorerProxyMarshaler.js:76:91)\n    at Object.finalizeOperation (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\CloudExplorerProxyMarshaler.js:99:76)\n    at Function.<anonymous> (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\StandardMarshaler.js:28:63)\n    at step (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:136:27)\n    at Object.next (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:117:57)\n    at C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:110:75\n    at new ZoneAwarePromise (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\zone.js\\dist\\zone-node.js:551:29)\n    at Object.__awaiter (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:106:16)\n    at JsonPortMarshaler.standardMarshaler.(anonymous function) [as finalizeOperation] (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\StandardMarshaler.js:22:32)\n    at JsonPortMarshaler.onMessage (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\messaging\\marshaler.js:101:41)\n    at Port.EventTarget.dispatchEvent (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\common\\event.js:69:30)\n    at Port.processMessage (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\messaging\\communication.js:163:18)"
}

I'm having the same issue. Same error no matter how I try to add the account - not limited to URI (I've also tried a connection string and an account name and key)

Hey y'all, when you add the connection, in the main window are you on the tree panel or the account panel? Also, if you ignore the error and go to look for the thing you attached, is it there?

Hey y'all, when you add the connection, in the main window are you on the tree panel or the account panel? Also, if you ignore the error and go to look for the thing you attached, is it there?

I'm using the account panel (top left). My storage account is not available after I dismiss the error - there is no account added.

Ok just to make sure I understand, are you looking at the explorer panel:

Or the account panel:

When the error happens?
And if you click "Refresh All" and then drill down to the Local & Attached Storage Accounts does your account show up?

Having the same issue, seeing the account in the attached subscriptions but resfresh does not being successfull only showing an empty subscription.

Having the same issue.  Trying to add a storage account using storage name & key and it fails with attached error.
storageX.txt

The case where the error message talks about UIDs has been fixed. The other one is related to #985. Basically one connection is corrupted, and that causes loading all connections to fail.

"The case where the error message talks about UIDs has been fixed." Where? How do I access this fix? I have been through the links but they all reference this link. I have the latest storage emulator, the latest storage explorer and I have reinstalled both and still get this issue.

I'm with @johnstavely -- a bug fix is useless if it hasn't been released. When closing tickets can you at least give an idea about the release version to expect the change in?

I just updated to 1.12.0 and no longer get the error when adding.  It was tricky to then find where they showed under the storage view, as it showed up under "Local&Attached>Storage Accounts>".

authors: KBaroni LukeCarrier MRayermannMSFT johnstaveley josephmatt18 markus7h ryanhart vishwanath-uppala

repo: microsoft/AzureStorageExplorer


text:
'Search by prefix(case-sensitive)' is not localized in blob container's editor

Storage Explorer Version: 1.10.1
Build: 20190926.4
Branch: master
Platform/OS: Windows 10/ Linux Ubuntu 18.04/macOS High Sierra
Architecture: ia32/x64
Language: Chinese (zh-TW) / Chinese (zh-CN)
Regression From: Not a regression
Steps to reproduce:

Launch Storage Explorer using the command 'StorageExplorer.exe --lang=zh-CN'.
Expand a non-ADLS Gen2 storage account -> Blob Containers.
Create one blob container and check its editor.

Expect Experience:
'Search by prefix(case-sensitive)' is localized.
Actual Experience:
'Search by prefix(case-sensitive)' is not localized.

More Info:
This issue also reproduces for one file share.

Blob extension package version needs to be updated.

source: gharchive/issue

authors: craxal v-xianya

license_type: permissive

license_source: github-api


text:
Check forks & native branches activity on clean up

What is being addressed
The cleanup scripts doesn't check activity on forks or native branches, so it doesn't stop their environments.
How is this addressed

Add the PR's updatedAt date in case we don't find a commit (happens when the ref is from a fork)
Add activity check to native branches as well

/test-force-approve
tests don't cover this change

/test-force-approve

/test-force-approve
as per https://github.com/microsoft/AzureTRE/pull/1579#issuecomment-1076545871

source: gharchive/pull-request

authors: stuartleeks tamirkamara

repo: microsoft/AzureTRE

license: MIT


text:
TypeError: object of type 'AspectRatioGroupedDataset' has no len()

Dear Authors:
Hi, thank you so much for your work and making it open source. It is amazing.
I have encountered an issue in the XdecoderPipeline.py when trying to finetune the model, I had this TypeError: object of type 'AspectRatioGroupedDataset' has no len(), I was wondering if anyone has encountered this and how I could solve it. Thank you! (I am running it on a mac)

Hi,
I remember seeing this error once and it was resolved by using the right environment and config setup. A few things to check:

Make sure the environment is correctly setup.
Check the dataset registration and make sure they are included in the config file. You can debug by running the raw code (there is a demo dataset already registered so you can directly run the code for validation.

Let me know if these help you solve the problem, and happy to answer any further questions.

Hi,
I remember seeing this error once and it was resolved by using the right environment and config setup. A few things to check:

Make sure the environment is correctly setup.
Check the dataset registration and make sure they are included in the config file. You can debug by running the raw code (there is a demo dataset already registered so you can directly run the code for validation.

Let me know if these help you solve the problem, and happy to answer any further questions.

Hi Thank you so much for your prompt reply! I have reinstalled detectron2 and that has seemed to solve that error, however I got another one which was

Could this be due to the fact that I am running it on MacOs?

Hi,
I remember seeing this error once and it was resolved by using the right environment and config setup. A few things to check:

Make sure the environment is correctly setup.
Check the dataset registration and make sure they are included in the config file. You can debug by running the raw code (there is a demo dataset already registered so you can directly run the code for validation.

Let me know if these help you solve the problem, and happy to answer any further questions.

Hi! Thank you so much for your prompt reply, and the issue has been resolved. I reinstalled and installed Detectron2, then ran the code again with the demo datasets. However, the process terminated, and I suspect it might be due to my MacBook not having enough memory. Do you know what the minimum requirements are for the memory in terms of fine-tuning the model? I want to ensure that my system is capable of handling it. Thank you!

Glad the issue was resolved! The minimum requirement to run the model was V100 in our experiments. If there is CUDA out of memory issue, you can reduce the batch size for training in train.sh (the default value was 4).

repo: microsoft/BiomedParse


text:
bug: sample multiplydialog causes an error with the dialog merge required for package manager

Describe the bug
When using the Javascript runtime, the package manager fails the "dialog merge" command scans the project for schema files.   This seems to be related to the sample multiply dialog code.
Version
current main
To Reproduce

Create a new bot
Navigate to project settings
Enable custom runtime
Get a new copy of the runtime. Choose Javascript.
Navigate to the package manager tab. (enable feature flag in settings if necessary)
PROBLEM! You will see the following error dialog:

Error MultiplyDialog: Interface Microsoft.IDialog is not defined

This is because the dialog merge command sees the MultiplyDialog.schema file.
INTERESTING NOTE ---
This does NOT happen if I first start the bot.  Something must happen during the start local process (perhaps the schema is updated?) that prevents this issue from happening.
Expected behavior
The disabled sample multiplydialog should not interfere with the schema merge process. This should be ignored until enabled.
Suggested fix
rename MultipleDialog.schema to MultiplyDialog.schema.sample

@benbrown , I don't know how to reproduce this issue, but I assume the fix is,

rename MultipleDialog.schema to MultiplyDialog.schema.sample
ask user to rename back it they want to enable custom schema in the doc
Is this your expected fix?
I am not quite understand this: "This does NOT happen if I first start the bot." How can I reproduce this issue?

@luhan2017 correct this should fix it.
Regarding my other comment -- I believe what happens is that when you start the bot the first time,he local publish process merges in the multiply dialog schema automatically. As a result, the multiplydialog schema is present in the main schemas/sdk.schema when the package manager goes to re-process the schema files, and does not cause this error.

repo: microsoft/BotFramework-Composer


text:
Getting Started - Bot Info exceeds its cell width in UI

Describe the bug
The Bot Info if text exceeds the length of the cell for the Bot Name and file location does not stay within cell boundries.

Version

Browser

[X] Electron distribution
[ ] Chrome
[ ] Safari
[ ] Firefox
[ ] Edge

OS

[ ] macOS
[X] Windows
[ ] Ubuntu

To Reproduce
Steps to reproduce the behavior:

Create Bot with a really long name and long path
Go to Bot Settings page and look in Bot Info Section

Expected behavior
Text is contained within cell / wrapped or concatenated

here's a shot in action


text:
Can't create Empty Bot with Azure Functions

Command failed: dotnet build PsychoAnamBotBot.csproj Unhandled exception. System.ObjectDisposedException: Cannot write to a closed TextWriter. Object name: 'StreamWriter'. at System.IO.StreamWriter.g__ThrowObjectDisposedException|76_0() at System.IO.StreamWriter.WriteLine(String value) at Microsoft.DotNet.Installer.Windows.TimestampedFileLogger.Dispose(Boolean disposing) at Microsoft.DotNet.Workloads.Workload.Install.NetSdkMsiInstallerClient.OnProcessExit(Object sender, EventArgs e)
On a new attempt the error is: Command failed: dotnet build PsychoAnamBotTest.csproj

On a new attempt the error is: Command failed: dotnet build PsychoAnamBotTest.csproj

Thanks @mfpt, I'm looking into this.

Thanks @mfpt for reporting.
I'm able to reproduce this issue.

Steps to reproduce:

Install Command line tools for Azure Functions per your OS from here
Open Composer 2.1.2
Create a new C# bot
Select the Runtime type to Azure Functions

See error

Enabling Composer in debug mode yielded the following logs:
  composer:extensions:plugin-runtimes BUILD THIS C# FUNCTIONS PROJECT! at /Users/ram/Documents/bot/BF/ComposerPlayground/ComposerAZFunctionBot/ComposerAZFunctionBot/... +0ms
  composer:extensions:plugin-runtimes Run dotnet user-secrets init... +0ms
  composer:extensions:plugin-runtimes Run dotnet build... +237ms
  composer Cleaning up failed project at  /Users/ram/Documents/bot/BF/ComposerPlayground/ComposerAZFunctionBot +2s
GET /api/status/e828f33d-8f54-44ba-aa71-d1dab8e1fc7f 500 10.266 ms - -

@mfpt, I will let the engineers know about this issue and report back. Thanks.

Hi @ceciliaavila,
I'm able to reproduce this issue. Assigning this to you for now.
Please let me know if you have any questions.
Thanks.

Hi @ramfattah and @mfpt,
We weren't able to reproduce the described issue. In our testing, the Azure Functions bot was created and started properly.
All our testings were done in a Windows environment and with different Node.js and Azure Functions Core Tools versions.
Which SO are you using?

Thank you @sw-joelmut,
I used Mac OS.
I can try to use Windows OS to see if I'm still able to reproduce.
Will report back soon.

Thank you @sw-joelmut,
With your help, I'm not able to reproduce this issue anymore after installing:

azure-functions-core-tools
dotnet SDK 3.1

https://github.com/microsoft/BotFramework-Composer/assets/38049078/d596727d-3fa4-468f-823c-81febcc9a77c
> dotnet --info

.NET SDK:
 Version:   7.0.302
 Commit:    990cf98a27

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.22621
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\7.0.302\

Host:
  Version:      7.0.5
  Architecture: x64
  Commit:       8042d61b17

.NET SDKs installed:
  4.1.426 [C:\Program Files\dotnet\sdk]
  7.0.302 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
  x86   [C:\Program Files (x86)\dotnet]
    registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

Not able to reproduce. Closing this issue.
@mfpt, please re-open if you continue to face this issue after installing above requirements mentioned.

authors: mfpt ramfattah sw-joelmut


text:
feat: pluggable recognizer build

Description

Task Item

Screenshots

covered in #4351


text:
fix: QnA import sign in

Description

Fix Sign in issue.
Update text.

Task Item
#minor

Screenshots

Coverage decreased (-0.002%) to 56.256% when pulling 3d1d6ec42693c79a1f84c27faff6ff38d4060013 on zhixzhan/qna-from-portal-refine into f8d78c49d3ad5872d535d30612804bfa3657c2b6 on main.

@zhixzhan It would be great to capture "All" references to the word KB in the codebase to Knowledge Base as a part of this PR.

Align to global auth PR https://github.com/microsoft/BotFramework-Composer/pull/8291
Closing this.

authors: coveralls srinaath zhixzhan


text:
Updating pcre to 8.43 to fix CVE-2019-20838.

Merge Checklist

All boxes should be checked before merging the PR (just tick any boxes which don't apply to this PR)

[X] The toolchain has been rebuilt successfully (or no changes were made to it)
[X] The toolchain/worker package manifests are up-to-date
[X] Any updated packages successfully build (or no packages were changed)
[X] All package sources are available
[X] cgmanifest files are up-to-date and sorted (./cgmanifest.json, ./toolkit/tools/cgmanifest.json, ./toolkit/scripts/toolchain/cgmanifest.json)
[X] All source files have up-to-date hashes in the *.signatures.json files
[X] sudo make go-tidy-all and sudo make go-test-coverage pass
[X] Documentation has been updated to match any changes to the build system
[X] Ready to merge

Summary

Updating pcre to 8.43 to fix CVE-2019-20838
Change Log

Updated pcre to 8.43.
Addressed linter's suggestions.

Does this affect the toolchain?

No.
Links to CVEs

https://nvd.nist.gov/vuln/detail/CVE-2019-20838

Test Methodology

Local package build.

Closing, since #305 already fixes this and other CVEs.

repo: microsoft/CBL-Mariner


text:
Methods from Derived Classes Not Bound in ClearScript

I may have misunderstood how binding works in ClearScript, but I'm encountering an issue where methods from derived classes are not accessible. We have a method createElement that returns an object of type IElement. However, the actual object being returned is an instance of a derived class, HtmlDivElement, which inherits from IElement. The problem is that ClearScript only binds methods and properties from IElement and its ancestors. This results in methods specific to HtmlDivElement being inaccessible from JavaScript.
Example:
public interface IElement
{
    void SetAttribute(string name, string value);
    string GetAttribute(string name);
}

public class HtmlDivElement : IElement
{
    public void SetAttribute(string name, string value)
    {
        // Set attribute logic
    }

    public string GetAttribute(string name)
    {
        // Get attribute logic
        return string.Empty;
    }

    public void SetDivSpecificMethod()
    {
        // Method specific to HtmlDivElement
    }
}

JavaScript Example:
var div = document.createElement("div");
div.setAttribute("id", "myDiv");

// This fails because methods specific to HtmlDivElement are not bound
div.setDivSpecificMethod(); // Error: setDivSpecificMethod is not a function

Question:
How can I ensure that ClearScript binds methods from the derived class (HtmlDivElement) so that they are accessible in JavaScript? Is there a way to expose these methods while keeping the inheritance structure intact?

Hi @herki18,
The behavior you're seeing is by design. It's an effect of a ClearScript feature called type restriction, which enhances the ability of script languages to consume strongly typed .NET APIs.

// This fails because methods specific to HtmlDivElement are not bound
div.setDivSpecificMethod(); // Error: setDivSpecificMethod is not a function

That's right. The variable div is effectively of type IElement. If this were C# code, you couldn't call SetDivSpecificMethod without casting, and the same applies to JavaScript.
Type restriction is important for .NET member binding. Suppose you wanted to pass div to a .NET method overloaded as follows:
void DoSomething(IElement element);
void DoSomething(HtmlDivElement element);

Without type restriction, the first overload would be inaccessible, and that's just the tip of the iceberg. .NET APIs often rely heavily on precise type tracking for correct member binding, ambiguity resolution, type inferencing, generic type and method instantiation, etc.
That said, ClearScript does give applications the following controls over type restriction:

Members whose return type is object are exempt from type restriction.
HostFunctions.cast is a script-callable type casting function.
ScriptMemberAttribute.Flags can disable type restriction for an individual member via ScriptMemberFlags.ExposeRuntimeType.
ScriptEngine.DisableTypeRestriction provides control at the script engine level.

Good luck!

Please reopen this issue if you have additional questions about this topic. Thank you!

repo: microsoft/ClearScript


text:
SuperNet training log file?

Hi~  谢谢你们完美的工作  你能给我提供你们训练超网的LOG文件吗  我发现在训练超网时prec1特别低  如下

我在训练spos的时候前6个epoch精度已经到30左右了

hello, I am hearing from you

Hi, thanks for your interest in our work. Can you share the settings for the experiment? Then, I could give more detailed helps.

你好，谢谢你的回答，我在这里用中文来表达我的一些疑问！
第一点，在论文里面的消融实验，你用的sta_num是481M的，即[4,4,5,4,4]的配置，你在消融实验里说你用single path one shot的方法超网精度能到63.5，但是我照着你的配置跑了一下single path one shot版本，精度只有58，如图，

而且，58的精度是在你设定prob为(0.05, 0.2, 0.05, 0.5, 0.05, 0.15)情况下，我也跑了均匀采样的方法，不用你设定的概率，现在没跑完，跑出来的精度如下，可以确定的是精度应该不能到58，

我很想知道你的63.5是怎么跑出来的，是在跑出来的超网上进行了进化搜索算法得到的精度吗？或者直接超网训练得到的精度就能到63.5，如果是这样，你能给我提供你超网训练的LOG文件吗？
第二点，关于meta_value,你的代码里实现如下（PrioritizedBoard.py）：

这里是torch.sigmoid(meta_value)还是代码里的torch.sigmoid(-weight)呢，如果是torch.sigmoid(-weight)，似乎说不通？
同时，我认为你们将meta_value用来加权valid loss和teacher loss是合理的，但是在子网的梯度反传的时候，由于meta_value的值是由output - item[5]作为输入得到的，那么在loss.backward()里面meta_value是不是也有关于当前子网的gradfn?我认为是不太合理的，是不是代码该这样写呢如下：

这样loss.backward()时就不会通过meta_value传递梯度到子网了吧，我不确定我理解的是否准确，但是当我这样做的时候跑出来的精度曲线和不这样做精度曲线如下：

上面是我设置了with torch.no_grad()情况下
第三点关于simulate_sgd_update,你们官方给的代码如下，我不理解为什么你们手动模拟当前子网梯度下降时，用的学习率是meta网络的学习率0.0001，而不用超网学习率？

如果你们允许，我希望加你们的微信深入沟通一下这些问题，因为我们最近在复现经典的nas论文，我的微信号是：Peach-Bacon

你好，
请问可以分享下实验的环境配置吗？不同的版本会带来结果上的差异
1）请问你是怎么evaluate supernet的呢？想确认下是否是不同的evaluate方式带来了差异
2）感谢指出代码里的问题！我们在重组代码时，并未发现这里的问题，实际上应为meta_value，即最合适的teacher。
3）这里只是恰巧选到了meta网络的lr。
我已经加了您的微信，如果有任何问题，很欢迎继续指出~

repo: microsoft/Cream


text:
Fix IsPartial check and escaping of assembly name

We previously didn't escape - and @ in the assembly name when using it for an identifier or namespace.  This addresses that.
Our IsPartial check wasn't complete where we didn't also check whether the parent classes were partial too to allow for us to generate source.  Due to that, we would generate source in such cases leading to a compiler error rather than raising a diagnostic as we did when the type itself wasn't partial.
Improve the code fixer to handle making parent types partial.
Add partial diagnostics for the GeneratedCustomBindableProperty attribute too

Fixes #1799
Fixes #1800

Failures are post build infra issues, tests passed.

repo: microsoft/CsWinRT


text:
Increasing model coverage of Tensor MP and Inference Engine.

@RezaYazdaniAminabadi
Let's continue the discussion we had in the offline chat here.
First of all I want to refactor the module_injection.py and inference/engine.py codes. The current codes consist of too many nested functions, and all logic is contained in one function, so it is a bit complicated. What do you think?

@stas00 it has been checked that the training can be implemented without the CUDA kernel. (The parallelformers can train almost all models with tensor parallel) However, I need to invest a little more time to use the CUDA kernel. What do you want to do?

Implement tensor parallel training & inference without CUDA kernel now.
Implement tensor parallel training & inference with CUDA kernel later.

I'd say, that since you already have it implemented, let's make it available to users now.
We just need to make sure that once CUDA kernels become available the user have little to no change in how they deploy the models, so ideally just having a flag to choose python vs. cuda and we sort out all the requirements changes behind the scenes.
So if we have a pure-python class, we just swap in a CUDA version of the same. And we will need a mechanism to build the extensions, so for example it can be Deepspeed or some other separate repo/package. This is so that transformers remains pure python.
Please let me know if you have any questions.

Then I will proceed with the two tasks in parallel.
First of all, on the HF transformers side, I will implement the tensor parallel feature without CUDA. At the same time, on the DeepSpeed side, I will refactor the code and find a way to use CUDA kernel together. By making the interface of both side the same, I will make HF transformers automatically use the DeepSpeed code when the CUDA kernel becomes available in the future.
Thanks.

@RezaYazdaniAminabadi
First off, you didn't answer (It's ok. I think you are busy now :) ), but I'll do some refactoring. It will be easier to work with cleaner code to support more models.

Hi @hyunwoongko
Sorry for the late reply, I was OOF for the last couple of days. I agree with you that the API needs some refactoring to help make it more readable. Please go ahead and submit a PR with your current changes and I will review and add changes as needed.
Thanks a lot.
Btw, most of the Bert- and GPT-like models that can easily benefit from the kernels to run faster inference. We can add the support orthogonal to the model-parallelization.
Best,
Reza

Currently, our team is developing our large-scale model for the Asian language, and most of my resources are used to this work, so I did not invest a lot of time in collaboration. Sorry. I plan to resume work from this weekend.
@RezaYazdaniAminabadi Are you not considering having your own implementation of mpu and tensor mp inside deepspeed? I want to implement mpu and tensor mp similar to those used in megatron lm. It also includes a training function, so it is no longer limited to deep speed-inference. How about making both inference and training possible?
I think there is some misalignment between each other.
@stas00 wants a training function based on tensor mp.
@RezaYazdaniAminabadi wants to improve deepspeed-inference.
How would you like to work?

A small correction - HF transformers wants both training and inference. It's just that we have a lot more users using the library for training. So there is definitely not misalignment between the two.
Remember that Deepspeed already has TP and PP, so they are just missing inference.
HF Transformers doesn't have those yet, hence the difference.

@stas00 where is deepspeed TP? Do you mean Megatron LM Integration?
I don't know where the TP is because I haven't tried all the features of DeepSpeed.

You know what - you're absolutely correct, @hyunwoongko! Deepspeed doesn't have it! I have just never tried to use it so I assumed that since many Deepspeed docs talk about 3D, it included tensor parallelism, but it doesn't! I just found this comment by Shaden:

Tensor slicing is a model-specific strategy, which is why DeepSpeed supports, but does not provide it.

(source: https://github.com/microsoft/DeepSpeed/issues/673#issuecomment-763842000)
I will need to update this doc to remove Deepspeed: https://huggingface.co/transformers/parallelism.html#tensor-parallelism
As it's incorrect!

@stas00 That's why we should we Megatron-LM for 3D parallelization. (nowadays, I am working with Megatron-LM everyday like you 😅) But, if we support all kinds of parallelization for Huggingface transformers, we need TP and MPU like Megatron, but without Megatron. (Because Megatron is too complex and I think it's not good to cover various model architectures)
I can implement TP and MPU into huggingface sides, but you want to integrate with deepspeed. That's why I told you we should adjust it. because It is not clear deepspeed team also want to implement TP and MPU inside deepseed. but they (MPU and TP) are necessary.

I can implement TP and MPU into huggingface sides. I already tested training feature of parallelformers and it works. but MPU and TP training feature are needed. but you told me you want to integrate through deepspeed.

I think we have a miscommunication here. I was trying to communicate that in the process of implementing a pure-python TP in HF transformers, it'd be great to keep in mind DS's cuda kernels so that we could swap those in when such are available. I have not suggested to implement TP through deepspeed. We want HF transformers to be independent for the core 3D parallelism functionality and optionally be sped up by various extensions.
@hyunwoongko, would it help for us to have a call so that we could flesh out the details and hopefully there will be more clarity about this. I remember you were concerned with language issues. Since my Korean consists of one word, I wonder if @jaketae would be interested to participate and help with translation. I'm in the PST.
If we do that perhaps let's invite @siddk as well, so that we perhaps could create a shared plan to integrate your and his projects, as you both are working on some overlapping solutions.

I'd be more than happy to assist with translation, though I doubt @hyunwoongko will have issues. Plus I know you're secretly hiding your Korean skills, @stas00.
We can coordinate a date and time for further discussion and proceed from there? Perhaps a when2meet?

I think it would be good to discuss the issue on the Huggingface side. This is a deepspeed repo.

no secret Korean yet. I'm good with languages, but Asian ones I find too hard for me - since my native tongue isn't tonal.
I'm arranging a slack channel for all involved, including Reza. and we can then sort things out including a meeting. hope should be added/invited in the next day or two once I find an admin who is not on vacation.

Good. Could you invite me slack channel?

@hyunwoongko, it is definitely exciting to bring the tensor-slicing in deepspeed both for training and inference. Looking forward to talking with you all :)

@RezaYazdaniAminabadi Great. Firstly, Is it ok to bring tensor MP and MPU without fused kernel into DeepSpeed? I hope DeepSpeed supports Tensor MP without Megatron LM. Let's implement pure python first, and we can discuss about applying kernel fusion.

authors: RezaYazdaniAminabadi hyunwoongko jaketae stas00

repo: microsoft/DeepSpeed


text:
Fix comms benchmark import issues and support MPI/slurm launching

Currently users need do either launch with ds_bench, deepspeed or append the DeepSpeed repo to their PYTHONPATH to avoid an import error when launching communication benchmarks.
These changes both fix the import errors by automatically appending the python sys path before import, and enable automatic MPI/slurm environment discovery similar to how we detect it in [deepspeed/comm.py](https://github.com/microsoft/DeepSpeed/blob/4ae3a3da0dfd19d7ab7a76e7c742ac12f44fc1c0/deepspeed/comm/comm.py#L664). This also enables MPI launching for both the pytorch distributed and deepspeed comm backends.
Tested on up to 2k GPUs.
I also updated the README to be more specific and comprehensive.
@jeffra @awan-10

These test failures look unrelated to the PR

Looks like some conflicts are showing up with the changes in https://github.com/microsoft/DeepSpeed/commit/0acf7e9c489def74d3ac05ee91eabfb8280a1dba.
The reason for runtime-checks like https://github.com/microsoft/DeepSpeed/blob/3879a503ae601deb7ad9a3758d488cc91889a08f/benchmarks/communication/all_gather.py#L40 and the lack of top-level deepspeed imports is to enable users to benchmark communication ops using pure torch.distributed even without deepspeed installed (e.g. users of NeMo, Megatron-LM, fairseq, ColossalAI, etc).
I'm of the opinion that we should preserve this behavior for the comms benchmarks (i.e. no deepspeed.* top-level imports and wrap all deepspeed- or torch.distributed-specific code in an if args.dist == 'torch/deepspeed' check), but I think someone from the deepspeed team like @tjruwase and/or @jeffra have the final say. Can one of you provide comment?

Conflicts are resolved, and CI failures look unrelated @jeffra

I have no idea why formatting CI tests are failing now. I ran pre-commit checks before committing? I can't seem to reproduce these on my end.

@Quentin-Anthony, check your pre-commit and clang-format versions. Upgrading worked for me.

Ok all checks are finally passing!

authors: Quentin-Anthony tjruwase


text:
Increase Max File Size limit

On multiple case i have used values greater than 4096. We shouldn't be limiting it to a low value, epically when it comes to circular logging. Increase this value to something like 30-40GB so we don't run into odd issues like this.

Removed size limits from the maxsize var.  Should now accept any valid int.

repo: microsoft/ExPerfWiz


text:
Design: Container Loading

Existing Flow

Possible Flows

CREATE_ALWAYS
Creates a new file, always.   If the specified file exists and is writable, the function   overwrites the file, the function succeeds

CREATE_NEW
Creates a new file, only if it does not already exist.   If the specified file exists, the function fails

OPEN_ALWAYS
Opens a file, always. If the specified file does not exist and   is a valid path to a writable location, the function creates a file

OPEN_EXISTING
Opens a file, only if it exists.   If the specified file does not exist, the function fails.

Considerations

Cancellation: #1069
IFrames
Pre-fetching
Logging
Lifecycle events
Error handling
Offline Creation
Creation from Snapshot/Summary

@anthony-murphy , @tylerbutler , can we move it somewhere to docs and close the issue? It has been around for 1.5 years mostly without any changes.

authors: anthony-murphy vladsud

repo: microsoft/FluidFramework


text:
The message sanitize tool doesn't scrub Sequence segment properties

There may be other cases too. Will need a compliance review before using with customer data.
Example from a sanitized message.json (after running packages\test\snapshots\dist\sanitize.js)--
"contents": "{\"address\":\"defaultComponent\",\"contents\":{\"content\":{\"address\":\"31597146-2bb1-4c92-9d6a-314f9ffdde23\",\"contents\":{\"pos1\":5839,\"pos2\":5847,\"props\":{\"spellingError\":[\"scripter\",\"script or\",\"scripture\"]},\"type\":2}},\"type\":\"op\"}}",

Tool has been updated to scrub more completely and should produce output compatible with the replay-tool, but is not compatible with the fluid preview.  That work is tracked separately (because it's not an issue of how the content is sanitized, but because either the sanitize tool or the fluid debugger needs to support additional file formats)

authors: KRyan heliocliu


text:
Documents not summarizing

We have received another incident of a document accumulating a large number (40k) of OPs without summarization. The ICM incident contains a Kusto query for the document.
We are seeing a lot of:
fluid:telemetry:DeltaManager:WrapError	Nack: Submit a summary before inserting additional operations		Nack: Submit a summary before inserting additional operations
fluid:telemetry:Container:ContainerWarning	Service busy/throttled: Nack: Submit a summary before inserting additional operations		Service busy/throttled: Nack: Submit a summary before inserting additional operations
fluid:telemetry:DeltaManager:WrapError	Nack: Submit a summary before inserting additional operations		Nack: Submit a summary before inserting additional operations
fluid:telemetry:Container:ContainerWarning	Service busy/throttled: Nack: Submit a summary before inserting additional operations		Service busy/throttled: Nack: Submit a summary before inserting additional operations
fluid:telemetry:Container:ContainerWarning	Service busy/throttled: Nack: Submit a summary before inserting additional operations		Service busy/throttled: Nack: Submit a summary before inserting additional operations
fluid:telemetry:DeltaManager:WrapError	Nack: Submit a summary before inserting additional operations		Nack: Submit a summary before inserting additional operations
fluid:telemetry:DeltaManager:WrapError	Nack: Submit a summary before inserting additional operations		Nack: Submit a summary before inserting additional operations

And the summarizer logs show the following pattern:
fluid:telemetry:SummaryManager:CreatingSummarizer
fluid:telemetry:Summarizer:StoppingSummarizer		parentNotConnected
(...)
fluid:telemetry:SummaryStatus:Behind
fluid:telemetry:SummaryManager:CreatingSummarizer
fluid:telemetry:Summarizer:StoppingSummarizer		parentNotConnected
(...)
fluid:telemetry:SummaryManager:CreatingSummarizer
fluid:telemetry:Summarizer:StoppingSummarizer		parentNotConnected
fluid:telemetry:SummaryStatus:Behind
fluid:telemetry:Summarizer:NotStarted		DifferentComputedSummarizer
fluid:telemetry:SummaryManager:RunningSummarizer_end
(...)
fluid:telemetry:SummaryManager:CreatingSummarizer
fluid:telemetry:SummaryStatus:Behind
fluid:telemetry:Summarizer:StoppingSummarizer		parentShouldNotSummarize
fluid:telemetry:SummaryStatus:Behind
fluid:telemetry:SummaryManager:CreatingSummarizer
fluid:telemetry:Summarizer:StoppingSummarizer		parentNotConnected

Incident 257910811 may be related, however that particular instance recovered by itself.
cc @vladsud @pleath

First, some related issue I've been looking yesterday, with deeper investigation into 2 recent docs that hit similar problems: https://github.com/microsoft/FluidFramework/issues/7589
When it comes to docId == "M%2F1pui%2FeRmFe%2FD8nLpqwO1YhTAYYENnGQUOL4g8Kx%2B4%3D", here are some observations:
First events started on 2021-08-31, for loaderId == "14e785a3-5e6e-45fd-838c-7f126c00d931", runtimeVersion = 0.46.0
What I see is that whenever interactive client disconnects due to 10K ops nack, summarizer client does not close and keeps loading just fine, but then eventually when it fully catches up ("connects"), it closes itself with fluid:telemetry:Summarizer:NotStarted	DifferentComputedSummarizer
I.e. fluid:telemetry:Summarizer:StoppingSummarizer / parentNotConnected event that fires before tells us that this summarizer should go away, but it proceeds forward despite of it to evaluate if last summary needs to be performed (which gets us out of this state), but it never happens as by the time summarizing client is fully up to date, it bails out due to DifferentComputedSummarizer check.
This query looks at overall picture - both successes & failures:

union Office_Fluid_FluidRuntime_*
| where Data_docId == "M%2F1pui%2FeRmFe%2FD8nLpqwO1YhTAYYENnGQUOL4g8Kx%2B4%3D"
| where Data_clientType == "noninteractive/summarizer"
| where Data_reason != "" or Data_error != "" or Data_eventName contains "Summarize_end" or Data_eventName contains "Summarize_cancel"
| where Data_eventName notcontains "DeltaManager"
| where Data_eventName notcontains "Container"
| project Event_Sequence, Event_Time, Data_eventName, Data_reason, Data_error, Data_clientType, Data_containerId

What I see from here is

For most part, when summaries are not going, parentNotConnected is followed by DifferentComputedSummarizer, and the later is the core reason we should look into why summaries are not being created.

Note that DifferentComputedSummarizer keyword does not exist in main any more. Looking through PRs, I've removed this code in https://github.com/microsoft/FluidFramework/pull/7230/files, which points to # 4 point in description of this PR on why this check was wrong and caused last Summary to not run properly - exactly the case we observer here!

There are cases where parentNotConnected is not followed by DifferentComputedSummarizer, but these cases are rarer, and it feels like it should be lower priority case to look into. For example, there is a block of such cases at the end (after 2021-09-07T17:29:35.75Z)

That said, if you look at nacks, you will see that there are a lot of nacks on 09/14. And looking at details (loaderId == "8adea59a-a1ec-4d1d-8855-8ff7ba60148c") you see that summarizer client was not attempted to be created for very long time after last summarizer quite. This looks to be https://github.com/microsoft/FluidFramework/pull/7321 problem. I checked - it is not in 0.46!

So to sum up, we need latest fixes. I have not checked, but I believe all of them are on 0.47 (need to double check), and all these sessions are for runtime = 0.46, so we are missing all of the proactive work I've done in this space.

This seems to be related to the old 10K ops issue, and possible fixed?

authors: andre4i pleath vladsud


text:
build(client): Bump build-tools dependencies to 0.46.0

Updates the build-tools to 0.46.0 in the client release group.
Primary changes:
The biggest changes in this release are in the type tests.

The tests are now output in order sorted by type name.
New in 0.46, the type tests no longer depend on any other build task, so the task config has been updated accordingly. Also, the default entrypoint that is tested in each package is now explicitly recorded in package.json (those changes are in #22495).

Additional changes:

The config now has a version property set to 1. This is a schema version that we can change as needed when we make schema changes to the config(s).
The "check:exports" task config for test-loader-utils needed manual updates because it does not have an api task. It has been updated with a custom task definition.
Several packages had task overrides that were incomplete/incorrect because they did not inherit the global task config. Those packages have been updated to inherit the global config.
The same packages from the previous step also have an additional task config for the "build:esnext" task to depend on typetests:gen; this is needed so that ESM and CJS can be built separately.

Not sure I understand the "so they can be built separately" bit. I think I understand that both ESM and CJS builds want the typetests already generated, so the build:esnext->typetests:gen dependency would make sense to me from that perspective. If there is something besides that ("built separately" makes me think "positioned in the build graph so that they can be run concurrently"), it's not clear to me.

Eventually, one should be able to run pnpm fluid-build -t build:esnext and build ONLY the ESM of all packages. That's what I mean by built separately. The CJS and ESM builds should not depend on each other, though they can share common deps (e.g. this one).
Without the build:esnext -> typetests:gen dependency, running that fluid-build command would not trigger typetests:gen in that package. The packages in question do not have a build:test task, so they're building tests as part of tsc/build:esnext -- that's why the package-local deps are needed. (However, right now there are some other tasks in the chain that end up mixing CJS and ESM builds, so there is other cleanup needed before we can build them independently. This just ensures we don't incur more debt.

Got it, thanks for the clarification 👍


text:
[python-package] Seeing unexpected behaviour in refit()

Description
Hi team,
I'm currently working on a classification task that involves incremental learning .To update the model with new data I am using refit() function.Now given an instance X=[x1,x2,x3,...,xn],and tree_index, I am  seeing X falling into different leaf node for initial and updated model which is not expected as refit does not change the structure of the main model.Can somebody please help me understand why it is happening ?
Reproducible example
Updating main model
updated_model = initial_model.refit(data = df_train[initial_model.feature_name()],label = df_train["target"],
                                  weight=df_train["sample_weights"],decay_rate=0.95)

example = df_test[df_test.index==0].values
l1 = initial_model.predict(example[fixed_model.feature_name()],num_iteration=1,pred_leaf=True) # get the leaf node of 1st tree of main model
l2 = updated_model.predict(example[updated_model.feature_name()],num_iteration=1,pred_leaf=True) # get the leaf node of 1st tree of updated model
l1,l2
(array([177], dtype=int32), array([187], dtype=int32)

I was expecting l1 and l2 to be equal
Environment info
LightGBM version : 4.4.0
python version : 3.8.3

Thanks for using LightGBM.
You're right that given two Booster objects, one of which was called by running .refit() on the other, the same sample should fall into the exact same leaf nodes by default. This is because .refit() preserves the tree structure.
I tried tonight with the latest development version of lightgbm (https://github.com/microsoft/LightGBM/commit/a9df7f113f7e603ed06dcc04baf2bc55f28ba090) and observed exactly that behavior:
import lightgbm as lgb
import pandas as pd
from sklearn.datasets import make_regression

X, y = make_regression(n_samples=10_000, n_features=5, random_state=708)
df_train = pd.DataFrame(X)

bst = lgb.train(
    train_set=lgb.Dataset(
        data=df_train,
        label=y
    ),
    params={
        "objective": "regression",
        "num_iterations": 10,
        "seed": 708,
        "deterministic": True,
        "force_row_wise": True,
        "num_threads": 1
    }
)

bst_new = bst.refit(
    data=df_train,
    label=y,
    decay_rate=0.95
)

Chose a sample and saw it fall into the same leaf nodes in both models' first tree:
example_case = df_train.head(1)

orig_leaf_index = bst.predict(example_case, num_iteration=1, pred_leaf=True)
new_leaf_index = bst_new.predict(example_case, num_iteration=1, pred_leaf=True)
print(f"leaf indices: old={orig_leaf_index}, new={new_leaf_index}")
# leaf indices: old=[10], new=[10]

But with different raw predicted values:
bst.predict(example_case, num_iteration=1, raw_score=True)
# array([-12.81107821]

bst_new.predict(example_case, num_iteration=1, raw_score=True)
# array([-17.67358137])

environment info (click me)

M2 Mac, macOS Sonoma 14.4.1
Python 3.11.9

conda env export:
name: lgb-dev
channels:
  - conda-forge
dependencies:
  - atk-1.0=2.38.0=hd03087b_2
  - aws-c-auth=0.7.22=hec39e38_2
  - aws-c-cal=0.6.14=h5db4892_1
  - aws-c-common=0.9.19=h99b78c6_0
  - aws-c-compression=0.2.18=h5db4892_6
  - aws-c-event-stream=0.4.2=h5eab607_12
  - aws-c-http=0.8.1=had10953_17
  - aws-c-io=0.14.8=hb5a7b21_5
  - aws-c-mqtt=0.10.4=h78534b8_4
  - aws-c-s3=0.5.9=h1755d02_3
  - aws-c-sdkutils=0.1.16=h5db4892_2
  - aws-checksums=0.1.18=h5db4892_6
  - aws-crt-cpp=0.26.9=h03bff2b_0
  - aws-sdk-cpp=1.11.329=hb37a6d0_3
  - brotli=1.1.0=hb547adb_1
  - brotli-bin=1.1.0=hb547adb_1
  - bzip2=1.0.8=h93a5062_5
  - c-ares=1.28.1=h93a5062_0
  - ca-certificates=2024.7.4=hf0a4a13_0
  - cairo=1.18.0=hd1e100b_0
  - certifi=2024.6.2=pyhd8ed1ab_0
  - cffi=1.16.0=py311h4a08483_0
  - cfgv=3.3.1=pyhd8ed1ab_0
  - click=8.1.7=unix_pyh707e725_0
  - cloudpickle=3.0.0=pyhd8ed1ab_0
  - cmakelint=1.4.3=pyhd8ed1ab_0
  - colorama=0.4.6=pyhd8ed1ab_0
  - coverage=7.5.0=py311hd23d018_0
  - cpplint=1.6.0=pyhd8ed1ab_0
  - cycler=0.12.1=pyhd8ed1ab_0
  - distlib=0.3.8=pyhd8ed1ab_0
  - exceptiongroup=1.2.0=pyhd8ed1ab_2
  - expat=2.6.2=hebf3989_0
  - filelock=3.14.0=pyhd8ed1ab_0
  - font-ttf-dejavu-sans-mono=2.37=hab24e00_0
  - font-ttf-inconsolata=3.000=h77eed37_0
  - font-ttf-source-code-pro=2.038=h77eed37_0
  - font-ttf-ubuntu=0.83=h77eed37_1
  - fontconfig=2.14.2=h82840c6_0
  - fonts-conda-ecosystem=1=0
  - fonts-conda-forge=1=0
  - fonttools=4.53.0=py311hd3f4193_0
  - freetype=2.12.1=hadb7bae_2
  - fribidi=1.0.10=h27ca646_0
  - gdk-pixbuf=2.42.11=h13c029f_0
  - gettext=0.22.5=h8fbad5d_2
  - gettext-tools=0.22.5=h8fbad5d_2
  - gflags=2.2.2=hc88da5d_1004
  - giflib=5.2.2=h93a5062_0
  - glog=0.7.1=heb240a5_0
  - gmp=6.3.0=hebf3989_1
  - graphite2=1.3.13=hebf3989_1003
  - graphviz=9.0.0=h3face73_1
  - gtk2=2.24.33=h7895bb2_4
  - gts=0.7.6=he42f4ea_4
  - harfbuzz=8.4.0=hbe0f7c0_0
  - icu=73.2=hc8870d7_0
  - identify=2.5.36=pyhd8ed1ab_0
  - iniconfig=2.0.0=pyhd8ed1ab_0
  - joblib=1.4.2=pyhd8ed1ab_0
  - kiwisolver=1.4.5=py311he4fd1f5_1
  - krb5=1.21.3=h237132a_0
  - lcms2=2.16=ha0e7c42_0
  - lerc=4.0.0=h9a09cb3_0
  - libabseil=20240116.2=cxx17_hebf3989_0
  - libarrow=16.1.0=h28dd788_6_cpu
  - libarrow-acero=16.1.0=h00cdb27_6_cpu
  - libarrow-dataset=16.1.0=h00cdb27_6_cpu
  - libarrow-substrait=16.1.0=hc68f6b8_6_cpu
  - libasprintf=0.22.5=h8fbad5d_2
  - libasprintf-devel=0.22.5=h8fbad5d_2
  - libblas=3.9.0=22_osxarm64_openblas
  - libbrotlicommon=1.1.0=hb547adb_1
  - libbrotlidec=1.1.0=hb547adb_1
  - libbrotlienc=1.1.0=hb547adb_1
  - libcblas=3.9.0=22_osxarm64_openblas
  - libcrc32c=1.1.2=hbdafb3b_0
  - libcurl=8.8.0=h7b6f9a7_1
  - libcxx=16.0.6=h4653b0c_0
  - libdeflate=1.20=h93a5062_0
  - libedit=3.1.20191231=hc8eb9b7_2
  - libev=4.33=h93a5062_2
  - libevent=2.1.12=h2757513_1
  - libexpat=2.6.2=hebf3989_0
  - libffi=3.4.2=h3422bc3_5
  - libgd=2.3.3=hfdf3952_9
  - libgettextpo=0.22.5=h8fbad5d_2
  - libgettextpo-devel=0.22.5=h8fbad5d_2
  - libgfortran=5.0.0=13_2_0_hd922786_3
  - libgfortran5=13.2.0=hf226fd6_3
  - libglib=2.80.0=hfc324ee_6
  - libgoogle-cloud=2.24.0=hfe08963_0
  - libgoogle-cloud-storage=2.24.0=h3fa5b87_0
  - libgrpc=1.62.2=h9c18a4f_0
  - libiconv=1.17=h0d3ecfb_2
  - libintl=0.22.5=h8fbad5d_2
  - libintl-devel=0.22.5=h8fbad5d_2
  - libjpeg-turbo=3.0.0=hb547adb_1
  - liblapack=3.9.0=22_osxarm64_openblas
  - libnghttp2=1.58.0=ha4dd798_1
  - libopenblas=0.3.27=openmp_h6c19121_0
  - libparquet=16.1.0=hcf52c46_6_cpu
  - libpng=1.6.43=h091b4b1_0
  - libprotobuf=4.25.3=hbfab5d5_0
  - libre2-11=2023.09.01=h7b2c953_2
  - librsvg=2.58.0=hb3d354b_1
  - libsqlite=3.45.3=h091b4b1_0
  - libssh2=1.11.0=h7a5bd25_0
  - libthrift=0.19.0=h026a170_1
  - libtiff=4.6.0=h07db509_3
  - libutf8proc=2.8.0=h1a8c8d9_0
  - libwebp=1.3.2=hf30222e_1
  - libwebp-base=1.3.2=h93a5062_1
  - libxcb=1.15=hf346824_0
  - libxml2=2.12.6=h0d0cfa8_2
  - libzlib=1.2.13=h53f4e23_5
  - llvm-openmp=18.1.8=hde57baf_0
  - lz4-c=1.9.4=hb7217d7_0
  - matplotlib=3.8.4=py311ha1ab1f8_2
  - matplotlib-base=3.8.4=py311h000fb6e_2
  - munkres=1.1.4=pyh9f0ad1d_0
  - mypy=1.10.0=py311hd23d018_0
  - mypy_extensions=1.0.0=pyha770c72_0
  - ncurses=6.4.20240210=h078ce10_0
  - nodeenv=1.8.0=pyhd8ed1ab_0
  - numpy=2.0.0=py311h4268184_0
  - openjpeg=2.5.2=h9f1df11_0
  - openssl=3.3.1=hfb2fe0b_1
  - orc=2.0.1=h47ade37_1
  - packaging=24.0=pyhd8ed1ab_0
  - pandas=2.2.2=py311h4b4568b_1
  - pandoc=3.1.13=hce30654_0
  - pango=1.52.2=hb067d4f_0
  - pcre2=10.43=h26f9a81_0
  - pillow=10.3.0=py311h0b5d0a1_0
  - pip=24.0=pyhd8ed1ab_0
  - pixman=0.43.4=hebf3989_0
  - platformdirs=4.2.1=pyhd8ed1ab_0
  - pluggy=1.5.0=pyhd8ed1ab_0
  - pre-commit=3.7.0=pyha770c72_0
  - psutil=5.9.8=py311h05b510d_0
  - pthread-stubs=0.4=h27ca646_1001
  - pyarrow=16.1.0=py311h35c05fe_2
  - pyarrow-core=16.1.0=py311hb5ba6a5_2_cpu
  - pycparser=2.22=pyhd8ed1ab_0
  - pyparsing=3.1.2=pyhd8ed1ab_0
  - pytest=8.2.2=pyhd8ed1ab_0
  - pytest-cov=5.0.0=pyhd8ed1ab_0
  - python=3.11.9=h932a869_0_cpython
  - python-dateutil=2.9.0=pyhd8ed1ab_0
  - python-graphviz=0.20.3=pyh717bed2_0
  - python-tzdata=2024.1=pyhd8ed1ab_0
  - python_abi=3.11=4_cp311
  - pytz=2024.1=pyhd8ed1ab_0
  - pyyaml=6.0.1=py311heffc1b2_1
  - re2=2023.09.01=h4cba328_2
  - readline=8.2=h92ec313_1
  - ruff=0.4.7=py311hd374d79_0
  - scikit-learn=1.5.1=py311hbfb48bc_0
  - scipy=1.14.0=py311hceeca8c_0
  - setuptools=69.5.1=pyhd8ed1ab_0
  - shellcheck=0.10.0=hecfb573_0
  - six=1.16.0=pyh6c4a22f_0
  - snappy=1.2.1=hd02b534_0
  - threadpoolctl=3.5.0=pyhc1e730c_0
  - tk=8.6.13=h5083fa2_1
  - toml=0.10.2=pyhd8ed1ab_0
  - tomli=2.0.1=pyhd8ed1ab_0
  - tornado=6.4.1=py311hd3f4193_0
  - typing_extensions=4.11.0=pyha770c72_0
  - tzdata=2024a=h0c530f3_0
  - ukkonen=1.0.1=py311he4fd1f5_4
  - virtualenv=20.26.0=pyhd8ed1ab_0
  - wheel=0.43.0=pyhd8ed1ab_1
  - xorg-libxau=1.0.11=hb547adb_0
  - xorg-libxdmcp=1.1.3=h27ca646_0
  - xz=5.2.6=h57fd34a_0
  - yaml=0.2.5=h3422bc3_2
  - zlib=1.2.13=h53f4e23_5
  - zstd=1.5.6=hb46c0d2_0
  - pip:
      - altgraph==0.17.4
      - annotated-types==0.7.0
      - attrs==24.2.0
      - backports-tarfile==1.2.0
      - build==1.2.1
      - charset-normalizer==3.3.2
      - check-wheel-contents==0.6.0
      - contourpy==1.3.0.dev1
      - delocate==0.11.0
      - docutils==0.21.2
      - idna==3.8
      - importlib-metadata==8.4.0
      - jaraco-classes==3.4.0
      - jaraco-context==6.0.1
      - jaraco-functools==4.0.2
      - keyring==25.3.0
      - lightgbm==4.5.0.99
      - macholib==1.16.3
      - markdown-it-py==3.0.0
      - mdurl==0.1.2
      - more-itertools==10.4.0
      - nh3==0.2.18
      - pkginfo==1.10.0
      - pydantic==2.8.2
      - pydantic-core==2.20.1
      - pydistcheck==0.7.1
      - pygments==2.18.0
      - pyproject-hooks==1.0.0
      - readme-renderer==44.0
      - requests==2.32.3
      - requests-toolbelt==1.0.0
      - rfc3986==2.0.0
      - rich==13.8.0
      - twine==5.1.1
      - urllib3==2.2.2
      - wheel-filename==1.4.1
      - zipp==3.20.1
prefix: /Users/jlamb/miniforge3/envs/lgb-dev

So to investigate this further, we're going to need your help.
Can you share a minimal, reproducible example? That'd be code that any of us can copy, paste, and run which reproduces the behavior you're seeing. You could use my example above as a starting point.
Your report leaves out a lot of important details, including the content of the training data. You may want to review "How to create a minimal, reproducible example" in the Stack Overflow docs (link) for some tips on how to report software questions like this.
If you're unable to reproduce this behavior, then could you provide the content of both models in text files? You could called .save_model() to produce those files.

authors: Abir-lal-Roy jameslamb

repo: microsoft/LightGBM

license: mit

license_source: bigquery


text:
clarify DEBUG-level log about tree depth

*TreeLearner::Train() methods have a DEBUG-level log like this:
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 6
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 7
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 8
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 8
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 7
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 9
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 8
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 9
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 7
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 9

This message is saying "ok, this tree is complete. It has n leaves and depth d". I think it's confusing to refer to the depth of that tree as max_depth, since that is also the name of a LightGBM parameter (https://lightgbm.readthedocs.io/en/latest/Parameters.html#max_depth). When I first saw logs like this, I thought "wait, why is LightGBM changing the max_depth parameter at each training iteration?".
This PR proposes changing that language to "depth" to avoid such confusion.
Linking to https://github.com/microsoft/LightGBM/issues/4026#issuecomment-808820872.

@jameslamb Please do not merge PRs when we have a pending blocking CI PR because it often results in outdated nightly releases and may confuse users.

oh! Sorry, did not consider the implications for nightly builds.

authors: StrikerRUS jameslamb


text:
Did the number of parameters take into account the parameters in the tunable classification head?

Thanks for releasing the code! I noticed that the reporting number of parameters of Lora module is 0.3 M for roberta-base. After experiments, I found that there are 0.5M parameters tunable in the sequence classification head (but it's the same for all baselines, so I am not arguing about the fairness). I wonder was I correct about the setting? Did the performances in the paper were the ones that also tune a classification head for classification tasks?

Hi Shengding,
You are right. We didn't include the classification head when counting parameters for all baselines, even though they were trained (otherwise they would be random). We'll add a note in our manuscript stating this explicitly.
P.S. My apology for the delayed response. I left Microsoft late last year and stopped receiving notifications.

authors: ShengdingHu edwardjhu

repo: microsoft/LoRA


text:
Add support to output the pre and post values that are graphed for treatment and comparison groups

When trying to understand the percentage improvement of a treatment, getting the outputs of the pre and post values of treatment and comparison would be super helpful. Bigger plus would be to have the treatment % impact outputted along with the set of outputs already printed to the CMD today.

Totally agree. These are super useful numbers to show when showing what your program has done.
Let’s think about how we’ll add this functionality to didDesigner. Feel free to submit a draft PR with the functions you’ll add and what they’ll do.

@HenokAddis Working on a draft now should be finished tonight.

https://github.com/microsoft/LongitudinalDifferenceInDifferencesPy/pull/8

merged the changes! Thanks @jlbuja!

authors: HenokAddis csbuja jlbuja


text:
Readme Tamil added

Tamil translation of Readme #559

@jlooper can you please review my PR

hi, @jlooper I have updated the PR can you please review it

repo: microsoft/ML-For-Beginners


text:
The property "environment.docker.base_dockerfile" is not passed through correctly in pipeline jobs

There seem to be a bug on the AML Pipeline side if a custom dockerfile is used. The runconfig property environment.docker.base_dockerfile is not being passed through correctly in pipeline jobs (for PythonScriptStep).
Is there any workaround, excluding building the docker image and specifying it with environment.docker.base_image?
This is the code snippet:
myenv.docker.enabled = True
with open("./ml_service/pipelines/build_env/DockerFile", "r") as f:
        dockerfile_contents_of_base_image=f.read()
myenv.docker.base_dockerfile=dockerfile_contents_of_base_image

run_config = RunConfiguration()
run_config.environment = myenv

train_step = PythonScriptStep(
        name="Test",
        script_name=e.train_script_path,
        compute_target=aml_compute,
        source_directory=e.sources_directory_train,
        runconfig=run_config,
        inputs=[model_path],
        arguments=[…],
        allow_reuse=False
    )

I tried adding myenv.docker.base_image = None, but it didn't help.

@LianaN based on the above-shared code snippet it looks perfect. However, I don't see the content of your docker file, if you can please share the docker file and error logs for further troubleshooting?
Also, how do you verify that the submitted image being used ? do you see a new image create in the ACR instance attached to your Azure Machine Learning workspace?

Related Open Issue

authors: LianaN sudivate

repo: microsoft/MLOpsPython


text:
MRTK3 and Mixed reality Toolkit do not work together in same project

Describe the bug
MRTK3 (15/15) and Mixed reality toolkit(10/10) when imported together or one after another in a single project show right controller prefab missing error  or corrupt hence both cannot be imported together in one project when imported individually it works fine
To reproduce

Steps to reproduce the behavior:
there is no  clear option to check out the unrequired assests while importing if there is such option we can cross out the dulpicate assets to avoid issue while installing
Expected behavior
to use some object collection tools of toolkit and rig or other elements tried  to import them together for smooth functioning
Screenshots
added a screenshot
Your setup (please complete the following information)

Unity Version tried it on 2021.3,2022.3.24,2022.2 also
MRTK Version or Commit [e.g. v2.0] [e.g. b026319]

Target platform (please complete the following information)

HoloLens
HoloLens 2
WMR immersive
OpenVR

Additional context
if its a configuration issue from myside please help me becuase i couldnt find any solution over documentation

bug with featuretool installation of mrtk3 and  toolkit together

For this one, MRTK 3 should be only one selected if doing new work.
So, would re-run the Microsoft Mixed Reality Feature tool & not select the "Mixed reality toolkit(10/10)" since well, that is the older toolkit.  Otherwise, either stick with MRTK 2.8.3 or go to MRTK 3  (not both at same time though).

Hi @nmanis ... unfortunately having MRTK2 and MRTK3 in the same project is not supported.  You'll have to remove all references to MRTK2 first, before updating to MRTK3.

authors: AMollis nmanis tithisingh22

repo: microsoft/MixedRealityToolkit-Unity


text:
profiles

Overview
Expected Behavior
Actual Behavior
Steps to reproduce
(Links to sample github project preferred)
Unity Editor Version
Mixed Reality Toolkit Release Version

Again too early :-)


text:
Assertion failed on expression: 'GetAssetScriptingType(*m_RefreshQueue.begin())

Describe the bug
I get the following error after importing the Foundation package in Unity 2019.1.14f1:
Assertion failed on expression: 'GetAssetScriptingType(*m_RefreshQueue.begin()) != AssetScriptingType::Script'

To reproduce
Steps to reproduce the behavior:

Import latest 2.2 test Foundation Package

Screenshots

Your Setup

Unity Version:  2019.1.14f1
MRTK Version: v2.2

@CDiaz-MS I couldn't repo this in 2019.2.15. Are you able to repo with official released packages?

@Troy-Ferrell  yes it still repros with the 2.2 release packages but I have only seen it in Unity 2019.1.14 so far.

@CDiaz-MS cool thanks for checking!

authors: CDiaz-MS Troy-Ferrell


text:
restore missing line in release notes

The release notes were missing a line of text separating the 2.3.0 hyperlinks and describing the next list.
This change adds back "This release of the Microsoft Mixed Reality Toolkit supports the following devices and platforms."

/azp run mrtk_docs


text:
Check if instancing is supported before attempting it in visual profiler

Overview
Instancing isn't supported on all platforms, so the visual profiler should only attempt it if supported. See the fixed bug below for more info.
Changes

Fixes: https://github.com/microsoft/MixedRealityToolkit-Unity/issues/8600

/azp run

authors: davidkline-ms keveleigh


text:
Add TorchTRTConversion pass

Describe your changes
This PR adds TorchTRTConversion pass which converts nn.Linear modules in transformer layers to TRTModule from torch-tensorrt. torch-tensorrt is an extension to torch where TensorRT engines can be used like normal nn.Module.
We require this conversion to take advantage of 2:4 sparsity.
I have not added unit test for the pass since the pass requires GPU and we don't have GPU support for ci tests yet. I will add a unit test in a follow up PR that uses aml system to run the pass and evaluation on GPU.
Checklist before requesting a review

[ ] Add unit tests for this change.
[ ] Make sure all tests can pass.
[x] Update documents if necessary.
[x] Format your code by running pre-commit run --all-files
[ ] Is this a user-facing change? If yes, give a description of this change to be included in the release notes.
Add TorchTRTConversion pass.

(Optional) Issue link

I think you might mean integration test to test TRTConversion. Is it possible that mock compile_trt_model to test the logic in the TorchTRTConversion in unit test?

I think you might mean integration test to test TRTConversion. Is it possible that mock compile_trt_model to test the logic in the TorchTRTConversion in unit test?

I wasn’t sure if it would fall under integration test if we use aml system + pass.
Yes, I can mock the function. That sounds like a good idea since that’s the only one that requires gpu. I can probably mock the cuda related code too.

authors: guotuofeng jambayk

repo: microsoft/Olive


text:
Rename with ctrl-r ctrl-r doesn't work correctly UI is not working.

This issue has been moved from a ticket on Developer Community.

Typing ctlr-r ctrl-r to do a rename, in a python file.
Expect: the rename dialog UI thingy pops up and allows me to type the new name
Actual: sometimes it comes up and works correctly, and sometimes it appears briefly and then disappears.  And after the disappearance subsequence attempts to use ctrl-r ctrl-r make a beep and say it is not available now.  However right click rename does work correctly - it is only the keyboard shortcut that is unreliable.

Original Comments
Feedback Bot on 3/13/2023, 06:10 PM:
(private comment, text removed)

Original Solutions
(no solutions)

This problem persists

authors: mcginnie vsfeedback

repo: microsoft/PTVS


text:
[GB18030] Create Python Application project error "Exception of type 'System.Exception' was thrown." with GB18030 PUA characters.

See https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1835748

related pr: https://devdiv.visualstudio.com/DevDiv/_git/VS/pullrequest/483664


text:
EditorState.json file generates property "AlignInContainer" that isn't recognized in Studio

In my YAML file, I have a container with some labels inside of it. When I pack the app and upload it to Studio, it complains about Label1.AlignInContainer: "Name isn't valid. This identifier isn't recognized."
The property "AlignInContainer" is being generated in the EditorState.json file by PowerApps-Language-Tooling:
"Label1": {
    "DynamicProperties": [
      {
        "Property": {
          "PropertyName": "FillPortions",
          "RuleProviderType": "Unknown"
        },
        "PropertyName": "FillPortions"
      },
      {
        "Property": {
          "PropertyName": "AlignInContainer",
          "RuleProviderType": "Unknown"
        },
        "PropertyName": "AlignInContainer"
      },
      {
        "Property": {
          "PropertyName": "LayoutMinHeight",
          "RuleProviderType": "Unknown"
        },
        "PropertyName": "LayoutMinHeight"
      },
      {
        "Property": {
          "PropertyName": "LayoutMinWidth",
          "RuleProviderType": "Unknown"
        },
        "PropertyName": "LayoutMinWidth"
      }
    ],
    ...
}

However, when I create an app from scratch in Studio, download it, and unpack it, the YAML file is identical, but in the EditorState.json file, there is no occurrence of "AlignInContainer".
Screen1 As screen:

    Container1 As groupContainer.manualLayoutContainer:
        DisplayMode: =DisplayMode.Edit
        Height: =768
        LayoutDirection: =LayoutDirection.Horizontal
        LayoutMode: =LayoutMode.Manual
        Width: =1366
        ZIndex: =1

        Label1 As label:
            Text: ="Text 1"
            ZIndex: =1

        Label2 As label:
            Text: ="Text 2"
            Y: =40
            ZIndex: =2

        Label3 As label:
            Text: ="Text 3"
            Y: =80
            ZIndex: =3

Chatter with @thelilyliu, the issue is fixed in the latest code, hence closing.

authors: suskumar-MSFT thelilyliu

repo: microsoft/PowerApps-Language-Tooling


text:
timeghsot: add icon image

Please check the following conditions for your PR.

[x] apiDefinition.swagger.json is validated using paconn validate command.
[x] apiProperties.json has a valid brand color. Invalid brand colors are #007ee5 and #ffffff.

@Marc-Hoch It is not required to add icons to your connector in the open source repo. Please close this PR.
If you are updating your connector's submission using commit id, Please download all the connector artifacts from open source repo and add the latest icon to the downloaded folder, then zip it to add in the submission form as we do not functionality to pull the icon from open source repo automatically.

authors: Marc-Hoch sriyen-msft

repo: microsoft/PowerPlatformConnectors


text:
Show new file size

It's be handy if you could show the new file size when selecting a image size to make it easier to scale photos where the website has an upload file size limit.

good idea but I'd group this in with https://github.com/microsoft/PowerToys/issues/1053 which will be an updated UX.  Current UX doesn't afford this style due to multi-selection

authors: AlquistArj crutkas

repo: microsoft/PowerToys