text:
[Event Request] Application “Tests-ERM”, Codeunit 134098 "ERM Invoice Disc. Distribution"

[Event Request] Application “Tests-ERM”, Codeunit 134098 "ERM Invoice Disc. Distribution"

Could you please insert an OnTestInitialize Event at the beginning of the procedures Initialize?

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.

authors: bc-ghost ttaeufer


text:
[Event Request] Codeunit 7301 "Whse. Jnl.-Register Line"

To allow serial numbers with different quantity than -1, 0 or 1, I need a publisher in function "InsertWhseEntry":
Currently:

shall be like:

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.

source: gharchive/issue

authors: Busslar bc-ghost

repo: microsoft/ALAppExtensions

license: MIT

license_type: permissive

license_source: github-api


text:
API: Support for Edm.Collection in ODataEdmType XML

Situation:
In order to use complex types in custom API's we can use the Table "OData Edm Type" to describe the desired type definition using xml. The BC-Standard API's use this to represent PaymentMethods and UOM-Conversions among a few other things. All the integrated Edm Types BC ships with have one thing in common: they don't nest further than 2 levels.
Therefore the following JSON payload can be published/received properly:
{
    "id": "ORDER1234",
    "customerName": "Jon Doe",
    "lines": [
        {
            "id": 10000,
            "description": "TestItem"
        }
    ]
}

To represent the "lines" - Part in our API Page Object we would need create a OdataEdmType (name: "LINES") with the following structure:
<ComplexType Name="lines">
    <Property Name="id" Type="Edm.Int32"></Property>
    <Property Name="description" Type="Edm.String"></Property>
</ComplexType>

Since we want an Array of the new Type we would use
ODataEDMType = 'Collection(LINES)';
to assign it to a Page Control.
However, trying to define more complex scenarios leads to a Problem.
Problem
Assuming the desired JSON representation gets a bit more complicated, for example we might want a single "line" to hold another array of applicable discounts:
{
    "id": "ORDER1234",
    "customerName": "Jon Doe",
    "lines": [
        {
            "id": 10000,
            "description": "TestItem",
            "discounts": [
                {
                    "id": "d10"
                    "discountPercent": 10
                },
                {
                    "id": "d3"
                    "discountPercent": 3
                },
            ]
        }
    ]
}

In my understanding according to the Edm Type Definition the way to do this would be to create a new EdmType "DISCOUNTS"
<ComplexType Name="discounts">
    <Property Name="id" Type="Edm.Int32"></Property>
    <Property Name="discountPercent" Type="Edm.String"></Property>
</ComplexType>

and extend the "LINES" type to:
<ComplexType Name="lines">
    <Property Name="id" Type="Edm.Int32"></Property>
    <Property Name="description" Type="Edm.String"></Property>
    <Property Name="discounts" Type="Collection(DISCOUNTS)"></Property>
</ComplexType>

However doing so will yield an Error when calling the WebService or trying to receive the OData Metadata:
{
    "error": {
    "code": "Unknown",
    "message": "The OData EDM complex type Collection(DISCOUNTS) that is referenced by lines was not found."
    }
}

Types embedded in the XML-Blob in the "OData EDM Type" Record do not seem to be checked for the "Collection" Keyword.
Question:
I need to be able to receive a complex JSON payload which does not at all map to BC Data. Out of hundreds of fields and nested Objects I'm only interested in very few fields. I am fully aware that using a Flow or some proxy Service that does the mapping this would be achievable very easily, however I would rather not resort to additional external services if this is doable in pure AL.
Is there any way to achieve this in the current version?
If yes, how?
If not, are there any plans on implementing this scenario in future releases?
I'm really looking forward responses regarding this issue.

@nikolakukrika - can you help?

It is not possible to implement the complex types outside of Microsoft, this feature is not extensible in SaaS. For OnPrem you could define your own complex types however I would not recommend it.
The current implementation of complex types has two major issues:

Performance - perf impact is large, APIs using complex types are much slower than APIs without them. The reason for slowness is the fact that we are generating them via AL, a lot of time is lost in .NET interoop and AL invocation.
Stability - if AL parsing / generation throws an error then the API is broken. We should really not handle the complex types via AL, but platform should generate them.

Because of these two problems we have decided to keep complex types as internal only feature and will remove most of the complex types in near future. The plan is to replace most of the complex types with API parts. API pages can nest as many levels as necessary, there is no 2 level limitation present in the product which is something I would strongly recommend. Address may be the only complex type that will stay, we need to see how to model this one.
For your APIs I would recommend following:

See if you can use Query objects to define a payload
If you have few records (less than 1000) then you can use temporary tables if you cannot build an API
You could implement an aggregate table as we did on the Sales Invoices and other sales documents, this would give you a fast read if there is a lot of data, however there is a cost of maintain the aggregate and writes are more expensive.
4.. You could also use parts for subparts of data. The subparts of data could be bound to the temporary tables. Then the users of the API can decide to $expand or call the parts separately. This would limit the performance hit of loading a lot of data.

@wisa1 - if you have more questions, you can join this yammer group and ask here:
https://www.yammer.com/microsoft.com/#/threads/inGroup?type=in_group&feedId=14737513
The name is APIs for Dynamics 365 Business Central"

authors: AlexanderYakunin nikolakukrika wisa1


text:
Event Request T36 - SalesHeader - OnShowDocDimOnBeforeUpdateLines

Can you add a new Event on Table 36 to function ShowDocDim called OnShowDocDimOnBeforeUpdateLines
procedure ShowDocDim()
var
    OldDimSetID: Integer;
    IsHandled: Boolean;
begin
    IsHandled := false;
    OnBeforeShowDocDim(Rec, xRec, IsHandled);
    if IsHandled then
        exit;

    OldDimSetID := "Dimension Set ID";
    "Dimension Set ID" :=
      DimMgt.EditDimensionSet(
        "Dimension Set ID", StrSubstNo('%1 %2', "Document Type", "No."),
        "Shortcut Dimension 1 Code", "Shortcut Dimension 2 Code");

    OnShowDocDimOnBeforeUpdateLines(Rec, xRec);   // NEW

    if OldDimSetID <> "Dimension Set ID" then begin
        Modify;
        if SalesLinesExist then
            UpdateAllLineDim("Dimension Set ID", OldDimSetID);
    end;
end;

[IntegrationEvent(false, false)]
local procedure OnShowDocDimOnBeforeUpdateLines(var SalesHeader: Record "Sales Header"; xSalesHeader: Record "Sales Header")
begin
end;

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.


text:
[Event Request] Codeunit 80 Sales-Post - OnAfterInsertReturnReceiptLine

Hi Team,
In Codeunit 80 we need possibility to have our additional checks. Having event in procedure PostSalesLine would be a perfect solution :

[IntegrationEvent(false, false)]
local procedure OnAfterInsertReturnReceiptLine(var SalesHeader: Record "Sales Header"; SalesLine: Record "Sales Line"; var xSalesLine: Record "Sales Line"; ReturnRcptHeader: Record "Return Receipt Header"; RoundingLineInserted: Boolean)

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.

authors: MantasGurinas bc-ghost


text:
[JS Templating] Bump templating version

Update the JS templating library version.
Microsoft Reviewers: Open in CodeFlow

squash merged due to android-ci work

source: gharchive/pull-request

authors: anna-dingler jwoo-msft

repo: microsoft/AdaptiveCards


text:
EventCounters Logs location traces vs customMetrics

Is this a document under https://docs.microsoft.com/en-us/azure/azure-monitor/
Yes, https://docs.microsoft.com/en-us/azure/azure-monitor/app/eventcounters#customizing-counters-to-be-collected.
Does the document belong to this repo?
My EventCounters appear as traces in Application Insights, but according to documentation should appear as customMetrics.
Also not sure what is happening but the traces security level seems completely random. I run eventCounters and the severity is "Critical". Then I created another event source and the severity is "Information". Is there even a way to configure these traces severity and other properties through the EventCounters?

Please share a minimal repro app showing this issue. Also add screenshots where the EventCounters are shown as traces instead of customMetrics.

@cijothomas So I discovered this is due to me using the EventSourceListener.
Again, I configured the EventCounter as in the docs https://docs.microsoft.com/pt-pt/azure/azure-monitor/app/eventcounters:
in Global.asax.cs:
void Application_Start(object sender, EventArgs e)
        {
            // Code that runs on application startup
            RouteConfig.RegisterRoutes(RouteTable.Routes);
            BundleConfig.RegisterBundles(BundleTable.Bundles);

            WebApplicationHelp.Configs.createEventSource("Sample.EventCounter.Minimal");
            Microsoft.ApplicationInsights.Extensibility.EventCounterCollector.EventCounterCollectionModule
                            collector = new Microsoft.ApplicationInsights.Extensibility.EventCounterCollector.EventCounterCollectionModule();
            collector.Counters.Add(new Microsoft.ApplicationInsights.Extensibility.EventCounterCollector.EventCounterCollectionRequest("Sample.EventCounter.Minimal", "Sample.EventCounter.Minimal Count"));
            collector.Initialize(Microsoft.ApplicationInsights.Extensibility.TelemetryConfiguration.Active);
        }

Then in the function I created a class according to the provided example https://docs.microsoft.com/pt-pt/dotnet/core/diagnostics/event-counters#conditional-counters:
namespace WebApplicationHelp
{
    [EventSource(Name = "Sample.EventCounter.Minimal")]
    public sealed class _EventCounter : EventSource
    {
        public static readonly _EventCounter Log = new _EventCounter();

        private System.Diagnostics.Tracing.EventCounter _requestCounter;

        private _EventCounter(){
            _requestCounter = new System.Diagnostics.Tracing.EventCounter("request-time", this);
        }

        public void Request(string url, long elapsedMilliseconds)
        {
            WriteEvent(1, url, elapsedMilliseconds);
            _requestCounter.WriteMetric(elapsedMilliseconds);
        }

        protected override void Dispose(bool disposing)
        {
            Dispose();
            //_requestCounter?.Dispose();
            _requestCounter = null;

            base.Dispose(disposing);
        }
    }
}

As you can see inside the Request method, the WriteEvent is a call from the superclass (EventSource) and the _requestCounter.WriteMetric is a call from the EventCounter class.
The WriteMetric call is reported as a Trace in Microsoft Application Insights. The requestCounter.Metric should be reported in the customMetrics, according to the docs.
However, even though I configured the EventCounter to be reported in Application_Start, by defining the EventSource Name to be collected, nothing appears in Application Insights.
Only the WriteEvent call appears as a trace.
Also in the ApplicationInsights.config file I have the following configuration:
<Add Type="Microsoft.ApplicationInsights.EventSourceListener.EventSourceTelemetryModule, Microsoft.ApplicationInsights.EventSourceListener">
      <Sources>
        <!-- Replace/add your own EventSources as necessary. -->
        <Add Name="Sample.EventCounter.Minimal" Level="LogAlways" />
      </Sources>
    </Add>

Please note that, before adding the EventSourceListener module https://www.nuget.org/packages/Microsoft.ApplicationInsights.EventSourceListener, I had just the EventCounterCollector module https://www.nuget.org/packages/Microsoft.ApplicationInsights.EventCounterCollector/.
Isn't the EventCounterCollector module supposed to work like this? Nothing reported to Application Insights, just the EventSourceListener works, but this is not very useful since it shows in Traces.

@cijothomas This is the repo of my app in case you want to test it: https://dev.azure.com/linkestagiodiogomoura/_git/SampleWebAppTransfer?path=%2FWebApplicationHelp.
Can you test it in 5 minutes and see what I am saying?
That would be much appreciated!
Also, in fact, the issue related to some methods of the EventCounter not working are related to my runtime version. I made a issue on .NET github and it was answered there:
https://github.com/dotnet/runtime/issues/58408#issuecomment-908818339.
Please help me with this one! I think I am very near to the solution, but something is missing or maybe this shouldn't be the approach for my runtime version, since all the docs are for .NET Core. Maybe the EventCounter class shouldn't be used for ASP.NET 4.8!

UPDATE:
I replicated the Unit Tests provided at:
https://github.com/microsoft/ApplicationInsights-dotnet/blob/0c5cd658495f78f3b70fd2db6207ba31c702371f/WEB/Src/EventCounterCollector/EventCounterCollector.Tests/EventCounterCollectionModuleTests.cs.
Turns out that maybe it is not even the event source Listener, but really the EventCounter produving those traces, since in this example is really only the EventCounter making the calls.
However, there is something odd happening.
Basically all the Asserts pass but the ones related to the count of the objects (another is commented but I think there is no problem).
I commented this Assert since the class EventCounterCollectorEventSource is internal to System.Diagnostics.Tracing.
                Assert.IsTrue(CheckEventReceived(eventListener.EventsReceived, nameof(EventCounterCollectorEventSource.IgnoreEventWrittenAsCounterNotInConfiguredList)));

And the only Asserts that do not work are related to count:
Assert.IsTrue(itemsReceived.Count >= 1);
Assert.AreEqual(expectedSum, sum);
Assert.AreEqual(expectedCount, count);

So this seems something Internal to my System???
Anyway event though there is no count, there appear Traces at Application Insights:
These traces have severity level critical but contain the name of the EventSource used by the EventCounter:
query_data.csv
Image with one of these traces:

Again, I have been able to send information with the requestCounter.WriteMetric of the previous post. But that is not at all what I inted. I want them to show in customMetrics.
Additionally that WriteMetric function was very ill functional since if I changed the EventID a weird runtime error Array out of Index appeared. But anyway those traces reported a URL and a Metric:

Check the Url and ellapsed miliseconds parameters that I passed are reported to Application Insights.

@cijothomas no help? Please provide some insights. If you could clone the repo will be great. As I said, everything is configured correctly so I don't really understand what's going on.

@esyker I am unsure what is the issue being faces.
EventCounter - are only expected to show up as CustomMetric. It looks like you are mixing it up with EventSource (https://www.nuget.org/packages/Microsoft.ApplicationInsights.EventSourceListener). which is expected to show up as Traces table. This is by design.
If this is not your question, please share the minimal repro app, along with a concise description of Expected behavior vs Actual behavior.

Might be i misunderstood something, but arent EventCounters also an EventSource, meaning that if we use EventSource listener for our regular "EventSources" we also end up with traces for our metrics?
This seems to be the behaviour I am observing at least.
If this is indeed the designed behaviour, how can we go about using both EventCounters and have the EventSourceListener filter out "counters" - because its generating a lot of traces!

@esyker not sure how much this is relevant to you anymore, but I had a support case for this and someone on the product group confirmed this is "by design".
Basically since EventCounters are also Event Sources, they might get registered twice (depending on how you register your event sources).
They recommended that Event Counters are disabled when configuring Event Sources:
services.ConfigureTelemetryModule<EventSourceTelemetryModule>((module, o) =>
{
    module.DisabledSources.Add(new DisableEventSourceRequest() { Name = "Microsoft-AspNetCore-Server-Kestrel" });
}

I ended up writing a ITelemetryProcessor for filtering instead:
public class FilteringTelemetryProcessor : ITelemetryProcessor
{
    private readonly ITelemetryProcessor next;

    public FilteringTelemetryProcessor(ITelemetryProcessor next)
    {
        this.next = next;
    }

    public void Process(ITelemetry item)
    {
        if (this.OkToSend(item))
        {
            this.next.Process(item);
        }
    }

    private bool OkToSend(ITelemetry item)
    {
        return item switch
        {
            TraceTelemetry traceTelemetry when IsEventPayloadTrace(traceTelemetry) => false,
            _ => true
        };
    }

    private static bool IsEventPayloadTrace(TraceTelemetry trace) => trace.Properties.TryGetValue("Payload", out var payload) && payload.Equals("System.Diagnostics.Tracing.EventPayload");
}

I hope this helps someone in the same boat as me.

authors: cijothomas esbenbach esyker

repo: microsoft/ApplicationInsights-dotnet


text:
Create New Project Terraform use demo generator

We ran into some issues and we are sorry about that!
The log below will provide you insights into why the provisioning failed. You can raise and issue with the logs here and we will try to help you.
Click on View Diagnostics button to share logs with us.

Error while creating release definition: Tasks with versions 'Replace tokens in terraform file:5., Install Terraform 0.12.3:0., Terraform : init:2., Terraform : plan:2., Terraform : apply -auto-approve:2.*' are not valid for deploy job 'Agent job' in stage Dev.

I solve it thanks .
this link helped me .
https://www.editcode.net/forum.php?mod=viewthread&tid=820823&extra=page%3D1&mobile=2

repo: microsoft/AzureDevOpsDemoGenerator


text:
An error occurs after attaching one blob using a container-level shared access signature URI

Storage Explorer Version: 1.10.1
Build: 20191031.6
Branch: rel/1.11.0
Platform/OS: Windows 10/ Linux Ubuntu 19.04/macOS High Sierra
Architecture: ia32/x64
Regression From: Not a regression
Steps to reproduce:

Expand one storage account -> Blob Containers.
Select one blob container which contains one blob at least.
Right click the blob then select ‘Get Shared Access Signature…’.
Check ‘Generate container-lever shared access signature URI ’ -> Create -> Copy the URI.
Open connect dialog -> Select use SAS URI -> Next.
Paste the copied URI -> Check the result.

Expect Experience:
No error occurs.
Actual Experience:

Successfully attached.

An error occurs when opening the blob container to which the blob belongs.

@craxal Craig, this is for regular blob, not ADLS Gen2.  While I am not getting the same error, but i also got an error on 1.10.1.  My error was in the Activity log saying: Error occurred while adding new connection: TypeError:Cannot read property 'forNext' of undefined.  However, the connection was established and I was able to open the blob.  v-xianya's experience was a bit different in 1.11.0 candidate.  Please investigate for 1.11.0.

Here is the stack from my error in 1.10.1.
{
  "name": "TypeError",
  "message": "Cannot read property 'forNext' of undefined",
  "__zone_symbol__error": {
    "originalStack": "Error: Cannot read property 'forNext' of undefined\n    at AppInsightsAsyncCorrelatedErrorWrapper.ZoneAwareError (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\zone.js\\dist\\zone-node.js:811:33)\n    at new AppInsightsAsyncCorrelatedErrorWrapper (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\applicationinsights\\out\\AutoCollection\\CorrelationContextManager.js:172:18)\n    at Function.ExceptionSerializer.deserialize (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\se-exception-serializer\\dist\\ExceptionSerializer.js:49:28)\n    at CloudExplorerProxyMarshaler._finalizeOperation (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\CloudExplorerProxyMarshaler.js:83:91)\n    at Object.finalizeOperation (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\CloudExplorerProxyMarshaler.js:107:80)\n    at Function.<anonymous> (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\StandardMarshaler.js:28:63)\n    at step (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:136:27)\n    at Object.next (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:117:57)\n    at C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:110:75\n    at new ZoneAwarePromise (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\zone.js\\dist\\zone-node.js:551:29)\n    at Object.__awaiter (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:106:16)\n    at JsonPortMarshaler.standardMarshaler.(anonymous function) [as finalizeOperation] (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\StandardMarshaler.js:22:32)\n    at JsonPortMarshaler.onMessage (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\messaging\\marshaler.js:101:41)\n    at Port.EventTarget.dispatchEvent (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\common\\event.js:69:30)\n    at Port.processMessage (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\messaging\\communication.js:163:18)",
    "zoneAwareStack": "Error: Cannot read property 'forNext' of undefined\n    at AppInsightsAsyncCorrelatedErrorWrapper.ZoneAwareError (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\zone.js\\dist\\zone-node.js:811:33)\n    at new AppInsightsAsyncCorrelatedErrorWrapper (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\applicationinsights\\out\\AutoCollection\\CorrelationContextManager.js:172:18)\n    at Function.ExceptionSerializer.deserialize (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\se-exception-serializer\\dist\\ExceptionSerializer.js:49:28)\n    at CloudExplorerProxyMarshaler._finalizeOperation (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\CloudExplorerProxyMarshaler.js:83:91)\n    at Object.finalizeOperation (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\CloudExplorerProxyMarshaler.js:107:80)\n    at Function.<anonymous> (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\StandardMarshaler.js:28:63)\n    at step (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:136:27)\n    at Object.next (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:117:57)\n    at C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:110:75\n    at new ZoneAwarePromise (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\zone.js\\dist\\zone-node.js:551:29)\n    at Object.__awaiter (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:106:16)\n    at JsonPortMarshaler.standardMarshaler.(anonymous function) [as finalizeOperation] (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\StandardMarshaler.js:22:32)\n    at JsonPortMarshaler.onMessage (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\messaging\\marshaler.js:101:41)\n    at Port.EventTarget.dispatchEvent (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\common\\event.js:69:30)\n    at Port.processMessage (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\messaging\\communication.js:163:18)"
  },
  "stack": "TypeError: Cannot read property 'forNext' of undefined\n    at Path.followPath (file:///C:/Program%20Files%20(x86)/Microsoft%20Azure%20Storage%20Explorer/resources/app/out/CloudHub/js/CloudExplorer/TreeNode/Path.js:31:35)\n    at file:///C:/Program%20Files%20(x86)/Microsoft%20Azure%20Storage%20Explorer/resources/app/out/CloudHub/js/CloudExplorer/TreeNode/Path.js:51:46\n    at Array.map (<anonymous>)\n    at file:///C:/Program%20Files%20(x86)/Microsoft%20Azure%20Storage%20Explorer/resources/app/out/CloudHub/js/CloudExplorer/TreeNode/Path.js:50:76",
  "originalStack": "Error: Cannot read property 'forNext' of undefined\n    at AppInsightsAsyncCorrelatedErrorWrapper.ZoneAwareError (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\zone.js\\dist\\zone-node.js:811:33)\n    at new AppInsightsAsyncCorrelatedErrorWrapper (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\applicationinsights\\out\\AutoCollection\\CorrelationContextManager.js:172:18)\n    at Function.ExceptionSerializer.deserialize (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\se-exception-serializer\\dist\\ExceptionSerializer.js:49:28)\n    at CloudExplorerProxyMarshaler._finalizeOperation (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\CloudExplorerProxyMarshaler.js:83:91)\n    at Object.finalizeOperation (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\CloudExplorerProxyMarshaler.js:107:80)\n    at Function.<anonymous> (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\StandardMarshaler.js:28:63)\n    at step (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:136:27)\n    at Object.next (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:117:57)\n    at C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:110:75\n    at new ZoneAwarePromise (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\zone.js\\dist\\zone-node.js:551:29)\n    at Object.__awaiter (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:106:16)\n    at JsonPortMarshaler.standardMarshaler.(anonymous function) [as finalizeOperation] (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\StandardMarshaler.js:22:32)\n    at JsonPortMarshaler.onMessage (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\messaging\\marshaler.js:101:41)\n    at Port.EventTarget.dispatchEvent (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\common\\event.js:69:30)\n    at Port.processMessage (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\messaging\\communication.js:163:18)",
  "zoneAwareStack": "Error: Cannot read property 'forNext' of undefined\n    at AppInsightsAsyncCorrelatedErrorWrapper.ZoneAwareError (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\zone.js\\dist\\zone-node.js:811:33)\n    at new AppInsightsAsyncCorrelatedErrorWrapper (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\applicationinsights\\out\\AutoCollection\\CorrelationContextManager.js:172:18)\n    at Function.ExceptionSerializer.deserialize (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\se-exception-serializer\\dist\\ExceptionSerializer.js:49:28)\n    at CloudExplorerProxyMarshaler._finalizeOperation (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\CloudExplorerProxyMarshaler.js:83:91)\n    at Object.finalizeOperation (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\CloudExplorerProxyMarshaler.js:107:80)\n    at Function.<anonymous> (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\StandardMarshaler.js:28:63)\n    at step (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:136:27)\n    at Object.next (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:117:57)\n    at C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:110:75\n    at new ZoneAwarePromise (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\zone.js\\dist\\zone-node.js:551:29)\n    at Object.__awaiter (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:106:16)\n    at JsonPortMarshaler.standardMarshaler.(anonymous function) [as finalizeOperation] (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\StandardMarshaler.js:22:32)\n    at JsonPortMarshaler.onMessage (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\messaging\\marshaler.js:101:41)\n    at Port.EventTarget.dispatchEvent (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\common\\event.js:69:30)\n    at Port.processMessage (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\messaging\\communication.js:163:18)",
  "__zone_symbol__stack": "TypeError: Cannot read property 'forNext' of undefined\n    at Path.followPath (file:///C:/Program%20Files%20(x86)/Microsoft%20Azure%20Storage%20Explorer/resources/app/out/CloudHub/js/CloudExplorer/TreeNode/Path.js:31:35)\n    at file:///C:/Program%20Files%20(x86)/Microsoft%20Azure%20Storage%20Explorer/resources/app/out/CloudHub/js/CloudExplorer/TreeNode/Path.js:51:46\n    at Array.map (<anonymous>)\n    at file:///C:/Program%20Files%20(x86)/Microsoft%20Azure%20Storage%20Explorer/resources/app/out/CloudHub/js/CloudExplorer/TreeNode/Path.js:50:76",
  "__zone_symbol__name": "TypeError",
  "__zone_symbol__message": "Cannot read property 'forNext' of undefined",
  "__zone_symbol__originalStack": "Error: Cannot read property 'forNext' of undefined\n    at AppInsightsAsyncCorrelatedErrorWrapper.ZoneAwareError (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\zone.js\\dist\\zone-node.js:811:33)\n    at new AppInsightsAsyncCorrelatedErrorWrapper (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\applicationinsights\\out\\AutoCollection\\CorrelationContextManager.js:172:18)\n    at Function.ExceptionSerializer.deserialize (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\se-exception-serializer\\dist\\ExceptionSerializer.js:49:28)\n    at CloudExplorerProxyMarshaler._finalizeOperation (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\CloudExplorerProxyMarshaler.js:83:91)\n    at Object.finalizeOperation (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\CloudExplorerProxyMarshaler.js:107:80)\n    at Function.<anonymous> (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\StandardMarshaler.js:28:63)\n    at step (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:136:27)\n    at Object.next (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:117:57)\n    at C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:110:75\n    at new ZoneAwarePromise (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\zone.js\\dist\\zone-node.js:551:29)\n    at Object.__awaiter (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:106:16)\n    at JsonPortMarshaler.standardMarshaler.(anonymous function) [as finalizeOperation] (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\StandardMarshaler.js:22:32)\n    at JsonPortMarshaler.onMessage (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\messaging\\marshaler.js:101:41)\n    at Port.EventTarget.dispatchEvent (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\common\\event.js:69:30)\n    at Port.processMessage (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\messaging\\communication.js:163:18)",
  "__zone_symbol__zoneAwareStack": "Error: Cannot read property 'forNext' of undefined\n    at AppInsightsAsyncCorrelatedErrorWrapper.ZoneAwareError (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\zone.js\\dist\\zone-node.js:811:33)\n    at new AppInsightsAsyncCorrelatedErrorWrapper (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\applicationinsights\\out\\AutoCollection\\CorrelationContextManager.js:172:18)\n    at Function.ExceptionSerializer.deserialize (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\se-exception-serializer\\dist\\ExceptionSerializer.js:49:28)\n    at CloudExplorerProxyMarshaler._finalizeOperation (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\CloudExplorerProxyMarshaler.js:83:91)\n    at Object.finalizeOperation (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\CloudExplorerProxyMarshaler.js:107:80)\n    at Function.<anonymous> (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\StandardMarshaler.js:28:63)\n    at step (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:136:27)\n    at Object.next (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:117:57)\n    at C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:110:75\n    at new ZoneAwarePromise (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\zone.js\\dist\\zone-node.js:551:29)\n    at Object.__awaiter (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\tslib\\tslib.js:106:16)\n    at JsonPortMarshaler.standardMarshaler.(anonymous function) [as finalizeOperation] (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\out\\app\\renderer\\marshalers\\StandardMarshaler.js:22:32)\n    at JsonPortMarshaler.onMessage (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\messaging\\marshaler.js:101:41)\n    at Port.EventTarget.dispatchEvent (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\common\\event.js:69:30)\n    at Port.processMessage (C:\\Program Files (x86)\\Microsoft Azure Storage Explorer\\resources\\app\\node_modules\\daytona\\pluginHost\\messaging\\communication.js:163:18)"
}

@v-xianya @JingLouMSFT Can you please share the exact URL you used to attach? Also check the Summary panel before you click Connect.
The following is an example of SAS URI to a blob:
https://cralvordtest02.blob.core.windows.net/blob-container-01/01.txt?st=2019-10-31T20%3A17%3A42Z&se=2019-11-01T20%3A17%3A42Z&sp=racwl&sv=2018-03-28&sr=c&sig=A28YIcipErl%2FcjiMIhulb6VePMHT%2BZbj%2BZb5bcPRZAI%3D
Trying to use this URL in the Connect dialog will give you these results:

Notice the resource name is the path to the blob, not the container. If you try to connect with this, you will get the "TypeError: Cannot read property 'forNext' of undefined" error.
You should only use SAS URIs to a blob container. You can do this by removing the blob from the URI path:
https://cralvordtest02.blob.core.windows.net/blob-container-01?st=2019-10-31T20%3A17%3A42Z&se=2019-11-01T20%3A17%3A42Z&sp=racwl&sv=2018-03-28&sr=c&sig=A28YIcipErl%2FcjiMIhulb6VePMHT%2BZbj%2BZb5bcPRZAI%3D
This URI should work perfectly.

Hi @craxal  Thanks for your updates! More information please see the blow.
The SAS URI:
https://4776retxnguu62.blob.core.windows.net/blobcontainer/blob.txt?st=2019-11-01T01%3A52%3A58Z&se=2019-11-02T01%3A52%3A58Z&sp=racwdl&sv=2018-03-28&sr=c&sig=W2j%2FBYXyVHFulp9Ggcaq%2B1hkWj6%2Blt71l9vWym8AnEQ%3D
The Connnection Summary:

Connect result:

Activities log: Successfully added new connection.
Tree view: The attched item shows well(blobcontainer/blob.txt(SAS))
Error: An error "The requested URI does not represent any resource on the server. RequestId:1dda927a-501e-006c-7a59-902de4000000 Time:2019-11-01T02:08:58.9196528Z" occurs when atuo opening the attached blob.

By the way, it works well if I remove the blob name from the above URI, because it attached one blob contianer in fact.

We'll address this in 12.0.0 and add some validation to prevent this confusion for users.

authors: craxal jinglouMSFT v-xianya

repo: microsoft/AzureStorageExplorer


text:
There is a successful activity log when pasting one blob container to emulator blob containers node which already has the same blob container

Storage Explorer Version: 1.23.0-dev
Build Number: 20220107.2
Branch: main
Platform/OS: Windows 10
How Found: Ad-hoc testing
Regression From: Not a regression
Steps to Reproduce

Install and run Azurite 3.14.3 -> Launch Storage Explorer.
Expand 'Local & Attached -> Storage Accounts -> (Emulator - Default Ports)(Key)'.
Blob Containers -> Create a blob container.
Right click the blob container -> Click 'Copy Blob Container'.
Right click Blob Containers node -> Click 'Paste Blob Container'.
Check whether there is a failed dialog indicating the blob container already exists.

Expected Experience
There is a failed activity log indicating the blob container already exists.
Actual Experience
There is a successful activity log.

Additional Context
This issue doesn't reproduce when creating a emulator blob container which already exists.

Since the paste is going through our old JS code, I'm going to put this in future for now. We should revisit this once emulator copy/paste for blob containers is using AzCopy code path.

Verified this issue on the build main/20220415.4. Fixed.

authors: MRayermannMSFT v-kellyluo


text:
Databricks workspace service UDR file missing entries for some regions.

The file
https://github.com/microsoft/AzureTRE/blob/main/templates/workspace_services/databricks/terraform/databricks-udr.json
Is missing entries for some regions, they need copying over from:
https://learn.microsoft.com/en-gb/azure/databricks/resources/supported-regions

updated eastus2 and created PR - https://github.com/microsoft/AzureTRE/pull/3568/commits/9efce6d7398b116e0aa7f967d1d334c8dae4a753

@marrobi is it still open?

Yes. Please feel free to add them.

authors: lordlinus marrobi nayeem01

repo: microsoft/AzureTRE


text:
Align airlock tags

Fixes #2121
What is being addressed
Some of the airlock resources weren't tagged correct which would impact the cost figures.
How is this addressed

Make sure all airlock resources have tags needed for cost figures
Remove data objects for things that could be passed between modules

/test

/test

/test

/test

authors: LizaShak tamirkamara


text:
LUIS Language Supports Error (Turkish)

LUIS service supports Turkish but it gives an error in the Composer.
https://docs.microsoft.com/en-us/azure/cognitive-services/luis/luis-language-support
Version

2.1.1

Browser

Chrome

OS

Windows

Screenshots

@cankanat, if you haven't already, can you please try passing in 'tr-tr' for the locale setting?

thank you @stevkan, it worked

authors: cankanat stevkan

repo: microsoft/BotFramework-Composer


text:
[Hackathon] Composer should support parallel instances

Is your feature request related to a problem? Please describe.
When developing bots, it's often helpful to have multiple bots open side by side so you can compare your own code to sample code, etc. We cannot do this in Composer without encountering unexpected behavior.
Describe the solution you'd like
The name of the bot you have open should be part of the path, so instead of http://localhost:3000/home it should be http://localhost:3000/ToDoBot/home for example. This could help in allowing different bots to be open in different browser tabs.

addressed in #2023
close this as composer can open parallel instances and #1330 addressed the remaining issues.

authors: hibrenda v-kydela


text:
feat: support app schema

Description
#minor
closes #4419
PVA prefers the naming convention app.*schema over the sdk.*schema.
This PR lets the project loader load 4 aliases of schema files:

app.schema -> sdk.schema
app.override.schema -> sdk.override.schema
app.uischema -> sdk.uischema
app.override.uischema -> sdk.override.uischema

The app.* schema files will take priority over sdk.* files.

Task Item

Screenshots

Coverage increased (+0.02%) to 55.732% when pulling 8e7fd5c0badf66bc363614005d1b7f5fc4da7fe5 on zeye/app-schema into 846d18f1d2c817b01e12271f553a4f3e1f2909a3 on main.


text:
Get version release about webchat.js using Javascript

Hi.
I download the file webchat.js from long time, should be 1 year.
We noticed that the browsers change some settings or they release new functions and yours probably have to fixed some things or add new functions.
I noticed all release in this page.
Probably I have this version [v4.15.7] but I am not right.
There is a way for get the version of file **webchat.js** using Javascript???
This file set the variable window.webChat then I ask if I can use the function for have the exact number version about webchat.js .
Like window.webchat.getVersionAboutThisFileBotFrameworkWebchat() (it is a example)
Best Regards

Paste the following code in the F12 console. It will list out all the versions of Web Chat and its dependencies.
[].map.call(document.head.querySelectorAll('meta[name^="botframework-"]'), function (meta) { return meta.outerHTML; }).join('\n')
This code snippet is documented in our bug template.

repo: microsoft/BotFramework-WebChat


text:
Address ipmitool CVE-2020-5208 (add comment for tooling)

Merge Checklist

All boxes should be checked before merging the PR (just tick any boxes which don't apply to this PR)

[x] The toolchain has been rebuilt successfully (or no changes were made to it)
[x] The toolchain/worker package manifests are up-to-date
[x] Any updated packages successfully build (or no packages were changed)
[x] All package sources are available
[x] cgmanifest files are up-to-date and sorted (./cgmanifest.json, ./toolkit/tools/cgmanifest.json, ./toolkit/scripts/toolchain/cgmanifest.json)
[x] All source files have up-to-date hashes in the *.signatures.json files
[x] sudo make go-tidy-all and sudo make go-test-coverage pass
[x] Documentation has been updated to match any changes to the build system
[ ] Ready to merge

Summary
CVE tooling was not detecting the patch for CVE-2020-5208, added a comment which it should be able to find.
Change Log

Add CVE comment to ipmitool so the tooling can find the patches

Does this affect the toolchain?

NO
Test Methodology

Local build of input-srpms

Tooling was recently updated to handle this case, work items are out of date. Abandoning the PR.

repo: microsoft/CBL-Mariner


text:
Adding the ccache and clamav packages.

Merge Checklist

All boxes should be checked before merging the PR (just tick any boxes which don't apply to this PR)

[X] The toolchain has been rebuilt successfully (or no changes were made to it)
[X] The toolchain/worker package manifests are up-to-date
[X] Any updated packages successfully build (or no packages were changed)
[x] All package sources are available
[x] cgmanifest files are up-to-date and sorted (./cgmanifest.json, ./toolkit/tools/cgmanifest.json, ./toolkit/scripts/toolchain/cgmanifest.json)
[X] All source files have up-to-date hashes in the *.signatures.json files
[X] sudo make go-tidy-all and sudo make go-test-coverage pass
[X] Documentation has been updated to match any changes to the build system
[x] Ready to merge

Summary

Adding the ccache and clamav packages as they're required by some of our partners.
Change Log

Added the ccache and clamav packages.

Does this affect the toolchain?

No.
Test Methodology

Local build.

fyi, looks like the new spec linter has some feedback: https://github.com/microsoft/CBL-Mariner/actions/runs/322486365
Haven't looked personally at them. @thcrain-msft Looks like the linter didn't flag an error even though it found changes.

authors: PawelWMS dmcilvaney


text:
[Issue] - Health Checker

Provide Version Number
Provide the version of the script that has the issue.
Describe the issue
A clear and concise description of the issue.
Expected behavior
A clear and concise description of what you expected to happen.
Script Output
If applicable, add the exception that wasn't handled.
Additional context
Add any other context about the problem here.

Closing issue as nothing is provided.

repo: microsoft/CSS-Exchange


text:
2022-10-27 BryanSoltis - v2.3.1 Updates

Removed Required attribute to Models/.ResourceType.cs ShortName property
Updated GitHub Action branch name trigger
Minor documentation grammar updates

Notification: @jocontr @dominicallen @EdPrice-MSFT

@microsoft-github-policy-service agree [company="Microsoft"]

2022-10-31 Updated Configuration page to export Admin configuration, if selected.

authors: BryanSoltis aznamingtool

repo: microsoft/CloudAdoptionFramework


text:
Change number of max_tokens from 1024

I am currently able to deploy, query, and shut down a model using the provided scripts.
However, unlike using DeepSpeed inference on its own, I am not able to figure out how to change the number of max generated tokens from 1024 to a different value.
I believe this is currently not supported, but I could be mistaken.
I believe the issue can be found with the code here:
https://github.com/microsoft/DeepSpeed-MII/blob/79b56af2b0cdbdd31845cecebfacc1b0090f9a97/mii/models/load_models.py#L73-L80
A value called max_tokens needs to be passed as an argument.
If I am correct, this should be a fairly simple fix.  I may create a PR for it if I can resolve it.

The fix is here. https://github.com/microsoft/DeepSpeed-MII/pull/129
Will close or support closing once this PR is merged.

repo: microsoft/DeepSpeed-MII


text:
Unable to execute mii-sd.py in benchmark

https://github.com/microsoft/DeepSpeed-MII/blob/main/examples/benchmark/txt2img/mii-sd.py
Hi, I encountered the following error when running the mi-sd.py file
AssertionError: text-to-image only supports ['Xibanya/sunset_city']
How to solve the problem?
Setup:
deepspeed==0.8.0
deepspeed-mii==0.0.4
diffusers==0.9.0

Hi @Yanxing-Shi this error often pops up if you haven't updated huggingface_hub or diffusers packages. Could you please update those and try again? Thanks!
pip3 install -U diffusers huggingface_hub

ok，@Einsum thank you！The problem is solved.

authors: Yanxing-Shi mrwyattii


text:
[BUG] memory overhead issue with optimizer leading to OOM

Could it be that unfused_optimizer is not careful at how it allocates memory when this condition occurs:
[2021-11-08 18:30:01,688] [INFO] [unfused_optimizer.py:275:_update_scale] Grad overflow on iteration: 2983
[2021-11-08 18:30:01,688] [INFO] [unfused_optimizer.py:276:_update_scale] Reducing dynamic loss scale from 65536.0 to 32768.0
[2021-11-08 18:30:01,689] [INFO] [unfused_optimizer.py:199:step] [deepspeed] fp16 dynamic loss scale overflow! Skipping step. Attempted loss scale: 65536.0, reducing to 32768.0

it tries to allocate more memory - a whooping 1GB!
    fp32_param.grad = fp16_param.grad.to(fp32_param.dtype)
RuntimeError: CUDA out of memory. Tried to allocate 1.00 GiB (GPU 1; 31.75 GiB total capacity; 24.28 GiB already allocated; 256.00 MiB free; 30.01 GiB reserved in total by PyTorch)RuntimeError
: CUDA out of memory. Tried to allocate 1.00 GiB (GPU 3; 31.75 GiB total capacity; 24.28 GiB already allocated; 276.00 MiB free; 30.01 GiB reserved in total by PyTorch)

I was running at 31 out of 32GB gpu memory used for many hours until the above occurred and it threw it into OOM.
Perhaps that logic needs to free memory it doesn't need any more first before allocating new memory?
e.g. this problem happened on a single SLURM job after running for 15h just fine.
This is with ZeRO-1 with Megatron-Deepspeed.
Thank you!
@tjruwase

Have you noticed the memory usage difference of recently release v0.5.6, compared to previous release?
After recent upgrade, I feel the memory usage of stage3 increased by a large amount.

This was ZeRO-1 only, so this is really the optimizer sharding so I can't comment on ZeRO-3.
Nevertheless I think it'd be a great idea to have some sort of regression test for memory usage for each of the ZeRO stages, if you'd like to work on that.
There are at least 4 possible groups of memory issues with any code modification (or the original code):

it uses more memory than before to do the same thing
a memory leak was introduced
it is not careful about how it allocates/frees tensors which leads to a temporary additional peak memory usage, which normally goes unnoticed, but if one was already on the brink of a 99% GPU memory usage, it OOMs.
memory fragmentation was created - typically this is not an issue with deepspeed, since it pre-allocates most of the buffers it uses and doesn't release those. But it's not always the case.

I suspect the 3rd case is what I'm dealing with at the moment.

@stas00, I did not realize this involved ZeRO-1. So basically, this is unfused optimizer + ZeRO-1? That configuration is not (well) tested :(. Do you have a quick repro?

Sorry, I should have written it out in the OP - I updated it now.
Unfortunately I don't have an easy way to reproduce it.
My suspicion is that the issue if any has been there all along, but as long as there was ~1GB of free memory it wasn't running into it, but since I dared to train on the brink of using all memory it happened.
So if it's something easy to see then we can improve it, and if it's very obscure then we would just know to leave enough free GPU memory for such situations. i.e. I won't ask to spend too much on it other than checking if perhaps it's something obvious and some tensor needs to be released on the target device before a new one gets copied over. For example it could be a very simple situation of:
a.cuda()
a = b.cuda()

If I'm not mistaken here, we have a temp situation where both a and b are cuda and there is a peak memory of size of b - i.e. 2x memory. So the no-peak memory solution is:
a.cuda()
del a
a = b.cuda()

does it make sense?

That makes sense. One would hope the pytorch garbage collector could help with this case.

pytorch doesn't have GC. It's all python's work. Once python frees a variable, pytorch automatically makes the memory available to new allocations (but it keeps it in its cache).
Pytorch can't be aware of how Python assigns to a variable. e.g. you can see here how even doing a+b+c creates a peak memory overhead: https://github.com/pytorch/pytorch/issues/27522#issuecomment-975041172
As I tried to briefly describe above Python doesn't necessarily free a variable when it goes out of scope or even when it gets deleted explicitly, since it depends on potentially other objects referencing to it. When the the reference count goes to 0 the object is freed. GC has a specific schedule when it performs its cycle when it identifies which objects can be freed. So at critical points one may have to call gc.collect explicitly in addition to del.  Though if the object is not referenced by any other object del should be sufficient to free memory.
In our situation of working with huge memory chunks, an explicit gc.collect call introduces no additional overhead that is of a significance.
There is an indepth guide on python GC if you're interested:

https://devguide.python.org/garbage_collector
and a few other resources:
http://cms.digi.com/resources/documentation/digidocs/90001537/#references/r_python_garbage_coll.htm
https://stackify.com/python-garbage-collection/

Sounds like this particular scenario might just be a corner case, but still needs to be confirmed. There might not be much we can do about it besides avoiding flying too close to the sun.

authors: stas00 tjruwase zarzen

repo: microsoft/DeepSpeed


text:
How to use torch.compile in DeepSpeed?

If I want to use the new feature of Pytorch2.0——torch.compile, what should I do? Where should I put the following code or just pass a command line parameter?
model = torch.compile(model)

@c-maxey Sorry, I still don't quite get it, can you show me a simple example？

@c-maxey Sorry, I still don't quite get it, can you show me a simple example？

Hey no problem.  Where your forward function is for your model, make it looks like the following:
@torch.compile
def forward(self,  )

@c-maxey Sorry, I still don't quite get it, can you show me a simple example？

Hey no problem. Where your forward function is for your model, make it looks like the following:
@torch.compile def forward(self, )

did this work? I use this way, but got a bug. I guess maybe the my cuda version is 11.4 which can't surpport torch2.0 compile?

authors: Cherryjingyao c-maxey noob-ctrl


text:
Add fp8-fused gemm kernel

This PR adds the new fused kernel for the Dense GeMM using fp8-quantized weight.

One thing that needs to be resolved before merging this, this kernel requires triton==2.3.0. This should be checked at runtime and communicated to users somehow.

Hi @jeffra. To clarify, does this kernel require exactly triton==2.3 or triton>=2.3?

Hi @jeffra. To clarify, does this kernel require exactly triton==2.3 or triton>=2.3?

@sfc-gh-reyazda would know better, I am not sure if we've tested with newer triton than 2.3.0. I have not personally tested this at least.

Hi @jeffra. To clarify, does this kernel require exactly triton==2.3 or triton>=2.3?

It needs that specific version, unfortunately triton keeps changing/improving and their APIs change too so it is hard to track it properly. That's also another motivation to move to cutlass soon and have a more solid implementation to work independent of other libraries. On the other hand, Triton gives the flexibility to run on various hardwares. So, it is always a tradeoff. I think we need to have some more discussions on such dependencies later in a different discussion.
Best,
Reza

authors: HeyangQin jeffra sfc-gh-reyazda


text:
Option: “--skip-git-ignored-files” causes fatal errors in logs

Option: “--skip-git-ignored-files” causes fatal errors in logs
Hi,
When running devskim with the --skip-git-ignored-files option, multiple errors are logged before the SARIF report is generated. The error messages seem to vary slightly depending on the environment where devskim is executed. Additionally, the error messages appear to be truncated, preventing a complete understanding of the issue.
To Reproduce:

Run devskim in a pipeline or locally with the --skip-git-ignored-files option.
Observe the errors in the logs.

Expected behavior:
When running devskim with the --skip-git-ignored-files option, no errors should be logged, and files specified in the .gitignore should be ignored during the analysis.
Screenshots:
The example screenshots are taken from a number of different repositories experiencing the same similar errors.
Github Actions

Azure Devops CI pipeline

Gitlab Job

Locally

Versions:

OS:
Docker image: python:3.12.4-alpine3.19
Locally: Windows 11 23H2

Devskim:
v1.0.33

Git:
Docker image: 2.43.4
Locally: 2.45.2.windows.1

Additional Information:

Removing the --skip-git-ignored-files option resolves the errors.
The errors flood the logs, making it difficult to identify other relevant information.
Once resolved, the --skip-git-ignored-files option should function as expected, ignoring files specified in the .gitignore without generating any errors.

Thanks for your time,
Tom

Thanks for the report. It looks to me like this error message indicates usage of this flag when scanning files that are not tracked by a Git work tree. The error message in particular here is not produced by DevSkim itself but is, I believe, coming from Git itself when we call out to git to check the ignore status.
I'll try to take a look to see if I can squelch the errors from appearing to the user (since they aren't actionable anyway) either by checking that a git repo exists before checking the ignore status, but it may also work to just redirect stderr so it doesn't present to the DevSkim user.
It's not clear to me from this report if this actually effects the behavior of DevSkim itself or the output report, so I'm working off the assumption that this is just an issue with the spurious error messages appearing about checking file status for files not in a work tree. But if that's incorrect, please let me know. I realize the error says fatal, but it shouldn't be from the perspective of DevSkim, its Fatal from the perspective of Git trying to check the ignore status.

Hello @gfs,
Thanks for getting back to me. It sounds like you have a plan!
I'm guessing you haven't seen this issue before? I've tried multiple different GitHub and Azure-hosted Git repos and was consistently seeing it both locally and in CI pipelines!
I'd need to confirm if the issue still occurs even when there's no actual error found. However, it was often showing up 10's to 100's of times in the output, depending (I guess) on the size of the repo. So I think cleaning it up would be a good idea regardless.
Thanks for your time,
Tom

repo: microsoft/DevSkim


text:
is SDK cache persisted?

Using the below code to cache the feature flag value in our console app
// To connect with feature flag CDN edge. options.Connect(this.oktaVerifyFeatureProviderConfiguration.RemoteConfigurationConnectionString); options.UseFeatureFlags(configure     => configure.CacheExpirationInterval         = TimeSpan.FromMinutes(this.oktaVerifyFeatureProviderConfiguration.RemoteConfigurationCacheExpiration));
when I restart the app, the value does not persist in SDK at all instead it makes a call to the server to download the configuration. is it expected?

Hi @rossgrambo ,
Good Morning.
do you have any update the on above question? like I would like to persist the value locally so that when the client WPF app close and opens again it retains the value otherwise before I close the app feature flag was on and after reopens it's on by default unless the network calls made and get the latest value.
Thanks,
Sathishkumar

@sathishkumarkaliavaradhan
There is no local (disk) persisted cache. Restarting the application means the library would need to contact the server to pull feature flag values.

repo: microsoft/FeatureManagement-Dotnet


text:
Some upcoming notices about logging types

Description

Add an "Upcoming" changeset note to advertised planned changes to a few APIs using the deprecated ITelemetryLogger type.
Mark a bunch of internal logging-related stuff as @internal

Breaking Changes
No, but announces some that will be done after internal.7.0 releases (see AB#5448)
Reviewer Guidance
Question: Is there a defined procedure for transitioning an API from public to internal, in terms of announcing / annotating the deprecation for public consumption?

@tylerbutler  / @CraigMacomber  - Tagging as release-blocking as well.  I thought I had already merged it!  Will unblock some feature work post-internal.7.0 release.

repo: microsoft/FluidFramework


text:
System.ExecutionEngineException   HResult=0x80131506   Message=Exception of type 'System.ExecutionEngineException' was thrown.

@yatli Please take a peek at this exception. Whenever I pass data in a method call I am getting an execution error; if  have a method defined as TSL protocol "Syn, void, void" that works. I've uploaded the project so that you can look at the project yourself; here is the link to my repro: https://github.com/TaviTruman/GraphEngineTripleStoreMemoryCloud

System.ExecutionEngineException
HResult=0x80131506
Message=Exception of type 'System.ExecutionEngineException' was thrown.

@yatli UPDATE: I think there is something wrong in the actual Service Fabric runtime as even the Trinity.ServiceFabric.SampleApplication sample is failing like this:

on the "m_trinity.Ping()" call.

@yatli I am working from the "Trinity.ServiceFabric.SampleApplication", Trinity.ServiceFabric.GraphEngineRemotingClient and Trinity.ServiceFabric.GraphEngineService.  I have tried a number of project configurations in an attempt to duplicate expected behavior and I've had some difficulty thus far.
The Trinity-GraphEngine Service Fabric implementation is mostly undocumented; however, the sample code and library source code is the best working examples of how to proceed. I have the time to dig into the code, and I'm fine with that, but others will not attempt to get started without additional documentation, working beyond what is supplied.
I am having problems getting coded workflows to generated expected results: no problem with storing and accessing data, but I am having trouble getting Trinity MessagePassing to work as expected. Also, as long as I use the Sample GE/SF suppled code, things seem to work as expected; however, when I create a new Service Fabric app and SF Services from scratch, then Trinity Message passing does not work as expected.
I am getting errors like this:

So I want to be very clear here; as long as I extend and modify the supplied samples, things seem to work well . I can add new productions to TSL and everything seems to work.

Passing simple data types over Sync and Async defined protocol definitions
Passing complex data types over Sync and Async defined protocol definitions

If, however, I create new projects (GE+Service Fabric), things don't work as expected; I have tied the following project configurations:

SF targeting .net461. netstandard20
SF targeting .net core 3.0

In order to create a working reference programming model, not based on supplied sample code,  I've written a few sample programs that use the TrinityClient API, Trinity.Network and Trinity.Config APIs - please note, these aren't Service Fabric instrumented programs.
I'm following coded workflows in the samples found in the GraphEngine.Client solution and those samples found in the "Samples" repro directory, I have been able to cerate programs that can send simple and complex data types; it all works as expected. I did this to convience myself the technology works and to gain  confidence as well.
I'm certain that I am doing something wrong, as this codebase has been around a long time, and it's  used by Microsoft commmerically and on many noteworthy technolgies.
Here is the error encounted when calling a generated RPC method passing a complex data type over a Sync defiend interface:

Here's one more bit of data and information:

{
"Timestamp": "2019-10-14T17:27:28.7587683-07:00",
"ProviderName": "Microsoft-ServiceFabric.Trinity.StatefulService",
"Id": 3,
"Message": "[P0-Wallflower]\t   at Trinity.Client.TrinityClientModule.TrinityClientModule.b__30_0(Int32 _)
at System.Collections.Concurrent.ConcurrentDictionary2.AddOrUpdate(TKey key, Func2 addValueFactory, Func3 updateValueFactory)        at Trinity.Client.TrinityClientModule.TrinityClientModule.RegisterClientHandler(RegisterClientRequestReader request, RegisterClientResponseWriter response)        at Trinity.Client.TrinityClientModuleBase._RegisterClientHandler(SynReqRspArgs args)        at Trinity.Network.Messaging.SynReqRspArgs.MessageProcess()",   "ProcessId": 57256,   "Level": "Error",   "Keywords": "0x0000F00000000001",   "EventName": "GraphEngineLogErr",   "ActivityID": null,   "RelatedActivityID": null,   "Payload": {     "message": "[P0-Wallflower]\t   at Trinity.Client.TrinityClientModule.TrinityClientModule.<RegisterClientHandler>b__30_0(Int32 _)          at System.Collections.Concurrent.ConcurrentDictionary2.AddOrUpdate(TKey key, Func2 addValueFactory, Func3 updateValueFactory)
at Trinity.Client.TrinityClientModule.TrinityClientModule.RegisterClientHandler(RegisterClientRequestReader request, RegisterClientResponseWriter response)
at Trinity.Client.TrinityClientModuleBase._RegisterClientHandler(SynReqRspArgs args)
at Trinity.Network.Messaging.SynReqRspArgs.MessageProcess()"
}
}

I'm going to close this issue because I have been able to solve part of the problem in part; I do feel there is something wrong with the behavior of the GE when working in the Service Fabric environment. As I don't have any working examples of anything more than just very simple functioning this is a taxing discovery task and process. I have a better working understanding of the Trinity API set the documentation is insufficient, but the working code is an available resource to learn from.
The Trinity Graph Engine is a very powerful technology and it supports many network programming models, powerful techniques, and amazing graph data management facilities; however, again, the existing documentation is too thin in my opinion. I'm also closing this issue because it documents a serious error and it has not received any support attention from the team and I feel like I need to learn and know more and it does not feel right to me to air the problem quite yet.
I'm able to improve the stability of my program's intended behavior as I learn more about the Trinity/GE technology in the Service Fabric environment - so the problems I'm running into is a result of my lack of knowledge and not the fault of the Graph Engine.

repo: microsoft/GraphEngine


text:
Padding in case of different number of nodes in batch

Hi,
I have a few questions about the node padding.
Firstly, is my assumption correct, that the adding of -inf values in "pad_attn_bias_unsqueeze" has the same purpose as the attention_mask in BERT, so that there will be no attention to padded nodes?
If this is correct, why do you add +1 to x in the padding functions? As the attention is restricted not to attend there anyway, there could be any values in the padded nodes, so 0 could still be just as a regular feature value.
I talk about the padding like in
def pad_2d_unsqueeze(x, padlen):
    x = x + 1  # pad id = 0 -> THIS LINE
    xlen, xdim = x.size()
    if xlen < padlen:
        new_x = x.new_zeros([padlen, xdim], dtype=x.dtype)
        new_x[:xlen, :] = x
        x = new_x
    return x.unsqueeze(0)

which is used to pad x.

Thanks for using Graphormer. Yes, we're used to use zero to represent padding nodes.

Yes, padded nodes are not attended, therefore you can assign to arbitrary category.

Thanks

authors: ChantalMP zhengsx

repo: microsoft/Graphormer


text:
Translation - Chinese

Translate the curriculum in to Chinese. Use this issue for all discussions during this work.

How do we deal with different zhs like zh-CN, zh-TW...?

If it helps, just as a note, right now I am translating to zh-cn :)

Good question. Adding @jlooper who has supported the translations in other repos.

hi, there should be one file for Simplified Chinese (README.zh-cn.md) and one for Traditional Chinese (README.zh-tw.md). Thank you!

Thank you, I'll take note and send in PRs to change the file names of the files I've translated from zh only to zh-cn.

I'll try translating the quiz-app. I think I can finish it in 10 days.

Hi all, I'm happy to contribute to this project, do we have any instructions on how?

Hey @max-sun-hongyi! Thanks for offering to help.
You can find some translation instructions here: https://github.com/microsoft/IoT-For-Beginners/blob/main/TRANSLATIONS.md
Can you coordinate with the rest of the folks here as to what files you'd like to translate. Fork the repo, translate, and once you are done, raise a PR and request a review from @Mireille-T. Once approved we'll get it merged.

Hi @max-sun-hongyi , thanks so much for your help!
Currently I've translated the main README.md as well as all but pi.md in 1-introduction-to-iot, which I am working on now. However, I understand there are a couple of mistakes in those translations; since Chinese is my second language, whilst the overall structure is present in the translations, the phrasing can be improved. If possible, I'd appreciate it if you would be willing to help me correct those, though if you want to work on the translation of another file, that's perfectly fine as well. Perhaps it would be nice if you could indicate it here so that there's no overlap in the work each of us are doing.
Thanks again! 😄

See https://github.com/microsoft/IoT-For-Beginners/issues/230

I'll try translating the quiz-app. I think I can finish it in 10 days.

Hi， when you finish it, can you give a message and url? thans. You do a good job

I'll try translating the quiz-app. I think I can finish it in 10 days.

Hi， when you finish it, can you give a message and url?
Thanks. You do a good job

@stevenwsg  #251 is completed. After it is merged, you can use the app in https://brave-island-0b7c7f50f.azurestaticapps.net/

I think we can follow this guidelines.

How do we deal with different zhs like zh-CN, zh-TW...?

I am glad to contribute  to this project.What is the progress of this?

How do we deal with different zhs like zh-CN, zh-TW...?

I am glad to contribute to this project.What is the progress of this?

Hello, currently we have files for zh-cn (Simplified Chinese), but not zh-tw (Traditional Chinese) in the translation folders.

How do we deal with different zhs like zh-CN, zh-TW...?

I am glad to contribute to this project.What is the progress of this?

Hello, currently we have files for zh-cn (Simplified Chinese), but not zh-tw (Traditional Chinese) in the translation folders.

OK. And how to know which  is not translated in a directly way?It's not effeciently to click to folder and check which  have not done.

@ishowman I think the easiest way is to still check the translations folder, though from what I remember, the content after Lesson 1 (other than the quizzes) has yet to be translated.
@jimbobbennett May I ask if there a more direct way of checking, as per the above query? Thank you!

I've submit a pr，will it auto assign a reviewer?

I've submit a pr，will it auto assign a reviewer?

Hello @ishowman, and thank you very much for the translation! I do not think the creation of a PR itself will automatically assign a reviewer, but you can either tag me @Mireille-T within the PR (preferred) or send a message here so I will be notified, then I can proceed to review the PR when I have the time. :smile:

Got here from the hacktoberfest-friendly tag. I would like to help translate the curriculum into zh-tw as I'm a native traditional Chinese speaker.

Happy to help with this translation, is there any task-list or I just pick up files that haven't being translated?

No task list - thought that would be a great addition to this issue! Maybe a good first step is to create a task list marking which ones have been done, then note here which ones you want to work on.

@JackLeeHal - one thing you could do to help I review https://github.com/microsoft/IoT-For-Beginners/pull/439. I don't read or speak Chinese, so would appreciate another set of eyes over this translation.

authors: ChungZH JackLeeHal Mireille-T ishowman jimbobbennett jks-liu jlooper max-sun-hongyi stevenwsg wamos wilsonmar

repo: microsoft/IoT-For-Beginners


text:
Feature Request: customizable early_stopping_tolerance

I have a situation where the default numerical tolerance (0.001) for early stopping is too large. My target has a gamma distribution and the LGB Regressor reaches convergence too early, when the numerous low target values are well approximated by the model, but the few large values are still underestimated. When I deactivate early stopping, I can see the loss metric still improving at the 4th or more decimal digit, past the best iteration reached during early stopping. It would be great to be able to set manually the tolerance.

@kryptonite0 Can you please provide a reproducible example or training logs at least?
As I know, we do not have any default numerical tolerance. Refer to:
https://github.com/microsoft/LightGBM/blob/fc991c9d7e07fd8e460ea24d1f5609f61fd2af3f/python-package/lightgbm/callback.py#L227
https://docs.python.org/3/library/operator.html#operator.lt
Linking https://github.com/dmlc/xgboost/issues/4982 here.

Ping @kryptonite0
As you can see from examples, there is no "default numerical tolerance (0.001)":
https://github.com/microsoft/LightGBM/blob/785e477856b9e9e9e69123caf175dbb8455c3b11/examples/python-guide/advanced_example.py#L52-L59
[1]	training's binary_logloss: 0.680894
[2]	training's binary_logloss: 0.672151
[3]	training's binary_logloss: 0.664753
[4]	training's binary_logloss: 0.656185
[5]	training's binary_logloss: 0.648174
[6]	training's binary_logloss: 0.641671
[7]	training's binary_logloss: 0.635597
[8]	training's binary_logloss: 0.628874
[9]	training's binary_logloss: 0.622432
[10]	training's binary_logloss: 0.616403
Finished first 10 rounds...

At present we do not have any "default numerical tolerance". But having customizable early stopping tolerance might be useful in some cases.

Closed in favor of being in #2302. We decided to keep all feature requests in one place.
Welcome to contribute this feature! Please re-open this issue (or post a comment if you are not a topic starter) if you are actively working on implementing this feature.

#4580 implemented this feature request for Python-package. Thank you very much @jmoralez !
I think we should reuse this issue as a feature request for adding the same functionality into R-package and core cpp code to not split the discussion. Refer to https://github.com/microsoft/LightGBM/pull/4580#issuecomment-910714636. So, I'm not excluding this issue from #2302 for now.

repo: microsoft/LightGBM

license: mit

license_source: bigquery


text:
Loss in performance by using init_score

Description
I need to use the init_score to provide a prior model but I'm seeing some behaviour I don't understand.
Setup

build a cv model with 20 trees

build a second cv model with 80 trees which uses the same folds as the first model, and feeds in cv predictions from first model as init_score

I have early stopping to avoid overfitting (second model stops at 2 trees)

first model alone has performance (cv metric) 2197.57

resulting combined performance is 2194.64

repeat the above but the first model has 10 trees and second has 90 trees

second model ends on 15 trees

first model alone has performance 2483.26

resulting combined performance is 2205.03

Query
I would have expected the two situations to give similar performance (due to using the same folds for both models, this setup should be the same as creating one model with 100 trees in both situations) . But I have seen over multiple examples that limiting the trees in the first model will lead to worse results. Is there any reason or intuition for this being the case?
Reproducible example
import lightgbm as lgb
import pandas as pd
import numpy as np
import sklearn.model_selection as skms

# RUN SCRIPT SECOND TIME BUT CHANGING num_iters TO 10
total_iters = 100
num_iters = 20

# create data
np.random.seed(5)
data = pd.DataFrame({
    "a": np.random.random(10_000),
    "b": np.random.random(10_000),
    "c": np.random.random(10_000),
    "d": np.random.random(10_000),
})
data["target"] = np.exp(5 + 3 * data["a"] + data["b"] - 2 * data["c"] + 1.5 * data["d"] + np.random.gamma(0.1, 1, 10_000))

# build first cv model
dataset = lgb.Dataset(
    data=data.drop(["target"], axis=1),
    label=data["target"],
    free_raw_data=False,
)
kf = skms.KFold(n_splits=3, shuffle=True, random_state=309)
kf_splits = kf.split(np.zeros(len(data)))

custom_folds = list()
for train_idx, test_idx in kf_splits:
    custom_folds.append((train_idx, test_idx))

cv_results = lgb.cv(
    params={
        "objective": "gamma",
        "boosting_type": "gbdt",
        "n_estimators": num_iters,
        "early_stopping": 5,
        "metric": "gamma_deviance",
    },
    train_set=dataset,
    folds=custom_folds,
    stratified=False,
    return_cvbooster=True,
)

# need cv preds to feed into second model - check my cv preds give same metric as lightgbm
print(cv_results["valid gamma_deviance-mean"])
def replicate_metrics(num_iters, model):
    list_metrics = []
    cv_preds = []
    for num_iter in range(1, num_iters + 1):
        metric_list = []
        for cv_idx, cv_fold in enumerate(custom_folds):
            mdl_temp = model.boosters[cv_idx]

            # predict from booster
            cv_preds_tmp = mdl_temp.predict(
                dataset.get_data().loc[cv_fold[1]],
                num_iteration=num_iter,
            )

            tmp = data["target"].loc[cv_fold[1]] / (cv_preds_tmp + 1.0e-9)

            metric_list.append(
                    2 * sum(tmp - np.log(tmp) - 1)
            )
            if num_iter == num_iters:
                cv_preds.append(cv_preds_tmp)
        list_metrics.append(np.mean(metric_list))

    cv_preds = (
        pd.DataFrame(
            {
                "idx": np.concatenate([idx[1] for idx in custom_folds]),
                "cv_pred": np.concatenate(
                    cv_preds
                ),
            }
        )
        .sort_values(by=["idx"])
        .reset_index(drop=True)
        .pop("cv_pred")
    )

    print(list_metrics)

    return cv_preds

cv_preds = replicate_metrics(len(cv_results["valid gamma_deviance-mean"]), cv_results["cvbooster"])

# second model
dataset2 = lgb.Dataset(
    data=data.drop(["target"], axis=1),
    label=data["target"],
    free_raw_data=False,
    init_score=np.log(cv_preds),
)

cv_results2 = lgb.cv(
    params={
        "objective": "gamma",
        "boosting_type": "gbdt",
        "n_estimators": total_iters - num_iters,
        "early_stopping": 5,
        "metric": "gamma_deviance",
    },
    train_set=dataset2,
    folds=custom_folds,
    stratified=False,
    return_cvbooster=True,
)
print(cv_results2["valid gamma_deviance-mean"])

Environment info
Package versions:
LightGBM: 4.5.0
numpy: 1.22.3
pandas: 1.4.1
sklearn: 1.1.1
Command(s) you used to install LightGBM
python -m venv venv
venv\Scripts\activate.ps1
python -m pip install -r requirements.txt

I think fundamentally, my initial thoughts were wrong as the init_score I'm feeding in is essentially "test" predictions. That leads to the question - is it possible to feed in different init_score for each fold?


text:
[BUG] Sending messages without monitoring causes Green Screen with the USB MIDI 2.0 driver

Describe the bug
Sending a lots of messages without monitoring causes Green Screen.
This issue only occurs if the device is running with the USB MIDI 2.0 driver (UsbMidi2.sys)
To Reproduce

Update ProtoZOA with the attached firmware.
Attach ProtoZOA to PC and update the driver to the USB MIDI 2.0 driver (UsbMidi2.sys).
Open console and make sure there is no monitoring command (midi endpoint monitor) running.
Run "midi endpoint send-message 0x40123456 0x00000000 -p 0 -c 125000 -i".
Choose "ZOA-216" as an endpoint.
Repeat step 4 to 5 several times.

Green Screen occurs.
I faced two types of Green Screen, "ATTEMPTED_SWITCH_FROM_DPC" and "KERNEL_SECURITY_CHECK_FAILURE".
Expected behavior
No Green Screen.
Installer Name or Version

Service: Windows.MIDI.Services.Developer.Preview.5.x64.1.0.24066.2126.exe
Driver: Updated_USBMIDI2_10.0.1.4.zip

Desktop (please complete the following information):

OS: Windows 11
OS Build: 26100.1

Device information, if this is with an external MIDI device:

Device: ProtoZOA w/attached firmware.
ZOA-216.uf2.zip

https://www.windowslatest.com/2024/04/06/windows-11-24h2-build-26100-causes-undocumented-issues-blocks-some-apps/
FYI  26001 is reported to be buggy.. I can't even download it anymore ATM.

Finally installed 26100.1 (after using a network cable and several retires)
Ran this command with Preview 5 midi driver on 26100 (with DYI MIDI 2.0 ARM hardware) for many times without issues.
I have an RP2040 ut no protozoa. Will try that FW

Loaded that FW in an off the shelf RP2040 board and ran that command many times.  it sens the data, Cannot reproduce the GSOD.

Got two crashes now of Windows 11.  (auto reboots)... but not when running the command. need further check.

Now fully reprodceuable. indeed an GSOD

Thank you for the repro and the crash dump, @MusicMaker.

https://www.windowslatest.com/2024/04/06/windows-11-24h2-build-26100-causes-undocumented-issues-blocks-some-apps/ FYI 26001 is reported to be buggy with several GSODs.. I can't even do Windows update to it anymore ATM. (stuck at an older version which by itself is also a known issue and suggests a clean install)

If you want to get off the Canary train, I was told Dev channel has the ACX / USB updates in it now, so USB MIDI 2 will work on it.

Indeed. noticed the Dev channel works too.

Needs further investigation and analysis.
@garydan42 thinking about it, the act of midi endpoint send-message will put driver into "run" state which will process the USB IN buffer and into double buffer. As there is no monitor or application, nothing will be feeding from double buffer. In code, the reaction is to drop data but then request to handle existing data.
We should actually respond with discarding oldest data and then putting new data in buffer. What is the design pattern for this?
However, does not explain GSOD - just noting what we should be doing.

@garydan42 This issue is due to the "Run" state being set when a send-message occurs but nothing is reading from driver. If the device has data provided and the driver is in run state, driver will populate the double buffer stream for UMP IN with UMP data. If there is nothing reading from the UMP Data from that stream, it will eventually fill.
I suspect we should just drop old data - but what is design pattern here? How do I ensure I am not conflicting on memory with other software that may be accessing stream? Is it same as when dealing with buffer for write?

The pin instance shouldn't be in the run state unless someone is retrieving buffers from the cross-process queue. The code that instantiates the cross-process buffer also creates the worker threads that does the reads and callbacks. The service should be retrieving these messages, and if there are no clients listening, dropping them.
That's not to say that the cross-process queue can't become full because messages are arriving faster than they can be processed or the callback blocks or hangs. The expectation is that the newest messages are dropped in that case, not that the oldest are dropped. Architecturally there's no way to drop the oldest messages because it's a producer/consumer fifo with the reader owning removing messages from the front of the fifo and the writer adding messages to the end. To drop the oldest messages the current read position would have to be advanced by the writer thread, but the reader may be actively reading at the current read position and modifying it would lead to data corruption.
Sent from Outlookhttp://aka.ms/weboutlook

From: Michael Loh @.>
Sent: Monday, June 24, 2024 5:31 PM
To: microsoft/MIDI @.>
Cc: Mention @.>; Assign @.>
Subject: Re: [microsoft/MIDI] [BUG] Sending messages without monitoring causes Green Screen with the USB MIDI 2.0 driver (Issue #307)
@garydan42https://github.com/garydan42 This issue is due to the "Run" state being set when a send-message occurs but nothing is reading from driver. If the device has data provided and the driver is in run state, driver will populate the double buffer stream for UMP IN with UMP data. If there is nothing reading from the UMP Data from that stream, it will eventually fill.
I suspect we should just drop old data - but what is design pattern here? How do I ensure I am not conflicting on memory with other software that may be accessing stream? Is it same as when dealing with buffer for write?
—
Reply to this email directly, view it on GitHubhttps://github.com/microsoft/MIDI/issues/307#issuecomment-2187443202 or unsubscribehttps://github.com/notifications/unsubscribe-auth/AOXSFVA77GNRVMTM37KQLBLZJCF3VBFKMF2HI4TJMJ2XIZLTTGBKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDUOJ2WLJDOMFWWLLTXMF2GG2C7MFRXI2LWNF2HTAVFOZQWY5LFUVUXG43VMWSG4YLNMWVXI2DSMVQWIX3UPFYGLAVFOZQWY5LFVI2DCNRUGYZDIMBVHCSG4YLNMWUWQYLTL5WGCYTFNSBKK5TBNR2WLKRUGE4DAMBQGQZDQM5ENZQW2ZNJNBQXGX3MMFRGK3ECUV3GC3DVMWVDKNRTGU2DGNJRGA3KI3TBNVS2S2DBONPWYYLCMVWIFJLWMFWHKZNKGU3DIOJSGYYTMMBYURXGC3LFVFUGC427NRQWEZLMQKSXMYLMOVS2UNRYGMZDAMJUHA2DRJDOMFWWLKLIMFZV63DBMJSWZAVFOZQWY5LFVI3DSOBVG44DQMRQGOSG4YLNMWUWQYLTL5WGCYTFNSWHG5LCNJSWG5C7OR4XAZNMJFZXG5LFINXW23LFNZ2KM5DPOBUWG44YQKSHI6LQMWVHEZLQN5ZWS5DPOJ42K5TBNR2WLKJUHE2TSNJZGQ3TDAVEOR4XAZNFNFZXG5LFUV3GC3DVMWVDEMRTG4ZDGNBTHE2YFJDUPFYGLJLMMFRGK3FFOZQWY5LFVI2DCNRUGYZDIMBVHCBKI5DZOBS2K3DBMJSWZJLWMFWHKZNKGQYTQMBQGA2DEOBTQKSHI6LQMWSWYYLCMVWKK5TBNR2WLKRVGYZTKNBTGUYTANUCUR2HS4DFUVWGCYTFNSSXMYLMOVS2UNJWGQ4TENRRGYYDRAVEOR4XAZNFNRQWEZLMUV3GC3DVMWVDMOBTGIYDCNBYGQ4IFJDUPFYGLJLMMFRGK3FFOZQWY5LFVI3DSOBVG44DQMRQGOTXI4TJM5TWK4VGMNZGKYLUMU.
You are receiving this email because you were mentioned.
Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

I have confirmed that the issue has been solved.

Service: Windows.MIDI.Services.In-Box.Service.-.Developer.Preview.6.1.0.24194.2233-x64.exe
Driver: [interim preview build] 240713-USBMidi2-x86-TESTSIGNED.zip (DriverVer = 07/13/2024,23.1.50.504)
OS: Windows 11 24H2 (OS Build 26100.1150)

That's not to say that the cross-process queue can't become full because messages are arriving faster than they can be processed or the callback blocks or hangs. The expectation is that the newest messages are dropped in that case,
This makes most sense, but could that not lead to a stall or a new reader process retrieving  a lot of old messages ? another way could be to reset the queue when the queue overflows, but then nothing should be reading it, otherwise it could end up reading a non valid UMP message and it needs to handle to skip over corrupt data.

@garydan42 I am thinking that the correct response here should be that on processing a continuous reader data input, if it looks like another message could fill cross-process queue to full, should stop continuous reader - this should cause USB stack to start to queue and use its own flow control. Then some event (maybe reads from cross-process queue check to see if continuous reader needs to be restarted) would start the continuous reader once enough space exists for servicing a max size USB message.
This of course would take a bit of rework to driver - and should be made a feature request.

@garydan42 @Psychlist1972 I feel that this original issue can be closed and that a new one indicating dropping USB data in large transfers should be created. Green screens no longer occurring I believe but fix may have created a new issue.

Closing as fixed in the August 2024 driver release. If you find it not to be when testing, will reopen.

authors: AmeNote-Michael MusicMaker Psychlist1972 garydan42 m-komo symdeb

repo: microsoft/MIDI


text:
Misleading tutorial for next-auth integration with ms graph lib

I have followed nextjs auth tutorial , which title states there is integration with @microsoft/microsoft-graph-client package. I think this is wrong and misleading, because actually tutorial is all about next-auth integration with azure-ad, and there is no dependency code on MS Graphs library and no dependency on MS Teams (which is stated at the beginning of this tutorial).
I googled this tutorial, because I was looking how to integrate MS Teams with Nextjs (next-auth).
There is big chance, I am wrong somewhere, I would appreciate some clarification then.

Hi @ricardasjak. How are you? You can see the usage of  @microsoft/microsoft-graph-client in the second tutorial (Reminder App). I decided to divide it into two parts, because this tutorial is more dedicated to students and not for Pro-Developers. Therefore, to help the development of beginners people, I decided to divide the tutorial into 2 parts.
I may be changing the title to Part 01 and Part 02. I think it would actually avoid confusion in the textual interpretation of the developed workshops.
You can follow usage of @microsoft/microsoft-graph-client in that file here:
https://github.com/glaucia86/msgraph-nextjs-workshops/blob/main/demo-02/reminder-app/pages/api/getPresence.ts
I apologize in advance for the confusion in the title. However, it is not misleading. The use of the word sounded very strong to me here in this context.
If you need any help with how to make use of the package, you can contact me anytime I call you via Discord.
Yours sincerely,
Glaucia Lemos

I may be changing the title to Part 01 and Part 02. I think it would actually avoid confusion in the textual interpretation of the developed workshops.
You can follow usage of @microsoft/microsoft-graph-client in that file here:
https://github.com/glaucia86/msgraph-nextjs-workshops/blob/main/demo-02/reminder-app/pages/api/getPresence.ts

Hello, thank you very much for your response. I am sorry for my direct style, I am not English speaker in the end.
Given example seems about right, I will take closer look another day.
Consider this issue resolved.

Thank you for your feedback @ricardasjak! No problem :)
Reading your feedback and the title can cause some confusion. I will open a PR following up your advice! Thank you for your contribution about it! We really appreciate it!

repo: microsoft/MicrosoftCloud


text:
Add Leap and Tap to Place to Release Notes

Overview
Added Leap Motion support to the top of the release notes with a new gif and added Tap to Place.
Changes

Fixes: #7798

/azp run mrtk_docs

repo: microsoft/MixedRealityToolkit-Unity


text:
Investigate Server URL info changing/getting lost during copying when filtering an OpenAPI Document

Reference: https://github.com/microsoft/OpenAPI.NET/issues/1799#issuecomment-2318308032

Closing this for now to get confirmation for https://github.com/microsoft/OpenAPI.NET/issues/1799#issuecomment-2320843019


text:
Incorrect colorization after using triple quoted string

I added triple quotes around code, and colorization is incorrect. Reopening the file fixes it.

May be related to https://github.com/Microsoft/PTVS/issues/4236

I retested this with latest from master to see if today's fixes took care of it, but no, not quite.

Also reported at https://developercommunity.visualstudio.com/content/problem/275331/python-triple-quotes.html

This is still an issue in VS2019 version 16.2.1
It seems that when you use triple quotes, the following line is in red (as it's supposed to be) but all subsequent lines after that, within the quotes, aren't.

authors: bedroomlab huguesv

repo: microsoft/PTVS


text:
Python Azure function support

This issue has been moved from a ticket on Developer Community.

Currently only VS Code supports creating Azure Functions in Python; Visual Studio 2022 supports only C#.
It would be really nice if we had Python Azure Function support in VS as well.

Original Comments
Feedback Bot on 5/7/2023, 09:15 PM:
(private comment, text removed)

Moving this issue to discussion as an enhancement request for comments and upvotes.


text:
New-PartnerAccessToken (or any other cmdlets of the module) are recognized

Steps to reproduce
Install the module on Powershell Core 6.2.0-preview.3
Install the module version 1.5.1902.5
run cmdlet
get-command New-PartnerAccessToken -Module PartnerCenter.NetCore

Expected behavior
Should output
CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Cmdlet          New-PartnerAccessToken                             1.5.1902.5 PartnerCenter.NetCore
``

## Actual behavior

Outputs:
```ps
get-command : The term 'New-PartnerCustomerUser' is not recognized as the name of a cmdlet, function, script file, or operable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ get-command New-PartnerCustomerUser -Module PartnerCenter.NetCore
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : ObjectNotFound: (New-PartnerCustomerUser:String) [Get-Command], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException,Microsoft.PowerShell.Commands.GetCommandCommand

Diagnostic logs
Will be shared if needed
Environment
get-command : The term 'New-PartnerCustomerUser' is not recognized as the name of a cmdlet, function, script file, or operable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ get-command New-PartnerCustomerUser -Module PartnerCenter.NetCore
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : ObjectNotFound: (New-PartnerCustomerUser:String) [Get-Command], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException,Microsoft.PowerShell.Commands.GetCommandCommand

This started after installation of:
Security Update  KB4489899     NT AUTHORITY\SYSTEM  3/21/2019 12:00:00 AM

The dll is locking somehow:

Hi @Agazoth,
If the problem presist after restarting I would recommend updating to the release candidate for PowerShell 6.2opening an issue using the PowerShell issue tracker.

@Agazoth if you are still having issues please let us know.

authors: Agazoth isaiahwilliams

repo: microsoft/Partner-Center-PowerShell


text:
Broken links for In-person Instructor Led pages

Many of the links to "In-person Instructor Led" pages are broken, e.g. Cloud to Cloud Migration (there are many more).

Confirmed that in Cloud to Cloud migration (github.com/microsoft/PartnerResources/blob/main/LearningPlanResources/Azure/Infrastructure/Cloud to Cloud Migration.md) most of the expert-level links that are instructor led are 404.
Confirmed that in VMWare Migration (github.com/microsoft/PartnerResources/blob/main/LearningPlanResources/Azure/Infrastructure/VMWare Migration.md) those expert-level instructor led links are 404.
Likely others, but will rely on team to check their own areas.  Thanks for raising.

We've notified across the SET re links and to be checking across LPR's. If you cannot find where the new links are please do best effort to do some discovery. Raise issues SPECIFICALLY on which Learning Plan resource if there are errors and you cannot determine path forward (i.e. did due diligence in finding updated material but could not find, etc.).

authors: Jeep-Girl bhitney petertuton

repo: microsoft/PartnerResources