text:
[BUG] System.ArgumentNullException: Value cannot be null. (Parameter 'queueName') when running in node functions app

Library name and version
Microsoft.Azure.WebJobs.Extensions.Storage.Queues 5.0.0
Describe the bug
We are writing a node based function app importing Microsoft.Azure.WebJobs.Extensions.Storage.Queues 5.0.0 via the Microsoft.Azure.Functions.ExtensionBundle extension bundle in our host.json however we cannot get our functions app to run successfully with a queue trigger function
Expected behavior
Functions run successfully
Actual behavior
[2022-02-23T17:08:14.183Z] A host error has occurred during startup operation '6ce5d430-f5a3-4772-bcdd-bd9e00f16456'.
[2022-02-23T17:08:14.184Z] System.ArgumentNullException: Value cannot be null. (Parameter 'queueName')
[2022-02-23T17:08:14.188Z]    at Microsoft.Azure.WebJobs.Extensions.Storage.Common.HostQueueNames.IsHostQueue(String queueName)
[2022-02-23T17:08:14.188Z]    at Microsoft.Azure.WebJobs.Extensions.Storage.Queues.Listeners.QueueListenerFactory.CreateQueueProcessor(QueueClient queue, QueueClient poisonQueue, ILoggerFactory loggerFactory, IQueueProcessorFactory queueProcessorFactory, QueuesOptions queuesOptions, IMessageEnqueuedWatcher sharedWatcher)
[2022-02-23T17:08:14.188Z]    at Microsoft.Azure.WebJobs.Extensions.Storage.Queues.Listeners.QueueListenerFactory.CreateAsync(CancellationToken cancellationToken)
[2022-02-23T17:08:14.188Z]    at Microsoft.Azure.WebJobs.Extensions.Storage.Queues.Triggers.QueueTriggerBinding.CreateListenerAsync(ListenerFactoryContext context)
[2022-02-23T17:08:14.188Z]    at Microsoft.Azure.WebJobs.Host.Indexers.FunctionIndexer.ListenerFactory.CreateAsync(CancellationToken cancellationToken) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Indexers\FunctionIndexer.cs:line 474
[2022-02-23T17:08:14.188Z]    at Microsoft.Azure.WebJobs.Host.Listeners.HostListenerFactory.CreateAsync(CancellationToken cancellationToken) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Listeners\HostListenerFactory.cs:line 71
[2022-02-23T17:08:14.188Z]    at Microsoft.Azure.WebJobs.Host.Listeners.ListenerFactoryListener.StartAsyncCore(CancellationToken cancellationToken) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Listeners\ListenerFactoryListener.cs:line 45
[2022-02-23T17:08:14.188Z]    at Microsoft.Azure.WebJobs.Host.Listeners.ShutdownListener.StartAsync(CancellationToken cancellationToken) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Listeners\ShutdownListener.cs:line 29
[2022-02-23T17:08:14.188Z]    at Microsoft.Azure.WebJobs.JobHost.StartAsyncCore(CancellationToken cancellationToken) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\JobHost.cs:line 97
[2022-02-23T17:08:14.188Z]    at Microsoft.Azure.WebJobs.Script.ScriptHost.StartAsyncCore(CancellationToken cancellationToken) in D:\a\1\s\src\WebJobs.Script\Host\ScriptHost.cs:line 273
[2022-02-23T17:08:14.189Z]    at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)
[2022-02-23T17:08:14.189Z]    at Microsoft.Azure.WebJobs.Script.WebHost.WebJobsScriptHostService.UnsynchronizedStartHostAsync(ScriptHostStartupOperation activeOperation, Int32 attemptCount, JobHostStartupMode startupMode) in D:\a\1\s\src\WebJobs.Script.WebHost\WebJobsScriptHostService.cs:line 292
[2022-02-23T17:08:14.215Z] Stopping JobHost
System.ArgumentNullException: Value cannot be null. (Parameter 'provider')
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Azure.Functions.Cli.Actions.HostActions.StartHostAction.RunAsync() in D:\a\1\s\src\Azure.Functions.Cli\Actions\HostActions\StartHostAction.cs:line 354
   at Azure.Functions.Cli.ConsoleApp.RunAsync[T](String[] args, IContainer container) in D:\a\1\s\src\Azure.Functions.Cli\ConsoleApp.cs:line 66

Reproduction Steps
Create a node functions app and add this to host.json
"extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle",
    "version": "[3.3.0, 4.0.0)"
  }

Create a function with a function.json that looks like this:
{
  "bindings": [
    {
      "name": "queueItem",
      "type": "queueTrigger",
      "direction": "in",
      "queueName": "delete",
      "connection": "AzureStorage"
    }
  ],
  "scriptFile": "../dist/delete/index.js"
}

attempt to run with CLI_DEBUG=1 func start --verbose
Environment
node 16
azure functions tools v4

@ksstott I have tried this in my local environment, and it is working fine.

Can you please check what all extensions version you have in your local environment?

I have the same issue.
Azure.Data.Tables 12.4.3
Microsoft.Azure.Functions.Extensions 1.1.0
And these 2 from a legacy package reference: Microsoft.Azure.Search (10.1.0) | WinfowsAzure.Storage 9.3.3

Hi @ksstott, @kvigor

Is it time triggered, or direct Queue triggered?
the error says the queue name is null, cam u makes sure you are passing the queue name
Can u run the function app in local dev machine as part of local debug and check?

Thanks
Saurabh

Timer Trigger.
I've resolved the...
"Value cannot be null. (Parameter 'queueName') error as tied to dev settings being added to staging settings. I don't ever swap those slots and the Azure activity logs don't even show the settings in staging being updated which is fishy. But, I still have an issue with the second part of that issue going on.
When our Messaging API published messages to the queue and subsequent blob store the Azure Function app seems to not be able to access it as shown below.
@.***
From: SaurabhSharma-MSFT @.>
Sent: Monday, February 28, 2022 2:52 PM
To: Azure/azure-sdk-for-net @.>
Cc: Kevon Hayes @.>; Mention @.>
Subject: Re: [Azure/azure-sdk-for-net] [BUG] System.ArgumentNullException: Value cannot be null. (Parameter 'queueName') when running in node functions app (Issue #27164)
Hi @ksstotthttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fksstott&data=04|01||5ff3410124e7408e251508d9fafc324e|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637816783343075284|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=HIEEz%2FApN0rFan0JLYt5ZYQafS%2Fgh%2BzdPqIGiYnux58%3D&reserved=0, @kvigorhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fkvigor&data=04|01||5ff3410124e7408e251508d9fafc324e|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637816783343075284|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=8vjXvlykJHNk8cBrtm6xbJoCn09za6XaYgQIIz2peKw%3D&reserved=0

Is it time triggered, or direct Queue triggered?
the error says the queue name is null, cam u makes sure you are passing the queue name
Can u run the function app in local dev machine as part of local debug and check?

Thanks
Saurabh

Reply to this email directly, view it on GitHubhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fazure-sdk-for-net%2Fissues%2F27164%23issuecomment-1054651197&data=04|01||5ff3410124e7408e251508d9fafc324e|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637816783343075284|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=qlwxTpKqTbtmDzpDeJQhSQMiK%2BlkZNlCwxipZn%2BQjVQ%3D&reserved=0, or unsubscribehttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAAGDO3NCOWMZ7GGDXG7TH4LU5PN7ZANCNFSM5PFAF45A&data=04|01||5ff3410124e7408e251508d9fafc324e|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637816783343075284|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=YBFUl5c3bD8hfZjexzGBt9nqA0alT%2BeVcHJyG92D73k%3D&reserved=0.
Triage notifications on the go with GitHub Mobile for iOShttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04|01||5ff3410124e7408e251508d9fafc324e|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637816783343075284|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=ZqDLvAIdfnOXhH44W%2FF6lOFLba3y%2FpYnzLU0%2Bh%2B2H5s%3D&reserved=0 or Androidhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04|01||5ff3410124e7408e251508d9fafc324e|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637816783343075284|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=zLwacrBTXXkSTpuvG1G6eb5NvTcQhDFInnkN31iPy40%3D&reserved=0.
You are receiving this because you were mentioned.Message ID: @.***>

Sorry I had to delete the previous message I thought is was a correspondence our support ticket we have open with MS on this issue.

Yes this is a timer trigger
This queue name was erroneous, and this queue error is fixed.
I can no longer run locally, however, sometimes the Azure function runs fine and other times it just errors out. This makes it hard to debug especially when I'm not changing the payload sent to the queue.

@kvigor Thanks for the confirmation.  We are proceeding with the closure of this case. Thanks.

authors: SaurabhSharma-MSFT ksstott kvigor


text:
[QUERY] Driving Directions

Library name and version
azure-sdk-for-net
Query/Question
Hello ..
We are working on fleet management system that use Console map for drivers and this consoles are connected to central unit which dispach routes and waypoints for each day .. our question is how we can render the routing api result on map and display it as Driving mode with instructions, we looked into azure maps sample sdk but no example for it founded unfortunately
our goal is get the route results and render it with instruction when driver start working on it.
if we are missing some thing please let us know
( The Console app is will be builded with Blazor MAUI .Net 6 and will run on Android Tablet as Hybrid app )
there is some pictures explain our goal as an attachment

Best Regard's
Environment
.NET Core 5 / 6
Blazor WASM
Blazor MAUI

Label prediction was below confidence level 0.6 for Model:ServiceLabels: 'Azure.Core:0.11213854,Service Bus:0.07461427,Storage:0.074002735'

Adding Service team to look into this.

Hi @scsJames ,
Currently Azure Maps is unable to do turn by turn navigation. This is not supported yet. Not sure this answers your question or not?

source: gharchive/issue

authors: azure-sdk dubiety navba-MSFT scsJames

repo: Azure/azure-sdk-for-net

license: MIT

license_type: permissive

license_source: github-api


text:
[BUG] Exception reading keyvault

Library name and version
azure.identity 1.8.0
Describe the bug
We are seeing this exception sometimes when running the below code in .net 6
The below code is running in a scoped service and is called when the app starts up.
            SecretClientOptions options = new SecretClientOptions()
            {
                Retry =
                {
                    Delay= TimeSpan.FromSeconds(2),
                    MaxDelay = TimeSpan.FromSeconds(16),
                    MaxRetries = 5,
                    Mode = RetryMode.Exponential
                }
            };

            string tenantId = _configuration.GetSection("Azure:JwtTenantId").Value;
            string clientId = _configuration.GetSection("Azure:JwtClientId").Value;
            string clientSecret = _configuration.GetSection("Azure:JwtClientSecret").Value;

            ClientSecretCredential clientSecretCredential = new ClientSecretCredential(tenantId, clientId, clientSecret);

            var client = new SecretClient(new Uri(KeyVaultUri), clientSecretCredential, options);

            var keyvaultSecret = client.GetSecret("JwtSecret");

Expected behavior
should not throw an exception
Actual behavior
throws exception
"Azure.Identity.AuthenticationFailedException: ClientSecretCredential authentication failed: Collection was modified; enumeration operation may not execute.\r\n ---> System.InvalidOperationException: Collection was modified; enumeration operation may not execute.\r\n at System.Collections.Generic.List1.Enumerator.MoveNext()\r\n at System.Net.Http.Headers.HttpHeaders.ReadStoreValues[T](Span1 values, Object storeValue, HttpHeaderParser parser, Int32& currentIndex)\r\n at System.Net.Http.Headers.HttpHeaders.GetStoreValuesAsStringOrStringArray(HeaderDescriptor descriptor, Object sourceValues, String& singleValue, String[]& multiValue)\r\n at System.Net.Http.Headers.HttpHeaders.GetEnumeratorCore()+MoveNext()\r\n at Azure.Core.Pipeline.HttpClientTransport.GetHeaders(HttpHeaders headers, HttpContent content)+MoveNext()\r\n at Azure.Core.HttpPipelineMessageHandler.ToHttpResponseMessage(Response response)\r\n at Azure.Core.HttpPipelineMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)\r\n at System.Net.Http.HttpClient.g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)\r\n at Microsoft.Identity.Client.Http.HttpManager.ExecuteAsync(Uri endpoint, IDictionary2 headers, HttpContent body, HttpMethod method, ILoggerAdapter logger, CancellationToken cancellationToken)\r\n at Microsoft.Identity.Client.Http.HttpManager.ExecuteWithRetryAsync(Uri endpoint, IDictionary2 headers, HttpContent body, HttpMethod method, ILoggerAdapter logger, Boolean doNotThrow, Boolean retry, CancellationToken cancellationToken)\r\n at Microsoft.Identity.Client.Http.HttpManager.SendPostAsync(Uri endpoint, IDictionary2 headers, HttpContent body, ILoggerAdapter logger, CancellationToken cancellationToken)\r\n at Microsoft.Identity.Client.Http.HttpManager.SendPostAsync(Uri endpoint, IDictionary2 headers, IDictionary2 bodyParameters, ILoggerAdapter logger, CancellationToken cancellationToken)\r\n at Microsoft.Identity.Client.OAuth2.OAuth2Client.ExecuteRequestAsync[T](Uri endPoint, HttpMethod method, RequestContext requestContext, Boolean expectErrorsOn200OK, Boolean addCommonHeaders, Func2 onBeforePostRequestData)\r\n at Microsoft.Identity.Client.OAuth2.OAuth2Client.GetTokenAsync(Uri endPoint, RequestContext requestContext, Boolean addCommonHeaders, Func2 onBeforePostRequestHandler)\r\n at Microsoft.Identity.Client.OAuth2.TokenClient.SendHttpAndClearTelemetryAsync(String tokenEndpoint, ILoggerAdapter logger)\r\n at Microsoft.Identity.Client.OAuth2.TokenClient.SendTokenRequestAsync(IDictionary2 additionalBodyParameters, String scopeOverride, String tokenEndpointOverride, CancellationToken cancellationToken)\r\n at Microsoft.Identity.Client.Internal.Requests.ClientCredentialRequest.FetchNewAccessTokenAsync(CancellationToken cancellationToken)\r\n at Microsoft.Identity.Client.Internal.Requests.ClientCredentialRequest.ExecuteAsync(CancellationToken cancellationToken)\r\n at Microsoft.Identity.Client.Internal.Requests.RequestBase.RunAsync(CancellationToken cancellationToken)\r\n at Microsoft.Identity.Client.ApiConfig.Executors.ConfidentialClientExecutor.ExecuteAsync(AcquireTokenCommonParameters commonParameters, AcquireTokenForClientParameters clientParameters, CancellationToken cancellationToken)\r\n at Azure.Identity.AbstractAcquireTokenParameterBuilderExtensions.ExecuteAsyncT\r\n at Azure.Identity.MsalConfidentialClient.AcquireTokenForClientCoreAsync(String[] scopes, String tenantId, Boolean async, CancellationToken cancellationToken)\r\n at Azure.Identity.MsalConfidentialClient.AcquireTokenForClientAsync(String[] scopes, String tenantId, Boolean async, CancellationToken cancellationToken)\r\n at Azure.Identity.ClientSecretCredential.GetToken(TokenRequestContext requestContext, CancellationToken cancellationToken)\r\n --- End of inner exception stack trace ---\r\n at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage)\r\n at Azure.Identity.ClientSecretCredential.GetToken(TokenRequestContext requestContext, CancellationToken cancellationToken)\r\n at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetHeaderValueFromCredentialAsync(TokenRequestContext context, Boolean async, CancellationToken cancellationToken)\r\n at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetHeaderValueAsync(HttpMessage message, TokenRequestContext context, Boolean async)\r\n at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetHeaderValueAsync(HttpMessage message, TokenRequestContext context, Boolean async)\r\n at Azure.Security.KeyVault.ChallengeBasedAuthenticationPolicy.AuthorizeRequestOnChallengeAsyncInternal(HttpMessage message, Boolean async)\r\n at Azure.Security.KeyVault.ChallengeBasedAuthenticationPolicy.AuthorizeRequestOnChallenge(HttpMessage message)\r\n at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean async)\r\n at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.Process(HttpMessage message, ReadOnlyMemory1 pipeline)\r\n at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory1 pipeline)\r\n at Azure.Core.Pipeline.RedirectPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean async)\r\n at Azure.Core.Pipeline.RedirectPolicy.Process(HttpMessage message, ReadOnlyMemory1 pipeline)\r\n at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean async)\r\n at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean async)\r\n at Azure.Core.Pipeline.RetryPolicy.Process(HttpMessage message, ReadOnlyMemory1 pipeline)\r\n at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory1 pipeline)\r\n at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory1 pipeline)\r\n at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory1 pipeline)\r\n at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory1 pipeline)\r\n at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory1 pipeline)\r\n at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory1 pipeline)\r\n at Azure.Core.Pipeline.HttpPipeline.Send(HttpMessage message, CancellationToken cancellationToken)\r\n at Azure.Security.KeyVault.KeyVaultPipeline.SendRequest(Request request, CancellationToken cancellationToken)\r\n at Azure.Security.KeyVault.KeyVaultPipeline.SendRequestTResult\r\n at Azure.Security.KeyVault.Secrets.SecretClient.GetSecret(String name, String version, CancellationToken cancellationToken)\r\n at Velocity.Akouba.Services.Config.KeyVaultConfigService.GetJwtSecret()"
Reproduction Steps
happens on the startup of the website after deployment, but is intermittent.
Environment
.net core 6 running in IIS in Rackspace
Using Azure.Identity 1.8.0 and Azure.Security.KeyVault.Secrets 4.4.0

@AlexanderSher :  This looks to trace back to header-related functionality in Azure.Core.  Do you mind taking a look, please?

Thank you for your feedback.  Tagging and routing to the team member best able to assist.

I am experiencing exactly the same issue.

Would you be willing to test if this reproduce with Azure.Identity 1.9.0?

I believe my issue was because I didn't had permissions in one of the keyvaults and it was throwing that error without any information

We disabled the use of the key vault since it was not reliable for us. So I can't try to reproduce as it was only in production where it happened.Sent from my iPhoneOn May 25, 2023, at 5:31 PM, github-actions[bot] @.***> wrote:
Hi @ncsugrad94, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>


text:
[FEATURE REQ] Provide a SHA2-supported BlobProperties.ContentHash

Library name
Azure.Storage.Blobs
Please describe the feature.
The current ContentHash property returns an MD5 hash. Unfortunately, our internal code analysis tools flag any usage of MD5 as weak cryptography which requires lengthy exception request processes.
Would it be possible to include a new ContentHash property that uses a SHA2-based hash algorithm?

System.Security.Cryptography.SHA256.Create()
System.Security.Cryptography.SHA384.Create()
System.Security.Cryptography.SHA512.Create()

@WilliamXieMSFT Apologies for the late reply. Thanks for reaching out to us and reporting this issue. We are looking into this issue and we will provide an update.

@WilliamXieMSFT I tried the below approach and I was able to set the Content-MD5 property on my blob using sha512 algorithm hash.
$hash = (Get-FileHash -Algorithm SHA512 -Path "C:\myfolder\test.txt").Hash

# upload the blob to the storage

az storage blob update --account-name MyStorageAccount  --account-key "XXXXXXXXX" --container-name MyContainer
--name test.txt --content-md5 $hash --debug

Could you please let me know if this helps ?

Hi @navba-MSFT, appreciate the reply. The workaround requires having control over uploading the dump file, and forces client apps to have additional logic checking which algorithm is used for the content hash (not to mention confusion since the name of the property mentions MD5).
Would it be possible to have both a Content-Hash property as well as a Content-Hash-Algorithm property to aid in determining which algorithm was used?
I believe the Content-MD5 property is auto-generated by Azure Storage, and I'm sure there are historical reasons for this. The concern is that for products to meet FIPS compliance checks, any usage of MD5 will trigger blocking bugs. If there could be a way to set the default to a SHA2 algorithm instead, that would greatly reduce the number of exceptions we must file.
Thanks for the consideration.

This would require support from the storage service, and is very unlikely to be prioritized.

Hi since this is a storage service request, and not an SDK request we recommend discussing this among the azure support community.
https://azure.microsoft.com/en-us/support/community/
Closing this issue as this is a non-SDK issue.

authors: WilliamXieMSFT amnguye navba-MSFT seanmcc-msft


text:
Update Microsoft.Azure.Management.ResourceManager.csproj

Update Microsoft.Azure.Management.ResourceManager version to request a new release.
All SDK Contribution checklist:
This checklist is used to make sure that common guidelines for a pull request are followed.

[x] Please open PR in Draft mode if it is:

Work in progress or not intended to be merged.
Encountering multiple pipeline failures and working on fixes.

[x] If an SDK is being regenerated based on a new swagger spec, a link to the pull request containing these swagger spec changes has been included above.
[x] I have read the contribution guidelines.
[x] The pull request does not introduce breaking changes.

General Guidelines and Best Practices

[x] Title of the pull request is clear and informative.
[x] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[x] Pull request includes test coverage for the included changes.

SDK Generation Guidelines

[x] The generate.cmd file for the SDK has been updated with the version of AutoRest, as well as the commitid of your swagger spec or link to the swagger spec, used to generate the code. (Track 2 only)
[x] The *.csproj and AssemblyInfo.cs files have been updated with the new version of the SDK. Please double check nuget.org current release version.

Additional management plane SDK specific contribution checklist:
Note: Only applies to Microsoft.Azure.Management.[RP] or Azure.ResourceManager.[RP]

[ ] Include updated management metadata.
[ ] Update AzureRP.props to add/remove version info to maintain up to date API versions.

Management plane SDK Troubleshooting

If this is very first SDK for a services and you are adding new service folders directly under /SDK, please add new service label and/or contact assigned reviewer.

If the check fails at the Verify Code Generation step, please ensure:

Do not modify any code in generated folders.
Do not selectively include/remove generated files in the PR.
Do use generate.ps1/cmd to generate this PR instead of calling autorest directly.
Please pay attention to the @microsoft.csharp version output after running generate.ps1. If it is lower than current released version (2.3.82), please run it again as it should pull down the latest version.

Note: We have recently updated the PSH module called by generate.ps1 to emit additional data. This would help reduce/eliminate the Code Verification check error. Please run following command:
  `dotnet msbuild eng/mgmt.proj /t:Util /p:UtilityName=InstallPsModules`

Old outstanding PR cleanup
Please note:
If PRs (including draft) has been out for more than 60 days and there are no responses from our query or followups, they will be closed to maintain a concise list for our reviewers.

Thank you for approving the PR.
Can you please help merge it as I do not have the permission?
Thanks,
Di
From: Mingzhe Huang @.>
Sent: Tuesday, September 21, 2021 8:01 PM
To: Azure/azure-sdk-for-net @.>
Cc: Di Wu @.>; Author @.>
Subject: Re: [Azure/azure-sdk-for-net] Update Microsoft.Azure.Management.ResourceManager.csproj (#24129)
@archerzz approved this pull request.

You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fazure-sdk-for-net%2Fpull%2F24129%23pullrequestreview-760411426&data=04|01|wud%40microsoft.com|688b9d050a18460ea1a508d97d753058|72f988bf86f141af91ab2d7cd011db47|1|0|637678764548447682|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=8CyNyaUajI2WGc9Fry0WaNSgcDAToVYS6ae1r37EfcA%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFOVDVQEMUH2QDTGRYEMN43UDFBGJANCNFSM5EM6NE3Q&data=04|01|wud%40microsoft.com|688b9d050a18460ea1a508d97d753058|72f988bf86f141af91ab2d7cd011db47|1|0|637678764548457642|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=7TCuiwBYZrXsPZoiz8RGE01JCDpARNEwoqyRiBTahNI%3D&reserved=0.
Triage notifications on the go with GitHub Mobile for iOShttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04|01|wud%40microsoft.com|688b9d050a18460ea1a508d97d753058|72f988bf86f141af91ab2d7cd011db47|1|0|637678764548457642|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=nJMmpmA7BaUJf1M4z%2B5MuXw1ppTHCwbTE74LUAe2XG0%3D&reserved=0 or Androidhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04|01|wud%40microsoft.com|688b9d050a18460ea1a508d97d753058|72f988bf86f141af91ab2d7cd011db47|1|0|637678764548467594|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=YZhglSPZvtoziUmkhpUIQi1o6adjVbUBZdokgSqO65I%3D&reserved=0.

source: gharchive/pull-request


text:
Update TROUBLESHOOTING.md

Update links from #enabling-and-configuring-logging to #enable-and-configure-logging
Contributing to the Azure SDK
Please see our CONTRIBUTING.md if you are not familiar with contributing to this repository or have questions.
For specific information about pull request etiquette and best practices, see this section.

Thank you for the contribution, @DavidAbraham082!


text:
Increment version for eventhub releases

Increment package version after release of Azure.Messaging.EventHubs

API change check
API changes are not detected in this pull request.


text:
Support overrides of token issuer from environment variables

This pr intends to add support for overrides of issuer for token validation.
We will introduce reading the following environments from application settings of azure functions.
AuthenticationEvents__TokenIssuer_V1
AuthenticationEvents__TokenIssuer_V2
AuthenticationEvents__OIDCMetadataUrl
With the above combined with AuthenticationEvents__TenantId and AuthenticationEvents__AudienceAppId a developer can define their own application for token validation.
Example usages:
[FunctionName("onTokenIssuanceStart")]
public async static Task<AuthenticationEventResponse> Run(
    [AuthenticationEventsTrigger(
    TenantId = "00000001-0000-0000-0000-000000000000",
    AudienceAppId = "00000002-0002-0002-0002-000000000002",
    OIDCMetadataUrl= "https://login.windows.net/00000001-0000-0000-c000-000000000000/.well-known/openid-configuration",
    TokenIssuerV1 = "https://sts.windows.net/00000001-0000-0000-c000-000000000000",
    TokenIssuerV2 = "https://login.microsoftonline.com/00000001-0000-0000-c000-000000000000/v2.0")] TokenIssuanceStartRequest request, ILogger log)
{

Environment variables advance definition

API change check
APIView has identified API level changes in this PR and created following API reviews.
Microsoft.Azure.WebJobs.Extensions.AuthenticationEvents

@hakimms Tried adding you as a reviewer but your name isn't coming up. Can you give this a review please?

authors: HarmanDhunna azure-sdk


text:
Installation instructions for azure-search-documents shows --pre

On https://pypi.org/project/azure-search-documents/ for recently-GA'd azure-search-documents, the installation section shows:
pip install azure-search-documents --pre

This is in the README for 11.0.

Fixed in #12404

repo: Azure/azure-sdk-for-python


text:
[Metrics Advisor] Release metricsadvisor beta 1

Feature implementation

[x] [Metrics Advisor] implement DataFeed functionality
[x] [Metrics Advisor] implement AnomalyAlertingConfiguration functionality
[x] [Metrics Advisor] implement AnomalyDetectionConfiguration functionality
[x] [Metrics Advisor] implement Alerting querying methods
[x] [Metrics Advisor] implement Detection querying methods
[x] [Metrics Advisor] implement Feedback methods
[x] [Metrics Advisor] implement Hook methods
[x] [Metrics Advisor] implement IngestionStatus methods
[x] [Metrics Advisor] implement Metric methods

Infrastructure

[x] [Metrics Advisor] Initial commit to feature branch
[x] [Metrics Advisor] Add CI build pipeline
[ ] [Metrics Advisor] enable live test pipeline

Doc

[x] [Metrics Advisor] Create/improve readme
[x] [Metrics Advisor] Ensure proper reference docs for public API surface

Samples

[x] [Metrics Advisor] create samples

Released! https://pypi.org/project/azure-ai-metricsadvisor/


text:
DataLakeFileClient.upload_data overwrite flag

It seems that I cannot use upload_data without setting the overwrite flag to true. Maybe I am doing something wrong:
file1 = my_file_system_client.get_file_client("path/to/NONEXISTING_file.txt")
file1.upload_data(data) # Trying to upload to a non-existing file
# Raises: azure.core.exceptions.ResourceNotFoundError: (PathNotFound) The specified path does not exist.

file1 = my_file_system_client.get_file_client("path/to/NONEXISTING_file.txt")
file1.upload_data(data, overwrite=True) # Trying to upload to a non-existing file, with overwrite
# Works!

file1 = my_file_system_client.get_file_client("path/to/existing.txt")
file1.upload_data(data) # Trying to upload to a existing file, without overwrite
# Raises: azure.storage.filedatalake._generated.models._models_py3.StorageErrorException: Operation returned an invalid status 'The condition specified using HTTP conditional header(s) is not met.'

file1 = my_file_system_client.get_file_client("path/to/existing.txt")
file1.upload_data(data, overwrite=True) # Trying to upload to a existing file, with overwrite
# Works!

If this is the case for everyone, I am questioning the use of the overwrite flag... or at least its default value of False?

Duplicate - closed

Duplicate - closed


text:
CosmosDB Trigger > retrieve documents directly

It would be very useful to have directly Document(s) that trigger python function like in C# or js. If I try to retrieve the document with a subsequent query, I'm not sure if I give me the exact document.

My feeling is that your question belongs on the DocDB repo?
https://github.com/Azure/azure-documentdb-python
If not, could you describe a little more, since I'm not able to understand what code you are talking about :(
Thank you,

Probably I have not explained well. The problem is when I create a function on Azure in python and I want to trigger it with cosmosDB (when a document is loaded or edited). The trigger arrives correctly, but unlike javascript or c#, I am not returned the document (s) that triggered the event. I can not query the DB after the event because I would risk not having the exact document (s).

Ok, didn't understand you're talking about Azure Function. @asavaritayal ? Thoughts?

Closing this issue as non-SDK issue. If the design doesn't still allow in current Azure Function to do this scenario properly, please do a feedback on the Azure Function service on the Azure Portal.

authors: lmazuel mirkomancin


text:
Automatic PR from RestAPI-PR1466

Generated from RestAPI PR: https://github.com/Azure/azure-rest-api-specs/pull/1466

Codecov Report

Merging #1328 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1328   +/-   ##
=======================================
  Coverage   56.04%   56.04%
=======================================
  Files        2691     2691
  Lines       71246    71246
=======================================
  Hits        39932    39932
  Misses      31314    31314

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f477734...4d7ef11. Read the comment docs.

Not relevant

authors: AutorestCI codecov-io lmazuel


text:
[Cosmos] remove double signatures on container methods

Addressing https://github.com/Azure/azure-sdk-for-python/issues/23355

API changes have been detected in azure-cosmos. You can review API changes here

authors: azure-sdk simorenoh


text:
add back import for run samples script

Failing on our live tests: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=1785379&view=results

@kristapratico https://github.com/Azure/azure-sdk-for-python/pull/25721 already merged.

@kristapratico #25721 already merged. Sorry for the inconvenience :(

Nice, thank you!

authors: kristapratico scbedd


text:
[Personalizer] fix pylint next

fixes #31599
tried to regenerate -- failed :( so manually updating

API change check
API changes are not detected in this pull request.


text:
[AutoPR storage/resource-manager] Move Storage to Py v4.x

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/6296

(message created by the CI based on PR content)

This PR has been merged into https://github.com/Azure/azure-sdk-for-python/pull/5713


text:
Lint code for Python syntax errors and undefined names

Many of these undefined names can be solved with import typing or by importing other modules while others can be solved by using six.string_types and six.text_type or by following Python porting best practice use feature detection instead of version detection.
The F63 issues are about the confusion between identity and equality in Python.  Use ==/!= to compare str, bytes, and int literals are instances where a == b is True but a is b is False (or vice versa).

Can one of the admins verify this patch?

The F63 instances will raise SyntaxWarnings on Python >= 3.8 so it is best to fix them now. https://docs.python.org/3.8/whatsnew/3.8.html#porting-to-python-3-8

@mayurid @lmazuel can you take a look?

Hi @cclauss
Sorry for the very late discussion on this one, missed your PR :(
We are using a combination of pylint/black nowadays in our main CI, and we don't plan to integrate flake8 in our pipelines.
Also, you introduced it in the "updated_pr" pipeline, which is not designed for that (that's a bot to update packaging information automatically).
For all those reasons, I'm closing this PR for now, but I'll be happy to discuss our engsys in an issue if you believe there is important improvement we should still do.
Thanks for your contribution!

authors: Petermarcu adxsdk6 cclauss lmazuel


text:
Storage samples + ref docs

Make samples executable
Add read me to samples
Fix dedents
add missing rtypes

Can one of the admins verify this patch?


text:
Board Review: Form Recognizer

The Basics

Service team responsible for the client library: Azure SDK Team
Link to documentation describing the service: https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/overview
Contact email: @mayurid

About this client library

Name of the client library: Form Recognizer
Languages for this review: Java and Javascript
Link to the service REST APIs: https://github.com/Azure/azure-rest-api-specs/blob/master/specification/cognitiveservices/data-plane/FormRecognizer/preview/v2.0/FormRecognizer.json
and https://westus2.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2-preview/operations/AnalyzeWithCustomForm

Artifacts required (per language)
We use an API review tool (apiview) to support .NET and Java API reviews.  For Python and TypeScript, use the API extractor tool, then submit the output as a Draft PR to the relevant repository (azure-sdk-for-python or azure-sdk-for-js).
Java

Upload sources JAR to apiview.  Link: https://apiview.dev/Assemblies/Review/05d0102010704cfa977f9386ce622294
Link to samples for champion scenarios: https://github.com/Azure/azure-sdk-for-java/tree/master/sdk/formrecognizer/azure-ai-formrecognizer/src/samples

Javascript

Upload the api as a Draft PR.  Link to PR: https://github.com/Azure/azure-sdk-for-js/pull/10129/files
Link to samples for champion scenarios: https://github.com/Azure/azure-sdk-for-js/tree/master/sdk/formrecognizer/ai-form-recognizer/samples/typescript/src

Champion Scenarios
A champion scenario is a use case that the consumer of the client library is commonly expected to perform.  Champion scenarios are used to ensure the developer experience is exemplary for the common cases.  You need to show the entire code sample (including error handling, as an example) for the champion scenarios.
Recognize and extract field values from custom forms

Target audience: LOB developers (~90%)
Code samples: Java | JS

Recognize and extract field values from Receipt documents

Target audience: Day 0 developers, LOB developers (~30%)
Code samples: .Java | JS

Recognizing/Extracting strongly typed receipt fields from RecognizedForm

Target audience: Day 0 developers, LOB developers (~30%)
Code samples: .Java | JS

Recognize and extract OCR Content and Tables from Forms

Target audience: LOB developers (~20%)
Code samples: .Java | JS

Train a model from custom forms

Target audience: Data Scientists (~50%)
Code samples: Java | JS

Manage custom models

Target audience: Data Scientists (~50%)
Code samples: Java | JS

Agenda for the review
A board review is generally split into two parts, with additional meetings as required
Part 1 - Introducing the board to the service:

Review of the service (no more than 10 minutes).
Review of the champion scenarios.
Get feedback on the API patterns used in the champion scenarios.

After part 1, you may schedule additional meetings with architects to refine the API and work on implementation.
Part 2 - the "GA" meeting

Scheduled at least one week after the APIs have been uploaded for review.
Will go over controversial feedback from the line-by-line API review.
Exit meeting with concrete changes necessary to meet the quality bar.

Thank you for your submission

@lilyjma Updated to include link to JavaScript API listing (draft PR on azure-sdk-for-js repo).

[MS internal only] Video part1, part2

(Thanks to Maggie and Kyle for the notes!)
Java

Last meeting, want to have options parameter be the only param on the overload
FieldText model has been changed to FieldData
- Decided to rename to call it FieldData because it has more elements than just the text in the field, this is consistent with Python naming

FormRecognizerAsyncClient

Added overload to pass in buffer type to PollerFlux
Parameter order question - should order be (String modelId, formUrl)

ACTION: All languages look at parameter order on FormRecognizerAsyncClient.PollerFlux
FOLLOW UP: Look into lightening up the model package by moving account related model properties apart from training based models
FOLLOW UP: Look into IOException usage
FOLLOW UP: RecognizerOptions has a naming concern, does this fit with other languages?
JS/TS

Same renames as Java above
Decided not to do a table-indexer, instead use flat cells array to be consistent with other languages and service
- Could add convenience functions later, if necessary

FOLLOW UP: JS uses "valueType", should python be doing the same thing?

authors: lilyjma samvaity willmtemple

repo: Azure/azure-sdk


text:
Board Review:

Thank you for submitting this review request. Thorough review of your client library ensures that your APIs are consistent with the guidelines and the consumers of your client library have a consistently good experience when using Azure.
The Architecture Board reviews Track 2 libraries only. If your library does not meet this requirement, please reach out to Architecture Board before creating the issue.
Please reference our review process guidelines to understand what is being asked for in the issue template.
To ensure consistency, all Tier-1 languages (C#, TypeScript, Java, Python) will generally be reviewed together.  In expansive libraries, we will pair dynamic languages (Python, TypeScript) together, and strongly typed languages (C#, Java) together in separate meetings.
For Tier-2 languages (C, C++, Go, Android, iOS), the review will be on an as-needed basis.
Before submitting, ensure you adjust the title of the issue appropriately.
Note that the required material must be included before a meeting can be scheduled.
Contacts and Timeline

Responsible service team:
Main contacts:
Expected code complete date:
Expected release date:

About the Service

Link to documentation introducing/describing the service:
Link to the service REST APIs:
Link to GitHub issue for previous review sessions, if applicable:

About the client library

Name of the client library:
Languages for this review:

Artifacts required (per language)
Please read through “API Review” section here to understand how these artifacts are generated. It is critical that these artifacts are present and are in the right format. If not, the language architects cannot review them with the SDK Team’s API review tool.
.NET

APIView Link:
Link to Champion Scenarios/Quickstart samples:

Java

APIView Link:
Link to Champion Scenarios/Quickstart samples:

Python

APIView Link:
Link to Champion Scenarios/Quickstart samples:

TypeScript

APIView Link:
Link to Champion Scenarios/Quickstart samples:

For all other languages, send a request to the Architecture Board to discuss the best format on individual basis.
Thank you!

Closing issue as it seems to have been created by mistake.

authors: bellaariffinofficials lilyjma


text:
panic: close /root/.azcopy/...: file already closed

Which version of the AzCopy was used? azcopy version 10.22.2
Which platform are you using? Ubuntu 22.04
What command did you run? AZCOPY_CONCURRENCY_VALUE=128 azcopy bench --mode='Upload' "https://....blob.core.windows.net/bench" --file-count 50 --size-per-file 128M

What problem was encountered?
(Running cleanup job to delete files created during benchmarking with cleanup jobID 8410e5c5-bdeb-7c4f-568e-6de9fcd40848
Cleanup 50/50     panic: close /root/.azcopy/488ce77e-69d8-eb44-4ddf-d1cdbadc7679: file already closed

goroutine 1949 [running]:
github.com/Azure/azure-storage-azcopy/v10/common.PanicIfErr(...)
	/home/vsts/work/1/s/common/lifecyleMgr.go:698
github.com/Azure/azure-storage-azcopy/v10/common.(*jobLogger).CloseLog(0xc00033bc80)
	/home/vsts/work/1/s/common/logger.go:120 +0x93
github.com/Azure/azure-storage-azcopy/v10/common.(*lifecycleMgr).Exit(0xc00039e510, 0xc0cac6c870?, 0x0)
	/home/vsts/work/1/s/common/lifecyleMgr.go:376 +0xde
github.com/Azure/azure-storage-azcopy/v10/cmd.(*CookedCopyCmdArgs).ReportProgressOrExit(0xc0000bf680, {0x1296ce8, 0xc00039e510})
	/home/vsts/work/1/s/cmd/copy.go:1870 +0x42d
github.com/Azure/azure-storage-azcopy/v10/common.(*lifecycleMgr).InitiateProgressReporting.func1()
	/home/vsts/work/1/s/common/lifecyleMgr.go:592 +0x2f2
created by github.com/Azure/azure-storage-azcopy/v10/common.(*lifecycleMgr).InitiateProgressReporting
	/home/vsts/work/1/s/common/lifecyleMgr.go:564 +0xb5

How can we reproduce the problem in the simplest way? You should be able to run the above
Have you found a mitigation/solution? Seems like I can run this command without setting AZCOPY_CONCURRENCY_VALUE=128

@RenaudWasTaken, thanks for reporting this.  We will investigate and get back to you soon.

Closing this issue as a PR was merged that will be released as part of 10.23.0 which will resolve this issue.

authors: RenaudWasTaken gapra-msft seanmcc-msft

repo: Azure/azure-storage-azcopy


text:
Srinivas

Please provide a succinct description of the issue.
Repro steps
Provide the steps required to reproduce the problem

Step A

Step B

Expected behavior
Provide a description of the expected behavior.
Actual behavior
Provide a description of the actual behavior observed.
Known workarounds
Provide a description of any known workarounds.
Related information
Provide any related information

Programming language used
Links to source

Com

Comme

Not sure why this issue was created - no info was provided?

authors: mathewc spallanti

repo: Azure/azure-webjobs-sdk-script


text:
BYOB Python input bindings return empty strings

Python input bindings return empty strings. Missing write to stream for binding input case.
https://github.com/Azure/azure-webjobs-sdk-script/blob/9cc600461bb176ba768dfa2787f835693f8d6bac/src/WebJobs.Script/Description/ScriptFunctionInvokerBase.cs#L81

@mathewc - can you review this with @glennamanns when you're back?

@mathewc will take a look at this before next week

authors: christopheranderson glennamanns


text:
Simplify ParameterDeclarationSyntax to remove ParameterDefaultValueSyntax

Created from comment in #2870. It should be possible to remove ParameterDefaultValueSyntax entirely.

Let's get this done for v0.5. It'll simplify some of the code dealing with parameter declarations.

authors: anthony-c-martin majastrz

repo: Azure/bicep


text:
Types system should be able to handle more complex type definitions

Created out of discussion from #7840
Take a case like keyVault:
resource kv 'Microsoft.KeyVault/vaults@2021-11-01-preview' = {
  ...
  properties: {
    ...
    accessPolicies: [] // required if "enableRbacAuthorization" property is not provided
    enableRbacAuthorization: true // required if "accessPolicies" is not provided
  }
}

Neither accessPolicies nor enableRbacAuthorization are explicitly required, but one of them is required if the other is not included. We need a way to express this complexity in our type system and validate it.

Yes please. oneOf unions are very useful from an API design POV. These are really common in Kubernetes types, so have the capability to validate these in Bicep would be mega-valuable.

authors: alex-frankel rynowak


text:
[Question] How can we share lzs across environments?

The hierarchy model and landing zone composition prescribed by CAF makes sense to me and am using this successfully. However, I haven't seen any examples of how to properly do shared services.
For example, suppose I deploy my network hub at level2 in environment X. In level3 of environment X I can reference that hub and peer to it, all good. But now a new team comes on board, and I'd like to deploy environment Y.
How can I reference the hub from environment Y? Am I misunderstanding the purpose of environments?

If i understand you correctly, you can grab state from any environment as long as you share the same storage accounts for state.
When you call in your remote states below, you can just give the name of the file that needs to be read in. I haven't tried this but you could try using folder/name.tfstate to see if it calls in subdirectories so you maintain a bit of order.
Fundamentally the point is environments are a fictious construct and its really just a case of accessing the right state file in the storage account. Perhaps we can look at adding support for multiple storage accounts as right now everything is in one.
landingzone = {
  backend_type        = "azurerm"
  global_settings_key = "foundations"
  level               = "level2"
  key                 = "shared_services"
  tfstates = {
    foundations = {
      level   = "lower"
      tfstate = "caf_foundations.tfstate"
    }
    launchpad = {
      level   = "lower"
      tfstate = "launchpad_prod.tfstate"
    }
    launchpad_dev = {
      level   = "lower"
      tfstate = "launchpad_dev.tfstate"
    }
  }
}

Thanks for the reply @rdbartram.

you can grab state from any environment as long as you share the same storage accounts for state

Yes however this is exactly my point - for the states to be under the same set of storage accounts (launchpads), your Rover -env argument must be the same.
Suppose I have an Azure tenancy with 4 subscriptions under it, central, dev, test and prod. I first want to deploy the my foundations (level1) and shared_services (level2) to the central subscription. Lets suppose I use rover [...] -env central. Now in order to access these I also have to put all my dev/test/prod resources also under the storage accounts referred to by -env central, not ideal.
Intuitively I would want to have a separate Rover environment (and hence a separate set of storage  accounts) for each of dev/test/prod. But in this case which of them contains the shared services levels? And how can the other envs look up say the central hub if it's in a different set of launchpad storage accounts it has no reference to?
One way around this is to have a completely separate tenancy for each of dev/test/prod, implying a separate Rover env for each. Though I think during your live stream you said you generally dont like this model?
So in general my question is how are the community laying these out and what is the intended best practice for Rover environment structure.

Continued in https://github.com/aztfmod/rover/discussions/231

asjuil.uliwsd.

authors: brk3 gaoyarui rdbartram

repo: Azure/caf-terraform-landingzones


text:
[Bug]:  GraphQL type doesn't match any entity name or singular type in the runtime config.

What happened?
A bug happened! (Or I'm doing something wrong)
Error message in Postman
In postman:  POST  http://localhost:5000/graphql
CosmosDB emulator running
schema.gql:
  id: String
  iothubname: String
  Body: Body
  SystemProperties: SystemProperties
  Properties: Properties
}

type Body {
  la_activity_device_mac_date: String
  action: String
  la_activity_device_mac: String
  la_activity_index: Int
  la_activity_dev_ref: String
  la_activity_block: String
  la_activity_activity: String
  la_activity_mode: Int
  la_activity_tag: String
  la_activity_date: String
  la_activity_time: String
  la_activity_timestamp: Int
  la_activity_quantity: Int
}

type SystemProperties {
  iothubconnectiondeviceid: String
  iothubconnectionauthmethod: String
  iothubconnectionauthgenerationid: String
  iothubcontenttype: String
  iothubcontentencoding: String
  iothubenqueuedtime: String
  iothubmessagesource: String
}

type Properties {
  action: String
}

The GraphQL Query in Postman:
    activities(filter: { Body: { la_activity_device_mac: {eq: "DCDA0C56FA18"}}}){
    items {
        id
        Body {
            la_activity_device_mac_date
            la_activity_device_mac
            la_activity_index
            la_activity_dev_ref
            la_activity_tag
            la_activity_date
            la_activity_time
            la_activity_quantity
        }
        Properties {
            action
        }

        }
    }
}

This query results in the error message posted below.
If I run a query without a filter as follows, it returns the record.
Query:
    activities{
    items {
        id
        Body {
            la_activity_device_mac_date
            la_activity_device_mac
            la_activity_index
            la_activity_dev_ref
            la_activity_tag
            la_activity_date
            la_activity_time
            la_activity_quantity
        }
        Properties {
            action
        }

        }
    }
}

The result:
    "data": {
        "activities": {
            "items": [
                {
                    "id": "843e7f39-a565-4379-ae3b-99f49afba24b",
                    "Body": {
                        "la_activity_device_mac_date": "DCDA0C56FA18-2024-03-29",
                        "la_activity_device_mac": "DCDA0C56FA18",
                        "la_activity_index": 0,
                        "la_activity_dev_ref": "am1",
                        "la_activity_tag": "8D01EF395A",
                        "la_activity_date": "2024-03-29",
                        "la_activity_time": "12:24:06",
                        "la_activity_quantity": 1
                    },
                    "Properties": {
                        "action": "addActivity"
                    }
                }
            ]
        }
    }
}

Any advice as to what is being done incorrectly?
Version
Microsoft.DataApiBuilder 0.10.23+573117a7ff19d9d946343a92ad8d0f32e649954c
What database are you using?
CosmosDB NoSQL
What hosting model are you using?
Static Web Apps (SWA)
Which API approach are you accessing DAB through?
GraphQL
Relevant log output
{
    "errors": [
        {
            "message": "GraphQL type doesn't match any entity name or singular type in the runtime config.",
            "extensions": {
                "code": "BadRequest"
            }
        }
    ]
}

Code of Conduct

[X] I agree to follow this project's Code of Conduct

In the repository: src/Service.Tests/CosmosTests/QueryFilterTests.cs
There are some very complex nested filters. Do these tests pass on CosmosDb, and if so, why is my example above failing.
Any help will be appreciated.

repo: Azure/data-api-builder


text:
Porting CK-RX65N samples from GCC compiler to CCRX

■ Overview of changes
I ported the following CK-RX65N projects from a GCC compiler version project to a CCRX compiler version..

ewf_info_tx_ryz014
ewf_netx_duo_azure_iot_embedded_sdk_pnp_ryz014
ewf_netx_duo_azure_iot_embedded_sdk_ryz014
ewf_netx_duo_azure_sdk_pnp_temp_controller_ryz014

I also modified some source codes to adjust to CCRX compiler.
■ Effects on other files
I only added the CCRX projects to CK_RX65N_CCRX folder and did not change the other files.

@microsoft-github-policy-service agree company="Renesas Electronics Corporation"

repo: Azure/embedded-wireless-framework


text:
Integrate running AzGovViz as part of the starter pipeline

Collect info and check into Repo as diffs

Assigning this to Jeremiah as part of the Hydration Kit future

Adding to backlog

authors: apybar techlake

repo: Azure/enterprise-azure-policy-as-code


text:
Extend create options by chunks

Hi, I write some code for issue #26.
Please do code reviews and merge this to master repository. I had to add git submodule with azure/iotedge to avoid duplicating code written in class StringEx.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: alipek sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

Thanks a lot for the PR @alipek - could you make the changes the that @neumanndaniel is suggesting (thanks for review)?

@veyalla changes suggested by @neumanndaniel are on branch now.

/LGTM

@alipek Have you done a test build with the Dockerfile? I only got it working with the following adjustments to dotnet restore, dotnet publish and the COPY --from.
FROM microsoft/dotnet:sdk AS build-env
WORKDIR /app

# Copy csproj and restore as distinct layers
ADD ./ ./

RUN dotnet restore ./Microsoft.Azure.VirtualKubelet.Edge.Provider

# Copy everything else and build
COPY . ./
RUN dotnet publish ./Microsoft.Azure.VirtualKubelet.Edge.Provider -c Release -o out

# Build runtime image
FROM microsoft/dotnet:2.1-runtime
WORKDIR /app
COPY --from=build-env /app/Microsoft.Azure.VirtualKubelet.Edge.Provider/out .
EXPOSE 5000
CMD ["/usr/bin/dotnet", "Microsoft.Azure.VirtualKubelet.Edge.Provider.dll"]

authors: alipek msftclas neumanndaniel veyalla

repo: Azure/iot-edge-virtual-kubelet-provider


text:
Create storage versions of API types

For each object type defined in an API, defines a storage variation designed for maximum cross-version compatibility

All properties are optional
Enumerations are stored as their base type

Also, to further highlight the fact these are not intended to be externally visible, most comments are stripped.
Here's a diff between v20170901 and v20170901s of Batch:

Still to do (will be coming in separate PRs):

Conversion functions between API and Storage versions (will include generated unit tests)
Conversion functions between Storage versions (will include generated unit tests)
PropertyBag support on storage types

Abandoning this PR for now; will create a new PR once the branch is in a worthwhile state.


text:
node created by karpenter, cannot pull ACR image.

Version

Karpenter Version: v0.4.0

Kubernetes Version: v1.27.9
Expected Behavior
node can pull ACR image.
Actual Behavior
Events:
Type     Reason            Age                   From               Message

Warning  FailedScheduling  12m                   default-scheduler  0/10 nodes are available: 10 Insufficient nvidia.com/gpu. preemption: 0/10 nodes are available: 10 No preemption victims found for incoming pod..
Warning  FailedScheduling  11m                   default-scheduler  0/10 nodes are available: 10 Insufficient nvidia.com/gpu. preemption: 0/10 nodes are available: 10 No preemption victims found for incoming pod..
Normal   Scheduled         7m5s                  default-scheduler  Successfully assigned default/nginx-deployment-5bbff74cf8-79pmf to aks-t4-spot-pgpck
Normal   Nominated         8m9s (x3 over 12m)    karpenter          Pod should schedule on: nodeclaim/t4-spot-npr4z
Normal   Pulling           5m45s (x4 over 7m5s)  kubelet            Pulling image "ottoacr.azurecr.io/nginx:v1"
Warning  Failed            5m45s (x4 over 7m5s)  kubelet            Failed to pull image "ottoacr.azurecr.io/nginx:v1": rpc error: code = Unknown desc = failed to pull and unpack image "ottoacr.azurecr.io/nginx:v1": failed to resolve reference "ottoacr.azurecr.io/nginx:v1": failed to authorize: failed to fetch anonymous token: unexpected status from GET request to https://ottoacr.azurecr.io/oauth2/token?scope=repository%3Anginx%3Apull&service=ottoacr.azurecr.io: 401 Unauthorized
Warning  Failed            5m45s (x4 over 7m5s)  kubelet            Error: ErrImagePull
Warning  Failed            5m18s (x6 over 7m4s)  kubelet            Error: ImagePullBackOff
Normal   BackOff           118s (x20 over 7m4s)  kubelet            Back-off pulling image "ottoacr.azurecr.io/nginx:v1"

Only classic nodepool can pull ACR image, because the managed identity bind vmss.
But karpenter node don't have any identiy, karpentermsi don't bind it.
Steps to Reproduce the Problem
az aks update -n karpenter -g karpenter  --attach-acr ottoacr
Resource Specs and Logs
Back-off pulling image "ottoacr.azurecr.io/nginx:v1"
Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Are you using self-hosted Karpenter or NAP? If NAP, I am wondering if this could be a duplicate of https://github.com/Azure/karpenter-provider-azure/issues/197 (fixed, but not rolled out yet).

Are you using self-hosted Karpenter or NAP? If NAP, I am wondering if this could be a duplicate of https://github.com/Azure/karpenter-provider-azure/issues/197 (fixed, but not rolled out yet).

I use self-hosted Karpenter, can updated the v0.4.0 version.

I think you are right, we should be propagating at least agentpool identity into node identities. Here is a draft PR which has the updated create-values and karpenter-values-template: https://github.com/Azure/karpenter-provider-azure/pull/301. I won't be able to test it until later, but give it a try if you have a chance.

@ottodeng We are fairly certain #301 fixes this, please retest (and re-open this issue if needed)

Hello,
I have updated to version 0.5.0, and the managed identity has been added to the VM. However, the issue persists. The node created by Karpenter still cannot pull the ACR image and shows a 401 error.
Here are the steps I've taken:

Updated Karpenter to version 0.5.0.
Verified that the managed identity is added to the VM.

Despite these steps, I am still encountering the same issue. Could you please provide further assistance on how to resolve this?

(discussed in https://github.com/Azure/karpenter-provider-azure/issues/411)

authors: ATymus comtalyst ottodeng tallaxes

repo: Azure/karpenter-provider-azure


text:
Configure WebUI to work with IoT Hub service

Hi,
I have an IoT Hub running service over Azure cloud. How can I configure the pcs-remote-monitoring-webui running locally to connect directly to my IoT Hub over Azure cloud without using the IoT accelerator remote monitoring solution or docker?
Thanks.

@yaronA  pcs-remote-monitoring-webui is not intended to connect to IOT Hub directly. You need all the supporting microservices that we have in remote-monitoring.
The main reason we developed solution accelerators remote monitoring solution is to create a wrapper around IOThub and various other Azure resources, so the customers don't have to create a solution of their own.

repo: Azure/pcs-remote-monitoring-webui


text:
Release pipeline is not getting triggered when triggering the build pipeline using GitHub Actions

On a relevant change, we run a GitHub action which then trigger the azure build pipeline. We have configured the pre-deployment condition that whenever there is a build from main branch it should  trigger the release pipeline.  In our case, it is not getting triggered.  But if we manually run the build pipeline, the whole flow is working fine.
We did some debugging on our end and found out that, while firing the azure pipeline API, Actions is passing a random value "943" as a build reason in API body. It should pass one of the build reasons mentioned here.
With "943" as build reason, we are getting the below error:
System.Runtime.Serialization.SerializationException: Enum value '943' is invalid for type 'Microsoft.TeamFoundation.Build.WebApi.BuildReason' and cannot be serialized. Ensure that the necessary enum values are present and are marked with EnumMemberAttribute attribute if the type has DataContractAttribute attribute.   at System.Runtime.Serialization.EnumDataContract.WriteEnumValue(XmlWriterDelegator writer, Object value)   at System.Runtime.Serialization.XmlObjectSerializerWriteContext.WriteDataContractValue(DataContract dataContract, XmlWriterDelegator xmlWriter, Object obj, RuntimeTypeHandle declaredTypeHandle)   at System.Runtime.Serialization.XmlObjectSerializerWriteContext.SerializeWithoutXsiType(DataContract dataContract, XmlWriterDelegator xmlWriter, Object obj, RuntimeTypeHandle declaredTypeHandle)   at System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerialize(XmlWriterDelegator xmlWriter, Object obj, Boolean isDeclaredType, Boolean writeXsiType, Int32 declaredTypeID, RuntimeTypeHandle declaredTypeHandle)   at WriteBuildToXml(XmlWriterDelegator , Object , XmlObjectSerializerWriteContext , ClassDataContract )   at System.Runtime.Serialization.ClassDataContract.WriteXmlValue(XmlWriterDelegator xmlWriter, Object obj, XmlObjectSerializerWriteContext context)   at System.Runtime.Serialization.XmlObjectSerializerWriteContext.WriteDataContractValue(DataContract dataContract, XmlWriterDelegator xmlWriter, Object obj, RuntimeTypeHandle declaredTypeHandle)   at System.Runtime.Serialization.XmlObjectSerializerWriteContext.SerializeWithoutXsiType(DataContract dataContract, XmlWriterDelegator xmlWriter, Object obj, RuntimeTypeHandle declaredTypeHandle)   at System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerialize(XmlWriterDelegator xmlWriter, Object obj, Boolean isDeclaredType, Boolean writeXsiType, Int32 declaredTypeID, RuntimeTypeHandle declaredTypeHandle)   at System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerializeReference(XmlWriterDelegator xmlWriter, Object obj, Boolean isDeclaredType, Boolean writeXsiType, Int32 declaredTypeID, RuntimeTypeHandle declaredTypeHandle)   at WriteBuildCompletedEventToXml(XmlWriterDelegator , Object , XmlObjectSerializerWriteContext , ClassDataContract )   at System.Runtime.Serialization.ClassDataContract.WriteXmlValue(XmlWriterDelegator xmlWriter, Object obj, XmlObjectSerializerWriteContext context)   at System.Runtime.Serialization.XmlObjectSerializerWriteContext.WriteDataContractValue(DataContract dataContract, XmlWriterDelegator xmlWriter, Object obj, RuntimeTypeHandle declaredTypeHandle)   at System.Runtime.Serialization.XmlObjectSerializerWriteContext.SerializeAndVerifyType(DataContract dataContract, XmlWriterDelegator xmlWriter, Object obj, Boolean verifyKnownType, RuntimeTypeHandle declaredTypeHandle, Type declaredType)   at System.Runtime.Serialization.XmlObjectSerializerWriteContext.SerializeWithXsiType(XmlWriterDelegator xmlWriter, Object obj, RuntimeTypeHandle objectTypeHandle, Type objectType, Int32 declaredTypeID, RuntimeTypeHandle declaredTypeHandle, Type declaredType)   at System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerialize(XmlWriterDelegator xmlWriter, Object obj, Boolean isDeclaredType, Boolean writeXsiType, Int32 declaredTypeID, RuntimeTypeHandle declaredTypeHandle)   at System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerializeReference(XmlWriterDelegator xmlWriter, Object obj, Boolean isDeclaredType, Boolean writeXsiType, Int32 declaredTypeID, RuntimeTypeHandle declaredTypeHandle)   at WriteServiceEventToXml(XmlWriterDelegator , Object , XmlObjectSerializerWriteContext , ClassDataContract )   at System.Runtime.Serialization.ClassDataContract.WriteXmlValue(XmlWriterDelegator xmlWriter, Object obj, XmlObjectSerializerWriteContext context)   at System.Runtime.Serialization.XmlObjectSerializerWriteContext.WriteDataContractValue(DataContract dataContract, XmlWriterDelegator xmlWriter, Object obj, RuntimeTypeHandle declaredTypeHandle)   at System.Runtime.Serialization.XmlObjectSerializerWriteContext.SerializeWithoutXsiType(DataContract dataContract, XmlWriterDelegator xmlWriter, Object obj, RuntimeTypeHandle declaredTypeHandle)   at System.Runtime.Serialization.DataContractSerializer.InternalWriteObjectContent(XmlWriterDelegator writer, Object graph, DataContractResolver dataContractResolver)   at System.Runtime.Serialization.DataContractSerializer.InternalWriteObject(XmlWriterDelegator writer, Object graph, DataContractResolver dataContractResolver)   at System.Runtime.Serialization.XmlObjectSerializer.WriteObjectHandleExceptions(XmlWriterDelegator writer, Object graph, DataContractResolver dataContractResolver)   at System.Runtime.Serialization.XmlObjectSerializer.WriteObject(XmlDictionaryWriter writer, Object graph)   at System.Runtime.Serialization.XmlObjectSerializer.WriteObject(Stream stream, Object graph)   at Microsoft.ServiceBus.Messaging.BrokeredMessage..ctor(Object serializableObject, XmlObjectSerializer serializer)   at Microsoft.VisualStudio.Services.Cloud.ServiceBusPublishHelper.GetBrokeredMessage(MessageBusMessage input, String messageBusIdentifier) in D:\v2.0\P1\_work\5\s\Vssf\Sdk\CloudServer\ServiceBus\ServiceBusPublishHelper.cs:line 54   at Microsoft.VisualStudio.Services.Cloud.ServiceBusPublishHighAvailabilityConnection.PublishMessagesInternal(IVssRequestContext requestContext, String messageBusIdentifier, MessageBusMessage[] messagesToPublish, List 1 messages, Boolean publishToSecondary) in D:\v2.0\P1\_work\5\s\Vssf\Sdk\CloudServer\ServiceBus\ServiceBusPublishHighAvailabilityConnection.cs:line 165   at Microsoft.VisualStudio.Services.Cloud.ServiceBusPublishHighAvailabilityConnection.PublishHelper(IVssRequestContext requestContext, String messageBusIdentifier, MessageBusMessage[] messagesToPublish, Boolean publishToSecondary) in D:\v2.0\P1\_work\5\s\Vssf\Sdk\CloudServer\ServiceBus\ServiceBusPublishHighAvailabilityConnection.cs:line 141
We created a test pipeline and fired the build API using "triggered" as build reason and our whole pipeline is working fine.
Possible Solution: Instead of just passing a random value as build reason, pass a valid reason string as mentioned in this doc.

The reason why are we getting 943 is that this action is not using the updated azure-devops-node-api version. Enum value for "triggered" got changed in this PR - https://github.com/microsoft/azure-devops-node-api/pull/368/files?file-filters[]=.json&file-filters[]=.ts#diff-2400cd68b4f3536f59bb07ee7f99a260ce1cd558d0bfe051d6f2c8f9fa9519b8L1129.
We should update this action to use the latest API to avoid such misunderstandings.
Since the triggered value is 943, we are passing this value here to the pipeline API. That is causing the issue to trigger the release pipeline. Instead of passing BuildInterfaces.BuildReason.Triggered as reason, we can just pass "triggered" to resolve this issue.

repo: Azure/pipelines


text:
Publish as npm module

Look at making queryjs an npm module so it can be a dependency for the JS SDK build process

Any update on this?

Refactoring will wait a bit, the new Node.js Server (https://github.com/Azure/azure-mobile-apps-node/blob/master/package.json#L23) and the next JS SDK (https://github.com/Azure/azure-mobile-services/blob/offline/sdk/Javascript/package.json#L16) now take this via package.json to here.
@danderson00 and @shrishrirang can probably outline the current plans now for JS consumption.

authors: joewood phvannor

repo: Azure/queryjs

license_source: bigquery


text:
Fixed ordering for logical volume creation.

Problem
Merge the code for 3.8.3.1 release in sdaf. This release contains a fix that plagues single server installations due to the faulty order of the lv creation within vg_sap. lv_usrsapinstall was being created before the other lvs which was resulting in the entire space allocated to the vg_sap being allocated to the lv_usrsapinstall. This was failing the other lv creations.
Solution
lv_usrsapinstall creation is now made the last lv created so that all remaining space is allocated to it post the creation of lv_sapmnt and lv_usrsap.
Tests
Deploy single server systems using by the sdaf release 3.8.3.1 from ACSS portal. The create infra procedure should succeed.
Proof of Testing
Query to view the results for createinfra test:
https://waasservices.eastus.kusto.windows.net/WaaSKustoDB?query=H4sIAAAAAAAEAG2RTUsDQQyG7%2FsrwpwqbGFZQUGpILXCHvzAFXoQD2kmtaO7M0sm21Lwxzur0A8qzCFD3id5k8wR6wdWcTRbs9fsGzYrFoZnYXKRX13LtWLbwQ1YVNb0H5VFeT4uLsblJRTF1e8724F1v4gkrlMXfGXBeRhVXlk8NoepuCdeOIZeiJOagld0PoJBuwpkdpq71NfHgXsz8yBfyyZsHrFl8w6TCZipcDJX%2BaVgVOlJe%2BH%2F4aeOBQcDaSod8MGgqXsiZsvW5GDu0TUpGgx2Ej6Z9GQb%2BVHNmUiQabD85yiHmmXtiG9J3drptrL5wZBD%2FJHAHPb51CqIZYHF9kh5cgWMdJ1lP74BzVW1AQAA&web=0
WaaSMetricEvent
| where PreciseTimeStamp > datetime(2023-06-27 00:00:00)
| where SubscriptionId in (InternalSubscriptions)
| where ResourceId contains "adhoc"
| where Dimensions["WorkflowName"] == "CreateInfrastructure"
| where Dimensions["OperationState"] in ("Succeeded", "Failed")
| project PreciseTimeStamp, Dimensions["ErrorCodeName"], ServiceActivityId, ResourceId, Region, ActivityId
| order by ResourceId, PreciseTimeStamp asc;
Query to view the results for install test:
https://waasservices.eastus.kusto.windows.net/WaaSKustoDB?query=H4sIAAAAAAAEAG2RTUsDMRCG7%2FsrhpwqbGFZQUGpIFphD37gCj2Ih2ky2mg2WSazWwr%2BeBOFflAhhwnv%2B%2BSdmSwQ23sStno%2BkpfiG9YrYoInJm0jvdiOWsGuhyswKCTpPqmr%2BnRanU3rc6iqi99zsgXbYRk1215s8I0B62HSeCH26PaluCOeKYaBNSW3Dl7Q%2BggKzSpotfXcplwfM%2FeqFoG%2F3l1YP2BH6g1mM1CNj4LOZcUFNP9zjz0x5uw0kGQy96baQWsiQ0aVoO7QulTl3noOn6TlaBHlwZtz5sA3wdBfMyW0xKPVdK3FjlY2jSn35sv1RwJL2OkpKrAhhuXmwHn0ARj1ZVH8AGqZJGKwAQAA&web=0
WaaSMetricEvent
| where PreciseTimeStamp > datetime(2023-06-27 00:00:00)
| where SubscriptionId in (InternalSubscriptions)
| where ResourceId contains "adhoc"
| where Dimensions["WorkflowName"] == "InstallWorkload"
| where Dimensions["OperationState"] in ("Succeeded", "Failed")
| project PreciseTimeStamp, Dimensions["ErrorCodeName"], ServiceActivityId, ResourceId, Region, ActivityId
| order by ResourceId, PreciseTimeStamp asc;
1 SCS failure observed which was transient and succeeded on retry
These are the other errors observed which are not related to ansible:
InstallHostUnreachable
InstallServiceError
AzureVMIsNotInSupportedProvisioningState

Closing this PR as this does not fix the issue. The sapinstall logical volume needs to created at the very end for robust volume group setup. Will create a PR for this shortly.

repo: Azure/sap-automation


text:
Restrict access to specifc users

By default everything seems to be public facing, but I'd like to expose my static web app to select people within my organization. I've had a look at the docs and I'm not quite sure how I would block access to all users except the ones I explicitly invite via the Role management blade in the Azure portal.
I've added the following routes.json file to the app_location folder (= root folder of my Hugo app):
{
    "routes": [
        {
            "route": "/login",
            "serve": "/.auth/login/aad"
        },
        {
            "route": "/*",
            "allowedRoles": [
                "authenticated"
            ]
        }
    ],
    "platformErrorOverrides": [
        {
            "errorType": "Unauthenticated",
            "statusCode": "302",
            "serve": "/login"
        }
    ]
}

... but it seems to completely ignore  it 🤷.
This will also allow anyone who is able to authenticate against our AAD tenant, not just the people I invited.
I'm using the Azure DevOps task to build and deploy my Hugo app, should that matter.

The built-in auth allows anyone from the several identity providers we support authenticate and they are all assigned the authenticated role. If you want to lock down to certain users, you need to generate invitations to add a user to a custom role. You then use that custom role name (instead of authenticated) in your routes config to lock down the site.
We are working on a feature to allow you to configure AAD auth to lock it down to users in your tenant, but that’s not yet available.

Ok, so I've added a user and gave him a custom role docsreader. Any invitees seem to get the anonymous and authenticated role as well.  I've opted to use my personal Google account to test things out, instead of AAD, but the method should be the same:

I want to block access for everyone, accept for the administrator ( = me ) and anyone who invited using the Role  Management function.
My routes.json:
{
    "routes": [
        {
            "route": "/*",
            "allowedRoles": ["docsreader","administrator"]
        },
        {
            "route": "/login",
            "rewrite": "/.auth/login/google"
        },
        {
            "route": "/.auth/login/aad",
            "statusCode": "404"
        },
        {
            "route": "/.auth/login/github",
            "statusCode": "404"
        },
        {
            "route": "/.auth/login/twitter",
            "statusCode": "404"
        }
    ],
    "responseOverrides": {
        "401": {
            "redirect": "/login",
            "statusCode": 302
        }
    }
}

This somehow still isn't working, anyone is still able to access it.
My repo:
.
+-- azure-pipelines.yml
+-- docs
|   +-- routes.json
|   +-- assets
|   +-- content
|   +-- static
|   +-- themes
|   +-- ...

My pipeline:
trigger:
  - v1.0

pool:
  vmImage: ubuntu-latest

steps:
  - task: AzureStaticWebApp@0
    inputs:
      app_location: "docs"
      app_build_command: "hugo"
      api_location: "api"
      output_location: "public"
    env:
      azure_static_web_apps_api_token: $(deployment_token)
      HUGO_ENV: production
      HUGO_VERSION: "0.74.3"

Hey @smholvoet , one reason this could happen would be if the routes.json file is not being copied to the output location. If you ensure it is copied over to the public folder then it may unblock you.
You can also try the newer (as routes.json is our older, preview config file) staticwebapp.config.json file which if placed in the app_location will automatically be uploaded for you.

authors: anthonychu miwebst smholvoet

repo: Azure/static-web-apps


text:
chore: repository governance

Repository governance update
This PR was automatically created by the AVM Team hive-mind using the grept governance tool.
We have detected that some files need updating to meet the AVM governance standards.
Please review and merge with alacrity.
Grept config source: git::https://github.com/Azure/Azure-Verified-Modules-Grept.git//terraform
Thanks! The AVM team :heart:

Supersceeded by #33


text:
chore: repository governance

Repository governance update
This PR was automatically created by the AVM Team hive-mind using the grept governance tool.
We have detected that some files need updating to meet the AVM governance standards.
Please review and merge with alacrity.
Grept config source: git::https://github.com/Azure/Azure-Verified-Modules-Grept.git//terraform
Thanks! The AVM team :heart:

Supersceeded by #56


text:
Cannot add custom policy definitions using \lib

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Versions

terraform: 1.0.4
azure provider: 2.71
module: 0.4.0
Description
I have added the lib folder for custom archetypes. I also have some custom policy assignments there. I ran into a situation where I wanted to create custom policy definition. I have named it "policy_definition_my_custom_policy.json". To my surprise, terraform plan does not do anything or "recognize" the policy definition. Is this by design?  And if I wanted to add one or two custom policies, how to do it?

Steps to Reproduce

create lib folder in the repo
add "policy_definition_custom_policy.json"
run terraform plan

Expected behaviour
I'd expect the module to grab the policy definition the same way as it does with custom policy assignments.

Screenshot

Hey @tohov,
Have you added the policy key name to an archetype definition in the policy definitions list that is used for one of your management groups?
Example here: https://github.com/Azure/terraform-azurerm-caf-enterprise-scale/blob/edd4ec093c5d3aa2468cf58dc867ae20e0698254/modules/archetypes/lib/archetype_definitions/archetype_definition_es_root.tmpl.json#L13
Let us know 👍

Nice. No I haven't. 😄 My bad. Thanks for the help @jtracey93! It works like a charm now. No bugs.

Superb! Not a problem at all and glad to help 👍
Will close this issue off now 👍

I would also recommend that you check out our feature which enables you to Expand built in archetype definitions.
This should significantly reduce any maintenance overhead on managing custom archetype definitions.

Note that you can also exclude specific items from the built-in archetype definitions too (guidance in the same Wiki page).

authors: jtracey93 krowlandson tohov


text:
Go dep issues

Running in vscode.  Using devcontainer GO 1.16.  I tried to sort this out but couldn't.  Would be nice to include a working .devcontainer for easy building.  I ended up build #55 by hand.
# github.com/coreos/etcd/clientv3/balancer/resolver/endpoint
/go/pkg/mod/github.com/coreos/etcd@v3.3.17+incompatible/clientv3/balancer/resolver/endpoint/endpoint.go:114:78: undefined: resolver.BuildOption
/go/pkg/mod/github.com/coreos/etcd@v3.3.17+incompatible/clientv3/balancer/resolver/endpoint/endpoint.go:182:31: undefined: resolver.ResolveNowOption
# github.com/coreos/etcd/clientv3/balancer/picker
/go/pkg/mod/github.com/coreos/etcd@v3.3.17+incompatible/clientv3/balancer/picker/err.go:37:44: undefined: balancer.PickOptions
/go/pkg/mod/github.com/coreos/etcd@v3.3.17+incompatible/clientv3/balancer/picker/roundrobin_balanced.go:55:54: undefined: balancer.PickOptions
make: *** [Makefile:8: install] Error 2

Fixed on #81

authors: Nepomuceno pearcec

repo: Azure/terraform-azurerm-naming


text:
[Bug][tcgc]: Does not honor encodedName correctly

Describe the bug
I try to migrate Search service to use tsp.
Here are the converted tsps: https://github.com/Azure/azure-rest-api-specs/blob/search_tsp/specification/search/Azure.Search
You can see in https://github.com/Azure/azure-rest-api-specs/blob/search_tsp/specification/search/Azure.Search/models-service.tsp#L6446 I have
@pagedResult
model ListIndexesResult {
    /**
     * The indexes in the Search service.
     */
    @visibility("read")
    @items
    @encodedName("application/json", "value")
    indexes: SearchIndex[];
}

I expected that in the generated code, it should attempt to parse "value". But unfortunately it generated deserialized["indexes"]
https://github.com/Azure/azure-sdk-for-python/blob/search_11.5.0_tsp/sdk/search/azure-search-documents/azure/search/documents/_generated/operations/_operations.py#L3730
Reproduction
Try to generate Python code using https://github.com/Azure/azure-rest-api-specs/blob/search_tsp/specification/search/Azure.Search
Checklist

[x] Follow our Code of Conduct
[x] Check that this issue is about the Azure libraries for typespec. For bug in the typespec language or core libraries file it in the TypeSpec repo
[x] Check that there isn't already an issue that request the same bug to avoid creating a duplicate.
[x] The provided reproduction is a minimal reproducible example of the bug.

But another related issue is TCGC does not take renaming for the path. Will fix it.

repo: Azure/typespec-azure


text:
TypeScript RLC multipart documentation

Document existing support for multipart in TypeScript RLC generation

No changes needing a change description found.

authors: azure-sdk timovv


text:
What is the next thing?

I heard that this project has been deactivated in favor of a different approach. What is the next thing?

VDC modules and governance best practices got merged into Enterprise Control Plane (ECP).

authors: Izzmo jorgecotillo

repo: Azure/vdc


text:
Create Kubernetes objects from the extension

I would like to be able to create Kubernetes objects from the extension. The experience would be as follows:

right click the type of object you want to create from the extension pane
click create
the editor opens YAML file, pre-populated with the template for the object type I want to create

Of course it would not work for all objects in the tree (such as Nodes), but for anything that can be created with a declarative file, it should work.
Thoughts?

What's the user story for this, as a developer? Can you write the scenario out? Radu might want that, or Ralph, but for prioritization I'd love to hear the "common developer workflow" story.

User story:

Radu is working on a simple application. Suddenly, he needs to add credentials for a database connection string, and he heard that secrets might be the way to go in Kubernetes. As he never created a secret before, or is unfamiliar / unwilling to write Kubernetes YAML by hand, from scratch, this would spare Radu the Google search for Kubernetes secret, then the copy-paste from the Kubernetes docs, a right-click -> create secret -> pre-populated YAML file might be the experience he needs.

Now replace  with any Kubernetes object. Does it make sense?
Thanks!

yes, more generally, I would start with specific objects and work up from there. Secrets/configmaps are good starts. :-)

Hey guys, would be also nice to edit ConfigMaps and Secrets from the extension. :)

Raised a new issue #579 for the specific case of ConfigMaps and Secrets because those are more specific, we have more concrete use cases and it's a bit more bounded.

authors: itowlson radu-matei squillace vdefeo

repo: Azure/vscode-kubernetes-tools


text:
We should have some way of detecting that our dependencies need to be upgraded

We should at the very least have our dependencies upgraded to the minimum recommended version, if not the latest available.

not meeting any specific bar

authors: brentschmaltz mafurman


text:
WIP: Add Continuous Integration Testing & PR Validation

We gotta run our tests and enforce style guidance!

We're running into some permissions trouble. Until that is resolved, I've opened #18 which may offer a path of less resistance.

TravisCI seems to fulfill all of our needs for the time being. I'm going to close this in favor of #18.


text:
Adding Dialog Prompts to Remove Smartcard (at times that make sense)

Summary
Why the CBA dialog flow is being changed again
Feedback from testers of the NFC dialog flow showed that it would be beneficial to have dialog prompts that let the user know when they should remove their smartcard. For example, after the first NFC tap, once the client determines that there are/are not any errors and have read off the certificates, we should let the user know that they do not need to hold their smartcard to the device anymore.
There are two main places that a user should see such a dialog for NFC:

After initially tapping in order to retrieve certs from the smartcard (before the cert picker)
After tapping in order to confirm PIN and sign with a private key

There is a potential use for the removal prompt dialog in the USB flow:
-After signing with the private key, and other processes involving the smartcard have completed.
- UPDATE: Upon positive feedback, this is included in this PR.
Including this prompt in the USB flow could potentially decrease the amount of configuration change related errors, depending upon when most users will plug in their smartcard. If they plug in their smartcard only during the times our activity is running (when the authentication WebView is showing), then implementing such a change will keep the configuration changes occurring within our activities, which have the appropriate configChanges flags. The flip side of this is that a user who plugged in their smartcard within the host app activity will experience a configuration change within the host app activity upon our activities destroying. Logic has been added to detect if this is the case, and no removal prompt will be displayed if a smartcard is still detected by the end of the flow.
What main changes were made

Added a new ShowSmartcardRemovalPromptDialog method to DialogHolder to be called when we want a user to remove their smartcard (after NFC taps, before sending auth results, for NFC and USB).

corresponding Dialog is SmartcardRemovalPromptDialog

Made changes to WebViewAuthorizationFragment and AzureActiveDirectoryWebViewClient to set a callback that sends the auth result and finishes (in order to delay sending results until the user removes their smartcard). This callback will be invoked immediately if the cba challenge handler is null, and otherwise will be invoked based on the status of the challenge handler. A callback is needed here in order to make sure the user removes their smartcard before our library's activity destroys.
onPausedSmartcardDiscovery renamed to prepForNextUserInteraction. Though we are stopping discovery in NFC's case, the emphasis is put on giving some time to prepare for an expected user interaction, which would be pausing for the user to remove their smartcard (in NFC case; USB implementation of this method just invokes callback).
Got rid of the onCancelCba() method in smartcard dialogs. This was originally used to run logic when a smartcard was unexpectedly unplugged. To keep more consistent with the connectionCallback variable of the CBA managers, I created a DisconnectedCallback local variable that should be set for unexpected/expected unplugging on the USB side, or expected removal on the NFC side.  Actually ended up keeping the onCancelCba() logic, as I was overthinking things. The method was renamed to be onUnexpectedUnplug() to be more specific and to avoid confusion with the cancelCallbacks.

Was able to get rid of callback generics for AbstractSmartcardCertBasedAuthManager.

Consolidated the local cancelCallback interfaces attached to each dialog into a universal ICancelCbaCallback. Done in https://github.com/AzureAD/microsoft-authentication-library-common-for-android/pull/1972#pullrequestreview-1316779182
Consolidated the local dismissCallback interfaces attached to each dialog into a universal IDismissCallback Done in https://github.com/AzureAD/microsoft-authentication-library-common-for-android/pull/1975
-Added a disconnect method to the NFC manager that should disconnect a smartcard and wait for the user to remove the smartcard.

Testing
The Units tests associated with CBA have been updated to check for SmartcardRemovalPromptDialog being shown at correct times.
I've manual tested different scenarios to make sure that the changes made to AzureActiveDirectoryWebVIewClient does not interfere with regular auth. I'll continue to manual test to make sure all bases that I can think of are covered.

I want to note that the removal prompt dialog is no longer being dismissed automatically in the NFC flow because of an issue I am investigating, where certain Android devices are getting blocked when the removal method is being called.
A dismiss button was added to the dialog so that the user does not get blocked here. I'm going to continue to investigate this issue with YubiCo so that the remove() YubiKit method can be brought back in upon next release.


text:
How to handle client clock being skewed, not synced with actual time?

Library

[ ] msal@1.x.x or @azure/msal@1.x.x
[X] @azure/msal-browser@2.11.1
[ ] @azure/msal-node@1.x.x
[ ] @azure/msal-react@1.x.x
[ ] @azure/msal-angular@0.x.x
[ ] @azure/msal-angular@1.x.x
[ ] @azure/msal-angular@2.x.x
[ ] @azure/msal-angularjs@1.x.x

Description
Is there any guidance on how to deal with client clock being skewed. Eg imagine a scenario where the client's clock thinks it is currently 18:00 UTC and has a cached token that is valid until 19:00 UTC. The client app then makes an api call using the cached token since it thinks it is still valid. However, the actual time is 20:00 UTC, and the server rejects the request with a 401 because the token is expired (the token expires around 19:00 UTC, and the current time is 20:00 UTC)
We see evidence of this happening in our current application, where the client side log's show the request being made at an incorrect time like 18:36 UTC (the timestamp is from client side clock) but when we search up the request in backend logs, the time is 19:32 UTC and the client had passed a token that is valid until 19:00 UTC.
I was wondering if this scenario has been encountered before, if so, is there any guidance on how to handle such situations? In my opinion, it looks like the solution boils down to the client app being aware that the time is incorrect and should react to that (display some error message or force fetch a new token)
Thanks!
Source

[X] Internal (Microsoft)
[ ] Customer request

@thoo1 Are we talking about access tokens or refresh tokens? If it's access tokens, the token expiration date returned in the response should be calculated based on the client's time. Tokens are valid by default for 1 hour from when they are issued. When you call acquireTokenSilent, it will check the current expiration time based on the client side calculation and renew the token if it is past this time.

We see evidence of this happening in our current application, where the client side log's show the request being made at an incorrect time like 18:36 UTC (the timestamp is from client side clock) but when we search up the request in backend logs, the time is 19:32 UTC and the client had passed a token that is valid until 19:00 UTC

This shouldn't be the case. So lets say the client's clock is skewed 1 hr before the server's clock (using your example). If the client acquired a token at 18:00, the token should have a exp claim, assuming 1hr token lifetime, based on the server's clock of 20:00 in its claims (19:00 + 1hr)  (double checking this, but pretty sure that this is how it works), but MSAL will set the expiration date to 19:00. When the token is validated, it should be based on the time of the server's clock.
Are you seeing different behavior? Check the exp claim of the token.

In the above image you can see there is a slight difference in the expiration times - this is because the expiration used by MSAL is calculated separately than the one from the auth server.
There is a tokenRenewalOffsetSeconds configuration option but this is usually to offset seconds or minutes, not an entire hour.

The client app then makes an api call using the cached token since it thinks it is still valid

How does this happen? The client should be using acquireTokenSilent to retrieve a new token to make sure that the token is still valid. If it isn't valid, acquireTokenSilent should be renewing the token. acquireTokenSilent does this based on the client clock, not the exp claim in the token.

the server rejects the request with a 401

By "the server" do you mean the resource server? How does the token validation happen?

Hi @pkanher617 thanks for taking your time to look into this. I can answer some of your questions

The client app then makes an api call using the cached token since it thinks it is still valid
How does this happen? The client should be using acquireTokenSilent to retrieve a new token to make sure that the token is still valid. If it isn't valid, acquireTokenSilent should be renewing the token. acquireTokenSilent does this based on the client clock, not the exp claim in the token.

Sorry for the confusion in my wording, when I said client app I meant the msal-library, we do just use acquireTokenSilent and don't do anything special on top. I meant to express that msal-browser library thought the token was still valid (as it looks at the client clock, not the exp claim) and our code sent the request with the token. Since the client clock was about an hour behind, the token in reality was expired and the server had rejected it.

This shouldn't be the case. So lets say the client's clock is skewed 1 hr before the server's clock (using your example). If the client acquired a token at 18:00, the token should have a exp claim, assuming 1hr token lifetime, based on the server's clock of 20:00 in its claims (19:00 + 1hr) (double checking this, but pretty sure that this is how it works), but MSAL will set the expiration date to 19:00. When the token is validated, it should be based on the time of the server's clock.

I understand what you're saying, and I agree. If the client clock is consistently 1 hour behind the server, that is fine as long as long as the server validates the token using the exp claim (which will be based on server time). Let me validate this with the backend team, that they are using the exp claim (I don't see any other claim that could be used).
There is one small nuance. What happens if the client clock is not consistently 1 hour behind? I'll see if I can validate this in the logs first before I raise this topic further
Thanks for your answer! I'll do a bit more digging

I think we need to be really clear here about which actors are performing specific actions. There is a client, which calls ATS to get tokens. This client should never be sending expired tokens. ATS always calculates lifetimes based on the client's clock, so it should never return a token that has passed the lifetime.
There is a resource server, which validates the access token. If this service is out of sync, there are going to be issues no matter what.
Finally, there is the auth server which is the one that accepts the login credentials and issues tokens.

Since the client clock was about an hour behind, the token in reality was expired and the server had rejected it.

If the token was used within an hour of when it was issued, the only way it would be expired on the service but not the client would be if the client clock is somehow moving slower than the server's clock, or the resource service is also skewed. Expiration for ATS is calculated based on lifetime, not exact timestamps.

There is one small nuance. What happens if the client clock is not consistently 1 hour behind? I'll see if I can validate this in the logs first before I raise this topic further

Again, the expiration timestamp claim of the token is always based on the auth server's clock, not the client. It depends on the resource service validating the token. If that resource service is out of sync, there isn't much we can do. As I said above, the only way the client would be sending expired tokens is if the clock is somehow moving at a different rate than the auth server's clock. Meaning, 1hr on the service is >1hr on the client.

Closing as this doesn't appear to be a problem with the library. Please open a new issue if you determine there is any action we need to take, thanks!