text:
[VULN] Security vulnerability detected in com.azure:azure-storage-blob (via transient dependency) [CVE-2023-44487]

Hello,
Since it is publicly available information, so reporting it here as an issue.
The latest available azure-storage-blob library (12.24.0, ref: https://mvnrepository.com/artifact/com.azure/azure-storage-blob/12.24.0) includes a known vulnerability via transient dependency.
Vulnerability description:

SNYK report: https://security.snyk.io/vuln/SNYK-JAVA-IONETTY-5953332
CVE: https://www.cve.org/CVERecord?id=CVE-2023-44487

How to fix:
Quote from SNYK vulnerability description:

Upgrade io.netty:netty-codec-http2 to version 4.1.100.Final or higher.

Dependency tree
Dependency tree of the com.azure:azure-storage-blob library with transient dependencies that contains vulnerability:
[INFO] +- com.azure:azure-storage-blob:jar:12.24.0:compile
[INFO] |  +- com.azure:azure-core-http-netty:jar:1.13.6:compile
[INFO] |  |  +- io.netty:netty-codec-http2:jar:4.1.97.Final:compile

Can you, please, fix this?

Thank you for reporting this @dbelyaev.
The Azure SDKs don't use the affect code in Netty, HTTP server, so they aren't affected by the CVE. With that said we will be updating to the latest version of Netty so the CVE warnings go away.

@dbelyaev we've updated out dependency and which will be reflected in azure-storage-blobs next release. Echoing what @alzimmermsft stated our SDKs were not affected by the CVE. Thank you.

authors: alzimmermsft dbelyaev jairmyree


text:
Add tests to use OkHTTP

Most of our client libraries use azure-core-http-netty as the Http client by default. So, all our tests run using netty but we should have tests that swap out azure-core-http-netty with azure-core-http-okhttp and run the tests so we ensure all changes are tested with both http clients.

Should we extend the test framework to handle running permutations at a test suite level where we could plug in Netty or OkHttp for an entire test run?

Customer-reported issue for OkHttp client - https://github.com/Azure/azure-sdk-for-java/issues/8183

@sima-zhu could you assist with this issue please?

Looking this

Testing over whether OkHttp and Netty populate different request and response in record-json files.

Spot check and confirmed from test json and Anu, Okhttp and netty generate the same http request and response.
Current options:

Parameterized run on okhttp and netty. Not very ideal since we takes too long to run netty test suite. It will introduce more burden to each build if we setup two test suites.
(Optimal) Add new env variable and run different http client on different pipelines, which will not change the overall structure too much.
For this approach, need to involve eng team, since we do not support different test env sets for service. The assumption is each live test only based on one test env set. @danieljurek  is helping to look into the feasibility.

Fixed in #8679

source: gharchive/issue

authors: alzimmermsft joshfree sima-zhu srnagar

repo: Azure/azure-sdk-for-java

license: MIT

license_type: permissive

license_source: github-api


text:
Support AllIgnoreCase and IgnoreCase keywords in Cosmos

New feature, support AllIgnoreCase and IgnoreCase keywords in Cosmos database to ignore case query.

Fixed issue: https://github.com/Azure/azure-sdk-for-java/issues/12721

/azp run java - cosmos - ci

/azp run java - cosmos - ci

@kushagraThapar, Ok, i will add now.

source: gharchive/pull-request

authors: moarychan saragluna


text:
[Automation] Generate Fluent Lite from datafactory#package-2018-06

[Automation] Generate Fluent Lite from datafactory#package-2018-06

/azp run java - datafactory - tests.mgmt

authors: azure-sdk weidongxu-microsoft


text:
[AutoPR] cognitiveservices/data-plane/Face

Created to accumulate context: cognitiveservices/data-plane/Face

Build output:

Build output:


text:
ServiceBus: POM dependency and remove unneccesary version

PR is after Service bus SDK lib migration: no changes #3428
It includes POM file changes:

Module dependency
Remove unnecessary version
Enabled Spotbugs and Checkstyle

fixes #3429

Can one of the admins verify this patch?

Please get the tests passing, and then re-request a PR review then. Thanks.

@yvgopal  @JonathanGiles  @conniey  All changes are set. WIll create SpotBugs P1 and SpotBugs P2 PRs after this PR. Thank you for your time.

authors: JonathanGiles azuresdkci mssfang


text:
Prepare Tables Patch Release September 2023

Prepare Tables patch release for September 2023.

API change check
API changes are not detected in this pull request.

authors: azure-sdk jairmyree


text:
Increment versions for cosmos releases

Increment package versions for cosmos releases

API change check
API changes are not detected in this pull request.


text:
mgmt, local generation for AppService package-2023-12

Description
Please add an informative description that covers that changes made by the pull request and link all relevant issues.
If an SDK is being regenerated based on a new swagger spec, a link to the pull request containing these swagger spec changes has been included above.
All SDK Contribution checklist:

[ ] The pull request does not introduce [breaking changes]
[ ] CHANGELOG is updated for new features, bug fixes or other significant changes.
[ ] I have read the contribution guidelines.

General Guidelines and Best Practices

[ ] Title of the pull request is clear and informative.
[ ] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[ ] Pull request includes test coverage for the included changes.

API change check
API changes are not detected in this pull request.


text:
SAS Token Generation with TokenCredential

Is your feature request related to a problem? Please describe.
In building out the Azure Functions Plugin for the Serverless Framework, we'd like to simplify the process of authentication for deployment of Azure Functions. Users can log in via interactive login or by using a service principal, which is then used to create a TokenCredential. It seems like most operations are available with the blob storage SDK, but it seems like creating a SAS token requires a SharedKeyCredential.
Describe the solution you'd like
Is there any way to generate a blob SAS token with a TokenCredential? I'd like to use the SAS token to update the Azure Functions WEBSITE_RUN_FROM_PACKAGE setting to point to a newly deployed zip file.
Describe alternatives you've considered

Requiring users to supply account key. This would present a challenge as the storage account creation is part of the initial deployment process. We'd like to keep the initial experience of using the framework to be as easy and streamlined as possible.

Discovered a solution. For posterity, you can get keys via @azure/arm-storage in this way:
// this.credentials is from the AuthResponse received when authenticating via Service Principal
// or interactive login
const context = new StorageManagementClientContext(this.credentials, this.subscriptionId)
const storageAccounts = new StorageAccounts(context);
const keys = await storageAccounts.listKeys(this.resourceGroup, this.storageAccountName);
return keys.keys[0].value;

repo: Azure/azure-sdk-for-js


text:
[AutoPR datafactory/resource-manager] typo: datafactory/resource-manager/Microsoft.DataFactory

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/4573

This PR has been merged into https://github.com/Azure/azure-sdk-for-js/pull/578


text:
Please Make AzureKeyVaultConfigurationSource public OR (Better) add an Insert Extension Method that takes an index

Library or service name.
Azure.Extensions.AspNetCore.Configuration.Secrets
Is your feature request related to a problem? Please describe.
The problem right now is that the AddAzureKeyVault method appends the keyvault configuration to the list of configuration sources which by default already has command line, environment variables and config files (to name a few) automatically added by AspNetCore.  In virtually every case AzureKeyVault is your baseline and then environment variables etc. override those baseline values.
Thus, in virtually every case beyond the Hello World example, you'll want AzureKeyVault added as the first source with the config.Sources.Insert method.
Right now this isn't possible because the IConfigurationSource (AzureKeyVaultConfigurationSource) for AzureKeyVault is internal (you can use reflection which isn't optimal)
So the request is, either make AzureKeyVaultConfigurationSource public so that we can create our own instance and then use config.Sources.Insert, and/or create an extension method InsertAzureKeyVault() that takes an index (i.e. 0) so that we can put it in the right order so that all other sources of configuration override azure key vault as is the basic use case for this per above.
I'm not sure what the value is in it being marked internal anyhow.

Thank you for your feedback.  Tagging and routing to the team member best able to assist.

Good suggestion.
There are a few workarounds before we implement the change:

Copy the AzureKeyVaultConfigurationSource into you project (https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/extensions/Azure.Extensions.AspNetCore.Configuration.Secrets/src/AzureKeyVaultConfigurationSource.cs). It's pretty simple and AzureKeyVaultConfigurationProvider is public.
Use the extension method but modify the builder.Sources list after adding.l

builder.AddKeyvault();
builder.Insert(0, builder.Sources[builder.Sources.Count - 1]);
builder.RemoveAt(builder.Sources.Count - 1)

Thanks!

authors: JohnGalt1717 jsquire pakrym

repo: Azure/azure-sdk-for-net


text:
[ACR] [BlobClient] Investigate whether support is needed for digest hash algo other than sha256

We're currently using only SHA256 hash for computing the digest of a blob or manifest.
Investigate whether there is a need to support any other digest calculation algorithms.

Per: https://docs.docker.com/registry/spec/api/#content-digests,

compliant implementations should use sha256.

We need to confirm this for OCI as well.


text:
Docs onboarding: Sanity check packages before updating versions

Today: Validate that package installs
Future: Use containerized build when available

This is solved by some of our smoke tests and other things.

authors: danieljurek kurtzeborn


text:
[BUG] Azure.Messaging.EventHubs.Processor - UpdateCheckpointAsync fails with NullReferenceException

Library name and version
Azure.Messaging.EventHubs 5.8.0
Describe the bug
My app has always finished with this line of code after handling an event from my event hub:
await eventArgs.UpdateCheckpointAsync(eventArgs.CancellationToken);
When I upgrade to version 5.8.0 of Azure.Messaging.EventHubs and Azure.Messaging.EventHubs.Processor the very same line of code (UpdateCheckpointAsync) fails with this exception:
{     "_typeTag": "NullReferenceException",     "TargetSite": "System.Threading.Tasks.Task UpdateCheckpointAsync(System.String, Int64, System.Nullable`1[System.Int64], System.Threading.CancellationToken)",     "Message": "Object reference not set to an instance of an object.",     "Data": [],     "InnerException": null,     "HelpLink": null,     "Source": "Azure.Messaging.EventHubs.Processor",     "HResult": -2147467261,     "StackTrace": "   at Azure.Messaging.EventHubs.EventProcessorClient.UpdateCheckpointAsync(String partitionId, Int64 offset, Nullable`1 sequenceNumber, CancellationToken cancellationToken)n   at EventHandler.HandleEvent(ProcessEventArgs eventArgs) in /app/EventHandler.cs:line 67n   at EventHubProcessorService.ProcessEventHandler(ProcessEventArgs eventArgs) in /app/EventHubProcessorService.cs:line 87" }
If I downgrade the version of the EventHubs-nuget packages to 5.7.5 the issue goes away and the app runs happily.
Expected behavior
Based on the changelog/release notes for version 5.8.0 of Azure.Messaging.EventHubs and Azure.Messaging.EventHubs.Processor I expect to be able to update without changing anything in my code, at least not anything regarding await eventArgs.UpdateCheckpointAsync(eventArgs.CancellationToken);.
Actual behavior
The actual behavior is that my app does no longer work properly.
Reproduction Steps
Handle any event using version 5.8.0 of Azure.Messaging.EventHubs and Azure.Messaging.EventHubs.Processor and perform await eventArgs.UpdateCheckpointAsync(eventArgs.CancellationToken); when event is handled.
Environment
App is a dotnet 7 minimal API running in docker container in kubernetes. Runtime image is mcr.microsoft.com/dotnet/aspnet:7.0-alpine-amd64.
Output from dotnet --info:
Host:
Version:      7.0.3
Architecture: x64
Commit:       0a2bda10e8
.NET SDKs installed:
No SDKs were found.
.NET runtimes installed:
Microsoft.AspNetCore.App 7.0.3 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 7.0.3 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
Other architectures found:
None
Environment variables:
Not set
global.json file:
Not found
Learn more:
https://aka.ms/dotnet/info
Download .NET:
https://aka.ms/dotnet/download

Encountering this too, fixed by going back to 5.7.5

Hi @smafre.  Thank you for reaching out and we regret that you're experiencing difficulties.  I'm unable to repro in our test suite; we'll start to investigate further.  Are you able to provide the full stack trace for the exception, by chance?

@JoshLove-msft:  This seems like it may be related to the OTel changes.   I'd appreciate your thoughts on whether there's a specific configuration that may help to repro.

authors: jsquire kordoulakis smafre


text:
Batch set blob tier to cold tier not working

Which service(blob, file, queue, table) does this issue concern?
Blob
Which version of the SDK was used?
Azure.Storage.Blobs,12.16.0
Azure.Storage.Blobs.Batch, 12.13.0
Which platform are you using? (ex: .NET Core 2.1)
.Net 6
What problem was encountered?
I have enable preview  for ColdTier and run some tests.
batch.SetBlobsAccessTierAsync to Cold tier throw exception.
{Azure.RequestFailedException: The value for one of the HTTP headers is not in the correct format.
RequestId:beee5c35-701e-001b-2e48-9803c21e25ce
Time:2023-06-06T07:25:48.3504850Z
Status: 400 (The value for one of the HTTP headers is not in the correct format.)
ErrorCode: InvalidHeaderValue

Content:
<?xml version="1.0" encoding="utf-8"?>
<Error>InvalidHeaderValue<Message>The value for one of the HTTP headers is not in the correct format.
RequestId:beee5c35-701e-001b-2e48-9803c21e25ce
Time:2023-06-06T07:25:48.3504850Z</Message></Error>

Headers:
x-ms-error-code: InvalidHeaderValue
x-ms-request-id: beee5c35-701e-001b-2e48-9803c21e25ce
x-ms-version: 2022-11-02
x-ms-client-request-id: ab938e5f-83b9-45d0-8005-a82f68282596
Content-Length: 255
Content-Type: application/xml
Server: Windows-Azure-Blob/1.0

   at Azure.Storage.Blobs.Specialized.BlobBatch.<SetBlobAccessTier>b__26_0(Response response)
   at Azure.Storage.Blobs.Specialized.DelayedResponse.SetLiveResponse(Response live, Boolean throwOnFailure)
   at Azure.Storage.Blobs.Specialized.BlobBatchClient.UpdateOperationResponses(IList`1 messages, Response rawResponse, Boolean throwOnAnyFailure, Boolean async, CancellationToken cancellationToken)}

How can we reproduce the problem in the simplest way?
Enable preview for ColdTier, create a storage account in Canada Central, upload a few blobs and run the following test code sample.
 var batchUris = new List<Uri>();
   var service = new BlobServiceClient(connectionString);
   var container = service.GetBlobContainerClient(containerName);
   await foreach (var blob in container.GetBlobsAsync())
   {
       var blobClient = container.GetBlobClient(blob.Name);
       var p = await blobClient.GetPropertiesAsync();
       System.Console.WriteLine($"{blobClient.Uri},{p.Value.AccessTier}");
       if (p.Value.AccessTier == "Cool")
       {
           batchUris.Add(blobClient.Uri);
       }
   }
   var batch = service.GetBlobBatchClient();
   var rsp = await batch.SetBlobsAccessTierAsync(batchUris, AccessTier.Cold);

Have you found a mitigation/solution?
I try to use blobClient.SetAccessTierAsync(AccessTier.Cold), it works.
But we have storage account with billions of blobs, I prefer to use batch to reduce the request calls.

Hi, So this isn't a SDK issue so I have to close the issue.
I can still answer your question about this though. Cold Tier isn't supported by the Blob Batch API.
We're looking to update documentation around this to ensure this gets communicated.
Any follow up you would have to submit an Azure Support Ticket for the Storage Service.
https://learn.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request

Thanks @amnguye, I will submit an Azure support ticket.

Unrelated to the SDK:
As an update to anyone who finds this post, Azure Blob Storage Cold tier is GA for Blob Batch operations now.
See this announcement:
https://azure.microsoft.com/en-us/updates/generally-available-azure-blob-storage-cold-tier-support-on-blob-batch-operations/

authors: QinglongLuo amnguye


text:
[BUG] BlobContainerClient.ExistsAsync throws exception after upgrading 12.20.0  => 12.21.0

Library name and version
Azure.Storage.Blobs
Describe the bug
After upgrading from 12.20.0  => 12.21.0 BlobContainerClient.ExistsAsync(...) fails with the following exception:
Unhandled exception. System.AggregateException: One or more errors occurred. (Failed to compare two elements in the array.)
 ---> System.InvalidOperationException: Failed to compare two elements in the array.
 ---> System.TypeInitializationException: The type initializer for 'Azure.Storage.StorageSharedKeyPipelinePolicy' threw an exception.
 ---> System.Globalization.CultureNotFoundException: Only the invariant culture is supported in globalization-invariant mode. See https://aka.ms/GlobalizationInvariantMode for more information. (Parameter 'name')
en-US is an invalid culture identifier.
   at System.Globalization.CultureInfo..ctor(String name, Boolean useUserOverride)
   at Azure.Storage.StorageSharedKeyPipelinePolicy..cctor()
   --- End of inner exception stack trace ---
   at Azure.Storage.StorageSharedKeyPipelinePolicy.<>c.<BuildCanonicalizedHeaders>b__6_0(HttpHeader x, HttpHeader y)
   at System.Collections.Generic.ArraySortHelper`1.InsertionSort(Span`1 keys, Comparison`1 comparer)
   at System.Collections.Generic.ArraySortHelper`1.IntroSort(Span`1 keys, Int32 depthLimit, Comparison`1 comparer)
   at System.Collections.Generic.ArraySortHelper`1.IntrospectiveSort(Span`1 keys, Comparison`1 comparer)
   at System.Collections.Generic.ArraySortHelper`1.Sort(Span`1 keys, Comparison`1 comparer)
   --- End of inner exception stack trace ---
   at System.Collections.Generic.ArraySortHelper`1.Sort(Span`1 keys, Comparison`1 comparer)
   at System.Collections.Generic.List`1.Sort(Comparison`1 comparison)
   at Azure.Storage.StorageSharedKeyPipelinePolicy.BuildCanonicalizedHeaders(StringBuilder stringBuilder, HttpMessage message)
   at Azure.Storage.StorageSharedKeyPipelinePolicy.BuildStringToSign(HttpMessage message)
   at Azure.Storage.StorageSharedKeyPipelinePolicy.OnSendingRequest(HttpMessage message)
   at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNextAsync(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.InnerProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.RedirectPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.InnerProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Storage.Blobs.ContainerRestClient.GetPropertiesAsync(Nullable`1 timeout, String leaseId, CancellationToken cancellationToken)
   at Azure.Storage.Blobs.BlobContainerClient.GetPropertiesInternal(BlobRequestConditions conditions, Boolean async, CancellationToken cancellationToken)
   at Azure.Storage.Blobs.BlobContainerClient.ExistsInternal(Boolean async, CancellationToken cancellationToken)
   at Azure.Storage.Blobs.BlobContainerClient.ExistsAsync(CancellationToken cancellationToken)
   at <my code>

I'm running my production environment using an invariant culture.
Expected behavior
BlobContainerClient.ExistsAsync(...) to return if the container exists
Actual behavior
An exception is thrown.
Reproduction Steps
Call BlobContainerClient.ExistsAsync(...) after building code using:
dotnet publish vehicle-listing/vehicle-listing.application/vehicle-listing.application.csproj \
        --configuration Release \
        --output /app \
        --self-contained false \
        /p:UseAppHost=false \
        --nologo

And running in a mcr.microsoft.com/dotnet/aspnet:8.0-alpine Docker container
Environment
mcr.microsoft.com/dotnet/aspnet:8.0-alpine Docker container running on x64.

Duplicate of https://github.com/Azure/azure-sdk-for-net/issues/45052

We released a patch release to fix this issue. Please upgrade accordingly.
https://www.nuget.org/packages/Azure.Storage.Common/12.20.1
https://www.nuget.org/packages/Azure.Storage.Blobs/12.21.1
https://www.nuget.org/packages/Azure.Storage.Files.DataLake/12.19.1
https://www.nuget.org/packages/Azure.Storage.Queues/12.19.1
https://www.nuget.org/packages/Azure.Storage.Files.Shares/12.19.1

authors: amnguye invertedtomato


text:
[BUG] System.TypeLoadException

Tried to set running newest bits. When started, application fails with System.TypeLoadException.
Error

Exception or Stack Trace
System.TypeLoadException
HResult=0x80131522
Message=Method 'get_EventBehaviourForContinueAsNew' in type 'DurableTask.ServiceBus.ServiceBusOrchestrationService' from assembly 'DurableTask.ServiceBus, Version=2.0.10.0, Culture=neutral, PublicKeyToken=d53979610a6e89dd' does not have an implementation.
Source=
StackTrace:

Project file
<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp2.2</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.Azure.DurableTask.AzureStorage" Version="1.6.0" />
    <PackageReference Include="Microsoft.Azure.DurableTask.Core" Version="2.1.1" />
    <PackageReference Include="Microsoft.Azure.DurableTask.ServiceBus" Version="2.0.10" />
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..\SharedLib.NetCore\SharedLib.NetCore.csproj" />
  </ItemGroup>

</Project>

Sample
static void Main(string[] args)
       {

           string hubName = "DtfSampleHub2";
           IOrchestrationServiceInstanceStore instanceStore = new AzureTableInstanceStore(hubName, m_StorageConnStr);

           var orchestrationServiceAndClient = new ServiceBusOrchestrationService(m_ConnStr, hubName, instanceStore, null, null);

           orchestrationServiceAndClient.CreateIfNotExistsAsync().Wait();

           m_TaskHubWorker = new TaskHubWorker(orchestrationServiceAndClient);

           m_TaskHubWorker.AddTaskOrchestrations(typeof(TimerOrchestration));
           m_TaskHubWorker.AddTaskOrchestrations(typeof(HelloOrchestration));
           m_TaskHubWorker.AddTaskActivities(typeof(SchedulerTask), typeof(UserInputTask), typeof(PlayFileTask));

           m_TaskHubWorker.StartAsync().Wait();

           Thread.Sleep(int.MaxValue);
       }

Thank you for opening this issue! We are routing it to the appropriate team for follow up.

@ddobric Microsoft.Azure.DurableTask.ServiceBus is referencing the old client, WindowsAzure.ServiceBus. Which is not related to this client here.

This issue should be assigned to DurableTask framework (https://github.com/Azure/durabletask)

Microsoft.Azure.DurableTask.ServiceBus prior to version 2.1.3 did not support DotNetCore. Version 2.0.10 of Microsoft.Azure.DurableTask.ServiceBus has Microsoft.Azure.DurableTask.Core version 2.0.10 as a dependency, the project it set to reference a newer version explictly.
Recommend using Microsoft.Azure.DurableTask.ServiceBus 2.1.3 and not explicitly calling in a different version of Microsoft.Azure.DurableTask.Core.

authors: SeanFeldman ddobric kurtzeborn shankarsama simonporter


text:
various new features for datafactory sdk

Description
Add several features to Data Factory SDK

This checklist is used to make sure that common guidelines for a pull request are followed.

[x] I have read the contribution guidelines.
[x] The pull request does not introduce breaking changes.

General Guidelines

[x] Title of the pull request is clear and informative.
[x] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[x] Pull request includes test coverage for the included changes.

SDK Generation Guidelines

[x] If an SDK is being regenerated based on a new swagger spec, a link to the pull request containing these swagger spec changes has been included above.
[x] The generate.cmd file for the SDK has been updated with the version of AutoRest, as well as a link to the swagger spec, used to generate the code.
[x] The project.json and AssemblyInfo.cs files have been updated with the new version of the SDK.

Can one of the admins verify this patch?

@ninishaha Can you please provide a link to the hydra-specs-pr pull request corresponding with these changes in the PR comments?  We can take a look after that.

@markcowl do you mean the document of the released features I added in the SDK? https://docs.microsoft.com/en-us/azure/data-factory/data-factory-data-movement-activities

@ninishaha I see, the models are all in customizations, will take a look.

@azuresdkci add to whitelist

@ninishaha,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

@cormacpayne  replaced them with fake strings.

@ninishaha Note that if these were real credentials, they may have been compromised, because all commits are public.  If this is the case, please report this on http://reportitnow, and follow the instructions there.

@ninishaha I don't see any tests for the new functionality.  Please add sufficient tests to prevent important regressions.  You may file a test issue in the repo and assign to your team, if you fix withon one sprint.

authors: azuresdkci markcowl msftclas ninishaha


text:
Constant declarations for Spark configurations to be used by powershell

Description
Adding new constant declarations which will be used by Azure HDInsight powershell

This checklist is used to make sure that common guidelines for a pull request are followed.

[x ] I have read the contribution guidelines.
[x ] The pull request does not introduce breaking changes.

General Guidelines

[x ] Title of the pull request is clear and informative.
[x] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[ ] Pull request includes test coverage for the included changes. - Not necessar

SDK Generation Guidelines - Not required

[ ] If an SDK is being regenerated based on a new swagger spec, a link to the pull request containing these swagger spec changes has been included above.
[ ] The generate.cmd file for the SDK has been updated with the version of AutoRest, as well as the commitid of your swagger spec or link to the swagger spec, used to generate the code.
[ ] The *.csproj and AssemblyInfo.cs files have been updated with the new version of the SDK.

Can one of the admins verify this patch?

Can one of the admins verify this patch?

@jeffersonezra,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

@shefaliv @nemanja88 Can one of you please sign off on these changes?

@azuresdkci add to whitelist

@azuresdkci retest this please

@azuresdkci test this please

@jeffersonezra this build failed earlier due to a bad test for DataLake.Store.
I have restarted the CI, once this passes, this is good to go.

authors: azuresdkci jeffersonezra markcowl msftclas shahabhijeet


text:
skip sample in playback

Description
Skip sample when running in playback mode. The sample Docker client and SSH client require a real network connection.
This checklist is used to make sure that common guidelines for a pull request are followed.

[ ] I have read the contribution guidelines.
[ ] The pull request does not introduce breaking changes.

General Guidelines

[ ] Title of the pull request is clear and informative.
[ ] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[ ] Pull request includes test coverage for the included changes.

SDK Generation Guidelines

[ ] If an SDK is being regenerated based on a new swagger spec, a link to the pull request containing these swagger spec changes has been included above.
[ ] The generate.cmd file for the SDK has been updated with the version of AutoRest, as well as the commitid of your swagger spec or link to the swagger spec, used to generate the code.
[ ] The *.csproj and AssemblyInfo.cs files have been updated with the new version of the SDK.

@milismsft,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

authors: milismsft msftclas


text:
Refactor TDE setup.

Also removed dead code in SqlManagementTestUtilities and removed unneeded RecordedDelegatingHandler class.

Description

This checklist is used to make sure that common guidelines for a pull request are followed.

[x] I have read the contribution guidelines.
[x] The pull request does not introduce breaking changes.

General Guidelines

[x] Title of the pull request is clear and informative.
[x] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[x] Pull request includes test coverage for the included changes.

SDK Generation Guidelines

[x] If an SDK is being regenerated based on a new swagger spec, a link to the pull request containing these swagger spec changes has been included above.
[x] The generate.cmd file for the SDK has been updated with the version of AutoRest, as well as the commitid of your swagger spec or link to the swagger spec, used to generate the code.
[x] The *.csproj and AssemblyInfo.cs files have been updated with the new version of the SDK.

@azuresdkci test this please

authors: jaredmoo shahabhijeet


text:
[Text Analytics] Implement IsEnvironmentReadyAsync

Resolves: https://github.com/Azure/azure-sdk-for-net/issues/35452.

API change check
API changes are not detected in this pull request.

/azp run net - textanalytics - tests

/azp run net - textanalytics - tests

/azp run net - textanalytics - tests

/azp run net - textanalytics - tests

Looks good, however, the issue seems to be not TA specific. Should the pipeline add some latency after the resource creation step to mitigate it?

We don't want to add latency to all pipelines. Few, hopefully, need it. See my review for what I suggested to inject latency when needed.

authors: azure-sdk heaths joseharriaga


text:
Autorest Regen Preview to 3.0.0-alpha.20240308.3 by Wei Hu from refs/pull/4187/merge

Triggered from https://github.com/Azure/autorest.csharp/pull/4187

API change check
API changes are not detected in this pull request.


text:
Increment version for provisioning releases

Increment package version after release of Azure.Provisioning.AppContainers

API change check
API changes are not detected in this pull request.


text:
Unable to filter by multiple provisioningState on deployment list

Package Name:
azure-mgmt-resource
Package Version:
22.0.0
Operating System:
Ubuntu
Python Version:
3.10.8

Describe the bug
I am unable to retrieve deployments when filtering by provisioning state with OR operator. I tried the filter query with REST API and it works.
azure.core.exceptions.HttpResponseError: (InvalidProvisioningStateFilter) Invalid $filter 'provisioningState eq 'accepted' or provisioningState eq 'updating'' specified in the query string.
Code: InvalidProvisioningStateFilter
Message: Invalid $filter 'provisioningState eq 'accepted' or provisioningState eq 'updating'' specified in the query string.

To Reproduce
Steps to reproduce the behavior:
ResourceManagementClient(...).deployments.list_by_resource_group(
       resource_group_name=<resource_group_name>,
       filter="provisioningState eq 'accepted' or provisioningState eq 'updating'",
)

Expected behavior
I expect to retrieve the list of deployments with provisioning state accepted or running.

Hi @wiwski
Please try the following:
ResourceManagementClient(...).deployments.list_by_resource_group(
       resource_group_name=<resource_group_name>,
       filter="provisioningState eq '{accepted}' or provisioningState eq '{updating}'",
)

Hi @Wzb123456789,
Thanks for the support.
I am getting the same error with your suggestion.
ongoing_deployments = self._resource_mgmt_client.deployments.list_by_resource_group(
            resource_group_name=self.resource_group_name,
            filter="provisioningState eq '{accepted}' or provisioningState eq '{updating}'",
)

Code: InvalidProvisioningStateFilter
Message: Invalid $filter 'provisioningState eq '{accepted}' or provisioningState eq '{updating}'' specified in the query string.

Hi, any update on this ? Is this something that is not just posible at the REST API level ?

@wiwski sorry for the delay on this one.  I am able to reproduce this and as per my understanding it looks like a limitations at services end as the Deployments REST API is also not able to handle filters with multiple conditions.
Please check the below response -

@Wzb123456789 Can you please look into this.

Hello,
Any update on this issue ?
I just noticed ne (for not equal) filter is not available as well. It would be great at least to have this (to filter out Succeeded deployment).
Thanks !

Please help on this issue @ChenxiJiang333

got it

Hi @wiwski, provisioningState eq 'accepted' or provisioningState eq 'updating should be a correct odata syntax, yet I don't know somehow the api does not accept it, since it's a service issue, python sdk can not make any change on it. So I would transfer this issue to seek help from service team.

repo: Azure/azure-sdk-for-python


text:
Fix storage file datalake readme and samples issues

Fix issue https://github.com/Azure/azure-sdk-for-python/issues/11358

Add the correct link for "Azure Portal", "azure PowerShell", '"Azure CLI".
Modify the CLI command for creating the storage account.

@jongio for notification.

@xiafu-msft - What are your thoughts on this PR?


text:
[synapse][artifacts] release for composite-v6

swagger definition:
https://github.com/Azure/azure-rest-api-specs/tree/main/specification/synapse/data-plane
swagger SHA:
a528b7824623c322bc917393654ff3d8ce0f7aed
cmd:
autorest C:\dev\azure-rest-api-specs\specification\synapse\data-plane\readme.md --python  --python-sdks-folder=C:\dev\azure-sdk-for-python\sdk  --use=@autorest\python@6.2.12 --use=@autorest\modelerfour@4.24.3 --version=3.9.2 --version-tolerant=False --package-mode=dataplane

API change check
APIView has identified API level changes in this PR and created following API reviews.
azure-synapse-artifacts

Hi @annatisch Could you help review the PR or ApiView: https://github.com/Azure/azure-sdk-for-python/pull/28231#issuecomment-1375693274 ? Thanks!

https://dev.azure.com/azure-sdk/internal/_build/results?buildId=2111746&view=results

authors: azure-sdk msyyc


text:
[Eventhub] Stress testing checkpoint store

Adding a wrapper around the BlobCheckpointStore to allow injecting different levels of latency and/or network errors.
This should helpfully help us repro behaviour seen in issue #28338 where frequent partition rebalancing was occurring where a checkpoint storage account was in a different region to the EventHub.

API change check
API changes are not detected in this pull request.

authors: annatisch azure-sdk


text:
[AutoRelease] t2-containerregistry-2023-09-06-76862(can only be merged by SDK owner)

https://github.com/Azure/sdk-release-request/issues/4497
Live test success
https://dev.azure.com/azure-sdk/internal/_build?definitionId=981
BuildTargetingString
azure-mgmt-containerregistry
Skip.CreateApiReview
true

issue link:https://github.com/Azure/sdk-release-request/issues/4497

/check-enforcer override

/check-enforcer override

authors: ChenxiJiang333 azure-sdk msyyc


text:
App Config Provider - KV Options Fix and Tests

Description
Fixes bug added in the last pr on key_vault_options usage.
Adds tests for key_vault_options
All SDK Contribution checklist:

[X] The pull request does not introduce [breaking changes]
[X] CHANGELOG is updated for new features, bug fixes or other significant changes.
[X] I have read the contribution guidelines.

General Guidelines and Best Practices

[C] Title of the pull request is clear and informative.
[C] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[X] Pull request includes test coverage for the included changes.

API change check
API changes are not detected in this pull request.


text:
[AutoRelease] t2-mobilenetwork-2023-10-14-25824(can only be merged by SDK owner)

https://github.com/Azure/sdk-release-request/issues/4648
Live test success
https://dev.azure.com/azure-sdk/internal/_build?definitionId=4616

issue link:https://github.com/Azure/sdk-release-request/issues/4648


text:
Re-Release Distro 1.2.0: Enable CI

Distro release failed. Enabling CI.

API change check
API changes are not detected in this pull request.

Superfluous: https://github.com/Azure/azure-sdk-for-python/pull/33913/files

authors: azure-sdk jeremydvoss


text:
[AutoRelease] t2-containerorchestratorruntime-2024-09-24-22106(can only be merged by SDK owner)

https://github.com/Azure/sdk-release-request/issues/5494
Recording test success

issue link:https://github.com/Azure/sdk-release-request/issues/5494

/azp run prepare-pipelines

Hi @ChenxiJiang333 , the python SDK looks good to me. Is there any ETA when the PR will be merged and published? Thanks!

Hi @ChenxiJiang333 , the python SDK looks good to me. Is there any ETA when the PR will be merged and published? Thanks!

the target release date would be around 10/24

authors: ChenxiJiang333 HE-Xinyu azure-sdk


text:
[AutoPR databricks/resource-manager] databricks: switching parameters to be a proper struct

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/6917

(message created by the CI based on PR content)


text:
[AutoPR] machinelearningservices/resource-manager

Created to accumulate context: machinelearningservices/resource-manager

(message created by the CI based on PR content)


text:
cargo fmt sdk/iot_deviceupdate/src/client.rs

Other PR was failing without this file formatted.

Seems no longer necessary as we've done a number of full builds without seeing an issue.

authors: cataggar heaths

repo: Azure/azure-sdk-for-rust


text:
test-proxy push hanging on large add operations

To be clear, this is hanging because we're missing handling an event somewhere.
I say this because if I end the test-proxy push as it is hung, all the files immediately show up as added. This tells me there is something happening internally in the ProcessStartInfo that we aren't handling. It's not a matter of an assumption going wrong.
After speaking with a couple people who have extensively dealt with this, I was advised to register an outputDataReceived handler. RefDocs.

And now it's not repro-ing in tests 👎

I believe this is a conflict between process.StandardOutput.ReadToEnd(); and Process.WaitForExit().
I am claiming (with no real proof) that when invoking an operation with a lot of output, we are filling an internal buffer. Given that we're synchronously reading from the process, we may end up in a deadlock.
This is due to the fact that the child is waiting for the parent to empty some of it's buffer, and the parent process is waiting for the child to exit.
I am going to swap to registering asynchronous read, and get the output that way.

repo: Azure/azure-sdk-tools


text:
[tsp-client] tsp client fails to clean TempTypeSpecFiles

emitter-package.json is as below.
{
    "main": "dist/src/index.js",
    "dependencies": {
      "@azure-tools/typespec-powershell": "0.0.x",
      "@azure-tools/typespec-azure-core": "0.44.0",
      "@azure-tools/typespec-azure-resource-manager": "0.44.0",
      "@azure-tools/typespec-client-generator-core": "0.44.2",
      "@typespec/compiler": "0.58.1",
      "@typespec/http": "0.58.0",
      "@typespec/openapi": "0.58.0",
      "@typespec/openapi3": "0.58.0",
      "@typespec/rest": "0.58.0",
      "@typespec/versioning": "0.58.0"
    }
}

The issue could be reproduced by running following command.
tsp-client init -c
https://github.com/dolauli/azure-rest-api-specs/blob/82a148896900c9bf89693d8dadea6a4acca5bf5c/specification/sphere/Sphere.Management/tspconfig.yaml

Where there any updates to your emitter recently? I can't get the same error, I'm seeing this output now:
C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\TempTypeSpecFiles\node_modules\@autorest\powershell\dist\plugins\llcsharp-v2.js - Error: ENOENT: no such file or directory, mkdir 'C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\generated\api'
    at mkdirSync (node:fs:1380:26)
    at TspHostImpl.writeFile (C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\TempTypeSpecFiles\node_modules\@autorest\powershell\dist\utils\tsp-host.js:28:32)
    at State.writeFile (C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\TempTypeSpecFiles\node_modules\@autorest\powershell\dist\utils\model-state.js:85:29)
    at C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\TempTypeSpecFiles\node_modules\@autorest\powershell\dist\plugins\llcsharp-v2.js:14:74
    at w (C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\TempTypeSpecFiles\node_modules\@azure-tools\codegen-csharp\dist\namespace.js:98:42)
    at ServiceNamespace.writeFiles (C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\TempTypeSpecFiles\node_modules\@azure-tools\codegen-csharp\dist\namespace.js:114:19)
    at C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\TempTypeSpecFiles\node_modules\@azure-tools\codegen-csharp\dist\project.js:24:64
    at Array.map (<anonymous>)
    at Object.pall (C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\TempTypeSpecFiles\node_modules\@azure-tools\codegen\dist\text-manipulation.js:150:30)
    at Project.writeFiles (C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\TempTypeSpecFiles\node_modules\@azure-tools\codegen-csharp\dist\project.js:24:25)/ENOENT: no such file or directory, mkdir 'C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\generated\api'
node:internal/process/promises:289
            triggerUncaughtException(err, true /* fromPromise */);
            ^

Error: ENOENT: no such file or directory, mkdir 'C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\generated\api'
    at mkdirSync (node:fs:1380:26)
    at TspHostImpl.writeFile (C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\TempTypeSpecFiles\node_modules\@autorest\powershell\dist\utils\tsp-host.js:28:32)
    at State.writeFile (C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\TempTypeSpecFiles\node_modules\@autorest\powershell\dist\utils\model-state.js:85:29)
    at C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\TempTypeSpecFiles\node_modules\@autorest\powershell\dist\plugins\llcsharp-v2.js:14:74
    at w (C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\TempTypeSpecFiles\node_modules\@azure-tools\codegen-csharp\dist\namespace.js:98:42)
    at ServiceNamespace.writeFiles (C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\TempTypeSpecFiles\node_modules\@azure-tools\codegen-csharp\dist\namespace.js:114:19)
    at C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\TempTypeSpecFiles\node_modules\@azure-tools\codegen-csharp\dist\project.js:24:64
    at Array.map (<anonymous>)
    at Object.pall (C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\TempTypeSpecFiles\node_modules\@azure-tools\codegen\dist\text-manipulation.js:150:30)
    at Project.writeFiles (C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\TempTypeSpecFiles\node_modules\@azure-tools\codegen-csharp\dist\project.js:24:25) {
  errno: -4058,
  code: 'ENOENT',
  syscall: 'mkdir',
  path: 'C:\\Users\\caperal\\repos\\testpwsh\\sdk\\spheremanager\\spheremanager\\C:\\Users\\caperal\\repos\\testpwsh\\sdk\\spheremanager\\spheremanager\\generated\\api'
}

Node.js v20.11.0

The first lines of the error seem to be pointing to an issue in one of the emitter's files:
C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\TempTypeSpecFiles\node_modules\@autorest\powershell\dist\plugins\llcsharp-v2.js - Error: ENOENT: no such file or directory, mkdir 'C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\generated\api'
    at mkdirSync (node:fs:1380:26)
    at TspHostImpl.writeFile (C:\Users\caperal\repos\testpwsh\sdk\spheremanager\spheremanager\TempTypeSpecFiles\node_modules\@autorest\powershell\dist\utils\tsp-host.js:28:32)

I did not change the emitter, but I changed a dependency -- autorest/powershell. I think you may pin autorest/powershell to an old version by adding "@autorest/powershell": "4.0.703" in emitter-package.json.

After you updated your dependency are you still running into the first issue or are you seeing something like the error I shared? I wonder if the emitter is kicking off separate processes that conflict with what tsp-client is doing.

You've jogged my memory. Let me review my code.

My bad. Proved to be a bug in my emitter. I forget to add await when calling an async function. tsp-client works fine after I fixed my issue.

authors: catalinaperalta dolauli


text:
New common script to create API review request from artifact

Common script that will be consumed by each language pipeline to create API review

The following pipelines have been queued for testing:java - templatejs - templatenet - templatepython - template You can sign off on the approval gate to test the release stage of each pipeline. See eng/common workflow

The following pipelines have been queued for testing:java - templatejs - templatenet - templatepython - template You can sign off on the approval gate to test the release stage of each pipeline. See eng/common workflow

The following pipelines have been queued for testing:java - templatejs - templatenet - templatepython - template You can sign off on the approval gate to test the release stage of each pipeline. See eng/common workflow

The following pipelines have been queued for testing:java - templatejs - templatenet - templatepython - template You can sign off on the approval gate to test the release stage of each pipeline. See eng/common workflow

The following pipelines have been queued for testing:java - templatejs - templatenet - templatepython - template You can sign off on the approval gate to test the release stage of each pipeline. See eng/common workflow

/azp run azure-sdk-tools - sync - eng-common

The following pipelines have been queued for testing:java - templatejava - template - testsjs - templatenet - templatenet - template - testspython - templatepython - template - tests You can sign off on the approval gate to test the release stage of each pipeline. See eng/common workflow

@weshaggard Please let me know if any additional change is required. I have changes for .Net and Java but waiting to get this merged.

The following pipelines have been queued for testing:java - templatejava - template - testsjs - templatenet - templatenet - template - testspython - templatepython - template - tests You can sign off on the approval gate to test the release stage of each pipeline. See eng/common workflow

authors: azure-sdk praveenkuttappan


text:
[Service Bus] Implement readme template

There's a new README.md template we should be using (guidelines).

Opened in wrong repo, admins can delete.

repo: Azure/azure-sdk


text:
Naming convention for service name in our repos

As part of the repo structure guidelines we have been hitting a number of questions on what we should exactly name the <service name> folders in the guidelines outlined at https://github.com/Azure/azure-sdk/blob/master/docs/engineering-system/repo-structure.md.
Currently the guideline is:
<service name> - Should be the short name for the azure service. It will be used to group all the various packages for a given service, including the management and data-plane packages, as well as any multi-api (e.g. or different profile versions (e.g AzureStack). Any shared artifacts like release notes or versions should go into this root repo. These names should match across all the different language repos and by default should be what is in the specification folder in the azure-rest-api-specs repo.
However the names that are in the specification repo and in the different languages currently are not always consistent. In fact other places are also inconsistent like the docs vs the portal. On top of that a lot of the names are all lowercase with no word separators which can make them difficult to read in a number of cases.
One proposal would be to use the names that we list on our public Azure product page at https://azure.microsoft.com/en-us/services/. They seem to use lower kebab-casing some examples that keep coming up would be as follows:

event-hubs https://azure.microsoft.com/en-us/services/event-hubs/
service-bus https://azure.microsoft.com/en-us/services/service-bus/
cosmos-db https://azure.microsoft.com/en-us/services/cosmos-db/
key-vault https://azure.microsoft.com/en-us/services/key-vault/
storage https://azure.microsoft.com/en-us/services/storage/

So if we use those names then the root sdk examples for all the languages would be:
sdk\event-hubs
sdk\service-bus
sdk\cosmos-db
sdk\key-vault
sdk\stoarge

@adrianhall is this a topic we could get into an upcoming review slot?
Arch Board PTAL @KrzysztofCwalina @bterlson @JeffreyRichter @JonathanGiles @johanste

I think .NET developers would prefer .NET casing, i.e. pascal casing.

The current guideline is:

The current guideline is:
pick a package name that allows the consumer to tie the namespace to the service being
used.  As a default, use the compressed service name at the end of the namespace.  The
namespace does **NOT** change when the branding of the product changes, so avoid the
use of marketing names that may change.  (See below for examples).

However, I defer "compressed" to the language-specific guidelines - whether it's camel case or kebab case or snake case or whatever.  I understand that the GitHub repo should be named consistently, so your engineering system guidelines should define what "compressed" means in that context.   That's a job for @maggiepint

There is I suppose an option where the repo for the language follows the convention that the language follows for packages. Trying to think what the impact of this choice would be on the build system or reporting over time.

Further thought on this - we have chosen snake case for the names of the repos themselves. I'm pretty sure we're not going to change that, so maybe snake case is the best choice for the subfolders as well? I think it creates consistency across a lot of urls which is always good.

For the Java perspective - I have no opinion nor preference, and I don't believe Java users will either. In this case, I vote for consistency across all languages for the reason Maggie states.

I believe we are all sold on this now.
Within the repostructure.md file is the following statement:
These names should match across all the different language repos and by default should be what is in the specificiation folder in the [azure-rest-api-specs](https://github.com/azure/azure-rest-api-specs) repo.

This should cover all the cases.  If not, put in a PR to adjust the repostructure.md (which is moving on Monday to docs/policies/repostructure.md)

authors: JonathanGiles KrzysztofCwalina adrianhall maggiepint weshaggard


text:
Board Review: Introducing Azure App Configuration Provider for Python

Thank you for starting the process for approval of the client library for your Azure service.  Thorough review of your client library ensures that your APIs are consistent with the guidelines and the consumers of your client library have a consistently good experience when using Azure.
The Architecture Board reviews Track 2 libraries only. If your library does not meet this requirement, please reach out to Architecture Board before creating the issue.
Please reference our review process guidelines to understand what is being asked for in the issue template.
Before submitting, ensure you adjust the title of the issue appropriately.
Note that the required material must be included before a meeting can be scheduled.
Contacts and Timeline

Service team responsible for the client library: Azure App Configuration
Main contacts: mametcal, zhenlwa
Expected stable release date for this library: N/A

About the Service

Link to documentation introducing/describing the service: https://learn.microsoft.com/en-us/azure/azure-app-configuration/overview
Link to the service REST APIs: N/A, this library is building on top of the existing Azure SDK for Python.
Is the goal to release a Public Preview, Private Preview, or GA? GA

About the client library

Name of client library: azure-appconfiguration-provider
Link to library reference documentation: https://learn.microsoft.com/en-us/azure/azure-app-configuration/quickstart-python-provider?tabs=windowscommandprompt
https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/appconfiguration/azure-appconfiguration-provider
Is there an existing SDK library? If yes, provide link: https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/appconfiguration/azure-appconfiguration

Step 1: Champion Scenarios
Ultimately the library should be easy to use for common scenarios that developers want. Consider the following questions when thinking about champion scenarios:

What is the app the developer is building that uses your client library?
Who is the end-user of the application (the developer's customer)?
What features of the API need to be explained in the sample so that someone could use this API in real app?
How does the authentication workflow look?

See Champion Scenario section here.
Code is appreciated but optional. Pseudocode is fine.
https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/appconfiguration/azure-appconfiguration-provider/samples

Accessing your Application's configuration in a managed way.
Enables using Secrets from Key Vault as configuration through Key Vault references that are auto resolved by the library.

Step 2: Quickstart Samples (Optional)
https://learn.microsoft.com/en-us/azure/azure-app-configuration/quickstart-python-provider?tabs=windowscommandprompt
Thank you for your submission!

Scheduled for Dec 14th from 2:05PM - 4PM PST

Link to the API Review https://apiview.dev/Assemblies/Review/f5435417b399458eb7e5fd127bc55095

Recording (MS INTERNAL ONLY)

authors: mrm9084 ronniegeraghty tg-msft


text:
Board Review: Azure.ResourceManager.Help Namespace Review (Microsoft.Help Resource Provider)

Thank you for submitting this review request. Thorough review of your client library ensures that your APIs are consistent with the guidelines and the consumers of your client library have a consistently good experience when using Azure.
The Architecture Board reviews Track 2 libraries only. If your library does not meet this requirement, please reach out to Architecture Board before creating the issue.
Please reference our review process guidelines to understand what is being asked for in the issue template.
To ensure consistency, all Tier-1 languages (C#, TypeScript, Java, Python) will generally be reviewed together.  In expansive libraries, we will pair dynamic languages (Python, TypeScript) together, and strongly typed languages (C#, Java) together in separate meetings.
For Tier-2 languages (C, C++, Go, Android, iOS), the review will be on an as-needed basis.
Before submitting, ensure you adjust the title of the issue appropriately.
Note that the required material must be included before a meeting can be scheduled.
Contacts and Timeline

Responsible service team: Aravind Bharatha (arbhara@microsoft.com); Joshua Min (joshuamin@microsoft.com); Nithya Shivam (nithyasivam@microsoft.com)
Main contacts: Aravind Bharatha (arbhara@microsoft.com); Joshua Min (joshuamin@microsoft.com); Nithya Shivam (nithyasivam@microsoft.com)
Expected code complete date:
Expected release date: 03/30/2023

About the Service

Link to documentation introducing/describing the service: https://review.learn.microsoft.com/en-us/rest/api/azure-rest-preview/?branch=result_7590b404-e4be-4a9f-821d-ab3ea9fbb55f&view=azure-rest-preview
Link to the service REST APIs: https://github.com/Azure/azure-rest-api-specs/pull/23056
Link to GitHub issue for previous review sessions, if applicable:  Link to our preview review. We would like to redo the review as we find HelpRp (Rp word in HelpRp is redundant and would like to request the word "Help" in our sdk's as our RP name is Microsoft.Help. I added the swagger documentation for context about our service above.)

About the client library

Name of the client library:
Languages for this review:

• Java
Resource Management - Help
azure-resourcemanager-help
API View Link: https://apiview.dev/Assemblies/Review/73135cd3a34243349ee51728aad80224
• JavaScript
Resource Management - Help
@azure/arm-help
API View Link: https://apiview.dev/Assemblies/Review/d360d9e658664b20ad829dc20a6c45a5
• Go/Golang
Resource Management - Help
sdk/resourcemanager/help/armhelp
API View link: Failed to generate (Working with pr reviewer. Look at swagger pr for more details)
• .NET (Future)
Resource Management - Help
Azure.ResourceManager.Help
API View Link: N/A
• Swagger
API view link: https://apiview.dev/Assemblies/Review/a38db001941d4debac155a8202e88dff
Thank you!

@BharathaAravind,
I believe we already reviewed and approved MGMT SDK Namespaces for this spec in this issue: https://github.com/Azure/azure-sdk-pr/issues/854

@ronniegeraghty Yeah I added the link to previously approved spec in the description. After discussing internally, we would like to do the review again and propose few more options other than HelpRp. Ex: Help, AzureHelp, HelpApi, AzHelp, ArmHelp etc

To me just "Help" is too generic. Coudl we call it SelfHelp or SelfDiagnostics?

After an email thread here are the agreed upon namespaces for each language:

.NET: Azure.ResourceManager.SelfHelp
Java: azure-resourcemanager-selfhelp (com.azure.resourcemanager.selfhelp)
Go: sdk/resourcemanager/help/selfhelp
JavaScript: @azure/arm-selfhelp
Python: azure-mgmt-selfhelp

The namespaces above have been provided to our architect team and are being reviewed.

Go should be sdk/resourcemanager/selfhelp/armselfhelp

@JeffreyRichter & @BharathaAravind I updated my comment with the namespaces to reflect your comments.

@ronniegeraghty - Can we consider this as our approved namespaces?

@ronniegeraghty - Can we consider this as our approved namespaces based on the email thread conversation?

At EOB today the architect's week to object to the names will be up and the names will be considered approved. I'll comment again when the approval is final.

The following namespaces are approved:
.NET: Azure.ResourceManager.SelfHelp
Java: azure-resourcemanager-selfhelp (com.azure.resourcemanager.selfhelp)
Go: sdk/resourcemanager/selfhelp/armselfhelp
JavaScript: @azure/arm-selfhelp
Python: azure-mgmt-selfhelp

authors: BharathaAravind JeffreyRichter KrzysztofCwalina ronniegeraghty


text:
Set StatusCode on HttpRequestException

Today when we create an HttpRequestException, the status code of a request is set as a part of the Message for the exception.
Since .NET 5, the HttpRequestException has a property for StatusCode.
I would propose this property should be set to allow for easier access to the status code.
To avoid having to do string parsing on the Message.

To avoid having to do string parsing on the Message

In which case would you do string parsing to which Message?

Today when we create an HttpRequestException,

Could you describe more about when would you create an HttpRequestException when using Azure SignalR?

I updated the post, providing more information and a use case

authors: skystedt vicancy

repo: Azure/azure-signalr


text:
Connection close error is not readable.

Describe the bug
improve error message on connection close.
Bad case:
Error: Connection disconnected with error 'Error: Server returned an error on close: Connection closed with an error.'.

The error message are from signalr application layer, and prefixes are defined in the js signalr code https://github.com/dotnet/aspnetcore/blob/98c4546ddccfe2f62c4dc239fabd0c92848b255d/src/SignalR/clients/ts/signalr/src/HubConnection.ts#L643

authors: JialinXin vicancy


text:
please help to cap azcopy memory useage

Which version of the AzCopy was used?
Note: The version is visible when running AzCopy without any argument
Version:10.17.0
Which platform are you using? (ex: Windows, Mac, Linux)
Linux. alpine docker container
What command did you run?
Job-Command sync https://xxxxxx.file.core.windows.net/xxxx/Housekeeping?se=2023-03-29t09%3A49%3A35z&sharesnapshot=2023-03-28t09%3A49%3A35.0000000z&sig=-REDACTED-&sp=rl&sr=s&sv=2019-07-07 https://xxxx.file.core.windows.net/xxxx/Housekeeping?se=2023-03-29t09%3A49%3A36z&sig=-REDACTED-&sp=rwdl&sr=s&sv=2019-07-07 --preserve-smb-info=true --preserve-smb-permissions=true --recursive=true --log-level=Error --delete-destination=true
What problem was encountered?
Since we are syncing about 1M to 2M files in Azure file share in K8S pods. azcopy always exceed the pod memory limitation and cause the pod failure with OOMKilled.
How can we reproduce the problem in the simplest way?
Have you found a mitigation/solution?

Hi @wingfeng ! Please consider breaking down the jobs into smaller parts. I would also suggest to use the AZCOPY_BUFFER_GB  (view this document for more information: https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-optimize#optimize-memory-use)

Thanks @siminsavani-msft, we have already use this parameter as below
  containers:           - name: azcopy             image: {8}             command:               - "azcopy"              - "sync"              - "{9}"              - "{10}"                     - "--preserve-smb-info=true"              - "--preserve-smb-permissions=true"              - "--recursive={11}"              - "--log-level=Error"              - "--delete-destination=true"             env:             - name: AZCOPY_CONCURRENCY_VALUE               value: "194"             - name: AZCOPY_CONCURRENT_SCAN               value: "512"             - name: AZCOPY_CONCURRENT_FILES               value: "194"             - name: AZCOPY_BUFFER_GB               value: "4"             - name: AZCOPY_LOG_LOCATION               value: /mnt/afs/azcopy_logs/{3}             resources:               requests:                 cpu: 400m                 memory: 2Gi               limits:                 cpu: 4000m                 memory: 16Gi
but it seems no work fine as we want. some pods are almost 3 times and 4 times then 4GB.
Here are some memory usage from our environment.
kubctl top pods -n $ns
NAME                                           CPU(cores)   MEMORY(bytes)
azcopysync-xxx1-exb-gef-28012706-74w9x         4001m        4468Mi
azcopysync-xxx1-xx1-28012708-w7dfw            3978m        1455Mi
largeazcopysync-xxx1-xx--2-28012700-czkh6    3997m        15321Mi
largeazcopysync-xx-xxx--14-28012700-wchn5   3999m        12875Mi

I see. Maybe you try adding --cap-mbps=<value> to cap the throughput data rate?
The documentation linked earlier has more information about this flag.

--cap-mbps only help to cap the throughput data rate in document. does it work for cap memory usage too?
actually we don't want to cap the throughput. we want it to sync as fast as it can. we want to cap azcopy memory usage to make it run stabilize in K8S

Piggybacking onto this issue report. I'm running into similar issues as @wingfeng with large copies. The documentation indicates that the buffer is not a hard cap, but it does not clearly communicate in which situations the buffer size is relevant, or how it is calculated\used.
In my large test job (40TB), the RAM difference between jobs with AZCOPY_BUFFER_GB set or not set is minimal, with consistent RAM usage at about 15GB without the buffer set, or 14.5GB with the buffer set to 8GB.
Setting the --cap-mbps can help in some situations, but it's an indirect effort, not a proportional reduction.
I suspect for many users, there's less of a concern with setting a hard limit on RAM usage, and more a concern with leaving enough RAM available for the OS and other applications to function. I know in my environment that large jobs will run up sufficent RAM usage to eventually cause the OS to crash due to lack of resources. Maybe an application tweak to let AZCopy always leave 5% or 10% of system memory free might be a useful feature addition?

Another suggestion for large data sets is to break down the job. Here are some suggestions on how to break down jobs: https://github.com/Azure/azure-storage-azcopy/wiki/Performance:-small-blobs#job-size.

Closing due to lack of activity, and sharing suggestion above. Please feel free to reopen the issue in case you feel your issue hasn't been resolved.

authors: cdiskan gapra-msft siminsavani-msft wingfeng

repo: Azure/azure-storage-azcopy


text:
replaced AzCopy version update arm64 to x86_64 package

Addressed mismatch error: package architecture (arm64) does not match system (amd64) in the release pipeline.
Replaced the app architecture to be use a safer and more machine compatible x86_64 package
ADO Item

@wonwuakpa-msft can you please test and valid this Update version stage weather its running fine and updating the expected azcopy version in the latest_version.txt file?

authors: dphulkar-msft wonwuakpa-msft


text:
BlobStorage: Possibility of downloading a batch of blobs

Which service(blob, file, queue) does this issue concern?
blob
Which version of the SDK was used? Please provide the output of pip freeze.
azure-storage-blob==1.4.0
azure-storage-common==1.4.0
What problem was encountered?
I am missing the possibility of downloading/reading a batch of blobs (specifically a "folder") -- a functionality which is already present in the Azure CLI (az storage blob download-batch). I am currently using the service.get_blob_to_text(container: str, path: str), and I imagine the syntax being service.get_blobs_to_text(container: str, paths: iterable).
Have you found a mitigation/solution?
Currently I am reading the files one-by-one, which generates a performance overhead.

Hi @casparjespersen, thanks for reaching out!
We already have this feature on our roadmap, but I unfortunately do not have a timeline to share yet.

I created a Python wrapper for the Azure CLI to do downloads / uploads in batches, see here for more details:
Installation:
pip install azurebatchload

See the GitHub repo for more details and examples.

Other than @erfannariman 's noble attempt, is there any update for this getting support from the offical sdk?

authors: casparjespersen erfannariman ilayn zezha-msft

repo: Azure/azure-storage-python

license: mit

license_source: bigquery


text:
How to handle timeout for Uploading a blob in Azure Storage using Python SDK?

Which version of the SDK was used? Please provide the output of pip freeze.
azure-storage-blob==12.13.1
What problem was encountered?
I am trying to upload a blob to Azure Storage using Python SDK but I am unable to manage the timeout. The behavior of all timeout values are same.
from azure.storage.blob import BlobServiceClient, ContentSettings
from datetime import datetime

STORAGE_CONNECTION_STRING = ''
container = ''
blobFileName = ''
contentType = ''

def upload_blob(localFilePath):
    blob_service_client = BlobServiceClient.from_connection_string(STORAGE_CONNECTION_STRING)
    blob_client = blob_service_client.get_blob_client(container=container, blob=blobFileName)
    content_setting = ContentSettings(content_type=contentType)
    with open(localFilePath, "rb") as data:
        blob_client.upload_blob(data, overwrite=True, content_settings=content_setting, timeout=30)

@retry(Exception, tries=5, delay=1)
def uploadImageToAzure(imagePath):
    try:
        upload_blob(imagePath)
    except Exception as ex:
        print(f'Exception: {ex}, Type:{type(ex)}, Time:{datetime.now()}')
        raise Exception(ex)

    print('Successfully Uploaded the File')
    return True

uploadImageToAzure(imagePath)

To check the behavior of timeout. I just turn off my internet during uploading the blob.
Test Cases:
I just pick the random values of timeout and saw the behavior of the above code

timeout=1
Exception: <urllib3.connection.HTTPSConnection object at 0x7fe5d848e9a0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution, Type:<class 'azure.core.exceptions.ServiceRequestError'>, Time:2023-03-28 09:25:42.543262 Exception: <urllib3.connection.HTTPSConnection object at 0x7fe5d83c30a0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution, Type:<class 'azure.core.exceptions.ServiceRequestError'>, Time:2023-03-28 09:27:04.949785 Exception: <urllib3.connection.HTTPSConnection object at 0x7fe5d83da940>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution, Type:<class 'azure.core.exceptions.ServiceRequestError'>, Time:2023-03-28 09:28:28.508035 Exception: <urllib3.connection.HTTPSConnection object at 0x7fe5d837a340>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution, Type:<class 'azure.core.exceptions.ServiceRequestError'>, Time:2023-03-28 09:29:58.473318 Exception: <urllib3.connection.HTTPSConnection object at 0x7fe5d8393be0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution, Type:<class 'azure.core.exceptions.ServiceRequestError'>, Time:2023-03-28 09:31:19.184696
timeout=2
Exception: <urllib3.connection.HTTPSConnection object at 0x7f19f0164940>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution, Type:<class 'azure.core.exceptions.ServiceRequestError'>, Time:2023-03-28 09:46:25.328502 Exception: <urllib3.connection.HTTPSConnection object at 0x7f19f009b0a0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution, Type:<class 'azure.core.exceptions.ServiceRequestError'>, Time:2023-03-28 09:47:50.919091 Exception: <urllib3.connection.HTTPSConnection object at 0x7f19f00b2940>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution, Type:<class 'azure.core.exceptions.ServiceRequestError'>, Time:2023-03-28 09:49:14.144449 Exception: <urllib3.connection.HTTPSConnection object at 0x7f19f0056340>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution, Type:<class 'azure.core.exceptions.ServiceRequestError'>, Time:2023-03-28 09:50:43.947462 Exception: <urllib3.connection.HTTPSConnection object at 0x7f19f006ebe0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution, Type:<class 'azure.core.exceptions.ServiceRequestError'>, Time:2023-03-28 09:52:05.272197
timeout=4
Exception: <urllib3.connection.HTTPSConnection object at 0x7f2a08699910>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution, Type:<class 'azure.core.exceptions.ServiceRequestError'>, Time:2023-03-28 09:57:27.401257 Exception: <urllib3.connection.HTTPSConnection object at 0x7f2a085d41f0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution, Type:<class 'azure.core.exceptions.ServiceRequestError'>, Time:2023-03-28 09:58:55.699520 Exception: <urllib3.connection.HTTPSConnection object at 0x7f2a085ea910>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution, Type:<class 'azure.core.exceptions.ServiceRequestError'>, Time:2023-03-28 10:00:18.315710 Exception: <urllib3.connection.HTTPSConnection object at 0x7f2a096ce310>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution, Type:<class 'azure.core.exceptions.ServiceRequestError'>, Time:2023-03-28 10:01:37.800908 Exception: <urllib3.connection.HTTPSConnection object at 0x7f2a096e6bb0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution, Type:<class 'azure.core.exceptions.ServiceRequestError'>, Time:2023-03-28 10:03:02.293558
timeout=16
Exception: <urllib3.connection.HTTPSConnection object at 0x7ff8c83d7880>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution, Type:<class 'azure.core.exceptions.ServiceRequestError'>, Time:2023-03-28 10:19:15.195291 Exception: <urllib3.connection.HTTPSConnection object at 0x7ff8c830f130>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution, Type:<class 'azure.core.exceptions.ServiceRequestError'>, Time:2023-03-28 10:20:47.769388 Exception: <urllib3.connection.HTTPSConnection object at 0x7ff8c8323880>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution, Type:<class 'azure.core.exceptions.ServiceRequestError'>, Time:2023-03-28 10:22:14.996163 Exception: <urllib3.connection.HTTPSConnection object at 0x7ff8c82c8280>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution, Type:<class 'azure.core.exceptions.ServiceRequestError'>, Time:2023-03-28 10:23:37.736138 Exception: <urllib3.connection.HTTPSConnection object at 0x7ff8c82ddb20>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution, Type:<class 'azure.core.exceptions.ServiceRequestError'>, Time:2023-03-28 10:25:00.846901

What do you mean by "I am unable to manage the timeout"? Are you saying that in testing the various timeout values you are seeing the actual timeout does not change? I doubt you are actually testing the network timeout here. With your network disconnected, it is failing to resolve a domain name. "Temporary failure in name resolution" is reported in each case. That is likely a different timeout value which you are not controlling with the timeout parameter you specify. That parameter probably controls the timeout once the connection is actually established.


text:
404 error when using bindingName to trigger a function using HttpTrigger

I am working on creating a new function that needs to have a different URL path.
Expected URL to be :  http://localhost:7071/api/getCustomerAccounts/v1/customer-accounts/{userID}
I have configured  the userID as a binding in the API method  as below:
@FunctionName("getCustomerAccounts")
public HttpResponseMessage getCustomerAccounts(@HttpTrigger(name = "req", methods = {"get", "post"}, authLevel = AuthorizationLevel.ANONYMOUS) HttpRequestMessage request, @BindingName("userId") String userId,
final ExecutionContext context) {
context.getLogger().info("getCustomerAccounts API invoked.");
return request.createResponse(200, "Hello, " + userId);
}
I have also modified the function.json as below:
{
"scriptFile" : "..\customerAccountsAPI-1.0-SNAPSHOT.jar",
"entryPoint" : "com.sce.demo.api.customeraccounts.CustomerAccountsAPI.getCustomerAccounts",
"bindings" : [ {
"type" : "httpTrigger",
"name" : "req",
"direction" : "in",
"authLevel" : "anonymous",
"methods" : [ "get", "post" ],
"route": "v1/customer-accounts/{userId}"
},{
"type" : "String",
"name" : "userId",
"direction" : "in",
"authLevel" : "anonymous",
"methods" : [ "get", "post" ]
},
{
"type" : "http",
"name" : "$return",
"direction" : "out"
} ],
"disabled" : false
}
Now when I invoke the URL as  http://localhost:7071/api/getCustomerAccounts/v1/customer-accounts/123
I am getting a 404 error. Can you please let me know what am I missing? Also, I do not get to see the function.json, initially when I create the function using the mvn archetype command. I get to see after doing a clean package.  Is it an expected behavior and why so?

Few things here - you need to specify the route in your Java attribute to be the custom route you want.  Then the @BindingName("") value will map to the route parameter you specified.  For example here is a simple Function I wrote that returns the username route parameter which is an alphabetical string
package io.hollan.hello;

import com.microsoft.azure.serverless.functions.annotation.*;
import com.microsoft.azure.serverless.functions.*;

/**
 * Azure Functions with HTTP Trigger.
 */
public class Function {
    @FunctionName("hello")
    public String hello(@HttpTrigger(name = "req", methods = {"get", "post"}, authLevel = AuthorizationLevel.ANONYMOUS, route = "customers/{username:alpha}") HttpRequestMessage<String> request,
                                     @BindingName("username")String username,
                                     final ExecutionContext context) {
        context.getLogger().info("Java HTTP trigger processed a request.");

        return username;
    }
}

authors: getmckenzie jeffhollan

repo: Azure/azure-webjobs-sdk-script


text:
Expose function level TraceWriter to Binding Extensions

TraceWriter in ScriptBindingProvider and ExtensionConfigContext are host level trace writers. We need a TraceWriter that can log at function level
Repro steps
Provide the steps required to reproduce the problem
EnableTestSend on NH output binding:
{
"type": "notificationHub",
"name": "notification",
"hubName": "scripthub",
"connection": "scripthub_NOTIFICATIONHUB",
"direction": "out",
"enableTestSend": true,
"platform": "wns"
}
Expected behavior
NH debug logs show up in logs window
Actual behavior
NH debug logs show up in LogStream
Known workarounds
N/A
Related information
Provide any related information

Programming language used
Links to source

Do you mean that the actual behavior is that the logs only show in host logs, not function logs?
It might be the case that this is only an issue for extensions written on the "new model". E.g. your BuildFromAttribute func that creates your async collector doesn't have access to the actual invocation binding context that contains the per invocation file logger that we set up.

Yes. Correct. Logs only show up on host logs not function logs as Async collector does not have access to tracewriter for the invoker.

Is it possible to extend the ExtensionConfigContext.Trace to also log at function level? This would be super since we are using dependency injection in implementation of IExtensionConfigProvider to setup our logger. See code here

authors: jansoren mathewc pragnagopa


text:
Allow a template to specify required binding extensions

Templates should be able to pull in any extensions that they require. This would need to be defined in the template metadata, and when the user goes to create a function, the extensions are added to their function app.
Note that if the UX is being derived from (as proposed in https://github.com/Azure/azure-functions-ux/issues/1698), then the extension would actually have to be in place before we can paint the properties.
That may mean we need a multi-step process in the UX. After selecting a template:

(If you don't have any of the required extensions) See a warning that it details what extensions the template requires that you don't have. Click a button to install the extensions and continue.
See the template UX as normal.

If we want to go with that plan, we may need to create a work item in the UX repo for the multi-step process. There may be other solutions to this problem, though.

Is this for @soninaren?

Likely, yes.

authors: lindydonna mattchenderson

repo: Azure/azure-webjobs-sdk-templates


text:
Ensure TOC generation is working as expected

I noticed we're still manually maintaining the TOC (https://github.com/MicrosoftDocs/azure-resource-manager-schema-reference-pr/commit/933151be00425973457460890552e5f71143c1d4#diff-94f18876c75bfb0db040d9d8be40e6a52cf6e50dd6fece924982108282aec478). Can this be improved?

Closed by #38

repo: Azure/bicep-refdocs-generator


text:
[Feature Request]: Virtual Machine: make extensionDSCConfig secure

Description
The parameter extensionDSCConfig  object in the bicep-module for virtual machine is not secure. Therefore the password in the object under protectedSettings.configurationArguments.RegistrionKey.Password is not secure en will be visible when deploying.
In the module Virtual Machine the protected settings will process to the extension module and in the extension module the protectedSettings parameter is secure. But not in the module Virtual Machine.

Hey @rahalan,
I just migrated this issue over from CARML. Please take a look and triage if still relevant :)

repo: Azure/bicep-registry-modules


text:
feat: avm/ptn/data/private-analytical-workspace V0.1

Description

This is the initial implementation of the proposed pattern module detailed in proposal. https://github.com/Azure/Azure-Verified-Modules/issues/949 It includes documentation, unit tests, and PowerShell post-unit tests.
Pipeline Reference

Pipeline

Type of Change

[ ] Update to CI Environment or utilities (Non-module affecting changes)
[x] Azure Verified Module updates:

[ ] Bugfix containing backwards-compatible bug fixes, and I have NOT bumped the MAJOR or MINOR version in version.json:

[ ] Someone has opened a bug report issue, and I have included "Closes #{bug_report_issue_number}" in the PR description.
[ ] The bug was found by the module author, and no one has opened an issue to report it yet.

[x] Feature update backwards compatible feature updates, and I have bumped the MINOR version in version.json.
[x] Breaking changes and I have bumped the MAJOR version in version.json.
[x] Update to documentation

Checklist

[x] I'm sure there are no other open Pull Requests for the same update/change
[x] I have run Set-AVMModule locally to generate the supporting module files.
[x] My corresponding pipelines / checks run clean and green without any errors or warnings

@jbinko We have added examples for Bicep parameter files to the Readme. This has been applied to all published modules but needs to be done for PRs as well. Can you please update your branch and run the Set-AVMModule utility as detailed here. It is required for the validation pipeline to succeed and the contribution to be published.
Please reach out if any support is needed.

Updated documentation

All review comments are resolved.
Ready for next review.
BTW - I cannot run pipeline and execute tests. Something is wrong at the step "Set environment"
E: Unable to locate package powershell
It looks like generic issue. I'm seeing this for other pipelines as well.

All review comments are resolved. Ready for next review.
BTW - I cannot run pipeline and execute tests. Something is wrong at the step "Set environment" E: Unable to locate package powershell
It looks like generic issue. I'm seeing this for other pipelines as well.

Oh wow. Can't say I've seen that before. 👀
Any ideas @eriqua? If it does not work at all, I guess we could merge the PR into an upstream branch once it is ready to validate it there. But let's only go there if there is no other way as it would be much better to find out 'where' this issue comes from also for future contibutions.

All review comments are resolved. Ready for next review.
BTW - I cannot run pipeline and execute tests. Something is wrong at the step "Set environment" E: Unable to locate package powershell
It looks like generic issue. I'm seeing this for other pipelines as well.

3 comments left 💪 Thanks @jbinko

All review comments are resolved. Ready for next review.
BTW - I cannot run pipeline and execute tests. Something is wrong at the step "Set environment" E: Unable to locate package powershell
It looks like generic issue. I'm seeing this for other pipelines as well.

Oh wow. Can't say I've seen that before. 👀 Any ideas @eriqua? If it does not work at all, I guess we could merge the PR into an upstream branch once it is ready to validate it there. But let's only go there if there is no other way as it would be much better to find out 'where' this issue comes from also for future contibutions.

@AlexanderSehr only as a transient error unfortunately. From the history of @jbinko's fork this seems pretty consistent.
@jbinko did you get any other pipeline to pass the set-environment point?

Small addition: It seems to be related to: https://github.com/PowerShell/PowerShell/issues/23197#issuecomment-2402934692
Currently testing if we still need the explicit pwsh version installation (which was added due to a bug of a version). If we do, then we need to hardcode the runner version

OK, all changes completed on my side. Ready for merge. Tests and deployments are passing. @AlexanderSehr pipeline succeeds if I do changes  you proposed in the file \avm\utilities\pipelines\sharedScripts\Set-EnvironmentOnAgent.ps1. Those changes are not included in this PR. I removed them after try. I expect this will be changed in the main in original repo.

Hey @jbinko,
there are 2 final comments hidden behind this GitHub feature 😄

Hey @jbinko, there are 2 final comments hidden behind this GitHub feature 😄

Thanks @AlexanderSehr - got it. I fixed the remove of default test location and virtual network name propagation into dependencies files in tests.
Should be ready for merge now.

Hey @jbinko,
please not that many of the module's post-deployment tests did fail in the upstream workflow (ref). That means the module was not published. Could you please check if you can reproduce the errors you can see in the workflow's log? The deployment seemed to have worked.

Hey @jbinko, please not that many of the module's post-deployment tests did fail in the upstream workflow (ref). That means the module was not published. Could you please check if you can reproduce the errors you can see in the workflow's log? The deployment seemed to have worked.

Thanks @AlexanderSehr, I created new PR #3626 which addresses the post deployment tests issue.
The tests are run by default in the fixed location North Europe

authors: AlexanderSehr eriqua jbinko


text:
feat: Update AKS Automatic mode examples to indicate PREVIEW status

Description

Add documentation to test files to indicate that the AKS Automatic mode examples are in preview.
Reduce count in test files.
Pipeline Reference

Pipeline

Type of Change

[ ] Update to CI Environment or utilities (Non-module affecting changes)
[ ] Azure Verified Module updates:

[ ] Bugfix containing backwards-compatible bug fixes, and I have NOT bumped the MAJOR or MINOR version in version.json:

[ ] Someone has opened a bug report issue, and I have included "Closes #{bug_report_issue_number}" in the PR description.
[ ] The bug was found by the module author, and no one has opened an issue to report it yet.

[ ] Feature update backwards compatible feature updates, and I have bumped the MINOR version in version.json.
[ ] Breaking changes and I have bumped the MAJOR version in version.json.
[x] Update to documentation

Checklist

[x] I'm sure there are no other open Pull Requests for the same update/change
[x] I have run Set-AVMModule locally to generate the supporting module files.
[x] My corresponding pipelines / checks run clean and green without any errors or warnings

override PR #3646


text:
Support for msbuild variables & Directory.Build.props

Hi!
If csproj contains msbuild variables the "azds prep" fails with message:  Failed to prepare the current working directory: Project is not a dotnetcore project.
My csproj contains:
<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <TargetFramework>$(NetCoreTargetVersion)</TargetFramework>
    <AspNetCoreHostingModel>$(AspNetCoreHostingModel)</AspNetCoreHostingModel>
    <DockerDefaultTargetOS>$(DockerDefaultTargetOS)</DockerDefaultTargetOS>
  </PropertyGroup>

Then I have a Directory.Build.props file that sets the $(NetCoreTargetVersion), $(AspNetCoreHostingModel) and $(DockerDefaultTargetOS) to desired values.
I am using CLI version
Azure Dev Spaces CLI (Preview)
0.1.20190320.5
API v2.17

Good catch @eiximenis.  We had not accounted for that scenario - we'll work on a fix.

authors: YuzorMa eiximenis

repo: Azure/dev-spaces


text:
Remove link.Paused

Asking the link mux to evaluate its state is fine in any case.
There was a corner case where this would incorrectly be set.  If a
transfer frame is receiving from link.muxFlow() the link wouldn't be
marked as paused even though the credit was consumed.  Fixing this is
complex and doesn't make sense for an unnecessary feature.

[ ] The purpose of this PR is explained in this or a referenced issue.
[ ] Tests are included and/or updated for code changes.
[ ] Updates to CHANGELOG.md are included.
[ ] MIT license headers are included in each file.

This should also fix the flaky multi-part frame test failure.

repo: Azure/go-amqp


text:
swupdate image update requierments

Hi,
I'm kind of new to ADU and swupdate, so please forgive my ignorance.
We are thinking of using pi-gen to create images to be used on Raspberry Pi CM4.
We want to use ADU with swupdate and identity service. The image should also include IoT Edge Runtime (moby-engine etc.).
Is that possible or do swupdate require something (like uboot?) that Raspberry Pi OS doesn't have, to be able to make image updates.
As we want to use IoT Edge Runtime we have to use Raspberry Pi OS, as the current Yocto meta-layer with IoT Edge Runtime is "stuck" at version 0.9 and we want to use version 1.2, that supports identity service.
When doing a image update, how do you keep config, log files and similar, unchanged? The last thing we want to do is to update the OS and at the same time remove the connection string in the config.toml file, or something similar.
Thanks

Apologies for the late ack on this. I'll talk about this with @Nox-MSFT and others and report back once we have thorough answers for your questions.

I haven't used the pi-gen project to generate rpi images so cannot comment on feasibility but a really quick look it seems like you could add deviceupdate-agent package to 00-packages or 00-packages-nr.
The deviceupdate reference image along with other layers/recipes is built with yocto swupdate layer with uboot but I don't think its required per se.
One way is to omit data partition(s) from the image and apps should put configs/logs/etc in the data partition e.g. ln -s /data/etc/adu /etc/adu

The reference yocto swupdate image treats /adu partition as a data partition and survives .swu updates.

Closing due to inactivity. If additional information is needed, please feel free to create a new issue!

authors: FinitelyFailed esha0611 jw-msft

repo: Azure/iot-hub-device-update


text:
Checkout mlops-project-template@cindy-test to s/mlops-project-template

Describe the bug or the issue that you are facing
Is it correct to use a branch different to main?
Starting: Checkout mlops-project-template@cindy-test to s/mlops-project-template
Steps/Code to Reproduce
Same as https://github.com/Azure/mlops-v2/blob/main/documentation/deployguides/deployguide_ado.md
Section: Set up source repository with Azure DevOps
Expected Output
I dont have an error per see, but its weird to me that its using a named branch (cindy-test) instead of main.
Versions
I am using: initialise-project.yml
from mlops-v2 repo
Which platform are you using for deploying your infrastrucutre?
Azure DevOps (ADO)
If you mentioned Others, please mention which platformm are you using?
No response
What are you using for deploying your infrastrucutre?
Pre-Deployed
Are you using Azure ML CLI v2 or Azure ML Python SDK v2
Azure ML Python SDK v2
Describe the example that you are trying to run?
I am just following the section:
Setup MLOps V2 and a New MLOps Project in Azure DevOps
from:
https://github.com/Azure/mlops-v2/blob/main/documentation/deployguides/deployguide_ado.md
We use ADO not GHA

looks like when I imported the repos, the default branches are not the main ones

Same for mlops-v2 repo here on next run, which shows branch feathr as default (and main as failed, with further beautifully named main_xxx branches)

authors: corticalstack levalencia


text:
Support for Application Insights?

I created a custom telemetry initiualizer cfr ServiceFabric. it works in local debugging but not when i deploy to azure, any plans for this? In  local debug I do not seem to be able to tracing dependcie calls either without plumbing offcourse
In microservices this is a must (imo)

The SF module wouldn't work here and that wouldn't be our goal.  You should be able to build something very similar from the context passed via environment variables to each container.  We would eventually get to this if someone from the community doesn't get to it first.  App Insights folks have been driving some really cool work with OpenTelemetry that I have been keeping a close eye on as well.

yes I have written my own TelemetryInitializer which effectively reads the environment variables. It works fine on my devbox but not when i publish because I assume the module or extension is not enabled there. I will try to fiddle around in the arm template a bit to get it enabled
snippit of what I get below from my local devbox

authors: jellebens mattrowmsft

repo: Azure/service-fabric-mesh-preview


text:
[Feature]: Sensible to perform integration/unit tests using PowerShell's Pester (with psake)?

Module Name
N/A
Description
Just a question here -- I was wondering if the testing code documentation restricts testing of terraform module's to just the terratest golang framework.
I have been using a combination of the following PowerShell (Core) modules lately with my Terraform projects and find it to be very fluent:

Az: Azure Suite of PowerShell Modules
Pester: Tests
psake: Task Runner
Plaster: Module templating and best practices
PSDepend: Dependency Management and Versioning
Crescendo: (Optional) Could be used to create custom wrapper utilities for classic terraform-cli commands/scenarios (for example converting terraform plan to JSON or setting up the remote-backend / service principal, etc.)
CredentialManager: Manage credentials and authentication
platyPS: Generate documentation similar to terraform-docs does (although I still prefer terraform-docs here).

Similar to how PSRule (see Validate infrastructure as code (IaC) and objects using PowerShell rules) works nicely with Bicep or ARM Templates; I feel like these PowerShell tools really integrate nicely with the Terraform infrastructure as code ecosystem, specifically for Azure infrastructural deployments.
It also is a more friendly interface to use for a typical Azure developer that is more likely used to PowerShell syntax and development over Go, especially if having to write complex tests (notice the similarities between Pester/psake/DSC syntax and Terraform configuration syntax?).
Just wanted to see if anyone had any food for thoughts on this subject - any criticism or feedback would be much appreciated.
Thanks!

Jimmy

Example usage
No response
Other information
No response

Hi @jimbrig
Go is idiomatic of Terraform, historically. Due to this, I have created a fluent testing module in go, it allows code that is much more easy to read. It is built atop terratest.
See here: https://github.com/Azure/terratest-terraform-fluent
That's not to say we can't investigate other testing frameworks, but this might be worth considering.

@matt-FFFFFF - Awesome, thanks for that - will check it out. I'm starting to try and learn Go regardless due to the fact it teaches design patterns and best practices relevant across any language/framework.

Thanks @jimbrig for the suggestions! I'll take a look at the tools that you recommended.

CredentialManager: Manage credentials and authentication

We're using MSI as the authentication method, we're also considering OIDC/).

authors: jimbrig lonegunmanb matt-FFFFFF

repo: Azure/terraform-azure-modules


text:
chore: repository governance

Repository governance update
This PR was automatically created by the AVM Team hive-mind using the grept governance tool.
We have detected that some files need updating to meet the AVM governance standards.
Please review and merge with alacrity.
Grept config source: git::https://github.com/Azure/Azure-Verified-Modules-Grept.git//terraform
Thanks! The AVM team :heart:

Supersceeded by #31


text:
chore: repository governance

Repository governance update
This PR was automatically created by the AVM Team hive-mind using the grept governance tool.
We have detected that some files need updating to meet the AVM governance standards.
Please review and merge with alacrity.
Grept config source: git::https://github.com/Azure/Azure-Verified-Modules-Grept.git//terraform
Thanks! The AVM team :heart:

Supersceeded by #183


text:
chore: repository governance

Repository governance update
This PR was automatically created by the AVM Team hive-mind using the grept governance tool.
We have detected that some files need updating to meet the AVM governance standards.
Please review and merge with alacrity.
Grept config source: git::https://github.com/Azure/Azure-Verified-Modules-Grept.git//terraform
Thanks! The AVM team :heart:

Supersceeded by #21