text:
[Hub Generated] Review request for Microsoft.DataProtection to add version preview/2022-09-01-preview

This is a PR generated at OpenAPI Hub. You can view your work branch via this link.
ARM API Information (Control Plane)
Azure 1st Party Service can try out the Shift Left experience to initiate API design review from ADO code repo.  If you are interested, may request engineering support by filling in with the form https://aka.ms/ShiftLeftSupportForm.
Changelog
Add a changelog entry for this PR by answering the following questions:

What's the purpose of the update?

[ ] new service onboarding
[x] new API version
[ ] update existing version for new feature
[ ] update existing version to fix swagger quality issue in s360
[ ] Other, please clarify

When are you targeting to deploy the new service/feature to public regions? Please provide the date or, if the date is not yet available, the month.
When do you expect to publish the swagger? Please provide date or, the the date is not yet available, the month.
By default, Azure SDKs of all languages (.NET/Python/Java/JavaScript for both management-plane SDK and data-plane SDK, Go for management-plane SDK only ) MUST be refreshed with/after swagger of new version is published. If you prefer NOT to refresh any specific SDK language upon swagger updates in the current PR, please leave details with justification here.

Contribution checklist (MS Employees Only):

[x] I commit to follow the Breaking Change Policy of "no breaking changes"
[x] I have reviewed the documentation for the workflow.
[x] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

Applicability: :warning:
If your changes encompass only the following scenarios, you should SKIP this section, as these scenarios do not require ARM review.

Change to data plane APIs
Adding new properties
All removals

Otherwise your PR may be subject to ARM review requirements. Complete the following:

[x] Check this box if any of the following apply to the PR so that the label "ARMReview" and "WaitForARMFeedback" will be added by bot to kick off ARM API Review.  Missing to check this box in the following scenario may result in delays to the ARM manifest review and deployment.

Adding a new service
Adding new API(s)
Adding a new API version
-[ ] To review changes efficiently, ensure you copy the existing version into the new directory structure for first commit and then push new changes, including version updates, in separate commits. You can use OpenAPIHub to initialize the PR for adding a new version. For more details refer to the wiki.

[x] Ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[x] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If you have any breaking changes as defined in the Breaking Change Policy, request approval from the Breaking Change Review Board.
Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Additional details on the process and office hours are on the Breaking Change Wiki.
NOTE: To update API(s) in public preview for over 1 year (refer to Retirement of Previews)
Please follow the link to find more details on PR review process.

Most of the changes in this API have already been reviewed and approved by ARM team in a PR raised in swagger's private repo -
https://github.com/Azure/azure-rest-api-specs-pr/pull/7480

/azp run unifiedPipeline

    "state": {

What's the difference between On and AlwaysOn? If they are different, please add descriptions for the enum values to clarify (x-ms-enum allows per-value descriptions).

Refers to: specification/dataprotection/resource-manager/Microsoft.DataProtection/preview/2022-09-01-preview/dataprotection.json:6661 in 3d96f0c. [](commit_id = 3d96f0cc1dd10bc8960dec57b23c1e7ff75063bf, deletion_comment = False)

"/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataProtection/backupVaults/{vaultName}/deletedBackupInstances/{backupInstanceName}/undelete": {

ARM soft-delete pattern uses restore for this action.

Refers to: specification/dataprotection/resource-manager/Microsoft.DataProtection/preview/2022-09-01-preview/dataprotection.json:2560 in 3d96f0c. [](commit_id = 3d96f0cc1dd10bc8960dec57b23c1e7ff75063bf, deletion_comment = False)

    "state": {

What's the difference between On and AlwaysOn? If they are different, please add descriptions for the enum values to clarify (x-ms-enum allows per-value descriptions).
Refers to: specification/dataprotection/resource-manager/Microsoft.DataProtection/preview/2022-09-01-preview/dataprotection.json:6661 in 3d96f0c. [](commit_id = 3d96f0c, deletion_comment = False)

Added description for enum values. Thanks for the suggestion

/azp run unifiedPipeline

/azp run unifiedPipeline

    "state": {

Nicely done.

In reply to: 1260293048

Refers to: specification/dataprotection/resource-manager/Microsoft.DataProtection/preview/2022-09-01-preview/dataprotection.json:6661 in 3d96f0c. [](commit_id = 3d96f0cc1dd10bc8960dec57b23c1e7ff75063bf, deletion_comment = False)

"/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataProtection/backupVaults/{vaultName}/deletedBackupInstances/{backupInstanceName}/undelete": {

Makes sense.

In reply to: 1260293405

Refers to: specification/dataprotection/resource-manager/Microsoft.DataProtection/preview/2022-09-01-preview/dataprotection.json:2560 in 3d96f0c. [](commit_id = 3d96f0cc1dd10bc8960dec57b23c1e7ff75063bf, deletion_comment = False)

    "state": {

Recommend adding per-value descriptions for this property as well (not blocking ARM signoff).

Refers to: specification/dataprotection/resource-manager/Microsoft.DataProtection/preview/2022-09-01-preview/dataprotection.json:5505 in b503351. [](commit_id = b503351dcd921a9006e589736180b86a71603623, deletion_comment = False)

    "resourceGuardOperationRequests": {

Description would be helpful.

Refers to: specification/dataprotection/resource-manager/Microsoft.DataProtection/preview/2022-09-01-preview/dataprotection.json:7157 in b503351. [](commit_id = b503351dcd921a9006e589736180b86a71603623, deletion_comment = False)

@amchandn - Signed off for ARM with comments.

    "state": {

Recommend adding per-value descriptions for this property as well (not blocking ARM signoff).
Refers to: specification/dataprotection/resource-manager/Microsoft.DataProtection/preview/2022-09-01-preview/dataprotection.json:5505 in b503351. [](commit_id = b503351, deletion_comment = False)

thanks for the suggestion. We'll plan and take up this exercise to add enum descriptions for all enum values in our swagger in upcoming versions.

    "resourceGuardOperationRequests": {

Description would be helpful.
Refers to: specification/dataprotection/resource-manager/Microsoft.DataProtection/preview/2022-09-01-preview/dataprotection.json:7157 in b503351. [](commit_id = b503351, deletion_comment = False)

thanks for the suggestion. We'll plan and take up this exercise to add all missing descriptions in our swagger in upcoming versions.

/azp run

authors: amchandn jianyexi mentat9


text:
Add MaintenanceWindow into MaintenanceConfigurationProperties

ARM API Information (Control Plane)
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Azure 1st Party Service can try out the Shift Left experience to initiate API design review from ADO code repo.  If you are interested, may request engineering support by filling in with the form https://aka.ms/ShiftLeftSupportForm.
Changelog
Add a changelog entry for this PR by answering the following questions:

What's the purpose of the update?

[ ] new service onboarding
[ ] new API version
[ ] update existing version for new feature
[ ] update existing version to fix swagger quality issue in s360
[ ] Other, please clarify

When are you targeting to deploy the new service/feature to public regions? Please provide the date or, if the date is not yet available, the month.
When do you expect to publish the swagger? Please provide date or, the the date is not yet available, the month.
By default, Azure SDKs of all languages (.NET/Python/Java/JavaScript for both management-plane SDK and data-plane SDK, Go for management-plane SDK only ) MUST be refreshed with/after swagger of new version is published. If you prefer NOT to refresh any specific SDK language upon swagger updates in the current PR, please leave details with justification here.

Contribution checklist (MS Employees Only):

[ ] I commit to follow the Breaking Change Policy of "no breaking changes"
[ ] I have reviewed the documentation for the workflow.
[ ] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

Applicability: :warning:
If your changes encompass only the following scenarios, you should SKIP this section, as these scenarios do not require ARM review.

Change to data plane APIs
Adding new properties
All removals

Otherwise your PR may be subject to ARM review requirements. Complete the following:

[ ] Check this box if any of the following apply to the PR so that the label "ARMReview" and "WaitForARMFeedback" will be added by bot to kick off ARM API Review.  Missing to check this box in the following scenario may result in delays to the ARM manifest review and deployment.

Adding a new service
Adding new API(s)
Adding a new API version
-[ ] To review changes efficiently, ensure you copy the existing version into the new directory structure for first commit and then push new changes, including version updates, in separate commits. You can use OpenAPIHub to initialize the PR for adding a new version. For more details refer to the wiki.

[ ] Ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If you have any breaking changes as defined in the Breaking Change Policy, request approval from the Breaking Change Review Board.
Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Additional details on the process and office hours are on the Breaking Change Wiki.
NOTE: To update API(s) in public preview for over 1 year (refer to Retirement of Previews)
Please follow the link to find more details on PR review process.

The errors scanned by Swagger ModelValidation is irrelevant to the changes in this PR. I'll help bypass the check after the identifier issue got fixed.

source: gharchive/pull-request

repo: Azure/azure-rest-api-specs

license: MIT

license_type: permissive

license_source: github-api


text:
updated resource id in examples Azure Monitor for SAP

updated resource id in examples Azure Monitor for SAP for API version 2021-12-01-preview
ARM API Information (Control Plane)
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Azure 1st Party Service can try out the Shift Left experience to initiate API design review from ADO code repo.  If you are interested, may request engineering support by filling in with the form https://aka.ms/ShiftLeftSupportForm.
Changelog
Add a changelog entry for this PR by answering the following questions:

What's the purpose of the update?

[ ] new service onboarding
[ ] new API version
[ ] update existing version for new feature
[ ] update existing version to fix swagger quality issue in s360
[ ] Other, please clarify

When are you targeting to deploy the new service/feature to public regions? Please provide the date or, if the date is not yet available, the month.
When do you expect to publish the swagger? Please provide date or, the the date is not yet available, the month.
By default, Azure SDKs of all languages (.NET/Python/Java/JavaScript for both management-plane SDK and data-plane SDK, Go for management-plane SDK only ) MUST be refreshed with/after swagger of new version is published. If you prefer NOT to refresh any specific SDK language upon swagger updates in the current PR, please leave details with justification here.

Contribution checklist (MS Employees Only):

[ ] I commit to follow the Breaking Change Policy of "no breaking changes"
[ ] I have reviewed the documentation for the workflow.
[ ] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

Applicability: :warning:
If your changes encompass only the following scenarios, you should SKIP this section, as these scenarios do not require ARM review.

Change to data plane APIs
Adding new properties
All removals

Otherwise your PR may be subject to ARM review requirements. Complete the following:

[ ] Check this box if any of the following apply to the PR so that the label "ARMReview" and "WaitForARMFeedback" will be added by bot to kick off ARM API Review.  Missing to check this box in the following scenario may result in delays to the ARM manifest review and deployment.

Adding a new service
Adding new API(s)
Adding a new API version
-[ ] To review changes efficiently, ensure you copy the existing version into the new directory structure for first commit and then push new changes, including version updates, in separate commits. You can use OpenAPIHub to initialize the PR for adding a new version. For more details refer to the wiki.

[ ] Ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If you have any breaking changes as defined in the Breaking Change Policy, request approval from the Breaking Change Review Board.
Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Additional details on the process and office hours are on the Breaking Change Wiki.
NOTE: To update API(s) in public preview for over 1 year (refer to Retirement of Previews)
Please follow the link to find more details on PR review process.

@M2skills , can you revert the changes on the root folder files and also resolve the conflicts?

@M2skills , feel free to re-open this PR if needed.


text:
New API for communication call settings [Private preview]

Data Plane API Specification Update Pull Request

[!TIP]
Overwhelmed by all this guidance? See the Getting help section at the bottom of this PR description.

Introducing new provisioning API for ACS calling, that will allow customers to store call related settings on ACS resource level and ACS participant level.
TypeSpec link
PR review workflow diagram
Please understand this diagram before proceeding. It explains how to get your PR approved & merged.

API Info: The Basics
Most of the information about your service should be captured in the issue that serves as your API Spec engagement record.

Link to API Spec engagement record issue:

Is this review for (select one):

[x] a private preview
[ ] a public preview
[ ] GA release

Change Scope
This section will help us focus on the specific parts of your API that are new or have been modified. Please share a link to the design document for the new APIs, a link to the previous API Spec document (if applicable), and the root paths that have been updated.

Design Document:
Previous API Spec Doc: N/A
Updated paths: N/A

Viewing API changes
For convenient view of the API changes made by this PR, refer to the URLs provided in the table
in the Generated ApiView comment added to this PR. You can use ApiView to show API versions diff.
Suppressing failures
If one or multiple validation error/warning suppression(s) is detected in your PR, please follow the
Swagger-Suppression-Process
to get approval.
❔Got questions? Need additional info?? We are here to help!

 Contact us!
The Azure API Review Board is dedicated to helping you create amazing APIs. You can read about our mission and learn more about our process on our wiki.

💬 Teams Channel
💌 email

Click here for links to tools, specs, guidelines & other good stuff
Tooling

Open API validation tools were run on this PR. Go here to see how to fix errors
Spectral Linting

Guidelines & Specifications

Azure REST API Guidelines
OpenAPI Style Guidelines
Azure Breaking Change Policy

Helpful Links

Schedule a data plane REST API spec review

Getting help

First, please carefully read through this PR description, from top to bottom.
If you don't have permissions to remove or add labels to the PR, request write access per aka.ms/azsdk/access#request-access-to-rest-api-or-sdk-repositories
To understand what you must do next to merge this PR, see the Next Steps to Merge comment. It will appear within few minutes of submitting this PR and will continue to be up-to-date with current PR state.
For guidance on fixing this PR CI check failures, see the hyperlinks provided in given failure
and https://aka.ms/ci-fix.
If the PR CI checks appear to be stuck in queued state, please add a comment with contents /azp run.
This should result in a new comment denoting a PR validation pipeline has started and the checks should be updated after few minutes.
If the help provided by the previous points is not enough, post to https://aka.ms/azsdk/support/specreview-channel and link to this PR.

fix https://github.com/Azure/azure-rest-api-specs/issues/30399

From our prior conversations, this is only for private preview.
If we're going with the new Swagger route, then we should include the TypeSpec sources.

authors: DominikMe jiriburant


text:
nsp async association api spec changes

Choose a PR Template
Switch to "Preview" on this description then select one of the choices below.
Click here to open a PR for a Data Plane API.
Click here to open a PR for a Control Plane (ARM) API.

@shublnu please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
@microsoft-github-policy-service agree [company="{your company}"]

Options:

(default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.

@microsoft-github-policy-service agree

(when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.

@microsoft-github-policy-service agree company="Microsoft"

Contributor License Agreement

@microsoft-github-policy-service agree company="Microsoft"

Choose a PR Template description and add a description to this pr to have the Purpose of PR and due diligence sections added

The first commit needs to be an exact copy of the previous API version. All new changes should only be added in the subsequent commits.
This allows the reviewer to get a clear understanding of the actual changes being introduced. With the way the PR is raised now, it is not possible for the reviewer to tell what the changes are. Please either abandon the PR and raise another one with the recommendation or create a new set of commits on this PR following the recommendation. If you are doing the later option please indicate which commit is the exact copy of the previous version.

authors: razvanbadea-msft shublnu


text:
New Put API for updating vault security config

Latest improvements:
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Contribution checklist:

[x] I have reviewed the documentation for the workflow.
[x] Validation tools were run on swagger spec(s) and have all been fixed in this PR.
[ ] The OpenAPI Hub was used for checking validation status and next steps.

ARM API Review Checklist

[ ] Service team MUST add the "WaitForARMFeedback" label if the management plane API changes fall into one of the below categories.
adding/removing APIs.
adding/removing properties.
adding/removing API-version.
adding a new service in Azure.

Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs.

[ ] If you are blocked on ARM review and want to get the PR merged urgently, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.
Please follow the link to find more details on API review process.

You don't have permission to trigger SDK Automation.
Please add yourself to Azure group from opensource portal if you are MSFT employee,
or please ask reviewer to add comment *** /openapibot sdkautomation ***.
Please ask tih@microsoft.com (or NullMDR in github) for additional help.

/azp run automation - sdk

/azp run automation - sdk

/azp run automation - sdk

Can one of the admins verify this patch?


text:
Update mountTarget type definition

Latest improvements:
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Contribution checklist:

[x] I have reviewed the documentation for the workflow.
[x] Validation tools were run on swagger spec(s) and have all been fixed in this PR.
[ ] The OpenAPI Hub was used for checking validation status and next steps.

ARM API Review Checklist

[ ] Service team MUST add the "WaitForARMFeedback" label if the management plane API changes fall into one of the below categories.
adding/removing APIs.
adding/removing properties.
adding/removing API-version.
adding a new service in Azure.

Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs.

[ ] If you are blocked on ARM review and want to get the PR merged urgently, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.
Please follow the link to find more details on API review process.

/azp run automation - sdk

URGENT, This is a bug fix that already was put in other API versions see pr 9078

Can one of the admins verify this patch?

authors: AutorestCI audunn azuresdkci


text:
azure-sdk-for-c-arduino, can't see telemetry data from devices in AZURE,ESP32

Describe the bug
can't see telemetry data from devices in AZURE,ESP32
To Reproduce
example from libarary
https://github.com/Azure/azure-sdk-for-c-arduino/blob/main/examples/Azure_IoT_Hub_ESP32/readme.md
Expected behavior
It looks like the device is not sending payloads
Additional context
see my discussion details on stackoverflow
https://stackoverflow.com/questions/76687945/cant-see-telemetry-data-from-devices-in-azure-esp32
In the end result, I'm interested in data transmission via cellular communication, ESP32 and SIM7000 module connected to AZURE IoT HUB
Setup :

OS: [Windows10]

IDE: ARDUINO

Version of the Library used: Last available

[ ] Bug Description Added

[ ] Repro Steps Added

[ ] Setup information Added

Problem solved in https://github.com/Azure/azure-sdk-for-c/issues/2611

source: gharchive/issue

repo: Azure/azure-sdk-for-c


text:
Split test capture step out from build step

Currently the builds run gtest_discover_tests as a final step of the tests.  This step should be split out into its own step to speed up the builds and help isolate points of failure.

Related issue tracking resolving the failures:
https://github.com/Azure/azure-sdk-for-cpp/issues/1607

Having isolated the point of failure to GTest_Discover_Tests , splitting the step out is pointless and actually lengthens the build process as it being part of the cmake files, the only way to have it executed or not is to run cmake config / build twice. which overall lengthen the build process.
We cannot have only the discovery test step run , but we can actually increase the timeout of the discovery from 5 s currently ( which is too short ) to something more apropriate.
To quote the cmake gtest documentation for discovery timeout "Most test executables will enumerate their tests very quickly, but under some exceptional circumstances, a test may require a longer timeout. The default is 5. " the exceptional circumstances are not elaborated further.
Why the current implementation of the timeout is not working is due to the fact that it is in a macro nobody calls , protected by an ENV that nobody sets thus it never really executes.

authors: RickWinter ahsonkhan gearama

repo: Azure/azure-sdk-for-cpp


text:
Setting up keyvault live test

Setting up KeyVault live testing

/azp run cpp - keyvault

/azp run cpp - keyvault

/azp run cpp - keyvault

/azp run cpp - keyvault

/azp run cpp - keyvault

/azp run cpp - keyvault


text:
armsubscription.Subscription: missing Tags

Bug Report

import path: /sdk/resourcemanager/subscription/armsubscription
SDK version: latest
go version: go version go1.18.3 darwin/arm64

Subscription struct doesn't include Tags:
type Subscription struct {
	// The authorization source of the request. Valid values are one or more combinations of Legacy, RoleBased, Bypassed, Direct
	// and Management. For example, 'Legacy, RoleBased'.
	AuthorizationSource *string `json:"authorizationSource,omitempty"`

	// The subscription policies.
	SubscriptionPolicies *Policies `json:"subscriptionPolicies,omitempty"`

	// READ-ONLY; The subscription display name.
	DisplayName *string `json:"displayName,omitempty" azure:"ro"`

	// READ-ONLY; The fully qualified ID for the subscription. For example, /subscriptions/00000000-0000-0000-0000-000000000000.
	ID *string `json:"id,omitempty" azure:"ro"`

	// READ-ONLY; The subscription state. Possible values are Enabled, Warned, PastDue, Disabled, and Deleted.
	State *SubscriptionState `json:"state,omitempty" azure:"ro"`

	// READ-ONLY; The subscription ID.
	SubscriptionID *string `json:"subscriptionId,omitempty" azure:"ro"`
}

https://github.com/Azure/azure-sdk-for-go/blob/main/sdk/resourcemanager/subscription/armsubscription/zz_generated_models.go

@mblaschke Thanks for your feedback. The return body of Get operation of subscription does not contains tag for now. As this is a feature request, I'll involve service team to have a look.

Adding Service team to look into this feature request.

according to https://docs.microsoft.com/en-us/rest/api/resources/subscriptions/get it returns the tags
This is not really a feature as the old sdk also provides subscription tags.

Swagger default tag still using old 2016 api-version. @anyone from service team, please help to do the upgrade.

@tadelesh I will check with the Service team offline and update this github thread.

You might want to use all the models under “resourcemanager/resources” instead of “resourcemanager” (might has the old version, not sure why it hasn’t been removed).
Please give it a try with https://github.com/Azure/azure-sdk-for-go/blob/main/sdk/resourcemanager/resources/armsubscriptions/zz_generated_models.go (I do see “tags” in subscription struct) instead of https://github.com/Azure/azure-sdk-for-go/blob/main/sdk/resourcemanager/subscription/armsubscription/zz_generated_models.go (without tags).

@Grayer123 Is the package of subscription/armsubsciption replaced by resources/armsubscriptions totally? If so, we could deprecate the former one and let the customer use the later one.

@tadelesh  Got a confirmation from @Grayer123 that it should be replaced.

@navba-MSFT Thanks. @mblaschke You could use package github.com/Azure/azure-sdk-for-go/resources/armsubscriptions.

authors: Grayer123 mblaschke navba-MSFT tadelesh

repo: Azure/azure-sdk-for-go


text:
Enhancement: combine event hub in-memory and blob checkpointing feature

Feature Request
This is regarding the checkpointing feature in the Event Hub Go client. I'm wondering if we can combine the benefits of in-memory checkpoints and persistent checkpointing to blob storage. Persisting to blob storage on every checkpoint has resulted in slower performance for my team's application (which requires low-latency processing of telemetry). In-memory persistence, on the other hand, means data will be lost when the application dies or restarts.
I propose adding an option to only persist to blob storage after a time interval or after a certain number of checkpoints.

I propose adding an option to only persist to blob storage after a time interval or after a certain number of checkpoints.

We've generally left this pattern up to the application writer. Generally your app wants full visibility when you bypass certain areas of safety.
Now, your application is in full control of when UpdateCheckpoint is called, so if you want to do a "only write after 'x' time" style pattern it's a simple update to wrap the update with your own logic:
if time.Since(lastCheckpointTime) > arbitraryDuration {
	if err := partitionClient.UpdateCheckpoint(context.TODO(), events[len(events)-1], nil); err != nil {
		return err
	}

	lastCheckpointTime = time.Now()
}

The compromise, as you mention, is whether you're okay with possibly reprocessing events if you lose a processor instance and have to start again.
Closing as we're not going to add this to the SDK at this time.

authors: ifeify richardpark-msft


text:
[Release] sdk/resourcemanager/trafficmanager/armtrafficmanager/2.0.0-beta.1

https://github.com/Azure/sdk-release-request/issues/3321

@Alancere Please help to change to minor version.

authors: azure-sdk tadelesh


text:
Update Changelog, ACS reorg and delete extra services for v12

Thanks you for your contribution to the Azure-SDK-for-Go! We will triage and review it as quickly as we can.
As part of your submission, please make sure that you can make the following assertions:

[ ] I'm not making changes to Auto-Generated files which will just get erased next time there's a release.

If that's what you want to do, consider making a contribution here: https://github.com/Azure/autorest.go

[ ] I've tested my changes, adding unit tests where applicable.
[ ] I've added Apache 2.0 Headers to the top of any new source files.
[ ] I'm submitting this PR to the dev branch, or I'm fixing a bug that warrants its own release and I'm targeting the master branch.
[ ] If I'm targeting the master branch, I've also added a note to CHANGELOG.md.
[ ] I've mentioned any relevant open issues in this PR, making clear the context for the contribution.

#861


text:
Sync eng/common directory with azure-sdk-tools for PR 1386

Sync eng/common directory with azure-sdk-tools for PR https://github.com/Azure/azure-sdk-tools/pull/1386  See eng/common workflow

/check-enforcer reset

authors: azure-sdk weshaggard

repo: Azure/azure-sdk-for-ios


text:
Mechanism for Mono to Clone Swift PM Repos

The goal here is for us to have fine-grained, independent version control for SwiftPM modules the way we do in other SDKs and with CocoaPods while maintaining the benefits of a mono-repo.
To facilitate this, we will have a per-module Package.swift and CHANGELOG.md. Each module will be represented in manifest.yml with its path in the mono-repo and its mirror repo name.
Whenever the mono-repo is updated, we can update the mirror repo with:
python3 eng/scripts/sync_repo.py <MODULE>
This will copy that branch to the root of the mirror, which can then be target by SwiftPM. In the current model, users target the "AzureSDK" package, and then specify individual modules, whereas with this change, the target will be the module.
The current example of this is AzureCore, which will mirror to azure-sdk-for-ios-core.

This PR seems to have two strategies in play. One strategy that has been discussed is having a copy of all the modules in a separate repository (synchronized from the mono-repo). But there also appears to be a reference to an azure-sdk-for-ios-core repository as well (which makes me think that this is just breaking out core separately, and will be managed as a separate repo (incl the engineering system etc).

This PR seems to have two strategies in play. One strategy that has been discussed is having a copy of all the modules in a separate repository (synchronized from the mono-repo). But there also appears to be a reference to an azure-sdk-for-ios-core repository as well (which makes me think that this is just breaking out core separately, and will be managed as a separate repo (incl the engineering system etc).

@mitchdenny thanks for the catch. I updated the reference to SwiftPM-AzureCore. Also, @weshaggard I updated the sync_repo.py script so it no longer requires any metadata file.

@weshaggard @mitchdenny this PR should be ready for review.

Another thing that needs to be worked out is the tagging strategy when a release goes out. Is the plan just to replace the contents of the repo and then commit and tag? So we could end up with something like this:
commit1: 1.0.0-beta.11
commit2: 1.0.0
commit3: 1.1.0-beta.1
commit4: 1.0.1

There would be corresponding tags on the monorepo like:
commit1: AzureCore_1.0.0-beta.11
commit2: AzureCore_1.0.0 (albeit not anytime soon)
commit3: AzureCore_1.1.0-beta.1
commit4: AzureCore_1.0.1
However, if that were the sequence in both repos, I don't see how either would be any less confusing. \

I think there will also need to be some content in the README that explains how/why these repos are done this way. Imagine a scenario where someone inherits an iOS code-base and they go and look at their dependencies and they see one of our swiftpm-* links. They may not have previously worked with the Azure SDK before so wouldn't know why we have these satellite repos.

I think, most likely, they wouldn't think twice about the SwiftPM repo but would be surprised to learn of the existence of the mono-repo. There would be no reference to azure-sdk-for-ios in their files, but if they wanted to file a bug or contribute, the README would direct them to the mono-repo. But you are right, we should call this out in some fashion somewhere.

authors: mitchdenny tjprescott


text:
Deprecation of Service Bus Processor Stop API

Today Service Bus Processor exposes two APIs that complete the message pumping - stop and close.
The mainline use case is - the application wants the Processor to pump the message forever. When/If the application shuts down, it calls close API to dispose of the Processor.
As the engineering team recently reevaluated the past design choices, we identified that the stop API involves quite a complexity; supporting it correctly involves a reasonable (allocation, coordination) overhead and contributes to engineering costs. Additionally, the above mainline use case (that does not use stop) pays the price of this overhead.
The engineering team came to the conclusion that the mainline use case will greatly benefit from deprecating and removing stop , and saves engineering and maintenance costs.
Going forward, starting Processor that was stopped before is not recommended and this feature may be deprecated in future. Recommendation is to close the Processor instance and create a new one to restart processing.

The April 2023 version (v7.13.4) of Service Bus SDK includes a log message in the warning level to communicate this upcoming change; this is the pr.

We run a async "short-lived" client for receving messages and has tested recreating clients with the use of close(). But this does not behave in a gracefull way today. This is a list of my top values from the logs when we call close() after a few days in production with version 7.13.4 of the java lib (I guess some of the messages could be ignored, but still...)

Cannot perform operation 'abandoned' on a disposed receiver.
The receiver didn't receive the disposition acknowledgment due to receive link closure.
java.lang.InterruptedException
Cannot subscribe. Processor is already terminated
Cannot perform operations on a disposed set.
Delivery not on receive link.
Cannot perform operation 'renewMessageLock' on a disposed receiver

Maybe a combination of first calling stop(), and the close() a short while later could be good choice?

We definitely need a way for in-process messages to complete just before close, otherwise it's very easy to run into edge cases where the work completes around processing a message, but the complete call then fails because the processor was stopped/closed.

authors: MT-Jacobs anuchandy trefsahl

repo: Azure/azure-sdk-for-java


text:
[QUERY] RequestRetryOptions seems did not work at the SpringBoot

Query/Question
My app is SpringBoot 3.1.0 and the package is using spring cloud azure.
<dependency>
           <groupId>com.azure.spring</groupId>
            <artifactId>spring-cloud-azure-starter-storage-blob</artifactId>
            <version>5.5.0</version>
</dependency>

This Java app uses multiple threads to upload images(jpeg, png) into Azure Blob Storage. Each thread is uploading one image, but I face intermittently upload timeout.
how to justify when there is a timeout,  I set the timeout as 30 seconds because I think it should not be over 30 seconds to upload a tiny image. Please see my following attached code.
In order to cushion this timeout issue, I want to use the re-try logic to failover if over 30 seconds

Why is this not a Bug or a feature Request?
Now I have already introduced a re-try logic as the following code, but I CAN NOT see the re-try log record when I intentionally make the image upload timeout into a quite small value eg: 10 ms, and the re-try account is 3, but I can not see three-times re-try execution log!
    @Bean
    public BlobServiceClient getBlobServiceClient() {
        RequestRetryOptions requestRetryOptions = new RequestRetryOptions(RetryPolicyType.EXPONENTIAL, 3, 1, 10000L, 20000L, null);

        return new BlobServiceClientBuilder().
                retryOptions(requestRetryOptions).
                connectionString("******").buildClient();

    }

    @Async("taskExecutor")
    public CompletableFuture<String> migrateImageToAzureStorage(Image image, BlobContainerClient forecastOrObsContainer, byte[] imageData) {

        String imageAzureBlobName = image.getImageId().toString() + guessFileExtension(imageData);
        try {
            BlobClient blobClient = forecastOrObsContainer.getBlobClient(imageAzureBlobName);

            try (ByteArrayInputStream imageStream = new ByteArrayInputStream(imageData)) {
                blobClient.uploadWithResponse(new BlobParallelUploadOptions(imageStream).setTier(AccessTier.HOT).setRequestConditions(new BlobRequestConditions()),
                        Duration.ofMillis(10ms),
                        Context.NONE);
                LogUtil.info("Uploaded Image "+image.getImageId()+" to Azure Blob Storage HOT Tier as " + blobClient.getBlobUrl());
            }

            image.setObjectStoreUrl(blobClient.getBlobName());

            return CompletableFuture.completedFuture(blobClient.getBlobName());
        } catch (IOException e) {
            LogUtil.error("Failed for closing stream when UPLOAD image "+image.getImageId()+" for Azure as "+imageAzureBlobName, e);
        } catch (IllegalStateException e) {
            LogUtil.error("Timeout for UPLOADING image "+image.getImageId()+" for Azure ", e);
        } catch (Exception e) {
            LogUtil.error("Failed for uploading image "+image.getImageId()+" for Azure ", e);
        }

        // return a failed future
        CompletableFuture<String> failedFuture = new CompletableFuture<>();
        failedFuture.completeExceptionally(new MigrationException("Failed for uploading image "+image.getImageId()+" for Azure "));
        return failedFuture;
    }

Setup (please complete the following information if applicable):

OS: [e.g. iOS]
IDE: [e.g. IntelliJ]

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

[x] Query Added
[x] Setup information Added

@alzimmermsft
Thanks for your details explanation.
I removed the timeout setting from BlobClient and assigned it to the RequestRetryOptions. Now it works and the following is the sudo code, Please have a look.
But may I check with you about several things

Could you please share some official documents to describe the timeout difference between among BlobClient and RequestRetryOptions?
I may miss something because I found it hard to catch the azure upload timeout exception after running out of re-try count, eg: what is the particular exception type, so I have to use a crude way to match the exception string which you can see it among my code
Seems I have to add the HttpLogOptions to let the azure related logs be represented, eg: the following logs
seems I have to fetch out these logs with "az.sdk.message", but I still will miss some part, eg: the second log, in this instance, is there any way I can use to fetch all these logs in one time?

{"@timestamp":"2023-09-12T02:24:29.712Z","ecs.version":"1.2.0","log.level":"INFO","message":"{\"az.sdk.message\":\"HTTP request\",\"method\":\"PUT\",\"url\":\"https://host.blob.core.windows.net/forecastimage/1565385274.jpeg\",\"tryCount\":\"3\",\"contentLength\":236608141}","process.thread.name":"parallel-4","log.logger":"com.azure.storage.blob.implementation.BlockBlobsImpl$BlockBlobsService.upload"}

{"@timestamp":"2023-09-12T02:24:33.717Z","ecs.version":"1.2.0","log.level":"WARN","message":"[id: ***********, L:/********:***** - R:host.blob.core.windows.net/********:*****] Last write attempt timed out; force-closing the connection.","process.thread.name":"reactor-http-kqueue-3","log.logger":"io.netty.handler.ssl.SslHandler"}

    @Bean
    public BlobServiceClient getBlobServiceClient() {
        RequestRetryOptions requestRetryOptions = new RequestRetryOptions(RetryPolicyType.FIXED, retryMaxCount, tryTimeoutInMs/1000, retryDelayInMs, maxRetryDelayInMs, null);

        HttpLogOptions httpLogOptions = new HttpLogOptions().setLogLevel(HttpLogDetailLevel.BASIC);

        BlobServiceClient blobServiceClient =
                new BlobServiceClientBuilder().httpLogOptions(httpLogOptions).retryOptions(requestRetryOptions).connectionString(connectStr).buildClient();

        return blobServiceClient;
    }

    @Async("taskExecutor")
    public CompletableFuture<String> migrateImageToAzureStorage(Image image, BlobContainerClient forecastOrObsContainer, byte[] imageData) {

        String imageAzureBlobName = image.getImageId().toString() + guessFileExtension(imageData);

        // record the size of each image and represent at the log
        double imageSizeInMB = imageData.length / 1024.0 / 1024.0;

        try {
            BlobClient blobClient = forecastOrObsContainer.getBlobClient(imageAzureBlobName);

            try (ByteArrayInputStream imageStream = new ByteArrayInputStream(imageData)) {
                blobClient.uploadWithResponse(new BlobParallelUploadOptions(imageStream).setTier(AccessTier.HOT).setRequestConditions(new BlobRequestConditions()),
                        null,
                        Context.NONE);
                LogUtil.info("Uploaded Image "+image.getImageId()+", Size(MB): " +imageSizeInMB+" to Azure Blob Storage HOT Tier as " + blobClient.getBlobUrl());
            }

            image.setObjectStoreUrl(blobClient.getBlobName());

            return CompletableFuture.completedFuture(blobClient.getBlobName());
        } catch (IOException e) {
            LogUtil.error("Failed for closing stream when UPLOAD image "+image.getImageId()+", Size(MB): " +imageSizeInMB+ " for Azure as "+imageAzureBlobName, e);
        } catch (Exception e) {
            // crude way to check the azure blob upload timeout exception
            if(e.getMessage().contains("Did not observe any item or terminal signal within ")) {
                LogUtil.error("Timeout for UPLOADING image "+image.getImageId()+", Size(MB): " +imageSizeInMB+" for Azure ", e);
            } else {
                LogUtil.error("Exception happened during uploading image "+image.getImageId()+", Size(MB): " +imageSizeInMB+" for Azure ", e);
            }
        }

        // return a failed future
        CompletableFuture<String> failedFuture = new CompletableFuture<>();
        failedFuture.completeExceptionally(new MigrationException("Failed for uploading image "+image.getImageId()+", Size(MB): " +imageSizeInMB+" for Azure "));
        return failedFuture;
    }

@alzimmermsft could you please have a look for the above message when you feel available, thank you so much.

Could you please share some official documents to describe the timeout difference between among BlobClient and RequestRetryOptions?

This is something that wasn't well documented and I'm working on adding this in a few places. So far, I have a PR opened adding this documentation to azure-core's README: https://github.com/Azure/azure-sdk-for-java/pull/36710/files#diff-b8dc45bc6fad5f70e59b49cda551e56ae6668c49fa0fd026c09b74537b9abed1R161
@ibrahimrabab could you look at porting this documentation to the Storage READMEs as well?
@JonathanGiles where is the best place to put documentation like this in https://learn.microsoft.com/en-us/azure/developer/java/sdk/?

I may miss something because I found it hard to catch the azure upload timeout exception after running out of re-try count, eg: what is the particular exception type, so I have to use a crude way to match the exception string which you can see it among my code

Thanks for making note of this, looking at this it's a bit tricky and something the SDKs need to clean up. Depending on the timeout that triggered different exceptions will be thrown.
If the timeout happened at the apiCall(Duration timeout) level it will throw an IllegalStateException. If the timeout happened at the HttpPipeline or HttpClient layer it will throw a TimeoutException.
For the purposes of your application as you removed the usage of the apiCall(Duration timeout) I'd check for TimeoutException.
@JonathanGiles @srnagar @lmolkova we should do a review on the exceptions thrown in timeout scenarios to make sure they are standardized.

Seems I have to add the HttpLogOptions to let the azure related logs be represented, eg: the following logs
seems I have to fetch out these logs with "az.sdk.message", but I still will miss some part, eg: the second log, in this instance, is there any way I can use to fetch all these logs in one time?

I don't fully understand the question about fetching all the logs at one time. If you're asking how to fetch all Azure SDK logs I would recommend doing that based on the log.logger included in the log and look for com.azure.* loggers.

This is something that wasn't well documented and I'm working on adding this in a few places. So far, I have a PR opened adding this documentation to azure-core's README: https://github.com/Azure/azure-sdk-for-java/pull/36710/files#diff-b8dc45bc6fad5f70e59b49cda551e56ae6668c49fa0fd026c09b74537b9abed1R161
@ibrahimrabab could you look at porting this documentation to the Storage READMEs as well? @JonathanGiles where is the best place to put documentation like this in https://learn.microsoft.com/en-us/azure/developer/java/sdk/?

I'll ping you this week to discuss Alan - but generally speaking this seems like good content for the troubleshooting push I'm doing over at learn.microsoft.com

@JonathanGiles @srnagar @lmolkova we should do a review on the exceptions thrown in timeout scenarios to make sure they are standardized.

@alzimmermsft Please file an issue and start the ball rolling on this! Loop us in ASAP.

@JonathanGiles @alzimmermsft
Thank you so much for the detailed information and your valuable time.

Closed, question was answered, and documentation was updated to be clearer.

authors: JonathanGiles alzimmermsft ibrandes kensinzl


text:
[BUG] Spring Cloud Azure 5.9.0 update requires JDK 21

Describe the bug
Updating to Spring Cloud Azure 5.9.0 caused build failure complaining about the class version. I wasn't able to find any information about the JDK 21 requirement in the release so I assume this is not intentional. Please kindly let me know in case I missed something! Thank you!
Exception or Stack Trace
    Caused by: java.lang.UnsupportedClassVersionError: com/azure/spring/cloud/autoconfigure/implementation/context/AzureGlobalConfigurationEnvironmentPostProcessor has been compiled by a more recent version of the Java Runtime (class file version 65.0), this version of the Java Runtime only recognizes class file versions up to 61.0
    	at java.base/java.lang.ClassLoader.defineClass1(Native Method)
    	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
    	at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150)

To Reproduce
Steps to reproduce the behavior:

Check out https://github.com/nagyesta/lowkey-vault-example/commit/f14417508a6cb433636455f9b115c21fa89e59d7
Build using JDK 17
Observe errors

Code Snippet
N/A
Expected behavior
The project is working with JDK 17
Screenshots
N/A
Setup (please complete the following information):

OS: any
IDE: any
Library/Libraries:

"com.azure.spring:spring-cloud-azure-starter-keyvault-secrets:5.9.0"
"com.azure.spring:spring-cloud-azure-starter-keyvault:5.9.0"

Java version: 17 (below 21)
App Server/Environment: any
Frameworks: Spring Boot

Additional context
Affects this PR: https://github.com/nagyesta/lowkey-vault-example/pull/274
Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

[x] Bug Description Added
[x] Repro Steps Added
[x] Setup information Added

Meanwhile, Microsoft Azure Functions do not support Java 21 :)

@saragluna @vcolin7 could you please investigate this apparent regression (requiring JDK 21 to run)?

Yes, I'm looking into this.

Sorry for the inconvenience, we will release a hotfix ASAP.

Spring Cloud Azure 5.9.1 now is released

Looks like 5.9.1 works well. Thank you

authors: Netyyyy OleksandrShkurat joshfree nagyesta saragluna vcolin7


text:
Add support for groupby queries

Java sdks currently do not support group by queries. Support for the same should be added.

Added support for groupby in v4.1.0  https://github.com/Azure/azure-sdk-for-java/blob/master/sdk/cosmos/azure-cosmos/CHANGELOG.md#410-2020-06-25


text:
vm createOrUpdate not working

When I try to e.g. add a data disk to a vm I get an error that the createoption for the os disk is required and cannot be null. When I try to set the createoption of the os disk to DiskCreateOptionTypes.EMPTY I get "error":{"code":"PropertyChangeNotAllowed","target":"osDisk.createOption","message":"Changing property 'osDisk.createOption' is not allowed."}
The issue seems to be that the DataDisk.createOption and OSDisk.createOption do not get filled on retrieval of a virtual machine. I changed the field to a string and adapted the setter and getter method which seems to work

https://github.com/Azure/azure-rest-api-specs/issues/275

Not sure if that issue is related to mine. In my case, the createOption is not getting filled when I get() a virtual machine. When I try to update the VM, I get " is required and cannot be null " because the field was not set on get()

This is fixed in beta2. Please verify.

authors: MSSedusch jianghaolu selvasingh


text:
Azure spring data cosmos version schema

The former design of Spring version schema will make two separate artifacts to cope with Spring Data 2.2.x and 2.3.x, but if we let Spring BOM manage our versions we could use one artifact to support both versions.
The dependency management section in a pom file could also affect the transitive versions: https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#bill-of-materials-bom-poms.

/azp run java - cosmos - tests

/azp run java - cosmos - tests


text:
Sync eng/common directory with azure-sdk-tools for PR 1327

Sync eng/common directory with azure-sdk-tools for PR https://github.com/Azure/azure-sdk-tools/pull/1327  See eng/common workflow

/check-enforcer evaluate

/check-enforcer evaluate

authors: azure-sdk benbp


text:
Fixing checkstyle breaks from upgrade to 8.40

Fixing indentation in IdentityClient

/azp run java - anomalydetector - ci


text:
Sync stack

Incorporating feedback:

rename synchronous -> sync
rename getContent -> getBodyAsBinaryData
rename setContent -> setBody(BinaryData)

API change check for com.azure:azure-core
API changes have been detected in com.azure:azure-core. You can review API changes here

API change check for com.azure:azure-core-http-jdk-httpclient
API changes have been detected in com.azure:azure-core-http-jdk-httpclient. You can review API changes here
API changes
-         public JdkHttpClientProvider()
+         public JdkHttpClientProvider()

API change check for com.azure:azure-core-http-netty
API changes have been detected in com.azure:azure-core-http-netty. You can review API changes here
API changes
-         public NettyAsyncHttpClientProvider()
+         public NettyAsyncHttpClientProvider()

API change check for com.azure:azure-core-http-okhttp
API changes have been detected in com.azure:azure-core-http-okhttp. You can review API changes here
API changes
-         public OkHttpAsyncClientProvider()
+         public OkHttpAsyncClientProvider()
-         public OkHttpAsyncHttpClientBuilder followRedirects(boolean followRedirects)

API change check for com.azure:azure-core-test
API changes have been detected in com.azure:azure-core-test. You can review API changes here

API change check for com.azure:azure-core-tracing-opentelemetry
API changes have been detected in com.azure:azure-core-tracing-opentelemetry. You can review API changes here
API changes
+         @Override public HttpResponse processSync(HttpPipelineCallContext context, HttpPipelineNextPolicy next)

API change check for com.azure:azure-storage-common
API changes have been detected in com.azure:azure-storage-common. You can review API changes here
API changes
+         @Override public HttpResponse processSync(HttpPipelineCallContext context, HttpPipelineNextPolicy next)
+         @Override public HttpResponse processSync(HttpPipelineCallContext context, HttpPipelineNextPolicy next)
+             @Override public HttpResponse processSync(HttpPipelineCallContext context, HttpPipelineNextPolicy next)
+         @Override public HttpResponse processSync(HttpPipelineCallContext context, HttpPipelineNextPolicy next)
+         @Override public HttpResponse processSync(HttpPipelineCallContext context, HttpPipelineNextPolicy next)

API change check for com.azure:azure-storage-blob
API changes are not detected in this pull request for com.azure:azure-storage-blob

API change check for com.azure:azure-storage-blob-batch
API changes are not detected in this pull request for com.azure:azure-storage-blob-batch

API change check for com.azure:azure-storage-blob-nio
API changes are not detected in this pull request for com.azure:azure-storage-blob-nio

authors: azure-sdk kasobol-msft


text:
Bug Fix: Passing clientOptions to HttpPipelineBuilder in buildPipeline

resolves #28783

API change check
API changes are not detected in this pull request.

authors: azure-sdk ibrahimrabab


text:
[AutoPR mariadb/resource-manager] Update MariaDB api version

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/5280

This PR has been merged into https://github.com/Azure/azure-sdk-for-java/pull/2387


text:
[Automation] Generate SDK based on TypeSpec 0.15.8

[Automation] Generate SDK based on TypeSpec 0.15.8

/check-enforcer override

authors: azure-sdk weidongxu-microsoft


text:
[Automation] Generate Fluent Lite from Swagger security#package-composite-v3

[Automation] Generate Fluent Lite from Swagger security#package-composite-v3

API change check
API changes are not detected in this pull request.


text:
Confused about @azure/ms-node-auth vs @azure/identity vs MSAL.js

Package Name: @azure/identity
Package Version: All
Is the bug related to documentation in

[x] README.md
[x] source code documentation
[x] SDK API docs on https://docs.microsoft.com

Describe the bug
It is unclear what the difference is between @azure/ms-node-auth, @azure/identity, and MSAL.js. I do not know which lib to use when. If there is a doc that explains it, I cannot find one.
Expected behavior
A doc explaining each library and when they should be used with examples.

Thanks for reporting @southpolesteve
@jonathandturner, @daviwil, @sadasant,
I was thinking that at the very least, we should do a couple of things here

Update the readme for @azure/identity with notes on

how it integrates with MSAL
the fact that it uses v2 of AAD apis
that not all credential classes can be used in the browser
any scenario where one should use MSAL.js directly instead of @azure/identity

Resolve #11359
See what improvements we can do to Authenticate with the Azure management modules for JavaScript

@southpolesteve Can you list the top app types or identity issues, from your perspective, a customer is trying to figure out when asking this question?

@diberry I think you covered it well.
Maybe your first case captures this, but I think it can be broken in two:

I am writing an app that needs many social logins (FB, GitHub, etc) only for identity purposes
I am writing an app that specifically needs AAD login to access Azure. Like an internal app that lets my employees configure things that might be Azure resources under the hood.

I also think there is a "Just make it work any way you can" case for developers. I won't put an web app in production with vscode based login, but if it helps me get the app working or improves development, I want that to be possible.

@southpolesteve Hello Steve! I'll be going through your feedback and making an update to our documentation as soon as possible. I'll get in contact with you in case I'm missing something. Thank you for submitting this issue!

@sadasant @ramya-rao-a Do you want to be included on Dev Center changes or localize this issue to your own SDK content?

@diberry I'd like to be included in the Dev Center changes! If anything, for exposure.

Hi again! Just to mention that I'll provide more information next week.

@southpolesteve :
I believe that the questions @diberry is working on will be helpful for you! Let me answer some of the other things you mention here.

It is unclear what the difference is between @azure/ms-node-auth, @azure/identity, and MSAL.js

@azure/ms-node-auth is the older authentication library! We continue to maintain it, but we're not adding new features. We're planning some better integration with it to facilitate users to move towards our newer library.
@azure/identity is our newer library! This is the one that should be used if you want to authenticate any of our clients with the Azure services.
MSAL.js is a library that connects to the system-specific keychains to provide a same authentication experience across environments. @azure/identity uses MSAL.js under the hood!

When should each library be used?

Even though all of them can be used, @azure/identity is the library that should be used with the Azure SDK for JS clients.
@ramya-rao-a :

Update the readme for @azure/identity

I've made an issue! https://github.com/Azure/azure-sdk-for-js/issues/12669 . I'll follow up this week.

See what improvements we can do to Authenticate with the Azure management modules for JavaScript.

I'll take a look and I'll make some notes!

While I move ahead with the readme update etc, how else can I be useful here? Please let me know if I'm missing something!

On MSAL:
MSAL does offer several features that are not yet available in our SDK, but we will be adding support as soon as possible. These include more control on the caching and the storing of the credentials. However, we're working as closely as possible with the MSAL team, so we should be able to level up with them in a couple of months, as far as I'm understanding.
It's in our interest to request people to use the @azure/identity library as much as possible, instead of the possible alternatives, since direct customer feedback will help us make this experience better for everyone.

@sadasant thanks for the explainer, I've been wondering the differences for awhile! What is the recommendation for providing browser-based authentication (e.g. for webapps needing a credential)? @azure/identity has a browser method that works well, but the package is only supported by a small number of Azure service libraries. I'm using @azure/arm-resources and @azure/arm-compute which require the older msRest.ServiceClientCredentials type of credential object.

@sadasant thanks for the explainer, I've been wondering the differences for awhile! What is the recommendation for providing browser-based authentication (e.g. for webapps needing a credential)? @azure/identity has a browser method that works well, but the package is only supported by a small number of Azure service libraries. I'm using @azure/arm-resources and @azure/arm-compute which require the older msRest.ServiceClientCredentials type of credential object.

@seanknox You can use the @azure/ms-rest-browserauth package when working with @azure/arm-resources and @azure/arm-compute packages for authentication needs in the browser. The readmes on these packages should have a code snippet that shows this:

Code snippet in readme for compute
Code snippet in readme for resources

@seanknox You can use the @azure/ms-rest-browserauth package when working with @azure/arm-resources and @azure/arm-compute packages for authentication needs in the browser. The readmes on these packages should have a code snippet that shows this:

Code snippet in readme for compute
Code snippet in readme for resources

@ramya-rao-a @azure/ms-rest-browserauth requires creating an AD app to authenticate users. Is that the only option for browser authentication, or is there another way can users authenticate directly to Microsoft auth, like @azure/identity's InteractiveBrowserCredential method?

@ramya-rao-a @azure/ms-rest-browserauth requires creating an AD app to authenticate users. Is that the only option for browser authentication, or is there another way can users authenticate directly to Microsoft auth, like @azure/identity's InteractiveBrowserCredential method?

@seanknox Hello hello! I wonder if a credential like @azure/identity's DeviceCodeCredential can work for you. Would that be useful? In ms-rest-nodeauth we have interactiveLoginWithAuthResponse, which is similar.

@seanknox Hello hello! I wonder if a credential like @azure/identity's DeviceCodeCredential can work for you. Would that be useful? In ms-rest-nodeauth we have interactiveLoginWithAuthResponse, which is similar.

I understand that this wouldn't be on the browser though. Would it be possible to move to @azure/identity instead?

I understand that this wouldn't be on the browser though. Would it be possible to move to @azure/identity instead?

@azure/ms-rest-browserauth requires creating an AD app to authenticate users. Is that the only option for browser authentication, or is there another way can users authenticate directly to Microsoft auth, like @azure/identity's InteractiveBrowserCredential method?

@seanknox All credential classes in @azure/identity make use of the client id and therefore require you to create an app registration. The ones that don't default to the one corresponding to Azure CLI. So yes, the recommended way is to create an app registration and pass the clientId when creating the credential.
@sadasant The packages @azure/arm-resources and @azure-arm-compute do not support @azure/identity. So, @seanknox won't be able to use it.

@azure/ms-rest-browserauth requires creating an AD app to authenticate users. Is that the only option for browser authentication, or is there another way can users authenticate directly to Microsoft auth, like @azure/identity's InteractiveBrowserCredential method?

@seanknox All credential classes in @azure/identity make use of the client id and therefore require you to create an app registration. The ones that don't default to the one corresponding to Azure CLI. So yes, the recommended way is to create an app registration and pass the clientId when creating the credential.
@sadasant The packages @azure/arm-resources and @azure-arm-compute do not support @azure/identity. So, @seanknox won't be able to use it.

Why don't they support @azure/identity? I'm interested in making it work.
If it makes sense, is it because of Continuous Access Evaluation (CAE) challenge based authentication? I believe this is important for ARM resources. We're adding support for CAE this month.

Why don't they support @azure/identity? I'm interested in making it work.
If it makes sense, is it because of Continuous Access Evaluation (CAE) challenge based authentication? I believe this is important for ARM resources. We're adding support for CAE this month.

This has nothing to do with CAE
All the management plane packages (the ones dealing with resource management) at the moment are auto generated. The generated code works with the credentials from @azure/ms-rest-nodeauth and @azure/ms-rest-browserauth. They are of a different shape than the TokenCredential interface which is implemented by all the credentials in the @azure/identity package. We do have a feature request to update the code generator to generate code that will work with the credentials from @azure/identity as well. But it will take a while to update the code generator and re-generate over 100 management plane packages.

This has nothing to do with CAE
All the management plane packages (the ones dealing with resource management) at the moment are auto generated. The generated code works with the credentials from @azure/ms-rest-nodeauth and @azure/ms-rest-browserauth. They are of a different shape than the TokenCredential interface which is implemented by all the credentials in the @azure/identity package. We do have a feature request to update the code generator to generate code that will work with the credentials from @azure/identity as well. But it will take a while to update the code generator and re-generate over 100 management plane packages.

@seanknox Please log an issue in the repo for @azure/ms-rest-browserauth for more on that package
We have https://github.com/Azure/azure-sdk-for-js/issues/12669 tracking improvements to the @azure/identity package which we will tackle this month.
We are independently tracking other efforts to improve documentation around auth. So, closing this issue.
Thanks for your patience everyone

@seanknox Please log an issue in the repo for @azure/ms-rest-browserauth for more on that package
We have https://github.com/Azure/azure-sdk-for-js/issues/12669 tracking improvements to the @azure/identity package which we will tackle this month.
We are independently tracking other efforts to improve documentation around auth. So, closing this issue.
Thanks for your patience everyone

then how about library @azure/msal-browser, this lib is using PublicClientApplication to achieve browser login and use graph or other web API. what distinguish it from @azure/identity?

then how about library @azure/msal-browser, this lib is using PublicClientApplication to achieve browser login and use graph or other web API. what distinguish it from @azure/identity?

@leolumicrosoft,
@azure/identity contains multiple credential classes, all following the TokenCredential interface. You would need to use these credentials when using our newer set of libraries.
When in browser, the only credential that applies from @azure/identity is the InteractiveBrowserCredential which at the moment uses the msal package. We are in the process of moving to use @azure/msal-browser instead. See #13155 and #13263
You are free to use @azure/msal-browser directly as long as you create your own credential class that follows the interface expected by the client constructor in the Azure package that you are using

The client constructors in the new JS packages require a credential that follows the TokenCredential interface
The client constructors in the rest of the JS packages in this repo require a credential that follows the ServiceClientCredential interface. An example can be found at Authenticating with an existing token

@leolumicrosoft,
@azure/identity contains multiple credential classes, all following the TokenCredential interface. You would need to use these credentials when using our newer set of libraries.
When in browser, the only credential that applies from @azure/identity is the InteractiveBrowserCredential which at the moment uses the msal package. We are in the process of moving to use @azure/msal-browser instead. See #13155 and #13263
You are free to use @azure/msal-browser directly as long as you create your own credential class that follows the interface expected by the client constructor in the Azure package that you are using

The client constructors in the new JS packages require a credential that follows the TokenCredential interface
The client constructors in the rest of the JS packages in this repo require a credential that follows the ServiceClientCredential interface. An example can be found at Authenticating with an existing token

Thank you, @ramya-rao-a. Thank you for the detail reply, I started to understand more of AD related javascript SDK.
I highlight a few points I learned through experiment as it might be helpful to people who just started explore AD authenticate topic. Please correct anything that is inaccurate.

DefaultAzureCredential in @azure/identity is to be used in backend service application code, or it is used in a local running application to get credential. But this library is not used in frontend code such as JavaScript in html.

Browser javascript code can utilize @azure/msal-browser or earlier @azure/ms-node-auth. @azure/msal-browser uses auth code flow which allows stricter control of protected resource access, while @azure/ms-node-auth uses implicit flow

Azure services exposes RESTful endpoints which need token. There are two types of token, one is for the resource management or control plane REST api. This token can be get through credential.getToken("https://management.azure.com/.default"). such as listing all data storage account in your azure account.

The other type of RESTful endpoint need token from each specified service, for example
azure keyvault: credential.getToken("https://vault.azure.net/.default")
azure digitaltwin: credential.getToken("https://digitaltwins.azure.net/.default")

Both 3 and 4 can be tested using postman after you get the token by using DefaultAzureCredential in a simple locally run script with "az login".
These points maybe very basic but it still give me three days to have a clearer insight into them.

authors: diberry leolumicrosoft ramya-rao-a sadasant seanknox southpolesteve

repo: Azure/azure-sdk-for-js


text:
Remove core-tracing dependency in core-auth

The @azure/core-auth package was meant to be a light weight one to hold the types and interfaces to be used by anyone trying to implement the credentials used in our latest packages. In #11359, we are discussing using it in our older code generator so that the older packages can make use of @azure/identity as well. Since this package has a dependency on @azure/core-tracing only for types, we now end up pulling un-necessary tracing dependency as well.
This issue is to consider removing the dependency on core-tracing from core-auth and instead duplicate the two types we pull in i.e. SpanOptions and SpanContext
cc @xirzec, @joheredi

I'm good with duplicating.

authors: ramya-rao-a xirzec


text:
[Identity] Should we add support for tenantId on the ManagedIdentityCredential?

On getToken, the ManagedIdentityCredential can receive a tenant Id since the GetTokenOptions type supports it. However, the ManagedIdentityCredential does not send the tenant Id on the outgoing requests. Should we support this?

@sadasant I believe you cannot support ternant id on managed identity, since this is calling a localhost endpoint this is not how the flow is designed to work.
The update we did in the Python doc, is to say that when you implement the "get_token" protocol, you may ignore silently tenant_id if you can't do anything with it, as this parameter should be seen as a hint of how to get a valid token (it's designed with challenge of KV in mind), not as a requirement that it has to be this tenant_id. This means if the hint doesn't apply to this implementation it's safe to ignore it.

@lmazuel oh ok! Gotcha.
So, my approach to solve this issue will be just to add tests. Thank you!

I've been beating my head against the wall on this. From what I'm reading if I want to use ManagedIdentityCredential I can't specify the tenant ID, which I thought I could do. However, when I do, I always pull the managed identity from the default tenant and not the tenant that I specify. Which makes sense now.
My understanding is that when I have a multi-tenant app (Azure Function) that uses a system managed identity, that a Managed Identity will be created in other tenants when an admin consents to the access in those tenants. How then do I consume the managed identity in the other tenant, so that my multi-tenant function can access the Graph API of the other tenant? Everything I try just returns an access token for the tenant that is hosting the app.
My code looks like this:
              var credential = new DefaultAzureCredential();
               var token = credential.GetToken(
                   new Azure.Core.TokenRequestContext(
                       new[] { "https://graph.microsoft.com/.default" }, null, null, "<Tenant B ID>"));
               var accessToken = token.Token;```

authors: appleoddity lmazuel sadasant


text:
Missing H1 for Quantum Area

The following page on Docs are using H2 header elements in place of an H1 title header. Docs throws a warning when there is no H1 element at the top of the page.
https://github.com/MicrosoftDocs/azure-docs-sdk-node/blob/main/docs-ref-services/preview/quantum-jobs-readme.md
This appears to have been imported from the following readme file: sdk/quantum/quantum-jobs/README.md
The readme file should be modified to use H1 title headers and re-imported to Docs, or the import process needs to be modified to change the headers.

Label prediction was below confidence level 0.6 for Model:ServiceLabels: 'Storage:0.23352472,Azure.Core:0.13711227,Docs:0.04791413'

Tracking in #22206

authors: azure-sdk v-alje xirzec


text:
OpenAI: GetChatCompletionsOptions.n - can n have a more obvious name

The property n - The number of chat completions choices that should be generated for a chat completions - this needs a better name such as totalAllowedChatCompletions.
Please update source.

As unfortunate as it is, this name is used in the API: https://platform.openai.com/docs/api-reference/chat/create#chat/create-n and the SDK is using the same names.
/cc @bterlson @johanste

@deyaaeldeen How did that get past API review board?

This API is owned by OpenAI.

Ok.

authors: deyaaeldeen diberry johanste


text:
[Service Bus] "message" word added to "createBatch", "CreateBatchOptions" and "tryAdd"

PR for #11878

/azp run js - servicebus - tests

/azp run js - servicebus - tests


text:
[core] - added changelog entries for recent changes

I forgot the changelogs. I always forget the changelogs 🤷

/check-enforcer override


text:
Post release automated changes for eventgrid releases

Post release automated changes for azure-arm-eventgrid

@qiaoza Please take a look at the merge conflicts in this PR

close this one as eventgrid has already been GAed

authors: azure-sdk qiaozha ramya-rao-a


text:
imagebuilder release

https://github.com/Azure/sdk-release-request/issues/3930

API change check
APIView has identified API level changes in this PR and created following API reviews.
azure-arm-imagebuilder


text:
Move identity variable setting into test-resources-pre.ps1

Updating the identity live tests to remove logic and env setting from yaml in favor of a dedicated keyvault config and powershell script. This will make it possible to improve local and sovereign cloud testing, and make cross-language config updates more easily.
Related: https://github.com/Azure/azure-sdk-for-net/pull/38473

API change check
API changes are not detected in this pull request.

Working through some testing issues

Live tests: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3080889&view=results


text:
[AutoPR hdinsight/resource-manager] [HDInsight] - Support KV URL

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/4449

This PR has been merged into https://github.com/Azure/azure-sdk-for-js/pull/515


text:
[Add methods:] Add method to retrieve results of completed train/analyze LROs

Per .NET guidelines: https://azure.github.io/azure-sdk/dotnet_introduction.html#dotnet-longrunning
Outstanding design issues here: https://github.com/Azure/azure-sdk-for-python/pull/9963#discussion_r388289970

Constructors have been added to Operation classes to resume LRO.

repo: Azure/azure-sdk-for-net


text:
Azure QnA maker fails to create a knowledge base by raising ExtractionFailure error code

Describe the bug
Azure QnA Maker SDK fails to create knowledgebase from a URL powered by an Azure Http-Triggered function while the function is publicly available and it only accepts GET requests. The response of the function is a pure string return by "return new OkObjectResult(responseString)"
Expected behavior
Knowledgebase should be created from the given URL.
Actual behavior (include Exception or Stack Trace)
The following error message is produced:

Unsupported / Invalid url(s). Failed to extract Q&A from the source

To Reproduce
Run the following code snippet in a console application:
var createKbDto = new CreateKbDTO
        {
            Name = request.Name,
            QnaList = new List<QnADTO>(),
            Urls = new List<string>
            {        "https://caccea77.ngrok.io/api/jobdescription/facade/2034/43D672205B3106BE3273C60FE423C632"
            }
        };

        var createKb = await client.Knowledgebase.CreateAsync(createKbDto);
        var createdOp = await MonitorOperationAsync(client, createKb);
        return GetKbId(createdOp);

Environment:

Microsoft.Azure.CognitiveServices.Knowledge.QnAMaker" Version="1.1.0"
IDE and version : Visual Studio 16.5.4
Environment:
.NET Core SDK (reflecting any global.json):
Version:   3.1.201
Commit:    b1768b4ae7

Runtime Environment:
OS Name:     Windows
OS Version:  10.0.18363
OS Platform: Windows
RID:         win10-x64
Base Path:   C:\Program Files\dotnet\sdk\3.1.201\
Host (useful for support):
Version: 3.1.3
Commit:  4a9f85e9f8
.NET Core SDKs installed:
2.1.802 [C:\Program Files\dotnet\sdk]
2.2.207 [C:\Program Files\dotnet\sdk]
2.2.402 [C:\Program Files\dotnet\sdk]
3.0.100-rc1-014190 [C:\Program Files\dotnet\sdk]
3.1.100-preview3-014645 [C:\Program Files\dotnet\sdk]
3.1.201 [C:\Program Files\dotnet\sdk]
.NET Core runtimes installed:
Microsoft.AspNetCore.All 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.1.17 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.1.17 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.0.0-rc1.19457.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.0-preview3.19555.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.17 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.0.0-rc1-19456-20 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.0-preview3.19553.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 3.0.0-rc1-19456-20 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 3.1.0-preview3.19553.2 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

@dfulcer
Hi Dean,
To be able to reproduce the problem please feed the following URL to QnAMaker's SDK:
https://caccea77.ngrok.io/api/jobdescription/facade/2034/43D672205B3106BE3273C60FE423C632
Feeding this URL to QnAMaker portal results in the same error message too:

Unsupported / Invalid url(s). Failed to extract Q&A from the source

Does the web content produced by the URL above lack anything that QnAMaker expects it? What causes QnAMaker to reject the content?
/cc @milad-simcoeai @miladghafoori

@dfulcer @jsquire
Any updates on this ticket? The application insights is quite silent in terms of logging what goes wrong.
/cc @milad-simcoeai

I am just the source of initial triage in this case.  Unfortunately, that means that I don't have any insight to offer.

@jsquire just wondering, who's the lead on QnA Maker to have them engaged a quickly as possible? It's a showstopper and I'm sure there are or will be other folks experiencing the same problem.

With regret, I do not know.  This is not a library which the Azure SDK team owns at this point.  The QnA Maker service team will need to assist, but beyond that I don't have insight.  Each Azure service team has their own triage process once an issue has been identified and tagged.  In this case, it would appear that they've identified @dfulcer as the point of contact.
If this is a show stopping issue, I'd recommend opening an Azure support ticket.  That will be a more formal and expedient route for support with a proper escalation path.  That would ensure that someone is actively working to engage the proper folks for attention.
My apologies that I don't have a better answer for you.

This thread has came to the team now :(
This is related to the QnAMaker extraction logic, not specific to SDK. As per the error, the provided URL content didn't support our extraction standard to generate any QnAs. However, I see that the provided URL doesn’t exist anymore. Please close the thread if it's too late, or share a valid URL so that we can investigate. Thanks!

authors: Arash-Sabet jsquire rokulka


text:
[BUG] BlobContainerClient.CreateIfNotExistsAsync returns a null Response object when container already exists.

Describe the bug
The bug happened here:
https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/storage/Azure.Storage.Blobs/src/BlobContainerClient.cs#L1080
BlobContainerClient.CreateIfNotExistsAsync() returns default or null if said container already exists.
Expected behavior
It should return a Response object with the proper error code
Actual behavior (include Exception or Stack Trace)
It returns null
Environment:

Azure.Storage.Blobs 12.8.1
Hosting platform or OS and .NET runtime version:

.NET SDK (reflecting any global.json):
 Version:   5.0.201
 Commit:    a09bd5c86c

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19041
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\5.0.201\

Host (useful for support):
  Version: 5.0.4
  Commit:  f27d337295

.NET SDKs installed:
  1.0.4 [C:\Program Files\dotnet\sdk]
  2.1.500 [C:\Program Files\dotnet\sdk]
  2.1.812 [C:\Program Files\dotnet\sdk]
  2.2.207 [C:\Program Files\dotnet\sdk]
  3.1.202 [C:\Program Files\dotnet\sdk]
  5.0.201 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.All 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.24 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.26 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.24 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.26 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 1.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 1.1.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.24 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.26 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.4 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.13 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.4 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

IDE and version : Microsoft Visual Studio Community 2019 Version 16.9.2

Hi,
CreateIfNotExists returning null or default is an expected result if the container already exists. A response of Response<BlobContainerInfo> can not be returned because that would mean that a successful container exists. Users are expecting that if they receive a null or default response from this API that the container already exists (and that we don't throw an exception). If the container does not exist and was created a Response<BlobContainerInfo> will be returned. (If we were to stop returning default or null in the case of the container already existing, this would be a breaking change).
If you're looking for this method to throw an exception upon seeing a BlobErrorCode of ContainerAlreadyExists, please use the regular Create method.

authors: Arkatufus amnguye


text:
Explore multiple calls to GZip WriteTo and TryComputeLength methods

To be completed before GA on Sept 29

Can we WriteTo and then continue to add Json?
Add test for write to the stream, get the length, and then write some more
Are gzip streams appendable?

https://gist.github.com/KrzysztofCwalina/f94e76a50c78968fe9c7b3df99a73eed


text:
[BUG] ObjectDisposedException when mocking HttpResponseMessage content via a System.Net.Http.DelegatingHandler on the underlying HTTP client.

Library name and version
Azure.ResourceManager.Resources 1.6.0, Azure.ResourceManager 1.7.0, Azure.Core 1.34.0
Describe the bug
I'm encountering an ObjectDisposedException in our unit test environment that uses a DelegatingHandler to mock out response content. See the repro steps for relevant code snippets.
After debugging, I discovered the following:

In Azure.Core.Pipeline.HttpClientTransport::ProcessAsync(HttpMessage message, bool async), the StringContent gets read as a MemoryStream and is passed to the SDK's response abstraction PipelineResponse which is an IDisposable that when disposed, will dispose the underlying HttpResponseMessage and its content.
In Azure.Core.Pipeline.ResponseBodyPolicy, if the content stream is a non-seekable stream and message.BufferResponse is true, a setter message.Response.ContentStream = bufferedStream is called. This invokes the overridden setter in Azure.Core.Pipeline.PipelineResponse which nulls the Content on HttpResponseMessage

This step does not occur in the unit test setup because responseContentStream.CanSeek is true for a MemoryStream

In Azure.ResourceManager.Resources.ArmDeploymentResource::UpdateAsync(...) the HttpMessage is disposed after CreateOrUpdateAtScopeAsync is done, which will dispose the StringContent on the response since it was not nulled by the previous point.
The SDK then wraps the response with a ResourcesArmOperation which leads to the disposed exception.

Expected behavior
Be able to mock a long running operation's intermediate response at the HTTP layer via DelegatingHandler.
Actual behavior
System.ObjectDisposedException: Cannot access a closed Stream.
   at System.IO.__Error.StreamIsClosed()
   at System.IO.MemoryStream.get_Length()
   at Azure.Core.NextLinkOperationImplementation.IsFinalState(Response response, HeaderSource headerSource, Nullable`1& failureState, String& resourceLocation)
   at Azure.Core.NextLinkOperationImplementation.Create(HttpPipeline pipeline, RequestMethod requestMethod, Uri startRequestUri, Response response, OperationFinalStateVia finalStateVia, Boolean skipApiVersionOverride, String apiVersionOverrideValue)
   at Azure.Core.NextLinkOperationImplementation.Create[T](IOperationSource`1 operationSource, HttpPipeline pipeline, RequestMethod requestMethod, Uri startRequestUri, Response response, OperationFinalStateVia finalStateVia, Boolean skipApiVersionOverride, String apiVersionOverrideValue)
   at Azure.ResourceManager.Resources.ResourcesArmOperation`1..ctor(IOperationSource`1 source, ClientDiagnostics clientDiagnostics, HttpPipeline pipeline, Request request, Response response, OperationFinalStateVia finalStateVia, Boolean skipApiVersionOverride, String apiVersionOverrideValue)
   at Azure.ResourceManager.Resources.ArmDeploymentResource.<UpdateAsync>d__20.MoveNext()

Reproduction Steps
The mock delegating handler:
public class MockRequestHandler : DelegatingHandler
{
    protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    {
        // ...

        var response = new HttpResponseMessage();
        response.Content = new StringContent(/* mock json */);

        // it does NOT call base.SendAsync()

        // ...

        return Task.FromResult(response);
    }

The http client:
var webRequestHandler = new WebRequestHandler { AllowAutoRedirect = false };

var httpClient = new HttpClient(
    handler: HttpClientFactory.CreatePipeline(innerHandler: webRequestHandler, handlers: delegatingHandlers), // mock handler goes here
    disposeHandler: true);

The ArmClient is initialized as follows:
var armClientOptions = new ArmClientOptions
{
  // ...
  Transport = new HttpClientTransport(httpClient) // http client with the delegating handler
};

var armClient = new ArmClient(tokenCredential, default, armClientOptions);

The ArmClient call:
var armDeploymentSdkResource = armClient.GetArmDeploymentResource(/* ResourceIdentifier */);
var deploymentOperation = await armDeploymentSdkResource
    .UpdateAsync(WaitUntil.Started, deploymentRequestInput, this.CancellationToken);

Environment
Windows 11
System.Runtime.InteropServices.RuntimeInformation.FrameworkDescription = ".NET Framework 4.8.9167.0"
JetBrains Rider 2023.2.1

Thank you for your feedback.  Tagging and routing to the team member best able to assist.

//cc: @m-nash, @annelo-msft
I think this may turn out to be the same root cause as #38219, with what we currently suspect is the root cause discussed here.

Thanks, @jsquire!  I was planning to spend some time looking at https://github.com/Azure/azure-sdk-for-net/issues/38219 today, so I'll look at this one as well while I'm doing that.

@kalbert312, thanks for a really nice investigation and repro case!  I have confirmed that this is the same issue as we're looking at in https://github.com/Azure/azure-sdk-for-net/issues/38219.  I'm going to close it as a duplicate, but I'm also tagging the other one as Azure.Core, and will try to turn around a fix soon.  Thanks for reporting this!

Reopening this as no-longer a duplicate of the first one.

authors: annelo-msft jsquire kalbert312