text:
datalake.Store data-plane definition casing inconsistency

I was recently using the Node.js datalake.Store SDK and found a casing-inconsistency here:
https://github.com/Azure/azure-rest-api-specs/blob/master/specification/datalake-store/data-plane/Microsoft.DataLakeStore/stable/2016-11-01/filesystem.json#L1671
"FileStatusResult": {
  "properties": {
    "fileStatus": {
      "$ref": "#/definitions/FileStatusProperties",
      "readOnly": true,
      "description": "the file status object associated with the specified path."
    }
  },
  "description": "Data Lake Store filesystem file status information response."
}

The response that I get back from the service actually looks like this:
{
  "FileStatus": {
    "length": 0,
    "pathSuffix": "",
    "type": "DIRECTORY",
    "blockSize": 0,
    "accessTime": 1490302583819,
    "modificationTime": 1490302583819,
    "replication": 0,
    "permission": "770",
    "owner": "2e6c02d2-a364-4530-9137-d17403996cbf",
    "group": "2e6c02d2-a364-4530-9137-d17403996cbf",
    "aclBit": false
  }
}

As you can see, the top level property in the response is FileStatus, but in the specification file it's fileStatus. Case sensitivity matters. Please fix the specification or the service.

@daschult - please reassign this to @lewu-msft

@sumantmehtams and I talked about this, and he indicated that they aren't interested in having a JavaScript SDK anymore, so this doesn't matter. I'm closing this issue.

authors: daschult ro-joowan


text:
[Network] Invalid swagger: Missing required definition property 'nextHopType' + unused definition "AzureAsyncOperationResult"

Semantic validation on network.json spec 2016-09-01 results in invalid swagger:
[ { [Error: The Swagger document(s) are invalid]
errors:
[ { code: 'MISSING_REQUIRED_DEFINITION_PROPERTY',
message: 'Definition requires property but it is not defined: nextHopType',
path:
[ 'definitions',
'ExpressRouteCircuitRoutesTableSummary',
'required',
'0' ] } ],
failedValidation: true,
warnings:
[ { code: 'UNUSED_DEFINITION',
message: 'Definition is defined but is not used: #/definitions/AzureAsyncOperationResult',
path: [ 'definitions', 'AzureAsyncOperationResult' ] } ],
code: 'INTERNAL_ERROR' } ] }

@Nilambari @deepakrajendranmsft - Can you guys take a look at this issue?

@dihan0604 Can you Please take a look at it? As Ding worked on this.

Also, api-version 2016-03-30 reports Duplicate API Path error as well:
azure-rest-api-specs/arm-network/2016-03-30/swagger/network.json should be valid Swagger
Errors
MISSING_REQUIRED_DEFINITION_PROPERTY : #/definitions/ExpressRouteCircuitRoutesTable/required/0 : Definition requires property but it is not defined: nextHopType
MISSING_REQUIRED_DEFINITION_PROPERTY : #/definitions/ExpressRouteCircuitRoutesTableSummary/required/0 : Definition requires property but it is not defined: nextHopType
DUPLICATE_API_PATH : #/paths//subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/connections/{virtualNetworkGatewayConnectionName}/sharedkey : API path (or equivalent) already defined: /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/connections/{virtualNetworkGatewayConnectionName}/sharedkey

@dihan0604 have you had a chance to look at this?

This has been fixed, Link: https://github.com/Azure/azure-rest-api-specs/pull/855

source: gharchive/issue

authors: Nilambari amarzavery dihan0604 jhendrixMSFT veronicagg

repo: Azure/azure-rest-api-specs

license: MIT

license_type: permissive

license_source: github-api


text:
Add teams data type to office data connector

MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Contribution checklist:

[V] I commit to follow the Breaking Change Policy of “no breaking changes
[V] I have reviewed the documentation for the workflow.
[V] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

[ ] Ensure to check this box if one of the following scenarios meet updates in the PR, so that label “WaitForARMFeedback” will be added automatically to involve ARM API Review. Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs, all “removals” and “adding a new property” no more require ARM API review.

Adding new API(s)
Adding a new API version
Adding a new service

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If there are following updates in the PR, ensure to request an approval from API Review Board as defined in the Breaking Change Policy.

[ ] Removing API(s) in stable version
[ ] Removing properties in stable version
[ ] Removing API version(s) in stable version
[ ] Updating API in stable version with Breaking Change Validation errors
[ ] Updating API(s) in preview over 1 year

Please follow the link to find more details on PR review process.

You don't have permission to trigger SDK Automation.
Please add yourself to Azure group from opensource portal if you are MSFT employee,
or please ask reviewer to add comment *** /openapibot sdkautomation ***.
Please ask tih@microsoft.com (or NullMDR in github) for additional help.

Can one of the admins verify this patch?

@xccc-msft can you please review my pr?

/openapibot sdkautomation

/azp run automation - sdk

/azp run automation - sdk

/azp run automation - sdk

/azp run automation - sdk

@almogTsur Do you know who can help review from service side? I cannot find from CODEOWNERS.

@xccc-msft, i reviewed the PR, looks good

@xccc-msft can you please merge this pr? I don't have the permission for it.
@ofshomro is from the service team and he reviewed the PR and approve from his end.

source: gharchive/pull-request

authors: AutorestCI almogTsur azuresdkci ofshomro xccc-msft


text:
BRP prune backup store API

MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Contribution checklist:

[x] I commit to follow the Breaking Change Policy of “no breaking changes
[x] I have reviewed the documentation for the workflow.
[x] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

[x] Ensure to check this box if one of the following scenarios meet updates in the PR, so that label “WaitForARMFeedback” will be added automatically to involve ARM API Review. Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs, all “removals” and “adding a new property” no more require ARM API review.

Adding new API(s)
Adding a new API version
Adding a new service

[x] Please ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[x] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If there are following updates in the PR, ensure to request an approval from API Review Board as defined in the Breaking Change Policy.

[ ] Removing API(s) in stable version
[ ] Removing properties in stable version
[ ] Removing API version(s) in stable version
[ ] Updating API in stable version with Breaking Change Validation errors
[ ] Updating API(s) in preview over 1 year

Please follow the link to find more details on PR review process.

/azp run automation - sdk

/azp run automation - sdk

/azp run automation - sdk

/azp run automation - sdk

/azp run automation - sdk

/azp run automation - sdk

/azp run automation - sdk

/azp run automation - sdk

Hi @poetw , as indicated in breaking change check result, adding new path is only allowed in a new api version or please get approval from breaking change review board.

Hi @poetw , as indicated in breaking change check result, adding new path is only allowed in a new api version or please get approval from breaking change review board.

Hello Ray,
Varun asked me to ignore the breaking change check since this isn't a breaking change from ARM perspective.

Hi @poetw
        Peter Wu (AZURESTACK)
        FTE , as indicated in breaking change check result, adding new path is only allowed in a new api version or please get approval from breaking change review board.

Hello Ray,
Varun asked me to ignore the breaking change check since this isn't a breaking change from ARM perspective.

from ARM yes, but in my sign-off I did say it could be breaking to sdk

Hi @poetw
        Peter Wu (AZURESTACK)
        FTE , as indicated in breaking change check result, adding new path is only allowed in a new api version or please get approval from breaking change review board.

Hello Ray,
Varun asked me to ignore the breaking change check since this isn't a breaking change from ARM perspective.

from ARM yes, but in my sign-off I did say it could be breaking to sdk

Hello Varun,
By SDK do you mean the powershell cmdlets to be released? The change is going to add a new cmdlet to the admin powershell modules which is similar to the ARM API addition. Is Bala in charge of the SDK?

/azp run automation - sdk

/azp run automation - sdk

Hmm not sure why there're a bunch of warnings on Build Log style etc...

Please make the PR against master branch not stackadmin. @poetw

pls follow this for CI errors
https://github.com/Azure/azure-rest-api-specs/blob/master/documentation/ci-fix.md

@raych1 Whom should we reach out for SDK breaking change? WE already have a ARM sign off as adding an API is not a breaking change.
These APIs are specific to AzureStack Admin, We only ship powershell at the moment. There are no SDKs being shipped for these AzureStack Admin APIs. THese APIs are not exposed in azure

/azp run automation - sdk

/azp run automation - sdk

@raych1 Whom should we reach out for SDK breaking change? WE already have a ARM sign off as adding an API is not a breaking change.
These APIs are specific to AzureStack Admin, We only ship powershell at the moment. There are no SDKs being shipped for these AzureStack Admin APIs. THese APIs are not exposed in azure

@poetw , please provide business justification in the PR comment and why you must have these change(s), and how external customer impact can be mitigated. Please ensure to follow breaking change policy to request breaking change review and approval before proceeding swagger PR review.

@raych1 Whom should we reach out for SDK breaking change? WE already have a ARM sign off as adding an API is not a breaking change.
These APIs are specific to AzureStack Admin, We only ship powershell at the moment. There are no SDKs being shipped for these AzureStack Admin APIs. THese APIs are not exposed in azure

@poetw , please provide business justification in the PR comment and why you must have these change(s), and how external customer impact can be mitigated. Please ensure to follow breaking change policy to request breaking change review and approval before proceeding swagger PR review.

Updated the PR description.

@poetw , is there any update on the breaking change request?

@poetw @raych1 Just had a conversation with the review team and the breaking change is exempted. The PR is approved. COuld you please go ahead and merge this

/azp run

Hello @raych1, Bala went to the board and got an exemption on this PR, could you help merge it as I'm not authorized to do so?
Thanks!

authors: AutorestCI bganapa poetw raych1 varunkumta


text:
Move server service level objectives to servers.json

This checklist is used to make sure that common issues in a pull request are addressed. This will expedite the process of getting your pull request merged and avoid extra work on your part to fix issues discovered during the review process.
PR information

[x] The title of the PR is clear and informative.
[x] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For information on cleaning up the commits in your pull request, see this page.
[x] Except for special cases involving multiple contributors, the PR is started from a fork of the main repository, not a branch.
[] If applicable, the PR references the bug/issue that it fixes.
[x] Swagger files are correctly named (e.g. the api-version in the path should match the api-version in the spec).

Quality of Swagger

[x] I have read the contribution guidelines.
[x] My spec meets the review criteria:

[x] The spec conforms to the Swagger 2.0 specification.
[x] The spec follows the guidelines described in the Swagger checklist document.
[] Validation tools were run on swagger spec(s) and have all been fixed in this PR.

This change is

Hey @nathannfan - so all commits to this PR are purely moving stuff from one spec to another? No other additions/deletions expected?

@balajikris yep, just doing this to rework the categories we're generating in docs.microsoft.com.  I did have to copy SubResource to servers.json as a dependency, but nothing new.

For existing warns/errors, there is a open workitem that we're tracking. This PR doesn't create any new violations. I've reviewed the changes and I'm okay with it. Merging in as is.

Thanks, @balajikris!

No modification for NodeJS

 https://github.com/Azure/azure-sdk-for-ruby/pull/780

 https://github.com/Azure/azure-sdk-for-python/pull/1177

authors: AutorestCI balajikris nathannfan


text:
OperationailInsights: Cluster identity description change.

MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Changelog
Please ensure to add changelog with this PR by answering the following questions.

What's the purpose of the update?

[ ] new service onboarding
[ ] new API version
[ ] update existing version for new feature
[ ] update existing version to fix swagger quality issue in s360
[x] Other, please clarify

When you are targeting to deploy new service/feature to public regions? Please provide date, or month to public if date is not available yet.
When you expect to publish swagger? Please provide date, or month to public if date is not available yet.
If it's an update to existing version,  please select SDKs of specific language and CLIs that require refresh after swagger is published.

[ ] SDK of .NET (need service team to ensure code readiness)
[ ] SDK of Python
[ ] SDK of Java
[ ] SDK of Js
[ ] SDK of Go
[ ] PowerShell
[ ] CLI
[ ] Terraform
[x] No, no need to refresh for updates in this PR

Contribution checklist:

[x] I commit to follow the Breaking Change Policy of “no breaking changes
[x] I have reviewed the documentation for the workflow.
[x] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

[ ] Ensure to check this box if one of the following scenarios meet updates in the PR, so that label “WaitForARMFeedback” will be added automatically to involve ARM API Review. Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs, all “removals” and “adding a new property” no more require ARM API review.

Adding new API(s)
Adding a new API version
Adding a new service

[ ] Please ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If there are following updates in the PR, ensure to request an approval from API Review Board as defined in the Breaking Change Policy.

[ ] Removing API(s) in stable version
[ ] Removing properties in stable version
[ ] Removing API version(s) in stable version
[ ] Updating API in stable version with Breaking Change Validation errors
[ ] Updating API(s) in preview over 1 year

Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Addition details on the process and office hours are on the Breaking change Wiki.
Please follow the link to find more details on PR review process.

Cluster identity type description change.

/azp run

@msyyc Python SDK for Track2 is failed. Please have a look. Thanks.

/azp run


text:
ACR API version 2021-09-01

MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Changelog
Add a changelog entry for this PR by answering the following questions:

What's the purpose of the update?

[ ] new service onboarding
[x] new API version
[ ] update existing version for new feature
[ ] update existing version to fix swagger quality issue in s360
[ ] Other, please clarify

When are you targeting to deploy the new service/feature to public regions? Please provide the date or, if the date is not yet available, the month.
When do you expect to publish the swagger? Please provide date or, the the date is not yet available, the month.
If updating an existing version, please select the specific langauge SDKs and CLIs that must be refreshed after the swagger is published.

[ ] SDK of .NET (need service team to ensure code readiness)
[ ] SDK of Python
[ ] SDK of Java
[ ] SDK of Js
[ ] SDK of Go
[ ] PowerShell
[ ] CLI
[ ] Terraform
[ ] No refresh required for updates in this PR

Contribution checklist:

[ ] I commit to follow the Breaking Change Policy of "no breaking changes"
[ ] I have reviewed the documentation for the workflow.
[ ] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

Applicability: :warning:
If your changes encompass only the following scenarios, you should SKIP this section, as these scenarios do not require ARM review.

Change to data plane APIs
Adding new properties
All removals

Otherwise your PR may be subject to ARM review requirements. Complete the following:

[ ] Check this box if any of the following apply to the PR so that label “WaitForARMFeedback” will be added automatically to begin ARM API Review. Failure to comply may result in delays to the manifest.

Adding a new service
Adding new API(s)
Adding a new API version
-[ ] To review changes efficiently, ensure you copy the existing version into the new directory structure for first commit and then push new changes, including version updates, in separate commits.

[ ] Ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If any of the following scenarios apply to the PR, request approval from the Breaking Change Review Board as defined in the Breaking Change Policy.

[ ] Removing API(s) in a stable version
[ ] Removing properties in a stable version
[ ] Removing API version(s) in a stable version
[ ] Updating API in a stable or public preview version with Breaking Change Validation errors
[ ] Updating API(s) in public preview over 1 year (refer to Retirement of Previews)

Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Addition details on the process and office hours are on the Breaking change Wiki.
Please follow the link to find more details on PR review process.

@fengzhou-msft There are a couple of issues where I will need guidance from the swagger team. There are two failures in the Azure OpenAPI pipeline:

SDK azure-sdk-for-net failed: For this GA API, we are not including some features from the past preview APIs, such as Transfer. However, we have checked in Transfer tests in the .NET SDK repo. The tests are currently failing because some of these definitions are not included in the current swagger.

What is your guidance on this? Could we get an exception for this test? Our .NET SDK is not automatically published.

SDK azure-sdk-for-go-track2: I'm unfamiliar with this stage, as it was usually skipped in previous PRs. It seems that it can't find a the Go Module, but the SDK azure-sdk-for-go step succeeded (previous to running this step). Any guidance on this?

cc @jaysterp

You can ignore the dotnet SDK test failure, it will not block this PR. But make sure there are only expected failures.
@ArcturusZhang for go SDK questions.

What is your base api? you need to follow our documentation: https://armwiki.azurewebsites.net/rp_onboarding/process/api_review.html basically these steps:

Create your new folder (RP namespace, and/or  api version)
Copy and paste your baseline swagger (from your previous api version)
Rename the files to match your needs
This step is super important, now commit & push this first iteration
Start making changes to your swagger (add the new definitions)

Please abandon this PR and create one following these steps, otherwise I'll have to review your entire swagger and will cause in delays or me reviewing existing items.

authors: AntonioVT fengzhou-msft jorgecotillo


text:
Adding oauth2AllowImplicitFlow property to ApplicationCreateParameters and ApplicationUpdateParameters

Adding oauth2AllowImplicitFlow property to ApplicationCreate and ApplicationUpdate parameters.
Tested with local change using azure-sdk-node to see if the proprety is getting applied on newly created application.

This checklist is used to make sure that common issues in a pull request are addressed. This will expedite the process of getting your pull request merged and avoid extra work on your part to fix issues discovered during the review process.
PR information

[x] The title of the PR is clear and informative.
[x] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For information on cleaning up the commits in your pull request, see this page.
[x] Except for special cases involving multiple contributors, the PR is started from a fork of the main repository, not a branch.
[ ] If applicable, the PR references the bug/issue that it fixes.
[x] Swagger files are correctly named (e.g. the api-version in the path should match the api-version in the spec).

Quality of Swagger

[x] I have read the contribution guidelines.
[x] My spec meets the review criteria:

[x] The spec conforms to the Swagger 2.0 specification.
[x] The spec follows the guidelines described in the Swagger checklist document.
[ ] Validation tools were run on swagger spec(s) and have all been fixed in this PR.

@ppathan,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

No modification for AutorestCI/azure-sdk-for-node

 https://github.com/Azure/azure-sdk-for-python/pull/1456

 https://github.com/Azure/azure-sdk-for-ruby/pull/988

authors: AutorestCI msftclas ppathan


text:
[Hub Generated] Publish private branch 'addCmkApiToStable'

This is a PR generated at OpenAPI Hub. You can view your work branch via this link.
Changelog
Add a changelog entry for this PR by answering the following questions:

What's the purpose of the update?

[ ] new service onboarding
[ ] new API version
[ ] update existing version for new feature
[ ] update existing version to fix swagger quality issue in s360
[ ] Other, please clarify

When are you targeting to deploy the new service/feature to public regions? Please provide the date or, if the date is not yet available, the month.
When do you expect to publish the swagger? Please provide date or, the the date is not yet available, the month.
If updating an existing version, please select the specific language SDKs and CLIs that must be refreshed after the swagger is published.

[ ] SDK of .NET (need service team to ensure code readiness)
[ ] SDK of Python
[ ] SDK of Java
[ ] SDK of Js
[ ] SDK of Go
[ ] PowerShell
[ ] CLI
[ ] Terraform
[ ] No refresh required for updates in this PR

Contribution checklist:

[ ] I commit to follow the Breaking Change Policy of "no breaking changes"
[ ] I have reviewed the documentation for the workflow.
[ ] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

Applicability: :warning:
If your changes encompass only the following scenarios, you should SKIP this section, as these scenarios do not require ARM review.

Change to data plane APIs
Adding new properties
All removals

Otherwise your PR may be subject to ARM review requirements. Complete the following:

[ ] Check this box if any of the following apply to the PR so that label "WaitForARMFeedback" will be added automatically to begin ARM API Review. Failure to comply may result in delays to the manifest.

Adding a new service
Adding new API(s)
Adding a new API version
-[ ] To review changes efficiently, ensure you are using OpenAPIHub to initialize the PR for adding a new version. More details, refer to the wiki.

[ ] Ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If any of the following scenarios apply to the PR, request approval from the Breaking Change Review Board as defined in the Breaking Change Policy.

[ ] Removing API(s) in a stable version
[ ] Removing properties in a stable version
[ ] Removing API version(s) in a stable version
[ ] Updating API in a stable or public preview version with Breaking Change Validation errors
[ ] Updating API(s) in public preview over 1 year (refer to Retirement of Previews)

Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Addition details on the process and office hours are on the Breaking change Wiki.
Please follow the link to find more details on PR review process.

The PR is created based on the updates in the private branch. The updates in the PR has already been reviewed and approved with this PR Azure/azure-rest-api-specs-pr/6981


text:
[Hub Generated] Publish private branch 'appplatform/release/s213'

This is a PR generated at OpenAPI Hub. You can view your work branch via this link.
ARM API Information (Control Plane)
Azure 1st Party Service can try out the Shift Left experience to initiate API design review from ADO code repo.  If you are interested, may request engineering support by filling in with the form https://aka.ms/ShiftLeftSupportForm.
Changelog
Add a changelog entry for this PR by answering the following questions:

What's the purpose of the update?

[ ] new service onboarding
[ ] new API version
[ ] update existing version for new feature
[ ] update existing version to fix swagger quality issue in s360
[ ] Other, please clarify

When are you targeting to deploy the new service/feature to public regions? Please provide the date or, if the date is not yet available, the month.
When do you expect to publish the swagger? Please provide date or, the the date is not yet available, the month.
By default, Azure SDKs of all languages (.NET/Python/Java/JavaScript for both management-plane SDK and data-plane SDK, Go for management-plane SDK only ) MUST be refreshed with/after swagger of new version is published. If you prefer NOT to refresh any specific SDK language upon swagger updates in the current PR, please leave details with justification here.

Contribution checklist (MS Employees Only):

[ ] I commit to follow the Breaking Change Policy of "no breaking changes"
[ ] I have reviewed the documentation for the workflow.
[ ] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

Applicability: :warning:
If your changes encompass only the following scenarios, you should SKIP this section, as these scenarios do not require ARM review.

Change to data plane APIs
Adding new properties
All removals

Otherwise your PR may be subject to ARM review requirements. Complete the following:

[ ] Check this box if any of the following apply to the PR so that the label "ARMReview" and "WaitForARMFeedback" will be added by bot to kick off ARM API Review.  Missing to check this box in the following scenario may result in delays to the ARM manifest review and deployment.

Adding a new service
Adding new API(s)
Adding a new API version
-[ ] To review changes efficiently, ensure you copy the existing version into the new directory structure for first commit and then push new changes, including version updates, in separate commits. You can use OpenAPIHub to initialize the PR for adding a new version. For more details refer to the wiki.

[ ] Ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If you have any breaking changes as defined in the Breaking Change Policy, request approval from the Breaking Change Review Board.
Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Additional details on the process and office hours are on the Breaking Change Wiki.
NOTE: To update API(s) in public preview for over 1 year (refer to Retirement of Previews)
Please follow the link to find more details on PR review process.

The PR is created based on the updates in the private branch. The updates in the PR has already been reviewed and approved with this PR Azure/azure-rest-api-specs-pr/9828

Approved as this PR is to publish above reviewed PR from private spec repo.

Talked to @yuwzho offline, she will create a new PR later to fix modelvalidation errors.


text:
[ACS JobRouter] Python sdk customizations

Python SDK Customizations. Teams post (internal)

@iscai-msft can you please review these changes?

authors: sarkar-rajarshi tjprescott


text:
Modify CircuitBreaker and BackendPool Contract

ARM (Control Plane) API Specification Update Pull Request

[!TIP]
Overwhelmed by all this guidance? See the Getting help section at the bottom of this PR description.

[!NOTE]
As of January 2024 there is no PR assignee. This is expected. See https://aka.ms/azsdk/pr-arm-review.

PR review workflow diagram
Please understand this diagram before proceeding. It explains how to get your PR approved & merged.

Click here to see the details of Step 1, Breaking Changes review
If you are in purview of Step 1 of the diagram, follow the Breaking Changes review process.
IMPORTANT! This applies even if you believe your PR was mislabeled, for any reason, including tool failure.

Click here to see the details of Step 2, ARM review
See https://aka.ms/azsdk/pr-arm-review.

 Click here to see the diagram footnotes
Diagram footnotes
[1] See ARM review queue (for PR merge queues, see [2]).
[2] public repo merge queue, private repo merge queue (for ARM review queue, [1])
The ARM reviewer on-call engineer visits the merge queue twice a day, so the approximate ETA for merges is 12 - 24 hours.

Purpose of this PR
What's the purpose of this PR? Check the specific option that applies. This is mandatory!

[ ] New resource provider.
[ ] New API version for an existing resource provider. (If API spec is not defined in TypeSpec, the PR should have been generated using OpenAPI Hub).
[ ] Update existing version for a new feature. (This is applicable only when you are revising a private preview API version.)
[ ] Update existing version to fix OpenAPI spec quality issues in S360.
[ ] Other, please clarify:

edit this with your clarification

Due diligence checklist
To merge this PR, you must go through the following checklist and confirm you understood
and followed the instructions by checking all the boxes:

[ ] I confirm this PR is modifying Azure Resource Manager (ARM) related specifications, and not data plane related specifications.
[ ] I have reviewed following Resource Provider guidelines, including
ARM resource provider contract and
REST guidelines (estimated time: 4 hours).
I understand this is required before I can proceed to the diagram Step 2, "ARM API changes review", for this PR.

Additional information

 Viewing API changes
For convenient view of the API changes made by this PR, refer to the URLs provided in the table
in the Generated ApiView comment added to this PR. You can use ApiView to show API versions diff.

Suppressing failures
If one or multiple validation error/warning suppression(s) is detected in your PR, please follow the
suppressions guide to get approval.

Getting help

First, please carefully read through this PR description, from top to bottom. Please fill out the Purpose of this PR and Due diligence checklist.
To understand what you must do next to merge this PR, see the Next Steps to Merge comment. It will appear within few minutes of submitting this PR and will continue to be up-to-date with current PR state.
For guidance on fixing this PR CI check failures, see the hyperlinks provided in given failure
and https://aka.ms/ci-fix.
For help with PR workflow diagram Step 2 (ARM review), see https://aka.ms/azsdk/pr-arm-review.
If the PR CI checks appear to be stuck in queued state, please add a comment with contents /azp run.
This should result in a new comment denoting a PR validation pipeline has started and the checks should be updated after few minutes.
If the help provided by the previous points is not enough, post to https://aka.ms/azsdk/support/specreview-channel and link to this PR.

@m-h-s there is only one comment, if you can not use booleans true or false for property acceptRetryAfterHeader property
https://github.com/Azure/azure-rest-api-specs/pull/22978#issuecomment-1517001062
ARM holds us accountable for this everytime we go for review.

/pr RequestMerge

authors: m-h-s solankisamir


text:
chore: jsonfmt peering

Latest improvements:
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Contribution checklist:

[ ] I have reviewed the documentation for the workflow.
[ ] Validation tools were run on swagger spec(s) and have all been fixed in this PR.
[ ] The OpenAPI Hub was used for checking validation status and next steps.

ARM API Review Checklist

[ ] Service team MUST add the "WaitForARMFeedback" label if the management plane API changes fall into one of the below categories.
adding/removing APIs.
adding/removing properties.
adding/removing API-version.
adding a new service in Azure.

Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs.

[ ] If you are blocked on ARM review and want to get the PR merged urgently, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.
Please follow the link to find more details on API review process.

Automation for azure-sdk-for-python

Automation for azure-sdk-for-go

Can one of the admins verify this patch?

authors: AutorestCI azuresdkci nschonni


text:
chore: jsonfmt mysql

Latest improvements:
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Contribution checklist:

[ ] I have reviewed the documentation for the workflow.
[ ] Validation tools were run on swagger spec(s) and have all been fixed in this PR.
[ ] The OpenAPI Hub was used for checking validation status and next steps.

ARM API Review Checklist

[ ] Service team MUST add the "WaitForARMFeedback" label if the management plane API changes fall into one of the below categories.
adding/removing APIs.
adding/removing properties.
adding/removing API-version.
adding a new service in Azure.

Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs.

[ ] If you are blocked on ARM review and want to get the PR merged urgently, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.
Please follow the link to find more details on API review process.

Automation for azure-sdk-for-python

Automation for azure-sdk-for-go

Can one of the admins verify this patch?


text:
Test Portal Automation

MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Contribution checklist:

[ ] I have reviewed the documentation for the workflow.
[ ] Validation tools were run on swagger spec(s) and have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

[ ] Service team MUST add the "WaitForARMFeedback" label if the management plane API changes fall into one of the below categories.

adding/removing APIs.
adding/removing properties.
adding/removing API-version.
adding a new service in Azure.

Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs.

[ ] If you are blocked on ARM review and want to get the PR merged urgently, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.
Please follow the link to find more details on API review process.

/azp run automation - sdk

Can one of the admins verify this patch?

authors: AutorestCI LianwMS azuresdkci


text:
update cdn java md

MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Contribution checklist:

[ ] I have reviewed the documentation for the workflow.
[ ] Validation tools were run on swagger spec(s) and have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

[ ] Service team MUST add the "WaitForARMFeedback" label if the management plane API changes fall into one of the below categories.

adding/removing APIs.
adding/removing properties.
adding/removing API-version.
adding a new service in Azure.

Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs.

[ ] If you are blocked on ARM review and want to get the PR merged urgently, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.
Please follow the link to find more details on API review process.

/azp run automation - sdk

Can one of the admins verify this patch?

authors: AutorestCI ChenTanyi azuresdkci


text:
app crashes on init of AzureData; Unable to invoke no-args constructor for class com.azure.data.model.service.ResourceLocation

Environment

Android version multiple in range from 6.0+:
Azure.Android SDK:
Azure.Android SDK version: 0.4.0

Bug Summary
2020-07-01 15:30:28.500 16055-16068/? E/zygote64: Failed sending reply to debugger: Broken pipe
2020-07-01 15:30:40.584 16055-16187/sogo.android.sogogolf E/AndroidRuntime: FATAL EXCEPTION: pool-10-thread-1
Process: sogo.android.sogogolf, PID: 16055
java.lang.RuntimeException: Unable to invoke no-args constructor for class com.azure.data.model.service.ResourceLocation. Registering an InstanceCreator with Gson for this type may fix this problem.
at com.google.gson.internal.ConstructorConstructor$14.construct(ConstructorConstructor.java:228)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:212)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:131)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:222)
at com.google.gson.Gson.fromJson(Gson.java:927)
at com.google.gson.Gson.fromJson(Gson.java:892)
at com.google.gson.Gson.fromJson(Gson.java:841)
at com.google.gson.Gson.fromJson(Gson.java:813)
at com.azure.data.util.json.ResourceWriteOperationAdapter.deserializeDetails(ResourceWriteOperationAdapter.kt:93)
at com.azure.data.util.json.ResourceWriteOperationAdapter.deserialize(ResourceWriteOperationAdapter.kt:37)
at com.azure.data.util.json.ResourceWriteOperationAdapter.deserialize(ResourceWriteOperationAdapter.kt:15)
at com.google.gson.internal.bind.TreeTypeAdapter.read(TreeTypeAdapter.java:69)
at com.google.gson.Gson.fromJson(Gson.java:927)
at com.google.gson.Gson.fromJson(Gson.java:892)
at com.google.gson.Gson.fromJson(Gson.java:841)
at com.google.gson.Gson.fromJson(Gson.java:813)
at com.azure.data.service.ResourceWriteOperationQueue$load$$inlined$safeExecute$lambda$1.run(ResourceWriteOperationQueue.kt:133)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636)
at java.lang.Thread.run(Thread.java:764)
Caused by: java.lang.UnsupportedOperationException: Abstract class can't be instantiated! Class name: com.azure.data.model.service.ResourceLocation
at com.google.gson.internal.UnsafeAllocator.assertInstantiable(UnsafeAllocator.java:120)
at com.google.gson.internal.UnsafeAllocator$1.newInstance(UnsafeAllocator.java:49)
at com.google.gson.internal.ConstructorConstructor$14.construct(ConstructorConstructor.java:225)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:212)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:131)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:222)
at com.google.gson.Gson.fromJson(Gson.java:927)
at com.google.gson.Gson.fromJson(Gson.java:892)
at com.google.gson.Gson.fromJson(Gson.java:841)
at com.google.gson.Gson.fromJson(Gson.java:813)
at com.azure.data.util.json.ResourceWriteOperationAdapter.deserializeDetails(ResourceWriteOperationAdapter.kt:93)
at com.azure.data.util.json.ResourceWriteOperationAdapter.deserialize(ResourceWriteOperationAdapter.kt:37)
at com.azure.data.util.json.ResourceWriteOperationAdapter.deserialize(ResourceWriteOperationAdapter.kt:15)
at com.google.gson.internal.bind.TreeTypeAdapter.read(TreeTypeAdapter.java:69)
at com.google.gson.Gson.fromJson(Gson.java:927)
at com.google.gson.Gson.fromJson(Gson.java:892)
at com.google.gson.Gson.fromJson(Gson.java:841)
at com.google.gson.Gson.fromJson(Gson.java:813)
at com.azure.data.service.ResourceWriteOperationQueue$load$$inlined$safeExecute$lambda$1.run(ResourceWriteOperationQueue.kt:133)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636)
at java.lang.Thread.run(Thread.java:764)
Steps to Reproduce
Thats a tough one unf. It seems that somehow AzureData gets into a state where it will no longer initialise. When that happens, the app will crash at the very first call to init AzureData, ie.
AzureData.configure(applicationContext, COSMOS_ACCOUNT_NAME, COSMOS_MASTER_KEY, PermissionMode.All /* or PermissionMode.Read */)
Expected Behavior
doesnt crash app
Actual Behavior
crashes app
Really appreciate if someone can look into this as its affecting multiple of our Android users

hey @NateRickard hope your well ! Did you have any sort of eta on this bug fix ? It's pretty urgent from our side. Even if in the interim there is a workaround. Thanks for the help on this.

hi folks any update on this one?

Hey @SogoGolf, just doing some wild guessing here. Based on your repro steps, you mentioned that this can occur as soon as you first configure AzureData, before you even execute any operations. To me, this points to a possibility that you are using the offline data feature of the library, and the local data store has been corrupted. Are you using offline data? If so, can you try disabling it?:
AzureData.offlineDataEnabled = false

If you require the offline data feature, you could try catching the exception and deleting the cache directory when this situation occurs. The ResourceCache class has a convenient purge() method for this, but unfortunately the class is internal. But I believe the following should achieve the same result:
try {
    AzureData.configure(applicationContext, COSMOS_ACCOUNT_NAME, COSMOS_MASTER_KEY, PermissionMode.All /* or PermissionMode.Read */)
}
catch (e: Exception) {
    val cacheDir = File(applicationContext.cacheDir, "com.azuredata.data")
    if (cacheDir.exists) {
        cacheDir.deleteRecursively()
    }

    AzureData.configure(applicationContext, COSMOS_ACCOUNT_NAME, COSMOS_MASTER_KEY, PermissionMode.All /* or PermissionMode.Read */)
}

You may also need to delete any shared preferences files that start with the prefix "com.azure.data.lookup", but I'm not sure if that's necessary.

hi @bsiegel , thanks for your help with this. actually yes we are using / have enabled "offline" mode. we'll try disabling it as a first step.

@bsiegel we removed the offline data flag offlineDataEnabled and it seems like it might have solved the problem. We're monitoring crash reports and waiting for a few more users to report back before closing, will let you know in next few days.

authors: SogoGolf bsiegel

repo: Azure/azure-sdk-for-android


text:
[BUG] JSON parser should [de]serialize according to IEEE

See https://en.wikipedia.org/wiki/IEEE_754

@ahsonkhan : Again, I think you know more about this than I do so you should decide how to best handle JSON numbers.

authors: JeffreyRichter sergey-shandar

repo: Azure/azure-sdk-for-c


text:
Deploy ARM template using the Go SDK

Hello!
Can you deploy an ARM template using the Go SDK?
Here is the scenario: I recently contributed to a project where the requirement was to create a simple VM based on VHD file you created locally.
You had to upload the VHD in a storage account, then create all necessary infrastructure for the VM - virtual network, subnet, public IP address, network interface, network security group and so on.
Here is the actual project for that.
It would be much simpler (and easier for versioning and so on) if I could write an ARM template and then from my Go application (maybe) set some parameters and deploy it.
While I'm not an expert on this SDK, I haven't seen in possible to actually take an ARM template and deploy it.
Is this something possible? If not, is it a scenario that is worth exploring?
Also, what is currently the recommended way of achieving what I was talking about earlier?
Thanks!

Howdy @radu-matei, always good to hear from you!
Yes, using the Azure SDK for Go, you are able to do use ARM templates using "Deployments".
The package that one looking to utilize ARM templates will find most useful can be found here:

Hi, @marstr!
Thanks a lot for the fast reply, helpful information.

authors: marstr radu-matei

repo: Azure/azure-sdk-for-go


text:
adds a claims field placeholder

partial #14284 adds a claims field placeholder to unlock Microsoft Graph SDKs work.

[x] The purpose of this PR is explained in this or a referenced issue.
[x] The PR does not update generated files.

These files are managed by the codegen framework at Azure/autorest.go.

~[ ] Tests are included and/or updated for code changes.~ N/A
[x] Updates to CHANGELOG.md are included.
[x] MIT license headers are included in each file.

I'm reluctant to add this field apart from a full CAE implementation because by itself it has no function and invites confusion. Can you elaborate on how this unblocks the Graph SDK? Even if azcore.TokenCredential had the API it would need to support CAE, actually doing so would require custom credential types (or perhaps a hacky adapter?) because no azidentity credential will claim to support CAE or request tokens with particular claims.

I'm reluctant to add this field apart from a full CAE implementation because by itself it has no function and invites confusion. Can you elaborate on how this unblocks the Graph SDK? Even if azcore.TokenCredential had the API it would need to support CAE, actually doing so would require custom credential types (or perhaps a hacky adapter?) because no azidentity credential will claim to support CAE or request tokens with particular claims.

@chlowell thanks for the reply.
mainly because I had to return an error when I see the claim coming back from the http adapter. Having the field would allow all the infrastructure to be in place in the Microsoft Graph SDK, even though "it wouldn't do anything". And then when you ship the full implementation of CAE in the credentials, simply updating the reference would light it up in the Microsoft Graph SDKs.
See this issue for more details

I see how this would be convenient by enabling you to do some of your CAE work in advance. However, I still think it's best to wait on this until we actually support CAE. Setting aside for the moment that I don't want to ship nonfunctional API, I see practical problems. I also don't want to commit to (release as stable) part of a feature in isolation from the rest of it. Hypothetically then--I don't think we should really do this--I suppose we would release this field in an azcore beta and then maintain parallel beta and stable versions until we add CAE support. That's a lot of work for a field that doesn't do anything 😅

alright, thanks for the rationale. We'll keep returning errors that point to the issues in this repo then until the implementation work is done. Hopefully this will help driving the demand from the customer side. If you could share timelines on the issues it'd be helpful as well. Closing.

authors: baywet chlowell


text:
[azmetrics] add sovereign cloud support

Closes https://github.com/Azure/azure-sdk-for-go/issues/22547

Adding sovereign cloud support to azmetrics
Adding weekly live testing for the US Gov and China clouds

/azp run go - azmetrics - weekly

/azp run go - azmetrics - weekly


text:
[BUG] Add Rule Fail with Exception (SDK)

Describe the bug
We are using Azure SDK for Java (azure-resourcemanager library), version: 2.24.0.
While using the SDK to add new rule, we get the following error.
This happened in some scenarios using multiple ASGs that are linked to same Vnet.
In other case one of them is located under a different resource group.
We also got this error message when adding a new rule without using ASG at all.
We created the same rule manually throw azure portal, the rule was added to NSG and no error message appeared.
Exception or Stack Trace
"code": "AllApplicationSecurityGroupsInSecurityRuleMustBeFromSameVnet",
"message": "Security Rule /subscriptions/"subscription_id"/resourceGroups/CM/providers/Microsoft.Network/networkSecurityGroups/CMvm2-nsg has Application Security Groups (ASGs) from multiple virtual networks (/subscriptions/"subscription_id"/resourceGroups/CM/providers/Microsoft.Network/virtualNetworks/CM-VNET, /subscriptions/"subscription_id"/resourceGroups/CM_QA/providers/Microsoft.Network/virtualNetworks/CM_QA-VNET).
All ASGs on the Security Rule must be from the same virtual network."
To Reproduce
As I mentioned it is reproduced occasionally but the scenario that is most common is adding a rule with ASGs linked to the same VM while one of them is located on different resource group.
We created the same rule manually throw azure portal, the rule was added to NSG and no error message appeared.
Code Snippet
nsg.update().defineRule(“ruleName”)
.allowInbound()
.withSourceApplicationSecurityGroup(“/subscriptions/"subscription_id"/resourceGroups/CM_QA/providers/Microsoft.Network/applicationSecurityGroups/asg_nva”).fromAnyPort()
.withDestinationApplicationSecurityGroup(“/subscriptions/"subscription_id"/resourceGroups/CM_QA/providers/Microsoft.Network/applicationSecurityGroups/ASG_api”,
“/subscriptions/"subscription_id"/resourceGroups/CM_QA/providers/Microsoft.Network/applicationSecurityGroups/ASG_web”)
.toAnyPort().withAnyProtocol().withPriority(107)
.attach().apply
Expected behavior
a new rule should be added without exception error.
Setup (please complete the following information):
OS: windows\linux
IDE: IntelliJ
Library/Libraries: com.azure.resourcemanager:azure-resourcemanager:2.24.0com.azure.resourcemanager:azure-resourcemanager-network:2.24.0
Java version: 8
we suspect that the problem is related to the SDK because same rules are working properly through Azure portal,
the rule was added to NSG and no error message appeared.

Hi,
any update regarding this issue ?

@NatanelY
Is this error message true in your case?
"Security Rule /subscriptions/"subscription_id"/resourceGroups/CM/providers/Microsoft.Network/networkSecurityGroups/CMvm2-nsg has Application Security Groups (ASGs) from multiple virtual networks (/subscriptions/"subscription_id"/resourceGroups/CM/providers/Microsoft.Network/virtualNetworks/CM-VNET, /subscriptions/"subscription_id"/resourceGroups/CM_QA/providers/Microsoft.Network/virtualNetworks/CM_QA-VNET).
All ASGs on the Security Rule must be from the same virtual network."
It seems indeed say that there is 2 different Vnet.
Portal and SDK may use different api-version. 2.24.0 SDK would be the latest 2022-09-01 on Network. Sometimes, a later api-version would have improved validation on backend that try tell error before it get into the system.

@weidongxu-microsoft
Hi,
Yes this is the error as I mentioned in the case.
I checked again the following scenarios:

I tried to add a new rule to NSG1.
I used 2 ASGs that linked to VNET1, so that one is source and the other is destination.
VNET1 and NSG1 are under the same resource group.
It was added successfully through SDK.

I tried to add a new rule to NSG2.
I used 2 ASGs that linked to VNET1, so that one is source and the other is destination.
VNET1 and NSG2 are under different resource groups.
this operation failed with the error message.

So I think that there is another dependency that is not documented:
ASGs should be under the same VNET and the VNET should be located under the same resource group with the NSG that we want to implement the rule.

Thanks for the details.
@XiaofeiCao Could you double check in SDK and maybe with backend (if necessary)?
It seems not very reasonable that the setting to NSG get different behavior on whether the VNet in same resource group or not (resource group usually just a logic thing),

It doesn't make sense to me too. I'll have that checked.

Hi, thanks for your response.
Our ASGs are under different resource groups.
meaning ASG1 is under rgName2, and linked to Vnet1.
We also encountered weird scenario:
We tried to add a rule to the NSG; with Service tag in source and IP address in destination.
The operation failed and we got same error message as mentioned above.
Maybe the NSG includes rule\s which contain ASGs which doesn't align with the SDK validations, but we would expect to get relevant error when an attempt to add rule doesn't include ASGs.

Let me try to get this straight. Is this the configuration in your failure case?

Resource
Resource Group

ASG1
RG2

ASG2
RG1

VNET1
RG1

NSG2
RG2

correct

Correct, maybe we tried to do some
"Portal and SDK may use different api-version."
I have another question regarding this quote, when do you plan to upgrade the portal SDK to 2.24.0?

Thanks @NatanelY for confirmation,
You can try our newly released 2.26.0. A new Network api-version has been used since 2.25.0.
SDK is released monthly.

authors: NatanelY XiaofeiCao weidongxu-microsoft

repo: Azure/azure-sdk-for-java


text:
Can you increase the version of the dependency com.fasterxml.jackson.dataformat:jackson-dataformat-xml above 2.15.3 ?

Query/Question
At the moment the  library uses the dependency com.fasterxml.jackson.dataformat:jackson-dataformat-xml in version 2.13.5, which is quite behind the actual version or the version used by Spring Boot 3.
Unfortunately any version higher than 2.15.3 leads to a crash of the library. I add a file with the crash report. Please see below.
However Spring Boot 3 enforces a higher version.
So could you please increase the version for com.fasterxml.jackson.dataformat:jackson-dataformat-xml used by the library?
Why is this not a Bug or a feature Request?
Its a bit of a grey area here, I fear.
Setup (please complete the following information if applicable):

OS: Windows
IDE: IntelliJ
Library/Libraries: com.azure:azure-storage-file-datalake:12.18.2

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

[X] Query Added
[X] Setup information Added
exception.txt

@alzimmermsft will this be resolved by moving Storage to azure-xml ?

Hi @TreasureM, thanks for filing this issue and including a stack trace of the exception thrown.
This doesn't appear to be an issue with the dependencies being used by the Storage SDK, as the exception shows that later versions than 2.13.5 are being used, the issue is that there is a dependency issue with the Jackson dependencies aligning on a version. The exception that Java.isIterationType is due to this as it was added in Jackson Databind 2.16.0, aligning the Jackson dependencies should resolve this.
@joshfree, yes, switching to azure-xml would resolve this as Jackson would no longer be used to serialize XML, removing the jackson-dataformat-xml dependency.

/unresolve
We have a version conflict from our Spring Boot Application when we request the  dependency com.fasterxml.jackson.dataformat:jackson-dataformat-xml with the message
"Caused by: java.lang.ClassCastException: class com.fasterxml.jackson.dataformat.yaml.YAMLFactory cannot be cast to class com.fasterxml.jackson.dataformat.xml.XmlFactory (com.fasterxml.jackson.dataformat.yaml.YAMLFactory and com.fasterxml.jackson.dataformat.xml.XmlFactory are in unnamed module of loader 'app')"
Can we resolve this by using azure-xml or do we have to wait till you change those dependencies?

//resolve
We found the core problem. But it would still be intersting to know, when azure-xml will come in use here. ;)

(Was just in the process of replying to the previous comment, but glad you found the solution to that)
I'll need to discuss the timeline with the owners of the Java Storage SDKs, and can update here once there is more concrete information. The backing dependency for this migration is still in beta (azure-xml) but we're actively validating it with beta releases of azure-data-tables and the admin portion of azure-messaging-servicebus. Roughly, if those go well azure-xml should be nearing GA status and then will come the work to migrate the Storage libraries to it.

authors: TreasureM alzimmermsft joshfree


text:
Fix the issue 10448: Azure storage file data lake Sample code is wrong

Fix the issue https://github.com/Azure/azure-sdk-for-java/issues/10448

Thank you for submitting this PR. We really appreciate the help!

authors: Melanie-Ma rickle-msft


text:
Prepare to release spring-experimental related artifacts.

As title, prepare to release spring-experimental related artifacts.
I'll use different PR to upgrade artifacts versions in different release pipeline.

API change check
API changes are not detected in this pull request.

Hi, @saragluna , please help to review this PR when you have time.

Closing this PR because spring-experimental should be released after spring & bom.

authors: azure-sdk chenrujun


text:
Increment versions for eventhubs releases

Increment package versions for eventhubs releases

API change check
API changes are not detected in this pull request.


text:
[AutoPR] cognitiveservices/resource-manager

Created to accumulate context: cognitiveservices/resource-manager

Build output:


text:
[Automation] Generate Fluent Lite from netapp#package-preview-2023-05

[Automation] Generate Fluent Lite from netapp#package-preview-2023-05

API change check
APIView has identified API level changes in this PR and created following API reviews.
com.azure.resourcemanager:azure-resourcemanager-netapp

/azp run java - netapp - mgmt - tests


text:
Cannot use DataLakePathClient to do things with the root path

Package Name: @azure/storage-file-datalake
Package Version: 12.2.0
Operating system: any
[x] nodejs

version: 12

[ ] browser

name/version:

[ ] typescript

version:

Is the bug related to documentation in

[ ] README.md
[ ] source code documentation
[ ] SDK API docs on https://docs.microsoft.com

Describe the bug
If I create a DataLakePathClient with a URL like https://accountname.dfs.core.windows.net/containername/somefolder then things work as expected. But, if I try to use a URL that points to the root path, such as either:

https://accountname.dfs.core.windows.net/containername/ (this one I would most expect to work)
https://accountname.dfs.core.windows.net/containername (this one seems to point more to the container, I'm ok if it doesn't work)

then the SDK throws the following error:
message:'Unable to extract blobName and containerName with provided information.'

which appears to originate from here:

because blobName is an empty string.
If I debug into the SDK and avoid the error by temporarily changing the parsed blobName to be something not empty string before the if check, and then back to empty string after the if check, then things work as expected.
To Reproduce
Steps to reproduce the behavior:

Create a DataLakePathClient which points to the root directory.

Expected behavior
The root is a valid path, so I should be able to use a path client with it.
Screenshots
N/A
Additional context
N/A

Yes, this is a bug.
Need to find out what's the right blobName for root path. We should allow empty blobName as long as it works with the SAS logic.

Thanks for the fast resolution @ljian3377 !

Thanks for the fast resolution @ljian3377 !

repo: Azure/azure-sdk-for-js


text:
Add Data plane SDK for Azure Remote Rendering

Just starting work on this.

/azp run prepare-pipelines

/azp run prepare-pipelines

/azp run js - mixedreality-remoterendering - tests

/azp run js - mixedreality-remoterendering - tests

/azp run js - mixedreality-remoterendering - ci

/azp run js - core - ci

Hi there. I believe this PR is ready for review, so I'm pinging all the automatically added reviewers. Could some of you please take a look? Thanks :) (Apologies if this is just noise for you. It's not obvious to me who is most suitable.)
@benbp , @bterlson , @chradek , @mikeharder , @mitchdenny , @praveenkuttappan , @ramya-rao-a , @xirzec

@chradek can you do a first pass?
Also I wonder if this package should be @azure/mixedreality-remote-rendering

/azp run js - core - ci

/azp run js - mixedreality-remoterendering - tests

@xirzec, @benbp, @chradek Hi. This PR has been open for a long while without any responses. I'd like to get sign-off or, if not, some further feedback I can act on. Thanks!

Looks good for the most part AFAICT! I left a few comments

@deyaaeldeen Thanks for your review. I was able to apply most of your suggestions.

/azp run js - mixedreality-remoterendering - tests

/azp run js - mixedreality-remoterendering - tests

/azp run js - mixed-reality-remote-rendering - tests

/azp run prepare-pipelines

/azp run js - mixed-reality-remote-rendering - tests

/azp run js - mixed-reality-authentication - tests

authors: MalcolmTyrrell xirzec


text:
[storage-blob]Manually change to not send x-ms-encryption-algorithm header when customer doesn't indicate CPK info.

[storage-blob]Manually change to not send x-ms-encryption-algorithm header when customer doesn't indicate CPK info.

Manually modifying generated code doesn't sound like a good idea.  @sarangan12 do you know if there is a way to achieve this using swagger transformation?  This is a parameter from https://raw.githubusercontent.com/Azure/azure-rest-api-specs/564d0137d08defea63d9de2413ef5336fbfe2e6d/specification/storage/data-plane/Microsoft.BlobStorage/preview/2021-04-10/blob.json
    "EncryptionAlgorithm": {
      "name": "x-ms-encryption-algorithm",
      "x-ms-client-name": "encryptionAlgorithm",
      "type": "string",
      "in": "header",
      "required": false,
      "enum": [
        "AES256"
      ],
      "x-ms-enum": {
        "name": "EncryptionAlgorithmType",
        "modelAsString": false
      },
      "x-ms-parameter-location": "method",
      "x-ms-parameter-grouping": {
        "name": "cpk-info"
      },
      "description": "The algorithm used to produce the encryption key hash. Currently, the only accepted value is \"AES256\". Must be provided if the x-ms-encryption-key header is provided."
    },

Please open an item tracking the generation upgrade for the fix, or add a case covering, otherwise the change will be override by next generation.

authors: EmmaZhu XiaoningLiu jeremymeng


text:
Post release automated changes for communication releases

Post release automated changes for azure-communication-chat

API change check
API changes are not detected in this pull request.


text:
[identity] Merge release for 3.3.1 to upstream

Packages impacted by this PR
@azure/identity
Describe the problem that is addressed by this PR
Added some validations for tenant and scope in this hotfix
Checklists

[x] Added impacted package name to the issue description
[x] Added a changelog (if necessary)

API change check
API changes are not detected in this pull request.

authors: KarishmaGhiya azure-sdk


text:
Post release automated changes for identity releases

Post release automated changes for azure-identity

API change check
API changes are not detected in this pull request.


text:
[Azure Monitor OpenTelemetry] Update Code Samples in README

Packages impacted by this PR
@azure/monitor-opentelemetry
Describe the problem that is addressed by this PR
Code samples should be consistent and be syntactically correct in TypeScript
#28744
Checklists

[x] Added impacted package name to the issue description
[ ] Does this PR needs any fixes in the SDK Generator?** (If so, create an Issue in the Autorest/typescript repository and link it here)
[x] Added a changelog (if necessary)

API change check
API changes are not detected in this pull request.

authors: JacksonWeber azure-sdk


text:
Delete eng/tools/upload-to-storage-blob.ps1

This is a dead script we no longer use so removing.

/check-enforcer override

authors: jeremymeng weshaggard


text:
Update rush and pnpm

cc: @Azure/azure-sdk-eng

https://dev.azure.com/azure-sdk/internal/_build/results?buildId=181975&view=results


text:
[BUG] Unable to use interactive login from VSCode Dev Container

x-post from this stackoverflow question
Describe the bug
When using devcontainers on a project where we are relying on MSIs for app-to-app communication, we are utilizing DefaultAzureCredential as the class to load either from the local developer's machine or, on Azure, the MSI of the running service. In order to get this working, I found we needed to add the includeInteractiveCredentials: true parameter to the constructor. This results in an actual browser window getting popped from when the code is hit (pretty cool, honestly). If we didn't have this parameter set, it wasn't picking up any user creds to access the resources.
The problem we're having is when we load our solution in its devcontainer, the browser login that would normally pop at debug time to pull the local user's credentials isn't allowed to start up. Instead we are met with the following error:

Azure.Identity: DefaultAzureCredential authentication failed. InteractiveBrowserCredential authentication failed. Microsoft.Identity.Client: Unable to open a web page using xdg-open. See inner exception for details. Possible causes for this error are: xdg-open is not installed or it cannot find a way to open an url - make sure you can open a web page by invoking from a terminal: xdg-open https://www.bing.com . System.Diagnostics.Process: No such file or directory.

We then added xdg-utils to our apt-get call in our Dockerfile but now we are met with
/usr/bin/xdg-open: 870: /usr/bin/xdg-open: www-browser: not found
/usr/bin/xdg-open: 870: /usr/bin/xdg-open: links2: not found
/usr/bin/xdg-open: 870: /usr/bin/xdg-open: elinks: not found
/usr/bin/xdg-open: 870: /usr/bin/xdg-open: links: not found
/usr/bin/xdg-open: 870: /usr/bin/xdg-open: lynx: not found
/usr/bin/xdg-open: 870: /usr/bin/xdg-open: w3m: not found
xdg-open: no method available for opening 'https://login.microsoftonline.com/...'

I've gone so far as clicking the link/pasting it in to my host OS browser, logging in, then looking at the port that's attempted for callback (e.g. localhost:44663) and immediately doing a "forward port" from the VSCode command pallet but this also failed saying the state object doesn't match.
Additionally, doing an az login w/in the container doesn't alleviate the issue.
How are we supposed to use local Azure credentials from within a Dev Container? Keep in mind we'd like to avoid having to create a Service Principal in Azure just to get local development working.
Expected behavior
Interactive login could simply use Device Code flow instead of trying to pop a browser and it'd work fine everywhere. Regardless, I expect to be able to use my user credentials to log in to Azure from w/in a VS Code dev container.
Actual behavior (include Exception or Stack Trace)
See above
To Reproduce
Steps to reproduce the behavior (include a code snippet, screenshot, or any additional information that might help us reproduce the issue)
https://github.com/brandonh-msft/functions-storage-managed-identity#how-we-obtain-local-user-and-azure-msi-credentials-within-the-function
Environment:

Name and version of the Library package used

Azure.Identity 1.1.1
Azure.Storage.Blobs 12.4.1

Hosting platform or OS and .NET runtime version

Windows 10, netcore 3.1

IDE and version

VSCode latest

Other information
For those wondering, the error we get without setting interactive=true is

An exception of type 'Azure.Identity.CredentialUnavailableException' occurred in System.Private.CoreLib.dll but was not handled in user code: 'DefaultAzureCredential failed to retrieve a token from the included credentials. EnvironmentCredential authentication unavailable. Environment variables are not fully configured. ManagedIdentityCredential authentication unavailable, no managed identity endpoint found. SharedTokenCacheCredential authentication unavailable. No accounts were found in the cache.'

@brandonh-msft Thanks for filing this issue. I'll need to investigate if there is another way we could launch the browser process in this case.
However, since you mentioned you're able to run az login in the dev container, you could try the latest Azure.Identity preview 1.2.0-preview.2. In this preview we have updated the DefaultAzureCredential so that it can authenticate through the Azure CLI, as well the Azure Account VS Code plugin. I'm not 100% sure the latter will work from a dev container, but if you are running the Azure CLI in the dev container it should work.

I'll give this a try ASAP

No dice. Using the sample I linked here's the error I get w/ the new nupkg:

Azure.Identity: DefaultAzureCredential authentication failed. Azure.Identity: VisualStudioCodeCredential authentication failed. Azure.Identity: Unable to load shared library 'libsecret-1.so.0' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: liblibsecret-1.so.0: cannot open shared object file: No such file or directory.

I then added libsecret-1-0 to my devcontainer's dockerfile and tried again:

Azure.Identity: DefaultAzureCredential authentication failed. Azure.Identity: VisualStudioCodeCredential authentication failed. Azure.Identity: An error was encountered while reading secret from keyring, domain:'140', code:'0', message:'Cannot autolaunch D-Bus without X11 $DISPLAY'.

I pushed this work to a new branch on my sample

It looks like you're running into an issue where the VisualStudioCodeCredential is failing in a way which is failing authentication where it should be raising a CredentialUnavailableException so that the DefaultAzureCredential authentication flow can continue to the AzureCliCredential. @AlexanderSher can we file an issue to raise CredentialUnavailableException in the case that libsecret is not available or the d-bus daemon is not running.
@brandonh-msft you should be able to get passed this issue by disabling the VisualStudioCodeCredential via the DefaultAzureCredentialOptions like so:
var credential = new DefaultAzureCredential(new DefaultAzureCredentialOptions { DisableVisualStudioCodeCredential = true });

@brandohn-msft I don't see anyway we can actually launch a browser in this scenario as the container doesn't have a browser installed, or a UI stack for that matter. It's possible that VS Code could add a different mechanism for authenticating in a Dev Container as it is able to communicate with the container via SSH. However, unfortunately it seems the InteractiveBrowserCredential is not supported from inside a container. If you disagree please feel free to reopen the issue.
Also, please let me know if you're still having trouble getting authentication working via the Azure CLI. I think this is the most viable work around in this case.

@schaabs :

@brandonh-msft you should be able to get passed this issue by disabling the VisualStudioCodeCredential via the DefaultAzureCredentialOptions like so:

Thanks I'll try this in the linked sample and report back.

@brandohn-msft I don't see anyway we can actually launch a browser in this scenario as the container doesn't have a browser installed, or a UI stack for that matter.

Yeah this is why I suggested having it use Device Code flow as its "interactive" implementation instead of trying to pop a browser.

Also, please let me know if you're still having trouble getting authentication working via the Azure CLI. I think this is the most viable work around in this case.

Not sure what you mean by this - I did do az login in the course of testing the new Preview SDK....

by disabling the VisualStudioCodeCredential via the DefaultAzureCredentialOptions like so:
var credential = new DefaultAzureCredential(new DefaultAzureCredentialOptions { DisableVisualStudioCodeCredential = true });

in case anybody else comes across this item, the code is:
new DefaultAzureCredential(new DefaultAzureCredentialOptions { ExcludeVisualStudioCodeCredential = true })

@brandonh-msft

Yeah this is why I suggested having it use Device Code flow as its "interactive" implementation instead of trying to pop a browser.

This is an interesting suggestion that I missed this on the first read-through of this issue.
The only problem I forsee with this is that currently the DeviceCodeCredential doesn't do any interaction on it's own, and relies on the developer to provide a callback which will display the device code to the user. It's possible that if we were to add this to the end of the DefaultAzureCredential authentication flow, we could provide a default implementation of this callback that displays the device code to the console. My concern with this would be if the app didn't have a console which is visible to the user, the application would appear to hang until the timeout of the device code is reached.
Another possibility is that we could allow the user to specify this callback as an option when creating the DefaultAzureCredential, and only fallback to the DeviceCodeCredential if this callback has been specified. This unfortunately makes this behavior not exactly default. What are your thoughts on this?

Also, please let me know if you're still having trouble getting authentication working via the Azure CLI. I think this is the most viable work around in this case.
Not sure what you mean by this - I did do az login in the course of testing the new Preview SDK....

I meant let me know if authentication through the Azure CLI is still not working once you disable the VisualStudioCodeCredential to work around the issue you hit with libsecret not running inside the container. Thank you for adding the correct code snippet.

Good news! I added the ExcludeVisualStudioCodeCredential = true option to the DefaultAzureCredential ctor and it worked perfectly. I event hit a resource in a different tenant than my default one without issue. So it seems adding that exception will solve the issue we're having and fall back to the creds pulled by az login - should we keep this issue open to cover that work?
Regarding the option for Device Code, while I'm not an expert, it seems that the current way of popping a browser to auth also has some sort of callback, does it not? Would it not be similar to this?
If port mapping, etc. are of concern we have the ability to forward ports from the host to the devcontainer using the forwardPorts specification of devcontainer.json (in fact we do this to enable debugging of Azure Functions while w/in the dev container).

authors: brandonh-msft schaabs

repo: Azure/azure-sdk-for-net


text:
Query/Question

version of the Library package used: [e.g. Azure.Storage.Blobs 12.2.0]

Hosting platform or OS and .NET runtime version (dotnet --info output for .NET Core projects): [e.g. Azure AppService or Windows 10 .NET Framework 4.8]
sekrts IDE and version : [e.g. Visual Studioin https://github.com/Azure/azure-sdk-for-net/issues/14957__

Closing, as this appears to be a mistaken submission.  Please feel free to reopen if you believe that further discussion is needed.


text:
[BUG]Microsoft.Azure.WebJobs.Extensions.EventHubs 5.0.0-beta.5 will make unit test start failed.

Describe the bug
If I add Microsoft.Azure.WebJobs.Extensions.EventHubs 5.0.0-beta.5 to project, I can't run unit test for my project.
Expected behavior
I can run unit test success.
Actual behavior (include Exception or Stack Trace)
I can't run the unit test, and in test outputs, it said:

To Reproduce
Steps to reproduce the behavior (include a code snippet, screenshot, or any additional information that might help us reproduce the issue)

add this to a project
create unit test for that project.

Environment:

Microsoft.Azure.WebJobs.Extensions.EventHubs 5.0.0-beta.5
Windows 10 .NETCore 3.1
Visual Studio 16.10

Thank you for your feedback.  Tagging and routing to the team members best able to assist.

Hi @yunair,
Can you include more information on what specific unit tests you are trying to run?

I can't repro this in demo, so close this.

authors: JoshLove-msft jsquire yunair


text:
ManagedIdentityCredential has no token caching support

Query/Question
In the documentation about App Authentication (https://docs.microsoft.com/en-us/dotnet/api/overview/azure/service-to-service-authentication) it is mentioned that for any new application we should be using the new Azure.Identity library for authentication. Yet the new library doesn't cache tokens when requested using GetTokenAsync, while the "legacy" library does.
First of all, the lack of caching is not actually mentioned anywhere in this update article. We have found this out the hard way after migration. Should probably mention it?
But more importantly, what's the plan going forward? Is it expected that from now on we should be taking care of token caching ourselves? Or will internal cache be added to Azure.Identity at some point in the future?

Thank you for your feedback.  Tagging and routing to the team members best able to assist.

Hi @ncksol - The reference documentation for GetToken does mention this (example here.
In practice though, some credentials do cache because of the underlying implementation provided by MSAL. We also allow for some configuration of token cache for scenarios where you need to share cache across credential instances. See this sample for more information.
Which credential are you using where caching didn't seem to be available?

Hi @christothes. That example is not very relevant to my case. I am trying to connect to azure sql db using connection string that doesn't have credentials. For that purpose I am doing:
var tokenRequestContext = new TokenRequestContext(AzureSqlScopes);
var token = new DefaultAzureCredential().GetToken(tokenRequestContext);
sqlConnection.AccessToken = token.Token;

Previously we were using legacy library:
var tokenProvider = new AzureServiceTokenProvider();
var accessToken = await tokenProvider.GetAccessTokenAsync(ResourceId);
sqlConnection.AccessToken = accessToken .Token;

In the legacy scenario we've observed that token is being cached internally, while using new Azure.Identity approach we are spamming Azure with token requests.

Sounds like you are using the Managed Identity credential. Since this is not a credential that is implemented with MSAL, there is no caching for this currently. It's possible that MSAL will implement some changes in the future that will allow us to use it for caching, but we have no plans to duplicate the caching logic implemented by the MSAL library.

@christothes so would you recommend us to continue using AzureServiceTokenProvider for the time being? Since it does seem to have the caching. We got thrown off by that documentation, I've linked, that recommended to use Azure.Identity.

If built-in token caching for Managed Identity authorization is the most important feature for your scenario then yes.

Related https://github.com/Azure/azure-sdk-for-net/issues/24046

duplicate of #25361

authors: christothes jsquire ncksol pakrym


text:
[BUG] ObjectReplicationSourceProperties is null

Library name and version
Azure.Storage.Blobs v12.10.0
Describe the bug
We are testing the Object Replication feature for blobs. Based on this link  we should be able to check the replication status of the blobs through the ObjectReplicationSourceProperties. However, the property is coming as null for all blobs. As a result, blob.ObjectReplicationSourceProperties[0].Rules[0].ReplicationStatus does not work and throws null reference exception. We are testing this on an account where the replication has already taken place and the replicated blobs can be seen in the destination account. However, the ObjectReplicationSourceProperties is coming as null for blobs in both source and destination accounts. In the Azure portal, the replication properties are visible.
Expected behavior
ObjectReplicationSourceProperties[0].Rules[0].ReplicationStatus should return complete or failure for blobs in storage accounts where replication policy has been applied.
Actual behavior
ObjectReplicationSourceProperties is coming as null.
Reproduction Steps

Followed the steps in this page for the portal to create the replication policies: https://docs.microsoft.com/en-us/azure/storage/blobs/object-replication-configure?tabs=portal

Used the code below to list and iterate through the blobs. Replace the variables in <> for the account, container, access key
`
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Azure;
using Azure.Storage;
using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;

namespace TestBlobReplication
{
class Program
{
//Azure storage account information
private static readonly string storageAccountName = <STORAGE ACCOUNT NAME>;
private static readonly string storageContainerName = <CONTAINER NAME>;
private static readonly string AzureStoragePrimaryKey = <PRIMARY ACCESS KEY>;
private static readonly string AzureStorageConnectionString = $"DefaultEndpointsProtocol=https;AccountName={storageAccountName};AccountKey={AzureStoragePrimaryKey};EndpointSuffix=core.windows.net";
    private static BlobContainerClient blobContainerClient;

    public async Task<BlobContainerClient> GetBlobContainerReferenceAsync(string containerName)
    {
        var blobContainerClient = new BlobContainerClient(AzureStorageConnectionString, containerName);
        bool exists = await blobContainerClient.ExistsAsync();

        if (exists)
        {
            Console.WriteLine($"{DateTime.Now} \n2. Found Container \n{containerName}\n");
        }
        else
        {
            await blobContainerClient.CreateAsync();
            Console.WriteLine($"{DateTime.Now} Blob container: {containerName} inside storage account: {blobContainerClient.AccountName} does not exist, container created");
            //throw new InvalidOperationException("Storage Account/Container not found");
        }

        return blobContainerClient;
    }

    public async Task<IList<BlobItem>> ListBlobMetadataAsync(
        string prefix,
        string continuationToken,
        int maxResultCount = 5000)
    {
        // TODO: make this.container auto initialize
        IAsyncEnumerable<Page<BlobItem>> blobPages = blobContainerClient.GetBlobsAsync(
            BlobTraits.Metadata,
            BlobStates.None,
            prefix)
            .AsPages(continuationToken, maxResultCount);

        IAsyncEnumerator<Page<BlobItem>> enumerator = blobPages.GetAsyncEnumerator();
        IList<BlobItem> resultSegment = new List<BlobItem>();
        string newContinuationToken = null;

        try
        {
            while (await enumerator.MoveNextAsync())
            {
                Page<BlobItem> blobPage = enumerator.Current;
                foreach (BlobItem blobItem in blobPage.Values)
                {
                    resultSegment.Add(blobItem);
                }
                newContinuationToken = blobPage.ContinuationToken;
            }
        }
        finally
        {
            await enumerator.DisposeAsync();
        }

        return resultSegment;
    }

    static async Task Main(string[] args)
    {
        Console.WriteLine($"{DateTime.Now} Beginning operations...");
        Program p = new Program();

        blobContainerClient = await p.GetBlobContainerReferenceAsync(storageContainerName);
        //BlobClient blobClient = blobContainerClient.GetBlobClient(blobId);
        IList<BlobItem> blobs = await p.ListBlobMetadataAsync(null, null);

        foreach (var blob in blobs)
        {
            var metadata = blob.Metadata;
            var createdTime = blob.Properties.CreatedOn;
            var replicated = blob.ObjectReplicationSourceProperties[0].Rules[0].ReplicationStatus;
        }
    }
}

}
`
Environment
.NET CORE 3.1
Visual Studio 2019

@asgoe I see that you are using GetBlobsAsync in your code.   You may need to use DownloadAsync() or DownloadContentAsync() to get the ObjectReplicationSourceProperties.  (See below)

Thanks
Saurabh

Thanks @SaurabhSharma-MSFT, we are trying to find a way to validate the replication for all the blobs in a storage account for data move. Are recommendation on what would be the best way to do so? As of now it seems that we will have to list all the blobs, filter out based on last modified time and then download and check the replication status. This can be a slow process if our account has a large number of blobs, especially considering that we will be taking a customer downtime when we perform this validation.

@SaurabhSharma-MSFT I am still getting the ObjectReplicationSourceProperties as null after calling the download operation on the blob
See below, lines 93-95 is the code that I added to download the blobs that are received as part of the list operation:

This is the screenshot of the blob's properties from the portal:

@SaurabhSharma-MSFT any feedback on ObjectReplicationSourceProperties still coming as null?

@asgoe

I agree with you that you don't need to download the blob every time to check for this property and this should be returned as part of GetBlobsAsync.  I am redirecting this issue to service team to look into this and guide with your Null issue with DownloadAsync method. Thank you.

Can I get some feedback on this? This is blocking our design.

@jsquire, @xgithubtriage, its been more than 2 weeks without any action on this. Can this please be assigned to the correct team? This is a blocker for us.

Hi @asgoe.   I'm sorry to hear that there hasn't been further engagement.  Unfortunately, this has already been triaged to the correct team and I have no insight into its current state.    @amishra:  Is there any insight that you're able to provide?

Tagged the wrong person @jsquire Jesse :)
@asgoe, lets sort it out over email and we can put the summary here. I do apologize for the delay

Tagged the wrong person @jsquire Jesse :)

Apologies; that's my mistake.

@SaurabhSharma-MSFT I am still getting the ObjectReplicationSourceProperties as null after calling the download operation on the blob See below, lines 93-95 is the code that I added to download the blobs that are received as part of the list operation:
This is the screenshot of the blob's properties from the portal:

Also regarding this issue, it's expected behavior that the service won't return back a ObjectReplicationSourceProperties in BlobDownloadInfo when you go to download the destination blob. If you go to download the source blob, the BlobDownloadInfo.ObjectReplicationSourceProperties will be populated, and the BlobDownloadInfo.ObjectReplicationDestinationPolicyId will be null.

Fixed in 12.11 (https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/storage/Azure.Storage.Blobs/CHANGELOG.md#12110-beta1-2021-11-03)

authors: SaurabhSharma-MSFT amishra-dev amnguye asgoe jsquire


text:
[FEATURE REQ] Bring back the TableEntityAdapter for Azure.Data.Tables

Library name
Azure.Data.Tables
Please describe the feature.
In the "old" SDKs such as Microsoft.WindowsAzure.Storage.Table, there was a class called TableEntityAdapter which implemented the ITableEntity interface and allowed to write and read any POCO (given some restrictions on property types are met) without having the POCO implement ITableEntity itself. See https://github.com/Azure/azure-storage-net/blob/07d66a4917ab03cb5e7262774235d2d2f7e65752/Lib/Common/Table/TableEntityAdapter.cs
As of now, I don't see any replacement for this in the new Azure.Data.Tables SDK.
The background for this request is that I'd like to define my storage entities without requiring them to have a dependency on the Azure.Data.Tables SDK.

Thank you for your feedback.  Tagging and routing to the team member best able to assist.

I would like to see this done in a way that continues to allow for the independent use of the "Flatten" and "ConvertBack" functionality.  For my use, I find it simpler to use the EntityPropertyConverter functionality without use of the TableEntityAdapter.

Any update on this? At the moment you still have to implement ITableEntity in your class...

We're considering this feature for next semester.

You can use Flatten and ConvertBack methods directly via this nuget https://www.nuget.org/packages/ObjectFlattenerRecomposer.Core
This updated version also supports collection type properties and othe unsopported types in the azure sdk.

authors: ChristianEder LockTar christothes doguvolkanarslan ebussa jsquire