text:
Add WAM support in Azure CLI

Is your feature request related to a problem? Please describe.
A near-term future release of MSAL Python will enable broker-based authentication through Web Account Manager (WAM) on Windows devices. The benefits of authentication through WAM are described in this document under the "WAM value proposition" heading.
Describe the solution you'd like
Upgrade MSAL Python (upon release of the latest version including WAM support) and enable authentication through WAM.
Additional context
@rayluo is currently driving the work to enable WAM support within MSAL Python and the functionality will be ready for testing by mid-February. Ahead of release we would like the Azure CLI team to run tests against WAM functionality / identify any bugs in a feature branch @rayluo will provide.

WAM support

I think there is a lot more than bumping MSAL:

How will az account clear and az logout work? Currently az account clear deletes ~/.azure/msal_token_cache.bin, if MSAL uses WAM, how can accounts be deleted?
Will Azure CLI and Azure PowerShell share the same login context? What if Azure CLI deletes an account that is currently used by PowerShell, will Azure PowerShell be affected?
Will WAM slow down the execution speed of MSAL and Azure CLI?
Will MSAL HTTP cache still work?
Will WAM work with getting VM SSH certificate through data?

I think there is a lot more than bumping MSAL:

Indeed. While the integration is currently designed to be as simple as bumping MSAL, the implication is serious.

How will az account clear and az logout work? Currently az account clear deletes ~/.azure/msal_token_cache.bin, if MSAL uses WAM, how can accounts be deleted?

There will be a new broker (WAM) layer operates beneath MSAL Python. Each layer has its own cache mechanism.
Based on your description, it sounds like Azure CLI simply deletes MSAL's token file without using MSAL's remove_account() API. Either way, it has been good enough to reset Azure CLI into an not-yet-signed-in state, although the subsequent az login sign-in might not actually prompt user to enter their password again (is that true?), because the browser still holds a valid signed-in state. After the broker integration, the az login experience would probably remain the same, it is just that the valid signed-in state is kept in broker (WAM), not in browser.
We are also looking into how MSAL's remove_account() will also wire up with broker's corresponding mechanism.

Will Azure CLI and Azure PowerShell share the same login context? What if Azure CLI deletes an account that is currently used by Azure PowerShell, will Azure PowerShell be affected?

When/if MSAL's remove_account() removes the account in WAM, all apps using broker/wam mechanism will share the same signed state, so one app can affect the rest.

3. Will WAM slow down the execution speed of MSAL and Azure CLI?

The acquire_token_silent() call feels still fast enough. The acquire_token_interactive() seems tangibly slower. MSAL does not really have a control on this part. Please test it and report back.

4. Will MSAL HTTP cache still work with WAM?

Not for broker/wam. But broker/wam is only for public clients, AzureCLI and MSAL Python still need to support Confidential Clients i.e. Service Principals. And, currently broker only supports Windows, but the vast majority of MSAL downloads are from Linux platform (is it also true for Azure CLI?). So, our effort on http_cache will not be wasted.

5. Will WAM work with getting VM SSH certificate through `data`?

We were not able to test this for you. You would need to follow our prerequisite No.1 to setup the proper redirect_uri, and then ping me, I'll test it for you.

6. Is it possible to turn off WAM integration and use the current token cache file?

The MSAL layer operates on top of WAM layer. MSAL will already look for an access token (AT) from MSAL's own token cache first. If it finds an AT, broker/WAM will not kick in.
Currently, other than "prerequisite no. 2", MSAL Python does not provide a more granular way to turn on/off broker. Please conduct your integration tests and see whether we will run into any scenario that a more granular control is desirable.

7. What if the system doesn't support WAM, like old versions of Windows, Windows Server?

MSAL Python will automatically fall back to the non-broker behavior.

authors: jiasli medhir rayluo yonzhan


text:
Add support for 'tags' parameter in 'az postgres flexible-server replica create'

Related command

az postgres flexible-server replica create
Is your feature request related to a problem? Please describe.

Currently, when using the az cli command, the postgres flexible server replica is created inheriting the primary server tags, however ideally, it should be possible to create the replica with its own tags.
Describe the solution you'd like

Add support for the 'tags' parameter as existing in many other commands
Describe alternatives you've considered

Add tags from the portal
Additional context

Thank you for opening this issue, we will look into it.

Adding Service team to look into this

@pwakano Thanks for reporting this. We've triaged this issue and supporting this additional parameter in the CLI command wouldn't be sufificient. Even if we send the tags in the create replica payload, our control plane is currently overriding whatever list of values you pass with a copy of the tags from the primary server.
We have also created a work item for control plane owners to triage and determine if changing current behavior wouldn't break any other functionality. If that is the case, once the control plane behavior is changed, it would make sense to surface this --tags parameter.
Meanwhile, instead of changing those tags interactively via the UI of Azure portal, you can pipeline your az postgres flexible-server replica create with the az tag commands as explained in https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/tag-resources-cli.

source: gharchive/issue

authors: nachoalonsoportillo navba-MSFT pwakano yonzhan

repo: Azure/azure-cli

license: MIT

license_type: permissive

license_source: github-api


text:
az Command does not output anything and returns errorcode 1

Describe the bug
The az command suddenly stopped working.
I get no console output whatsoever, only errorcode 1.
I tried uninstalling and re-installing the current version 2.53.1, but this did not help.
Related command
az
Errors
The command does not output anything. Error code on the console is 1.
Issue script & Debug output
Can't provide, I don't get any output at all.
Expected behavior
If az has a problem, it should at least output an error message.
Environment Summary
Can't provide, I don't get any output at all.
Additional context
No response

Thank you for opening this issue, we will look into it.

It could be possible that the az command is broken.

Where are you running Azure CLI, Windows, Linux or MacOS?
Which shell are you using?
Could you share a screenshot of what you are seeing?

I had the same issue.
I reinstalled Azure CLI, tried different PowerShell consoles, and updated PowerShell to the latest version.
However, the problem persisted.

There was nothing related in the EventLog.
OS: Windows 11 with the latest updates.
The issue, as it turns out, was with my system (cmd.exe). After some Windows updates, it stopped working as it should have (CMD exited immediately after running).
I fixed it by executing the following command:
%SystemRoot%\System32\reg.exe DELETE "HKCU\Software\Microsoft\Command Processor" /v AutoRun /f

authors: gingters jiasli stanislav-fedorenko yonzhan


text:
Add cloud-shell and interactive-mode to Feedback

[ ] Move az feedback telemetry store to new AI account
[ ] Add custom dimension fields for "IsCloudShell", "IsInteractiveMode", "DefaultOutput", "HashedMacAddress" and "InstallationId"
[ ] Change az feedback to accept an "empty" NPS value, and if provided, error out with message "If you change your mind and want to provide us feedback, we'll still be here!"
[ ] When a customer provides feedback and gives an email address, can we save that in the profile and prepopulate it?  Or maybe default to their az account show login?
In addition, I think we should move to a three question model (ask pro and con) as it seems lots of folks are using 0 just to exit the command which adds lots of noise to our numbers.
We appreciate your feedback!  This survey is only three questions and should take less than a minute"
"How likely is it you would recommend our Azure CLI to a friend or colleague? [0 to 10]:
"What can we improve on?"
"What do we do really well?"
"Do you want to provide an email we could follow up with?"

we can live w/o it

authors: JasonRShaver yugangw-msft


text:
num-results option for az storage blob list

Hello,
My blob container is expected to have more than 100,000+ blob files. I would like to use az storage blob list to recursively get a list of all blobs in the container. If I do not specify --num-results would I receive the entire list of blobs with my request?
Thanks very much for your help
Chanaka

Environment summary
Install Method (e.g. pip, interactive script, apt-get, Docker, MSI, edge build) / CLI version (az --version) / OS version / Shell Type (e.g. bash, cmd.exe, Bash on Windows)

Related to #3481.

Hello TJPREScott,

Is this different from #5474?

Hello TJPREScott,
Yes, az storage blob list does seem to work recursively by listing filenames but the documentation is confusing on --num-results option. I would like to get all the results without paging. My container is expected to have over 100,000 + blob files. Will this work if I don't include --num-results option?
Thanks for your help

@troydai

@chanaka-jostle yes, az storage blob list will by default list all the blobs in a container (note this might take some time depending on how many blobs you have), we page for all the blobs for you before returning everything.
Including the --num-results parameter will limit the result to whatever you set, with a maximum of 5000 items.

Thanks for the reply!

authors: chanaka-jostle tjprescott williexu


text:
some VM sizes got by az vm list-skus --location <locationname> are not available on the Azure portal

For example:  Standard_H16r in West Europe, from the command output, it's Available:
{
"additionalProperties": {},
"apiVersions": null,
"capabilities": [
{
"additionalProperties": {},
"name": "LowPriorityCapable",
"value": "True"
}
],
"capacity": null,
"costs": null,
"family": null,
"kind": null,
"locationInfo": [
{
"additionalProperties": {},
"location": "westeurope",
"zones": [
"2",
"1"
]
}
],
"locations": [
"westeurope"
],
"name": "Standard_H16r",
"resourceType": "virtualMachines",
"restrictions": [],
"size": "H16r",
"tier": "Standard"
}
But in Azure portal , the size is in 'Not Available' list.

The output vm list-sizes is not accurate, as it does not take into account any restrictions placed on your subscription by CAS layer due to capacity or offer type. Hence please use az vm list-skus which is also what Portal does.
I am closing this issue, as there is nothing much client tools can do

Hi yugangw-msft,  the output is from az vm list-skus,  not az vm list-sizes，please check!

@leifei87 , are you able to create the vm with this size through cli?

No, please see the following errors, actually the core in the location is very enough, I think the mean of the error is the core is exceeding for Standard H Family vCPUs:
x-ms-routing-request-id: WESTEUROPE:20180403T045948Z:fc6a4cbd-2cb8-4184-a9c5-c430885384ba
Error message: {
"error": {
"code": "OperationNotAllowed",
"message": "Operation results in exceeding quota limits of Core. Maximum allowed: 8, Current in use: 0, Additional requested: 16. Please read more about quota increase at http://aka.ms/corequotaincrease."
}
}

This error is expected. I believe there is a separated quota for H family.  I guess portal might cross check the quota and hide images which could be over the limit?
So far, I don't think CLI has any bugs here.  If you do need to create such VMs, you will have to contact customer service to upper the quota.

I see ,thanks


text:
[App Service] Fix #14701: implement functionapp create --assign-identity

Description

fix issue: https://github.com/Azure/azure-cli/issues/14701
This PR implements  az functionapp create --assign-identity
No tests added, since identity assigning of functionapp and webapp is calling exactly the same implementation.
Testing Guide

History Notes

[Component Name 1] BREAKING CHANGE: az command a: Make some customer-facing breaking change.
[Component Name 2] az command b: Add some customer-facing feature.

This checklist is used to make sure that common guidelines for a pull request are followed.

[ ] The PR title and description has followed the guideline in Submitting Pull Requests.

[ ] I adhere to the Command Guidelines.

functionapp

@cxznmhdcxz @tzhanl - I looked at the code, but I'm not sure how the --assign-identity parameter will surface on the command without explicitly adding it.  Can you verify that the parameter is there and help is rendered for it when I az functionapp create -h

/azp run

/azp run

I looked at the code, but I'm not sure how the --assign-identity parameter will surface on the command without explicitly adding it. Can you verify that the parameter is there and help is rendered for it when I az functionapp create -h

@jongio I checked, and it's all good. Code about parameters mapping is already updated in the last PR.

ADO pipeline result https://github.com/Azure/azure-cli/pull/14702/checks?check_run_id=991745040 all passed, but not reflected in github. Merge this PR.

source: gharchive/pull-request

authors: cxznmhdcxz jongio qwordy yonzhan yungezz


text:
{Storage} Announce incoming breaking change

Description

We will introduce a braking change in next version and here is to announce the incoming breaking change.
Testing Guide

History Notes

[Component Name 1] BREAKING CHANGE: az command a: Make some customer-facing breaking change.
[Component Name 2] az command b: Add some customer-facing feature.

This checklist is used to make sure that common guidelines for a pull request are followed.

[ ] The PR title and description has followed the guideline in Submitting Pull Requests.

[ ] I adhere to the Command Guidelines.

[ ] I adhere to the Error Handling Guidelines.

Storage

authors: Juliehzl yonzhan


text:
[AMS] Removing deprecated variable 'identifier_uri' from create sp method

Description

Addressing #20217
Testing Guide

History Notes

[Component Name 1] BREAKING CHANGE: az command a: Make some customer-facing breaking change.
[Component Name 2] az command b: Add some customer-facing feature.

This checklist is used to make sure that common guidelines for a pull request are followed.

[ ] The PR title and description has followed the guideline in Submitting Pull Requests.

[ ] I adhere to the Command Guidelines.

[ ] I adhere to the Error Handling Guidelines.

AMS

/azp run

authors: hivyas jiasli yonzhan


text:
{Core} Make the recommendation for SSLError more accurate

Description
Not all SSLErrors are caused by untrusted self-signed certificate. For example, when a firewall blocks https://management.azure.com/, an requests.exceptions.SSLError will be raised:
$ PYTHONPATH="/lib64/az/lib/python3.9/site-packages" python3.9 -c "import requests; print(requests.get('https://management.azure.com/').status_code)"
...
requests.exceptions.SSLError: HTTPSConnectionPool(host='management.azure.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1129)')))

This PR refines the recommendation message to make it more accurate.
Testing Guide
TBD

Core

authors: jiasli yonzhan


text:
Direct users to github when unexpected error is raised.

Closes #8644

This checklist is used to make sure that common guidelines for a pull request are followed.

[ ] The PR has modified HISTORY.rst describing any customer-facing, functional changes. Note that this does not include changes only to help content. (see Modifying change log).

[ ] I adhere to the Command Guidelines.

Text looks like this:

Warning now looks like this:

does this look good with everyone?


text:
FeedIterator ReadNextAsync is getting ServiceUnavailable (503); Substatus: 20001 after upgrading cosmos sdk nuget

We are continuously addressing and improving the SDK, if possible, make sure the problem persist in the latest SDK version.

Describe the bug
After updating .net sdk from 3.31.1 to 3.35.4 we started getting ServiceUnavailable (503); Substatus: 20001 on FeedIterator ReadNextAsync
To Reproduce
It's happening randomly only for calls from server on azure east us calling to cosmos account on south east asia
Expected behavior
Not throwing errors and retrieving next results page
Actual behavior
throws  ServiceUnavailable (503); Substatus: 20001
ActivityId: 936c91da-9f89-4a18-96de-740cd00ee62b; Reason: (The request failed because the client was unable to establish connections to 4 endpoints across 1 regions. Please check for client resource starvation issues and verify connectivity between client and server
Environment summary
SDK Version: 3.35.4
OS Version (e.g. Windows, Linux, MacOSX) Windows .net 6
Additional context
verify SNAT ports
looking at diagnostics we see no thread starvation and no high CPU
using direct mode and TokenCredential for authenticate
the cosmos account has only one region defined
DirectCalls":["(410, 20001)":5 seems we get status Gone (410)

@oshvartz Can you please share the full exception/diagnostics?

some of it PII so I will mask it
Response status code does not indicate success: ServiceUnavailable (503); Substatus: 20001; ActivityId: 936c91da-9f89-4a18-96de-740cd00ee62b; Reason: (The request failed because the client was unable to establish connections to 4 endpoints across 1 regions. Please check for client resource starvation issues and verify connectivity between client and server. More info: https://aka.ms/cosmosdb-tsg-service-unavailable\r\nActivityId: 936c91da-9f89-4a18-96de-740cd00ee62b, Microsoft.Azure.Cosmos.Tracing.TraceData.ClientSideRequestStatisticsTraceDatum, Windows/10.0.20348 cosmos-netstandard-sdk/3.32.0);; Diagnostics:["Summary":["DirectCalls":["(410, 20001)":5],"GatewayCalls":["(200, 0)":2]],"name":"Typed FeedIterator ReadNextAsync","start datetime":"2024-03-04T12:11:26.133Z","duration in milliseconds":34776.1538,"data":["Client Configuration":["Client Created Time Utc":"2024-03-04T10:42:16.0517487Z","MachineId":"vmId:34250944-e978-46b5-92c0-40c70200594f","VM Region":"eastus","NumberOfClientsCreated":20,"NumberOfActiveClients":20,"ConnectionMode":"Direct","User Agent":"cosmos-netstandard-sdk/3.38.1|10|X64|Microsoft Windows 10.0.20348|.NET 6.0.2|N|","ConnectionConfig":["gw":"(cps:50, urto:6, p:False, httpf: False)","rntbd":"(cto: 5, icto: -1, mrpc: 30, mcpe: 65535, erd: True, pr: ReuseUnicastPort)","other":"(ed:False, be:False)"],"ConsistencyConfig":"(consistency: NotSet, prgns:[], apprgn: )","ProcessorCount":16],"Query Correlated ActivityId":"fb63afa0-c3b2-41c4-9050-49e1242dc503"],"children":[["name":"Create Query Pipeline","duration in milliseconds":0.0985,"children":[["name":"Get Container Properties","duration in milliseconds":0.0157,"children":[["name":"Get Collection Cache","duration in milliseconds":0.0013]]]]],["name":"Get Partition Key Ranges","duration in milliseconds":0.0358,"children":[["name":"Try Get Overlapping Ranges","duration in milliseconds":0.021]]],["name":"[,FF) move next","duration in milliseconds":34775.6354,"children":[["name":"Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler","duration in milliseconds":34775.2689,"children":[["name":"Microsoft.Azure.Cosmos.Handlers.DiagnosticsHandler","duration in milliseconds":34775.2349,"data":["System Info":["systemHistory":[["dateUtc":"2024-03-04T12:11:02.0148100Z","cpu":16.888,"memory":27545132.000,"threadInfo":["isThreadStarving":"False","threadWaitIntervalInMs":0.0554,"availableThreads":32766,"minThreads":16,"maxThreads":32767],"numberOfOpenTcpConnection":294],["dateUtc":"2024-03-04T12:11:12.0305392Z","cpu":15.610,"memory":27786328.000,"threadInfo":["isThreadStarving":"False","threadWaitIntervalInMs":0.0536,"availableThreads":32762,"minThreads":16,"maxThreads":32767],"numberOfOpenTcpConnection":294],["dateUtc":"2024-03-04T12:11:22.0461804Z","cpu":17.853,"memory":27739792.000,"threadInfo":["isThreadStarving":"False","threadWaitIntervalInMs":0.0249,"availableThreads":32766,"minThreads":16,"maxThreads":32767],"numberOfOpenTcpConnection":294],["dateUtc":"2024-03-04T12:11:32.0617628Z","cpu":22.943,"memory":27695196.000,"threadInfo":["isThreadStarving":"False","threadWaitIntervalInMs":0.0501,"availableThreads":32766,"minThreads":16,"maxThreads":32767],"numberOfOpenTcpConnection":294],["dateUtc":"2024-03-04T12:11:42.0770009Z","cpu":21.402,"memory":27630724.000,"threadInfo":["isThreadStarving":"False","threadWaitIntervalInMs":0.2541,"availableThreads":32765,"minThreads":16,"maxThreads":32767],"numberOfOpenTcpConnection":294],["dateUtc":"2024-03-04T12:11:52.0917364Z","cpu":26.024,"memory":27632252.000,"threadInfo":["isThreadStarving":"False","threadWaitIntervalInMs":0.0977,"availableThreads":32766,"minThreads":16,"maxThreads":32767],"numberOfOpenTcpConnection":294]]]],"children":[["name":"Microsoft.Azure.Cosmos.Handlers.TelemetryHandler","duration in milliseconds":34775.228,"children":[["name":"Microsoft.Azure.Cosmos.Handlers.RetryHandler","duration in milliseconds":34775.2217,"children":[["name":"Microsoft.Azure.Cosmos.Handlers.RouterHandler","duration in milliseconds":34775.1692,"children":[["name":"Microsoft.Azure.Cosmos.Handlers.TransportHandler","duration in milliseconds":34775.1656,"children":[["name":"Microsoft.Azure.Documents.ServerStoreModel Transport Request","duration in milliseconds":34774.248,"data":["Client Side Request Stats":["Id":"AggregatedClientSideRequestStatistics","ContactedReplicas":[],"RegionsContacted":[],"FailedReplicas":["rntbd://****8:14017/apps/75084cf5-ad02-4e67-94ce-893215572b76/services/1a857c93-bffc-46d2-ad92-dc77354b8dd8/partitions/ce520ef0-bbaf-42d9-b2a5-ac15bce6ba96/replicas/133490440942662799p","rntbd://cdb-ms-prod-southeastasia1-fd13-az03.documents.azure.com:14050/apps/75084cf5-ad02-4e67-94ce-893215572b76/services/1a857c93-bffc-46d2-ad92-dc77354b8dd8/partitions/ce520ef0-bbaf-42d9-b2a5-ac15bce6ba96/replicas/133525767419518240s","rntbd://cdb-ms-prod-southeastasia1-fd13-az01.documents.azure.com:14332/apps/75084cf5-ad02-4e67-94ce-893215572b76/services/1a857c93-bffc-46d2-ad92-dc77354b8dd8/partitions/ce520ef0-bbaf-42d9-b2a5-ac15bce6ba96/replicas/133525446851132963s","rntbd://cdb-ms-prod-southeastasia1-fd13-az02.documents.azure.com:14049/apps/75084cf5-ad02-4e67-94ce-893215572b76/services/1a857c93-bffc-46d2-ad92-dc77354b8dd8/partitions/ce520ef0-bbaf-42d9-b2a5-ac15bce6ba96/replicas/133525466106987757s"],"ForceAddressRefresh":[["No change to cache":["rntbd://cdb-ms-prod-southeastasia1-fd13-az03.documents.azure.com:14017/apps/75084cf5-ad02-4e67-94ce-893215572b76/services/1a857c93-bffc-46d2-ad92-dc77354b8dd8/partitions/ce520ef0-bbaf-42d9-b2a5-ac15bce6ba96/replicas/133490440942662799p","rntbd://cdb-ms-prod-southeastasia1-fd13-az01.documents.azure.com:14332/apps/75084cf5-ad02-4e67-94ce-893215572b76/services/1a857c93-bffc-46d2-ad92-dc77354b8dd8/partitions/ce520ef0-bbaf-42d9-b2a5-ac15bce6ba96/replicas/133525446851132963s","rntbd://cdb-ms-prod-southeastasia1-fd13-az02.documents.azure.com:14049/apps/75084cf5-ad02-4e67-94ce-893215572b76/services/1a857c93-bffc-46d2-ad92-dc77354b8dd8/partitions/ce520ef0-bbaf-42d9-b2a5-ac15bce6ba96/replicas/133525466106987757s","rntbd://cdb-ms-prod-southeastasia1-fd13-az03.documents.azure.com:14050/apps/75084cf5-ad02-4e67-94ce-893215572b76/services/1a857c93-bffc-46d2-ad92-dc77354b8dd8/partitions/ce520ef0-bbaf-42d9-b2a5-ac15bce6ba96/replicas/133525767419518240s"]],["No change to cache":["rntbd://cdb-ms-prod-southeastasia1-fd13-az03.documents.azure.com:14017/apps/75084cf5-ad02-4e67-94ce-893215572b76/services/1a857c93-bffc-46d2-ad92-dc77354b8dd8/partitions/ce520ef0-bbaf-42d9-b2a5-ac15bce6ba96/replicas/133490440942662799p","rntbd://cdb-ms-prod-southeastasia1-fd13-az01.documents.azure.com:14332/apps/75084cf5-ad02-4e67-94ce-893215572b76/services/1a857c93-bffc-46d2-ad92-dc77354b8dd8/partitions/ce520ef0-bbaf-42d9-b2a5-ac15bce6ba96/replicas/133525446851132963s","rntbd://cdb-ms-prod-southeastasia1-fd13-az02.documents.azure.com:14049/apps/75084cf5-ad02-4e67-94ce-893215572b76/services/1a857c93-bffc-46d2-ad92-dc77354b8dd8/partitions/ce520ef0-bbaf-42d9-b2a5-ac15bce6ba96/replicas/133525466106987757s","rntbd://cdb-ms-prod-southeastasia1-fd13-az03.documents.azure.com:14050/apps/75084cf5-ad02-4e67-94ce-893215572b76/services/1a857c93-bffc-46d2-ad92-dc77354b8dd8/partitions/ce520ef0-bbaf-42d9-b2a5-ac15bce6ba96/replicas/133525767419518240s"]]],"AddressResolutionStatistics":[["StartTimeUTC":"2024-03-04T12:11:32.9063699Z","EndTimeUTC":"2024-03-04T12:11:33.1339908Z","TargetEndpoint":"https://geo-asia-asisf-prod-db.documents.azure.com//addresses/?$resolveFor=dbs%2F8iMIAA%3D%3D%2Fcolls%2F8iMIAKYo9Hc%3D%2Fdocs&$filter=protocol eq rntbd&$partitionKeyRangeIds=0"],["StartTimeUTC":"2024-03-04T12:11:53.9059232Z","EndTimeUTC":"2024-03-04T12:11:54.1312003Z","TargetEndpoint":"https://**//addresses/?$resolveFor=dbs%2f8iMIAA%3d%3d%2fcolls%2f8iMIAKYo9Hc%3d%2fdocs&$filter=protocol eq rntbd&$partitionKeyRangeIds=0"]],"StoreResponseStatistics":[["ResponseTimeUTC":"2024-03-04T12:11:32.9060438Z","ResourceType":"Document","OperationType":"Query","RequestSessionToken":"0:15#6621133","LocationEndpoint":"https:///","StoreResult":["ActivityId":"936c91da-9f89-4a18-96de-740cd00ee62b","StatusCode":"Gone","SubStatusCode":"TransportGenerated410","LSN":-1,"PartitionKeyRangeId":null,"GlobalCommittedLSN":-1,"ItemLSN":-1,"UsingLocalLSN":true,"QuorumAckedLSN":-1,"SessionToken":null,"CurrentWriteQuorum":-1,"CurrentReplicaSetSize":-1,"NumberOfReadRegions":-1,"IsValid":false,"StorePhysicalAddress":"rntbd://cdb-ms-prod-southeastasia1-fd13-az03.documents.azure.com:14017/apps/75084cf5-ad02-4e67-94ce-893215572b76/services/1a857c93-bffc-46d2-ad92-dc77354b8dd8/partitions/ce520ef0-bbaf-42d9-b2a5-ac15bce6ba96/replicas/133490440942662799p","RequestCharge":0,"RetryAfterInMs":null,"BELatencyInMs":null,"ReplicaHealthStatuses":["(port: 14017 | status: UnhealthyPending | lkt: 3/4/2024 12:10:31 PM)","(port: 14332 | status: UnhealthyPending | lkt: 3/4/2024 12:10:31 PM)","(port: 14049 | status: UnhealthyPending | lkt: 3/4/2024 12:10:31 PM)","(port: 14050 | status: UnhealthyPending | lkt: 3/4/2024 12:10:31 PM)"],"transportRequestTimeline":["requestTimeline":[["event": "Created", "startTimeUtc": "2024-03-04T12:11:26.1337653Z", "durationInMs": 0.0059],["event": "ChannelAcquisitionStarted", "startTimeUtc": "2024-03-04T12:11:26.1337712Z", "durationInMs": 0.0209],["event": "Pipelined", "startTimeUtc": "2024-03-04T12:11:26.1337921Z", "durationInMs": 0.1109],["event": "Transit Time", "startTimeUtc": "2024-03-04T12:11:26.1339030Z", "durationInMs": 6771.9708],["event": "Failed", "startTimeUtc": "2024-03-04T12:11:32.9058738Z", "durationInMs": 0]],"serviceEndpointStats":["inflightRequests":1,"openConnections":1],"connectionStats":["waitforConnectionInit":"False","callsPendingReceive":0,"lastSendAttempt":"2024-03-04T12:10:42.3068110Z","lastSend":"2024-03-04T12:10:42.3068546Z","lastReceive":"2024-03-04T12:10:42.5291730Z"],"requestSi"

Have you looked into the aka.ms link in the exception? https://aka.ms/cosmosdb-tsg-service-unavailable

What is the volume of these errors per hour?
Are they happening in the same machine or across all machines in the region?
HTTP connectivity does not seem to be affected, are you sure the TCP port range (10000-20000) is fully available? The error means the SDK tried to all 4 endpoints and all the endpoints had timeouts.: cdb-ms-prod-southeastasia1-fd13-az03.documents.azure.com:14017, cdb-ms-prod-southeastasia1-fd13-az01.documents.azure.com:14332, cdb-ms-prod-southeastasia1-fd13-az02.documents.azure.com:14049, cdb-ms-prod-southeastasia1-fd13-az03.documents.azure.com:14050. They all have different ports, this sounds like a networking problem.

"NumberOfClientsCreated":20,"NumberOfActiveClients":20

There are 20 client instances created, is this by design? Is the application working with 20 different Cosmos DB accounts? Reference: https://learn.microsoft.com/azure/cosmos-db/nosql/troubleshoot-dotnet-sdk-request-timeout?tabs=cpu-new#create-multiple-client-instances

server on azure east us calling to cosmos account on south east asia

Network in-between regions goes through unmonitored network infrastructure, it can face network problems, high latency and timeouts. Has this been always the deployment infrastructure?

After updating .net sdk from 3.31.1 to 3.35.4

Is the SDK update the only change that was done? If so, does rolling it back solve the issue? The error is network related, unless there were any other code changes beyond the SDK that might be playing a part (like the increase in client instances).

To clarify, nothing in the exception diagnostics shows the SDK is doing anything wrong. It is facing timeouts, retrying, and attempting to rediscover addresses.

@ealsur
just few answers - we did try following the TSG
we have two application versions - one using sdk 3.31.1 and one upgraded Microsoft.Azure.Cosmos to 3.35.4 - only then we see the issue - rolling back the issue disappears
it's happing in all VMs in the region and only for this account - maybe some configuration\timeout was change it's very reproducible getting ~60-70 errors per hour
can you advise what can we do to overcome this issue

Is the issue only affecting the queries that now use PartitionKeyBuilder? Are other queries working well? Are the queries with PartitionKeyBuilder working sometimes too?
What about the Number of Clients? There are 20, is this expected?
One potential reason is: Queries with Hierarchical Partition Key (PartitionKeyBuilder) can be cross-partition queries. Cross-partition queries will use more connections than a normal query. 20 clients in the machine might be putting some pressure on resources.
Is it possible to attach the rest of the diagnostics?

@ealsur we have in the process ~20 different cosmos accounts - is number of clients are for the same account ?
The issue is for document with only one partition key, and we pass it as part of the queue so not cross partition
Is it possible to attach the rest of the diagnostics? - this was truncated in my logs, so this is what I have too.
Update the same code with 3.31.2-preview didn't reproduced the error trying with 3.33.0 and I'll update

with only one partition key,

Are you using PartitionKeyBuilder setting only 1 value? PartitionKeyBuilder is used for Hierarchical Partition Keys. Hierarchical Partition Keys when the number of values is less than the number of paths defined in the container (for example, the container is created with /country, /state, /city and you use PartitionKeyBuilder just specifying country), then it can be a cross partition query.

["event": "Transit Time", "startTimeUtc": "2024-03-04T12:11:26.1339030Z", "durationInMs": 6771.9708],["event": "Failed", "startTimeUtc": "2024-03-04T12:11:32.9058738Z", "durationInMs": 0]],

This means the SDK sent the request and the response was not returned in 6700 milliseconds, so it timed out.
The VM is in East US
,"VM Region":"eastus"

And the Cosmos DB region is in South east asia
rntbd://cdb-ms-prod-southeastasia1-fd13-az01.documents.azure.com:14332

There isn't much more we can do to analyze this on the SDK end. Your machine is doing a cross-continental network request, and it's timing out. What networks are in the middle inbetween East US and South East Asia are public networks owned by unknown companies.

updating that it may be related #4360 since we were on the edge of dynamic port - I will update

@ealsur
Sorry for the long delay in responding - we had mitigation downgrading to version 3.33.0 so we postponed investigating it. We thought that #4360 (dynamic ports) was related but we were able reproduced it in an isolated environment and saw it was still happening. We did binary search on the SDK version and saw it started from 3.35.0 so we reviewed the change log and saw #3902 - which is part of 3.35.0 the default timeout was reduced from 10 to 6. We increase the timeout back to 10 manually and the issue resolved. Our feedback was that the error indicated on connectivity issue - 20001 and not 20002 (client timeout) so we followed the relevant TSG which talked about ports\threads\high CPU in the client. I think that maybe 20002 sub status should be returned in this case. In addition, I looked for any explanation of why the default time was reduced to 6 seconds? How was this value  selected ?
Thanks

Both 20001 and 20002 point to the same problem, it just happens at different times. 20001 is related to the inability to establish connections, 20002 is related to experiencing timeouts during requests: https://learn.microsoft.com/en-us/azure/cosmos-db/nosql/troubleshoot-service-unavailable
The TSG steps are the same for both, at least I don't see a difference in the document. Both can be caused by the same causes.
The previous timeout was 10 seconds, the new one is 6 seconds, But the truth is that a request should not take that long to complete. Having 7 seconds of latency on requests is not normal. The 6 seconds was changed to be inline with other SDKs (like Java) and because Cosmos DB has a maximum processing time of 5 seconds, assuming worst case scenario where the request took exactly 5 seconds on the service, it still has 1 full second of network time.
Are you normally working with more than 6 seconds of network latency and consider that normal behavior?

authors: ealsur oshvartz


text:
3.9.0-preview2: Fix preview version

Pull Request Template
Description
3.9.0 hasn't been released so the preview version should be 3.9.0-preview2
Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How is 'preview2" suffix generated?

How is 'preview2" suffix generated?

The way it has been done in the past is modifying the csproj to add it. I will add it when I create the release branch. As part of my PR to refactor the csproj I will fix this so it can be passed in.


text:
Documentation: Fixes method descriptions with sql query text vs definition

Description
The description for method overloads that accept a sql query as a string vs a QueryDefinition object are misleading for some
as discussed in this issue.
This PR fixes them.
Type of change
Documentation
Closing issues
resolves Azure/azure-sdk-for-net#25878

/azp run

authors: PramodValavala-MSFT ealsur


text:
Documentation: Expand EUAP in Comments

Description
The abbreviation for Early Updates Access Program (EUAP) is not expanded in the description for those regions. This PR updates the description accordingly.
Type of change

[x] This change requires a documentation update

Closing issues
resolves Azure/azure-sdk-for-net#32456

/azp run


text:
Request: Single-Developer Mode

If possible, I'd like to have a single-developer mode that:

Caches all ARM responses in a machine/user-local file.
Invalidates cache entries based on changes that are applied (or re-reads those ARM responses after deployment changes).

This would vastly speed up single-developer, single-machine iterative deployments.

In the meantime, you could use the --debug flag and pipe to a file.
What are you wanting to do with the data in the file?  Do you need it structured?

Sorry if I wasn't clear.
I have no use for the file. What I want is to be able to change one property on one resource without waiting 10 minutes for my entire infrastructure to be checked for changes.
If there was a "single developer" mode, then azd wouldn't need to check the resources I haven't changed.

#376 Seems like this is relevant here. We've started investigation into supporting provision caching which would avoid that wait you're seeing.

provision caching relied on client-state and assumed that the resource group is only managed through azd via azd provision.

Client-side provision caching is exactly what I was thinking, yes.

Perfect - if it's alright, I will close this issue in favour of that one. We are currently spiking on this and should hopefully have improved UX here soon.
Please feel free to reopen if need be.

authors: StephenCleary jongio savannahostrowski

repo: Azure/azure-dev


text:
az ad sp create-for-rbac example appears to disagree with actual behavior

Hi,
I wanted to check my understanding of how the --name flag is meant to be used for create-for-rbac.
Doc in question Create an Azure service principal with Azure CLI 2.0.
The example begins by looking up the appId for MyDemoWebApp:
az ad app list --display-name MyDemoWebApp
{
    "appId": "a487e0c1-82af-47d9-9a0b-af184eb87646d",
     ...
}

The creates a service principal, passing appId as the name:
az ad sp create-for-rbac --name {appId} --password "{strong password}"
{
  "appId": "a487e0c1-82af-47d9-9a0b-af184eb87646d",
  "displayName": "MyDemoWebApp",
  "name": "http://MyDemoWebApp",
  ...
}

Expected output of create-for-rbac command would be closer to:
{
  "appId": "<NEW_APP_ID>",
  "displayName": "a487e0c1-82af-47d9-9a0b-af184eb87646d",
  "name": "http://a487e0c1-82af-47d9-9a0b-af184eb87646d",
  ...
}

CLI docs and Azure/azure-cli#1698 seem to imply --name is used for display name / app URI.

Should this example be updated?
What is the correct method to associated a service principal and existing AD application (similar to what the example currently appears to be demonstrating)?

Thanks

Hi @c-n-d:
The Service Principal article has been rewritten since this issue was created. If you could go over it and see if the confusion regarding --name has been resolved to your satisfaction, I'd appreciate it, and can close out this issue.
Thanks!
https://docs.microsoft.com/en-us/cli/azure/create-an-azure-service-principal-azure-cli

Closing due to lack of activity.

authors: c-n-d sptramer

repo: Azure/azure-docs-cli-python


text:
Current version Az Powershell RM

How much longer will the current Az Powershell RM be supported in production environment?   When will it be completely stopped the service?  Some of our customers are planning to keep using their current version and concerning if/when it is completely stopped using or any fee will be charged to switch to the NEW PowerShell Az module...

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 65abff75-73f1-3033-d816-e03e0d7fe4ff
Version Independent ID: 524598ca-76aa-c8f5-cde1-d6bd626b757b
Content: Introducing the Azure PowerShell Az module
Content Source: docs-conceptual/azurermps-6.13.0/new-azureps-module-az.md
Service: azure-powershell
GitHub Login: @sptramer
Microsoft Alias: sttramer

@markcowl Please respond with some detailed information about AzureRM lifecycle support.

The biggest complaint I have is that the login method is the same as the CLI which is just horrendous having to copy a login URL and then an access code every time you want to start a session. PowerShell AzureRM had a much better login method because you can just easily type your credentials.

Login shall be fixed on 1/15/2019 per article https://docs.microsoft.com/en-us/powershell/azure/migrate-az-1.0.0?view=azps-1.0.0#temporary-default-device-code-login-instead-of-web-browser-prompt

The new module does not incur any additional fees. The new module was released only to offer better support, more consistent names, and work with PowerShell 6. AzureRM will continue to be supported with bug fixes, but receive no new features. @markcowl can provide information on the support window, and that information will be added to the migration docs.

Keeping the login UX the same as the AzureRm module should have been a required feature for a 1.0.0 release.

authors: Yliche artisticcheese cpdohert sptramer v-samiya

repo: Azure/azure-docs-powershell


text:
Docs.MS Release Updates for azure-sdk-template

Update docs metadata and targeting for release of azure-sdk-template

Docs Build status updates of commit 9f57748:
:clock10: Full build: the average full build time is 252 min(s) 10 sec(s), this is based on the last 872 build(s) for this repository.

Docs Build status updates of commit 9f57748:
:warning: Validation status: warnings

File
Status
Preview URL
Details

docs-ref-services/sdk-template-readme-pre.md
:warning:Warning
View (azure-java-stable) View (azure-java-preview) View (azure-java-legacy)
Details

docs-ref-services/messaging-eventhubs-readme.md
:warning:Warning
View (azure-java-stable) View (azure-java-preview) View (azure-java-legacy)
Details

package.json
:white_check_mark:Succeeded

docs-ref-services/sdk-template-readme-pre.md

Line 56: [Warning-InvalidBookmark] Invalid link: 'Introduction'. The file docs-ref-services/sdk-template-readme-pre.md doesn't contain a bookmark named 'introduction'.
Line 56: [Warning-InvalidBookmark] Invalid link: 'Introduction'. The file docs-ref-services/sdk-template-readme-pre.md doesn't contain a bookmark named 'introduction'.
Line 56: [Warning-InvalidBookmark] Invalid link: 'Introduction'. The file docs-ref-services/sdk-template-readme-pre.md doesn't contain a bookmark named 'introduction'.

docs-ref-services/messaging-eventhubs-readme.md

[Warning-DuplicateUids] Uid(azure.java.sdk.landingpage.services.eventhub.Client) has already been defined in legacy/docs-ref-autogen/overview/EventHubs/Client.yml.
Line 43: [Warning-InvalidBookmark] Invalid link: 'Including the package'. The file docs-ref-services/messaging-eventhubs-readme.md doesn't contain a bookmark named 'including-the-package'.
Line 43: [Warning-InvalidBookmark] Invalid link: 'Including the package'. The file docs-ref-services/messaging-eventhubs-readme.md doesn't contain a bookmark named 'including-the-package'.
Line 43: [Warning-InvalidBookmark] Invalid link: 'Including the package'. The file docs-ref-services/messaging-eventhubs-readme.md doesn't contain a bookmark named 'including-the-package'.
Line 56: [Warning-InvalidBookmark] Invalid link: 'Default SSL library'. The file docs-ref-services/messaging-eventhubs-readme.md doesn't contain a bookmark named 'default-ssl-library'.
Line 56: [Warning-InvalidBookmark] Invalid link: 'Default SSL library'. The file docs-ref-services/messaging-eventhubs-readme.md doesn't contain a bookmark named 'default-ssl-library'.
Line 56: [Warning-InvalidBookmark] Invalid link: 'Default SSL library'. The file docs-ref-services/messaging-eventhubs-readme.md doesn't contain a bookmark named 'default-ssl-library'.

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

repo: Azure/azure-docs-sdk-java


text:
Docs.MS Release Updates for azure-resourcemanager-msi

Update docs metadata and targeting for release of azure-resourcemanager-msi

Docs Build status updates of commit 5dde7d8:
:clock10: Full build: the average full build time is 279 min(s) 9 sec(s), this is based on the last 923 build(s) for this repository.

Docs Build status updates of commit 5dde7d8:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs-ref-services/resourcemanager-msi-readme.md
:white_check_mark:Succeeded
View (azure-java-stable) View (azure-java-preview) View (azure-java-legacy)

package.json
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

merge to https://github.com/Azure/azure-docs-sdk-java/pull/1377