text:
FuntionApp Encountered an (InternalServerError) from host runtime

I’m trying to hit the HTTPTrigger Function App which is built and deployed. It was working well till last Thursday(09/04/2020)
But, from last 2 days, I’m getting host runtime issue.
Also, tried re-deploying the code to Azure to see if that solve the Issue. But, even the re-deployment of the code isn’t working.
Logs FYR while re-deploying the functionApp from VSCode:
Investigative information
Please provide the following:

Timestamp: Logs are provided below
Function App name: MarketingInsightsFuntionApp
Core Tools version: azure-core==1.3.0

Expected behavior
Function is expected to take HTTP Requests and also accept re-deployments of new code
Actual behavior
Currently, the Function app is encountering host run time exceptions not allowing any new HTTP requests
Contents of the requirements.txt file:
azure-functions
gremlinpython==3.4.6
futures
azure-core==1.3.0
azure-cosmos==4.0.0b6
Logs:: (NZST 16/04/2020)
2:04:04 pm marketingInsightsFunctionApp: Syncing triggers...
2:04:12 pm marketingInsightsFunctionApp: Syncing triggers (Attempt 2/6)...
2:04:26 pm marketingInsightsFunctionApp: Syncing triggers (Attempt 3/6)...
2:04:48 pm marketingInsightsFunctionApp: Syncing triggers (Attempt 4/6)...
2:05:30 pm marketingInsightsFunctionApp: Syncing triggers (Attempt 5/6)...
2:06:53 pm marketingInsightsFunctionApp: Syncing triggers (Attempt 6/6)...
2:06:58 pm: Error: Encountered an error (InternalServerError) from host runtime.

Storage Account is restored which has been deleted


text:
_ProcessLoop takes 2000ms when ulTimeoutMilliseconds < 2000 and nothing to process

I'm noticing a strange pattern where when there's nothing for _ProcessLoop to process, it takes exactly 2000ms to run.
Quick overview of how I have things set up. Main loop task does the work to queue messages, then there's an azure task that runs constantly and does nothing but maintain Azure connectivity and constantly run _ProcessLoop when connected.
Here's the code in the azure task:

PROCESS_LOOP_TIMEOUT_MS is 500, but even when set to 0 the behavior is the same. Here's the log of a typical run when there's nothing to do. Note the difference is EXACTLY 2000ms, and usually is.

When attempting to send a message in the main loop, I run _ProcessLoop a single time after _SendTelemetry to attempt to deliver the message right away. This allows me to get a failure if we suddenly become disconnected, and I can then store the message for sending later, or if the device happens to reboot or crash.

This used to be set to PROCESS_LOOP_TIMEOUT_MS. Setting the timeout to 0 from 500 made a big difference in the amount of time spent on this task. In this case, it takes just 360ms, and is often lower.

Here, I make a direct method call to the device (no work here, just returning a variable value). Because there's something to do, it takes just 1080ms, even with calling _SendCommandResponse.

The reason this even matters is because during this 2 second period, any direct method call will not be processed and must wait until the next or subsequent iteration. A response can take upwards of 5 seconds or more. Any idea why when there's nothing for _ProcessLoop to do, it takes exactly 2 full seconds to finish, but when there is something to do it finishes quickly?

Hi @rtheil-growlink
I'm thinking it could be this from the coreMQTT docs:
https://github.com/FreeRTOS/coreMQTT/blob/19d198c8cb03c00307608b3008b62d433b6f4353/source/include/core_mqtt.h#L445-L461
Specifically, I think number 1 might be happening. We have a transport send/receive delay of 5000ms currently in our ADU sample:
https://github.com/Azure-Samples/iot-middleware-freertos-samples/blob/1f3be178a6bce3dd072a8e85580b11ab73765ae3/demos/sample_azure_iot_adu/sample_azure_iot_adu.c#L113
Can you try modifying that to see if a lower number gets you better performance?

Hey @rtheil-growlink
Any updates on this?

Thanks for those updates, Dane. I'm going to look deeper into this when I have a chance. For now it's not a huge issue, but something I'd like to improve in future firmware versions. I'll close this for now and reopen in the future if needed.

source: gharchive/issue

authors: danewalton rtheil-growlink

repo: Azure/azure-iot-middleware-freertos

license: MIT

license_type: permissive

license_source: github-api


text:
External visibility on DatasyncClient's ServiceHttpClient

Would it be possible to change the ServiceHttpClient from internal to public or have some sort of getter to retrieve the Http Context that was used in DatasyncClient? One use case I am seeing right now is when you hook into the SynchronizationProgress event.
It would be nice to have access to HttpClient or at least the HttpContext.Request of the sync request.
In the context I am using my extended implementation, it would be nice to retrieve query string parameters from the original to be sent back on a following request when the SynchronizationProgress SynchronizationEventType.ItemWasStored triggers.

The SyncContext doesn't have access to the query parameters used.  They are generated on the fly when the LINQ query is executed.   The SyncContext never sees anything about the HTTP request.
What are you actually trying to solve for here?

I would like be able to reuse Datasync Client's internal HttpClient to send a request on behalf of the client to an endpoint.
Basically, I am in the process of wrapping a layer around Datasync for MongoDB and made an AbstractOfflineStore to enable storing files through GridFS.
I can create a new HttpClient and point it to the endpoint I need to retrieve the actual files but it would be preferable if I could use the one configured for the client since it's got some HttpPipeline configuration that I need to have access to from within that wrapper.
For now, I am looking at passing down the HttpPipeline from DatasyncClientOptions to my custom AbstractOfflineStore.

Any reason why you can't use DatasyncClient.SendAsync(HttpRequestMessage, CancellationToken) for this purpose?

(To adjust that - yes, I'm working on making an underlying HttpClient be available, but the mechanism for re-use is already there)

authors: acidindustries adrianhall

repo: Azure/azure-mobile-apps

license: mit

license_source: bigquery


text:
Azure OpenAI REST API extensions completions returning 'NoneType' object has no attribute 'get' for multi-turn conversations

Using API version 2023-06-01-preview.
API call to POST {your-resource-name}/openai/deployments/{deployment-id}/extensions/chat/completions responds with:
{"error": {"requestid": "2867bb5c-a1aa-471a-8af0-b6f80682c65a", "code": 500, "message": "'NoneType' object has no attribute 'get'"}}
if messages array is contains more than one message.
Body that responds with 200:
{'messages': [{'role': 'user', 'content': 'what is data governance'}], 'temperature': 0.0, 'max_tokens': 800, 'top_p': 1.0, 'stop': None, 'stream': False, 'dataSources': [{'type': 'AzureCognitiveSearch', 'parameters': {'endpoint': {AZURE_OPENAI_ENDPOINT}, 'key': {AZURE_SEARCH_KEY}, 'indexName': {AZURE_SEARCH_INDEX}, 'fieldsMapping': {FIELD_MAPPINGS}, 'inScope': True, 'topNDocuments': '5', 'queryType': 'semantic', 'semanticConfiguration': {AZURE_SEMANTIC_CONFIG}, 'roleInformation': '{AZURE_OPENAI_SYSTEM_MESSAGE', 'embeddingEndpoint': '{AZURE_OPENAI_EMBEDDING_ENDPOINT}, 'embeddingKey': {AZURE_OPENAI_EMBEDDING_KEY}}]}
Body that responds with 500 and message NoneType has no attribute 'get':
{'messages': [{'role': 'user', 'content': 'what is data governance'}, {'role': 'assistant', 'content': "Data governance refers to the management of the availability, usability, integrity, and security of data in an organization. It involves a set of processes, roles, policies, standards, and metrics that ensure the effective and efficient use of information in enabling an organization to achieve its goals. It includes the use of technologies like Machine Learning and Artificial Intelligence to automatically classify, extract, and enrich physical and digital content and data. This means software can be trained to understand the meaning and context of information, classify it into categories, and automatically apply governance categories and retention schedules[doc5]. \n\nIn the context of AvePoint's Cloud Governance, data governance involves the lifecycle management of collaborative objects and spaces. It allows an organization to take a holistic approach to lifecycle management, including the creation of renewal processes, the configuration of renewal triggers based on inactivity, and the management of both active and inactive workspaces. It also includes the ability to configure outreach communications to users in multiple regions with language-specific renewal process direction and guidance[doc1]. \n\nMoreover, data governance also involves auditing the company's data before migration, which can save time, reduce the attack threat surface, and ensure the new system is secure, efficient, and in compliance. This process includes file analysis and tagging and classifying sensitive data[doc2]. \n\nIn summary, data governance is a comprehensive approach to managing, securing, and utilizing data effectively in an organization."}, {'role': 'user', 'content': 'summarize that into bullet points'}], 'temperature': 0.0, 'max_tokens': 800, 'top_p': 1.0, 'stop': None, 'stream': False, 'dataSources': [{'type': 'AzureCognitiveSearch', 'parameters': {'endpoint': {AZURE_OPENAI_ENDPOINT}, 'key': {AZURE_SEARCH_KEY}, 'indexName': {AZURE_SEARCH_INDEX}, 'fieldsMapping': {FIELD_MAPPINGS}, 'inScope': True, 'topNDocuments': '5', 'queryType': 'semantic', 'semanticConfiguration': {AZURE_SEMANTIC_CONFIG}, 'roleInformation': '{AZURE_OPENAI_SYSTEM_MESSAGE', 'embeddingEndpoint': '{AZURE_OPENAI_EMBEDDING_ENDPOINT}, 'embeddingKey': {AZURE_OPENAI_EMBEDDING_KEY}}]}
This error just appeared today

I got the same 3 days back..any solutions?

authors: carolinedpena nipun-threestops

repo: Azure/azure-rest-api-specs


text:
[DO NOT MERGE ] Introduce new scanners resource type

Introduce new scanners resource type

/azp run

/azp run

@eswatad - signed off from ARM side.

source: gharchive/pull-request

authors: RamyasreeChakka eswatad ruowan


text:
[Hub Generated] Review request for Microsoft.InstanceMetadataService to add version stable/2021-11-01

This is a PR generated at OpenAPI Hub. You can view your work branch via this link.
Data Plane API - Pull Request

API Info: The Basics
Most of the information about your service should be captured in the issue that serves as your engagement record.

Link to engagement record issue:

Is this review for (select one):

[ ] a private preview
[ ] a public preview
[ ] GA release

Change Scope
This section will help us focus on the specific parts of your API that are new or have been modified. Please share a link to the design document for the new APIs, a link to the previous Open API document (swagger) if applicable, and the root paths that have been updated.

Design Document:
Previous Open API Doc:
Updated paths:

❔Got questions? Need additional info?? We are here to help!

 Contact us!
The Azure API Review Board is dedicated to helping you create amazing APIs. You can read about our mission and learn more about our process on our wiki.

💬 Teams Channel
💌 email

Click here for links to tools, specs, guidelines & other good stuff
Tooling

Open API validation tools were run on this PR. Go here to see how to fix errors
Spectral Linting
Open API Hub

Guidelines & Specifications

Azure REST API Guidelines
OpenAPI Style Guidelines
Azure Breaking Change Policy

Helpful Links

Azure DevTools Wiki

Add a suppressions line for the new version (where they're all suppressing camelcase)

/azp run unifiedPipeline

/azp run unifiedPipeline

@weidongxu-microsoft it has already been in preview for some time and now we are moving it from preview to GA

Please check https://dev.azure.com/azure-sdk/internal/_wiki/wikis/internal.wiki/271/Azure-REST-API-Stewardship
Usually service should not start with a GA api-version?

What do you mean by this? This PR is just to add documentation for one of our new versions that we are moving public from experimental. I don't believe we've had to do this for any previous additions like this.

What do you mean by this? This PR is just to add documentation for one of our new versions that we are moving public from experimental. I don't believe we've had to do this for any previous additions like this.

No, they don't. If 2017-03-01 is already GA, and it is only adding a few properties, it does not need a review session.
My ask is most based on that there is no preview api-version ever, https://github.com/Azure/azure-rest-api-specs/tree/main/specification/imds/data-plane/Microsoft.InstanceMetadataService
But you and Anshul are saying "moving from preview to GA" or "moving from experimental to public", which I am not sure how I should interpret it, and whether to treat this as your first GA version or not.
If you are doing preview or experimental, you should use a preview api-version like 2021-11-01-preview.


text:
Microsoft.MachineLearningServices - Update readme.md

ARM API Information (Control Plane)
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Azure 1st Party Service can try out the Shift Left experience to initiate API design review from ADO code repo.  If you are interested, may request engineering support by filling in with the form https://aka.ms/ShiftLeftSupportForm.
Changelog
Add a changelog entry for this PR by answering the following questions:

What's the purpose of the update?

[ ] new service onboarding
[ ] new API version
[ ] update existing version for new feature
[ ] update existing version to fix swagger quality issue in s360
[x] Other, please clarify

When are you targeting to deploy the new service/feature to public regions? Please provide the date or, if the date is not yet available, the month.
When do you expect to publish the swagger? Please provide date or, the the date is not yet available, the month.
By default, Azure SDKs of all languages (.NET/Python/Java/JavaScript for both management-plane SDK and data-plane SDK, Go for management-plane SDK only ) MUST be refreshed with/after swagger of new version is published. If you prefer NOT to refresh any specific SDK language upon swagger updates in the current PR, please leave details with justification here.

Contribution checklist (MS Employees Only):

[x] I commit to follow the Breaking Change Policy of "no breaking changes"
[x] I have reviewed the documentation for the workflow.
[x] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

Applicability: :warning:
If your changes encompass only the following scenarios, you should SKIP this section, as these scenarios do not require ARM review.

Change to data plane APIs
Adding new properties
All removals

Otherwise your PR may be subject to ARM review requirements. Complete the following:

[ ] Check this box if any of the following apply to the PR so that the label "ARMReview" and "WaitForARMFeedback" will be added by bot to kick off ARM API Review.  Missing to check this box in the following scenario may result in delays to the ARM manifest review and deployment.

Adding a new service
Adding new API(s)
Adding a new API version
-[ ] To review changes efficiently, ensure you copy the existing version into the new directory structure for first commit and then push new changes, including version updates, in separate commits. You can use OpenAPIHub to initialize the PR for adding a new version. For more details refer to the wiki.

[ ] Ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If you have any breaking changes as defined in the Breaking Change Policy, request approval from the Breaking Change Review Board.
Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Additional details on the process and office hours are on the Breaking Change Wiki.
NOTE: To update API(s) in public preview for over 1 year (refer to Retirement of Previews)
Please follow the link to find more details on PR review process.

The Avocado failures for MISSING_APIS_IN_DEFAULT_TAG are expected. Those APIs are intentionally removed from 2022-10-01-preview (they were removed in an earlier API Version)


text:
Added Oakes API 2022-05-04 preview

ARM API Information (Control Plane)
Changelog

What's the purpose of the update?

[x] new API version

When are you targeting to deploy the new service/feature to public regions?

October 2022

When do you expect to publish the swagger?

October 2022

By default, Azure SDKs of all languages (.NET/Python/Java/JavaScript for both management-plane SDK and data-plane SDK, Go for management-plane SDK only ) MUST be refreshed with/after the swagger of the new version is published. If you prefer NOT to refresh any specific SDK language upon swagger updates in the current PR, please leave details with justification here.

Contribution checklist (MS Employees Only):

[x ] I commit to follow the Breaking Change Policy of "no breaking changes"
[ x] I have reviewed the documentation for the workflow.
[ ] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further questions about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

Applicability: :warning:
If your changes encompass only the following scenarios, you should SKIP this section, as these scenarios do not require ARM review.

Change to data plane APIs
Adding new properties
All removals

Otherwise your PR may be subject to ARM review requirements. Complete the following:

[ ] Check this box if any of the following apply to the PR so that the label "ARMReview" and "WaitForARMFeedback" will be added by bot to kick off ARM API Review.  Missing to check this box in the following scenario may result in delays to the ARM manifest review and deployment.

Adding a new service
Adding new API(s)
Adding a new API version
-[ ] To review changes efficiently, ensure you copy the existing version into the new directory structure for first commit and then push new changes, including version updates, in separate commits. You can use OpenAPIHub to initialize the PR for adding a new version. For more details refer to the wiki.

[ ] Ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If you have any breaking changes as defined in the Breaking Change Policy, request approval from the Breaking Change Review Board.
Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Additional details on the process and office hours are on the Breaking Change Wiki.
NOTE: To update API(s) in public preview for over 1 year (refer to Retirement of Previews)
Please follow the link to find more details on PR review process.
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Azure 1st Party Service can try out the Shift Left experience to initiate API design review from ADO code repo.  If you are interested, may request engineering support by filling in with the form https://aka.ms/ShiftLeftSupportForm.

Hi, @ittayo, For review efficiency consideration, when creating a new api version, it is required to place API specs of the base version in the first commit, and push new version updates into successive commits. You can use OpenAPIHub to initialize the PR for adding a new version. For more details refer to the wiki. Or you could onboard API spec pipeline

Please see this requirement, thanks.


text:
Add Registries 2023-04-01-Preview Swagger

ARM API Information (Control Plane)
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Azure 1st Party Service can try out the Shift Left experience to initiate API design review from ADO code repo.  If you are interested, may request engineering support by filling in with the form https://aka.ms/ShiftLeftSupportForm.
Changelog
Add a changelog entry for this PR by answering the following questions:

What's the purpose of the update?

[ ] new service onboarding
[x] new API version
[ ] update existing version for new feature
[ ] update existing version to fix swagger quality issue in s360
[ ] Other, please clarify

When are you targeting to deploy the new service/feature to public regions? Please provide the date or, if the date is not yet available, the month.
When do you expect to publish the swagger? Please provide date or, the the date is not yet available, the month.
By default, Azure SDKs of all languages (.NET/Python/Java/JavaScript for both management-plane SDK and data-plane SDK, Go for management-plane SDK only ) MUST be refreshed with/after swagger of new version is published. If you prefer NOT to refresh any specific SDK language upon swagger updates in the current PR, please leave details with justification here.

Contribution checklist (MS Employees Only):

[ ] I commit to follow the Breaking Change Policy of "no breaking changes"
[ ] I have reviewed the documentation for the workflow.
[ ] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

Applicability: :warning:
If your changes encompass only the following scenarios, you should SKIP this section, as these scenarios do not require ARM review.

Change to data plane APIs
Adding new properties
All removals

Otherwise your PR may be subject to ARM review requirements. Complete the following:

[x] Check this box if any of the following apply to the PR so that the label "ARMReview" and "WaitForARMFeedback" will be added by bot to kick off ARM API Review.  Missing to check this box in the following scenario may result in delays to the ARM manifest review and deployment.

Adding a new service
Adding new API(s)
Adding a new API version
-[ ] To review changes efficiently, ensure you copy the existing version into the new directory structure for first commit and then push new changes, including version updates, in separate commits. You can use OpenAPIHub to initialize the PR for adding a new version. For more details refer to the wiki.

[ ] Ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If you have any breaking changes as defined in the Breaking Change Policy, request approval from the Breaking Change Review Board.
Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Additional details on the process and office hours are on the Breaking Change Wiki.
NOTE: To update API(s) in public preview for over 1 year (refer to Retirement of Previews)
Please follow the link to find more details on PR review process.

      }

You should not declare location and tags yourself, instead please inherit allOf TrackedResouce in
https://github.com/Azure/azure-rest-api-specs/blob/main/specification/common-types/resource-management/v5/types.json

Refers to: specification/machinelearningservices/resource-manager/Microsoft.MachineLearningServices/preview/2023-04-01-preview/registries.json:537 in e6d249c. [](commit_id = e6d249c6944ef36903343f1cbe2e05c816749939, deletion_comment = False)

      }

You should not declare location and tags yourself, instead please inherit allOf TrackedResouce in
https://github.com/Azure/azure-rest-api-specs/blob/main/specification/common-types/resource-management/v5/types.json

unless there's some reason you can't?

Refers to: specification/machinelearningservices/resource-manager/Microsoft.MachineLearningServices/preview/2023-04-01-preview/registries.json:537 in 4197a16. [](commit_id = 4197a16, deletion_comment = False)

@TimLovellSmith, I think you're referring to PartialRegistryPartialTrackedResource as defined in this swagger? This dto inherits from PartialTrackedResource in Common. Do you know where that definition is located? The allOf should be pointing to that instead.

Similar feedback as this one : https://github.com/Azure/azure-rest-api-specs/pull/23085

EndpointServiceConnectionStatus

Cannot inherit. We have approved, pending, rejected, disconnected (4 states) whereas definition as seen here has just 3:

PartialManagedServiceIdentity

Our definition inherits ManagedServiceIdentityType from common types
Our definition does not have principalId, tenantId as defined here. Cannot inherit.
Our defintion of PartialUserAssignedIdentities does not include additionalProperties of principalId and clientId as defined here. Cannot inherit.

PartialRegistryPartialTrackedResource

Our definition no longer has 'location' property. (we removed location to fix that PATH Lint Diff error)
Cannot inherit from TrackedResource as it has 'location' property set as required.

PrivateEndpointConnectionARMResource

Cannot inherit. It is defined to inherit all of Resource object here. Our definition does not have name, type, or systemData.

PrivateEndpointConnectionProperties

Cannot inherit. Our definition of this includes a PrivateEndpoint object that has a subnetArmId. This is not defined in common types here

PrivateEndpointResource

Cannot inherit. We need 'id' variable to not be readOnly as defined here.

PrivateLinkServiceConnectionState

Cannot inherit. We define our status with EndpointServiceConnectionStatusthat has 4 states.

RegistryTrackedResource:

This we can inherit.

    "identity": {

Do you still need "identity" and "sku" in "PartialRegistryPartialTrackedResource" model? I assume, these two are not patch-able as of now. If not needed, please remove these.

Refers to: specification/machinelearningservices/resource-manager/Microsoft.MachineLearningServices/preview/2023-04-01-preview/registries.json:491 in 3f37fcd. [](commit_id = 3f37fcde976a6e7e85c6a94e719cdb81e4763184, deletion_comment = False)

authors: TimLovellSmith komo23x raosuhas visingla-ms


text:
Measure test coverage on Windows and consider adding to the CI

We currently measure on Linux only, so only the curl transport paths are showing up. The WinHttp code coverage is missing.
Extracted from https://github.com/Azure/azure-sdk-for-cpp/pull/4015#discussion_r1002065122
Our code coverage CI leg only runs on linux/curl, so we don't see the coverage number for Windows-specific transport adapter files:
https://dev.azure.com/azure-sdk/public/_build/results?buildId=1934177&view=codecoverage-tab

https://github.com/Azure/azure-sdk-for-cpp/blob/53c9da155244b72ac90ad27f515d4ea1e7505a01/eng/pipelines/templates/stages/platform-matrix.json#L143-L149
https://github.com/Azure/azure-sdk-for-cpp/blob/53c9da155244b72ac90ad27f515d4ea1e7505a01/eng/pipelines/templates/stages/platform-matrix-live.json#L37-L43
cc @danieljurek, @vhvb1989, @LarryOsterman

Azure DevOps supports one set of code coverage statistics for an entire pipeline run. This forced us into the current configuration we have today.
https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/test/publish-code-coverage-results?view=azure-devops#is-code-coverage-data-merged-when-multiple-files-are-provided-as-input-to-the-task-or-multiple-tasks-are-used-in-the-pipeline
To that end, we should probably focus on outputting code coverage in the logs but there wouldn't be an obvious way to bubble that information up and easily enforce it the way that we do with our linux/curl run today.

authors: ahsonkhan danieljurek

repo: Azure/azure-sdk-for-cpp


text:
Doc comment checker doesn't check fields within structs

While we validate doc comments for exported funcs/methods/types, it appears that we don't check for doc comments within structs.

Yep, we look at types but not their contents.
https://github.com/Azure/azure-sdk-for-go/blob/main/eng/tools/doccheck/main.go#L63-L67

I seem to remember this coming up before and us deciding to do nothing because generated types use docstrings from the API spec, which often don't follow the Go convention and generally don't admit a simple heuristic for making them so.

I think you're right. And in some cases, there are no doc strings in the OpenAPI at all, so generating a // Foo field... doc comment to satisfy the tool is pretty useless.

authors: chlowell jhendrixMSFT

repo: Azure/azure-sdk-for-go


text:
[AutoPR compute/resource-manager] Add AutomaticOSUpgradeProperties in GET VMImageVersion API

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/4031

This PR has been merged into https://github.com/Azure/azure-sdk-for-go/pull/2815


text:
[FEATURE REQ] Service Bus/EventHubs tracing conform to the semantic conventions of messaging systems

Service Bus SDK should conform and fit the above path used for other messaging systems but no rush as messaging semantic conventions may still change.
See this reference PR

merging it with #21684 - messaging semantic conventions live in azure-core-tracing-opentelemetery and not  library-specific

This can now be closed - Azure messaging libs follow otel messaging semconv

authors: lmolkova srnagar

repo: Azure/azure-sdk-for-java


text:
Add session token sent in the request to the backend in request diagnostics

To simplify session consistency investigations it would be useful to have the session token being sent to the backend/Gateway included in request diagnostics logs. The session token is scoped to the targeted partition in most cases - so at least in those cases it should not be a lot of overhead adding it to the diagnostics.
.Net link: https://github.com/Azure/azure-cosmos-dotnet-v3/issues/3107

PR: https://github.com/Azure/azure-sdk-for-java/pull/29516

requestSessionToken is available in the meantime in the diagnostics

authors: FabianMeiswinkel simorenoh xinlian12


text:
[FEATURE REQ] Add documentation of new properties maxSizeInMegabytes and defaultMessageTimeToLive

Is your feature request related to a problem? Please describe.
In the feature request https://github.com/Azure/azure-sdk-for-java/issues/35435 new properties maxSizeInMegabytes and defaultMessageTimeToLive were added https://github.com/Azure/azure-sdk-for-java/pull/36996
However, those new properties are not present in the documentation: https://learn.microsoft.com/en-us/azure/developer/java/spring-framework/configuration-properties-azure-service-bus
Describe the solution you'd like
Include the properties and description in the documentation https://learn.microsoft.com/en-us/azure/developer/java/spring-framework/configuration-properties-azure-service-bus
Additional context
We use spring-cloud-stream in our codebase with multiple binders which is decided at runtime. If using azure-service-bus binders, topics get auto-created with default configurations. We wanted to control Message Expiry time per Topic.
Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

[ x] Description Added
[ x] Expected solution specified

The doc has been updated, see:

https://learn.microsoft.com/en-us/azure/developer/java/spring-framework/spring-cloud-stream-support?tabs=SpringCloudAzure5x#consumer-properties-1
https://learn.microsoft.com/en-us/azure/developer/java/spring-framework/spring-cloud-stream-support?tabs=SpringCloudAzure5x#producer-properties-1

authors: fredysierra saragluna


text:
Prepare Event Hubs release

Prepare for Event Hubs release.
azure-messaging-eventhubs: 5.9.0
azure-messaging-eventhubs-checkpointstore-blob: 1.8.1

I think we are missing updates to the readme for the latest version release

authors: conniey samvaity


text:
Fix for resuming orderby queries from continuation  that includes undefined/null

Resuming orderby queries sorting that includes undefined/null items from continuation when the page breaks on undefined/null items needs including query literals and filters accordingly. This PR addresses them.
Also improves serialization of orderby items in continuation token to handle undefined/null properly.

/azp run java - cosmos - tests


text:
Fix version of nexus-staging-maven-plugin plugin to 1.6.8

This PR sets the version of nexus-staging-maven-plugin to 1.6.8 instead of using the latest version because 1.6.10 has an issue with publishing to Maven.

If we specify the version here... do we think we'll need to bump this version in the future? Or is pinning to 1.6.8 fine.

We'll remove the hardcoded version after a newer version of the nexus plugin is released fixing the bug.

/check-enforcer override


text:
Add Meter.isEnabled() for more optimal usage

Adds Meter.isEnabled() that users can call to check if metrics implementation is found at all. Helps to shave off a bit more perf when everything is noop.

API change check
APIView has identified API level changes in this PR and created following API reviews.
azure-core

authors: azure-sdk lmolkova


text:
Fix Cosmos DB Samples issue #31136

Fixes #31136
@josefree, @KrzysztofCwalina, @kyle-patterson and @xinlian12 for notification.

@hongli750210 - why do we change this to 0? -1 should work. I wonder how you tested it?

@kushagraThapar
Jar version:
azure-cosmos: 4.37.0-alpha.20220926.2
Test step as follows：

Create Azure Cosmos DB for NoSQL in Azure portal.
Replace the properties of "CosmosClientBuilder" with the endpoint and primary key of the created Azure Cosmos DB
Run main method of test class AnalyticalStorageCodeSnippet

Test Result:
BadRequest exception occurs, exception infomation as follows:
Exception in thread "main" {"ClassName":"CosmosException","userAgent":"azsdk-java-cosmos/4.36.0-alpha.20220906.1 Windows11/10.0 JRE/17.0.4.1","statusCode":400,"resourceAddress":"https://cosmosdb4lh-westus.documents.azure.com:443/dbs/testDB/colls","error":"{"code":"BadRequest","message":"Message: {\"Errors\":[\"The input content is invalid - 'analyticalStorageTtl' is not a valid property in the current payload.\"]}\r\nActivityId: 0dd21efd-de4f-4f1d-b16e-1b126035f8e7, Request URI: /apps/6d65d659-c668-473c-a798-50083b67853c/services/8a8a2418-cb71-44f0-9615-d7e2adcf7a98/partitions/2b75256c-d3cc-4f29-89bd-5cb7e0b8c93f/replicas/133119468144684133p, RequestStats: , SDK: Microsoft.Azure.Documents.Common/2.14.0, StatusCode: BadRequest","additionalErrorInfo":null}","innerErrorMessage":"Message: {"Errors":["The input content is invalid - 'analyticalStorageTtl' is not a valid property in the current payload."]}\r\nActivityId: 0dd21efd-de4f-4f1d-b16e-1b126035f8e7, Request URI: /apps/6d65d659-c668-473c-a798-50083b67853c/services/8a8a2418-cb71-44f0-9615-d7e2adcf7a98/partitions/2b75256c-d3cc-4f29-89bd-5cb7e0b8c93f/replicas/133119468144684133p, RequestStats: , SDK: Microsoft.Azure.Documents.Common/2.14.0, StatusCode: BadRequest","causeInfo":null,"responseHeaders":"{Transfer-Encoding=chunked, x-ms-last-state-change-utc=Thu, 03 Nov 2022 11:00:17.017 GMT, x-ms-current-replica-set-size=4, Server=Microsoft-HTTPAPI/2.0, x-ms-request-duration-ms=1.674, x-ms-session-token=0:-1#15, lsn=15, x-ms-request-charge=0, x-ms-schemaversion=1.14, x-ms-transport-request-id=5224, x-ms-number-of-read-regions=1, x-ms-current-write-quorum=3, x-ms-cosmos-quorum-acked-llsn=15, x-ms-quorum-acked-lsn=15, Date=Fri, 04 Nov 2022 03:28:16 GMT, x-ms-activity-id=0dd21efd-de4f-4f1d-b16e-1b126035f8e7, Strict-Transport-Security=max-age=31536000, x-ms-xp-role=1, x-ms-global-Committed-lsn=15, x-ms-cosmos-llsn=15, x-ms-gatewayversion=version=2.14.0, x-ms-serviceversion=version=2.14.0.0, Content-Type=application/json}","requestHeaders":"[x-ms-cosmos-allow-tentative-writes=true, Accept=application/json, x-ms-date=Fri, 04 Nov 2022 03:28:15 GMT, Content-Type=application/json]","cosmosDiagnostics":{"userAgent":"azsdk-java-cosmos/4.36.0-alpha.20220906.1 Windows11/10.0 JRE/17.0.4.1","activityId":"0dd21efd-de4f-4f1d-b16e-1b126035f8e7","requestLatencyInMs":1189,"requestStartTimeUTC":"2022-11-04T03:28:15.377422300Z","requestEndTimeUTC":"2022-11-04T03:28:16.566755Z","responseStatisticsList":[],"supplementalResponseStatisticsList":[],"addressResolutionStatistics":{},"regionsContacted":["west us"],"retryContext":{"statusAndSubStatusCodes":null,"retryCount":0,"retryLatency":0},"metadataDiagnosticsContext":{"metadataDiagnosticList":null},"serializationDiagnosticsContext":{"serializationDiagnosticsList":[{"serializationType":"CONTAINER_SERIALIZATION","startTimeUTC":"2022-11-04T03:28:15.378422100Z","endTimeUTC":"2022-11-04T03:28:15.379422100Z","durationInMilliSecs":1.0}]},"gatewayStatistics":{"sessionToken":"0:-1#15","operationType":"Create","resourceType":"DocumentCollection","statusCode":400,"subStatusCode":0,"requestCharge":0.0,"requestTimeline":[{"eventName":"connectionCreated","startTimeUTC":"2022-11-04T03:28:15.379422100Z","durationInMilliSecs":929.2115},{"eventName":"connectionConfigured","startTimeUTC":"2022-11-04T03:28:16.308633600Z","durationInMilliSecs":0.0},{"eventName":"requestSent","startTimeUTC":"2022-11-04T03:28:16.308633600Z","durationInMilliSecs":1.9978},{"eventName":"transitTime","startTimeUTC":"2022-11-04T03:28:16.310631400Z","durationInMilliSecs":252.6953},{"eventName":"received","startTimeUTC":"2022-11-04T03:28:16.563326700Z","durationInMilliSecs":1.0282}],"partitionKeyRangeId":null,"exceptionMessage":"Message: {"Errors":["The input content is invalid - 'analyticalStorageTtl' is not a valid property in the current payload."]}\r\nActivityId: 0dd21efd-de4f-4f1d-b16e-1b126035f8e7, Request URI: /apps/6d65d659-c668-473c-a798-50083b67853c/services/8a8a2418-cb71-44f0-9615-d7e2adcf7a98/partitions/2b75256c-d3cc-4f29-89bd-5cb7e0b8c93f/replicas/133119468144684133p, RequestStats: , SDK: Microsoft.Azure.Documents.Common/2.14.0, StatusCode: BadRequest","exceptionResponseHeaders":"{Transfer-Encoding=chunked, x-ms-last-state-change-utc=Thu, 03 Nov 2022 11:00:17.017 GMT, x-ms-current-replica-set-size=4, Server=Microsoft-HTTPAPI/2.0, x-ms-request-duration-ms=1.674, x-ms-session-token=0:-1#15, lsn=15, x-ms-request-charge=0, x-ms-schemaversion=1.14, x-ms-transport-request-id=5224, x-ms-number-of-read-regions=1, x-ms-current-write-quorum=3, x-ms-cosmos-quorum-acked-llsn=15, x-ms-quorum-acked-lsn=15, Date=Fri, 04 Nov 2022 03:28:16 GMT, x-ms-activity-id=0dd21efd-de4f-4f1d-b16e-1b126035f8e7, Strict-Transport-Security=max-age=31536000, x-ms-xp-role=1, x-ms-global-Committed-lsn=15, x-ms-cosmos-llsn=15, x-ms-gatewayversion=version=2.14.0, x-ms-serviceversion=version=2.14.0.0, Content-Type=application/json}"},"systemInformation":{"usedMemory":"19034 KB","availableMemory":"4119974 KB","systemCpuLoad":"empty","availableProcessors":8},"clientCfgs":{"id":1,"machineId":"uuid:434d4ab0-9fea-401b-8556-ef576968718a","connectionMode":"DIRECT","numberOfClients":1,"clientEndpoints":{"https://cosmosdb4lh.documents.azure.com:443/":1},"connCfg":{"rntbd":"(cto:PT5S, nrto:PT5S, icto:PT0S, ieto:PT1H, mcpe:130, mrpc:30, cer:true)","gw":"(cps:1000, nrto:PT1M, icto:PT1M, p:false)","other":"(ed: true, cs: false, rv: false)"},"consistencyCfg":"(consistency: null, mm: true, prgns: [])"}}}

@hongli750210 - I just ran it locally and it works correctly for me, please see the below screenshot. Can you please try to set this property through azure portal directly and see how it works?
Also, worth pasting the complete stack trace of your exception.

@kushagraThapar
Atfer enable Azure Synapse Link of cosmos account, run main method of test class AnalyticalStorageCodeSnippet again, the test case successfully. So I think occured BadRequest exception is related to this setting.

Thank you @hongli750210 for the investigation, appreciate you looking into this.


text:
Remove continuation token doc

Description
Remove continuation token parameters from docs
Swagger README.md file used for regeneration: https://github.com/Azure/azure-rest-api-specs/blob/3e27c70e7c02c07b458bc0e94716c3d82d3fdd19/specification/loadtestservice/data-plane/readme.md
All SDK Contribution checklist:

[x] The pull request does not introduce [breaking changes]
[ ] CHANGELOG is updated for new features, bug fixes or other significant changes.
[x] I have read the contribution guidelines.

General Guidelines and Best Practices

[x] Title of the pull request is clear and informative.
[x] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[x] Pull request includes test coverage for the included changes.

API change check
API changes are not detected in this pull request.

@weidongxu-microsoft @samvaity Requesting review for removal of continuationToken as part of REST APIs. No code changes done - only a few doc changes


text:
Remove overlapping uses of getContext

Description
There is duplicated logic in adding HTTP headers in the context. Aggregates that.
EntityHelper.getContext performs the same action as: context.addData(AZURE_REQUEST_HTTP_HEADERS_KEY, new HttpHeaders())
All SDK Contribution checklist:

[x] The pull request does not introduce [breaking changes]
[x] CHANGELOG is updated for new features, bug fixes or other significant changes.
[x] I have read the contribution guidelines.

General Guidelines and Best Practices

[x] Title of the pull request is clear and informative.
[x] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[x] Pull request includes test coverage for the included changes.

API change check
API changes are not detected in this pull request.

authors: azure-sdk conniey


text:
Introduce JSON representation model

Description
This Pull Request introduces a java model that utilises DefaultJsonReader and DefaultJsonWriter from azure-json to serialise and represent the structure.
All SDK Contribution checklist:

[X] The pull request does not introduce [breaking changes]
[] CHANGELOG is updated for new features, bug fixes or other significant changes.
[X] I have read the contribution guidelines.

General Guidelines and Best Practices

[X] Title of the pull request is clear and informative.
[X] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[X] Pull request includes test coverage for the included changes.

Closed to open new draft


text:
[AutoPR cognitiveservices/resource-manager] Add CheckDomainAvailability resource type.

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/6347

This PR has been merged into https://github.com/Azure/azure-sdk-for-java/pull/3433


text:
Increment versions for batch releases

Increment package versions for batch releases

API change check
API changes are not detected in this pull request.


text:
Turn off shallow cloning.

This PR turns off shallow cloning in. I believe that shallow cloning and some replication/caching on the GitHub side might be causing intermittent build failures.

Merged after a quick chat with @JonathanGiles


text:
[Service Bus] Merge DispositionStatus and DispositionType

https://github.com/Azure/azure-sdk-for-js/pull/8406#discussion_r410582281

Done with #8707

authors: HarshaNalluru ramya-rao-a

repo: Azure/azure-sdk-for-js


text:
[service-bus] getMaxMessages needs to retry if link doesn't initialize

getMaxMessages had an edge case where the link being closed just after open would cause us to access a link that was undefined and throw a non-retryable exception.
This PR fixes that by checking that the link is defined, and throwing a retryable exception if it is not.
Fixes #15398

(NOTE: I'll need to backport this to release a hotfix since we're currently doing a preview/beta on main)

/azp run js - service-bus - tests

It's a pretty simple change. If anyone has a post-merge feedback let me know. :)


text:
appcomplianceautomation release

https://github.com/Azure/sdk-release-request/issues/3286

/azp run prepare-pipelines

/azp run js - appcompilanceautomation - mgmt - ci

/azp run js - appcomplianceautomation - mgmt - ci


text:
InheritDoc fails with IDT002 on explicit interface implementations in internal class

For example, the following code without any inheritdoc comment at all.

internal sealed class EventHubListener : IListener, IEventProcessorFactory, IScaleMonitorProvider
{
        void IListener.Cancel()
        {
            StopAsync(CancellationToken.None).Wait();
        }
}

Produces the following error:
  Microsoft.Azure.WebJobs.Extensions.EventHubs.original.xml: [IDT002] No matching documentation could be found for: M:Microsoft.Azure.WebJobs.EventHubs.EventHubListener.Microsoft#Azure#WebJobs#Host#Listeners#IListener#Cancel, which attempts to inherit from: M:Microsoft.Azure.WebJobs.Host.Listeners.IListener.Cancel

cc @christothes
FYI @maririos just in case

The package we reference doesn't have xml docs (tracked by https://github.com/Azure/azure-webjobs-sdk/issues/2627) but it shouldn't matter because the class is internal.

repo: Azure/azure-sdk-for-net


text:
When a field is an AvroUnion of Null and a Non-primitive type, incorrect code is generated

I was working with a schema (attached in .zip) with a field defined like so:
"fields": [
        {"name": "shouldNotBeObject", "type":
            [
                "null",
                {"type":"map", "values":
                    { "type": "array", "items":
                        {"type":"string","avro.java.string":"String"}
                    }
                }
            ]
        }
    ]

When running Microsoft.Hadoop.Avro.Tools codegen incorrect code is generated. The field is given type object on the property as well as for the parameter in the constructor. In this example, the field should be type Dictionary<string,IList<string>>.
I am a Microsoft FTE (admodlin). I reached out to Maxim Lukiyanov and he suggested I submit a bug and a PR. This is my first time doing this so please let me know if I did something wrong.

The Avro SDK is no longer supported by Microsoft.

authors: adammodlin tylerfox


text:
[BUG] Extension methods calling RegisterClientFactory() pollute the suggestion options in VS

Describe the bug
When coding with almost all types in the Azure NET client SDK, the completion options list has ~5 static helper methods that have a very particular use and simply clutter up the list in most cases. See screenshot below for details.
Expected behavior
Ideally, these should not show up. Now, I guess with VS picking up on potential options and auto-suggesting the needed 'usings', this problem has become worse. Users don't need to add using Microsoft.Extensions.Azure; in their code to be seeing these suggestions.
Actual behavior (include Exception or Stack Trace)
See screenshot below to see what users see when looking at the completion list.
It may get worse with more components calling that static method and all those options showing up in the list.

@pakrym , what do you suggest to mitigate this issue?

It's tracked in roslyn https://github.com/dotnet/roslyn/issues/39689

Thanks for the context.
Seems like we may have to live with this behavior for at least some time.

authors: Mohit-Chakraborty pakrym


text:
[FEATURE REQ] Azure.Data.Tables TokenCredential

Library or service name.
What library or service is this request related to? [e.g. Azure.Storage.Blobs]
Azure.Data.Tables
Is your feature request related to a problem? Please describe.
What feature would you like to get added? What problem is it solving?
Many of the libraries in here allow you to initialize the credential clients with an implementation of the class
TokenCredentials that allow you to authenticate with, for example, oauth which table storage supports.
https://docs.microsoft.com/en-us/rest/api/storageservices/authorize-with-azure-active-directory
For example, i'm using a ITokenAcquisition from Identity.Web that allows me to use a token from Azure AD to instantiate an Service Client for Queue, just like:
queueServiceClient = new QueueServiceClient(serviceUri, new TokenAcquisitionTokenCredential(tokenAcquisition));
This would allow me, in this case, make use of the logged in Oauth user and the user_impersonation scope for Azure Storage and make calls in behalf of the user, for an automatic lockdown of resources.
Let me know if you need me to go deeper.

Thank you for your feedback.  Tagging and routing to the team member best able to assist.

TokenCredential authorization is currently not supported by the service. If they do add it, we will implement it in the client SDK.

I see now that this section does not mention the .table domain.
Appreciate the fast response time @christothes.

authors: christothes flipch jsquire


text:
[BUG] - EventProcessorHost got stuck throwing ObjectDisposedException

Describe the bug
After calling EventProcessorHost.UnregisterEventProcessorAsync it started to throw Exceptions in a loop
Expected behavior
Stop the processor host without errors.
Actual behavior (include Exception or Stack Trace)
System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'System.Net.WebSockets.ClientWebSocket'.
at System.Net.WebSockets.ClientWebSocket.ThrowIfNotConnected()
at System.Net.WebSockets.ClientWebSocket.SendAsync(ArraySegment1 buffer, WebSocketMessageType messageType, Boolean endOfMessage, CancellationToken cancellationToken)    at Microsoft.Azure.Amqp.Transport.WebSocketTransport.WriteAsync(TransportAsyncCallbackArgs args)    at Microsoft.Azure.Amqp.AsyncIO.AsyncBufferWriter.Write(TransportAsyncCallbackArgs args) --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Microsoft.Azure.EventHubs.Amqp.AmqpEventHubClient.<CreateConnectionAsync>d__31.MoveNext() --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Microsoft.Azure.Amqp.FaultTolerantAmqpObject1.d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Azure.Amqp.Singleton1.<GetOrCreateAsync>d__13.MoveNext() --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Microsoft.Azure.Amqp.Singleton1.d__13.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Azure.EventHubs.Amqp.AmqpPartitionReceiver.d__15.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Azure.Amqp.FaultTolerantAmqpObject1.<OnCreateAsync>d__6.MoveNext() --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Microsoft.Azure.Amqp.Singleton1.d__13.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Azure.Amqp.Singleton`1.d__13.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Azure.EventHubs.Amqp.AmqpPartitionReceiver.d__13.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.Azure.EventHubs.Amqp.AmqpPartitionReceiver.d__13.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Azure.EventHubs.PartitionReceiver.d__30.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Azure.EventHubs.Amqp.AmqpPartitionReceiver.d__18.MoveNext()
To Reproduce
I can't reproduce. It only happened once
Environment:

Microsoft.Azure.EventHubs.Processor 4.2.0
.NET Framework 4.7.2
Windows Server 2019

//cc: @serkantkaraca, @kasun04

Thank you for your feedback.  Tagging and routing to the team best able to assist.

@felipepessoto The Microsoft.Azure.EventHubs package is deprecated. A newer package Azure.Messaging.EventHubs is available as of February 2020. While the older package will continue to receive critical bug fixes, we strongly encourage you to upgrade. Read the migration guide for more details.
@serkantkaraca Is there any way we can help @felipepessoto here?

@felipepessoto where did the client receive the failures? At UnregisterEventProcessorAsync call or in ProcessErrorAsync?

In ProcessErrorAsync. The error seems to happen when the Event Hub is deleted while the EventProcessorHost is running.

Per @felipepessoto's feedback,  marking as addressed and closing.

authors: felipepessoto jsquire ramya-rao-a serkantkaraca


text:
Compute Module Documentation Incorrect Parsing of Line Endings and Breaks.

In the Az.Compute Module there are a lot of instances of incorrectly escaped line endings and breaks showing up as <li&gt
This is showing up in documentation and has been pointed out to us by various customers. Please advise us on if this is a Swagger issue, and how we can fix this?
https://github.com/Azure/azure-docs-sdk-dotnet/blob/e6d5d173ebe06623700c29d06272e649f385be71/xml/Microsoft.Azure.Management.Compute.Models/VirtualMachine.xml#L622

Thank you for your feedback.  Tagging and routing to the team member best able to assist.

The docs team need to fix it, will check with docs team later.

authors: ArthurMa1978 haagha jsquire


text:
[BUG] ReceiveDeferredMessageAsync with PeekLock doesn't renew lock

Library name and version
Azure.Messaging.ServiceBus 7.6.0
Describe the bug
When calling DeferMessageAsync on a ServiceBusReceiver, the lock of the message you retrieve isn't renewed.
Meaning if the lock duration is expired one will not be able to acknowledge the message has been "Completed".
Expected behavior
The lock was renewed and one can Complete the message.
Actual behavior
The lock is not renewed and trying to Complete the message will throw
"The lock supplied is invalid. Either the lock expired, or the message has already been removed from the queue, or was received by a different receiver instance. (MessageLockLost)".
Reproduction Steps
Create a deferred message, wait for its lock to expires then run the following:
    async Task ProcessDeferred(int maximumMessageToSearch)
    {
        var messages = await receiver.PeekMessagesAsync(maximumMessageToSearch);

        if (messages != null && messages.Any())
        {
            foreach (var deferred in messages.Where(m => m.State == ServiceBusMessageState.Deferred))
            {
                Console.WriteLine($"Processing deferred message {deferred.MessageId}");
                var message = await receiver.ReceiveDeferredMessageAsync(deferred.SequenceNumber);

                // { add you processing here }

                await receiver.CompleteMessageAsync(message);

                Console.WriteLine($"Deferred message {deferred.MessageId} has been processed.");
            }
        }
    }

    await ProcessDeferred(10);

Environment
.Net 6

@serge-portima can you clarify how long your processing is taking? The messages received using the ServiceBusReceiver (whether active or deferred) won't have their locks renewed automatically, so if your processing is taking longer than the queue's lock duration, the lock will be lost.

@serge-portima can you clarify how long your processing is taking? The messages received using the ServiceBusReceiver (whether active or deferred) won't have their locks renewed automatically, so if your processing is taking longer than the queue's lock duration, the lock will be lost.

Not long, the processing time doesn't matter, one could retrieve a message and try to complete is as next operation it will fail as the lock of a deferred message you retrieve is not acquired/renewed.
The step to reproduce the bug is:
Imagine your topic lock duration is set to 5 seconds:

Send a message
Receive it
Defer it
Wait for 6 (5 + 1) seconds
Receive the deferred message (lock not acquired/renewed)
Try to complete it

Here a complete example on how to reproduce the issue.
`using Azure.Identity;
using Azure.Messaging.ServiceBus;
const string ToDefer = nameof(ToDefer);
const string UnDefer = nameof(UnDefer);
const string TopicName = "demo-deferred-sj";
const string ServiceBusNamespace = "#######";
const string SubscriptionName = "all";
var oneSecondMoreThanLockDuration = TimeSpan.FromSeconds(6);
var userName = System.Environment.GetEnvironmentVariable("USERNAME");
var serviceBusClient = new ServiceBusClient(
$"{ServiceBusNamespace}.servicebus.windows.net",
new VisualStudioCredential()
);
var sender = serviceBusClient.CreateSender(TopicName);
var processor = serviceBusClient.CreateProcessor(TopicName, SubscriptionName, new ServiceBusProcessorOptions
{
AutoCompleteMessages = false
});
var receiver = serviceBusClient.CreateReceiver(TopicName, SubscriptionName);
async Task SendMessagesAsync(string kind, string param = "", DateTimeOffset? scheduleTime = null)
{
var messageId = $"{kind}|{userName}|{param}|{Guid.NewGuid()}";
Console.WriteLine($"Sending {messageId}"
+ (scheduleTime.HasValue ? $" scheduled for {scheduleTime}" : "")
);
var serviceBusMessage = new ServiceBusMessage()
{
MessageId = messageId
};
if (scheduleTime.HasValue)
{
serviceBusMessage.ScheduledEnqueueTime = scheduleTime.Value;
}
await sender.SendMessageAsync(serviceBusMessage);
}
async Task ProcessMessageAsync(ProcessMessageEventArgs args)
{
Console.WriteLine($"Handling {args.Message.MessageId} ");
var messageParts = args.Message.MessageId.Split('|');
var kind = messageParts[0];
var user = messageParts[1];
var param = messageParts[2];

if (user != userName)
{
    Console.WriteLine($"Caution handling message of another user: {user}");
}

switch (kind)
{
    case ToDefer:
        await args.DeferMessageAsync(args.Message);
        var scheduleTime = args.Message.EnqueuedTime + oneSecondMoreThanLockDuration;
        await SendMessagesAsync(UnDefer, args.Message.SequenceNumber.ToString(), scheduleTime);
        break;
    case UnDefer:
        var deferredMessage = await receiver.ReceiveDeferredMessageAsync(long.Parse(param));
        Console.WriteLine($"Deferd message {deferredMessage.MessageId} processed");
        await args.CompleteMessageAsync(args.Message);
        try
        {
            // THIS WOULD DELETE THE MESSAGE IF THE LOCK WAS STILL ON
            await receiver.CompleteMessageAsync(deferredMessage);
        }
        catch (Exception ex)
        {
            Console.WriteLine($"Ignoring {ex.Message}");
        }

        break;
}

}
processor.ProcessMessageAsync += ProcessMessageAsync;
processor.ProcessErrorAsync += eventArgs =>
{
Console.WriteLine(eventArgs.Exception.Message);
return Task.CompletedTask;

};
var cancellationTokenSource = new CancellationTokenSource();
await processor.StartProcessingAsync(cancellationTokenSource.Token);
await SendMessagesAsync(ToDefer);
Console.ReadKey();
cancellationTokenSource.Cancel();`

I've only been able to repro the issue when I put a breakpoint and intentionally wait for the lock duration to expire. When receiving the deferred messages using the receiver, there is no automatic lock renewal happening, so you would need to complete the message within 5 seconds. If I let the application run without breaking, it completes the deferred message without throwing an exception. Can you try increasing the lock duration to see if you are still seeing the issue?

I've only been able to repro the issue when I put a breakpoint and intentionally wait for the lock duration to expire. When receiving the deferred messages using the receiver, there is no automatic lock renewal happening, so you would need to complete the message within 5 seconds. If I let the application run without breaking, it completes the deferred message without throwing an exception. Can you try increasing the lock duration to see if you are still seeing the issue?

Did you adapt the value of "oneSecondMoreThanLockDuration" to match your topic's subscription settings? You wouldn't need to set a breakpoint if you did. At least that's how it behave on my side.

I've only been able to repro the issue when I put a breakpoint and intentionally wait for the lock duration to expire. When receiving the deferred messages using the receiver, there is no automatic lock renewal happening, so you would need to complete the message within 5 seconds. If I let the application run without breaking, it completes the deferred message without throwing an exception. Can you try increasing the lock duration to see if you are still seeing the issue?

Did you adapt the value of "oneSecondMoreThanLockDuration" to match your topic's subscription settings? You wouldn't need to set a breakpoint if you did. At least that's how it behave on my side.

Yes, I actually updated the lock duration to be 5 seconds to match your description.

Here's my configuration, maybe there something in there which leads to this undesirable behavior.

And with that I always get the same result

authors: JoshLove-msft serge-portima


text:
[Eventhub] ConsumerCanReadFromAllPartitionsStartingWithLatest failing in nightly runs

Eventhub nightly test runs are failing with:

Error message
The cancellation token should not have been signaled.
Expected: False
But was:  True
Stack trace
at Azure.Messaging.EventHubs.Tests.EventHubConsumerClientLiveTests.ConsumerCanReadFromAllPartitionsStartingWithLatest() in D:\a_work\1\s\sdk\eventhub\Azure.Messaging.EventHubs\tests\Consumer\EventHubConsumerClientLiveTests.cs:line 2305
at Azure.Messaging.EventHubs.Tests.EventHubConsumerClientLiveTests.ConsumerCanReadFromAllPartitionsStartingWithLatest() in D:\a_work\1\s\sdk\eventhub\Azure.Messaging.EventHubs\tests\Consumer\EventHubConsumerClientLiveTests.cs:line 2314
at Azure.Messaging.EventHubs.Tests.EventHubConsumerClientLiveTests.ConsumerCanReadFromAllPartitionsStartingWithLatest() in D:\a_work\1\s\sdk\eventhub\Azure.Messaging.EventHubs\tests\Consumer\EventHubConsumerClientLiveTests.cs:line 2317
at NUnit.Framework.Internal.TaskAwaitAdapter.GenericAdapter1.BlockUntilCompleted()    at NUnit.Framework.Internal.MessagePumpStrategy.NoMessagePumpStrategy.WaitForCompletion(AwaitAdapter awaiter)    at NUnit.Framework.Internal.AsyncToSyncAdapter.Await(Func1 invoke)
at NUnit.Framework.Internal.Commands.TestMethodCommand.RunTestMethod(TestExecutionContext context)
at NUnit.Framework.Internal.Commands.TestMethodCommand.Execute(TestExecutionContext context)
at NUnit.Framework.Internal.Execution.SimpleWorkItem.<>c__DisplayClass4_0.b__0()
at NUnit.Framework.Internal.ContextUtils.<>c__DisplayClass1_0`1.b__0(Object _)

at Azure.Messaging.EventHubs.Tests.EventHubConsumerClientLiveTests.ConsumerCanReadFromAllPartitionsStartingWithLatest() in D:\a_work\1\s\sdk\eventhub\Azure.Messaging.EventHubs\tests\Consumer\EventHubConsumerClientLiveTests.cs:line 2305

For more details check here:

https://dev.azure.com/azure-sdk/internal/_build/results?buildId=1996003&view=results

@jsquire for notification.

This is a service timeout; they're intermittent but expected.

authors: AFengKK jsquire


text:
Azure Container Registry client library - Which TokenCredential to use for token/password?

Library name and version
Azure.Containers.ContainerRegistry 1.1.1
Query/Question
I am trying to use Azure.Containers.ContainerRegistry with token/password that I create in Azure Portal under my ACR->Tokens, where I have created a token and a custom Scope Map. But I cannot from the doucmentation, or from the default sub-classes of TokenCredential figure out which is the correct one to use. I have successfully used just the token/password combination with another library, so I would have thought there would be one that only required that data, but I cannot find it. Could you point me in the right direction?
Azure.Identity.AuthorizationCodeCredential
Azure.Identity.AzureCliCredential
Azure.Identity.AzureDeveloperCliCredential
Azure.Identity.AzurePowerShellCredential
Azure.Identity.ChainedTokenCredential
Azure.Identity.ClientAssertionCredential
Azure.Identity.ClientCertificateCredential
Azure.Identity.ClientSecretCredential
Azure.Identity.DefaultAzureCredential
Azure.Identity.DeviceCodeCredential
Azure.Identity.EnvironmentCredential
Azure.Identity.InteractiveBrowserCredential
Azure.Identity.ManagedIdentityCredential
Azure.Identity.OnBehalfOfCredential
Azure.Identity.SharedTokenCacheCredential
Azure.Identity.UsernamePasswordCredential
Azure.Identity.VisualStudioCodeCredential
Azure.Identity.VisualStudioCredential
Azure.Identity.WorkloadIdentityCredential

Thanks
Environment
No response

@edespong - You can use the DefaultAzureCredential. It attempts authentication through the following mechanisms, in this order, and stops when one succeeds:

Environment -> Workload Identity -> Managed Identity -> Visual Studio -> Visual Studio Code -> Azure CLI -> Azure PowerShell -> Azure Developer CLI -> Interactive browser.

An example of its usage can be found here. To learn more about DefaultAzureCredential, please refer to the documentation here.

Maybe I am misunderstanding, or my question was not clear.
I have an application where I want to use the client library, This application will run on machines without any Entra connection, Visual Studio, or PowerShell. I do not see how DefaultAzureCredential could work, as I will have generated differnet tokens for the different machines with different scopes that the client cannot possibly know unless the token is explicitly provided. The Environment and UsernamePasswordCredential looked promising, but they seem to be connected to Entra rather than the ACR-specific tokens.

Update, I tried creating my own implementation where I fetch a JWT bearer token for the ACR, like below but it fails due to:
{"errors":[{"code":"UNAUTHORIZED","message":"token invalid audience: the received access token has been obtained for wrong audience or resource \"myregistry.azurecr.io\". It should exactly match with one of the allowed audiences \"https://management.core.windows.net/\", \"https://management.core.windows.net\", \"https://management.azure.com/\", \"https://management.azure.com\"."}]}

Is there an underlying assumption that the client library may only be used if you have an Entry connection in some way?
public class AcrTokenCredential(string registry, string scope, string tokenName, string password) : TokenCredential {

    public override AccessToken GetToken(TokenRequestContext requestContext, CancellationToken cancellationToken) {
        return GetTokenAsync(requestContext, cancellationToken).Result;
    }

    public override async ValueTask<AccessToken> GetTokenAsync(TokenRequestContext requestContext, CancellationToken cancellationToken) {
        System.Net.Http.HttpClient httpClient = new();
        string basicCredential = $"{tokenName}:{password}";
        string basicBase64 = Convert.ToBase64String(System.Text.Encoding.UTF8.GetBytes(basicCredential));
        httpClient.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Basic", basicBase64);
        var response = await httpClient.GetAsync($"https://{registry}/oauth2/token?service={registry}&scope={scope}", cancellationToken);
        string content = await response.Content.ReadAsStringAsync(cancellationToken);
        TokenResponse tokenResponse = JsonSerializer.Deserialize<TokenResponse>(content)!;
        return new AccessToken(tokenResponse.access_token, DateTimeOffset.Now.AddMinutes(10));
    }

    private record TokenResponse(string access_token);
}

@jsquire is there a bot rule that appends identity here that we should refine? I saw a couple of incidents last week as well for issues like these?

@pallavit:  It's an AI prediction.   The match came back as Identity above 92% confidence in the prediction.   We will see occasional mistakes; accuracy improves every couple of months when we retrain against the corrected data.

@edespong - Thank you for reaching out. Currently, our Azure Container Registry SDK only supports Microsoft Entra ID authentication, formerly known as "Azure Active Directory". Token/Password authentication is not supported at the moment. While there are no immediate plans to add support for Token/Password authentication in the ACR SDK this semester, we appreciate your feedback and can consider it as a feature request in our backlog.
CC: @sandeep-sen

authors: ShivangiReja edespong jsquire pallavit


text:
[BUG] Azure.ResourceManager.Orbital SDK Issues

Library name and version
Azure.ResourceManager.Orbital
Describe the bug
We recently discovered 2 issues with the .NET SDK https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/orbital/Azure.ResourceManager.Orbital

EventHubUri is a misnomer, we expected a resource id, but the SDK expects a URI
EndPoint.ipAddress: - ip-address is not nullable in .NET TSDK

Expected behavior

EventHubUri accepts a resource id in the .NET SDK
Endpoint.ipAddress allows null for ip address

Actual behavior

EventHubUri does not accept a resource id in the .NET SDK
Endpoint.ipAddress does not allow null for ip address

Reproduction Steps

Create a contact profile resource with a eventhub resource id
Create a contact profile with null value for Endpoint.ipAddress

Environment
No response

Thank you for your feedback.  Tagging and routing to the team member best able to assist.

Workaround this from sdk side:

Rename it to eventHubId and add attribute "format": "arm-id".
Remove ipAddress out of required azure-rest-api-specs/specification/orbital/resource-manager/Microsoft.Orbital/stable/2022-11-01/orbital.json at main · Azure/azure-rest-api-specs (github.com)

Hi @thisisdevanshu ,  We have conducted multiple tests on PUT/subscriptions/<>/resourceGroups/<>/providers/Microsoft. Organizational/contactprofiles/<>? API version=2022-03-01 and found that the current API version does not support null values for the ipAddress property. If the value passed in is null, a RequestFailedException error will be thrown. May I ask if the version you are using is 2022-03-01? And how did you determine that the ipAddress property can be null? Look forward to your reply.


text:
[BUG] Missing StartWithHttpMessagesAsync in released ContainerInstance package

Describe the bug
The IContainerGroupsOperations in the published nuget package (2.0.0) doesn't have StartWithHttpMessagesAsync
Expected behavior
All the available operations should be present
Setup (please complete the following information):

OS: Windows 10
IDE : Visual Studio 2019
nuget 2.0.0

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

[ x ] Bug Description Added
[ n/a ] Repro Steps Added
[ x ] Setup information Added

Thank you for opening this issue! We are routing it to the appropriate team for follow up.
@shahabhijeet, can you kindly take a look at this?

If you look at the rest api definition there is a start container api but the library does not reference it. I think it's a simple case of just regenerating this library against the complete api definition. (From memory as it was a few weeks ago I raised this)

@samkreter,  please regenerate source code from swagger spec for containerinstance since StartWithHttpMessagesAsync was missed in nuget package 2.0.0? The process is available at https://github.com/Azure/adx-documentation-pr/blob/master/engineering/adx_netsdk_process.md#sdk-generation-from-updated-spec

@samkreter, any update?

I am currently running into this issue and thought maybe I could follow the process myself, since it does not seem complicated. Unfortunately it appears that the link is either dead or I do not have access.
@erich-wang is there a public url where I can see the process so that I may submit a pull request?

Close the issue as new package was released two months ago at https://www.nuget.org/packages/Microsoft.Azure.Management.ContainerInstance/3.0.0.

authors: erenes erich-wang nrandell praries880


text:
Create Test Automation for Azure.Identity running in Azure Hosted Environments

Epic: Create Test Automation for Azure.Identity running in Azure Hosted Environments

[ ] Azure Cloud Shell

[ ] /sdk/e2e test assets checked-in so that they can be referenced from Vendor Instructions
[ ] Language-specific instructions added to https://github.com/Azure/azure-sdk/issues/795

[ ] Azure VM

[ ] /sdk/e2e test assets checked-in so that they can be referenced from Vendor Instructions
[ ] Language-specific instructions added to https://github.com/Azure/azure-sdk/issues/815

[ ] Azure Kubernetes

[ ] /sdk/e2e test assets checked-in so that they can be referenced from Vendor Instructions
[ ] Language-specific instructions added to https://github.com/Azure/azure-sdk/issues/814

[ ] Azure WebJobs

[ ] /sdk/e2e test assets checked-in so that they can be referenced from Vendor Instructions
[ ] Language-specific instructions added to https://github.com/Azure/azure-sdk/issues/902

Updating issue as Functions and App Services have been moved to the backlog

dupe of #37471

authors: christothes joshfree


text:
Form Recognizer: Consider the design of FieldValue type

FieldValue type properties
Should this exposed as a generic? a dynamic?
Status: Won’t change. Introducing generics here would force user to do type checks or reflection. See IO.BinaryReader or Text.Json.JsonElement for similar implementations.
Priority: low
Effort: high

This will be confusing for users exploring the API in Intellisense:

Design prototyped. We can reopen this issue or create new ones from UX feedback. @mayurid
https://github.com/Azure/azure-sdk-for-net/issues/10333 replaces this.


text:
Enable transaction test

The service fix is deployed in US regions.

/azp run net - servicebus - tests


text:
[Sql] Bump API version

Contributing to the Azure SDK
Please see our CONTRIBUTING.md if you are not familiar with contributing to this repository or have questions.
For specific information about pull request etiquette and best practices, see this section.

API change check
APIView has identified API level changes in this PR and created following API reviews.
Azure.ResourceManager.Sql

authors: azure-sdk fengzhou-msft


text:
Convert pipelines to extend from 1es pipeline templates

Updating main ci/test pipelines to extend from 1es pipeline templates

For a general scan nothing bad jumps out at me. Of course, the details are in all the testing you are doing.

/azp run net - core - ci

/check-enforcer override
All CI is green, overriding to ignore the cancelled release pipelines I ran (since I don't want to release them)

authors: benbp weshaggard


text:
[Storage] [DataMovement] Preserve Share Files Properties and Attributes

Adding the ability to preserve properties and attributes of Share Files. It includes:

Cache Control
Content Disposition
Content Encoding
Content Language
Content Type
Metadata
NtfsFileAttributes
FileChangedOn
FileCreatedOn
FileLastWrittenOn

Seemingly unrelated change I made:

Added mock method to take in NtfsFileAttributes object instead of a string, to avoid unnecessary double parsing for tests

Things not yet done in this PR:

Preserving permission and setting permissions key in the ShareClient
Preserving properties and attributes for subdirectories

API change check
API changes are not detected in this pull request.

authors: amnguye azure-sdk


text:
Enable WriteCore feature for managedserviceidentity

We need to enable WriteCore feature to managedserviceidentity. Therefore add use-write-core: true and do a regen.

API change check
APIView has identified API level changes in this PR and created following API reviews.
Azure.ResourceManager.ManagedServiceIdentities


text:
Fixed customization for Hive/Oozie metastores

Description

There was a bug that would occur if a user specified an Hive/Oozie metastore when they had configurations like hive-site. This change will fix this scenario so the configurations are merged, but still raise an exception when there are duplicate configurations. There is no regeneration necessary for this change.

This checklist is used to make sure that common guidelines for a pull request are followed.

[ ] Please add REST spec PR link to the SDK PR
[ ] I have read the contribution guidelines.
[ ] The pull request does not introduce breaking changes.

General Guidelines

[ ] Title of the pull request is clear and informative.
[ ] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[ ] Pull request includes test coverage for the included changes.

SDK Generation Guidelines

[ ] If an SDK is being regenerated based on a new swagger spec, a link to the pull request containing these swagger spec changes has been included above.
[ ] The generate.cmd file for the SDK has been updated with the version of AutoRest, as well as the commitid of your swagger spec or link to the swagger spec, used to generate the code.
[ ] The *.csproj and AssemblyInfo.cs files have been updated with the new version of the SDK.

Thanks for the review. I've resolved the comment.

@dsgouda: We've decided to delay the other changes until later. You can merge these current changes.


text:
BlobClient().upload_blob(file) gets a 403

Code that is run
log.info("Begin Upload: {file} as {name} {type}".format(file=repr(file), name=name, type=content_type))
write_url = self._create_write_url(name) # produces a full url with SAS token with write permissions.
content_settings = ContentSettings(
    content_type=content_type,
    content_disposition="inline",
)
try:
    with BlobClient(write_url, logging_enable=True, retry_policy=ExponentialRetry(initial_backoff=2, retry_total=10)) as client:
        if type(file) is str:
            with open(file, 'rb') as f:
                client.upload_blob(f, content_settings=content_settings)
        else:
            client.upload_blob(file, content_settings=content_settings)
except:
    log.exception("Failure while uploading {file}".format(file=repr(file)))

Server Response:
<?xml version="1.0" encoding="utf-8"?>
<Error>
  AuthenticationFailed
  <Message>
    Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
    RequestId:***
    Time:2019-07-23T23:00:12.1569153Z
  </Message>
  <AuthenticationErrorDetail>Signature fields not well formed.</AuthenticationErrorDetail>
</Error>

The blob url I passed is valid and works with manual PUT requests. The debug logs show that the sas token on the url is url-decoded and the final PUT request is made using the (invalid) decoded sas token signature.

Hi @alexperovich thanks for reporting the bug. CCing @annatisch & @mayurid to help route this further

Thanks @alexperovich - we are aware of this and have a fix under review now.
This fix should be released in our next preview release.
The specific line with the fix is here:
https://github.com/Azure/azure-sdk-for-python/pull/6442/files#diff-19efb00002f2454f98f0c2539aeae6f3R274
Thanks again!

Hi @alexperovich - Preview 2 of storage has been released that should resolve this issue.
Can you confirm whether that's the case?
https://pypi.org/project/azure-storage-blob/12.0.0b2/

I'm assuming this is now fixed - please let me know if it needs to be reopened :)

I haven't had a chance to try it yet. I will let you know if I this issue isn't fixed.

authors: alexperovich annatisch loarabia

repo: Azure/azure-sdk-for-python


text:
PublicIPAddressSku - returned object instead of value

Hi
while creating public IP using python SDK, as a result I should get the python dict.
However, in one filed - sku - instead of getting an actual value (in this situation, it should be another dict) i'm getting object type.
   pubip_args = {
        'location': res_region,
        'sku': {
            'name': 'basic'
        },
        'public_ip_allocation_method': 'Dynamic',
        'public_ip_address_version': 'IPv4',

    }
    async_operation = net_client.public_ip_addresses.create_or_update(RG_name, 'vpn-pub-ip', pubip_args)
    async_operation.wait()
    res = async_operation.result()

Printing the res variable:
{'additional_properties': {}, 'id': '/subscriptions/<id>/resourceGroups/vpn-network/providers/Microsoft.Network/publicIPAddresses/vpn-pub-ip', 'name': 'vpn-pub-ip', 'type': 'Microsoft.Network/publicIPAddresses', 'location': 'northeurope', 'tags': None, 'sku': <azure.mgmt.network.v2019_06_01.models._models_py3.PublicIPAddressSku object at 0x7f2a535d0a58>, 'public_ip_allocation_method': 'Dynamic', 'public_ip_address_version': 'IPv4', 'ip_configuration': None, 'dns_settings': None, 'ddos_settings': None, 'ip_tags': [], 'ip_address': None, 'public_ip_prefix': None, 'idle_timeout_in_minutes': 4, 'resource_guid': '2a607b2a-0842-4b74-96cf-a2baa00d769c', 'provisioning_state': 'Succeeded', 'etag': 'W/"f29c8d91-a8e6-4763-b109-bfc197ed7a6a"', 'zones': None}

Problematic snippet:
'sku': <azure.mgmt.network.v2019_06_01.models._models_py3.PublicIPAddressSku object at 0x7f2a535d0a58>,

@gruch4  thanks for your patience, this now has the attention of the SDN team and we will update you soon.

Hi service team, would you help to confirm whether the sku should be an object or a simple value?

authors: allegradomel changlong-liu gruch4


text:
[Storage] Fix newline in Doc for DelimitedTextDialect

Attempting to address #23008

Doc update and ci failures are unrelated. Will override.

/check-enforcer override


text:
[AutoPR logic/resource-manager] Adding in description of available filter options

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/3263

Codecov Report

Merging #2777 into restapi_auto_logic/resource-manager will not change coverage.
The diff coverage is 100%.

@@                         Coverage Diff                          @@
##           restapi_auto_logic/resource-manager    #2777   +/-   ##
====================================================================
  Coverage                                55.89%   55.89%
====================================================================
  Files                                     7161     7161
  Lines                                   150693   150693
====================================================================
  Hits                                     84227    84227
  Misses                                   66466    66466

Impacted Files
Coverage Δ

...azure/mgmt/logic/operations/sessions_operations.py
20.43% <ø> (ø)
:arrow_up:

...zure/mgmt/logic/operations/workflows_operations.py
20.6% <ø> (ø)
:arrow_up:

...gic/azure/mgmt/logic/operations/maps_operations.py
19.13% <ø> (ø)
:arrow_up:

...ure/mgmt/logic/operations/agreements_operations.py
19.13% <ø> (ø)
:arrow_up:

...perations/workflow_trigger_histories_operations.py
25% <ø> (ø)
:arrow_up:

.../azure/mgmt/logic/operations/schemas_operations.py
19.13% <ø> (ø)
:arrow_up:

.../mgmt/logic/operations/workflow_runs_operations.py
25% <ø> (ø)
:arrow_up:

...ogic/operations/workflow_run_actions_operations.py
22.78% <ø> (ø)
:arrow_up:

...azure/mgmt/logic/operations/partners_operations.py
19.13% <ø> (ø)
:arrow_up:

azure-mgmt-logic/azure/mgmt/logic/version.py
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d3ae137...faf09bd. Read the comment docs.

This PR has been merged into https://github.com/Azure/azure-sdk-for-python/pull/2782

authors: AutorestCI codecov-io


text:
[AutoPR] automation/resource-manager

Created to accumulate context: automation/resource-manager

(message created by the CI based on PR content)


text:
Swagger APIView should insert comments for all linter messages into the APIView

Linter messages from azure-openapi-validator should be inserted as automated comments in APIView.

Swagger APIView parser must run the linter as a subprocess and capture the results and then insert these into the APIView.

repo: Azure/azure-sdk-tools


text:
Full path override support for code owner matching

In order to help with special pipelines like aggregate-reports that live
in a general path we should allow an codeowner entry with the exact path
to that file so that it can override the any other paths like /eng/ etc.

@danieljurek what do you think about this approach to handle aggregate-reports? I did this mostly so we don't have to artificially move around the yml files but we can still assign owners to that pipeline by directly specifying the full path to the yml file.

The use of .Where in matching file paths will include ALL matches. This diverges from the original logic of GitHub where the LAST match is taken. It is correct to use .Where here to stay consistent with how we are also configuring failure notifications in the directory matching case.

For the exact path we are taking only the one match but you are correct that it could be more then one. Should we change the logic completely and only take the last match like what github does? I like being consistent and so that might be better here.

/azp run automation - build-failure-notification-subscriptions

/azp run automation - build-failure-notification-subscriptions

/azp run automation - build-failure-notification-subscriptions

/azp run automation - build-failure-notification-subscriptions

/azp run automation - build-failure-notification-subscriptions

/azp run automation - build-failure-notification-subscriptions


text:
leaked application settings between app service deployment slots?

context: I'm running a handful of functions on an App Service instance. I have two deployment slots: the primary and another called staging. Running Node 7.10 on my App Service instance. Functions runtime 1.0.10917
On a couple of occasions, I've run into some very odd behavior (one which is discussed in #1246) where it appears that functions/webjobs running on the same schedule, in differently deployment slots, appear to be leaking application settings.
For example, the most recent issue I ran into was with a payment processing job that essentially runs through and charges customers for services rendered. I woke up to find that my customers were sent two emails:

That there payment was successfully processed
That there payment failed

It's not uncommon to have a failed payment, but the issue here is that each customer was sent two emails for a single transaction. The transaction should either succeed or fail, not both. When I looked into the webjob log, indeed, the job only ran once on the production slot. However, it also ran on the staging slot as well. Given that it's the same job, on the same schedule, just pointing to a different database, this was expected.
The second part of this that was fishy was that in the email for the failed payments, we show the user the reason for failure, which is usually something from the payment processor. In this case, it was the following error message:
error: insert into "transaction" ("amount", "business_id", "entity_id", "entity_type", "kind", "type", "user_id") values ($1, $2, $3, $4, $5, $6, $7) - duplicate key value violates unique constraint "transaction_pkey"

The insert being referenced here occurs after a payment succeeds from our payment processor; we add a payment transaction to our own log. The interesting thing about the above statement is that while it cites a duplicated primary key violation, you'll notice that the primary key isn't something we've generated and tried to insert -- it's assigned from the sequence. From what I've read, it seems that this is due to a race condition where two inserts have tried to run simultaneously.
This seems to point to the webjob running two instance simultaneously. The only thing I can think is that somehow that the two identical jobs in different deployment slots were somehow using the same application settings even though they are clearly different.
That said, I want to point out a couple of important facts:

This particular job has been running for months without any major issues. The last time something like this happened was discussed in #1246.

I had just done a environment swap a couple of hours before all of this happened.

I'm hoping someone might be able to give me some insight on how the functions runtime works within the context of using it directly on an App Service instance rather than as a Functions app. But more importantly, is the scenario I described above something that others have encountered? If not, is it a possibility?
I'm really at a loss and this issue has caused a lot of problems that have made customers unhappy and resulted in a lot of lost time trying to clean up.
Thank you in advance for your help!

:bump:
Anyone have any insight on this?

Its difficult for us to reason about what is happening here because of your unusual setup. We would at least have a bunch of logs on our side if you were running in a Function App. I understand that this issue does not repro reliably, but can you try to somehow repro from a normal function app?

Thanks for the reply, Paul!
When you say that my setup is unusual, are you referring to the fact that I'm using the runtime as part of my app service instance?
Do you feel like I might gain some better stability and logging facilities if I moved to a functions app?
On Jun 5, 2017, 12:16 PM -0600, Paul Batum notifications@github.com, wrote:

Its difficult for us to reason about what is happening here because of your unusual setup. We would at least have a bunch of logs on our side if you were running in a Function App. I understand that this issue does not repro reliably, but can you try to somehow repro from a normal function app?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Yeah I'm referring to the fact that you're not using a Function App. We have systems that collect logs and telemetry for Function Apps and these help us troubleshoot problems, and potentially there are more variables in play when you are reusing the runtime code yourself. Its just really difficult for us to make headway on this type of problem given your current config.
Any thoughts on what a minimal repro would look like? What if you created a brand new function app with app insights enabled, added a timer trigger that logs the value of a sticky app setting, and then set up a slot with a different value of that app setting and then let them run for a week or so, swapping often? You could then query your logs in app insights for any log entries that appear to have ran in stage but used the prod setting or vice versa. If it repros in that configuration we'd have something very concrete to investigate.

Great to know. Do you feel that in the future, using the runtime in an App Service instance will be deprecated?
As for reproducing, that sounds like a good plan. I'll create both a functions app and a plain app service instance with some custom logging to see if anything changes.
I realize the situation seems a bit bizarre and most certainly an outlier, but I swear I'm not crazy 😄

Apologies for being glib, but for something to be "deprecated" it first has to be "supported" 😈 . But let me be clear to avoid any confusion - we absolutely intend to continue supporting Function Apps on dedicated app service plans. What we don't officially support is grabbing our code or our binaries and running it yourself. But its open source so we're not going to stop you, and when its not particularly onerous we'll do our best to make sure you're still successful.

Fair enough!
I suppose my confusion comes from the fact that as far as I can tell, it isn't clear whatsoever that using the binaries isn't a supported method. For example, this repo's wiki goes through using the method I use as a deployment option. Not being snarky by any means, but it would be good to make this more clear if this method of usage indeed is not supported.
I actually started using this SDK before Functions apps were considered a thing (publicly) and was pointed to this repo as a way to use the various storage bindings, etc within my app service as Webjobs. That said, I am now officially confused when it comes to the difference between a webjob and function 😄 Are Webjobs being sunsetted as well?

By "officially support" what I'm really referring to is that if you were paying for Azure Support, whether you could file a ticket about an issue when running in this configuration.
I can understand your confusion. The line between webjobs and functions can be blurry sometimes. Let me try to clear it up.
Nothing is being sunsetted. We continue to support webjobs. None of what I'm communicating here is a change in direction. One obvious difference between the functions packages and the webjobs packages is that the functions packages are still pre-release. There is no RTM release of those packages. For an example, see here.

Thanks for the clarification on support!
That said, I'm still a little hazy on the difference between webjobs and functions. Can you discuss any other significant difference?
Thank you!

This might help:
https://docs.microsoft.com/en-us/azure/azure-functions/functions-compare-logic-apps-ms-flow-webjobs

Closing as answered/stale.

authors: fabiocav jamesdixon paulbatum

repo: Azure/azure-webjobs-sdk-script


text:
Can this be used for things other than keyvault?

I'm trying to use AZWI to provide az login credentials to an Azure CLI pod inside kubernetes (using the Azure CLI docker image) and not sure if this is possible and if so how to actually do it. The documentation only talks about how to use AZWI to pull secrets from keyvault and it's unclear if other use cases are possible or how do to them.

If you need to use it with azure cli, you can run the following command in the pod
az login --federated-token "$(cat  $AZURE_FEDERATED_TOKEN_FILE)" --service-principal -u $AZURE_CLIENT_ID -t $AZURE_TENANT_ID

It works with all azure resources. We only have examples here with the key vault SDK reference. It would be similar with other azure resources.

@aramase doesn't work.. when I try to add that line it just hangs forever with no output

authors: aramase tdevopsottawa

repo: Azure/azure-workload-identity


text:
Implement scopes for modules

Implementation of https://github.com/Azure/bicep/blob/main/docs/spec/resource-scopes.md for modules only.

Closed by https://github.com/Azure/bicep/pull/771

repo: Azure/bicep