[ARM] Fix #31581: az deployment group create: Fix error message being hidden when template validation fails #31965
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Starting from roughly 1-2 months ago, az-cli started to not show the actual validation error if ARM/Bicep template validation fails, and instead just shows "The content for this response was already consumed". This was raised in issue Azure#31581. A fix was attempted in 6c88666, but this did not resolve the issue, because the "response was already consumed" exception was a RuntimeError (as per request 2.32.4), and the commit only caught StreamClosedError and StreamConsumedError. http_error.response.internal_response is a requests.models.Response, and StreamClosedError and StreamConsumedError are defined in azure sdk, so the request module would never throw them anyway. A fix could be to catch the RuntimeError, however upon further thinking and looking at the discussion in issue Azure#31709 I decided to simply remove the reference to internal_response, as was already suggested by @anthony-c-martin . The original source of the internal_response access seems to be PR Azure#18165 which was made to fix issue Azure#18134 ("[ARM] template validation commands show unexpected error messages for invalid templates"), however the original error was likely caused by using `.text` rather than `.text()` (not tested it, but I think it's likely given the shown message in the issue), so assuming there's no other reason why internal_response was used, I (also) think we should probably remove it. Testing: (.az-cli.venv) (1) (1.05s) tingmao-laptop (mao) /t/arm > az deployment group create -g test -n test --template-file arm.json /home/mao/.az-cli.venv/bin/az:4: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html __import__('pkg_resources').require('azure-cli==2.75.0') The content for this response was already consumed .. fix applied here .. (.az-cli.venv) (1) (1.198s) tingmao-laptop (mao) /t/arm > az deployment group create -g test -n test --template-file arm.json /home/mao/.az-cli.venv/bin/az:4: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html __import__('pkg_resources').require('azure-cli==2.75.0') {"code": "ResourceGroupNotFound", "message": "Resource group 'test' could not be found."} I also tested that it correctly display more complex error with causes in .details: {"code": "InvalidTemplateDeployment", "message": "The template deployment 'tingmao-vnet-change-test' is not valid according to the validation procedure. The tracking id is 'fc739efb-5f36-44aa-ae62-bd4903b7abcf'. See inner errors for details."} Inner Errors: {"code": "SecurityRuleParametersMissingForPortRanges", "target": "/subscriptions/.../resourceGroups/.../providers/Microsoft.Network/networkSecurityGroups/...", "message": "Required security rule parameters are missing for security rule with Id: /subscriptions/.../resourceGroups/.../providers/Microsoft.Network/networkSecurityGroups/.../securityRules/AnyOutbound. Security rule must specify either SourcePortRange or SourcePortRanges."} Fixes: Azure#31581 Original "response consumed" exception stack trace: Traceback (most recent call last): File "/home/mao/.az-cli.venv/lib/python3.12/site-packages/azure/core/polling/base_polling.py", line 636, in initialize _raise_if_bad_http_status_and_method(self._initial_response.http_response) File "/home/mao/.az-cli.venv/lib/python3.12/site-packages/azure/core/polling/base_polling.py", line 156, in _raise_if_bad_http_status_and_method raise BadStatus("Invalid return status {!r} for {!r} operation".format(code, response.request.method)) azure.core.polling.base_polling.BadStatus: Invalid return status 400 for 'POST' operation The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/mao/src/github.com/Microsoft/azure-cli/src/azure-cli/azure/cli/command_modules/resource/custom.py", line 653, in _deploy_arm_template_at_resource_group validation_poller = mgmt_client.begin_validate(resource_group_name, deployment_name, deployment) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/mao/.az-cli.venv/lib/python3.12/site-packages/azure/core/tracing/decorator.py", line 119, in wrapper_use_tracer return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/mao/.az-cli.venv/lib/python3.12/site-packages/azure/mgmt/resource/deployments/operations/_deployments_operations.py", line 5755, in begin_validate return LROPoller[_models.DeploymentValidateResult]( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/typing.py", line 1157, in __call__ result = self.__origin__(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/mao/.az-cli.venv/lib/python3.12/site-packages/azure/core/polling/_poller.py", line 229, in __init__ self._polling_method.initialize(client, initial_response, deserialization_callback) File "/home/mao/.az-cli.venv/lib/python3.12/site-packages/azure/core/polling/base_polling.py", line 641, in initialize raise HttpResponseError(response=initial_response.http_response, error=err) from err azure.core.exceptions.HttpResponseError: (InvalidTemplateDeployment) The template deployment 'tingmao-vnet-change-test-2' is not valid according to the validation procedure. The tracking id is '749ad07e-442e-40dc-a8cb-eaf38349f590'. See inner errors for details. Code: InvalidTemplateDeployment Message: The template deployment 'tingmao-vnet-change-test-2' is not valid according to the validation procedure. The tracking id is '749ad07e-442e-40dc-a8cb-eaf38349f590'. See inner errors for details. Exception Details: (SecurityRuleParametersMissingForPortRanges) Required security rule parameters are missing for security rule with Id: /subscriptions/.../resourceGroups/.../providers/Microsoft.Network/networkSecurityGroups/.../securityRules/AnyOutbound. Security rule must specify either SourcePortRange or SourcePortRanges. Code: SecurityRuleParametersMissingForPortRanges Message: Required security rule parameters are missing for security rule with Id: /subscriptions/.../resourceGroups/.../providers/Microsoft.Network/networkSecurityGroups/.../securityRules/AnyOutbound. Security rule must specify either SourcePortRange or SourcePortRanges. Target: /subscriptions/.../resourceGroups/.../providers/Microsoft.Network/networkSecurityGroups/... During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/mao/src/github.com/Microsoft/azure-cli/src/azure-cli/azure/cli/command_modules/resource/_utils.py", line 94, in _build_http_response_error_message error_txt = http_error.response.internal_response.text ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/mao/.az-cli.venv/lib/python3.12/site-packages/requests/models.py", line 926, in text if not self.content: ^^^^^^^^^^^^ File "/home/mao/.az-cli.venv/lib/python3.12/site-packages/requests/models.py", line 897, in content raise RuntimeError("The content for this response was already consumed") RuntimeError: The content for this response was already consumed Types traced: http_error: <class 'azure.core.exceptions.HttpResponseError'> .../site-packages/azure/core/exceptions.py http_error.response: <class 'azure.core.rest._requests_basic.RestRequestsTransportResponse'> .../site-packages/azure/core/rest/_requests_basic.py http_error.response.internal_response: <class 'requests.models.Response'> .../site-packages/requests/models.py Signed-off-by: Tingmao Wang <[email protected]>
micromaomao
requested review from
zhoxing-ms,
jsntcy and
yanzhudd
as code owners
|
Validation for Azure CLI Full Test Starting...
Thanks for your contribution! |
|
Hi @micromaomao, |
|
Validation for Breaking Change Starting...
Thanks for your contribution! |
|
Thank you for your contribution! We will review the pull request and get back to you soon. |
|
The git hooks are available for azure-cli and azure-cli-extensions repos. They could help you run required checks before creating the PR. Please sync the latest code with latest dev branch (for azure-cli) or main branch (for azure-cli-extensions). pip install azdev --upgrade
azdev setup -c <your azure-cli repo path> -r <your azure-cli-extensions repo path>
|
There was a problem hiding this comment.
Pull Request Overview
This PR fixes an issue where Azure CLI was hiding actual validation error messages when ARM/Bicep template validation fails, instead showing "The content for this response was already consumed". The fix removes problematic code that accessed the internal response object, which was causing the RuntimeError.
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| @@ -87,12 +87,7 @@ def _build_preflight_error_message(preflight_error): | |||
|
|
|||
|
|
|||
| def _build_http_response_error_message(http_error): | |||
There was a problem hiding this comment.
The function now directly calls http_error.response.json()['error'] without any error handling. Consider adding appropriate exception handling for cases where the response might not contain valid JSON or the expected 'error' key structure.
Copilot uses AI. Check for mistakes.
There was a problem hiding this comment.
This is existing behaviour.
microsoft-github-policy-service
bot
added
the
ARM
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Starting from roughly 1-2 months ago, az-cli started to not show the actual validation error if ARM/Bicep template validation fails, and instead just shows "The content for this response was already consumed". This was raised in issue
#31581.
A fix was attempted in 6c88666, but this did not resolve the issue, because the "response was already consumed" exception was a RuntimeError (as per request 2.32.4), and the commit only caught StreamClosedError and StreamConsumedError. http_error.response.internal_response is a requests.models.Response, and StreamClosedError and StreamConsumedError are defined in azure sdk, so the request module would never throw them anyway.
A fix could be to catch the RuntimeError, however upon further thinking and looking at the discussion in issue #31709 I decided to simply remove the reference to internal_response, as was already suggested by @anthony-c-martin .
The original source of the internal_response access seems to be PR #18165 which was made to fix issue #18134 ("[ARM] template validation commands show unexpected error messages for invalid templates"), however the original error was likely caused by using
.textrather than.text()(not tested it, but I think it's likely given the shown message in the issue), so assuming there's no other reason why internal_response was used, I (also) think we should probably remove it.Testing:
I also tested that it correctly display more complex error with causes in .details:
Fixes: #31581
Original "response consumed" exception stack trace:
Types traced:
Related command
Description
Testing Guide
History Notes
[Component Name 1] BREAKING CHANGE:
az command a: Make some customer-facing breaking change[Component Name 2]
az command b: Add some customer-facing featureThis checklist is used to make sure that common guidelines for a pull request are followed.
The PR title and description has followed the guideline in Submitting Pull Requests.
I adhere to the Command Guidelines.
I adhere to the Error Handling Guidelines.