diff --git a/doc/dev/mgmt/generation.md b/doc/dev/mgmt/generation.md index 20e16d919a22..4a35b3e45254 100644 --- a/doc/dev/mgmt/generation.md +++ b/doc/dev/mgmt/generation.md @@ -28,10 +28,10 @@ Autorest doesn't write the version number in the generated code, but a few indic A basic autorest command line will looks like this: ```shell -autorest readme.md --python --use="@microsoft.azure/autorest.python@~4.0.71" --python-mode=update --python-sdks-folder=/sdks/ --no-async --multiapi +autorest readme.md --python --use="@microsoft.azure/autorest.python@~4.0.71" --python-mode=update --python-sdks-folder=/sdks/ --no-async ``` -Which means "Generate the Python code for the Swagger mentioned in this readme, using autorest for Pyton v4.0.71 or above (but not v5), do not generate async files, generate multiapi if supported (if not ignore), and assume the package was already generated and it's an update" +Which means "Generate the Python code for the Swagger mentioned in this readme, using autorest for Python v4.0.71 or above (but not v5), do not generate async files, and assume the package was already generated and it's an update" In practical terms, this is not necessary since the Python SDK has the necessary tooling to simplify to just specify the readme.md: diff --git a/doc/dev/mgmt/mgmt_release.md b/doc/dev/mgmt/mgmt_release.md index 3a7d80133c17..8f3e378586f8 100644 --- a/doc/dev/mgmt/mgmt_release.md +++ b/doc/dev/mgmt/mgmt_release.md @@ -125,11 +125,7 @@ Example: - Added operation group TrafficManagerUserMetricsKeysOperations ``` -### Note on multi-api packages -If a package is using multi-api, this means it contains several Autorest generated folder. The tool will then build one report per Autorest generation. - -To simplify the change log call, the code report also build a "merged_report" that will merge correctly all api-versions and build a report suitable *for the default floating latest* Example of output if call with network: ```shell diff --git a/doc/dev/mgmt/multiapi.md b/doc/dev/mgmt/multiapi.md deleted file mode 100644 index 50843c42c6f2..000000000000 --- a/doc/dev/mgmt/multiapi.md +++ /dev/null @@ -1,119 +0,0 @@ -# Multi-api packaging and ARM SDK - -Several SDKs on this repository are able to support multi-api in a single package. If you're a SDK maintainer, this documentation explains the rational of it, and the technical details. - -## Overview and rationale - -### Multi-api, what does it mean? - -It means a given specific package is able to do calls to several different API versions of a given service in the same installation and Python process. Example: using `azure-mgmt-compute`, you can get a VM object using API version 2019-03-01, but also 2015-06-15 or 2018-04-01, etc. and not only using the latest API version that exists. - -### Why would I need to call anything else than the latest API version? - -Because there is different flavors of Azure that are not necessarily provided with the same set of API versions. For example Azure Government, Azure Stack usually needs a few months to get the latest available API version. - -### Why a multi-api package? - -Indeed, a simple solution would be to write down explicitly what version of SDK supports what API version. Example: 1.0 supports 2015-06-01, 2.0 supports 2017-07-01, etc. The story for customers then would be to pin the specific SDK version for the specific API version they need. However, this was considered unacceptable in an end-to-end scenario: -- It means you cannot install in the same Python environment packages that would target different cloud (Python doesn't allow installation of different versions of the same package together). Azure CLI or Ansible supports for different clouds would then be extremely complicated. -- This forces customers to use old SDK, that might have been fixed on different axis than API version (security fixes, new SDK features like async, etc.) -- Customers rarely needs only one package, but a set of them (storage, compute, network, etc.) and having to keep track of the correct list of packages is challenging. - -By providing package that supports multiple API versions we are solving all these issues: -- Simply install "azure-mgmt-compute" and you know it will provide all features for all clouds -- Install the latest one, it will provide all security and features and still contains all necessary API versions for all clouds. -- We can create the notion of "Azure Profile", which is a configuration file that configures automatically all the installed packages to the correct version of a given cloud. - -### Is the package too big then, all these API versions? - -Autorest for Python v4 has been designed to reduce the size of multi-api packages up to 80% in some scenarios. There is work in progress to do more and scale as necessary for a future version of Autorest. - -### Does some API version get deprecated some time and removed from packages? - -No, until ARM supports an API version it stays in the package. That could be a massive breaking change otherwise, to force customers to update an API version. - -## Technical details - -### Details about ARM, resource providers, resource types and Swaggers - -ARM is split by resource providers, themself split by resource types. The API version is actually defined at the resource type level (you can get an overview of equivalence between resource types and API version using the CLI: `az provider list`). - -To simplify multi-api packaging, it's recommended that each resource type are written in their own file, and that all operations in that file are prefixed by the same operation group. -Example: -Network interfaces operations are defines in a [network interface file](https://github.com/Azure/azure-rest-api-specs/blob/2a65faa9ddbf9970708ba507eeb8071a2d310b57/specification/network/resource-manager/Microsoft.Network/stable/2019-04-01/networkInterface.json), and all operations in this file are prefixed with `NetworkInterfaces_`. - -**Python multi-api packaging is based on the assumptions that it's true.** If it's not, it's usually ok but requires a little more subtle packaging (see final section here) - -Being that a given Swagger defines only *one* fixed API version, doing multi-api version in one package implies shipping several Swagger files into one package. This is archived by the `batch` directive of Autorest. More details on how to write Readme for Swagger in the specific page for it [swagger_conf.md](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/mgmt/swagger_conf.md). - -Python SDK team is responsible to design the correct set of tags to set for the `batch` node. Each line of the batch directive should contains only *one* api version to match the folder name used. this might require adding new tags in the readme.md that are specific to only one API version. These tags are usually suffixed by "-only" ([example with compute](https://github.com/Azure/azure-rest-api-specs/tree/main/specification/compute/resource-manager#tag-package-2019-03-01-only)) - - In order to simplify this work, there is script called [multi_api_readme_help](https://github.com/Azure/azure-sdk-for-python/blob/main/scripts/multi_api_readme_help.py) that will scan all the Swaggers of a given folder, and suggest: -- a list of tags for the main readme.md -- a batch configuration for the readme.python.md - -Note that this script is experimental, and the suggested output might need to be adjusted. - -### Azure profile - -An azure profile is an official, stamped by Microsoft, mapping between a set of resource types and the correct supported API versions. A profile is the only way right now to characterize a complete of set of ARM resources mapping together. Profiles are checked in in this place: https://github.com/Azure/azure-rest-api-specs/tree/main/profile - -### Overview of a multi-api client - -Main design guidelines: -- For people on public Azure, API must **NOT** be required parameters: - e.g. `client = ComputeManagementClient(credentials, sub_id)` -- For people that want a specific API version for a specific need, specifying API version should be possible - e.g. `client = ComputeManagementClient(credentials, sub_id, api_version='2018-06-01')` -- For people who target a single Azure Profile, specifying it should be be possible - e.g. `client = ComputeManagementClient(credentials, sub_id, profile=KnownProfile.v2018_06_01_hybrid)` - -The first condition has impact on models loading, by default they should load the latest API version transparently: -```python -# Loads the latest version of the model -from azure.mgmt.compute.models import VirtualMachineCreateParameter -``` - -### Structure of a multi-api package - -``` -o -`-- azure - `-- mgmt - `-- network - |-- __init__.py - |-- _configuration.py - |-- _network_management_client.py - |-- _operations_mixin.py - |-- models.py - |-- version.py - |-- v2019_04_01 - | |-- models - | |-- operations - | |-- __init__.py - | |-- _configuration.py - | |-- _network_management_client.py - | `-- version.py - `-- v2018_10_01 - |-- models - |-- operations - |-- __init__.py - |-- _configuration.py - |-- _network_management_client.py - `-- version.py -``` - -- There is as many folder per API versions we want to ship -- The files in the API version folders are generated by Autorest - -## Complicated scenarios - -### One operation group is defined across several files. - -If this is the same API version, since they will be packed together that's ok (for instance [compute](https://github.com/Azure/azure-rest-api-specs/blob/main/specification/compute/resource-manager/Microsoft.Compute/ComputeRP/stable/2019-03-01/compute.json) and [runcommands](https://github.com/Azure/azure-rest-api-specs/blob/main/specification/compute/resource-manager/Microsoft.Compute/ComputeRP/stable/2019-03-01/runCommands.json) shares `VirtualMachines_` but exists always in the same API version) - -If this is not the same API version, then we need to bend the rules a little: we need to understand the intent, and decide which API version we use as folder to ship both (example: [ACR 2019-05-01](https://github.com/Azure/azure-rest-api-specs/blob/main/specification/containerregistry/resource-manager/Microsoft.ContainerRegistry/stable/2019-05-01/containerregistry.json) and [registry build 2019-06-01-preview](https://github.com/Azure/azure-rest-api-specs/tree/main/specification/containerregistry/resource-manager/Microsoft.ContainerRegistry/preview/2019-06-01-preview) were shipped as [2019-06-01-preview](https://github.com/Azure/azure-sdk-for-python/tree/24e4a701802b82832d97f125c6c96567c8e4448e/sdk/containerregistry/azure-mgmt-containerregistry/azure/mgmt/containerregistry/v2019_06_01_preview) because they share `BuildRegistry_` operation group). - -## Possible improvements - -Current implementation assumes operation group are unique, and as discussed it's not always the case. Also, this limitation has impact on intellisense right now. Example, if a user types `compute_client.virtual_machines.` and hit the intellisense shortcut, users won't see any suggestions. It's because the `virtual_machines` property is dynamic and can change depending of dynamic configuration. diff --git a/doc/dev/mgmt/swagger/multi_api/readme.md b/doc/dev/mgmt/swagger/multi_api/readme.md deleted file mode 100644 index c03fc97c93f3..000000000000 --- a/doc/dev/mgmt/swagger/multi_api/readme.md +++ /dev/null @@ -1,67 +0,0 @@ -# Network - -> see https://aka.ms/autorest - -This is the AutoRest configuration file for Network. - ---- - -## Getting Started - -To build the SDK for Network, simply [Install AutoRest](https://github.com/Azure/autorest/blob/master/docs/install/readme.md) and in this folder, run: - -> `autorest` - -To see additional help and options, run: - -> `autorest --help` - ---- - -## Configuration - -### Basic Information - -These are the global settings for the Network API. - -``` yaml -title: NetworkManagementClient -description: Network Client -openapi-type: arm -tag: package-2019-04 -``` - - -### Tag: package-2019-04 - -These settings apply only when `--tag=package-2019-04` is specified on the command line. - -```yaml $(tag) == 'package-2019-04' -input-file: - - Microsoft.Network/stable/2019-04-01/applicationGateway.json -``` - ---- - -# Code Generation - -## Swagger to SDK - -This section describes what SDK should be generated by the automatic system. -This is not used by Autorest itself. - -``` yaml $(swagger-to-sdk) -swagger-to-sdk: - - repo: azure-sdk-for-python-track2 - - repo: azure-sdk-for-java - - repo: azure-sdk-for-go - - repo: azure-sdk-for-js - - repo: azure-sdk-for-node - - repo: azure-sdk-for-ruby - after_scripts: - - bundle install && rake arm:regen_all_profiles['azure_mgmt_network'] -``` - -## Python - -See configuration in [readme.python.md](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/mgmt/swagger/multi_api/readme.python.md) diff --git a/doc/dev/mgmt/swagger/multi_api/readme.python.md b/doc/dev/mgmt/swagger/multi_api/readme.python.md deleted file mode 100644 index 1dfe89377e12..000000000000 --- a/doc/dev/mgmt/swagger/multi_api/readme.python.md +++ /dev/null @@ -1,49 +0,0 @@ -## Python - -These settings apply only when `--python` is specified on the command line. - -``` yaml $(python) -azure-arm: true -license-header: MICROSOFT_MIT_NO_VERSION -package-name: azure-mgmt-servicetoreplace -package-version: 1.0.0b1 -no-namespace-folders: true -``` - -### Python multi-api - -Generate all API versions currently shipped for this package - -```yaml $(python) -clear-output-folder: true -multiapi: true -batch: - - tag: package-2019-04 - - tag: package-2019-02 - - multiapiscript: true -``` - -``` yaml $(multiapiscript) -output-folder: $(python-sdks-folder)/servicetoreplace/azure-mgmt-servicetoreplace/azure/mgmt/servicetoreplace/ -perform-load: false -``` - -### Tag: package-2019-04 and python - -These settings apply only when `--tag=package-2019-04 --python` is specified on the command line. -Please also specify `--python-sdks-folder=`. - -``` yaml $(tag) == 'package-2019-04' && $(python) -namespace: azure.mgmt.servicetoreplace.v2019_04_01 -output-folder: $(python-sdks-folder)/servicetoreplace/azure-mgmt-servicetoreplace/azure/mgmt/servicetoreplace/v2019_04_01 -``` - -### Tag: package-2019-02 and python - -These settings apply only when `--tag=package-2019-02 --python` is specified on the command line. -Please also specify `--python-sdks-folder=`. - -``` yaml $(tag) == 'package-2019-02' && $(python) -namespace: azure.mgmt.servicetoreplace.v2019_02_01 -output-folder: $(python-sdks-folder)/servicetoreplace/azure-mgmt-servicetoreplace/azure/mgmt/servicetoreplace/v2019_02_01 -``` diff --git a/doc/dev/mgmt/swagger_conf.md b/doc/dev/mgmt/swagger_conf.md index ef3a512b273d..5c6eef17ab18 100644 --- a/doc/dev/mgmt/swagger_conf.md +++ b/doc/dev/mgmt/swagger_conf.md @@ -14,11 +14,10 @@ In practical terms, we want to control the version of Autorest used, the output ## Writing the readme -Writing the readme is the responsibility of the Python SDK team. There is currently two types of templates for Python readmes: +Writing the readme is the responsibility of the Python SDK team. There is currently one type of template for Python readmes: - Readme that handles only one API version, and generates packages that handle one API version only -- Readme that handles several API versions, and generates packages with multiples API and profile supports -These templates can be found in the [single_api](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/mgmt/swagger/single_api) and the [multi_api](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/mgmt/swagger/multi_api) folders. +Templates can be found in the [single_api](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/mgmt/swagger/single_api) folder. ### Single API readmes @@ -27,31 +26,5 @@ This one is the most simple: - Copy the [readme.python.md](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/mgmt/swagger/single_api/readme.python.md) and replace `servicetoreplace` by your service name - Be sure the main [readme.md](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/mgmt/swagger/single_api/readme.md) contains a "swagger-to-sdk" section with Python -### Multi API readmes -When doing multi-api packages, it means you have shipping several "tags" of the main readme as one package. Autorest is calling this process a "batch" call, and this is the purpose of the "batch" section in [readme.python.md](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/mgmt/swagger/multi_api/readme.python.md). - -In order to be sure the correct tags exist, you can use the following script: -```shell -python ./scripts/multi_api_readme_help.py /azure-rest-api-specs/specification/service/resource-manager/ -``` - -This script will analyze the Swaggers available, and suggests on stdout: -- A list of tags for the main [readme.md](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/mgmt/swagger/multi_api/readme.md) -- A batch declaration for the [readme.python.md](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/mgmt/swagger/multi_api/readme.python.md) - -This script is not perfect that it *does* require manual review of the output and not a direct copy/paste. - -It's important for Python that tags represents only *ONE* unique API version. It's why it's pretty common that Python uses a set of tags that other languages don't use. - -Once you know the list of tags you need to generate: - -- Copy the [readme.python.md](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/mgmt/swagger/multi_api/readme.python.md) and replace `servicetoreplace` by your service name -- Update the batch list of [readme.python.md](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/mgmt/swagger/multi_api/readme.python.md) -- Be sure you have one tag section for each batch entry in [readme.python.md](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/mgmt/swagger/multi_api/readme.python.md) -- Be sure the main [readme.md](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/mgmt/swagger/multi_api/readme.md) contains a "swagger-to-sdk" section with Python with an `afterscripts` section like the one in the template. - -The `afterscripts` will execute a Jinja template to create a client to link together all the batch generated autorest ([example](https://github.com/Azure/azure-sdk-for-python/blob/4a7c67189591b052fe2b5769847ff68f7845386d/sdk/storage/azure-mgmt-storage/azure/mgmt/storage/_storage_management_client.py)) - -This script will infer a default `LATEST_PROFILE`. You can change this behavior and force the default API version if necessary ([example](https://github.com/Azure/azure-rest-api-specs/blob/49238f0b2917452311e71dd43c4164de70af3721/specification/authorization/resource-manager/readme.md#swagger-to-sdk)) diff --git a/doc/dev/private_package/get_private_package.md b/doc/dev/private_package/get_private_package.md index 7bcea74f4519..eea19c5aeb0b 100644 --- a/doc/dev/private_package/get_private_package.md +++ b/doc/dev/private_package/get_private_package.md @@ -10,19 +10,13 @@ Make sure your target tag is defined in `readme.md` and **default tag is same wi ![](default_tag.png) -## 2.Configure `readme.python.md` - -If there is no `Python multi-api` in `readme.python.md`(i.e. [datadog](https://github.com/Azure/azure-rest-api-specs/blob/main/specification/datadog/resource-manager/readme.python.md)), skip this step. - -If there is `Python multi-api` in `readme.python.md` (i.e. [network](https://github.com/Azure/azure-rest-api-specs/blob/main/specification/network/resource-manager/readme.python.md#python-multi-api)), you need additional configuration: [Python Multiapi Configuration](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/private_package/python_multiapi_configuration.md) - -## 3.Trigger pipeline +## 2.Trigger pipeline Submit a PR or draft PR to [Azure/azure-rest-api-specs](https://github.com/Azure/azure-rest-api-specs) ![](unreleased_package_guide_example1.png) -## 4.Get private package +## 3.Get private package Wait until pipelines finish, then there will be wheel and zip of the package. Just Click to download them. @@ -32,7 +26,7 @@ If there is no link in the figure above, it may be folded. You can also find it ![img.png](unreleased_package_guide_example3.png) -## 5.Build private package locally (backup solution) +## 4.Build private package locally (backup solution) Because of security issue, maybe there is no private link to download. Since there is still auto generated PR provided, you can build the private package locally based on the PR with [guidance](https://github.com/Azure/azure-sdk-for-python/wiki/Common-issues-about-Python-SDK#build-private-package-with-pr) diff --git a/doc/dev/private_package/python_multiapi_configuration.md b/doc/dev/private_package/python_multiapi_configuration.md deleted file mode 100644 index 3e30867fbb41..000000000000 --- a/doc/dev/private_package/python_multiapi_configuration.md +++ /dev/null @@ -1,33 +0,0 @@ -This doc describes how to configure multi-api package for Python. - -# Multi-Api Package - -When `readme.python.md` has `Python multi-api`, the package is multi-api package. - -# Configuration - -When you want to release a new version for multi-api package, check the following steps: - -## 1.target tag - -Make sure the target tag is defined in `readme.md`. For example, you want to release new tag `package-2021-05-01` for `azure-mgmt-network`, it should be defined in [readme.md](https://github.com/Azure/azure-rest-api-specs/tree/main/specification/network/resource-manager#tag-package-2021-05). - -the target tag should only contain files from one folder: - -![](one_folder.png) - -If your tag contains files from different folders, you need to define some split tags. For example: `ApplicationInsights` has [Tag: package-2021-11-01](https://github.com/Azure/azure-rest-api-specs/tree/main/specification/applicationinsights/resource-manager#tag-package-2021-11-01), but it contains different folders. If you want to publish the package for the tag, you need to split it to different tags: [sample](https://github.com/Azure/azure-rest-api-specs/pull/16799/files) - -![](different_folders.png) - -![](split_tag.png) - - - -## 2.Configure `readme.python.md` - -Let us set `azure-mgmt-network` as example, if you want to release `package-2021-03-01`, you need to add the following definition in `readme.python.md`: - -[Update readme.python.md by BigCat20196 · Pull Request #15818 · Azure/azure-rest-api-specs (github.com)](https://github.com/Azure/azure-rest-api-specs/pull/15818/files) - -![](python_config.png) \ No newline at end of file diff --git a/eng/.docsettings.yml b/eng/.docsettings.yml index 0a9925194637..715e5002af8b 100644 --- a/eng/.docsettings.yml +++ b/eng/.docsettings.yml @@ -147,7 +147,6 @@ known_content_issues: # dev tools - ['doc/dev/mgmt/swagger/single_api/readme.md', 'dev readme'] - - ['doc/dev/mgmt/swagger/multi_api/readme.md', 'dev readme'] - ['doc/dev/mgmt/README.md', 'dev readme'] - ['doc/dev/README.md', dev readme'] - ['doc/README.md', 'dev readme'] diff --git a/scripts/multi_api_readme_help.py b/scripts/multi_api_readme_help.py deleted file mode 100644 index c73744f32fd5..000000000000 --- a/scripts/multi_api_readme_help.py +++ /dev/null @@ -1,122 +0,0 @@ -import json -import logging -from pathlib import Path -import sys - - -_LOGGER = logging.getLogger(__name__) - -_TAG_PREFIX = """### Tag: package-{api_version}-only - -These settings apply only when `--tag=package-{api_version}-only` is specified on the command line. - -```yaml $(tag) == 'package-{api_version}-only' -input-file:""" -_TAG_SUFFIX = "```\n\n" - -_BATCH_PREFIX = """```yaml $(python) && $(multiapi) -batch:""" -_BATCH_SUFFIX = "```\n\n" - -_PY_NAMESPACE = """### Tag: package-{api_version}-only and python - -These settings apply only when `--tag=package-{api_version}-only --python` is specified on the command line. -Please also specify `--python-sdks-folder=`. - -``` yaml $(tag) == 'package-{api_version}-only' && $(python) -python: - namespace: $(python-base-namespace).{ns} - output-folder: $(python-sdks-folder)/$(python-base-folder)/{ns} -``` -""" - -def get_api_versions(root): - - api_versions = {} - prefixes_per_path = {} - - rp_folders = root.glob("Microsoft.*") - for rp_folder in rp_folders: - _LOGGER.info(f"Parsing folder {rp_folder}") - for preview_stable in rp_folder.iterdir(): - _LOGGER.info(f"Currently in {preview_stable}") - for api_version in preview_stable.iterdir(): - _LOGGER.info(f"Currently in {api_version}") - for swagger in api_version.glob("*.json"): - prefixes_per_path[swagger] = parse_swagger(swagger) - api_versions.setdefault(api_version.name, []).append(swagger.relative_to(root).as_posix()) - - # Try to detect when it's problematic. That's tough, the following logic is definitely - # not handling all the touch parts yet... - - # 1- If a file declare several prefixes, let's warning - for swagger_path, prefixed_used in prefixes_per_path.items(): - if len(prefixed_used) == 1: - _LOGGER.info(f"File {swagger_path} uses only one prefix: {prefixed_used}") - else: - _LOGGER.warn(f"File {swagger_path} uses several prefixes: {prefixed_used}") - - - # Let's print - print_tags(api_versions) - print_batch(api_versions) - print_python_namespace(api_versions) - -def print_tags(api_versions): - for api_version in sorted(api_versions.keys(), reverse=True): - swagger_files = api_versions[api_version] - print(_TAG_PREFIX.format(api_version=api_version)) - for swagger_file in swagger_files: - print("- {}".format(swagger_file)) - print(_TAG_SUFFIX) - - -def print_batch(api_versions): - print(_BATCH_PREFIX) - for api_version in sorted(api_versions.keys(), reverse=True): - print(f" - tag: package-{api_version}-only") - print(_BATCH_SUFFIX) - -def print_python_namespace(api_versions): - for api_version in sorted(api_versions.keys(), reverse=True): - swagger_files = api_versions[api_version] - print(_PY_NAMESPACE.format( - api_version=api_version, - ns="v"+api_version.replace("-", "_")) - ) - -def parse_swagger(swagger_path): - _LOGGER.info(f"Parsing {swagger_path}") - with swagger_path.open() as swagger: - parsed_swagger = json.load(swagger) - - api_version = parsed_swagger["info"]["version"] - - operations = operation_finder(parsed_swagger) - - prefixed_used = {op.split("_")[0] for op in operations if "_" in op} - return prefixed_used - -def operation_finder(swagger_root): - result = set() - for key, node in swagger_root.items(): - if key == "definitions": # Skipping some node - return result - if key == "operationId": - result.add(node) - # Can skip it now, only one operationId per node - return result - if isinstance(node, dict): - result |= operation_finder(node) - return result - - -if __name__ == "__main__": - logging.basicConfig(level=logging.DEBUG if "--debug" in sys.argv else logging.WARNING) - - root = Path(__file__).parent - - root = Path(sys.argv[1]).relative_to(root) - - _LOGGER.info(f"My root: {root}") - get_api_versions(root) diff --git a/scripts/multiapi-configuration-helper/README.md b/scripts/multiapi-configuration-helper/README.md deleted file mode 100644 index 2b244ce3c3ed..000000000000 --- a/scripts/multiapi-configuration-helper/README.md +++ /dev/null @@ -1,9 +0,0 @@ -# Overview -This script is to help Python SDK owner to configure multi-api for autorest more conveniently. - -# Usage -1. copy the script to the place under the same folder with `azure-rest-api-specs`. -2. execute script and follow the hint to input parameters: -``` -python python-multiapi-configuration-helper.py -``` \ No newline at end of file diff --git a/scripts/multiapi-configuration-helper/python-multiapi-configuration-helper.py b/scripts/multiapi-configuration-helper/python-multiapi-configuration-helper.py deleted file mode 100644 index 5c25ba86e4f4..000000000000 --- a/scripts/multiapi-configuration-helper/python-multiapi-configuration-helper.py +++ /dev/null @@ -1,193 +0,0 @@ -import os -import re -from glob import glob -from pathlib import Path -from typing import Set, List -from subprocess import check_call, call - - -def print_check(cmd: str): - check_call(cmd, shell=True) - - -def print_exec(cmd: str): - call(cmd, shell=True) - - -def step_into_rest_repo(): - os.chdir(str(Path(os.getcwd()) / 'azure-rest-api-specs')) - - -def git_clean(): - print_check('git checkout .') - print_check('git clean -fd') - print_check('git reset --hard HEAD') - - -class MultiApiConfigHelper: - """ - This class can generate private SDK package - """ - - def __init__(self): - self.service_name = '' - self.tag = '' - self.tag_files = {} # Dict[str, List(str)] - self.configured_files = set() # Set[str] - - def get_input(self): - print('Please commit your code before execute this script!!!') - self.service_name = input('Please input your target service link(e.g. ' - '"https://github.com/Azure/azure-rest-api-specs/blob/main/specification/applicationinsights/resource-manager" ' - 'or "applicationinsights"):\n') - if 'resource-manager' in self.service_name: - self.service_name = self.service_name.split('/')[-2] - tag = input('Please input your target tag(e.g. "Readme Tag: package-2022-01-11" or "package-2022-01-11"):\n') - self.tag = tag.split(':')[-1].strip() - - @staticmethod - def checkout_main_branch(): - step_into_rest_repo() - git_clean() - usr = 'Azure' - branch = 'main' - print_exec(f'git remote add {usr} https://github.com/{usr}/azure-rest-api-specs.git') - print_check(f'git fetch {usr} {branch}') - print_check(f'git checkout {usr}/{branch}') - - def step_into_service_folder(self): - root_path = os.getcwd() - target_service_info = f'{root_path}/specification/{self.service_name}/resource-manager' - result = glob(target_service_info) - if len(result) == 0: - raise Exception(f'do not find {target_service_info}') - elif len(result) > 1: - raise Exception(f'find multi target folder: {str(result)}') - os.chdir(str(Path(result[0]))) - - @staticmethod - def judge_tag(line: str) -> bool: - # ``` yaml $(tag) == 'package-2021-11-01' - elements = ['```', 'tag', '==', 'package-'] - result = [re.search(element, line) for element in elements] - return all(result) - - @staticmethod - def get_tags(line: str) -> Set[str]: - # extract 'package-2021-11-01' from "``` yaml $(tag) == 'package-2021-11-01'" - result = re.findall("[\'\"][^\'^\"]+[\'\"]", line) - if not result: - raise Exception(f'Can not find valid tag name in {line}') - return {item[1:-1] for item in result} - - @staticmethod - def get_file_name(line: str) -> str: - # - Microsoft.Insights/stable/2015-05-01/aiOperations_API.json - if '.json' in line and 'Microsoft.' in line: - line = line.strip('\n ') - line = line[1:] if line[0] == '-' else line - return line.strip() - return '' - - def get_tag_and_file(self, content: List[str], start_idx: int) -> int: - tags_name = self.get_tags(content[start_idx]) - end_idx = start_idx - for end_idx in range(start_idx + 1, len(content)): - if '```' in content[end_idx]: - break - - files = [] - for idx in range(start_idx + 1, end_idx): - file_name = self.get_file_name(content[idx]) - if file_name: - files.append(file_name) - for tag_name in tags_name: - if not self.tag_files.get(tag_name) and files: - self.tag_files[tag_name] = files - return end_idx + 1 - - def get_all_tag_files(self): - with open('readme.md', 'r') as file_in: - content = file_in.readlines() - - i = 0 - while i < len(content): - if self.judge_tag(content[i]): - i = self.get_tag_and_file(content, i) - else: - i += 1 - - @staticmethod - def judge_config(line: str) -> bool: - return 'batch' in line - - @staticmethod - def get_configured_tag_name(line: str) -> str: - # ' - tag: package-2017-10' - if 'multiapiscript' in line: - return '' - return line.split('tag:')[-1].strip('\n ') - - def get_configured_tags(self, content: List[str], start_idx: int, tags: List[str]): - end_idx = start_idx - for end_idx in range(start_idx + 1, len(content)): - if '```' in content[end_idx]: - break - - for idx in range(start_idx + 1, end_idx): - tag_name = self.get_configured_tag_name(content[idx]) - if tag_name: - tags.append(tag_name) - - def get_configured_files(self): - with open('readme.python.md', 'r') as file_in: - content = file_in.readlines() - - configured_files = [] - configured_tags = [] - for i in range(len(content)): - if self.judge_config(content[i]): - self.get_configured_tags(content, i, configured_tags) - break - for tag in configured_tags: - if tag not in self.tag_files: - raise Exception(f'find {tag} in readme.python.md but it is not in readme.md!') - configured_files.extend(self.tag_files[tag]) - - self.configured_files = set(configured_files) - - def get_missing_files(self): - missing_files = [] - if self.tag not in self.tag_files: - raise Exception(f'Do not find \"{self.tag}\" in \"{self.service_name}/readme.md\"' - f'({str(self.tag_files.keys())})') - print(f'find the following tiles in {self.tag}:') - for file_name in self.tag_files[self.tag]: - print(file_name) - if file_name not in self.configured_files: - missing_files.append(' - ' + file_name) - - print('Program done!') - if missing_files: - print(f'There are {len(missing_files)} files that are not configured in readme.python.md:') - for file_name in missing_files: - print(file_name) - else: - print('There are not missing files!!') - os.system("pause") - - def compare_tag(self): - self.step_into_service_folder() - self.get_all_tag_files() - self.get_configured_files() - self.get_missing_files() - - def run(self): - self.get_input() - self.checkout_main_branch() - self.compare_tag() - - -if __name__ == '__main__': - instance = MultiApiConfigHelper() - instance.run() diff --git a/sdk/appconfiguration/azure-mgmt-appconfiguration/CHANGELOG.md b/sdk/appconfiguration/azure-mgmt-appconfiguration/CHANGELOG.md index fae1aeeaaa0d..eb53ca892b11 100644 --- a/sdk/appconfiguration/azure-mgmt-appconfiguration/CHANGELOG.md +++ b/sdk/appconfiguration/azure-mgmt-appconfiguration/CHANGELOG.md @@ -44,7 +44,7 @@ ### Other Changes - - Changed to multiapi package(please refer to https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/mgmt/multiapi.md for more info) + - Changed to multiapi package(please refer to https://github.com/Azure/azure-sdk-for-python/blob/20a21f4bee6164eea84eab83eddd272d76ff9839/doc/dev/mgmt/multiapi.md for more info) ## 2.1.0 (2022-06-08)