-
Notifications
You must be signed in to change notification settings - Fork 159
feat: support workload identity setting in static PV mount #1566
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Binary file not shown.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,179 @@ | ||
| # Example of static PV mount with workload identity | ||
|
|
||
| > Note: | ||
| > - Available kubernetes version >= v1.20 | ||
|
|
||
| ## prerequisite | ||
|
|
||
|
|
||
| ### 1. Create a cluster with oidc-issuer enabled and get the credential | ||
|
|
||
| Following the [documentation](https://learn.microsoft.com/en-us/azure/aks/use-oidc-issuer#create-an-aks-cluster-with-oidc-issuer) to create an AKS cluster with the `--enable-oidc-issuer` parameter and get the AKS credentials. And export following environment variables: | ||
| ``` | ||
| export RESOURCE_GROUP=<your resource group name> | ||
| export CLUSTER_NAME=<your cluster name> | ||
| export REGION=<your region> | ||
| ``` | ||
|
|
||
|
|
||
| ### 2. Create a new storage account and fileshare | ||
|
|
||
| Following the [documentation](https://learn.microsoft.com/en-us/azure/storage/files/storage-how-to-use-files-portal?tabs=azure-cli) to create a new storage account and fileshare or use your own. And export following environment variables: | ||
| ``` | ||
| export ACCOUNT=<your storage account name> | ||
| export SHARE=<your fileshare name> | ||
| ``` | ||
|
|
||
| ### 3. Create managed identity and role assignment | ||
| ``` | ||
| export UAMI=<your managed identity name> | ||
| az identity create --name $UAMI --resource-group $RESOURCE_GROUP | ||
|
|
||
| export USER_ASSIGNED_CLIENT_ID="$(az identity show -g $RESOURCE_GROUP --name $UAMI --query 'clientId' -o tsv)" | ||
| export IDENTITY_TENANT=$(az aks show --name $CLUSTER_NAME --resource-group $RESOURCE_GROUP --query identity.tenantId -o tsv) | ||
| export ACCOUNT_SCOPE=$(az storage account show --name $ACCOUNT --query id -o tsv) | ||
| az role assignment create --role "Storage Account Contributor" --assignee $USER_ASSIGNED_CLIENT_ID --scope $ACCOUNT_SCOPE | ||
| ``` | ||
|
|
||
| ### 4. Create service account on AKS | ||
| ``` | ||
| export SERVICE_ACCOUNT_NAME=<your sa name> | ||
| export SERVICE_ACCOUNT_NAMESPACE=<your sa namespace> | ||
|
|
||
| cat <<EOF | kubectl apply -f - | ||
| apiVersion: v1 | ||
| kind: ServiceAccount | ||
| metadata: | ||
| name: ${SERVICE_ACCOUNT_NAME} | ||
| namespace: ${SERVICE_ACCOUNT_NAMESPACE} | ||
| EOF | ||
| ``` | ||
|
|
||
| ### 5. Create the federated identity credential between the managed identity, service account issuer, and subject using the `az identity federated-credential create` command. | ||
| ``` | ||
| export FEDERATED_IDENTITY_NAME=<your federated identity name> | ||
| export AKS_OIDC_ISSUER="$(az aks show --resource-group $RESOURCE_GROUP --name $CLUSTER_NAME --query "oidcIssuerProfile.issuerUrl" -o tsv)" | ||
|
|
||
| az identity federated-credential create --name $FEDERATED_IDENTITY_NAME \ | ||
| --identity-name $UAMI \ | ||
| --resource-group $RESOURCE_GROUP \ | ||
| --issuer $AKS_OIDC_ISSUER \ | ||
| --subject system:serviceaccount:${SERVICE_ACCOUNT_NAMESPACE}:${SERVICE_ACCOUNT_NAME} | ||
| ``` | ||
|
|
||
| ## option#1: static provision with PV | ||
| ``` | ||
| cat <<EOF | kubectl apply -f - | ||
| apiVersion: v1 | ||
| kind: PersistentVolume | ||
| metadata: | ||
| annotations: | ||
| pv.kubernetes.io/provisioned-by: file.csi.azure.com | ||
| name: pv-azurefile | ||
| spec: | ||
| capacity: | ||
| storage: 10Gi | ||
| accessModes: | ||
| - ReadWriteMany | ||
| persistentVolumeReclaimPolicy: Retain | ||
| storageClassName: azurefile-csi | ||
| mountOptions: | ||
| - dir_mode=0777 | ||
| - file_mode=0777 | ||
| - uid=0 | ||
| - gid=0 | ||
| - mfsymlinks | ||
| - cache=strict | ||
| - nosharesock | ||
| csi: | ||
| driver: file.csi.azure.com | ||
| # make sure volumeid is unique for every identical share in the cluster | ||
| # the # character is reserved for internal use | ||
| volumeHandle: unique_volume_id | ||
| volumeAttributes: | ||
| storageaccount: $ACCOUNT # required | ||
| shareName: $SHARE # required | ||
| clientID: $USER_ASSIGNED_CLIENT_ID # required | ||
| resourcegroup: $RESOURCE_GROUP # required, please make sure your account is NOT created under AKS node resource group(prefix with `MC_`) | ||
cvvz marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| # tenantID: $IDENTITY_TENANT #optional, only specified when workload identity and AKS cluster are in different tenant | ||
| # subscriptionid: $SUBSCRIPTION #optional, only specified when workload identity and AKS cluster are in different subscription | ||
| --- | ||
| apiVersion: apps/v1 | ||
| kind: StatefulSet | ||
| metadata: | ||
| name: statefulset-azurefile | ||
| labels: | ||
| app: nginx | ||
| spec: | ||
| podManagementPolicy: Parallel | ||
| serviceName: statefulset-azurefile | ||
| replicas: 1 | ||
| template: | ||
| metadata: | ||
| labels: | ||
| app: nginx | ||
| spec: | ||
| serviceAccountName: $SERVICE_ACCOUNT_NAME #required, Pod does not use this service account has no permission to mount the volume | ||
| nodeSelector: | ||
| "kubernetes.io/os": linux | ||
| containers: | ||
| - name: statefulset-azurefile | ||
| image: mcr.microsoft.com/oss/nginx/nginx:1.19.5 | ||
| command: | ||
| - "/bin/bash" | ||
| - "-c" | ||
| - set -euo pipefail; while true; do echo $(date) >> /mnt/azurefile/outfile; sleep 1; done | ||
| volumeMounts: | ||
| - name: persistent-storage | ||
| mountPath: /mnt/azurefile | ||
| updateStrategy: | ||
| type: RollingUpdate | ||
| selector: | ||
| matchLabels: | ||
| app: nginx | ||
| volumeClaimTemplates: | ||
| - metadata: | ||
| name: persistent-storage | ||
| spec: | ||
| storageClassName: azurefile-csi | ||
| accessModes: ["ReadWriteMany"] | ||
| resources: | ||
| requests: | ||
| storage: 10Gi | ||
| EOF | ||
| ``` | ||
|
|
||
| ## option#2: Pod with ephemeral inline volume | ||
| ``` | ||
| cat <<EOF | kubectl apply -f - | ||
| kind: Pod | ||
| apiVersion: v1 | ||
| metadata: | ||
| name: nginx-azurefile-inline-volume | ||
| spec: | ||
| serviceAccountName: $SERVICE_ACCOUNT_NAME #required, Pod does not use this service account has no permission to mount the volume | ||
| nodeSelector: | ||
| "kubernetes.io/os": linux | ||
| containers: | ||
| - image: mcr.microsoft.com/oss/nginx/nginx:1.19.5 | ||
| name: nginx-azurefile | ||
| command: | ||
| - "/bin/bash" | ||
| - "-c" | ||
| - set -euo pipefail; while true; do echo $(date) >> /mnt/azurefile/outfile; sleep 1; done | ||
| volumeMounts: | ||
| - name: persistent-storage | ||
| mountPath: "/mnt/azurefile" | ||
| volumes: | ||
| - name: persistent-storage | ||
| csi: | ||
| driver: file.csi.azure.com | ||
| volumeAttributes: | ||
| storageaccount: $ACCOUNT # required | ||
| shareName: $SHARE # required | ||
| clientID: $USER_ASSIGNED_CLIENT_ID # required | ||
| resourcegroup: $RESOURCE_GROUP # optional, specified when the storage account is not under AKS node resource group(which is prefixed with `MC_`) | ||
| # tenantID: $IDENTITY_TENANT # optional, only specified when workload identity and AKS cluster are in different tenant | ||
| # subscriptionid: $SUBSCRIPTION # optional, only specified when workload identity and AKS cluster are in different subscription | ||
| EOF | ||
| ``` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.