Added Custom Volume/Volume Mount Support

[Bobbins228]

Issue link

RHOAIENG-8262

What changes have been made

Added a two new config variables volumes which is used for adding custom volumes on a Ray Cluster and volume_mounts which does the same but for VolumeMounts.

Verification steps

Setup

Notebook server ODH/RHOAI/Local

• Clone this repository with git clone https://github.com/project-codeflare/codeflare-sdk.git
• Checkout this PR's branch
• Run poetry build - install if needed (pip install poetry)
• Run pip install --force-reinstall dist/codeflare_sdk-0.0.0.dev0-py3-none-any.whl
• Restart your notebook kernel

Testing

The volumes and volume_mounts config parameters are both lists that take the V1Volume and V1VolumeMount objects respectively.
Below I have given some examples for testing purposes.

Volumes/Volume Mounts configuration

from kubernetes.client import V1Volume, V1VolumeMount, V1EmptyDirVolumeSource, V1ConfigMapVolumeSource, V1KeyToPath, V1SecretVolumeSource
volume_mounts = [
    V1VolumeMount(
        mount_path="/home/ray/test1",
        name = "test"
    ),
    V1VolumeMount(
        mount_path = "/home/ray/test2",
        name = "test2",
    ),
    V1VolumeMount(
        mount_path = "/home/ray/test3",
        name = "test3",
    )
]

volumes = [
    V1Volume(
        name="test",
        empty_dir=V1EmptyDirVolumeSource(size_limit="2Gi"),
    ),
    V1Volume(
        name="test2",
        config_map=V1ConfigMapVolumeSource(
            name="test-config-map",
            items=[V1KeyToPath(key="test", path="data.txt")]
        )
    ),
    V1Volume(
        name="test3",
        secret=V1SecretVolumeSource(
            secret_name="test-secret"
    )
)
]

ClusterConfiguration

cluster = Cluster(ClusterConfiguration(
    name='raytest',
    namespace='default', # Update to your namespace
    num_workers=1,
    min_cpus=2,
    max_cpus=2,
    head_memory=3,
    min_memory=3,
    max_memory=3,
    num_gpus=0,
    head_gpus=0,
    image="quay.io/project-codeflare/ray:latest-py39-cu118",
    write_to_file=True, # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources 
    volume_mounts = volume_mounts,
    volumes = volumes
))

Test Config Map

kind: ConfigMap
apiVersion: v1
metadata:
  name: test-config-map
  namespace: YOUR_NAMESPACE
immutable: false
data:
  test: pass

Test Secret

• Create a secret on OpenShift by going to Workloads -> Secrets -> Create -> key/vaule secret
• Set the name to test-secret
• The key/value can be whatever you like

Verify

The volumes & volume mounts should be added to your worker and head pods.
You can exec into your pod and check that the emptyDir, Config Map volume and Secret Volume were mounted correctly

Checks

• I've made sure the tests are passing.
• Testing Strategy
  • Unit tests
  • Manual tests
  • Testing is not required for this change

[KPostOffice]

Overall LGTM. One tiny comment

[KPostOffice]

Conflict but then ready to come in I think

[KPostOffice]

/lgtm

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: KPostOffice

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [KPostOffice]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment