Enable Hyperpod telemetry #116

rsareddy0329 · 2025-07-11T07:53:41Z

PR Approval Steps

For Requester

Description
- Check the PR title and description for clarity. It should describe the changes made and the reason behind them.
- Ensure that the PR follows the contribution guidelines, if applicable.
Security requirements
- Ensure that a Pull Request (PR) does not expose passwords and other sensitive information by using git-secrets and upload relevant evidence: https://github.com/awslabs/git-secrets
- Ensure commit has GitHub Commit Signature
Manual review
1. Click on the Files changed tab to see the code changes. Review the changes thoroughly:
  - Code Quality: Check for coding standards, naming conventions, and readability.
  - Functionality: Ensure that the changes meet the requirements and that all necessary code paths are tested.
  - Security: Check for any security issues or vulnerabilities.
  - Documentation: Confirm that any necessary documentation (code comments, README updates, etc.) has been updated.
Check for Merge Conflicts:
- Verify if there are any merge conflicts with the base branch. GitHub will usually highlight this. If there are conflicts, you should resolve them.

For Reviewer

Go through For Requester section to double check each item.
Request Changes or Approve the PR:
1. If the PR is ready to be merged, click Review changes and select Approve.
2. If changes are required, select Request changes and provide feedback. Be constructive and clear in your feedback.
Merging the PR
1. Check the Merge Method:
  1. Decide on the appropriate merge method based on your repository's guidelines (e.g., Squash and merge, Rebase and merge, or Merge).
2. Merge the PR:
  1. Click the Merge pull request button.
  2. Confirm the merge by clicking Confirm merge.

* Introduce helm charts for hyperpod inference operator * Introduce helm charts for hyperpod inference operator * Introduce helm charts for hyperpod inference operator * Update Helm charts for inference operator, clean up to remove bedrock references. * Changes to 1. update image tag 2. Remove IAM policies for execution role 3. Rename to hyperpod-inference-operator prefix instead of deploymentoperator prefix * Removed binary from the code base. * Nit: Update the app name labels for sample yaml files. * Merge pull request aws#29 from mbnavali/main Introduce helm charts for hyperpod inference operator * Add crds, service account and region (aws#32) * Add CRDs and setup for region * Change annotation for SA * Remove default region * Add hyperpod inference classes **Description** Support jumpstart and custom model endpoints **Testing Done** Tested manually, will add unit tests in next few PRs * Refactor create inference function **Description** Refactor ModelEndpoint classes to let create happen in separate method instead of constructor **Testing Done** Manually tested in demo notebook * Add List, Delete, Describe endpoint features Tested manually in demo jupyter notebook * Add unit test and update class names **Description** **Testing Done** Unit test passes * Add end and setup.py * Update gitignore * Add setup.cfg * Fix HPEndpoint class and add optional values * remove utils.py * Make function classmethod and update unit tests * Fix bugs for inference endpoint * Small fixes * build: add mountpoint s3 csi driver, keda + cert-manager controllers as dependencies feat: add pv and pvc creation as part of helm * chore: add inference operator as dependency for HP Helm Chart, default disabled * feat: add support for jumpstart gated models * fix: remove stray symbol * fix: rename inference operator chart to match name in parent * change: sync charts with latest version of operator * doc: update readme.md identifying the inference operator as a subchart * Add HyperpodPytorchJob class (aws#39) * Add HyperpodPytorchJob * update to class methods * update to class methods * Address feedback * Fix bug --------- Co-authored-by: pintaoz <[email protected]> * Add tlsConfig to quick create * Revert "Add tlsConfig to quick create" This reverts commit 574351e. * Add tls config * Update CRD configs and minor updates * Add model_location to HPEndpont * Adding observability command to fetch details of grafana, prometheus and list of enabled metrics. * Training CLI implementation: create * Adding observability SDK experience and updating CLI command signature * Rename CLI commands to be consistent with SDK * Training CLI for Launch * Training CLI for Launch * Training CLI for Launch * Training CLI for Launch * Update JumpStartModel interface (aws#51) * Update JumpStartModel interface Tested in Jupyter notebook that endpoint can be successfully invoked * Add refresh method * remove debugging print * Update HPEndpoint classes Tested using example notebooks * Add example notebooks These notebooks haven't been cleaned up and they are for internal review only. Commands are supposed to change later * Add metadata class * Get Cluster Context * Update to HyperPodManager call * Cleanup import * Training CLI for Launch * Training CLI for Launch * Training CLI for Launch * Update HyperPodPytorchJob (aws#52) * Add HyperpodPytorchJob * update to class methods * update to class methods * Address feedback * Fix bug * Update HyperPodPytorchJob * Fix dependency * Add status * Add list_pods and get_logs_from_pod * Add error handling and metadata * Add example notebook * Fix bug --------- Co-authored-by: pintaoz <[email protected]> * E2E testing done for inference CLI * delete build * Revert accidental submodule pointer change * Update inference example notebook and fix bugs * Reformat code with black * Add get_logs function for inference * Update HyperPodPytorchJob to not use _HyperPodPytorchJob object (aws#63) * Add HyperpodPytorchJob * update to class methods * update to class methods * Address feedback * Fix bug * Update HyperPodPytorchJob * Fix dependency * Add status * Add list_pods and get_logs_from_pod * Add error handling and metadata * Add example notebook * Fix bug * Hide _HyperPodPytorchJob from user * Fix merge conflicts --------- Co-authored-by: pintaoz <[email protected]> * Update get_logs function to accept since_hour Tested in notebook * Separate get_logs and get_operator_logs methods * Update get_logs to class method * Add container name to get_logs function * Add container in get_logs_from_pod (aws#66) Co-authored-by: pintaoz <[email protected]> * change inference CLI directory, add inference CLI notebook, add get-logs and get-operator-logs * delete build * Training CLI for Launch - Changes per SDK HyperPodPytorchJob constructor (aws#64) * Training CLI for Launch * Training CLI for Launch --------- Co-authored-by: Roja Reddy Sareddy <[email protected]> * | * d2453d6 (rig-dev) Add notes about HMA patching * add cloudwatchtrigger and autoscalingspec to model.py and schema.json * Add exception handling and update example notebooks (aws#71) * Add exception handling and update example notebooks * Update HPEndpoint get status * Add unit tests for training sdk * Update util tests * Add training cli example notebook (aws#72) Co-authored-by: Roja Reddy Sareddy <[email protected]> * Address comments * fix tls flag issue, fsx endpoint successfully created with cli notebook * clear notebook outputs * minor update in notebook * minor change to notebook * Move Metadata model to common (aws#75) Co-authored-by: pintaoz <[email protected]> * REstructure HPCLI * Fix training cli unit tests * Fix list jobs test * Fixed logger Logger sometimes does not function properly. Tested in example notebook * Updates from Testing * Update import path * Revert lines from readme (should not have been updated) * unit test for inference CLI done * resolve merge conflicts * rebase with master * clean up * clean up recipes * Merging hyp and hyperpod commands in a common entry point as hyp * Removing not relevant directories and updating setup and pyproject (aws#87) * Add unit test and fix HyperPod Manager (aws#84) * Add unit test and fix HyperPod Manager 1. Default namespace can be set by HyperpodManager.set_context() 2. Added unit tests for inference * Remove debug print * update print for inference CLI for list and describe, bug fix for since-hours flag to support float, minor update to notebook (aws#85) * Append uuid to endpoint name (aws#90) * Add unit test and fix HyperPod Manager 1. Default namespace can be set by HyperpodManager.set_context() 2. Added unit tests for inference * Remove debug print * Append uuid to model name and endpoint name * minor fix in create method * Fix set_context in HyperPodManager (aws#91) * Add unit test and fix HyperPod Manager 1. Default namespace can be set by HyperpodManager.set_context() 2. Added unit tests for inference * Remove debug print * Append uuid to model name and endpoint name * minor fix in create method * Fix set_context in HyperPodManager * Add logging info for delete() * Remove Self from type hint (aws#92) * Add unit test and fix HyperPod Manager 1. Default namespace can be set by HyperpodManager.set_context() 2. Added unit tests for inference * Remove debug print * Append uuid to model name and endpoint name * minor fix in create method * Fix set_context in HyperPodManager * Add logging info for delete() * Remove Self in type hint This only supports python version 3.11+ * Minor documentation fixes for RIG Helm (aws#93) * Bug fix: Fixed create command job error (aws#94) Co-authored-by: Roja Reddy Sareddy <[email protected]> * [HyperPod Inference] Update RBAC with perms for KEDA, allow direct provision of operator image repository (aws#44) * change: add rbac perms for KEDA scaledobject * change: allow image.repository to be set directly via flag * change: consistently use namePrefix for app name and resources * fix: remove empty string as default value * fix: reference correct value for tls cert bucket URI fix: override empty image.repository values from domain map change: use shorter prefix for namespace change: do not require sageMakerEndpoint * Adding dynamic flag for dependencies installation (aws#95) * Add utils unit tests for training cli (aws#97) * Bug fix: Fixed create command job error * Add utils unit tests for training cli --------- Co-authored-by: Roja Reddy Sareddy <[email protected]> * Add instance type validation for JS model (aws#98) * Adding observability notebook (aws#96) * Inference dogfood notebook update (aws#99) * update print for inference CLI for list and describe, bug fix for since-hours flag to support float, minor update to notebook * change hyperpod to hyp in inferece cli notebook * update inference CLI notebook to reflect uuid change * Unique job name: Append uuid to training job name (aws#101) * Bug fix: Fixed create command job error * Add utils unit tests for training cli * Unique job name: Append uuid to training job name * Unique job name: Append uuid to training job name --------- Co-authored-by: Roja Reddy Sareddy <[email protected]> * Inference CLI update after dogfood (aws#102) * update print for inference CLI for list and describe, bug fix for since-hours flag to support float, minor update to notebook * change hyperpod to hyp in inferece cli notebook * update inference CLI notebook to reflect uuid change * update list and describe after dogfood callout, remove get_logs for inference CLI, update help text for CLI * Lookup standard Helm release name for RIG Helm installation (1ff9c) (aws#104) * Minor negative case update for Helm release name lookup during RIG Helm installation (aws#105) * Add JumpStart PublicHub model visualization utilities. (aws#106) * Add JumpStart PublicHub model visualization utilities. * Add JumpStart PublicHub model visualization utilities. * Update cli command noun to hyp-*, logging, list_jobs bug fix (aws#107) * Bug fix: Fixed create command job error * Add utils unit tests for training cli * Unique job name: Append uuid to training job name * Unique job name: Append uuid to training job name * Update command verb name to hyp, logging, list_jobs bug fix * Update command verb name to hyp, logging, list_jobs bug fix --------- Co-authored-by: Roja Reddy Sareddy <[email protected]> * Make metadata name same as endpoint name; Updated instance type validation (aws#110) Unit test passes and verified in jupyter notebook * Add integ test for training CLI and SDK (aws#100) * Add integ test for training cli * Add integ test for training sdk * relax pydantic version * fix pydantic version * return latest cluster and fix set cluster context test --------- Co-authored-by: adishaa <[email protected]> * baseline inference integration test for CLI and SDK, minor bug fixes (aws#111) * baseline inference integration test for CLI and SDK, minor bug fix for inference cli, clear inference sdk notebook output * clean up merge header * Remove UUID from training and Inference (aws#108) * Remove UUID from training and Inference * Fixes and PR comments * Fix * Fix logging * Fix * Update inference logging setup similar to training (aws#113) * Bug fix: Fixed create command job error * Add utils unit tests for training cli * Unique job name: Append uuid to training job name * Unique job name: Append uuid to training job name * Update command verb name to hyp, logging, list_jobs bug fix * Update command verb name to hyp, logging, list_jobs bug fix * Update inference logging setup similar to training --------- Co-authored-by: Roja Reddy Sareddy <[email protected]> * Change hp-pytorch-job to hyp-pytorch-job (aws#115) Co-authored-by: adishaa <[email protected]> * Add methods for list pods and namespaces (aws#114) Added unit test and tested in notebook * Minor change in training cli notebook: UUID removed (aws#117) * Bug fix: Fixed create command job error * Add utils unit tests for training cli * Unique job name: Append uuid to training job name * Unique job name: Append uuid to training job name * Update command verb name to hyp, logging, list_jobs bug fix * Update command verb name to hyp, logging, list_jobs bug fix * Update inference logging setup similar to training * Minor change in training cli notebook: UUID removed --------- Co-authored-by: Roja Reddy Sareddy <[email protected]> * Cleaner error messading for Endpoint invoke (aws#112) * Invoke Validation check * Fix * Bumping kubernetes python client version and updating observability command (aws#116) * change: add prefix to convert bucket name to s3 URI (aws#109) * Added type check on commands before invoking subprocess run (aws#118) * Bring HyperPodManager class util functions (aws#119) * Bring HyperPodManager class util functions Unit tests pass and verified in notebook * Update init * Add list_pods and get_logs for CLI (Update notebook, integ test, unit test) (aws#120) * baseline inference integration test for CLI and SDK, minor bug fix for inference cli, clear inference sdk notebook output * update integ test in progress * update cli code, notebook, integ and unit test to add list_pods and get_logs * clean up merge header * Update inference and training to only check kubeconfig on the first time (aws#122) Updated unit tests and verified in notebook * Update Readme to include Inference and Training (aws#121) * Update Readme to include Inference and Training * Update readme command * Documentation updates * Doc Updates * Move observability utils and constants; Rename set_context/get_context (aws#125) * Update inference and training to only check kubeconfig on the first time Updated unit tests and verified in notebook * Remove old unit tests * Revert "Remove old unit tests" This reverts commit e728e9864c853635f724e9a377fbe870f0f2e2a4. * Move observability utils and constants; Rename set_context/get_context * Updating template packages name and structure (aws#126) * Changelog updates (aws#128) * Changelog updates * Rebase and update * Fix * Readme update (aws#129) * Update Readme to include Inference and Training * Update readme command * Documentation updates * Doc Updates * Readme updates * Fix README.md * Remove Orchestrator from List Cluster * Changes to README.md * Fix the link * Remove orchestrator from README.md * Unit test fix (aws#127) * use unique basename for test file modules * fix unit tests * remove append_uuid test * fix failing test_invoke tests --------- Co-authored-by: adishaa <[email protected]> * Fix get_cluster_context runtime error (aws#130) * Remove Py38 Tests (aws#131) * Fix get_cluster_context runtime error * Remove Py38 fromtests * UNit test fixes (aws#132) * Fix get_cluster_context runtime error * Remove Py38 fromtests * Fix * Unit test fixes * Inference integ tests all passed in Chait's account (aws#135) * baseline inference integration test for CLI and SDK, minor bug fix for inference cli, clear inference sdk notebook output * update integ test in progress * update cli code, notebook, integ and unit test to add list_pods and get_logs * clean up merge header * inference integ tests all passing in chait's account * Update operator namespace string (aws#137) * Inference integ test passed on beta account (aws#140) * baseline inference integration test for CLI and SDK, minor bug fix for inference cli, clear inference sdk notebook output * update integ test in progress * update cli code, notebook, integ and unit test to add list_pods and get_logs * clean up merge header * inference integ tests all passing in chait's account * integ test passing on beta account * is_kubeconfig_loaded Fix (aws#139) * Test PR * Fix is_kubeconfig_loaded Class attribute bug * Include main branch in pull request target --------- Co-authored-by: Mahadeva N <[email protected]> Co-authored-by: Shantanu Tripathi <[email protected]> Co-authored-by: Zhaoqi <[email protected]> Co-authored-by: jzhaoqwa <[email protected]> Co-authored-by: Rahul Sahu <[email protected]> Co-authored-by: rvasahu-amazon <[email protected]> Co-authored-by: pintaoz-aws <[email protected]> Co-authored-by: pintaoz <[email protected]> Co-authored-by: Molly He <[email protected]> Co-authored-by: Amarjeet LNU <[email protected]> Co-authored-by: Roja Reddy Sareddy <[email protected]> Co-authored-by: rsareddy0329 <[email protected]> Co-authored-by: Chris Chan <[email protected]> Co-authored-by: adishaa <[email protected]> Co-authored-by: Aditi Sharma <[email protected]> Co-authored-by: chnnmz <[email protected]>

nargokul · 2025-07-11T19:49:44Z

src/sagemaker/hyperpod/common/telemetry/constants.py

+    SDK_DEFAULTS = 1
+    LOCAL_MODE = 2
+    REMOTE_FUNCTION = 3
+    MODEL_TRAINER = 4
+    ESTIMATOR = 5


Nit: Can remove these

removed additional ones

nargokul · 2025-07-11T19:54:12Z

src/sagemaker/hyperpod/inference/hp_endpoint.py

@@ -21,6 +25,7 @@ class HPEndpoint(_HPEndpoint, HPEndpointBase):
    metadata: Optional[Metadata] = Field(default=None)
    status: Optional[InferenceEndpointConfigStatus] = Field(default=None)

+    @_hyperpod_telemetry_emitter(Feature.HYPERPOD, "create_endpoint")


Do we need to track CLI and SDK requests separately ?

Added the telemetry to only SDK methods (training and inference), As CLI uses SDK underneath, did not add seperately for it. As it could be a double entry

nargokul · 2025-07-11T19:55:46Z

src/sagemaker/hyperpod/training/hyperpod_pytorch_job.py

@@ -83,6 +88,7 @@ def create(self, debug=False):
            handle_exception(e, self.metadata.name, self.metadata.namespace)

    @classmethod
+    @_hyperpod_telemetry_emitter(Feature.HYPERPOD, "list")


We need to be more specific in places like these to also include the CRD name .

If this is just Hyperpod list or HyperPod create, then this may not be detailed enough

added specific/unique names for each of the methods, especially for methods which have common names across training and inference.

nargokul · 2025-07-11T19:58:19Z

src/sagemaker/hyperpod/cli/commands/cluster.py

@@ -42,7 +42,7 @@
    TEMP_KUBE_CONFIG_FILE,
    OutputFormat,
 )


We also need to add telemetry for the functions like list_cluster

added for list_cluster

nargokul · 2025-07-11T19:58:50Z

src/sagemaker/hyperpod/common/telemetry/telemetry_logging.py

+        _requests_helper(url, 2)
+        logger.info("SageMaker Python SDK telemetry successfully emitted.")
+    except Exception:  # pylint: disable=W0703
+        logger.debug("SageMaker Python SDK telemetry not emitted!")


Should be a warning

done, updated.

Enable Hyperpod telemetry

0dedc06

rsareddy0329 requested a review from a team as a code owner July 11, 2025 07:53

rsareddy0329 temporarily deployed to auto-approve July 11, 2025 07:53 — with GitHub Actions Inactive

Enable Hyperpod telemetry

6e1ac1b

rsareddy0329 temporarily deployed to auto-approve July 11, 2025 19:06 — with GitHub Actions Inactive

Enable Hyperpod telemetry

bbe5e25

rsareddy0329 temporarily deployed to auto-approve July 11, 2025 19:40 — with GitHub Actions Inactive

nargokul requested changes Jul 11, 2025

View reviewed changes

nargokul reviewed Jul 11, 2025

View reviewed changes

rsareddy0329 and others added 2 commits July 11, 2025 14:34

Merge branch 'aws:main' into main-enable-telemetry

4bc6c69

Enable Hyperpod telemetry

a0d5daf

rsareddy0329 had a problem deploying to auto-approve July 11, 2025 21:34 — with GitHub Actions Error

rsareddy0329 temporarily deployed to auto-approve July 11, 2025 21:34 — with GitHub Actions Inactive

Enable Hyperpod telemetry

df2b257

rsareddy0329 temporarily deployed to auto-approve July 11, 2025 21:36 — with GitHub Actions Inactive

Enable Hyperpod telemetry

c7e8167

rsareddy0329 temporarily deployed to auto-approve July 11, 2025 22:29 — with GitHub Actions Inactive

rsareddy0329 temporarily deployed to auto-approve July 11, 2025 22:32 — with GitHub Actions Inactive

Aditi2424 approved these changes Jul 11, 2025

View reviewed changes

rsareddy0329 merged commit 664be9d into aws:main Jul 12, 2025
12 of 24 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Enable Hyperpod telemetry #116

Enable Hyperpod telemetry #116

Uh oh!

rsareddy0329 commented Jul 11, 2025

Uh oh!

nargokul Jul 11, 2025

Uh oh!

rsareddy0329 Jul 11, 2025

Uh oh!

nargokul Jul 11, 2025

Uh oh!

rsareddy0329 Jul 11, 2025

Uh oh!

nargokul Jul 11, 2025

Uh oh!

rsareddy0329 Jul 11, 2025

Uh oh!

nargokul Jul 11, 2025

Uh oh!

rsareddy0329 Jul 11, 2025

Uh oh!

nargokul Jul 11, 2025

Uh oh!

rsareddy0329 Jul 11, 2025

Uh oh!

Uh oh!

Uh oh!

Enable Hyperpod telemetry #116

Enable Hyperpod telemetry #116

Uh oh!

Conversation

rsareddy0329 commented Jul 11, 2025

PR Approval Steps

For Requester

For Reviewer

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!