Skip to content

Conversation

@ysiraichi
Copy link
Collaborator

This PR addresses issue #9659, caching bazel build artifacts on --disk_cache using GitHub cache action. This should allow PRs from external repositories (i.e. forks) to benefit from build caching. That's because remote cache is only enabled for PRs from within pytorch/xla repository.

In summary, this PR will add the following cache behavior:

  1. Every commit pushed to master or any release candidate branch (e.g. rX.X) will:
    • Create a new disk cache
    • Populate the cache as it builds PyTorch/XLA
    • Save the populated disk cache, associating it with the current branch, and the current commit
  2. Every PR on branch X (either master or a release candidate branch) will:
    • Try to restore the cache associated with X at the commit it's trying to merge with
    • Build PyTorch/XLA using the restored disk cache
      • If we don't actually find a cache to restore, we won't use a disk cache

Note that we only have 10GB of cache storage. So, in order to minimize it, I made the following decisions:

  • The disk caches created in (1) won't restore any cache in the beginning
    • Smaller caches
  • The disk caches restored in (2) won't be saved in the end
    • Fewer caches

@ysiraichi
Copy link
Collaborator Author

This PR should not use (save or restore) any cache, since caches are only created in push events to either master or release branches. We shall only see it being used when:

  • It gets pushed to master
  • A new PR (from a fork would be more noticeable) is opened after that

@ysiraichi
Copy link
Collaborator Author

@zhanyong-wan @ghpvnist This is a friendly ping. Could you take a look at this whenever you have some time?

Copy link
Collaborator

@zhanyong-wan zhanyong-wan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Let's give it a try!

@ysiraichi ysiraichi merged commit c8b09f5 into master Oct 31, 2025
37 of 38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants