Skip to content

Conversation

@gribnoysup
Copy link
Collaborator

The e2e tests for compass web on chrome right now are extremely flaky, most of them are randomly failing with "tab crashed". While there's a small chance that it's actually somehow caused by us, I think the chances are higher is that it's related to us using chrome canary for testing. I think it's really not required for us to run tests on canary, so to see if that's the root cause and to potentially resolve the flakes, I'm changing the logic a bit to use stable chrome when testing web (for FF we only can use latest unfortunately)

@gribnoysup gribnoysup requested a review from a team as a code owner October 31, 2025 13:52
@gribnoysup gribnoysup requested a review from mabaasit October 31, 2025 13:52
@gribnoysup gribnoysup added the no-title-validation Skips validation of PR titles (conventional commit adherence + JIRA ticket inclusion) label Oct 31, 2025
@gribnoysup gribnoysup changed the title chore(e2e): use chrome stable instead of canary for web tests fix(e2e): use chrome stable instead of canary for web tests COMPASS-10009 COMPASS-10010 COMPASS-10012 Oct 31, 2025
@gribnoysup gribnoysup removed the no-title-validation Skips validation of PR titles (conventional commit adherence + JIRA ticket inclusion) label Oct 31, 2025
@github-actions github-actions bot added the fix label Oct 31, 2025
@gribnoysup gribnoysup added no release notes Fix or feature not for release notes and removed fix labels Oct 31, 2025
@github-actions github-actions bot added the fix label Oct 31, 2025
Copy link
Collaborator

@addaleax addaleax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this meant to be temporary? In general, getting early warnings about changing browser behavior isn't a bad thing

@gribnoysup
Copy link
Collaborator Author

gribnoysup commented Oct 31, 2025

I wasn't planning to revert this, but I'm open to opinions. I don't think compass-web does anything that really requires us to test against the very latest (potentially really buggy) versions of browsers and the latest stable is probably ahead enough compared to what most of compass users would actually be using. I generally agree with you, but I think specifically canary is way too risky to have as a default for the bulk of the browser e2e tests. We can try either setting this to beta for chrome or to have a separate, non-blocking set of tests running against canary, what do you think?

@gribnoysup
Copy link
Collaborator Author

(I'll merge to leave the CI greener for the weekend, but really open to revisiting this later)

@gribnoysup gribnoysup merged commit fe6aab8 into main Oct 31, 2025
85 of 88 checks passed
@gribnoysup gribnoysup deleted the use-chrome-stable-for-e2e-tests branch October 31, 2025 15:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fix no release notes Fix or feature not for release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants