-
Notifications
You must be signed in to change notification settings - Fork 131
test(codspeed): fixup #1153: open_and_close_vvmをignoreする
#1169
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
test(codspeed): fixup #1153: open_and_close_vvmをignoreする
#1169
Conversation
CodSpeed Performance ReportMerging #1169 will degrade performances by 11.05%Comparing
|
| Benchmark | BASE |
HEAD |
Change | |
|---|---|---|---|---|
| ❌ | unload_and_load_vvm |
1.7 s | 1.9 s | -10.71% |
| ❌ | unload_and_load_vvm |
1.9 s | 2.1 s | -11.05% |
Footnotes
-
No successful run was found on
main(148dc75) during the generation of this report, so abef143 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩ -
10 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR addresses instability issues with the open_and_close_vvm benchmark test that was causing significant variance (±25%) in CodSpeed measurements. The change makes this specific benchmark ignored by default in CodSpeed continuous measurements while preserving the ability to run it manually when needed.
- Adds configuration option to control inclusion of the
open_and_close_vvmbenchmark - Sets the benchmark to be ignored by default to exclude it from CodSpeed measurements
- Updates documentation to reflect the new configuration option
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
遅れました、まぁ問題なさそう。
マージはお任せします。

内容
#1153 で用意したベンチのうち
open_and_close_vvmについて、これだけCodSpeed用の計測では凄まじく(具体的には±25%くらい)ブレが生じていた。そのためCodSpeedでの継続的な計測は不可能と判断し、デフォルトでignoreされるようにしてCodSpeedの対象から外す。関連 Issue
その他
[追記] このPRをマージし次第、
open_and_close_vvmを"archive"する。