-
Notifications
You must be signed in to change notification settings - Fork 13.9k
Make sure fmt-write-bloat doesn't vacuously pass on no symbols
#143669
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
Make sure `fmt-write-bloat` doesn't vacuously pass on no symbols Previously, the test only checks for the absence of certain panic symbols, but having no symbols was also a possible albeit vacuous way to satisfy this assertion. Fix this by checking we at least observe the `main` symbol which is always expected to be present. Noticed in #142841 (comment). r? `@ChrisDenton` try-job: x86_64-msvc-*
This comment was marked as outdated.
This comment was marked as outdated.
|
Hm. Let me try this locally on windows. |
|
The failure is not surprising to me. On Windows, symbols are not contained in the binary itself and are instead in a separate file (the .pdb). So the |
|
Oh of course... D'oh. Let me see if I can cook something up. |
This comment was marked as outdated.
This comment was marked as outdated.
fmt-write-bloat doesn't vacuously pass on no symbolsfmt-write-bloat doesn't vacuously pass on no symbols
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
[WIP] Make sure `fmt-write-bloat` doesn't vacuously pass on no symbols Previously, the test only checks for the absence of certain panic symbols, but having no symbols was also a possible albeit vacuous way to satisfy this assertion. Fix this by checking we at least observe the `main` symbol which is always expected to be present. Noticed in #142841 (comment). r? `@ChrisDenton` try-job: aarch64-apple try-job: x86_64-apple-1 try-job: x86_64-msvc-1 try-job: i686-msvc-1 try-job: x86_64-mingw-1
|
|
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as outdated.
This comment was marked as outdated.
[WIP] Make sure `fmt-write-bloat` doesn't vacuously pass on no symbols Previously, the test only checks for the absence of certain panic symbols, but having no symbols was also a possible albeit vacuous way to satisfy this assertion. Fix this by checking we at least observe the `main` symbol which is always expected to be present. Noticed in #142841 (comment). r? `@ChrisDenton` try-job: aarch64-apple try-job: x86_64-apple-1 try-job: x86_64-msvc-1 try-job: i686-msvc-1 try-job: x86_64-mingw-1
|
@bors r+ rollup=iffy (should be fine to rollup if need be though) |
Make sure `fmt-write-bloat` doesn't vacuously pass on no symbols This PR fixes the `tests/run-make/fmt-write-float/` run-make test to both (1) not vacuously pass on no symbols at all, and (2) compare panics symbols under optimizations versus no optimizations. ### Context Previously, the test only checks for the absence of certain panic symbols, but having no symbols was also a possible albeit vacuous way to satisfy this assertion. Noticed in #142841 (comment). r? `@ChrisDenton` try-job: aarch64-apple try-job: aarch64-msvc-1 try-job: x86_64-msvc-1 try-job: i686-msvc-1 try-job: x86_64-mingw-1
This comment has been minimized.
This comment has been minimized.
|
💔 Test failed - checks-actions |
|
Ah, debug assertions, right |
a51fb59 to
9059fc3
Compare
|
This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed. Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers. |
|
A few things of note since last review:
|
|
@bors try |
This comment has been minimized.
This comment has been minimized.
Make sure `fmt-write-bloat` doesn't vacuously pass on no symbols try-job: aarch64-apple try-job: aarch64-msvc-1 try-job: x86_64-msvc-1 try-job: i686-msvc-1 try-job: x86_64-mingw-1
|
The job Click to see the possible cause of the failure (guessed by this bot) |
|
💔 Test for 89c50a8 failed: CI. Failed jobs:
|
This test suffers from multiple issues that make it very, very difficult to fix, and even if fixed, it would still be too fragile. For some background context, this test tries to check that the optimization introduced in [PR-78122] is not regressed. The optimization is for eliding `usize` formatting machinery and padding code from the final binary. Previously, writing any `fmt::Arguments` would cause the `usize` formatting and padding machinery to be included in the final binary since indexing used in `fmt::write` generates code using `panic_bounds_check` (that prints the index and length). Those bounds check are never hit, since `fmt::Arguments` never contain any out-of-bounds indicies. The `Makefile` version of `fmt-write-bloat` was ported to the present `rmake.rs` test infra in [PR-128147]. However, this PR just tries to maintain the original test logic. The original test, it turns out, already have multiple limitations: - It only runs on non-Windows, since the `no_std` test of the original version tries to link against a `libc`. [PR-128807] worked around this by using a substitute name. We re-enabled this test in [PR-142841], but it turns out the assertions are too weak, it will even vacuously pass for no symbols at all. - In [PR-143669], we tried to make this test more robust by comparing the set of expected versus unexpected panic-related symbols, subject to if std was built with debug assertions. However, in working on [PR-143669], we've come to realize that this test is fundamentally very fragile: - The set of panic symbols depend on whether the standard library was built with or without debug assertions. - Different platforms often have different sets of panic machinery modules, functions and paths, and thus different sets of panic symbols. For instance, x86_64 msvc and i686 msvc have different panic codepaths. - This comes back to the way the test is trying to gauge the absence of panic symbols -- it tries to look for symbol substring matches for "known" panic symbols. This is fundamentally fragile, because the test is trying to peek into the symbols of the resultant binary post-linking, based on fuzzy matches (the symbols are mangled as well). Based on this assessment, we determined that we should remove this test. This is not intended to exclude the possibility of reintroducing a more robust version of this test. For instance, we could consider some kind of more controllable post-link "end product" integration codegen test suite. [PR-78122]: rust-lang#78122 [PR-128147]: rust-lang#128147 [PR-128807]: rust-lang#128807 [PR-142841]: rust-lang#142841 [PR-143669]: rust-lang#143669
|
Superseded by #148393. |
…=ChrisDenton Remove `tests/run-make/fmt-write-bloat/` This test suffers from multiple issues that make it very, very difficult to fix, and even if fixed, it would still be too fragile. So this PR removes `tests/run-make/fmt-write-bloat/`. This PR supersedes rust-lang#143669. r? `@ChrisDenton` (as you reviewed rust-lang#143669 and have context) ### Background context For some background context, this test tries to check that the optimization introduced in [PR-78122] is not regressed. The optimization is for eliding `usize` formatting machinery and padding code from the final binary. Previously, writing any `fmt::Arguments` would cause the `usize` formatting and padding machinery to be included in the final binary since indexing used in `fmt::write` generates code using `panic_bounds_check` (that prints the index and length). Those bounds check are never hit, since `fmt::Arguments` never contain any out-of-bounds indicies. The `Makefile` version of `fmt-write-bloat` was ported to the present `rmake.rs` test infra in [PR-128147]. However, that PR just tries to maintain the original test logic. ### Limitations and problems The original test, it turns out, already have multiple limitations: - It only runs on non-Windows, since the `no_std` test of the original version tries to link against a `libc`. [PR-128807] worked around this by using a substitute name. We re-enabled this test in [PR-142841], but it turns out the assertions are too weak, it will even vacuously pass for no symbols at all. - In [PR-143669], we tried to make this test more robust by comparing the set of expected versus unexpected panic-related symbols, subject to if std was built with debug assertions. However, in working on [PR-143669], we've come to realize that this test is fundamentally very fragile: - The set of panic symbols depend on whether the standard library was built with or without debug assertions. - Different platforms often have different sets of panic machinery modules, functions and paths, and thus different sets of panic symbols. For instance, x86_64 msvc and i686 msvc have different panic code paths. - This comes back to the way the test is trying to gauge the absence of panic symbols -- it tries to look for symbol substring matches for "known" panic symbols. This is fundamentally fragile, because the test is trying to peek into the symbols of the resultant binary post-linking, based on fuzzy matches (the symbols are mangled as well). Based on this assessment, we determined that we should remove this test. This is not intended to exclude the possibility of reintroducing a more robust version of this test. For instance, we could consider some kind of more controllable post-link "end product" integration codegen test suite. [PR-78122]: rust-lang#78122 [PR-128147]: rust-lang#128147 [PR-128807]: rust-lang#128807 [PR-142841]: rust-lang#142841 [PR-143669]: rust-lang#143669
Rollup merge of #148393 - jieyouxu:remove-fmt-write-bloat, r=ChrisDenton Remove `tests/run-make/fmt-write-bloat/` This test suffers from multiple issues that make it very, very difficult to fix, and even if fixed, it would still be too fragile. So this PR removes `tests/run-make/fmt-write-bloat/`. This PR supersedes #143669. r? `@ChrisDenton` (as you reviewed #143669 and have context) ### Background context For some background context, this test tries to check that the optimization introduced in [PR-78122] is not regressed. The optimization is for eliding `usize` formatting machinery and padding code from the final binary. Previously, writing any `fmt::Arguments` would cause the `usize` formatting and padding machinery to be included in the final binary since indexing used in `fmt::write` generates code using `panic_bounds_check` (that prints the index and length). Those bounds check are never hit, since `fmt::Arguments` never contain any out-of-bounds indicies. The `Makefile` version of `fmt-write-bloat` was ported to the present `rmake.rs` test infra in [PR-128147]. However, that PR just tries to maintain the original test logic. ### Limitations and problems The original test, it turns out, already have multiple limitations: - It only runs on non-Windows, since the `no_std` test of the original version tries to link against a `libc`. [PR-128807] worked around this by using a substitute name. We re-enabled this test in [PR-142841], but it turns out the assertions are too weak, it will even vacuously pass for no symbols at all. - In [PR-143669], we tried to make this test more robust by comparing the set of expected versus unexpected panic-related symbols, subject to if std was built with debug assertions. However, in working on [PR-143669], we've come to realize that this test is fundamentally very fragile: - The set of panic symbols depend on whether the standard library was built with or without debug assertions. - Different platforms often have different sets of panic machinery modules, functions and paths, and thus different sets of panic symbols. For instance, x86_64 msvc and i686 msvc have different panic code paths. - This comes back to the way the test is trying to gauge the absence of panic symbols -- it tries to look for symbol substring matches for "known" panic symbols. This is fundamentally fragile, because the test is trying to peek into the symbols of the resultant binary post-linking, based on fuzzy matches (the symbols are mangled as well). Based on this assessment, we determined that we should remove this test. This is not intended to exclude the possibility of reintroducing a more robust version of this test. For instance, we could consider some kind of more controllable post-link "end product" integration codegen test suite. [PR-78122]: #78122 [PR-128147]: #128147 [PR-128807]: #128807 [PR-142841]: #142841 [PR-143669]: #143669
This PR fixes the
tests/run-make/fmt-write-float/run-make test to both (1) not vacuously pass on no symbols at all, and (2) compare panics symbols under optimizations versus no optimizations.Context
Previously, the test only checks for the absence of certain panic symbols, but having no symbols was also a possible albeit vacuous way to satisfy this assertion.
Noticed in #142841 (comment).
r? @ChrisDenton
try-job: aarch64-apple
try-job: aarch64-msvc-1
try-job: x86_64-msvc-1
try-job: i686-msvc-1
try-job: x86_64-mingw-1