Add Baseline regression notes #3187

ddbeck · 2025-07-28T15:00:15Z

This is a work in progress toward fixing #1971.

This structures the existing Baseline regression comments as notes. I'm looking for early feedback on whether this structure for notes would be useful.

ddbeck · 2025-07-28T15:03:00Z

features/streams.yml

-# TODO: https://github.com/web-platform-dx/web-features/issues/1971
-# Status changed: https://github.com/web-platform-dx/web-features/pull/2358, https://github.com/web-platform-dx/web-features/pull/2491
-# 2024-12-19 — low → false — Regressed status to match Caniuse, which considers support beginning at BYOB shipping.
-# 2025-01-30 — false → high — Split BYOB into a separate "readable-byte-streams" feature. Linked that one to Caniuse.
-# References:
-# - https://caniuse.com/streams


I fussed over this for a while and ended up deciding to omit the note entirely. The first note would say that the feature regressed, the second note would be some new category (advance? unregression?) showing that we more or less reverted the second note. It seemed to me that the correct course of action in that scenario would be to withdraw the note, so that's what I've done.

I tend to agree. If our consumers really only care about baseline regressions, then we don't need a note here. Unless we have reasons to believe that the history of a feature would be useful. Which I don't think we do.

features/content-visibility.yml

ddbeck · 2025-08-30T18:29:17Z

features/popover.yml

+# notes:
+#   - date: 2024-09-18
+#     category: baseline-regression
+#     old_baseline_value: low
+#     new_baseline_value: false
+#     message: >
+#       Safari on iOS has a bug that prevents light dismiss (tapping outside the element to close it).
+#     citations:
+#       - https://bugs.webkit.org/show_bug.cgi?id=267688
+#       - https://github.com/mdn/browser-compat-data/issues/22927


When I opened this PR, I had written this note. However, the feature has since progressed. It occurred to me that we should not show irrelevant notes, so I commented it out. This is a rather new idea, so I'd like to delete this entirely, if we decide this is the right way to go.

Other options considered: some sort of "historic" flag on notes that are no longer relevant, or filtering historic notes from the published package. I figured both of those added complexity that wouldn't be very useful to anyone, least of all developers.

I almost feel like we should have a linting step in place which checks that notes.new_baseline_value is equal to status.baseline, and if not, deletes that particular note. I think keeping old notes about status regressions that no longer apply would be confusing. I don't think that web-features necessarily needs to keep track of the history of a feature. Its role is to document the platform as it is now.

ddbeck · 2025-08-30T18:37:38Z

I've updated this PR. Here's what's new:

The schema has been added. To match the description field, there is now a plain-text message field and a formatted message_html in the built package.
Regression notes that apply to a historic status are now forbidden. For example, if a feature regresses from high to low, then advances back to high, then the script that ingests the YAML treats that as an error. My working assumption is that such historic notes have no utility to web developers and ought to be dropped. This would result in the loss of one regression note. See my self-review for more detail and possible alternatives.
There are now guidelines for adding regression notes. I wrote the guidelines to emphasize impact on developers. This led to…
The regression notes have been revised. I tried to rewrite them to help developers make a decision about whether the regression is meaningful to their applications. This makes for much stronger notes, in my opinion.
The script that ingests the YAML files has been updated to handle the new fields and do the requisite conversions.

This PR now passes the tests and is ready for (real) review. I'll share this with the WebDX chat on Monday.

captainbrosset · 2025-09-01T09:11:19Z

features/popover.yml

+# notes:
+#   - date: 2024-09-18
+#     category: baseline-regression
+#     old_baseline_value: low
+#     new_baseline_value: false
+#     message: >
+#       Safari on iOS has a bug that prevents light dismiss (tapping outside the element to close it).
+#     citations:
+#       - https://bugs.webkit.org/show_bug.cgi?id=267688
+#       - https://github.com/mdn/browser-compat-data/issues/22927


I almost feel like we should have a linting step in place which checks that notes.new_baseline_value is equal to status.baseline, and if not, deletes that particular note. I think keeping old notes about status regressions that no longer apply would be confusing. I don't think that web-features necessarily needs to keep track of the history of a feature. Its role is to document the platform as it is now.

captainbrosset · 2025-09-01T09:12:30Z

features/streams.yml

-# TODO: https://github.com/web-platform-dx/web-features/issues/1971
-# Status changed: https://github.com/web-platform-dx/web-features/pull/2358, https://github.com/web-platform-dx/web-features/pull/2491
-# 2024-12-19 — low → false — Regressed status to match Caniuse, which considers support beginning at BYOB shipping.
-# 2025-01-30 — false → high — Split BYOB into a separate "readable-byte-streams" feature. Linked that one to Caniuse.
-# References:
-# - https://caniuse.com/streams


I tend to agree. If our consumers really only care about baseline regressions, then we don't need a note here. Unless we have reasons to believe that the history of a feature would be useful. Which I don't think we do.

captainbrosset · 2025-09-01T09:13:11Z

index.ts

+    // Ensure regression notes are still relevant
+    if (Array.isArray(data.notes)) {
+        for (const [index, note] of (data.notes as FeatureData["notes"]).entries()) {
+            if (note.new_baseline_value !== data.status.baseline) {


Ah cool! I see you've actually already done what I suggested in an earlier comment.

captainbrosset · 2025-09-01T09:15:08Z

types.ts

    /** Whether developers are formally discouraged from using this feature */
    discouraged?: Discouraged;
+    /** Notes about this feature */
+    notes?: BaselineRegressionNote[];


Do we really need this to be an array? I don't think we do, but I can see the point that not making it an array might, some day, block us, and require a major release to change it then.

Good question, @captainbrosset!

I think it's very likely that we'll have other note types and notes may need to coexist. My inclination was to introduce one note type, then add more. But I recognize it looks a little odd now because there really ought to be just one note at a time under the constraints presented here. But there's some likely follow up note types:

First, consider the mirror of a regression: a case where we've prevented a feature from advancing (think of it as "baseline-delay" categeory or something). There are two open cases of this right now, see #3228. It's possible (though rare) that a feature could have regressed and be prevented from advancing.

Second, there are cases where we might choose to not regress a feature, like #2957 (though it's still my preference to let it regress). It seems to me it might be wise for us to (publicly) acknowledge the bug, even if we do not believe it enough to regress the feature (something like "known-incompatibility", though ). That said, this type of note would be more tenuous (we might consider attaching notes to supported version number, for instance).

I imagine there are still further note types (e.g., something like solicitations for input on potentially deprecating or removing a feature, for instance, as in the case of XSLT), but I haven't given them a ton of thought yet.

I see. The notes property is broad enough that we might indeed use it for other things. Based on this, I'm fine on making this an array. However, if we do add other types of notes in the future, that will still be a breaking change, right? Cause the schema will change. That might be fine by the way.

ddbeck · 2025-09-12T14:05:01Z

@rviscomi would you be interested in re-reviewing this, now that there's a proper schema and such for it? I'm especially curious to know if this is something you'd be interested in building something against soon or not.

If you want to try playing with the data, then these steps ought to work:

Check out this branch.
Run npm install and npm run build:extended. This generates data.extended.json in the root of the repository.

rviscomi

I've tested this in Timebase (along with a patch to fix the note messages) to give an idea of how this might be integrated in developer tools/resources:

I've left a few comments in places where things don't appear as expected

rviscomi · 2025-09-12T19:44:38Z

index.ts

    }
+    if (Array.isArray(data.notes)) {
+        for (const note of data.notes as FeatureData["notes"]) {
+            const { text, html } = convertMarkdown(data.description);


This appears to copy the feature description into the note message. I think it should be something like this:

Suggested change

const { text, html } = convertMarkdown(data.description);

const { text, html } = convertMarkdown(note.message);

rviscomi · 2025-09-12T20:03:10Z

features/font-variant-position.yml

+    old_baseline_value: low
+    new_baseline_value: false
+    message: >
+      Chrome, Edge, and Safari do not implement font synthesis for missing superscript or subscript glyphs.


This message is partially contradicted by the support data for Safari:

web-features/features/font-variant-position.yml.dist

Lines 9 to 10 in 7fb07e2

safari: "9.1"

safari_ios: "9.3"

rviscomi · 2025-09-12T20:23:40Z

features/createimagebitmap.yml

+  - date: 2025-08-11
+    category: baseline-regression
+    old_baseline_value: high
+    new_baseline_value: low


Should the baseline_low_date be 2025-08-11 accordingly?

web-features/features/createimagebitmap.yml.dist

Line 6 in 7fb07e2

baseline_low_date: 2023-12-11

foolip · 2025-11-17T16:13:42Z

schemas/data.schema.json

+              },
+              "new_baseline_value": {
+                "description": "The `baseline` status value after the regression",
+                "enum": ["low", false],


How much value do we get in practice from letting the types exclude one of the possible Baseline statuses for old and new? Can we just use the generic definition and thus sidestep the stuff that quicktype is generating funny types for?

Add Baseline regression notes

34f003a

github-actions bot added the feature definition Creating or defining new features or groups of features. label Jul 28, 2025

ddbeck commented Jul 28, 2025

View reviewed changes

ddbeck added package:web-features schema Schema changes, proposals, and bugs minor version required This PR requires a minor version semver release (vX.Y+1.0) labels Jul 28, 2025

rviscomi reviewed Jul 28, 2025

View reviewed changes

features/content-visibility.yml Outdated Show resolved Hide resolved

ddbeck added 6 commits August 11, 2025 17:09

Merge branch 'main' into regression-notes

89c742b

Structure the Baseline value change

fccb850

Add regression note for createimagebitmap

241e4e5

Add Baseline regression notes to the schema

a96e279

Add Baseline regression note data guidelines

f632460

Revise regression notes to follow the new guidelines

e877da5

github-actions bot added the documentation Improvements or additions to documentation label Aug 30, 2025

Merge branch 'main' into regression-notes

7fb07e2

ddbeck commented Aug 30, 2025

View reviewed changes

ddbeck marked this pull request as ready for review August 30, 2025 18:30

captainbrosset reviewed Sep 1, 2025

View reviewed changes

rviscomi requested changes Sep 12, 2025

View reviewed changes

rviscomi mentioned this pull request Sep 15, 2025

Publish extended JSON in main web-features package #3060

Closed

Merge branch 'main' into regression-notes

29a5b9d

foolip approved these changes Nov 17, 2025

View reviewed changes

	const { text, html } = convertMarkdown(data.description);
	const { text, html } = convertMarkdown(note.message);

Add Baseline regression notes #3187

Are you sure you want to change the base?

Add Baseline regression notes #3187

Uh oh!

Conversation

ddbeck commented Jul 28, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ddbeck commented Aug 30, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ddbeck commented Sep 12, 2025

Uh oh!

rviscomi left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants