Prevent new lines written by write_str from breaking source maps
#971
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
thecrypticace
added a commit
to tailwindlabs/tailwindcss
that referenced
this pull request
May 8, 2025
Closes #13694 Closes #13591 # Source Maps Support for Tailwind CSS This PR adds support for source maps to Tailwind CSS v4 allowing us to track where styles come from whether that be user CSS, imported stylesheets, or generated utilities. This will improve debuggability in browser dev tools and gives us a good foundation for producing better error messages. I'll go over the details on how end users can enable source maps, any limitations in our implementation, changes to the internal `compile(…)` API, and some details and reasoning around the implementation we chose. ## Usage ### CLI Source maps can be enabled in the CLI by using the command line argument `--map` which will generate an inline source map comment at the bottom of your CSS. A separate file may be generated by passing a file name to `--map`: ```bash # Generates an inline source map npx tailwindcss -i input.css -o output.css --map # Generates a separate source map file npx tailwindcss -i input.css -o output.css --map output.css.map ``` ### PostCSS Source maps are supported when using Tailwind as a PostCSS plugin *in development mode only*. They may or may not be enabled by default depending on your build tool. If they are not you may be able to configure them within your PostCSS config: ```jsonc // package.json { // … "postcss": { "map": { "inline": true }, "plugins": { "@tailwindcss/postcss": {}, }, } } ``` ### Vite Source maps are supported when using the Tailwind CSS Vite plugin in *development mode only* by enabling the `css.devSourcemap` setting: ```js import tailwindcss from "@tailwindcss/vite"; import { defineConfig } from "vite"; export default defineConfig({ plugins: [tailwindcss()], css: { devSourcemap: true, }, }) ``` Now when a CSS file is requested by the browser it'll have an inline source map comment that the browser can use. ## Limitations - Production build source maps are currently disabled due to a bug in Lightning CSS. See parcel-bundler/lightningcss#971 for more details. - In Vite, minified CSS build source maps are not supported at all. See vitejs/vite#2830 for more details. - In PostCSS, minified CSS source maps are not supported. This is due to the complexity required around re-associating every AST node with a location in the generated, optimized CSS. This complexity would also have a non-trivial performance impact. ## Testing Here's how to test the source map functionality in different environments: ### Testing the CLI 1. Setup typical project that the CLI can use and with sources to scan. ```css @import "tailwindcss"; @utilty my-custom-utility { color: red; } /* to test `@apply` */ .card { @apply bg-white text-center shadow-md; } ``` 2. Build with source maps: ```bash bun /path/to/tailwindcss/packages/@tailwindcss-cli/src/index.ts --input input.css -o output.css --map ``` 3. Open Chrome DevTools, inspect an element with utility classes, and you should see rules pointing to `input.css` or `node_modules/tailwindcss/index.css` ### Testing with Vite Testing in Vite will require building and installing necessary files under `dist/*.tgz`. 1. Create a Vite project and enable source maps in `vite.config.js`: ```js import tailwindcss from "@tailwindcss/vite"; import { defineConfig } from "vite"; export default defineConfig({ plugins: [tailwindcss()], css: { // This line is required for them to work devSourcemap: true, }, }) ``` 2. Add a component that uses Tailwind classes and custom CSS: ```jsx // ./src/app.jsx export default function App() { return ( <div className="bg-blue-500 my-custom-class"> Hello World </div> ) } ``` ```css /* ./src/styles.css */ @import "tailwindcss"; @utilty my-custom-utility { color: red; } /* to test `@apply` */ .card { @apply bg-white text-center shadow-md; } ``` 3. Run `npm run dev`, open DevTools, and inspect elements to verify source mapping works for both utility classes and custom CSS. ### Testing with PostCSS CLI 1. Create a test file and update your PostCSS config: ```css /* input.css */ @import "tailwindcss"; @layer components { .card { @apply p-6 rounded-lg shadow-lg; } } ``` ```jsonc // package.json { // … "postcss": { "map": { "inline": true }, "plugins": { "/path/to/tailwindcss/packages/packages/@tailwindcss-postcss/src/index.ts": {} } } } ``` 2. Run PostCSS through Bun: ```bash bunx --bun postcss ./src/index.css -o out.css ``` 3. Inspect the output CSS - it should include an inline source map comment at the bottom. ### Testing with PostCSS + Next.js Testing in Next.js will require building and installing necessary files under `dist/*.tgz`. However, I've not been able to get CSS source maps to work in Next.js without this hack: ```js const nextConfig: NextConfig = { // next.js overwrites config.devtool so we prevent it from doing so // please don't actually do this… webpack: (config) => Object.defineProperty(config, "devtool", { get: () => "inline-source-map", set: () => {}, }), }; ``` This is definitely not supported and also doesn't work with turbopack. This can be used to test them temporarily but I suspect that they just don't work there. ### Manual source map analysis You can analyze source maps using Evan Wallace's [Source Map Visualization](https://evanw.github.io/source-map-visualization/) tool which will help to verify the accuracy and quality of source maps. This is what I used extensively while developing this implementation. It'll help verify that custom, user CSS maps back to itself in the input, that generated utilities all map back to `@tailwind utilities;`, that source locations from imported files are also handled correctly, etc… It also highlights the ranges of stuff so it's easy to see if there are off-by-one errors. It's easiest to use inline source maps with this tool because you can take the CSS file and drop it on the page and it'll analyze it while showing the file content. If you're using Vite you'll want to access the CSS file with `?direct` at the end so you don't get a JS module back. ## Implementation The source map implementation follows the ECMA-426 specification and includes several key components to aid in that goal: ### Source Location Tracking Each emittable AST node in the compilation pipeline tracks two types of source locations: - `src`: Original source location - [source file, start offset, end offset] - `dst`: Generated source location - [output file, start offset, end offset] This dual tracking allows us to maintain mappings between the original source and generated output for things like user CSS, generated utilities, uses of `@apply`, and tracking theme variables. It is important to note that source locations for nodes _never overlap_ within a file which helps simplify source map generation. As such each type of node tracks a specific piece of itself rather than its entire "block": | Node | What a `SourceLocation` represents | | ----------- | ---------------------------------------------------------------- | | Style Rule | The selector | | At Rule | Rule name and params, includes the `@` | | Declaration | Property name and value, excludes the semicolon | | Comment | The entire comment, includes the start `/*` and end `*/` markers | ### Windows line endings when parsing CSS Because our AST tracks nodes through offsets we must ensure that any mutations to the file do *not* change the lenth of the string. We were previously replacing `\r\n` with `\n` (see [filter code points](https://drafts.csswg.org/css-syntax/#css-filter-code-points) from the spec) — which changes the length of the string and all offsets may end up incorrect. The CSS parser was updated to handle the CRLF token directly by skipping over the `\r` and letting remaining code handle `\n` as it did previously. Some additional tweaks were required when "peeking" the input but those changes were fairly small. ### Tracking of imports Source maps need paths to the actual imported stylesheets but the resolve step for stylesheets happens inside the call to `loadStylesheet` which make the file path unavailable to us. Because of this the `loadStylesheet` API was augmented such that it has to return a `path` property that we can then use to identify imported sources. I've also made the same change to the `loadModule` API for consistency but nothing currently uses this property. The `path` property likely makes `base` redundant but elminating that (if we even want to) is a future task. ### Optimizing the AST Our optimization pass may intoduce some nodes, for example, fallbacks we create for `@property`. These nodes are linked back to `@tailwind utilities` as ultimately that is what is responsible for creating them. ### Line Offset Tables A key component to our source map generation is the line offset table, which was inspired by some ESBuild internals. It stores a sorted list of offsets for the start of each line allowing us to translate offsets to line/column `Position`s in `O(log N)` time and from `Position`s to offsets in `O(1)` time. Creation of the table takes `O(N)` time. This means that we can store code point offsets for source locations and not have to worry about computing or tracking line/column numbers during parsing and serialization. Only when a source map is generated do these offsets need to be computed. This ensures the performance penalty when not using source maps is minimal. ### Source Map Generation The source map returned by `buildSourceMap()` is designed to follow the [ECMA-426 spec](https://tc39.es/ecma426). Because that spec is not completely finalized we consider the result of `buildSourceMap()` to be internal API that may change as the spec chamges. The produces source map is a "decoded" map such that all sources and mappings are in an object graph. A library like `source-map-js` must be used to convert this to an encoded source map of the right version where mappings are encoded with base 64 VLQs. Any specific integration (Vite, PostCSS, etc…) can then use `toSourceMap()` from `@tailwindcss/node` to convert from the internal source map to an spec-compliant encoded source map that can be understood by other tools. ### Handling minification in Lightning Since we use Lightning CSS for optimization, and it takes in an input map, we generate an encoded source map that we then pass to lightning. The output source map *from lighting itself* is then passed back in during the second optimization pass. The final map is then passed from lightning to the CLI (but not Vite or PostCSS — see the limitations section for details). In some cases we have to "fix up" the output CSS. When this happens we use `magic-string` to do the replacement in a way that is trackable and `@amppproject/remapping` to map that change back onto the original source map. Once the need for these fix ups disappear these dependencies can go away. Notes: - The accuracy of source maps run though lightning is reduced as it only tracks on a per-rule level. This is sufficient enough for browser dev tools so should be fine. - Source maps during optimization do not function properly at this time because of a bug in Lightning CSS regarding license comments. Once this bug is fixed they will start working as expected. ### How source locations flow through the system 1. During initial CSS parsing, source locations are preserved. 2. During parsing these source locations are also mapped to the destinations which supports an optimization for when no utilities are generated. 3. Throughout the compilation process, transformations maintain source location data 4. Generated utilities are explicitly pointed to `@tailwind utilities` unless generated by `@apply`. 5. When optimization is enabled, source maps are remapped through lightningcss 6. Final source maps are written in the requested format (inline or separate file)
devongovett
reviewed
May 10, 2025
devongovett
force-pushed
the
feat/fix-source-map-license-comments
branch
from
390abc0 to
af95b25
Compare
devongovett
approved these changes
May 10, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Right now putting license comments in a file causes invalid source maps to be generated.
For example, running this file through lightning with source maps will produce incorrect generated locations:
Here's a screenshot of the source map visualizer for the generated CSS:
This happens because Lightning computes the column offset to be the length of the comment plus the new line character at the end and this happens for each license comment in the file. Additionally, license comments are one of the few things that basically preserve the content as is including new lines. So if the license comment itself contains new lines than the generated mappings can be pretty far off — as the screenshot above shows.
This PR fixes this by tracking new lines and resetting the appropriate state when printing the CSS in
write_str.