Skip to content

Commit ea5adbf

Browse files
authored
docs: TSAN with PyO3 projects (#4962)
* add docs on how to use TSAN * add sample rustup commands * fix markdown * reword and clarify
1 parent ccebbde commit ea5adbf

File tree

1 file changed

+59
-0
lines changed

1 file changed

+59
-0
lines changed

guide/src/debugging.md

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -334,3 +334,62 @@ To use these functions:
334334
1. Run the cell containing these functions in your Jupyter notebook
335335
2. Run `update_launch_json()` in a cell
336336
3. In VS Code, select the "Debug PyO3 (Jupyter)" configuration and start debugging
337+
338+
339+
## Thread Safety and Compiler Sanitizers
340+
341+
PyO3 attempts to match the Rust language-level guarantees for thread safety, but
342+
that does not preclude other code outside of the control of PyO3 or buggy code
343+
managed by a PyO3 extension from creating a thread safety issue. Analyzing
344+
whether or not a piece of Rust code that uses the CPython C API is thread safe
345+
can be quite complicated, since many Python operations can lead to arbitrary
346+
Python code execution. Automated ways to discover thread safety issues can often
347+
be more fruitful than code analysis.
348+
349+
[ThreadSanitizer](https://clang.llvm.org/docs/ThreadSanitizer.html) is a thread
350+
safety checking runtime that can be used to detect data races triggered by
351+
thread safety bugs or incorrect use of thread-unsafe data structures. While it
352+
can only detect data races triggered by code at runtime, if it does detect
353+
something the reports often point to exactly where the problem is happening.
354+
355+
To use `ThreadSanitizer` with a library that depends on PyO3, you will need to
356+
install a nightly Rust toolchain, along with the `rust-src` component, since you
357+
will need to compile the Rust standard library:
358+
359+
```bash
360+
rustup install nightly
361+
rustup override set nighty
362+
rustup component add rust-src
363+
```
364+
365+
You will also need a version of CPython compiled using LLVM/Clang with the same
366+
major version of LLVM as is currently used to compile nightly Rust. As of March
367+
2025, Rust nightly uses LLVM 20.
368+
369+
The [cpython_sanity docker images](https://github.com/nascheme/cpython_sanity)
370+
contain a development environment with a pre-compiled version of CPython 3.13 or
371+
3.14 as well as optionally NumPy and SciPy, all compiled using LLVM 20 and
372+
ThreadSanitizer.
373+
374+
After activating a nightly Rust toolchain, you can build your project using
375+
`ThreadSanitizer` with the following command:
376+
377+
```bash
378+
RUSTFLAGS="-Zsanitizer=thread" maturin develop -Zbuild-std --target x86_64-unknown-linux-gnu
379+
```
380+
381+
If you are not running on an x86_64 Linux machine, you should replace
382+
`x86_64-unknown-linux-gnu` with the [target
383+
triple](https://doc.rust-lang.org/rustc/platform-support.html#tier-1-with-host-tools)
384+
that is appropriate for your system. You can also replace `maturin develop` with
385+
`cargo test` to run `cargo` tests. Note that `cargo` runs tests in a thread
386+
pool, so `cargo` tests can be a good way to find thread safety issues.
387+
388+
You can also replace `-Zsanitizer=thread` with `-Zsanitizer=address` or any of
389+
the other sanitizers that are [supported by
390+
Rust](https://doc.rust-lang.org/beta/unstable-book/compiler-flags/sanitizer.html). Note
391+
that you'll need to build CPython from source with the appropriate [configure
392+
script
393+
flags](https://docs.python.org/3/using/configure.html#cmdoption-with-address-sanitizer)
394+
to use the same sanitizer environment as you want to use for your Rust
395+
code.

0 commit comments

Comments
 (0)