"Navigating the compiler from the outside"

[Michael-F-Bryan]

I'm currently working on a project which uses rustc internals to analyse a crate so I can then generate FFI bindings. Kinda like cbindgen, except where they focus on static linking at compile time or dynamically when your program gets loaded into memory, I'm trying to load a dynamic library at runtime using dlopen (*nix) and LoadLibrary (windows).

Seeing as this is a guide on how rustc works under the hood, would it be worth adding a chapter that mentions:

• how people can use rustc internals as a library,
• how you ask rustc_driver to run some code after a particular phase, and
• how you can extract information from an analysed crate?

My code is on GitLab so people in a similar situation or writing this chapter can use that as a reference.

---

This is slightly off topic, but I'm really impressed with how easy the compiler internals (especially the HIR and Session) are to work with once you understand how they're designed! It was initially a bit confusing because there are loads of data structures (unfortunately, not many have doc-comments) and lots of complexity, but after an hour or two of jumping around the crate docs I was able to be quite proficient in finding the information I want.

[Michael-F-Bryan]

Also, I remember seeing a tutorial a while back which steps you through using rustc_driver and rustc internals to analyse a crate's AST. It was published on GitHub by someone on the core team (maybe @alexcrichton?) and I think it did something like count the number of print statements, or another seemingly trivial task.

It'd be nice to add a link to those sorts of tutorials if they aren't too outdated because they help to give you a starting point to build off of.

[mark-i-m]

Hmm... So far the guide had been about how the compiler itself works, but I'm not opposed to adding this here. We could also add the shiny new rustdoc README...

[Michael-F-Bryan]

On second thoughts I don't think it'd be necessary to write anything specific to those using rustc internals as a library, per-se. It's probably enough to talk about how rustc_driver drives the entire compilation process and the ways people are able to hook in at various stages.

We could also add the shiny new rustdoc README

Rustdoc has a new README? 😮

[mark-i-m]

I frankly don't know that much about this topic, so I would defer to you on what would be most useful in such a chapter :)

Rustdoc has a new README? 😮

Indeed! rust-lang/rust#48283

[alexcrichton]

maybe @alexcrichton?

Unfortunately not me :(

[ehuss]

a tutorial a while back which steps you through using rustc_driver and rustc internals to analyse a crate's AST

Perhaps you are thinking of https://github.com/nrc/stupid-stats?

[Michael-F-Bryan]

Perhaps you are thinking of https://github.com/nrc/stupid-stats?

Yep, that's the one! Given how the compiler is currently going from being pass-based to query-based, does anyone know if this CompilerCalls mechanism still the best way to use rustc from the outside?