We should have a test that verifies the contents of configuration.md against the default serialized configuration so we know the configuration is up to date and accurate. We do something like this for the CLI arguments already.
Originally posted by @asbjornu in #4009 (comment)