Working with Sphinx and reStructuredText

Getting Started

The CIME documentation is written in reStructuredText (a markup language, like HTML, markdown or latex; see the bottom for some helpful resources) and is converted to publishable website by Sphinx. Sphinx can also generate the documentation in many other formats, such as PDF and LaTeX.

The CIME documentation is stored in the doc/source directory of the ESMCI/cime repository, and Sphinx will generate the documentation into the doc/build directory. However, as noted below, we typically do an out-of-source build.

Note: the build directory is ignored and nothing below it should be committed to the repository.

To publish the documentation to http://esmci.github.io/cime/, the build process should be done using a method that generates the HTML website in an appropriate subdirectory of the orphan gh-pages CIME branch.

The next two sections describe how to work with and publish the documentation.

Working with the docs

Note: for the purposes of these instructions, we will refer to the local directory containing your developmental clone (or fork) of CIME as $CIME.

The CIME documentation is contained within the $CIME/doc/source directory. When developing a feature, the documentation can be updated accordingly (if needed) by editing the reStructuredText files in $CIME/doc/source, and an updated version of the documentation should be built to verify your changes.

To build the documentation, you will need to have sphinx installed on your local machine (see Installing sphinx for details). The easiest way to install sphinx is by using pip:

pip install sphinx
pip install sphinxcontrib-programoutput
pip install git+https://github.com/esmci/sphinx_rtd_theme.git@version-dropdown-with-fixes

Note: you may need to update your PATH environment variable to include the path to the sphinx-build script. On a mac, the default location for this script is python-packages/bin/sphinx-build.

Then you can build a local copy of the documentation by navigating to $CIME/doc and issuing these commands:

make clean
make html

which will generate an HTML version of the website in $CIME/doc/build/html. You can view the documentation locally in your favorite web browser. (Note: make html will automatically run make api, which generates API documentation in-source. make clean cleans both the build directory and the in-source generated API documentation.)

Publishing the docs

When a feature is complete and merged into master, the updated documentation will need to be published.

Because the published documentation lives in an orphaned gh-pages branch, it is best to have a separate clone of just the gh-pages branch outside of your CIME development clone. First navigate out of your CIME clone to where you'd like to keep a copy of the published documentation and issue these commands:

git clone -b gh-pages [email protected]:ESMCI/cime.git cime-gh-pages
export CIMEDOCS=$PWD/cime-gh-pages

Note: If you're working in a fork, make sure to check out the gh-pages branch from your fork, NOT ESMCI/cime.

Then, you can navigate back into your development clone, and build a clean version of the documentation. The following assumes that you are building the documentation for master; if you are building for one of the release branches, change the BRANCH variable below.

cd $CIME/doc
export BRANCH=master
make BUILDDIR=${CIMEDOCS}/versions/${BRANCH} clean
make BUILDDIR=${CIMEDOCS}/versions/${BRANCH} html

(As above, note that make html automatically runs make api, which generates API documentation in-source, not in the given BUILDDIR. make clean cleans both the build directory and the in-source generated API documentation.)

Now, commit the new website:

cd $CIMEDOCS
git add .
git commit
git push origin gh-pages

Once those changes are pushed to github, http://esmci.github.io/cime/ will be automatically updated (it may take a few minutes for any cached versions to clear).

Note: if you're working in a fork, make sure to push to your fork and the docs will be hosted at http://UNAME.github.io/cime/, where UNAME is your github user name. Additionally, when you issue a PR for your feature, you can similarly issue a PR for the documentation. To do so, create a new pull request form your fork and set base fork to ESMCI/cime, base to gh-pages, head fork to UNAME/cime, and the compare to gh-pages.

Generating PDF

Sphinx can generate a PDF version of the documentation using a module add-on as follows:

1. Install rst2pdf

• use your package manager (or)
• pip install rst2pdf (or)
• easy_install rst2pdf

2. Edit $CIME/doc/source/conf.py to add rst2pdf to the list of extensions:

• extensions = ['rst2pdf.pdfbuilder']
  • This list will be empty if you accepted the defaults when the project was setup. If not, just append rst2pdf.pdfbuilder to the list.

3. Add a pdf_documents variable to conf.py

• pdf_documents = [(master_doc, u'CIME_Users_Guide', u'CIME Users Guide (PDF)', u'list-of-authors'),]
  • For all supported options, please check The Rst2pdf Handbook. There is also a Sphinx related section towards the end of the manual.

4. Generate the PDF

• cd $CIME/doc
• sphinx-build -b pdf source build/pdf

The generated PDF will be in the $CIME/doc/build/pdf directory.

Note: for convenience, you can add this command to the $CIME/doc/Makefile (or make.bat on Windows)

Some resources for learning markup with reStructuredText and using Sphinx.

• [reStructuredText Primer] (http://www.sphinx-doc.org/en/stable/rest.html)
• [ReST Syntax] (https://wiki.typo3.org/ReST_Syntax)
• [Sphinx (including how to get Sphinx)] (http://www.sphinx-doc.org/en/stable/)
• [reStructured syntax] (http://thomas-cokelaer.info/tutorials/sphinx/rest_syntax.html#tables)