Documentation

Documentation is hard

  • Good documentation is hard, and very expensive.

  • Bad documentation is detrimental.

  • Good documentation quickly becomes bad if not kept up-to-date with code changes.

  • Professional companies pay large teams of documentation writers.

Prefer readable code with tests and vignettes

If you don’t have the capacity to maintain great documentation, focus on:

  • Readable code

  • Automated tests

  • Small code samples demonstrating how to use the api

Comment-based Documentation tools

Documentation tools can produce extensive documentation about your code by pulling out comments near the beginning of functions, together with the signature, into a web page.

The most popular is Doxygen

Have a look at an example of some Doxygen output

Sphinx is nice for Python, and works with C++ as well. Here’s some Sphinx-generated output and the corresponding source code Breathe can be used to make Sphinx and Doxygen work together.

Roxygen is good for R.

Example of using Sphinx

Write some docstrings

We’re going to document our “greeter” example using docstrings with Sphinx.

There are various conventions for how to write docstrings, but the native sphinx one doesn’t look nice when used with the built in help system.

In writing Greeter, we used the docstring conventions from NumPy. So we use the numpydoc sphinx extension to support these.

""" 
Generate a greeting string for a person.

Parameters
----------
personal: str
    A given name, such as Will or Jean-Luc

family: str
    A family name, such as Riker or Picard
title: str
    An optional title, such as Captain or Reverend
polite: bool
    True for a formal greeting, False for informal.

Returns
-------
string
    An appropriate greeting
"""

Set up sphinx

Invoke the sphinx-quickstart command to build Sphinx’s configuration file automatically based on questions at the command line:

sphinx-quickstart

Which responds:

Welcome to the Sphinx 3.4.3 quickstart utility.

Please enter values for the following settings (just press Enter to
accept a default value, if one is given in brackets).

Selected root path: .

You have two options for placing the build directory for Sphinx output.
Either, you use a directory "_build" within the root path, or you separate
"source" and "build" directories within the root path.
> Separate source and build directories (y/n) [n]:

and then look at and adapt the generated config, which in our case is a file called conf.py in the doc/source/ directory of the project. This contains the project’s Sphinx configuration, as Python variables, for example:

#Add any Sphinx extension module names here, as strings. They can be
#extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
    "sphinx.ext.autodoc",  # Support automatic documentation
    "sphinx.ext.coverage", # Automatically check if functions are documented
    "sphinx.ext.mathjax",  # Allow support for algebra
    "sphinx.ext.viewcode", # Include the source code in documentation
    "numpydoc",            # Support NumPy style docstrings
]

To proceed with the example, we’ll copy a finished conf.py into our folder, though normally you’ll always use sphinx-quickstart

%%writefile greetings/doc/source/conf.py
# -- Project information -----------------------------------------------------

project = u"Greetings"
copyright = u"2021, James Hetherington"
author = "James Hetherington"

# The full version, including alpha/beta/rc tags
release = "0.1"


# -- General configuration ---------------------------------------------------

# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
    "sphinx.ext.autodoc",  # Support automatic documentation
    "sphinx.ext.coverage",  # Automatically check if functions are documented
    "sphinx.ext.mathjax",  # Allow support for algebra
    "sphinx.ext.viewcode",  # Include the source code in documentation
    "numpydoc",  # Support NumPy style docstrings
]

# Add any paths that contain templates here, relative to this directory.
templates_path = ["_templates"]

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = ["Thumbs.db", ".DS_Store"]

# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
source_suffix = ".rst"

# The master toctree document.
master_doc = "index"

# The name of the Pygments (syntax highlighting) style to use.
pygments_style = "sphinx"


# -- Options for HTML output -------------------------------------------------

# The theme to use for HTML and HTML Help pages.  See the documentation for
# a list of builtin themes.
html_theme = "alabaster"

# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ["_static"]


# -- Options for LaTeX output ------------------------------------------------

latex_elements = {}

# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title, author,
#  documentclass [howto, manual, or own class]).
latex_documents = [
    (
        "index",
        "Greetings.tex",
        u"Greetings Documentation",
        u"James Hetherington",
        "manual",
    ),
]


# -- Options for manual page output ------------------------------------------

# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
    ("index", "greetings", u"Greetings Documentation", [u"James Hetherington"], 1)
]

# -- Options for Texinfo output ----------------------------------------------

# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
#  dir menu entry, description, category)
texinfo_documents = [
    (
        "index",
        "Greetings",
        u"Greetings Documentation",
        u"James Hetherington",
        "Greetings",
        "One line description of project.",
        "Miscellaneous",
    ),
]
Overwriting greetings/doc/source/conf.py

Define the root documentation page

Sphinx uses RestructuredText another wiki markup format similar to Markdown.

You define an index.rst file to contain any preamble text you want. The rest is autogenerated by sphinx-quickstart

%%writefile greetings/doc/source/index.rst
Welcome to Greetings's documentation!
=====================================
Simple "Hello, James" module developed to teach research software engineering.

.. toctree::
   :maxdepth: 2
   :caption: Contents:


Functions
=========

.. autofunction:: greetings.greeter.greet


Indices and tables
==================

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
Overwriting greetings/doc/source/index.rst

Run sphinx

We can run Sphinx using:

%%bash
cd greetings/
sphinx-build doc/source doc/output
Running Sphinx v4.5.0
WARNING: html_static_path entry '_static' does not exist
loading pickled environment... done
[autosummary] generating autosummary for: index.rst
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 1 source files that are out of date
updating environment: 0 added, 1 changed, 0 removed
reading sources... [100%] index

WARNING: autodoc: failed to import function 'greeter.greet' from module 'greetings'; the following exception was raised:
No module named 'greetings'
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index

generating indices... genindex done
writing additional pages... search done
copying static files... done
copying extra files... done
dumping search index in English (code: en)... done
dumping object inventory... done
build succeeded, 2 warnings.

The HTML pages are in doc/output.

Sphinx output

Sphinx’s output is html. We just created a simple single function’s documentation, but Sphinx will create multiple nested pages of documentation automatically for many functions.