How do you keep documentation code examples up to date? Every time there is a new product update, you need to fix what has changed instead of spending time covering new features. However, detecting which documentation needs to be edited after an update is not always as straightforward as testing the output of a deterministic function. Someone must evaluate which concepts or guides have become outdated.

Still, there are some documentation areas where we can conveniently apply testing, automation, and other common development practicesβ€”like code examples.

In this article, we are going to see how to make the maintenance of the pieces of code that we can find in almost every documentation more enjoyable for developers. We are looking at ways to reduce the number of complaints received with the subject "The code examples are not working!" while significantly reducing the time spent fixing outdated snippets.

Note: The article is based on the Sphinx documentation system, but the same principles apply to any other documentation project with some custom development.

1. Splitting responsibilities

Amy, who works as a technical writer at BestTechnicalWritingConsulting, is documenting a SDK to order delivery pizza. The documentation page she is working on, written in restructuredText or MarkDown, looks similar to the following example:

Example ======= To order a pizza using the Python SDK, call the ``order`` method by passing the pizzas you want, your customer ID, and the delivery service as the parameters. The following examples show you how to order a Hawaiian pizza to take away. .. code-block:: python api = pizza.Api() api.order( pizzas=[ {'code':pizza.pizza_codes.HAWAIIAN, 'quantity':1} ], customer_id='dgarcia360', serviceType=pizza.service_types.TAKE_AWAY)

So, I guess that you already identified the mistake with the previous code snippet: Pineapple on pizza? Yikes! Well, you are mistaken β€˜cause adding pineapple to a pizza πŸπŸ• is just a minor issue! The real problem with the previous doc: the documentation page is mixing code and text in the same file.

Let's review in-depth the mixed responsibility issue with an example. This week, the Pizza SDK development team launches a new release. Which are the steps Amy should follow to update the documentation? She should:

  1. Detect every code example affected.
  2. Copy and paste every code example into a new file.
  3. Execute the code.
  4. Submit the changes.

Copying and pasting every code example (2) and testing them manually (3) are repetitive chores that can be automated. Additionally, following the approach mentioned above, Amy cannot ensure that the code tested in (3) is the same code displayed in the docs.

To start automating the process, Amy decides to move the piece of code into a separate file. For instance, she creates a new folder next to the documentation project to keep all the code examples organized.

The new folder will contain not only the code examples but also the necessary dependencies to run them. If the SDK were available in different languages, she could create an examples folder with subfolders for each language. Here is an example:

documentation-project/ β”œβ”€β”€ index.rst β”œβ”€β”€ order.rst β”œβ”€β”€ examples/ β”‚ β”œβ”€β”€ javascript β”‚ └── python β”‚ β”œβ”€β”€ order.py β”‚ β”œβ”€β”€ tests/ β”‚ └── requirements.txt

What is Amy achieving by separating the code from the text? Now she can:

  • execute the code examples and add tests.
  • use the code linter, to ensure the code is well-formatted and every bracket closes.
  • avoid copying and pasting code to verify that the code examples are working.

2. Don't repeat yourself

Now that we have the text and code separated, BestTechnicalWritingConsulting wants to render both text and code together. Regardless of the documentation system you use, the idea is to import the code from the files using a directive that allows you to do so.

With the Sphinx documentation system, we could use the directive literalinclude as follows:

.. literalinclude:: examples/python/order.py :language: python

This directive renders the content of the file order.py inside our documentation page. With this addition, all our code will also be reusable, as we could reference the same piece of code in multiple documentation pages. Moreover, we are ensuring that similar examples will always behave in the same way.

3. Highlighting subsets of code

Since we now have testable and reusable code examples, we can go a step further. In some situations, we only need to highlight a subset of the code instead of the complete file. We also want to skip the non-relevant code lines such as the license header, or the list of imports.

To achieve this, we can separate the code using opening and closing custom tags within comments. In our case, Amy decided to use the tags # block <number> to open and identify a code block and # endblock <number> to close it.

# block 01 api = pizza.Api() # endblock 01 # block 02 api.order( pizzas=[ {'code':pizza.pizza_codes.HAWAIIAN, 'quantity':1} ], customer_id='dgarcia360', serviceType=pizza.service_types.TAKE_AWAY) # endblock 02

... and then, detect those tags with the help of the directive.

.. literalinclude:: /pizza/order.py :language: python :start-after: # block 02 :end-before: # endblock 02

Avoid the trap of importing subsets of code by declaring the number of lines directly! If you define, for example, "render lines 1-10", you would have to check that every code line is still correct after every update.

Putting it all together

The development team launches a new version of the Pizza SDK. How can we check now that the code examples are still working?

  1. Update the latest SDK dependency into the code example folder.
  2. Run the tests and check which failed.
  3. Fix the code examplesβ€”get the green light! βœ…
  4. Push the code changes.

As you have seen, by following these practices, we have reduced the time spent maintaining our code examples. Interestingly, the time it took us to architect a Docs as Code solution is much less in comparison.

But, it’s not only about working less: the documentation will become more usable for your end-users since there are fewer chances to have code that does not work.