Updating documentation code examples can be a tedious chore, but it doesn't have to be. By separating code from text and automating testing, you can save time and reduce errors.

In this article, we are going to see how to make the maintenance of the pieces of code that we can find in almost every documentation more enjoyable and less error prone.

Note: The article is based on the Sphinx documentation system, but the same principles apply to any other documentation project with some custom development.

1. Split responsibilities

Imagine Amy, a technical writer at TechDocs Studio, tasked with documenting an SDK for ordering pizza deliveries. The documentation page she is working on, written in restructuredText or MarkDown, looks similar to the following example:

Example
=======

To order a pizza using the Python SDK, call the
``order`` method by passing the
pizzas you want, your customer ID, and the delivery
service as the parameters.

The following examples show you how to order a 
Hawaiian pizza to take away.

.. code-block:: python

    api = pizza.Api()

    api.order(
      pizzas=[
      {'code':pizza.pizza_codes.HAWAIIAN, 
        'quantity':1}
        ], 
        customer_id='dgarcia360', 
        serviceType=pizza.service_types.TAKE_AWAY)

So, I guess that you already identified the mistake with the previous code snippet: Pineapple on pizza? Well, you are mistaken because adding pineapple to a pizza is just a minor issue! The real problem with the previous doc: the documentation page is mixing code and text in the same file.

Let's review in-depth the mixed responsibility issue with an example.

This week, the Pizza SDK development team launches a new release. Which are the steps Amy should follow to update the documentation? She should:

1. Detect every code example affected.
2. Copy and paste every code example into a new file.
3. Execute the code.
4. Submit the changes.

Copying and pasting every code example (2) and testing them manually (3) are repetitive chores that can be automated. Additionally, following the approach mentioned above, Amy cannot ensure that the code tested in (3) is the same code displayed in the docs.

To start automating the process, Amy decides to move the piece of code into a separate file. For instance, she creates a new folder next to the documentation project to keep all the code examples organized.

The new folder will contain not only the code examples but also the necessary dependencies to run them. If the SDK were available in different languages, she could create an examples folder with subfolders for each language. Here is an example:

documentation-project/
├── index.rst
├── order.rst
├── examples/
│   ├── javascript
│   └── python
│       ├── order.py
│       ├── tests/
│       └── requirements.txt

What is Amy achieving by separating the code from the text? Now she can:

• execute the code examples and add tests.
• use the code linter, to ensure the code is well-formatted and every bracket closes.
• avoid copying and pasting code to verify that the code examples are working.

2. Add unit tests

Introducing unit tests ensures code examples remain functional and accurate. By creating tests to validate each example's behavior, both Amy and you can prevent errors and save time in the long run.

In the context of documentation code examples, unit tests can check the correctness of each example's output, ensuring that they accurately reflect the intended functionality.

Here's how to implement unit tests for the Pizza SDK documentation:

1. Identify testable scenarios: Begin by identifying key scenarios for each code example. For instance, consider creating tests to verify the process of placing an order for a Hawaiian pizza with specific parameters, ensuring the expected output aligns with the intended behavior.
2. Write test cases: Utilize a testing framework such as pytest or unittest to craft test cases. Each test case should address a specific scenario and assert the expected outcome. For instance, you might create a test case to confirm that the order function generates a confirmation message upon successfully placing an order.
3. Automate test execution: Integrate the unit tests into the documentation project's build process. This ensures that whenever modifications are made to the code examples or the SDK itself, the unit tests automatically validate the examples' functionality. By automating this process, you eliminate the need for manual testing of each example, thereby saving time and effort.
4. Continuous Integration (CI): Establish continuous integration (CI) pipelines to execute the unit tests automatically whenever new code is pushed to the repository. This ensures that any changes to the documentation or the SDK undergo immediate testing, enabling you to detect and rectify issues early in the development cycle.
5. Monitor Test Coverage: Keep a close eye on the test coverage to ensure that all code paths in the examples receive adequate testing. This practice helps identify any gaps in the test suite and guarantees comprehensive coverage of the documentation code examples.

By incorporating unit tests into the documentation code examples, you can confidently validate their functionality and accuracy with each SDK update. This proactive approach not only mitigates the risk of errors in the documentation but also streamlines the maintenance process, allowing you to focus on delivering top-notch documentation content.

3. Don't repeat yourself

Now that we have separated the text from the code and have unit tests in place, we aim to render them together seamlessly.

Regardless of the documentation system you are using, the idea is to import the code from the files using a directive that allows you to do so.

With the Sphinx documentation system, we could use the directive literalinclude as follows:

.. literalinclude:: examples/python/order.py
    :language: python

This directive renders the content of the file order.py inside the documentation page. With this addition, all our code will also be reusable, as we could reference the same piece of code in multiple documentation pages. Moreover, we are ensuring that similar examples will always behave in the same way.

4. Highlight subsets of code

Since we now have testable and reusable code examples, we can go a step further. In some situations, we only need to highlight a subset of the code instead of the complete file. We also want to skip the non-relevant code lines such as the license header, or the list of imports.

To achieve this, we can separate the code using opening and closing custom tags within comments. In our case, Amy decided to use the tags # block <number> to open and identify a code block and # endblock <number> to close it.

# block 01
api = pizza.Api()
# endblock 01

# block 02
api.order(
    pizzas=[
        {'code':pizza.pizza_codes.HAWAIIAN, 
        'quantity':1}
    ], 
    customer_id='dgarcia360', 
    serviceType=pizza.service_types.TAKE_AWAY)
# endblock 02

... and then, detect those tags with the help of the directive.

.. literalinclude:: /pizza/order.py
    :language: python
    :start-after: # block 02
    :end-before: # endblock 02

Avoid the trap of importing subsets of code by declaring the number of lines directly! If you define, for example, "render lines 1-10", you would have to check that every code line is still correct after every update.

Putting it all together

The development team launches a new version of the Pizza SDK. How can we check now that the code examples are still working?

1. Update the latest SDK dependency into the code example folder.
2. Run the tests and check which failed.
3. Fix the code examples—get the green light! ✅
4. Push the code changes.

As you have seen, by following these practices, we have reduced the time spent maintaining our code examples. Interestingly, the time it took us to architect a Docs as Code solution is much less in comparison.

But, it’s not only about working less: the documentation will become more usable for your end-users since there are fewer chances to have code that does not work.

If you found this article helpful, subscribe to our newsletter for more insightful content. Happy documenting!