September 1999. The Mars Climate Orbiter, a NASA spacecraft on a mission to study the climate and weather of Mars, is ready for its orbital insertion maneuver. From the ground, the navigation team instructs the spacecraft to enter a trajectory that would bring it to an altitude of 226 km (140 miles). But, after an expected loss of the signal, they can't re-establish contact. They will never hear from the spacecraft again.

Fast-forward a few months: after investigations, a board of experts did find the cause of the loss: they determined that a piece of software produced by Lockheed Martin was sending data in pound-force seconds. In contrast, the piece of software produced by NASA expected and interpreted the data as if it were in Newton seconds. Consequently, the spacecraft was headed into a trajectory that would bring it to an altitude of only 57 km (35 miles) instead of the expected 226 km. This altitude was too low for its survival.

Cost of the failure: 327.6 million dollars.

It would be unfair to blame only one single factor for this failure: the Mars Climate Orbiter incident was caused by the unfortunate interaction of several sub-optimal choices and practices. Quality documentation, however, is one of the ways that can prevent such incidents.

Writing excellent documentation is difficult. That's why we have collected here some of the questions that we think you should ask yourself when documenting your APIs. We will review the most important issues to keep in mind, and we will do that not through subjective "best practices" but with scientifically grounded suggestions.

Let's get started.

Is your documentation up to date?

In a joint research paper [^1] from Universities in Switzerland, Colombia, Colorado, and Virginia, the authors conducted a survey asking developers and other technical engineers with variegated levels of programming experience to indicate the importance of several of the issues that can be found in software documentation. The results? Among an extensive list of 25 issues related to the information content, here are the top 10 rated as most important by the participants:

1. Missing documentation for new feature/component (69%)

2. Installation, deployment, & release (68%)

3. Faulty tutorial (65%)

4. Missing user documentation (65%)

5. Outdated/Obsolete references (64%)

6. Inappropriate installation instructions (63%)

7. Developer guidelines (60%)

8. Erroneous code examples (59%)

9. Code-documentation inconsistency (59%)

10. Outdated installation instructions (54%)

Notice a pattern? Most of the reported issues have to do with outdated documentation. Similar results have been obtained by other researchers in the past. For example, according to Uddin and Robillard from McGill University [^2], the top problems in API documentation are (i) incompleteness, (ii) ambiguity, (iii)

unexplained examples, (iv) obsoleteness, (v) inconsistency and (vi) incorrectness. According to the research by Chen and Huang [^3], some of the most common problems in software documentation is that it is untrustworthy, inadequate, incomplete or does not even exist, lacks traceability, does not include its changes, and lacks integrity and consistency. You get the idea.

We cannot stress enough the importance of up-to-date documentation: outdated information makes the developers lose time and can cause frustration. It can be worse than having no documentation at all.

Keeping your documentation up to date is not something that you should think about. Establish a process, and let the development guide the update of the documentation almost spontaneously, just like it happens with testing. Practically speaking, if the developers are writing the documentation themselves, you may for example establish that for a pull request to be approved it should also include commits updating the documentation. If the documentation is instead curated by dedicated professionals, the developers could track each change by tools such as Atlassian Changeset so that whoever writes the documentation can be notified whenever something relevant changes. This is the way we do it at SignalWire, but any means of reliable coordination should work fine.

Less is more, more is less

According to a survey by Uddin and Robillard [^2], the top problems in the presentation of API documentation are (i) bloat, (ii) fragmentation, (iii) excess structural information and (iv) tangled information. To understand what this means, it is worth expanding on the definitions:

Bloat: the description of an API element or topic is verbose or excessively extensive. 

Fragmentation: the information related to an element or topic was fragmented or scattered over too many pages or sections.

Excess structural information: the description of an element contained redundant information about the elementʼs syntax or structure, which could be easily obtained through modern IDEs. 

Tangled information: the description of an API element or topic was tangled with information the respondent didnʼt need.

Unsurprisingly, the research from Aghajani et al. [^1] also shows similar concerns, with a majority of survey participants indicating superfluous or duplicate content as high-impact issues.

When using automated tools for generating the documentation, it is easy to fall for one or more of the above points. It is worth considering a practical example. In the following figure, we show the documentation for a TypeScript method. It was automatically generated by TypeDoc by analyzing the source code of the method:

Can you spot the issues in the above method documentation? First, information is fragmented (I'd like to see the details of the available options right here, not in another page). Then, there is tangled and excess structural information (I don't need to know inheritance information here). As for bloat, in this particular example we have the opposite problem.

By the way, the name of the method is inspired by our JavaScript SDK. Check it out if you are looking for a full-fledged custom video-conferencing library. The documentation is great.

Focus on what matters

It may be tempting to set up an automated documentation tool like TypeDoc or Javadoc, document all the methods, and call it a day. This, though, is just a small part of the process of documentation.

According to "A study of the effectiveness of usage examples in REST API documentation" by Sohan et al. [^4], developers are more productive and satisfied when examples are available. If examples are lacking, developers spend more time on trial-and-error approaches and have lower success rates. Which means, they are going to the competition.

Multiple studies have confirmed the importance of examples as a key learning resource. If you want to build great documentation, then you should dedicate an essential slice of your time to well-thought examples.

But it doesn't stop there. In "What makes APIs hard to learn? Answers from developers" [^5] and "A field study of API learning obstacles'' [^6], Professor Robillard and others highlight additional aspects and resources that, when lacking, severely affect the learning experience. These are API structure, Background, Technical environment, Process, documentation of intent, cookbooks for mapping usage scenarios to API elements, the penetrability of the API, and format and presentation of the documentation.

In short, in addition to your API reference, make sure you are also providing at least the following resources:

• Snippet examples, showing concise examples of accessing the API functionalities
• Tutorial examples, in which longer code segments are embedded in prose and intended to teach specific aspects of the API
• Application examples, showing the API in a real-world use case
• API structure, describing the general organization of the API, and background information, in case you are dealing with specific domains.

The users of your API will thank you.

To conclude

We have provided you a few suggestions on some of the most critical aspects of API documentation. We have focused on three main concerns: is your documentation up-to-date? Are you showing the right amount of content? Are you providing the most useful resources?

It is not a secret that developers don't like writing documentation. However, try to keep in mind that documentation is one of the most important aspects of the success of your product. When surveying developers, the most indicated cause of inadequate documentation is the lack of time to write it [^1]. Then, perhaps a change of mindset is needed. Documentation is not a nice-to-have addition: documentation is part of the product itself!

References:

[^1]: Aghajani, Emad, et al. "Software documentation: the practitioners' perspective." 2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE). IEEE, 2020.

[^2]: Uddin, Gias, and Martin P. Robillard. "How API documentation fails." IEEE software 32.4 (2015): 68-75.

[^3]: Chen, Jie-Cherng, and Sun-Jen Huang. "An empirical analysis of the impact of software development problem factors on software maintainability." Journal of Systems and Software 82.6 (2009): 981-992.

[^4]: Sohan, S. M., et al. "A study of the effectiveness of usage examples in REST API documentation." 2017 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE, 2017.

[^5]: Robillard, Martin P. "What makes APIs hard to learn? Answers from developers." IEEE software 26.6 (2009): 27-34.

[^6]: Robillard, Martin P., and Robert DeLine. "A field study of API learning obstacles." Empirical Software Engineering 16.6 (2011): 703-732.