Use GIT for documentation

Sometimes not only the documentation itself but also the process of working on it can be critical. For instance, in the case of projects, a significant portion of the work is related to preparing documentation, and an incorrect process can lead to errors and even data loss, which in turn can result in wasted time and loss of profit. However, even if this topic is not central to your work and is on the periphery, a proper process can still improve the quality of the document and save you time.

The approach outlined here, with an example of a specific implementation, has a low entry threshold. Technically, you could start working differently as soon as tomorrow.

Task Definition

You need to create some document or set of documents. This could be project documentation or documentation of your network, or something simpler, like describing processes within your company or department. Essentially, it pertains to any document or set of documents containing text, images, tables... To complicate matters,

1. this work implies collaborative efforts, involving a group or multiple groups of employees.
2. In the end, you want to have a document in a specific format, with corporate styling attributes, created according to a certain template. For the sake of clarity, let’s assume this is MS Word (.docx).

Ten years ago, the approach would have been clear-cut: we would create an MS Word document or documents and organize the work of making changes in some way.

And this approach is still valid. Large integrators also use it when creating project documentation. However, it is intuitively clear that if you are working intensively on a document with many edits and discussions over an extended period, this approach is not very convenient.

Example

I felt this issue quite sharply while working at a large integrator. The process of changing project documentation was as follows:

1. the engineer downloads the latest version of the MS Word (.docx) document
2. changes the title
3. makes edits in track mode
4. sends the document with changes to the architect
5. also sends a list of all corrections with comments
6. the architect analyzes the changes
7. if everything is okay, he copies the changes into the file with the latest version, changes the version, and uploads it to the shared resource.
8. If there are any comments, a discussion will be initiated (email or meetings)
9. Consensus is reached
10. Then points 3 - 9

While the work wasn't intensive, it somehow worked. However, at a certain point, this process became a bottleneck for the entire project and led to problems. The issue is that everything deteriorates as soon as changes are made frequently and simultaneously by multiple teams.

Thus, when we moved to the preliminary testing phase, different issues began to surface, and although they were minor, we often needed to update the documentation — four different teams, daily, practically simultaneously, with discussions. All these changes went through one engineer — the architect. The project design file was enormous, and as a result, the architect was inundated with routine work involving a lot of copying, editing, making many mistakes that had to be double-checked and resent, and overall it was close to chaos.

In this case, this approach, the approach of working on MS Word documents, worked with great strain and created problems.

Git, Markdown

Faced with the issue described in the example above, I began to investigate this matter.
I saw that the use of Git is becoming increasingly popular Markdown alongside Git in document creation.

Git is a development tool. But why not use it for the documentation process? In this case, the issue of multi-user collaboration is resolved. However, to fully leverage Git's capabilities, we need a text document format, and we need to find another tool, not MS Word, and for these purposes, Markdown fits perfectly.

Markdown is a simple markup language. It is designed to create well-formatted texts in regular TXT files. If we create our documents in Markdown, then the combination of Markdown and Git looks natural.

Everything would be fine, and this would be the end if it weren't for our second condition: "we need a document in a specific format, with corporate style attributes, created according to a specific template" (and we agreed at the beginning that, for clarity, this would be MS Word). In other words, if we decided to use Markdown, we would need to somehow convert this file into the required .docx format.

There are conversion programs between different formats, for example, Pandoc.
You can convert a Markdown file to .docx format with this program.
However, it is important to understand that, firstly, not everything in Markdown will be converted to MS Word and, secondly, MS Word is an entire country compared to the neat, but still small town of Markdown. There is a vast amount of content available in Word that simply does not exist in any form in Markdown. You can't just take your Markdown format and convert it to the desired MS Word format with specific Pandoc keys. So usually, after conversion, you have to manually "refine" the resulting .docx document, which can again be time-consuming and lead to errors.

If we could write a script that would automatically "finish" what Pandoc couldn't handle — that would be the ideal solution.

Due to the differences in functionality between MS Word and Markdown in general, I think it is impossible to solve this problem. However, can we approach it in relation to specific situations, specific requirements? My experience has shown that yes, it is possible, and most likely feasible for many, or perhaps even most situations.

Solving a specific task

In my case, after converting the file using Pandoc, I had to manually make additional modifications, namely

• adding fields in Word with automatic numbering of headings (caption) for tables and images
• changing styles for tables

I couldn't find how to do this with standard (Pandoc) or known tools. Therefore, I implemented a Python script using pywin32 package. As a result, I achieved full automation. Now I can convert my Markdown file into the required form of an MS Word document with one command.

See details here.

Note

In this example, I am, of course, transforming a certain abstract Markdown file, but the exact same approach was applied to a ‘production’ document, and as a result, I got almost exactly the same MS Word document that we previously obtained through manual formatting.

Overall, with pywin32, we gain nearly complete control over the MS Word document, allowing us to change it and format it according to our corporate standards. Of course, these same goals could be achieved using other tools, such as VBA macros, but I found it easier to use Python.

The brief formula for this approach is:

Markdown + Git -- (something) --> MS Word

It doesn't matter what 'something' is. In my case, it was Pandoc and Python with pywin32. You may have different preferences, but what's important is that it's possible. And that's the main message of this article.

In summary, the idea is that with this approach, you only work with the Markdown file and use Git for organizing collaborative work and version control, and only when necessary (for example, to provide documentation to the client) do you automatically create a file in the desired format (for example, MS Word).

Process

I think for many, the formula provided above is sufficient to understand how the documentation process can now be organized. However, I usually focus on network engineers, so I will outline how the work process might now look and how it differs from the approach of editing MS Word files.

To be specific, let's choose GitHub as the platform for working with Git. You should then create a repository and place the Markdown file or files you plan to work with in the master branch.

We will look at a simple process based on 'github flow'. Descriptions can be found both online and on Habr.

Let's assume four people are working on the documentation and you are one of them. Then four additional branches are created, for example, named after those individuals. Each works locally in their branch and makes changes using all the necessary git commands.

Once you have completed a certain piece of work, you create a pull request, thereby initiating the discussion of your changes. During the discussion, it may become apparent that you need to add or modify something else. In this case, you make the necessary changes and create an additional pull request. Eventually, your changes are accepted and merged with the master branch (or rejected).

Of course, this is quite a general description. I suggest you consult with your developers or knowledgeable individuals to create a detailed process. However, I want to point out that the barrier to entry for Git is quite low. This does not mean that the protocol is simple, but you can start with the basics. If you know nothing at all, I think that after spending a few hours or maybe days on learning and setting it up, you can start using it.

What is the benefit of this approach compared to, for example, the process described in the example above?

Actually, the processes are quite similar; you just replaced

copying a file -> creating a branch
copying text into the final file -> merging
copying the latest changes to yourself -> git pull/fetch
discussing in correspondence -> pull requests
track mode -> git diff
the last approved version -> master branch
backup (copying to a remote server) -> git push
…

Thus, you have automated everything that you had to do manually.

At a higher level, this allows you to

• create a clear, simple, and controlled process for documentation changes
• since the final document (in our example, MS Word) is created automatically, it reduces the chance of formatting errors.

Note

Given the above, it seems obvious that even if you are working on documentation alone, using Git can significantly ease your work.

All of this improves the quality of documentation and reduces the time required to create it. And a little bonus — you will learn Git, which will help you in automating your network 🙂

How to transition to the new process?

At the beginning of this article, I mentioned that you could start working in a new way as early as tomorrow. How to shift your work to this new pathway?

Here is a sequence of steps you will likely need to follow:

• if your document is very large, break it into parts.
• convert each part to Markdown (using Pandoc, for example)
• install one of the Markdown editors (I use Typora)
• you will likely need to adjust the formatting of the generated Markdown documents
• start applying the process described in the previous chapter
• at the same time, begin modifying the conversion script to suit your task (or create something of your own)

You don't have to wait until you've created and debugged the Markdown -> desired document output mechanism perfectly. The thing is, even if you can't fully automate the conversion of your Markdown files quickly, you can still do it in some form with Pandoc and then refine it to the final output manually. Usually, you don't need to do this often, only at the end of certain stages, and this manual work, while inconvenient, is still, in my opinion, quite acceptable during the debugging stage and shouldn't significantly 'slow down' the process.

Everything else (Markdown, Git, Pandoc, Typora) is already ready and requires little effort or time to start working with them.

Source: habr.com