How to read and fix 100,000 lines of code in a week

How to read and fix 100,000 lines of code in a week
At the beginning it is always difficult to understand a large and old project. Architecture assessment is one of the activities of an architect. Usually you have to work with large, old projects, and the results must be provided in a week.

How to evaluate a project of 100k or more lines of code per week and at the same time provide results that are really useful for the client.

Most architects and tech leads have experienced similar project reviews. It may look like a semi-formal process or a separate service like our company does, most of you have dealt with it in one way or another.

The original in English for your non-Russian speaking friends is here: Architecture Assessment in a week.

Approach in our company

I will tell you how it works in our company and how I do in such situations, but you can easily change this approach according to the needs of your project and company.

There are two types of architecture assessment.

Internal – we usually do it for projects within the company. Any project can request an architecture assessment for several reasons:

  1. The team thinks their project is perfect and that's suspicious. We have had such cases and often in such projects everything is far from perfect.
  2. The team wants to test their project and their solutions.
  3. The team knows it's bad. They may even list the main problems and causes, but they want a complete list of problems and recommendations for improving the project.

External is a more formal process than an internal evaluation. The client always comes only in one case, when everything is bad - very bad. Usually the client understands that there are global problems, but cannot correctly identify the causes and break them down into components.

Estimating an architecture for an external client is a more complex case. The process should be more formal. Projects are always big and old. They have a lot of problems, bugs and crooked code. A report on the work done should be ready within a few weeks maximum, which should contain the main problems and recommendations for improvement. Therefore, if we deal with the external evaluation of the project, then the internal one will be a couple of trifles. Let's consider the most difficult case.

Enterprise Project Architecture Assessment

A typical project to evaluate is a large, old, enterprise project with a lot of problems. A client comes to us and asks to fix his project. It's like with an iceberg, the client sees only the tip of his problems and does not know what is under the water (in the depth of the code).

Problems that the client may complain about and may be aware of:

  • Performance issues
  • Application Usability Issues
  • Continuous deployment
  • Lack of unit and other tests

Problems that the client is most likely not aware of, but they may be present in the project:

  • Safety problems
  • Design issues
  • Wrong architecture
  • Algorithmic errors
  • Inappropriate Technologies
  • Technical debt
  • Wrong development process

Formal architecture evaluation process

This is a formal process that we follow as a company, but you can customize it according to your company and project.

Request from client

The client asks to evaluate the architecture of the current project. A responsible person from our side collects basic information about the project and selects the necessary experts. Depending on the project, these may be different experts.

Solutions Architect - the main person responsible for evaluation and coordination (and often the only one).
Stack specific experts – .Net, Java, Python, and other technical specialists depending on the project and technologies
Cloud experts - these can be Azure, GCP or AWS cloud architects.
Infrastructure – DevOps, System administrator, etc.
Other experts – such as big data, machine learning, performance engineer, security expert, QA lead.

Gathering information about the project

You should collect as much information about the project as possible. You can use different techniques depending on the situation:

  • Questionnaires and other methods of communication through mail. The most inefficient way.
  • Online meetings.
  • Special tools for information exchange such as: Google doc, Confluence, repositories, etc.
  • "Live" meetings on the spot. The most efficient and most expensive way.

What do you need to get from the client?

Basic information. What is the project about. Its purpose and value. Main goals and plans for the future. Business goals and strategies. Main problems and desired result.

Project Information. Technology stack, frameworks, programming languages. On-premise or cloud deployment. If the project is in the cloud, what services are used. What architectural and design patterns were used.

non-functional requirements. All requirements related to performance, availability, usability of the system. Security requirements, etc.

Basic usecases and data flows.

Access to source code. The most important part! You should definitely get access to the repositories and documentation on how to build the project.

Access to infrastructure. It would be nice to get access to the stage or production infrastructure in order to work with a "live" system. It is a great success if the client has the tools to monitor the infrastructure and performance. We will talk about these tools in the next section.

Documentation. If the client has documentation, this is a good start. It may be outdated, but it's still a good start. Never trust documentation - test it with the client, on real infrastructure and in source code.

Architecture Assessment Process

How is it possible to process such a large amount of information in such a short time? First of all, parallelize the work.

DevOps should look into the infrastructure. Those lead into the code. Performance engineer to see performance metrics. A database specialist needs to dig deeper into data structures.

But this is the ideal case when you have a lot of resources. Typically, project evaluation is carried out by one to three people. You can even do the assessment yourself, which is often the case if you have the right knowledge and experience in all areas of the project. In this case, you need to automate all processes as much as possible.

Unfortunately, you will have to read the documentation manually. With proper experience, you can quickly understand the quality of the documentation. What is true and what is clearly not true. Sometimes you will see architectures in documentation that will never work in real life. This is a trigger for you to think about how it is done in reality in the project.

Useful Tools to Automate Project Estimation

Code evaluation is a simple exercise. You can use static code analyzers to show design, performance and security issues. Here are a few of them:

Frame 101 is a great tool for an architect. It will show you the big picture, dependencies between modules, and potential areas for refactoring. Like all good tools, it costs good money, but at the same time, you can take advantage of a 30-day trial version.

Sonar Qube - a good old tool. Tool for static code analysis. Allows you to identify bad code, bugs, security problems for more than 20 programming languages.

All cloud providers have infrastructure monitoring tools. This will allow you to correctly evaluate the effectiveness of the infrastructure in terms of cost and performance. For AWS it is trusted advisor. For Azure it's easy Azure Advisor.

Additional performance monitoring and logging will help you find performance issues at all levels. Starting from the database with inefficient queries, backend and ending with the frontend. Even if the client has not installed these tools before, you can integrate them into an existing system fairly quickly to identify performance issues.

As always, good tools are worth a lot. I can recommend a couple of paid tools. Of course, you can use open-source, but you will need more time for this. And this should be done in advance, and not in the process of evaluating the architecture.

New Relic – application performance evaluation tool
datadog – cloud service for system monitoring

There are many tools for security testing. This time I will recommend you a free system scan tool.

OWASP ZAP is a tool for scanning web applications for compliance with security standards.

Putting everything together.

Preparing a report

Start your report with data collected from the client. Describe the project goals, constraints, non-functional requirements. After that, all incoming data should be mentioned: source code, documentation, infrastructure.

Next step. List any issues you found manually or using automated tools. Put large auto-generated reports at the end of the applications section. There should be short and concise evidence of the problems found.
Prioritize the problems found on the error, warning, info scale. You can choose your own scale, but this one is generally accepted.

As a true architect, it is your responsibility to provide recommendations on how to fix the problems found. Describe the improvements and business value that the customer will experience. How to show business value from architecture refactoring we discussed earlier.

Prepare a roadmap with small iterations. Each iteration should contain time to complete, description, amount of resources needed for improvement, technical value, and business value.

We complete the architecture assessment and provide the client with a report

Never just mail a report. It may either not be read at all or read and not understood without proper explanation. In short, live communication helps eliminate misunderstandings between people. You should schedule a meeting with the client and talk about the problems found, focusing on the most significant ones. It is worth drawing the attention of the client to problems that he might not even be aware of. Such as security issues and explain how they might affect the business. Show your roadmap with improvements and discuss different options that are more suitable for the client. It can be time, resources, amount of work.

As a debriefing of your rally, send your report to the client.

In conclusion

Architecture assessment is a complex process. To conduct an assessment properly, you need to have enough experience and knowledge.

It's real - to provide the client with useful results for him and his business in just a week. Even if you are doing it alone.

Based on my experience, many improvements were pumped in the middle, and sometimes never even started. Those who chose the middle ground for themselves and made only a fraction of the improvements that were most beneficial to the business with minimal effort, significantly improved the quality of their product. Those who did nothing in a couple of years could completely close the project.

Your goal is to show the client the maximum improvement for the minimum price.

Other articles from the section architecture can be read at your leisure.

I wish you clean code and good architectural solutions.

Our facebook group Software Architecture and Development.

Source: habr.com

Add a comment