How to read and fix 100,000 lines of code in a week

How to read and fix 100,000 lines of code in a week
Getting started with a large, legacy project is always challenging. Architecture assessment is one of the architect's responsibilities. Typically, one has to work with substantial, old projects, and results must be delivered within a week.

How to evaluate a project with 100k or more lines of code in a week while providing truly valuable results for the client.

Most architects and tech leads have faced similar project evaluations. This may appear as a semi-formal process or as a separate service as done in our company; nonetheless, most of you have dealt with it.

The original in English for your non-Russian-speaking friends can be found here: Architecture Assessment in a week.

Our approach in the company

I'll explain how this works in our company and how I handle such situations, but you can easily adjust this approach according to your project's and company's needs.

There are two types of architecture assessments.

Internal – we typically conduct this for projects within the company. Any project can request an architecture assessment for several reasons:

  1. The team believes their project is perfect and that's questionable. We've seen such cases and often in such projects, everything is far from perfect.
  2. The team wants to verify their project and their solutions.
  3. The team knows that things are bad. They may even list the main issues and causes, but they want a comprehensive list of problems and recommendations for improving the project.

External – this is a more formal process than an internal assessment. A client comes in only when things are really bad – very bad. Usually, the client understands that there are global issues but cannot accurately identify the causes and break them down.

Architecture assessment for an external client is a more complex case. The process needs to be more formal. Projects are always large and old. They contain numerous problems, bugs, and poorly written code. The report on the work done must be ready within a few weeks at most, where the main issues and recommendations for improvement should be outlined. Therefore, if we manage to handle external project evaluations, an internal one will be a walk in the park. Let's consider the most challenging case.

Architecture assessment of an enterprise project

A typical project for assessment is a large, old enterprise project with many issues. The client comes to us asking to fix their project. It's like an iceberg; the client only sees the tip of their problems and is unaware of what lies beneath the surface of the code.

Problems that the client may complain about and may be aware of:

  • Performance issues
  • Usability problems in the application
  • Long deployment times
  • Lack of unit and other tests

Problems that the client is likely unaware of but may exist in the project:

  • Security issues
  • Design Issues
  • Incorrect architecture
  • Algorithmic errors
  • Inappropriate technologies
  • Technical debt
  • Incorrect development process

Formal architectural assessment process

This is the formal process we follow in the company, but you can adjust it to fit your company and project.

Client request

The client asks to assess the architecture of the current project. The responsible person on our side gathers basic information about the project and selects the necessary experts. Depending on the project, these may be different experts.

Solution Architect – the main person responsible for the assessment and coordination (and often the only one).
Stack specific experts – .Net, Java, Python, and other technical specialists depending on the project and technologies
Cloud experts – these may include Azure, GCP, or AWS cloud architects.
Infrastructure – DevOps, System administrators, etc.
Other experts – such as big data, machine learning, performance engineers, security experts, QA leads.

Gathering project information

You should gather as much information about the project as possible. You can use various techniques depending on the situation:

  • Questionnaires and other means of communication via email. The least effective method.
  • Online meetings.
  • Special tools for information exchange such as Google Docs, Confluence, repositories, etc.
  • In-person 'live' meetings. The most effective and most expensive method.

What should we obtain from the client?

Basic information. What the project is about. Its goals and value. Main objectives and future plans. Business goals and strategies. Key issues and desired outcomes.

Project InformationTechnology stack, frameworks, programming languages. On-premise or cloud deployment. If the project is in the cloud, which services are being used? What architectural and design patterns were applied?

Non-functional requirementsAll requirements related to the system's performance, availability, usability, security requirements, etc.

Basic use cases and data flows.

Access to the source codeThe most important part! You definitely need to gain access to the repositories and documentation on how to build the project.

Access to the infrastructureIt would be great to get access to stage or production infrastructure to work with a 'live' system. It's a big advantage if the client has monitoring tools for infrastructure and performance. We'll discuss these tools in the next section.

DocumentationIf the client has documentation, that's a good start. It might be outdated, but it's still a good starting point. Never trust documentation blindly—validate it with the client, on the actual infrastructure and in the source code.

Architecture assessment process

How can you process such a large amount of information in such a short time? First, parallelize the work.

The DevOps specialist should look into the infrastructure. The tech lead into the code. The performance engineer should check the performance metrics. The database specialist should delve deeper into the data structures.

But that’s the ideal case when you have plenty of resources. Typically, project assessment is done by one to three people. You might even conduct the assessment yourself, which often happens if you have the right knowledge and experience in all areas of the project. In that case, you need to automate processes as much as possible.

Unfortunately, you will have to manually read the documentation. With the right experience, you will be able to quickly assess the quality of the documentation. What is true and what clearly doesn't match reality. Sometimes you may encounter an architecture in the documentation that will never work in real life. This is a trigger for you to consider how it was actually implemented in the project.

Useful tools for automating project assessment

Code assessment is a simple exercise. You can use static code analyzers that will show design, performance, and security issues. Here are a few of them:

Structure 101 is an excellent tool for architects. It will show you the big picture, the dependencies between modules, and potential areas for refactoring. Like all good tools, it comes at a price, but you can take advantage of its 30-day trial version.

SonarQube is the old reliable tool. A static code analysis tool that can identify bad code, bugs, and security issues across more than 20 programming languages.

All cloud providers have tools for monitoring infrastructure. This will allow you to properly assess the efficiency of your infrastructure in terms of cost and performance. For AWS, this is trusted advisor. For Azure, it’s simply Azure Advisor.

Additional performance monitoring and logging will help identify performance issues at all levels, from databases with inefficient queries, through the backend, to the frontend. Even if the client has not previously installed these tools, you can quickly integrate them into the existing system to pinpoint performance problems.

As always, good tools come at a cost. I can recommend a couple of paid tools. Of course, you can use open-source options, but it will require more time. This should be done in advance, not while assessing the architecture.

New Relic is an application performance assessment tool
Datadog is a cloud system monitoring service

There are many tools for security testing. This time, I will recommend a free tool for system scanning.

OWASP ZAP is a tool for scanning web applications for compliance with security standards.

Let’s bring everything together.

Preparing the report

Start your report with the data collected from the client. Describe the project goals, constraints, and non-functional requirements. After this, make sure to mention all incoming data including the source code, documentation, and infrastructure.

Next step. Specify all the issues you found manually or using automated tools. Large auto-generated reports should be placed at the end in the appendix section. Here should be short and concise evidence of the identified problems.
Prioritize the identified issues on a scale of error, warning, info. You can choose your own scale, but this is a generally accepted one.

As a true architect, you are obliged to provide recommendations for resolving the identified issues. Describe the improvements and the business value the client will gain. How to demonstrate the business value from architecture refactoring we discussed earlier.

Prepare a roadmap with small iterations. Each iteration should include the time for execution, a description, the number of resources needed for improvement, technical value, and business value.

We conclude the architecture assessment and provide the client with a report

Never just send the report via email. It may either not be read at all or read without understanding without proper explanation. In short, live communication helps eliminate misunderstandings between people. You should schedule a meeting with the client and discuss the identified issues, emphasizing the most significant ones. It's worth bringing the client's attention to problems they might not even be aware of. Such as security issues and explain how they can impact the business. Show your roadmap with improvements and discuss various options more suitable for the client. This can include time, resources, and scope of work.

As a summary of your meeting, send your report to the client.

In conclusion

Architecture assessment is a complex process. To conduct an assessment properly, you need to have sufficient experience and knowledge.

It is indeed possible to provide the client with useful results for them and their business in just a week. Even if you are doing it alone.

From my experience, many improvements were stalled midway, and sometimes never started at all. Those who chose a middle ground and implemented only the most valuable improvements for the business with minimal labor costs significantly enhanced the quality of their product. Meanwhile, those who did nothing could end up shutting down the project in a couple of years.

Your goal is to show the client maximum improvements for a minimal price.

Other articles from the section architecture you can read in your spare time.

I wish you clean code and good architectural solutions.

Our Facebook group — Software Architecture and Development.

Source: habr.com

Buy reliable website hosting with DDoS protection, VPS VDS servers 🔥 Buy reliable website hosting with DDoS protection, VPS VDS servers | ProHoster