Hello, Habr!
In our work, our company frequently deals with various static code analysis tools (SAST). Out of the box, they all perform moderately. Of course, it all depends on the project and the technologies used in it, as well as how well these technologies are covered by analysis rules. In my opinion, one of the most important criteria for choosing a SAST tool is the ability to customize it to the specifics of your applications, namely to write and modify analysis rules, or as they are more commonly referred to, Custom Queries.
We mostly use Checkmarx — a very interesting and powerful code analyzer. In this article, I will share my experience with writing analysis rules for it.
Table of Contents
Introduction
To start, I would like to recommend one of the few articles in Russian about the specifics of writing queries for Checkmarx. It was published on Habr at the end of 2019 under the headline: .
It thoroughly covers how to write your first queries in CxQL (Checkmarx Query Language) for a certain test application and shows the basic principles of how analysis rules work.
I won’t repeat what is described there, although some overlaps will inevitably exist. In my article, I will try to compile a kind of "recipe book", a list of solutions to specific problems I've encountered during my work with Checkmarx. Many of these problems required considerable thought. Sometimes I lacked data in the documentation, and at times it was overall difficult to understand how to accomplish what was needed. I hope my experience and sleepless nights will not be in vain, and this "Custom Queries Recipe Book" will save you a few hours or a couple of nerve cells. So, let's get started!
General Information About Rules
First, let's review some basic concepts and the process of working with the rules for a better understanding of what will happen next. This is also necessary because it is not clearly stated or is heavily diluted across the documentation, making it inconvenient.
Rules are applied during scanning based on the preset selected at the start (a set of active rules). You can create an unlimited number of presets, and how you structure them depends on the specifics of your process. They can be grouped by languages or designated for each project. The number of active rules affects the speed and accuracy of the scanning.
Setting Up a Preset in the Checkmarx InterfaceRules are edited in a special tool called CxAuditor. This is a desktop application that connects to the server with Checkmarx. This tool has two modes of operation: editing rules and analyzing the results of a previously conducted scan.
CxAudit InterfaceRules in Checkmarx are categorized by languages, meaning that each language has its own set of queries. There are also some common rules that are applied regardless of the language, known as basic queries. For the most part, basic queries include searching for information that is used by other rules.
Categorizing Rules by LanguageRules can be 'Executable' and 'Non-Executable'. While the terminology may not be very accurate, the essence is that the results of 'Executable' rules will be displayed in the scanning results in the UI, whereas 'Non-Executable' rules are only needed for using their results in other queries (essentially just a function).
Determining Rule Type When CreatingYou can create new rules or modify/rewrite existing ones. To rewrite a rule, you need to find it in the tree, right-click, and select 'Override' from the dropdown menu. It is important to remember that new rules are not enabled in the presets by default and are inactive. To start using them, you need to activate them in the 'Preset Manager' menu in the tool. Rewritten rules retain their settings, meaning that if a rule was active, it will remain so and will be applied immediately.
Example of a new rule in the Preset Manager interfaceDuring execution, a "tree" of requests is built, showing dependencies. The first rules to execute are those that gather information, followed by those that use it. The results are cached, so if it's possible to use the results of an existing rule, it's better to do so, as it will reduce scanning time.
Rules can be applied at various levels:
Setting Up a Preset in the Checkmarx Interface
CxAudit Interface
Categorizing Rules by Language
Determining Rule Type When Creating
Example of a new rule in the Preset Manager interfaceFor the entire system — it will be used for any scanning of any project
At the team level — it will only apply to the scanning of projects within the selected team.
At the project level — it will apply to a specific project
Defining the level at which the rule will be applied
Defining the level at which the rule will be appliedA "Glossary" for Beginners
I'll start with a few things that raised questions for me, and I'll also show a number of techniques that will significantly simplify things.
Operations with Lists
- subtraction of one from another (list2 - list1)
* intersection of lists (list1 * list2)
+ addition of lists (list1 + list2)
& (logical AND) - combines lists on matches (list1 & list2), similar to intersection (list1 * list2)
| (logical OR) - combines lists on a broad search (list1 | list2)
Does not work with lists: ^ && || % \/ All Found Elements
Within the scanned language, you can obtain a list of all elements that Checkmarx has defined (strings, functions, classes, methods, etc.). This is a certain space of objects that can be accessed through All. So, to find an object with a specific name searchMe, you can search, for example, by name across all found objects:
// Такой запрос выдаст все элементы
result = All;
// Такой запрос выдаст все элементы, в имени которых присутствует “searchMe“
result = All.FindByName("searchMe");However, if you need to search in another language that for some reason was not included in the scan (like groovy in an Android project), you can expand our space of objects through the variable:
result = AllMembers.All.FindByName("searchMe");Functions for Flow Analysis
These functions are used in many rules, and here's a little cheat sheet on what they mean:
// Какие данные second влияют на first.
// Другими словами - ТО (second) что влияет на МЕНЯ (first).
result = first.DataInfluencedBy(second);
// Какие данные first влияют на second.
// Другими словами - Я (first) влияю на ТО (second).
result = first.DataInfluencingOn(second);Getting File Name/Path
There are several attributes that can be obtained from the results of the query execution (file name where the occurrence was found, line, etc.), but how to obtain and use them is not specified in the documentation. To do so, you need to access the LinePragma property, and within it will be the objects we need:
// Для примера найдем все методы
CxList methods = Find_Methods();
// В методах найдем по имени метод scope
CxList scope = methods.FindByName("scope");
// Таким образом можо получить путь к файлу
string current_filename = scope.GetFirstGraph().LinePragma.FileName;
// А вот таким - строку, где нашлось срабатывание
int current_line = scope.GetFirstGraph().LinePragma.Line;
// Эти параметры можно использовать по разному
// Например получить все объекты в файле
CxList inFile = All.FindByFileName(current_filename);
// Или найти что происходит в конкретной строке
CxList inLine = inFile.FindByPosition(current_line);It's worth noting that FileName actually contains the file path since we used the method GetFirstGraph.
Execution Results
Inside CxQL, there is a special variable result, which returns the result of executing your written rule. It is initialized immediately and you can write intermediate results to it, modifying and refining them during the process. However, if there is no assignment to this variable or function within the rule, returnthe execution result will always be null.
The following query will return nothing upon execution and will always be empty:
// Находим элементы foo
CxList libraries = All.FindByName("foo");But by assigning the execution result to the magic variable result, we will see what this call returns:
// Находим элементы foo
CxList libraries = All.FindByName("foo");
// Выводим, как результат выполнения правила
result = libraries
// Или еще короче
result = All.FindByName("foo");Using Execution Results of Other Rules
Rules in Checkmarx can be likened to functions in a traditional programming language. When writing a rule, you can certainly use results from other queries. For instance, there is no need to search for all method calls in the code every time, just call the necessary rule:
// Получаем результат выполнения другого правила
CxList methods = Find_Methods();
// Ищем внутри метод foo.
// Второй параметр false означает, что ищем без чувствительности к регистру
result = methods.FindByShortName("foo", false);This approach helps reduce code and significantly decreases execution time.
Troubleshooting
Logging
When working with the tool, sometimes it’s not possible to write the required query right away, and you have to experiment, trying various options. For such cases, the tool provides logging, which can be triggered as follows:
// Находим что-то
CxList toLog = All.FindByShortName("log");
// Формируем строку и отправляем в лог
cxLog.WriteDebugMessage (“number of DOM elements =” + All.Count);But it's important to remember that this method only accepts a string, so it won’t be possible to output the full list of found elements from the first operation's execution. The second option, used for debugging, is to periodically assign the execution result of the query to the magic variable result and see what comes out. This approach isn’t very convenient; you need to be sure that there are no overrides or operations with this in the code afterward, result or simply comment out the code located below. Alternatively, like I did, forget to remove several such calls from the final rule and be surprised why nothing works.
A more convenient way is to call the method return with the necessary parameter. In this case, the rule will finish executing and we will be able to see what the results of our writing are:
// Находим что-то
CxList toLog = All.FindByShortName("log");
// Выводим результат выполнения
return toLog
//Все, что написано дальше не будет выполнено
result = All.DataInfluencedBy(toLog)Login Issues
There are situations where you cannot access the CxAudit tool (which is used for writing rules). There can be many reasons for this, such as an unexpected crash, a sudden Windows update, BSOD, and other unforeseen circumstances beyond our control. In such cases, there may be an unfinished session in the database that prevents re-access. To fix this, you need to perform a few queries:
For Checkmarx up to 8.6:
// Проверяем, что есть залогиненые пользователи, выполнив запрос в БД
SELECT COUNT(*) FROM [CxDB].[dbo].LoggedinUser WHERE [ClientType] = 6;
// Если что-то есть, а на самом деле даже если и нет, попробовать выполнить запрос
DELETE FROM [CxDB].[dbo].LoggedinUser WHERE [ClientType] = 6;
For Checkmarx after 8.6:
// Проверяем, что есть залогиненые пользователи, выполнив запрос в БД
SELECT COUNT(*) FROM LoggedinUser WHERE (ClientType = 'Audit');
// Если что-то есть, а на самом деле даже если и нет, попробовать выполнить запрос
DELETE FROM [CxDB].[dbo].LoggedinUser WHERE (ClientType = 'Audit');Writing Rules
Now we come to the most interesting part. When you start writing rules in CxQL, it often feels like there is not enough documentation but rather a lack of real examples for solving specific tasks and a description of how queries work as a whole.
I will try to simplify things a bit for those who are starting to delve into the query language and provide several examples of using Custom Queries to solve specific tasks. Some of them are quite generic and can be applied in your company almost without changes, while others are more specific, but can also be used by adjusting the code to fit the specifics of your applications.
So, here are the tasks we frequently encountered:
Task: In the results of the rule execution, there are several Flows, and one of them is nested within another; it is necessary to keep one of them.
Solution: Indeed, sometimes Checkmarx shows several data flow movements, which may overlap and be a shortened version of others. For such cases, there is a special method ReduceFlow. Depending on the parameter, it will select the shortest or longest Flow:
// Оставить только длинные Flow
result = result.ReduceFlow(CxList.ReduceFlowType.ReduceSmallFlow);
// Оставить только короткие Flow
result = result.ReduceFlow(CxList.ReduceFlowType.ReduceBigFlow);Task: Expand the list of sensitive data that the tool reacts to
Solution: In Checkmarx, there are basic rules, the results of which are used by many other queries. By supplementing some of these rules with data specific to your application, you can immediately improve scanning results. Below is an example of a rule to get started:
General_privacy_violation_list
Let's add a few variables that are used in our application for storing sensitive information:
// Получаем результат выполнения базового правила
result = base.General_privacy_violation_list();
// Ищем элементы, которые попадают под простые регулярные выражения. Можно дополнить характерными для вас паттернами.
CxList personalList = All.FindByShortNames(new List<string> {
"*securityToken*", "*sessionId*"}, false);
// Добавляем к конечному результату
result.Add(personalList);Task: Expand the list of variables containing passwords
Solution: I would recommend paying attention to the basic password definition rule in the code and adding to it a list of variable names that are commonly used in your company.
Password_privacy_violation_list
CxList allStrings = All.FindByType("String");
allStrings.Add(All.FindByType(typeof(StringLiteral)));
allStrings.Add(Find_UnknownReference());
allStrings.Add(All.FindByType(typeof(Declarator)));
allStrings.Add(All.FindByType(typeof(MemberAccess)));
allStrings.Add(All.FindByType(typeof(EnumMemberDecl)));
allStrings.Add(Find_Methods().FindByShortName("get*"));
// Supplementing the default variable list
List pswdIncludeList = new List{"*password*", "*psw", "psw*", "pwd*", "*pwd", "*authKey*", "pass*", "cipher*", "*cipher", "pass", "adgangskode", "benutzerkennwort", "chiffre", "clave", "codewort", "contrasena", "contrasenya", "geheimcode", "geslo", "heslo", "jelszo", "kennwort", "losenord", "losung", "losungswort", "lozinka", "modpas", "motdepasse", "parol", "parola", "parole", "pasahitza", "pasfhocal", "passe", "passord", "passwort", "pasvorto", "paswoord", "salasana", "schluessel", "schluesselwort", "senha", "sifre", "wachtwoord", "wagwoord", "watchword", "zugangswort", "PAROLACHIAVE", "PAROLA CHIAVE", "PAROLECHIAVI", "PAROLE CHIAVI", "paroladordine", "verschluesselt", "sisma",
"pincode",
"pin"};
List pswdExcludeList = new List{"*pass", "*passable*", "*passage*", "*passenger*", "*passer*", "*passing*", "*passion*", "*passive*", "*passover*", "*passport*", "*passed*", "*compass*", "*bypass*", "pass-through", "passthru", "passthrough", "passbytes", "passcount", "passratio"};
CxList tempResult = allStrings.FindByShortNames(pswdIncludeList, false);
CxList toRemove = tempResult.FindByShortNames(pswdExcludeList, false);
tempResult -= toRemove;
tempResult.Add(allStrings.FindByShortName("pass", false));
foreach (CxList r in tempResult)
{
CSharpGraph g = r.data.GetByIndex(0) as CSharpGraph;
if(g != null && g.ShortName != null && g.ShortName.Length < 50)
{
result.Add(r);
}
}Task: Add the frameworks being used that are not supported by Checkmarx
Solution: All requests in Checkmarx are categorized by language, so the rules need to be supplemented for each language. Below are a few examples of such rules.
If libraries are used that supplement or replace the standard functionality, they can easily be added to the basic rule. Thus, anyone who uses it will immediately learn about new inputs. For example, logging libraries in Android — Timber and Loggi. There are no definitions for non-system calls in the basic rule set, so if a password or session identifier ends up in the log, we won't know about it. Let's try to add to the Checkmarx rules definitions of such methods.
A test code example that uses the Timber library for logging:
package com.death.timberdemo;
import android.support.v7.app.AppCompatActivity;
import android.os.Bundle;
import timber.log.Timber;
public class MainActivity extends AppCompatActivity {
private static final String TAG = MainActivity.class.getSimpleName();
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
Timber.e("Error Message");
Timber.d("Debug Message");
Timber.tag("Some Different tag").e("And error message");
}
}Here is an example of a request for Checkmarx that will allow adding the method call definitions from Timber as a data exit point from the application:
FindAndroidOutputs
// Получаем результат выполнения базового правила
result = base.Find_Android_Outputs();
// Дополняем вызовами, которые приходят из библиотеки Timber
CxList timber = All.FindByExactMemberAccess("Timber.*") +
All.FindByShortName("Timber").GetMembersOfTarget();
// Добавляем к конечному результату
result.Add(timber);Additionally, you can complement the neighboring rule that relates specifically to logging in Android:
FindAndroidLog_Outputs
// Получаем результат выполнения базового правила
result = base.Find_Android_Log_Outputs();
// Дополняем вызовами, которые приходят из библиотеки Timber
result.Add(
All.FindByExactMemberAccess("Timber.*") +
All.FindByShortName("Timber").GetMembersOfTarget()
);Also, if Android applications use for asynchronous work, it is good to additionally inform Checkmarx by adding the method for fetching data from the task getInputData:
FindAndroidRead
// Получаем результат выполнения базового правила
result = base.Find_Android_Read();
// Дополняем вызовом функции getInputData, которая используется в WorkManager
CxList getInputData = All.FindByShortName("getInputData");
// Добавляем к конечному результату
result.Add(getInputData.GetMembersOfTarget());Task: Search for sensitive data in plist files for iOS projects
Solution: Often, special files with the .plist extension are used to store various variables and values in iOS. Storing passwords, tokens, keys, and other sensitive data in these files is not recommended, as they can be easily extracted from the device.
Plist files have features that are not obvious to the naked eye but are important for Checkmarx. We will write a rule that will search for the data we need and notify us if passwords or tokens are mentioned somewhere.
An example of such a file in which a token for communication with the backend service is embedded:
DeviceDictionary
phone
iPhone 6s
privatekey
MIICXAIBAAKBgQCqGKukO1De7zhZj6+And a rule for Checkmarx that includes several nuances to consider when writing:
// Используем результат выполнения правила по поиску файлов plist, чтобы уменьшить время работы правила и
CxList plist = Find_Plist_Elements();
// Инициализируем новую переменную
CxList dictionarySettings = All.NewCxList();
// Теперь добавим поиск всех интересующих нас значений. В дальнейшем можно расширять этот список.
// Для поиска значений, как ни странно, используется FindByMemberAccess - поиск обращений к методам. Второй параметр внутри функции, false, означает, что поиск нечувствителен к регистру
dictionarySettings.Add(plist.FindByMemberAccess("privatekey", false));
dictionarySettings.Add(plist.FindByMemberAccess("privatetoken", false));
// Для корректного поиска из-за особенностей структуры plist - нужно искать по типу "If statement"
CxList ifStatements = plist.FindByType(typeof(IfStmt));
// Добавляем в результат, перед этим получив родительский узел - для правильного отображения
result = dictionarySettings.FindByFathers(ifStatements);Task: Searching for information in XML
Solution: Checkmarx has very convenient functions for working with XML and searching for values, tags, attributes, and more. Unfortunately, there is an error in the documentation that causes no example to work. Although this issue has been resolved in the latest version of the documentation — be careful if you are using earlier versions of the documents.
Here is an incorrect example from the documentation:
// Код работать не будет
result = All.FindXmlAttributesByNameAndValue("*.app", 8, “id”, "error- section", false, true);As a result of the attempt to execute, we will receive an error that All there is no such method… And this is true, as there is a special, separate object space for using functions that work with XML — cxXPath. Here is what a correct request looks like for finding a setting in Android that allows the use of HTTP traffic:
// Правильный вариант с использованием cxXPath
result = cxXPath.FindXmlAttributesByNameAndValue("*.xml", 8, "cleartextTrafficPermitted", "true", false, true);Let's break it down in a bit more detail, as the syntax for all functions is similar; once you've figured one out, you just need to select the relevant one. So, step by step by parameters:
"*.xml"— a file mask to search for8— language ID for which the rule applies"cleartextTrafficPermitted"— attribute name in XML"true"— value of this attributefalse— using regular expressions for the searchtrue— means the search will be case-insensitive
For example, a rule is used to identify insecure settings for network connections in Android that allow communication with the server via the HTTP protocol. An example of a setting containing the attribute cleartextTrafficPermitted with the value true:
example.com
secure.example.comTask: Limit results by file name/path
Solution: In one of the larger projects related to mobile application development for Android, we encountered false positives from a rule that defines the obfuscation settings. The thing is, this rule out of the box searches the file build.gradle for the settings that apply obfuscation rules for the release version of the app.
However, in large projects, there are often child files build.gradle, which are related to libraries included in the project. The peculiarity is that even if these files do not specify the need for obfuscation, during compilation, the settings of the parent build file will be applied.
Thus, the task is to eliminate triggers in child files that relate to libraries. They can be identified by the presence of the line apply 'com.android.library'.
An example of code from the file build.gradle, defining the need for obfuscation:
apply plugin: 'com.android.application'
android {
compileSdkVersion 24
buildToolsVersion "24.0.2"
defaultConfig {
...
}
buildTypes {
release {
minifyEnabled true
...
}
}
}
dependencies {
...
}Example file build.gradle for a library included in the project without such settings:
apply plugin: 'android-library'
dependencies {
compile 'com.android.support:support-v4:18.0.+'
}
android {
compileSdkVersion 14
buildToolsVersion '17.0.0'
...
}And the rule for Checkmarx:
ProGuardObfuscationNotInUse
// Поиск метода release среди всех методов в Gradle файлах
CxList releaseMethod = Find_Gradle_Method("release");
// Все объекты из файлов build.gradle
CxList gradleBuildObjects = Find_Gradle_Build_Objects();
// Поиск того, что находится внутри метода "release" среди всех объектов из файлов build.gradle
CxList methodInvokesUnderRelease = gradleBuildObjects.FindByType(typeof(MethodInvokeExpr)).GetByAncs(releaseMethod);
// Ищем внутри gradle-файлов строку "com.android.library" - это значит, что данный файл относится к библиотеке и его необходимо исключить из правила
CxList android_library = gradleBuildObjects.FindByName("com.android.library");
// Инициализация пустого массива
List<string> libraries_path = new List<string> {};
// Проходим через все найденные "дочерние" файлы
foreach(CxList library in android_library)
{
// Получаем путь к каждому файлу
string file_name_library = library.GetFirstGraph().LinePragma.FileName;
// Добавляем его в наш массив
libraries_path.Add(file_name_library);
}
// Ищем все вызовы включения обфускации в релизных настройках
CxList minifyEnabled = methodInvokesUnderRelease.FindByShortName("minifyEnabled");
// Получаем параметры этих вызовов
CxList minifyValue = gradleBuildObjects.GetParameters(minifyEnabled, 0);
// Ищем среди них включенные
CxList minifyValueTrue = minifyValue.FindByShortName("true");
// Немного магии, если не нашли стандартным способом :D
if (minifyValueTrue.Count == 0) {
minifyValue = minifyValue.FindByAbstractValue(abstractValue => abstractValue is TrueAbstractValue);
} else {
// А если всё-таки нашли, то предыдущий результат и оставляем
minifyValue = minifyValueTrue;
}
// Если не нашлось таких методов
if (minifyValue.Count == 0)
{
// Для более корректного отображения места срабатывания в файле ищем или buildTypes или android
CxList tempResult = All.NewCxList();
CxList buildTypes = Find_Gradle_Method("buildTypes");
if (buildTypes.Count > 0) {
tempResult = buildTypes;
} else {
tempResult = Find_Gradle_Method("android");
}
// Для каждого из найденных мест срабатывания проходим и определяем, дочерний или основной файлы сборки
foreach(CxList res in tempResult)
{
// Определяем, в каком файле был найден buildType или android методы
string file_name_result = res.GetFirstGraph().LinePragma.FileName;
// Если такого файла нет в нашем списке "дочерних" файлов - значит это основной файл и его можно добавить в результат
if (libraries_path.Contains(file_name_result) == false){
result.Add(res);
}
}
}This approach can be quite universal and useful not only for Android applications but also for other scenarios where it's necessary to determine the affiliation of results to a specific file.
Task: Add support for a third-party library if the syntax is not fully supported
Solution: The number of various frameworks used in the coding process is overwhelming. Of course, Checkmarx does not always recognize their existence, and our task is to teach it that certain methods belong specifically to this framework. Sometimes this is complicated by the fact that frameworks use function names that are widely used, making it difficult to determine the relationship of a particular call to a specific library.
The difficulty lies in the fact that the syntax of such libraries is not always recognized correctly, and experimentation is necessary to avoid receiving a large number of false positives. There are several options to improve scanning accuracy and address the problem:
The first option is if we know for sure that the library is used in a specific project and can apply the rule at the team level. However, if the team decides to adopt another approach or uses several libraries with overlapping function names, we could end up with an unpleasant situation of numerous false positives.
The second option is to perform a file search where the library is explicitly imported. With this approach, we can be confident that the necessary library is definitely applied in this file.
And the third option is to use the two previously mentioned approaches in combination.
As an example, let's analyze a well-known library in niche circles for the Scala programming language, specifically its functionality In general, to pass parameters in an SQL query, it is necessary to use an operator $, which substitutes data into a pre-formed SQL query. So, it is essentially a direct equivalent of a Prepared Statement in Java. However, if there is a need to dynamically construct an SQL query, for instance, if you need to pass table names, you can use the operator #$, which directly substitutes data into the query (practically, like string concatenation).
Code example:
// В общем случае - значения, контролируемые пользователем
val table = "coffees"
sql"select * from #$table where name = $name".as[Coffee].headOptionCheckmarx currently does not identify the use of Splicing Literal Values and skips the operators #$, so let's try to teach it to detect potential SQL injections and highlight the necessary places in the code:
// Находим все импорты
CxList imports = All.FindByType(typeof(Import));
// Ищем по имени, есть ли в импортах slick
CxList slick = imports.FindByShortName("slick");
// Некоторый флаг, определяющий, что импорт библиотеки в коде присутствует
// Для более точного определения - можно применить подход с именем файла
bool not_empty_list = false;
foreach (CxList r in slick)
{
// Если встретили импорт, считаем, что slick используется
not_empty_list = true;
}
if (not_empty_list) {
// Ищем вызовы, в которые передается SQL-строка
CxList sql = All.FindByShortName("sql");
sql.Add(All.FindByShortName("sqlu"));
// Определяем данные, которые попадают в эти вызовы
CxList data_sql = All.DataInfluencingOn(sql);
// Так как синтакис не поддерживается, можно применить подход с регулярными выражениями
// RegExp стоит использовать крайне осторожно и не применять его на большом количестве данных, так как это может сильно повлиять на производительность
CxList find_possible_inj = data_sql.FindByRegex(@"#$", true, true, true);
// Избавляемся от лишних срабатываний, если они есть и выводим в результат
result = find_possible_inj.FindByType(typeof(BinaryExpr));
}Task: Searching for used vulnerable functions in Open-Source libraries
Solution: Many companies use tools for controlling Open-Source (OSA practice), allowing for the detection of vulnerable library versions used in developed applications. Sometimes, it is not feasible to update such a library to a safe version. In some cases, there are functional limitations, while in others, there is simply no safe version. In such cases, a combination of SAST and OSA practices can help determine that functions that lead to exploiting vulnerabilities are not being used in the code.
But sometimes, especially when considering JavaScript, this may not be a trivial task. Below is a solution that may not be perfect, but it works, using the example of vulnerabilities in the component lodash in methods template and *set.
Examples of potentially vulnerable test code in a JS file:
/**
* Template example
*/
'use strict';
var _ = require("./node_modules/lodash.js");
// Use the "interpolate" delimiter to create a compiled template.
var compiled = _.template('hello <%= js %>!');
console.log(compiled({ 'js': 'lodash' }));
// => 'hello lodash!'
// Use the internal `print` function in "evaluate" delimiters.
var compiled = _.template('<% print("hello " + js); %>!');
console.log(compiled({ 'js': 'lodash' }));
// => 'hello lodash!'And when connected directly in HTML:
<!DOCTYPE html>
<html>
<head>
<title>Lodash Tutorial</title>
<script src="./node_modules/lodash.js"></script>
<script type="text/javascript">
// Lodash chunking array
nums = [1, 2, 3, 4, 5, 6, 7, 8, 9];
let c1 = _.template('<% print("hello " + js); %>!');
console.log(c1);
let c2 = _.template('<% print("hello " + js); %>!');
console.log(c2);
</script>
</head>
<body></body>
</html>We are looking for all our vulnerable methods that are listed in the vulnerabilities:
// Ищем все строки: в которых встречается строка lodash (предполагаем, что это объявление импорта библиотеки
CxList lodash_strings = Find_String_Literal().FindByShortName("*lodash*");
// Ищем все данные: которые взаимодействуют с этими строками
CxList data_on_lodash = All.InfluencedBy(lodash_strings);
// Задаем список уязвимых методов
List<string> vulnerable_methods = new List<string> {"template", "*set"};
// Ищем все наши уязвимые методы, которые перечисленны в уязвимостях и отфильтровываем их только там, где они вызывались
CxList vulnerableMethods = All.FindByShortNames(vulnerable_methods).FindByType(typeof(MethodInvokeExpr));
//Находим все данные: которые взаимодействуют с данными методами
CxList vulnFlow = All.InfluencedBy(vulnerableMethods);
// Если есть пересечение по этим данным - кладем в результат
result = vulnFlow * data_on_lodash;
// Формируем список путей по которым мы уже прошли, чтобы фильтровать в дальнейшем дубли
List<string> lodash_result_path = new List<string> {};
foreach(CxList lodash_result in result)
{
// Очередной раз получаем пути к файлам
string file_name = lodash_result.GetFirstGraph().LinePragma.FileName;
lodash_result_path.Add(file_name);
}
// Дальше идет часть относящаяся к html файлам, так как в них мы не можем проследить откуда именно идет вызов
// Формируем массив путей файлов, чтобы быть уверенными, что срабатывания уязвимых методов были именно в тех файлах, в которых объявлен lodash
List<string> lodash_path = new List<string> {};
foreach(CxList string_lodash in lodash_strings)
{
string file_name = string_lodash.GetFirstGraph().LinePragma.FileName;
lodash_path.Add(file_name);
}
// Перебираем все уязвимые методы и убеждаемся, что они вызваны в тех же файлах, что и объявление/включение lodash
foreach(CxList method in vulnerableMethods)
{
string file_name_method = method.GetFirstGraph().LinePragma.FileName;
if (lodash_path.Contains(file_name_method) == true && lodash_result_path.Contains(file_name_method) == false){
result.Add(method);
}
}
// Убираем все UknownReferences и оставляем самый "длинный" из путей, если такие встречаются
result = result.ReduceFlow(CxList.ReduceFlowType.ReduceSmallFlow) - result.FindByType(typeof(UnknownReference));Task: Searching for hardcoded certificates in the application
Solution: Often applications, especially mobile ones, use certificates or keys to access various servers or verify SSL-Pinning. From a security standpoint, storing such things in code is not the best practice. Let's try to write a rule that will search for similar files in the repository:
// Найдем все сертификаты по маске файла
CxList find_certs = All.FindByShortNames(new List<string> {"*.der", "*.cer", "*.pem", "*.key"}, false);
// Проверим, где в приложении они используются
CxList data_used_certs = All.DataInfluencedBy(find_certs);
// И для мобильных приложений - можем поискать методы, где вызывается чтение сертификатов
// Для других платформ и приложений могут быть различные методы
CxList methods = All.FindByMemberAccess("*.getAssets");
// Пересечение множеств даст нам результат по использованию локальных сертификатов в приложении
result = methods * data_used_certs;Task: Searching for compromised tokens in the application
Solution: It is often necessary to revoke compromised tokens or other important information present in the code. Of course, storing them within the source code is not the best idea, but situations vary. Thanks to CxQL queries, finding such items is quite straightforward:
// Получаем все строки, которые содержатся в коде
CxList strings = base.Find_Strings();
// Ищем среди всех строк нужное нам значение. В примере токен в виде строки "qwerty12345"
result = strings.FindByShortName("qwerty12345");Conclusion
I hope this article will be helpful for those who are starting their journey with the Checkmarx tool. Even those who have been writing their own rules for a while may find something useful in this guide.
Unfortunately, there is currently a lack of resources where one can gather new ideas during the rule development process for Checkmarx. That’s why we created , where we will share our developments so that anyone using CxQL can find something useful in it, as well as have the opportunity to share their work with the community. The repository is in the process of being filled and structured, so contributors are welcome!
Thank you for your attention!
Source: habr.com
