🥇Security audit of the MCS cloud platform

SkyShip Dusk by SeerLight

Building any service necessarily involves continuous work on security. Security is an ongoing process that includes constant analysis and improvement of product protection, monitoring news about vulnerabilities, and much more. This also includes audits. Audits are conducted both internally and by external experts who can significantly help with security, as they are not immersed in the project and have a fresh perspective.

This article discusses the fresh perspective of external experts who assisted the Mail.ru Cloud Solutions (MCS) team in testing their cloud service and what they discovered. For the 'external forces,' MCS chose Digital Security, known for its high level of expertise in cybersecurity circles. In this article, we will examine some interesting vulnerabilities found during the external audit — so that you can avoid similar pitfalls when creating your cloud service.

Product Description

Mail.ru Cloud Solutions (MCS) is a platform for building virtual infrastructure in the cloud. It includes IaaS, PaaS, and a marketplace of ready-made application images for developers. Taking into account the architecture of MCS, the security of the product needed to be verified in the following areas: protection of the virtualization environment infrastructure: hypervisors, routing, firewalls;

protection of client virtual infrastructure: isolation from each other, including networking, private networks in SDN;
OpenStack and its open components;
S3 developed in-house;
IAM: multi-tenant projects with a role-based model;
Vision (computer vision): API and vulnerabilities when working with images;
web interface and classic web attacks;
vulnerabilities of PaaS components;
API of all components.
Perhaps, everything significant for the subsequent history.

What work was done and why is it necessary?

The security audit is aimed at identifying vulnerabilities and configuration errors that may lead to leaks of personal data, modification of sensitive information, or disruption of service availability.

A security audit is aimed at identifying vulnerabilities and configuration errors that could lead to the leakage of personal data, modification of sensitive information, or disruption of service availability.

During the work, which lasts on average 1-2 months, auditors replicate the actions of potential attackers and search for vulnerabilities in both the client and server sides of the selected service. In the context of the MCS cloud platform audit, the following objectives were identified:

Analysis of authentication within the service. Vulnerabilities in this component would allow immediate access to others' accounts.
Examination of the role model and access control between different accounts. For an attacker, the ability to access another user's virtual machine is a desired goal.
Client-side vulnerabilities. XSS/CSRF/CRLF/etc. Is there a possibility to attack other users through malicious links?
Server-side vulnerabilities: RCE and various injections (SQL/XXE/SSRF, etc.). Server vulnerabilities are generally harder to find, but they can lead to the compromise of multiple users at once.
Analysis of user segment isolation at the network level. For an attacker, a lack of isolation significantly increases the attack surface on other users.
Business logic analysis. Is it possible to deceive the system and create virtual machines for free?

In this project, the work was conducted using a 'Gray-box' model: auditors interacted with the service with regular user privileges but partially had access to the API source code and could clarify details with developers. This is usually the most convenient and sufficiently realistic working model: internal information can still be gathered by an attacker; it is just a matter of time.

Identified vulnerabilities

Before an auditor begins sending various payloads (the data used to conduct an attack) to random locations, it is essential to understand how everything works and what functionality is represented. It may seem like a pointless exercise, as most places studied may not have vulnerabilities. But understanding the application structure and its logic is necessary to find the most complex attack vectors.

It's important to identify areas that appear suspicious or significantly differ from others. The first dangerous vulnerability was indeed found in this manner.

IDOR

IDOR vulnerabilities (Insecure Direct Object Reference) are among the most common types of vulnerabilities in business logic, allowing unauthorized access to objects in various ways. IDOR vulnerabilities create opportunities to access user information with varying levels of sensitivity.

One variant of IDOR involves manipulating access identifiers to perform actions on system objects (users, bank accounts, items in a cart). This can lead to unpredictable consequences, such as the ability to substitute the sender's account, allowing theft from other users.

In the case of MCS, auditors discovered an IDOR vulnerability related to insecure identifiers. The user's personal account used UUIDs for accessing any objects, which security professionals deemed impressively unbruteforceable (i.e., protected against brute-force attacks). However, for certain entities, it was found that ordinary predictable numbers were used to retrieve application user information. I think you can guess that it was possible to change the user ID by one, resend the request, and thereby obtain information bypassing the ACL (access control list, which defines access rules for data for processes and users).

Server Side Request Forgery (SSRF)

Open source products are beneficial because there are numerous forums with detailed technical descriptions of issues that arise and, if you're lucky, descriptions of their solutions. However, this has a downside: known vulnerabilities are also described in detail. For instance, the OpenStack forum has excellent descriptions of vulnerabilities [XSS] and [SSRF], which for some reason no one is in a hurry to fix.

A common application feature is the ability for users to send a link to the server that the server then accesses (for example, to upload an image from a specified source). Without adequate filtering of the links themselves or server responses, this functionality can easily be exploited by malicious actors.

SSRF vulnerabilities can significantly advance the development of an attack. An attacker can gain:

limited access to the targeted local network, for example, only to certain segments of the network and through a specific protocol;
full access to the local network if a downgrade from the application layer to the transport layer is possible, resulting in complete control over application-level traffic;
read access to local files on the server (if the file:/// scheme is supported);
and much more.

The SSRF vulnerability in OpenStack has long been known for its 'blind' nature: when accessing the server, you do not receive a response, but rather encounter different types of errors/delays depending on the request result. Based on this, port scanning can be executed on hosts within the internal network, leading to serious consequences that should not be underestimated. For instance, a product may have an API for the back office accessible only from the corporate network. Possessing documentation (remember, insiders), an attacker can use SSRF to access internal methods. For example, if a rough list of useful URLs is obtained, SSRF can be employed to navigate through them and execute requests—so to speak, transferring money from account to account or changing limits.

This is not the first instance of SSRF vulnerabilities being discovered in OpenStack. In the past, there was a possibility to upload VM ISO images via direct links, which also led to similar consequences. Currently, this feature has been removed from OpenStack. Apparently, the community considered this the simplest and most reliable solution to the problem.

And in this publicly available report from HackerOne (h1), the exploitation of the non-blind SSRF with the ability to read instance metadata leads to obtaining Root access to the entire Shopify infrastructure.

In MCS, two locations with similar functionality revealed SSRF vulnerabilities, but they were virtually impossible to exploit due to firewalls and other defenses. Nevertheless, the MCS team fixed the problem proactively, without waiting for the community.

XSS instead of uploading 'shells'

Despite hundreds of studies written, year after year, XSS (cross-site scripting) remains the most commonly encountered web vulnerability (or attack?).

File uploads are a favorite spot for any security researcher. Often, it's possible to upload arbitrary scripts (asp/jsp/php) and execute OS commands, which pentesters refer to as 'uploading a shell'. However, the popularity of such vulnerabilities works both ways: they are remembered, and countermeasures are developed against them, so lately, the likelihood of 'uploading a shell' has approached zero.

The attacking team (in the form of Digital Security) got lucky. Okay, in MCS, the content of uploaded files was checked on the server side; only images were allowed. But SVGs are images too. So, what can be dangerous about SVG images? The fact that they can embed fragments of JavaScript!

It turned out that the uploaded files are accessible to all users of the MCS service—meaning one can attack other cloud users, specifically the administrators.

Example of injection using an XSS attack on a phishing login form.

Examples of exploiting XSS attacks:

Why try to steal a session (especially since HTTP-Only cookies are in use everywhere, protected against theft by js scripts), if the uploaded script can immediately access the resource's API? In this case, the payload could change the server configuration via XHR requests, for example, adding the attacker's open SSH key and gaining SSH access to the server.
If the CSP policy (Content Security Policy) prohibits the injection of JavaScript, the attacker can manage without it. They can create a fake login form using pure HTML and phish for the administrator's password through advanced phishing: the phishing page for the user appears at the same URL, making it harder for the user to notice.
Finally, the attacker can arrange for a client-side DoS by setting cookies larger than 4 KB. The user only needs to open the link once—and the entire site becomes inaccessible until they figure out to clear their browser: in the overwhelming majority of cases, the web server will refuse to accept such a client.

Let's consider an example of another identified XSS, this time with a more cunning exploitation method. The MCS service allows grouping firewall settings. The discovered XSS was found in the group name. Its uniqueness was that the vector did not trigger immediately, not when viewing the list of rules, but when deleting a group:

In other words, the scenario played out as follows: an attacker creates a firewall rule with a 'load' in the name, the administrator eventually notices it and initiates the deletion process. This is when the malicious JS kicks in.

To protect against XSS in uploaded SVG images (if they cannot be avoided), the Digital Security team recommended developers of MCS:

Host files uploaded by users on a separate domain that has nothing to do with 'cookies'. The script will execute in the context of another domain and will not pose a threat to MCS.
In the HTTP response from the server, return the header 'Content-disposition: attachment'. This way, files will be downloaded by the browser rather than executed.

Additionally, there are now many ways available for developers to mitigate the risks of XSS exploitation:

Using the 'HTTP Only' flag can make session 'Cookies' headers inaccessible to malicious JavaScript;
a properly implemented CSP policy will significantly complicate XSS exploitation for an attacker;
modern templating engines like Angular or React automatically sanitize user data before rendering it in the user's browser.

Two-Factor Authentication Vulnerabilities

To enhance account security, users are always recommended to enable 2FA (two-factor authentication). Indeed, this is an effective method to prevent an attacker from gaining access to the service if the user's credentials have been compromised.

But does using a second factor of authentication always ensure account safety? There are security issues in the implementation of 2FA:

Brute Forcing OTP Codes (one-time codes). Despite being easy to exploit, issues like lack of brute force protection for OTPs are found even in large companies: the Slack case, the Facebook case.
Weak generation algorithms, such as the ability to predict the next code.
Logical errors, such as the ability to request someone else's OTP on your phone, like it was at Shopify.

In the case of MCS, 2FA is implemented based on Google Authenticator and Duo. The protocol itself has stood the test of time, but the implementation of code verification on the application side should be checked.

In MCS, 2FA is used in several places:

During user authentication. There is a protection mechanism against brute-forcing: the user has only a few attempts to enter the one-time password before input is blocked for a period of time. This blocks the possibility of guessing the OTP through trial and error.
When generating offline backup codes for performing 2FA and its deactivation. There was no protection against brute-forcing implemented here, which allowed for the regeneration of backup codes or the complete deactivation of 2FA if the account password and an active session were available.

Considering that the backup codes were within the same range of string values as those generated by the OTP application, the chance of guessing a code in a short time was significantly higher.

The process of guessing the OTP to deactivate 2FA using the Burp: Intruder tool.

Result

Overall, MCS as a product turned out to be secure. During the audit, the pentesting team was unable to access client VMs and their data, and the vulnerabilities found were quickly fixed by the MCS team.

However, it is important to note that security is an ongoing effort. Services are not static; they constantly evolve. It is impossible to develop a product completely devoid of vulnerabilities. However, vulnerabilities can be found in a timely manner and the chances of recurrence minimized.

All the vulnerabilities mentioned in MCS have now been fixed. To minimize the occurrence of new ones and reduce their lifespan, the platform team continues to: