Content-based data classification is an open problem. Traditional data loss prevention (DLP) systems tackle this by fingerprinting relevant data and monitoring endpoints for these fingerprints. Given the vast number of constantly changing data resources on Facebook, this approach is not only unscalable but also ineffective in determining where the data resides. This article is dedicated to an end-to-end system built for detecting sensitive semantic types on Facebook at scale and automatically ensuring data storage and access control.
The approach described here is our first end-to-end privacy system that attempts to address this issue by incorporating data signals, machine learning, and traditional fingerprinting methods to map and classify all data on Facebook. The described system is operational in a production environment, achieving an average F2 score of 0.9+ across various privacy classes while processing a large number of data resources across dozens of repositories. We present the translation of a Facebook publication on ArXiv regarding scalable data classification for ensuring security and privacy based on machine learning.
Introduction
Today, organizations collect and store vast amounts of data in various formats and locations [1]. The data is then consumed in many places, sometimes copied or cached multiple times, resulting in valuable and sensitive business information being scattered across numerous corporate data stores. When an organization is required to fulfill certain legal or regulatory requirements, such as complying with regulations during civil litigation, the need arises to collect data regarding the location of the necessary information. When a privacy order states that an organization must mask all Social Security Numbers (SSNs) when transmitting personal information to unauthorized parties, the natural first step is to locate all SSNs in the organization’s data stores. In such circumstances, data classification becomes crucial [1]. A classification system will allow organizations to automatically ensure compliance with confidentiality and security policies, such as implementing access management policies and data retention. Facebook presents a system built by us at Facebook that uses multiple data signals, scalable system architecture, and machine learning to detect sensitive semantic types of data.
Data detection and classification involve searching for and tagging data in a way that allows for quick and efficient retrieval of relevant information when necessary. The current process is mostly manual, involving studying relevant laws or regulations, determining which types of information should be deemed sensitive, and considering the various levels of sensitivity, followed by appropriately building classes and classification policies [1]. After data loss prevention (DLP) systems fingerprint the data and track endpoints downstream to obtain fingerprints. When working with a storage system that has a large number of assets and petabytes of data, such an approach simply does not scale.
Our goal is to build a data classification system that scales for both stable and unstable user data, without any additional restrictions on type or format of data. This is an ambitious goal, and naturally it comes with challenges. Any data record may be thousands of characters long.
Figure 1. Online and Offline Forecasting Streams
Therefore, we must represent it effectively, using a common set of features that can subsequently be integrated and easily transferred. These features must not only provide accurate classification but also ensure flexibility and scalability for the easy addition and discovery of new data types in the future. Secondly, we need to deal with large standalone tables. Stable data can be stored in tables that are many petabytes in size, which may lead to a decrease in scan speed. Thirdly, we must adhere to strict SLA classification for unstable data. This requires the system to be highly efficient, fast, and accurate. Finally, we need to ensure low-latency data classification for unstable data to perform real-time classification as well as for online use cases.
This article describes how we tackled the above challenges and presents a fast and scalable classification system that classifies data elements of all types, formats, and sources based on a common set of features. We expanded the system architecture and created a specialized machine learning model for rapid classification of offline and online data. This article is organized as follows: Section 2 presents the overall system design. Section 3 discusses the components of the machine learning system. Sections 4 and 5 cover related work and outline future directions.
Architecture
To address the challenges of stable data and online data at Facebook's scale, the classification system has two separate streams that we will discuss in detail.
Stable Data
Initially, the system needs to gather information about a multitude of Facebook's informational assets. For each storage location, some basic information is collected, such as the data center housing this data, the system handling that data, and the assets located in a specific data repository. This forms a metadata catalog that allows the system to efficiently retrieve data without overloading clients and resources utilized by other engineers.
This metadata catalog serves as a reliable source for all scanned assets and enables the tracking of the status of various assets. With this information, a prioritization of scheduling is established based on collected data and internal information from the system, such as the time of the last successful asset scan and its creation time, along with past memory and CPU requirements for this asset if it has been scanned before. Then, for each data resource (as resources become available), a job is called to perform the actual scanning of the resource.
Each job is a compiled binary file that executes Bernoulli sampling on the latest data available for each asset. The asset is divided into separate columns, where the classification result of each column is processed independently. Additionally, the system scans any saturated data within the columns. JSON, arrays, encoded structures, URLs, base64 serialized data, and much more are all scanned. This can significantly increase the scan execution time, as a single table may contain thousands of nested columns within a large binary object. json.
For each row selected from the data asset, the classification system extracts floating and textual objects from the content and links each object back to the column from which it was taken. The result of the object extraction stage is a map of all objects for each column found in the data asset.
What are the features for?
The concept of features is crucial. Instead of the float and text features, we can pass raw string samples that are directly extracted from each data resource. Additionally, machine learning models can be trained directly on each sample rather than on hundreds of feature calculations that only attempt to approximate the sample. There are several reasons for this:
- Privacy first: the most important aspect is that the concept of features allows us to store in memory only the samples we extract. This ensures that we retain samples solely for the intended purpose and never log them through our own efforts. This is especially important for volatile data, as the service must maintain some state of classification before providing a prediction.
- Memory: some samples can be thousands of characters long. Storing such data and transmitting it across system parts without necessity consumes many additional bytes. Two factors can combine over time, given that there are many data resources with thousands of columns.
- Feature aggregation: features present the results of each scan clearly through their set, allowing the system to combine the results of previous scans of the same data resource conveniently. This can be useful for aggregating scan results of a single data resource across multiple runs.
The features are then sent to the prediction service, where we use rule-based classification and machine learning to predict the labels of data for each column. The service relies on both rule classifiers and machine learning, selecting the best prediction provided from each prediction object.
Rule classifiers are manual heuristics; they use calculations and coefficients to normalize an object within a range of 0 to 100. Once such an initial score is generated for each data type and the column name associated with that data, falling into no 'blacklists', the rule classifier selects the highest normalized score among all data types.
Due to the complexity of classification, relying solely on manual heuristics results in low classification accuracy, especially for unstructured data. For this reason, we developed a machine learning system to handle the classification of unstructured data such as user content and addresses. Machine learning has enabled us to move away from manual heuristics and apply additional data signals (e.g., column names, data origins), significantly enhancing detection accuracy. We will delve deeper into our machine learning architecture later.
The forecasting service stores results for each column along with metadata related to the timing and status of the scan. Any consumers and downstream processes that depend on this data can read it from a daily published dataset. This dataset aggregates the results of all these scan tasks, or the real-time data catalog API. The published forecasts are the foundation for automatically applying privacy and security policies.
Finally, once the forecasting service records all the data and all forecasts are saved, our data catalog API can return all forecasts of data types for the resource in real time. Every day, the system publishes a dataset containing the latest forecasts for each asset.
Unstable data
Although the process described above is designed for stored assets, non-stored traffic is also considered part of organizational data and can be important. For this reason, the system provides a real-time classification forecast generation online API for any unstable traffic. The real-time forecasting system is widely used for classifying outgoing traffic, incoming traffic in machine learning models, and advertiser data.
Here, the API accepts two main arguments: the group key and the raw data to be predicted. The service performs the same object extraction as described above, grouping objects together for the same key. These features are also supported in the stored cache for recovery after failure. For each group key, the service ensures that it has seen enough samples in accordance with the process outlined above before calling the prediction service.
Optimization
To scan some storage, we use libraries and methods to optimize reading from hot storage [2] and ensure that there are no failures from other users accessing the same storage.
For extremely large tables (50+ petabytes), despite all optimizations and memory efficiency, the system struggles to scan and compute everything before memory runs out. Ultimately, scanning is fully computed in memory and is not saved during the scan. If large tables contain thousands of columns with unstructured data blobs, the task may fail due to insufficient memory resources when making predictions for the entire table. This will lead to reduced coverage. To combat this, we have optimized the system to use scan speed as a mediator for how well the system handles the current load. We use speed as a predictive mechanism to identify memory issues and when pre-computing the object map. In doing so, we utilize less data than usual.
Data signals
The classification system is only as good as the signals from the data. Here, we will examine all signals used by the classification system.
- Based on content: Of course, the primary and most important signal is the content itself. A Bernoulli sampling is performed for each data asset we scan, extracting features based on the data content. Many features arise from the content. There can be any number of floating objects representing calculations of how many times a certain type of sample has been observed. For instance, we might have counts of the number of emails seen in the sample, or features indicating how many emojis have been spotted. These feature calculations can be normalized and aggregated across various scans.
- Data provenance: an important signal that can help when the content has changed from the parent table. A common example is hashed data. When data in the child table is hashed, it often originates from the parent table, where it remains in plaintext. Provenance data helps classify certain types of data when they are not clearly readable or transformed from an upstream table.
- Annotations: another high-quality signal that aids in the identification of unstructured data. In fact, annotations and provenance data can work together to disseminate attributes across various data assets. Annotations help identify the source of unstructured data, while provenance data can assist in tracking the flow of that data throughout the repository.
- Data injection is a method where specific, unreadable characters are deliberately introduced into known sources with known data types. Then, whenever we scan content with the same unreadable sequence of characters, we can infer that the content originates from this known data type. This serves as another quality data signal, similar to annotations. The difference is that content-based detection helps discover injected data.
Metric Measurement
An important component is a strict methodology for measuring metrics. The primary metrics for iterating improvements in classification are the precision and recall of each label, with the F2 score being the most critical.
To calculate these indicators, an independent data asset labeling methodology is required, which does not depend on the system itself but can be used for direct comparison with it. Below, we will describe how we gather ground truth from Facebook and use it to train our classification system.
Collection of reliable data
We accumulate reliable data from each source listed below into its own table. Each table is responsible for aggregating the latest observable values from that specific source. Each source has a data quality check to ensure that the observable values for each source are of high quality and contain the latest data type labels.
- Logging platform configurations: specific fields in the hive tables are filled with data that pertains to a specific type. The use and distribution of this data serves as a reliable source of trustworthy data.
- Manual labeling: developers maintaining the system, as well as external labelers, are trained to label the columns. This generally works well for all types of data in the repository and can be a primary source of reliability for some unstructured data, such as message data or user content.
- Columns from parent tables can be marked or annotated as containing specific data, and we can track this data in downstream tables.
- Execution stream sampling: execution streams in Facebook carry data of a specific type. By using our scanner as a service architecture, we can sample streams that have known data types and pass them through the system. The system promises not to store this data.
- Sampling tables: large hive tables that are known to contain the entire data corpus can also be used as training data and passed through the scanner as a service. This is excellent for tables with a full range of data types, so sampling a column randomly is equivalent to sampling the entire set of that data type.
- Synthetic data: we can even use libraries that generate data on the fly. This works well for simple, publicly available data types like addresses or GPS.
- Data stewards: privacy programs typically employ data stewards to manually associate policies with data segments. This serves as a highly accurate source of reliability.
We combine every main source of reliable data into one corpus encompassing all this information. The biggest challenge with reliability is ensuring that it is representative of the data repository. Otherwise, classification engines may overfit. To combat this, all the aforementioned sources are utilized to ensure balance in training models or calculating metrics. Additionally, human labelers evenly sample various columns in the repository and label the data accordingly to keep the collection of reliable values unbiased.
Continuous integration
To ensure rapid iteration and improvement, it is crucial to always measure system performance in real-time. We can measure each classification improvement against the system today, allowing us to strategically focus on data for further enhancements. Here, we will examine how the system completes the feedback loop provided by reliable data.
When the planning system encounters an asset that is tagged by a reliable source, we plan two tasks. The first utilizes our production scanner and, thus, our production capabilities. The second task employs the latest assembly scanner with the most recent features. Each task writes its output to its own table, marking versions alongside the classification results.
This is how we compare the classification results of the release candidate and the production model in real-time.
While the datasets compare the features of RC and PROD, a multitude of variations of the ML classification engine of the forecasting service is logged. The latest built machine learning model, the current production model, and any experimental models. The same approach allows us to 'slice' different versions of the model (agnostic to our rule classifiers) and compare metrics in real time. This makes it easy to determine when an ML experiment is ready for production deployment.
Every night, RC features calculated for that day are sent to the ML training pipeline, where the model trains on the latest RC features and assesses its performance against a reliable dataset.
Each morning, the model completes training and is automatically published as experimental. It is automatically included in the experimental list.
Some results
Over 100 different types of data are labeled with high accuracy. Well-structured types, such as emails and phone numbers, are classified with an F2 score of over 0.95. Unstructured data types, such as user content and names, also perform very well, achieving F2 scores above 0.85.
A large number of individual columns of stable and unstable data are classified daily across all repositories. More than 500 terabytes are scanned daily in over 10 data stores. The coverage of most of these stores exceeds 98%.
Over time, classification has become very efficient, as classification tasks in the saved offline stream take an average of 35 seconds from scanning the asset to computing forecasts for each column.
Fig. 2. A diagram describing the continuous integration flow to understand how RC objects are generated and sent to the model.
Figure 3. A high-level diagram of the machine learning component.
Machine Learning System Component
In the previous section, we delved deeply into the architecture of the entire system, highlighting scale, optimization, and data flows in both offline and online modes. In this section, we will examine the forecasting service and describe the machine learning system that powers the forecasting service.
With over 100 data types and some unstructured content such as message data and user content, relying solely on manual heuristics leads to subpar classification accuracy, especially for unstructured data. For this reason, we have also developed a machine learning system to address the complexities of unstructured data. Utilizing machine learning allows us to start moving away from manual heuristics and work with features and additional data signals (e.g., column names, data provenance) to increase accuracy.
The implemented model learns vector representations [3] over dense and sparse objects separately. They are then combined to form a vector that passes through a series of batch normalization [4] stages and nonlinearities to produce the final output. The final result is a floating-point number between [0-1] for each label, indicating the probability that the example belongs to a given sensitivity type. Employing PyTorch for the model has allowed us to move faster, enabling developers outside the team to quickly make and test changes.
When designing the architecture, it was crucial to model sparse (e.g., text) and dense (e.g., numerical) objects separately due to their inherent differences. For the final architecture, performing hyperparameter tuning to find the optimal values for learning rate, batch size, and other hyperparameters was also important. Choosing the optimizer was another critical hyperparameter. We found that the popular optimizer Adamoften leads to overfitting, whereas the model with SGD more stable. There were additional nuances that we needed to incorporate directly into the model. For example, static rules that ensured the model makes a deterministic prediction when a feature has a certain value. These static rules are defined by our clients. We found that integrating them directly into the model led to the creation of a more self-sufficient and reliable architecture, as opposed to implementing a post-processing stage to handle these specific boundary cases. Also, note that during training, these rules are disabled so as not to interfere with the gradient descent training process.
Issues
One of the challenges was collecting high-quality, reliable data. The model requires reliability for each class so that it can learn associations between objects and labels. In the previous section, we discussed methods for data collection for both system measurement and model training. Analysis showed that data classes like credit card numbers and bank account numbers are not very common in our storage. This complicates the collection of large volumes of reliable data for training models. To address this issue, we developed processes for generating synthetic reliable data for these classes. We generate such data for sensitive types, including SSN, credit card numbers and IBAN-numbers for which the model could not predict previously. This approach allows for the processing of confidential types of data without the privacy risks associated with concealing actual confidential data.
In addition to the challenges of reliable data, there are open architectural issues that we are working on, such as change isolation and early stopping. Change isolation is important so that when various modifications are made to different parts of the network, the impact is isolated to specific classes and does not broadly affect overall predictive performance. Improving early stopping criteria is also crucial so that we can halt the training process at a stable point for all classes, rather than at a point where some classes are overfitting while others are not.
Importance of the feature
When a new feature is introduced into the model, we want to know its overall impact on the model. We also want to ensure that predictions are interpretable by humans so that we can accurately understand which features are used for each type of data. To achieve this, we have developed and implemented class-level feature importance for the PyTorch model. Note that this differs from overall feature importance, which is usually maintained because it does not tell us which features are important for a specific class. We measure the importance of a feature by calculating the increase in prediction error after shuffling the feature. A feature is considered "important" when shuffling its values increases the model's error because, in that case, the model relied on the feature for its predictions. A feature is "not important" when shuffling its values leaves the model's error unchanged because, in this situation, the model ignored it [5].
Feature importance for each class makes the model interpretable so that we can see what the model is focusing on when predicting a label. For example, when we analyze ADDR, we ensure that the feature related to the address, such as AddressLinesCount, ranks highly in the feature importance table for each class so that our human intuition aligns well with what the model has learned.
Rating
It is important to define a single success metric. We chose F2 — a balance between recall and precision (with a slight bias towards recall). Recall is more important for the privacy use case than precision, as it is crucial for the team not to miss any sensitive data (while still ensuring reasonable accuracy). The actual F2 performance evaluation data of our model goes beyond the scope of this article. However, with careful tuning, we can achieve a high (0.9+) F2 score for the most critical sensitive classes.
Related work
There are many algorithms for the automatic classification of unstructured documents using various methods, such as pattern matching, document similarity search, and various machine learning techniques (Bayesian methods, decision trees, k-nearest neighbors, and many others) [6]. Any of these can be used as part of the classification. However, the issue lies in scalability. The approach to classification in this article leans towards flexibility and performance. This allows us to support new classes in the future while maintaining low latency.
There is also a wealth of work on fingerprinting data. For example, the authors in [7] described a solution that focuses on the issue of capturing leaks of confidential data. The main assumption is the feasibility of fingerprinting data to match it with a set of known confidential data. The authors in [8] describe a similar privacy leakage issue, but their solution is based on a specific Android architecture and is classified only when user actions result in sending personal information or if there is a leak of user data in the base application. The situation here is somewhat different, as user data can also be highly unstructured. Therefore, we need a more sophisticated technique than fingerprinting.
Finally, to address the data scarcity for certain types of confidential data, we introduced synthetic data. There is a substantial body of literature on data augmentation; for instance, the authors in [9] investigated the role of noise injection during training and observed positive results in supervised learning. Our approach to privacy is different because introducing noisy data can be counterproductive, and instead, we focus on high-quality synthetic data.
Conclusion
In this article, we presented a system that can classify a data fragment. This enables us to create systems for ensuring compliance with privacy and security policies. We demonstrated that scalable infrastructure, continuous integration, machine learning, and high-quality data on data accuracy play a key role in the success of many of our privacy initiatives.
There are many directions for future work. This may include providing support for unstructured data (files), classifying not only the type of data but also the level of sensitivity, and using self-supervised learning during training by generating accurate synthetic examples. These examples, in turn, will help the model minimize losses to the greatest extent. Future work may also focus on the investigation workflow, where we go beyond detection and provide root cause analysis of various privacy violations. This will aid in cases such as sensitivity analysis (i.e., whether the sensitivity of the type of data is high (e.g., user IP) or low (e.g., internal IP of Facebook)).
Bibliography
- David Ben-David, Tamar Domany, and Abigail Tarem. Enterprise data classification using semantic web technologies. In Peter F. Patel-Schneider, Yue Pan, Pascal Hitzler, Peter Mika, Lei Zhang, Jeff Z. Pan, Ian Horrocks, and Birte Glimm, editors, The Semantic Web – ISWC 2010, pages 66–81, Berlin, Heidelberg, 2010. Springer Berlin Heidelberg.
- Subramanian Muralidhar, Wyatt Lloyd, Sabyasachi Roy, Cory Hill, Ernest Lin, Weiwen Liu, Satadru Pan, Shiva Shankar, Viswanath Sivakumar, Linpeng Tang, and Sanjeev Kumar. f4: Facebook’s warm BLOB storage system. In 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 14), pages 383–398, Broomfield, CO, October 2014. USENIX Association.
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 26, pages 3111–3119. Curran Associates, Inc., 2013.
- Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Francis Bach and David Blei, editors, Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pages 448–456, Lille, France, 07–09 Jul 2015. PMLR.
- Leo Breiman. Random forests. Mach. Learn., 45(1):5–32, October 2001.
- Thair Nu Phyu. Survey of classification techniques in data mining.
- X. Shu, D. Yao, and E. Bertino. Privacy-preserving detection of sensitive data exposure. IEEE Transactions on Information Forensics and Security, 10(5):1092–1103, 2015.
- Zhemin Yang, Min Yang, Yuan Zhang, Guofei Gu, Peng Ning, and Xiaoyang Wang. Appintent: Analyzing sensitive data transmission in Android for privacy leakage detection. Pages 1043–1054, 11 2013.
- Qizhe Xie, Zihang Dai, Eduard H. Hovy, Minh-Thang Luong, and Quoc V. Le. Unsupervised data augmentation.
Learn how to acquire an in-demand profession from scratch or Level Up your skills and salary by taking online courses at SkillFactory:
- (12 months)
- (12 weeks)
- (20 weeks)
- (20 weeks)
More Courses
- (9 months)
- (8 months)
- (9 months)
- (12 months)
- (18 months)
- (12 months)
- (9 months)
- (7 months)
Source: habr.com
