{"id":75636,"date":"2020-03-27T07:42:42","date_gmt":"2020-03-27T05:42:42","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/strukturirovanie-riskov-i-reshenij-pri-ispolzovanii-bigdata-dlya-polucheniya-oficzialnoj-statistiki"},"modified":"2020-03-27T07:42:42","modified_gmt":"2020-03-27T05:42:42","slug":"strukturirovanie-riskov-i-reshenij-pri-ispolzovanii-bigdata-dlya-polucheniya-oficzialnoj-statistiki","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/strukturirovanie-riskov-i-reshenij-pri-ispolzovanii-bigdata-dlya-polucheniya-oficzialnoj-statistiki","title":{"rendered":"Structuring Risks and Solutions When Using Big Data for Obtaining Official Statistics","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

Translator's Foreword<\/i><\/p>\n

I was intrigued by the material, primarily because of the table below:<\/p>\n

\"Structuring
\n
\nConsidering that statisticians (and Russians, at a genetic level) do not particularly favor anything that deviates from linear relationships, these individuals managed to advocate for the use of an activation function in a parabolic form to assess the risk associated with using Big Data in official statistics. Well done. Naturally, statisticians have added their note to this work \u2013 '1 Any errors and omissions are the sole responsibility of the authors. The opinions expressed in this document are personal and do not necessarily reflect the official position of the European Commission.' However, the work was published. I believe that is sufficient for today, and they (the authors) did not prohibit anyone from finding their scales in these aspects.<\/p>\n

The work is structured enough to distinguish where and how statistical methods differ from research methods for Big Data. In my opinion, the greatest benefit of this work will be in discussions with clients and for countering their statements like:<\/p>\n

\u2014 We collect statistics ourselves; what more do you wish to investigate?
\n \u2014 Present your results in such a way that we can reconcile them with our statistics. In this matter, the authors suggest that it would be worthwhile to read this work (3 How big is Big Data? Exploring the role of Big Data in Official Statistics<\/a><\/noindex>)<\/p>\n

In this work, the authors express their view on the level of risk. This parameter is placed in parentheses, not to be confused with a reference to sources.<\/p>\n

Second observation. The authors use the term BDS \u2013 this is analogous to the concept of Big Data. (presumably a nod to official statistics).
\n<\/a><\/noindex>
\nAuthors' Foreword<\/i><\/p>\n

An increasing number of statistical agencies are exploring the possibility of using large data sources for the preparation of official statistics. Currently, there are only a few examples where these sources have been fully integrated into actual statistical production. Consequently, the full extent of the implications brought about by their integration remains unclear. In the meantime, initial attempts have been made to analyze the conditions and impact of big data on various aspects of statistical production, such as quality or methodology. Recently, a target group has developed a qualitative framework for the preparation of statistical data based on big data in the context of the Economic Commission for Europe\u2019s big data project of the United Nations (UNECE). According to the European Statistical Code of Practice, providing high-quality statistical information is a primary objective of statistical agencies. Given that risk is defined as the impact of uncertainty on objectives (for example, by the International Organization for Standardization ISO 31000), we found it appropriate to categorize risks according to the dimensions of quality they affect.
\nThe proposed quality framework for statistical data derived from large data sources offers a structured representation of quality associated with all stages of the statistical business process, and thus can serve as the basis for comprehensive assessment and management of risks related to these new data sources. It introduces new quality dimensions that are specific to or highly relevant when using big data for official statistics, such as the institutional\/business environment or complexity. By employing these new quality dimensions, risks associated with the use of large data sources in official statistics can be identified more systematically.<\/p>\n

In this work, we aim to identify the risks posed by the use of big data in the context of official statistics. We adopt a systematic approach to defining risks within the proposed quality framework. Focusing on newly proposed quality metrics, we can describe risks that are currently absent or not impacting the production of official statistics. At the same time, we can identify existing risks that will be assessed quite differently when using big data for statistical purposes. We then move further in the risk management cycle and provide an assessment of the likelihood and impact of these risks. Since risk assessment entails subjectivity in attributing probability and impact to various risks, we measure the agreement among dozens of different stakeholders provided independently. We then offer risk mitigation options in accordance with four main categories: avoidance, reduction, sharing, and retention. According to ISO, one of the principles of risk management should be value creation, meaning that the resources for risk reduction should be less than those for inaction. In line with this principle, we will finally evaluate the potential impact of some risk mitigation measures on the quality of the final outcomes, aiming for a more comprehensive assessment of the use of Big Data for official statistics.<\/p>\n

1. Introduction<\/h2>\n

<\/p>\n

1.1. Background<\/h4>\n

\nThe development of big data was characterized by Kenneth Neil Cukier and Viktor Mayer-Sch\u00f6nberger in their article \"The Growth of Big Data\" (2. www.foreignaffairs.com\/articles\/139104\/kenneth-neil-cukier-and-viktor-mayer-schoenberger\/the-rise-of-big-data<\/a><\/noindex>) the term \"datafication.\" Datafication is described as the process of taking all aspects of life and turning them into data. For example, Facebook provides personal networks, sensors for all kinds of environmental conditions, smartphones for personal communication and movements, and wearable data for personal conditions. This leads to almost ubiquitous data collection and availability.<\/p>\n

As in many other sectors, official statistics have only recently begun to discuss the issue of big data at a strategic level. There is still no common and widely accepted understanding of the way forward, whether it is a challenge or an opportunity, whether it is small or large, etc. Within the High-Level Group on Modernization of Statistical Production and Services (3 How big is Big Data? Exploring the role of Big Data in Official Statistics: www1.unece.org\/stat\/platform\/download\/attachments\/99484307\/VirtualSprintBigDatapaper.docx?version=1&modificationDate=1395217470975&api=v2<\/a><\/noindex>), a First SWOT analysis, accompanied by a rough risk\/benefit analysis, was conducted. It was noted that \"a full risk analysis would also include aspects such as likelihood and impact, and may also be expanded to identify risk mitigation and management strategies.\"<\/p>\n

Although this document is far from a complete risk analysis, it aims to improve the situation precisely by creating the first structured overview. We would like to emphasize that this overview should be seen as a starting point for stimulating general discussion within the Official Statistical Community (OSC).<\/p>\n

1.2. Scope<\/h4>\n

\nThis article is dedicated solely to risks, excluding not only benefits but also strengths and weaknesses, opportunities, and threats. This means that \"risks of inaction\" (for example, the risk that the OSC will become uncompetitive with other players if it is not modernized) do not fall within the scope; these are rather threats. Instead, we try to highlight the risks that may arise (a) if the OSC takes advantage of the opportunities provided by big data and begins to develop or improve a specific \"Big Data-based Official Statistics Product\" (BOSP); (b) risks to the new \"business as usual,\" namely risks for official statistics based on \"big data\" production. (Since all production of official statistics involves risks, we limit ourselves to (b) risks specific to \"big data risks,\" i.e., risks that do not exist or are minimal for the \"traditional\" process of collecting official statistics.)<\/p>\n

1.3. Structure<\/h4>\n

\nIn Section 2, we present the fundamental principles related to this task, starting with the clearly necessary foundation for risk management and risk governance (Section 2.1). We also introduce a preliminary quality structure for statistical data derived from big data (Section 2.2), as linking the quality structure with risks serves two purposes:<\/p>\n