{"id":78146,"date":"2020-04-17T01:42:21","date_gmt":"2020-04-16T23:42:21","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/divizion-dannyh-2013-god-retrospektiva"},"modified":"2020-04-17T01:42:21","modified_gmt":"2020-04-16T23:42:21","slug":"divizion-dannyh-2013-god-retrospektiva","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/divizion-dannyh-2013-god-retrospektiva","title":{"rendered":"Data Division. 2013. Retrospective","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

In 2013 IBS<\/b>, which back then seemed to be creating Data Division<\/b>, asked me to do a brain dump (exclusively based on my experience with corporate oil and gas clients) regarding the problem area of Big Data and Data in general. I stumbled upon it after 7 years and found it amusing. Some things are obvious. Some turned out to be not entirely accurate, but... 7 years have passed. <\/p>\n

I wrote it in English and thought of translating it into Russian. Maybe something is still relevant today? (I will translate the bullet points, but I'll leave the tables in English out of laziness. Green means good, red means dangerous, blue means a dream). <\/p>\n

I will add minimal comments from 'today' italic<\/i>, to make it understandable and distinguishable.<\/p>\n

So, DATA! We need data...
\n<\/a><\/noindex>
\nThe Data Division is like the Blood Division because data can be compared to blood flowing through the veins and arteries of a business organism. However, while the blood is the same, the organisms are different, and thus productization<\/b> is very challenging, but it also represents an opportunity for development. <\/p>\n

There are people who notice the data right in front of them - these are We<\/b>.
\nAnd there are people who, unfortunately, do not see the data at all. This, again, unfortunately, includes our Clients<\/b>!<\/p>\n

\"Data
\n
\nSo, business axioms...<\/b><\/p>\n

    \n
  1. We sell to businesses<\/b>, not IT<\/b> (may all IT specialists forgive me right away) as we solve global issues, and also aim for more money.<\/li>\n
  2. All business problems concentrate around thematic industry verticals and require adequate specialization<\/b>.<\/li>\n
  3. Attempts to prove the value of 'data'<\/b> or, even more difficult, the value of 'data management' to businesses \u2013 this is eternal suffering and pain. Essentially, it's like going to someone who feels pretty well and saying: 'Dude, we are going to treat your blood, and, dude, it's going to be expensive!'<\/li>\n
  4. My 'wet dream' is to sell 'data extraction' and 'analytics' within a SaaS model to small and medium businesses<\/b>, which have dived into 123 cloud services with cool interfaces: project management, helpdesk, accounting, CRM, payroll, time reporting, marketing, ... you name it, and have buried themselves in data. Youcalc and Successfactors (there probably aren\u2019t such anymore<\/i>) are good!<\/li>\n
  5. Look for people who love to tinker with crunch <\/b>data. They are rare and strange (like coffee grounds readers), but key to business. A poet, for example, may be quite good at understanding correlation.<\/li>\n
  6. Engineers<\/b> are needed! They are necessary to turn the problems that Crunchers extract from data into solutions. The success or failure of the solution entirely depends on them. <\/li>\n
  7. Development opensource<\/b> projects represent enormous value and allow for the 'construction' of complex solutions practically 'from scratch'.<\/li>\n
  8. But\u2026 one must not forget that Hadoop is a library, and Lucene is also a library, and the gap between a library and an industrial product<\/b> is significant!<\/li>\n
  9. The solutions built will need to be substantially adapted, therefore modularity<\/b> and integrability<\/b> are key aspects. <\/li>\n
  10. Agile (God help us<\/i>) is a key technique in interaction with the client and hypothesis testing , of which there will be many.<\/b>Outsourcing any coding and UI is especially possible and necessary. All business analytics and specifications <\/li>\n
  11. of the backend should be retained<\/b> and regarded as a core competency. inside<\/b> Decision-makers in business must be constantly 'informed' about <\/li>\n
  12. the necessity of working correctly with data and the ongoing search for new ways to analyze it. The combination of technical and business competencies of our employees will help elevate the status of the entire organization as a whole. <\/b>\u2014 is an endless source of inspiration ( <\/li>\n
  13. Internet<\/b> there weren't so many kittens back then) regarding approaches to corporate data management despite the fact that tasks and scales differ significantly.<\/i>Technological postulates\u2026 <\/li>\n<\/ol>\n

    \"Data
    \n
    \nThere is enormous developmental potential in<\/b><\/p>\n

      \n
    1. simplifying how data is presented to people. One could call it 'iPhonization'.<\/b> Despite BI vendors claiming they directly<\/li>\n
    2. bring analytics to end-users , (and they are certainly moving in this direction) \u2013 a breakthrough has still not occurred. People simply do not understand<\/b>multidimensional data well.<\/b> A user interface presenting more or less complex loosely structured data in<\/li>\n
    3. a faceted format \u2013 also presents an endless number of problems. Conclusion: the flatter it is \u2013 the better.<\/b> A platform built on automatic data extraction from sources (which are not always intended for such extraction) is heavily dependent on the sources, the stability of connectors, and infrastructure. Any failure to deliver results will always be blamed on the platform (the messenger). <\/li>\n
    4. The platform built on automated data extraction from sources (which are not always intended for such extraction) is significantly dependent on these sources, the stability of the connectors, and the infrastructure. Any failure to deliver results will always lead to blame being placed on the platform (the messenger). Trust<\/b> \u2013 capital of this kind of platforms. Capital that is difficult to earn and easy to lose. <\/li>\n
    5. From a business perspective, there's no difference between analyzing Big Data and Just Data<\/b>. Often behind simple calculations like 2x2 lies the potential for millions of dollars. A good example is the data on the lifespan of infrastructure elements on the Norwegian shelf. When all the future maintenance dates for all equipment were plotted on a single timeline and it was discovered that in N years a veritable shelf Armageddon was approaching \u2014 one very wealthy individual got up from his seat, politely excused himself, and left the room saying, \"Excuse me, I don\u2019t have much time, I need to prepare a fleet...\"<\/li>\n
    6. Excel, and essentially a clear and concise tabular representation of data has enormous power and a great future. I believe in beautiful tables<\/b> (and still do<\/i>) and that's it!<\/li>\n
    7. The main bow of all this \"analytics\" is decision-making automation<\/b>. That's where the juiciest opportunities lie, but also the highest risks, because big opportunities come with big risks, hence the opportunities and the risks... \ud83d\ude42 Managing well drilling, for example...<\/li>\n
    8. If \"integrability\" is the key feature, then the data should de facto be represented as a service. REST<\/b> It's great, but one must not forget about optimization performance.<\/b>, which is often sacrificed for integrability, as computational power continues to grow. <\/li>\n
    9. Master data<\/b> is what needs to be localized, extracted, standardized, before addressing any business issues. Master data is small, but the problems with it are big! As the semantics brothers say \u2013 50% of all the world's problems stem from people calling the same things by different names, and the other 50% come from them calling different things the same name.<\/li>\n
    10. Any encapsulation<\/b> at the storage level limits the openness of the solution and leads to SILO-ification. It's fine if you're a large vendor, otherwise it\u2019s not so great. (Here, we are certainly not talking about the block level or AWS S3, which had already been around for 6 years back then, but about files<\/i>). <\/li>\n
    11. Relational modeling<\/b> is no longer our friend. RDF and key-value are cool! We have seen magical transformations of relational databases with models from 2000 tables to 15 tables, and none of the users lost anything. <\/li>\n
    12. The Internet works because there is URL<\/b> as the unified means of addressing. The importance of URLs, or rather URI<\/b> for an enterprise's informational resources is difficult to overestimate. <\/li>\n
    13. Text mining and NLP are popular<\/b>. On the Internet. But even in the corporate sector, significant success can be achieved by extracting structured data from unstructured corporate data. <\/li>\n
    14. Synergy<\/b> between structured data and information extracted from unstructured data, i.e. files \u2013 an analytical goldmine. <\/li>\n
    15. When extracting data, don't forget about rights and copyrights.<\/b>. <\/li>\n
    16. A data extraction company should establish adepartment of hackers<\/b>, in the good sense of the word. Inspired by the tough battle against the Yellow Pages' security systems from search bots. <\/li>\n
    17. Before working with data, it is necessary to \"see\"<\/b> it in its entirety. This is hard to explain. I think of table forms. Some might think of graphical representations, but any graph is already an interpretation. One way or another\u2026 \"see\"!<\/li>\n
    18. Reiterating the issue of user \"trust\" in the frontend. Trust in the connectors\/data generation processes, trust in the data, trust in the decisions made.<\/b>.<\/li>\n<\/ol>\n

      Source: habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

      \u0412 2013 \u0433\u043e\u0434\u0443 IBS, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u0442\u043e\u0433\u0434\u0430, \u043a\u0430\u0436\u0435\u0442\u0441\u044f, \u0441\u043e\u0437\u0434\u0430\u0432\u0430\u043b\u0438 \u0414\u0438\u0432\u0438\u0437\u0438\u043e\u043d \u0434\u0430\u043d\u043d\u044b\u0445, \u043f\u043e\u043f\u0440\u043e\u0441\u0438\u043b\u0438 \u043c\u0435\u043d\u044f \u0441\u0434\u0435\u043b\u0430\u0442\u044c \u0442\u0430\u043a\u043e\u0439 \u0431\u0440\u0435\u0439\u043d\u0434\u0430\u043c\u043f (\u0438\u0441\u043a\u043b\u044e\u0447\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u043d\u0430 \u0431\u0430\u0437\u0435 \u043e\u043f\u044b\u0442\u0430 \u0432\u0437\u0430\u0438\u043c\u043e\u0434\u0435\u0439\u0441\u0442\u0432\u0438\u044f \u0441 \u043a\u043e\u0440\u043f\u043e\u0440\u0430\u0442\u0438\u0432\u043d\u044b\u043c\u0438 \u043d\u0435\u0444\u0442\u0435\u0433\u0430\u0437\u043e\u0432\u044b\u043c\u0438 \u0437\u0430\u043a\u0430\u0437\u0447\u0438\u043a\u0430\u043c\u0438) \u043f\u043e \u043f\u043e\u0432\u043e\u0434\u0443 \u043f\u0440\u043e\u0431\u043b\u0435\u043c\u043d\u043e\u0439 \u043e\u0431\u043b\u0430\u0441\u0442\u0438 \u0411\u043e\u043b\u044c\u0448\u0438\u0445 \u0414\u0430\u043d\u043d\u044b\u0445, \u0434\u0430 \u0438 \u0414\u0430\u043d\u043d\u044b\u0445 \u0432\u043e\u043e\u0431\u0449\u0435. \u0412\u043e\u0442 \u044f \u043d\u0430\u0442\u043a\u043d\u0443\u043b\u0441\u044f \u043d\u0430 \u043d\u0435\u0433\u043e \u0441\u043f\u0443\u0441\u0442\u044f 7 \u043b\u0435\u0442 \u0438 \u043f\u043e\u043a\u0430\u0437\u0430\u043b\u043e\u0441\u044c \u0437\u0430\u0431\u0430\u0432\u043d\u043e. \u041d\u0435\u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u0432\u0435\u0449\u0438 \u043e\u0447\u0435\u0432\u0438\u0434\u043d\u044b. \u041d\u0435\u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u043d\u0435 \u0441\u043e\u0432\u0441\u0435\u043c \u0432\u0435\u0440\u043d\u044b \u043e\u043a\u0430\u0437\u0430\u043b\u0438\u0441\u044c, \u043d\u043e\u2026 7 […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":78147,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-78146","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47Data Division. 2013. Retrospective | ProHoster","description":"In 2013, IBS, which at that time seemed to be creating the Data Division, asked me to do a brain dump (solely based on my experience interacting with corporate oil and gas clients) regarding.","canonical_url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/divizion-dannyh-2013-god-retrospektiva","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"en_US","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0414\u0438\u0432\u0438\u0437\u0438\u043e\u043d \u0434\u0430\u043d\u043d\u044b\u0445. 2013 \u0433\u043e\u0434. \u0420\u0435\u0442\u0440\u043e\u0441\u043f\u0435\u043a\u0442\u0438\u0432\u0430 | ProHoster","og:description":"\u0412 2013 \u0433\u043e\u0434\u0443 IBS, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u0442\u043e\u0433\u0434\u0430, \u043a\u0430\u0436\u0435\u0442\u0441\u044f, \u0441\u043e\u0437\u0434\u0430\u0432\u0430\u043b\u0438 \u0414\u0438\u0432\u0438\u0437\u0438\u043e\u043d \u0434\u0430\u043d\u043d\u044b\u0445, \u043f\u043e\u043f\u0440\u043e\u0441\u0438\u043b\u0438 \u043c\u0435\u043d\u044f \u0441\u0434\u0435\u043b\u0430\u0442\u044c \u0442\u0430\u043a\u043e\u0439 \u0431\u0440\u0435\u0439\u043d\u0434\u0430\u043c\u043f (\u0438\u0441\u043a\u043b\u044e\u0447\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u043d\u0430 \u0431\u0430\u0437\u0435 \u043e\u043f\u044b\u0442\u0430 \u0432\u0437\u0430\u0438\u043c\u043e\u0434\u0435\u0439\u0441\u0442\u0432\u0438\u044f \u0441 \u043a\u043e\u0440\u043f\u043e\u0440\u0430\u0442\u0438\u0432\u043d\u044b\u043c\u0438 \u043d\u0435\u0444\u0442\u0435\u0433\u0430\u0437\u043e\u0432\u044b\u043c\u0438 \u0437\u0430\u043a\u0430\u0437\u0447\u0438\u043a\u0430\u043c\u0438) \u043f\u043e \u043f\u043e\u0432\u043e\u0434\u0443.","og:url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/divizion-dannyh-2013-god-retrospektiva","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-04-16T23:42:21+00:00","article:modified_time":"2020-04-16T23:42:21+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"78146","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 17:04:36","updated":"2022-09-30 17:30:05","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/78146","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/comments?post=78146"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/78146\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media\/78147"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media?parent=78146"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/categories?post=78146"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/tags?post=78146"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}