I-Habrastatistics: ukuhlaziya imibono yabafundi

Sawubona Habr. IN ingxenye edlule Ukuthandwa kwezigaba ezihlukahlukene zesayithi kwahlaziywa, futhi ngesikhathi esifanayo kwaphakama umbuzo - iyiphi idatha engakhishwa kumazwana ezihlokweni. Bengifuna nokuhlola i-hypothesis eyodwa, engizoxoxa ngayo ngezansi.


Idatha ibonakale ijabulisa kakhulu, futhi sakwazi nokuhlanganisa "isilinganiso esincane" sabahlaziyi. Iqhubeke ngaphansi kokusikwa.

Ukuqoqwa kwedatha

Ukuhlaziya, sizosebenzisa idatha yalo nyaka, 2019, ikakhulukazi njengoba sengilutholile kakade uhlu lwama-athikili ngefomu le-csv. Okusele nje ukukhipha amazwana esihlokweni ngasinye ngenhlanhla, agcinwa lapho, futhi akukho zicelo ezengeziwe okudingeka zenziwe.

Ukugqamisa amazwana avela esihlokweni, ikhodi elandelayo yanele:

r = requests.get("https://habr.com/ru/post/467453/")
data_html = r.text
comments = data_html.split('<div class="comment" id=')

comments_list = []
for comment in comments:
    body = Str(comment).find_between('<div class="comment__message', '<div class="comment__footer"').find_between('>', '</div>')# .replace('n', '-')
    if len(body) < 4: continue

    body = body.translate(str.maketrans(dict.fromkeys("tnrvf")))
    body = body.replace('"', "'").replace(',', " ").replace('<br>', ' ').replace('<p>', '').replace('</p>', '').replace('  ', ' ')

    user = Str(comment).find_between('data-user-login', '>').find_between('"', '"')
    date_str = Str(comment).find_between('<time class="comment__date-time comment__date-time_published', 'time>').find_between('>', '<')
    vote = Str(comment).find_between('<div class="voting-wjt', '</div>').find_between('<span', 'span>').find_between('>', '<')
    date = dateparser.parse(date_str)

    csv_data = "{},{},{},{}".format(user, date, vote, body)
    comments_list.append(csv_data)

Lokhu kusivumela ukuthi sithole uhlu lwamazwana abukeka kanje (iziteketiso zisusiwe ngenxa yezizathu zobumfihlo):

xxxxxxx,2019-02-06 11:50:00,0,А можно пример как именно?
xxxxxxx-02-24 16:15:00,+1,Побольше читайте независимые официальные источники чтобы таких вопросов не было.
xxxxxxx,2019-02-23 20:15:00,–5,А не важно главное в итоге в плюсе оказаться


Njengoba ubona, kumazwana ngamunye singathola igama lomsebenzisi, idethi, isilinganiso, nombhalo wangempela. Ake sibone ukuthi yini esingayithola kulokhu.

Ngendlela, ekuqaleni, umqondo wokuqoqa izilinganiso wawuhluke kancane - ukubona ukuthi abasebenzisi banikeza izilinganiso. Isibonelo, ungabheka i-YouTube - ngisho nevidiyo ekahle kakhulu, ngisho nevidiyo engenalo ulwazi oluqondile, okwereferensi noma ukukhishwa kwezindaba, isazuza inani elithile lokuncishiswa. I-hypothesis yayiwukuthi kukhona abasebenzisi abathi, ngokomtholampilo kuphela, abangathandi yonke into, mhlawumbe i-serotonin ayikhiqizwa ebuchosheni noma enye into. Mhlawumbe umuntu akasadingi ukuhlala ku-Habré, kodwa ukuphatha ukucindezeleka ... Kodwa njengoba kwavela, angikwazi ukubheka lokhu lapha, ngoba... uhlu lwalabo abanikeze izilinganiso alulondolozwanga kumazwana noma esihlokweni. Hhayi-ke, okungukuthi, sizosebenza ngedatha etholakalayo. Umphumela uba isilinganiso “sokuhlehla” - ungabona ukuthi yiziphi izilinganiso _receive_ ngabasebenzisi. Okungukuthi, ngokomthetho, nakho kuyathakazelisa.

Iyacubungula

Okokuqala, ukuphika kwendabuko. Lesi silinganiso, njengazo zonke ezedlule, asisemthethweni. Angiqinisekisi ukuthi angenzanga iphutha noma kuphi. Kulabo abathanda imininingwane yobuchwepheshe, ikhodi enemininingwane eyengeziwe inikezwa engxenyeni edlule.

Ngakho ake siqale. Amazwana alo nyaka, ka-2019 (ongakapheli), athathwe ukuze ahlaziywe. Ngesikhathi sokubhala, abasebenzisi babhale Amazwana we-448533, usayizi wefayela le-csv ngu 288MB. Inamandla, ihlaba umxhwele.

Isikhathi sokubhala

Ake siqoqe imibono ngehora, sihlukanise phakathi nesonto nezimpelasonto ngokuhlukana.

Lapha asinasithakazelo kumanani aphelele, kodwa kulawo ahlobene. Uma ubheka nje "njengoba injalo", bese kuvela lokhoоIningi lamazwana abhalwe ngesikhathi samahora okusebenza kusuka ku-10 kuya ku-18 😉 Ngakolunye uhlangothi, izindawo zesikhathi azinakwa lapha, ngakho umbuzo usavuliwe.

Ake sibheke ukusatshalaliswa kwamazwana unyaka wonke:

Futhi nokho iyazungeza; ukuhlinzwa kubonakala ngokucacile phakathi neviki - isikhathi seviki sibonakala ngokucacile, ngakho-ke singasho ngokuqiniseka okukhulu ukuthi abantu bafunda futhi babeka amazwana ngoHabr emsebenzini (kodwa lokhu akuqinisekile).

Ngendlela, kwakukhona umqondo wokuhlola i-hypothesis ukuthi inani lama-minuses noma ama-pluses atholakele lihlukile osukwini noma isikhathi sosuku, kodwa kwakungenakwenzeka ukuthola ubuhlobo - isikhathi esinikezwe isilinganiso asigcinwanga, futhi akukho ukuxhumana okuqondile nesikhathi sokuphawula.

Abasebenzisi

Yebo, angilazi inani eliqondile labasebenzisi kusayithi. Kodwa labo abashiye okungenani ukuphawula okukodwa kulo nyaka kuvele ukuthi cishe Abantu abangu-25000.

Igrafu yenani lemilayezo eshiywe ngabasebenzisi ibukeka ijabulisa kakhulu:

Ekuqaleni angizange ngikukholwe, kodwa kwakubonakala kungekho phutha. U-5% wabasebenzisi ushiya u-60% wemilayezo. I-10% - 74% yayo yonke imilayezo (okuyinto, ake ngikukhumbuze, kulo nyaka, izinkulungwane ezingama-450). Iningi livele lifunde isayithi, lishiye ukuphawula okungajwayelekile, noma lingawashiyi nhlobo (lawo, ngokwemvelo, awafakiwe ohlwini lwami).

Ukulinganisa

Asiqhubekele engxenyeni yokugcina nejabulisa kakhulu yezibalo - izilinganiso. Ngenxa yezizathu zobumfihlo, ngeke nginikeze iziteketiso eziphelele zabasebenzisi, noma ubani ofuna, ngicabanga ukuthi, uzozibona.

Ngu inani lamazwana kulo nyaka, abaphezulu aba-5 bathathwe yi-VoXXXX (amazwana angu-3377), 0xdXXXXX (amazwana angu-3286), strXXXX (amazwana angu-3043), i-AmXXXX (amazwana angu-2897) kanye ne-khXXXX (amazwana angu-2748).

Ngu inombolo yezinzuzo ezitholiwe, top 5 ithathwe amXXXX (1395 comments, ratings +3231/-309), tvXXXX (1544 comments, ratings +3231/-97), WhuXXXX (921 comments, ratings +2288/-13), MTXXXX (1328 comments, +1383 /-7) kanye namaXXXX (736 ukuphawula, isilinganiso +1340/-16).

Ngu isilinganiso esihle ngokuphelele (ekekho neyodwa i-negative rated comment) phezulu phezulu kuhlala Milfgard и I-Boomrum. Ngokuhlukile, ngethula iziteketiso zabo ngokugcwele, ngicabanga ukuthi zibafanele.

Ama-downsides nawo ayathakazelisa. Phezulu nge inani lamaminithi aqoqiwe kulo nyaka zithathwe yi-siXX (473 pluses, 699 minuses), khXX (1915 pluses, 573 minuses) kanye ne-nicXXXXX (456 pluses, 487 minuses). Kodwa njengoba ubona, laba basebenzisi banamazwana anele ngokwanele. Kodwa ngokusho ukususa ngokuphelele I-antitopic ihlanganisa i-vladXXXX (amazwana angu-55, amaminithi angu-84, ama-pluses angu-0), i-ekoXXXX (amazwana angu-77, imizuzu engu-92, 1 plus) kanye ne-iMXXXX (ukuphawula okungu-225, imizuzu engu-205, ukuhlanganisa okungu-12).

isiphetho

Angikwazanga ukubala konke okuhleliwe, kodwa ngithemba ukuthi bekuthakazelisa.

Njengoba ubona, ngisho nedathasethi enenombolo encane yezinkambu inganikeza idatha ethakazelisayo ukuze ihlaziywe. Kusekuningi okumele kugejwe, kusukela ekwakheni "ifu lamagama" kuya ekuhlaziyeni umbhalo. Uma imiphi imiphumela ethokozisayo ivela, izoshicilelwa.

Source: www.habr.com