Sevhisi yega yega iyo vashandisi vanogona kugadzira yavo yemukati (UGC - Mushandisi-yakagadzirwa zvemukati) inomanikidzwa kwete chete kugadzirisa matambudziko ebhizinesi, asiwo kuisa zvinhu zvakarongeka muGC. Yakashata kana yakaderera-yemhando yemukati modhiyo inogona pakupedzisira kuderedza kukwezva kwesevhisi kune vashandisi, kunyange kugumisa kushanda kwayo.
Nhasi tichakuudza nezve synergy pakati peYula neOdnoklassniki, iyo inotibatsira zvine mwero kushambadzira kuYula.
Synergy kazhinji chinhu chinobatsira, uye munyika yanhasi, kana tekinoroji nemaitiro zvichichinja nekukurumidza, zvinogona kushanduka kuita muponesi. Sei uchitambisa zviwanikwa zvishoma uye nguva kugadzira chimwe chinhu chakatogadzirwa uye chakaunzwa mupfungwa pamberi pako?
Isu takafunga chinhu chimwe chete patakanga takatarisana nebasa rakazara rekuenzanisa zvemukati zvemushandisi - mapikicha, zvinyorwa uye zvinongedzo. Vashandisi vedu vanorodha mamirioni ezvimedu zvemukati kuYula mazuva ese, uye pasina otomatiki kugadzirisa hazvigoneke zvachose kuyera iyi data nemawoko.
Naizvozvo, isu takashandisa yakagadzirira-yakagadzirwa moderation chikuva, iyo panguva iyoyo vatinoshanda navo kubva kuOdnoklassniki vakanga vapedza kusvika kune "inenge yakakwana."
Sei Odnoklassniki?
Mazuva ese, makumi emamiriyoni evashandisi anouya kusocial network uye anoburitsa mabhiriyoni ezvimedu zvemukati: kubva mafoto kuenda kumavhidhiyo uye zvinyorwa. Iyo Odnoklassniki moderation chikuva inobatsira kutarisa yakakura kwazvo mavhoriyamu e data uye kurwisa spammers uye bots.
Chikwata che OK moderation chakaunganidza ruzivo rwakawanda, sezvo yanga ichivandudza chishandiso chayo kwemakore gumi nemaviri. Izvo zvakakosha kuti ivo vaisangogona kugovera avo akagadzirira-akagadziriswa mhinduro, asi zvakare kugadzirisa mavakirwo epuratifomu yavo kuti ienderane nemabasa edu chaiwo.
Kubva zvino zvichienda mberi, kupfupika, isu tichangodaidza iyo OK moderation chikuva "puratifomu."
Kuti zvese zvinoshanda sei
Kutsinhana kwedata pakati peYula neOdnoklassniki kunotangwa kuburikidza .
Nei takasarudza chishandiso ichi:
- MuYula, kushambadza kwese kunoiswa-moderated, saka pakutanga mhinduro inowirirana yaisadikanwa.
- Kana ndima yakaipa ikaitika uye Yula kana Odnoklassniki isingawanikwe, kusanganisira nekuda kweimwe mitoro yepamusoro, ipapo data kubva kuKafka haizonyangarika chero kupi uye inogona kuverengwa gare gare.
- Nzvimbo yacho yakanga yatobatanidzwa neKafka, saka nyaya dzakawanda dzekuchengetedza dzakagadziriswa.
Pachishambadzo chega chega chakagadzirwa kana kugadziridzwa nemushandisi muYula, JSON ine data inogadzirwa, inoiswa muKafka kuti iongororwe inotevera. Kubva kuKafka, zviziviso zvinoiswa papuratifomu, kwavanotongwa otomatiki kana nemaoko. Kushambadzira kwakashata kwakavharwa nechikonzero, uye izvo izvo papuratifomu isingawani kutyorwa kunoiswa chiratidzo se "chakanaka." Zvadaro zvisarudzo zvose zvinodzorerwa kuYula uye kushandiswa mubasa.
Pakupedzisira, kuna Yula zvose zvinosvika kune zviito zviri nyore: tumira chiziviso kune Odnoklassniki platform uye udzorere chisarudzo "ok", kana kuti sei "ok".
Automatic processing
Chii chinoitika kushambadzo kana yasvika pachikuva? Imwe neimwe ad yakakamurwa kuita akati wandei:
- Zita,
- tsananguro,
- mapikicha,
- mushandisi-akasarudzwa chikamu uye chikamu chidiki che ad,
- mutengo.
Chikuva chinobva chaita kubatanidza kune chimwe nechimwe kuti chiwane zvakapetwa. Uyezve, zvinyorwa uye mapikicha zvakabatanidzwa maererano nehurongwa hwakasiyana.
Asati abatanidza, zvinyorwa zvinogadziriswa kuti zvibvise mavara akakosha, mavara akachinjwa uye mamwe marara. Iyo data yakagamuchirwa yakakamurwa kuva N-grams, imwe neimwe iyo yakakurumidza. Mhedzisiro yacho yakawanda yakasarudzika hashes. Kufanana pakati pezvinyorwa kunotsanangurwa ne pakati pezviviri zvinobuda seti. Kana kufanana kwakakura kudarika chikumbaridzo, ipapo zvinyorwa zvinobatanidzwa kuva sumbu rimwe. Kuti ikurumidze kutsvaga masumbu akafanana, MinHash uye Locality-sensitive hashing inoshandiswa.
Yakasiyana-siyana sarudzo yegluing mifananidzo yakagadzirirwa mafoto, kubva pakuenzanisa pHash mapikicha nekutsvaga makopi uchishandisa neural network.
Nzira yekupedzisira ndiyo yakanyanya "yakaoma". Kudzidzisa modhi, katatu emifananidzo (N, A, P) akasarudzwa umo N isina kufanana neA, uye P yakafanana neA (iri semi-duplicate). Ipapo iyo neural network yakadzidza kuita A uye P padhuze sezvinobvira, uye A neN kusvika pazvinogoneka. Izvi zvinoguma nemaitiro mashoma emanyepo zvichienzaniswa nekungotora embeddings kubva kune pre-yakadzidziswa network.
Kana iyo neural network inogamuchira mifananidzo sekuisa, inogadzira N (128) -dimensional vector kune yega yega uye chikumbiro chinoitwa kuongorora kuswedera kwemufananidzo. Zvadaro, chikumbaridzo chinoverengerwa apo mifananidzo yepedyo inoonekwa seyakafanana.
Iyo modhi inokwanisa nehunyanzvi kutsvaga spammers vanonyatso fotocha chigadzirwa chimwe chete kubva kumakona akasiyana kuitira kuti vapfuure pHash kuenzanisa.
Muenzaniso wemapikicha espam akanamirwa pamwechete neneural network seanodzokorora.
Padanho rekupedzisira, zviziviso zvakapetwa zvinotsvakwa panguva imwe chete nemavara uye mufananidzo.
Kana zviziviso zviviri kana kupfuura zvakanamatirwa pamwechete musumbu, sisitimu inotanga kuvharika otomatiki, iyo, uchishandisa mamwe maalgorithms, inosarudza kuti ndedzipi dzakapetwa dzekudzima uye dzekusiya. Semuenzaniso, kana vashandisi vaviri vaine mapikicha akafanana mune ad, sisitimu inovhara iyo ichangoburwa ad.
Kana zvangogadzirwa, masumbu ese anoenda kuburikidza neakatevedzana mafirita otomatiki. Sefa yega yega inopa zvibodzwa kune cluster: zvingaite sei kuti rine kutyisidzira kunoratidzwa nesefa iyi.
Semuenzaniso, sisitimu inoongorora tsananguro mune ad uye inosarudza zvingangove zvikamu zvayo. Zvadaro zvinotora iyo ine mukana mukuru uye inoienzanisa nechikamu chakatsanangurwa nemunyori wechishambadzo. Kana zvisingaenderane, iyo ad inovharirwa chikamu chisiri icho. Uye sezvo tiri vanhu vane mutsa uye vakatendeseka, tinoudza zvakananga mushandisi kuti ndeipi chikamu chaanofanira kusarudza kuitira kuti ad ipfuure mwero.
Chiziviso chekuvharira chikamu chisina kururama.
Kudzidza kwemichina kunonzwa kumba chaiko papuratifomu yedu. Semuenzaniso, nerubatsiro rwayo tinotsvaga mumazita uye tsananguro yezvinhu zvinorambidzwa muRussian Federation. Uye neural network modhi zvine hungwaru "wongorora" mifananidzo kuona kana ine maURL, zvinyorwa zve spam, nhamba dzenhare, uye "zvakarambidzwa" ruzivo rwakafanana.
Kana vari kuyedza kutengesa chigadzirwa chinorambidzwa chakavanzwa sechinhu chiri pamutemo, uye pasina mavara mune chero zita kana tsananguro, tinoshandisa mifananidzo yekumaka. Pamufananidzo wega wega, anosvika zviuru gumi nezviuru ma tag akasiyana anogona kuwedzerwa anotsanangura zviri mumufananidzo.
Vari kuedza kutengesa hookah nekuishandura sesamovar.
Mukuwirirana nemafirita akaomarara, akareruka anoshandawo, kugadzirisa zviri pachena matambudziko ane chekuita nemavara:
- antimat;
- URL uye nhamba yefoni detector;
- kutaurwa kwevatumwa pakarepo uye mamwe maitirwo;
- mutengo wakaderedzwa;
- zviziviso umo hapana chiri kutengeswa, nezvimwe.
Nhasi, kushambadza kwese kunopinda nemusefa weanopfuura makumi mashanu otomatiki mafirita anoedza kutsvaga chakashata mukushambadza.
Kana pasina chimwe chezviziviso chakashanda, zvino mhinduro inotumirwa kuYula kuti kushambadza "kunyanya" mukurongeka kwakakwana. Isu tinoshandisa mhinduro iyi isu pachedu, uye vashandisi vakanyoresa kumutengesi vanogamuchira chiziviso nezve kuwanikwa kwechigadzirwa chitsva.
Chiziviso chekuti mutengesi ane chigadzirwa chitsva.
Nekuda kweizvozvo, shambadziro yega yega "yakakura" nemetadata, imwe yacho inogadzirwa kana ad yagadzirwa (IP kero yemunyori, mushandisi-mumiriri, chikuva, geolocation, nezvimwewo), uye chasara ndicho chibodzwa chinoburitswa nesefa yega yega. .
Mitsetse yekuzivisa
Kana ad yarova papuratifomu, sisitimu inoiisa mune imwe yemitsara. Mutsara wega wega unogadzirwa pachishandiswa fomula yemasvomhu inosanganisa ad metadata nenzira inoona chero maitiro akaipa.
Semuenzaniso, unogona kugadzira mutsara wezviziviso muchikamu che "Mafoni Mafoni" kubva kune vashandisi veYula vanonzi vanobva kuSt.
Muenzaniso wezviziviso zvakatumirwa nemushandisi mumwechete mumaguta akasiyana.
Kana kuti unogona kuumba mitsetse zvichibva pazvibodzwa izvo neural network inopa kushambadzo, uchizvironga mukudzika.
Mutsetse wega wega, zvinoenderana nefomati yayo, unopa chibodzwa chekupedzisira kune ad. Ipapo iwe unogona kuenderera nenzira dzakasiyana:
- tsanangura chikumbaridzo icho chiziviso chichagamuchira imwe mhando yekuvhara;
- tumira ads ese ari mumutsara kune ma moderator kuti aongororwe nemaoko;
- kana kusanganisa zvakapfuura sarudzo: tsanangura otomatiki kuvharira chikumbaridzo uye tumira kune ma moderator ayo ads asina kusvika pachikumbaridzo ichi.
Sei mitsetse iyi ichidikanwa? Ngatitii mushandisi akaisa mufananidzo wepfuti. Iyo neural network inoipa iyo mamakisi kubva pa95 kusvika ku100 uye inosarudza ne99 muzana yechokwadi kuti pane chombo pamufananidzo. Asi kana kukosha kwezvibodzwa kuri pasi pe95%, kurongeka kwemuenzaniso kunotanga kuderera (ichi chinhu cheneural network models).
Nekuda kweizvozvo, mutsara unoumbwa unoenderana nemhando yezvibodzwa, uye iwo ads akagamuchira pakati pe95 ne100 anovharwa otomatiki se "Prohibited Products". Shambadziro dzine zvibodzwa pazasi makumi mapfumbamwe neshanu dzinotumirwa kune ma moderator kuti agadziriswe nemaoko.
Chocolate Beretta ine cartridges. Chete chekugadzirisa nemaoko! π
Manual moderation
Mukutanga kwa2019, ingangoita 94% yezvese kushambadzira muYula inogadziriswa otomatiki.
Kana iyo puratifomu isingakwanise kusarudza pane mamwe ads, inovatumira kune manyorero moderation. Odnoklassniki yakagadzira yavo chishandiso: mabasa evamodhita anobva angoratidza ruzivo rwese rwunodiwa kuti aite sarudzo nekukurumidza - kushambadza kwakakodzera kana kuvharwa, zvichiratidza chikonzero.
Uye kuitira kuti hutano hwebasa hurege kutambura panguva yekuenzanisa, basa revanhu rinogara richiongororwa. Semuenzaniso, mumutsara webasa, mutaridzi anoratidzwa "misungo" -zviziviso izvo zvatove zvakagadzirirwa zvakagadziriswa. Kana sarudzo yemutungamiri isingaenderane neyakapedzwa, mutariri anopiwa kukanganisa.
Paavhareji, moderator anopedza masekonzi gumi achitarisa imwe ad. Uyezve, nhamba yezvikanganiso haisi kupfuura 10% yezvose zvakasimbiswa ads.
Kuenzanisa kwevanhu
Vashandi veOdnoklassniki vakatoenda mberi uye vakatora mukana we "rubatsiro rwevateereri": vakanyora mutambo wemutambo wesocial network umo iwe unogona kukurumidza kumaka huwandu hwe data, uchiratidzira chimwe chiratidzo chakaipa - Odnoklassniki Moderator () Nzira yakanaka yekutora mukana werubatsiro rwevashandisi veOK vari kuyedza kuita kuti zvirimo zvinakidze.
Mutambo wekuti vashandisi vanomaka mafoto ane nhamba dzenhare pavari.
Chero mutsara wemashambadziro papuratifomu unogona kuendeswa kune iyo Odnoklassniki Moderator mutambo. Zvese zvinocherechedzwa nevashandisi vemutambo zvinozotumirwa kumamodhita emukati kuti aongororwe. Ichi chirongwa chinokutendera kuti uvhare kushambadza uko mafirita asati agadzirwa, uye panguva imwe chete kugadzira masampula ekudzidzisa.
Kuchengeta mibairo yekuyera
Isu tinochengetedza sarudzo dzese dzakaitwa panguva yekuenzanisa kuti tisagadzirise zvakare zviziviso zvatakatoita sarudzo.
Mamirioni emasumbu anogadzirwa zuva rega rega zvichienderana nekushambadzira. Nokufamba kwenguva, sumbu rimwe nerimwe rinonzi "chakanaka" kana "chakaipa." Imwe neimwe shambadziro nyowani kana ongororo yayo, ichipinda musumbu ine mucherechedzo, inogashira sarudzo kubva musumbu racho pacharo. Kune zviuru makumi maviri zvezvisarudzo zveotomatiki zvakadaro pazuva.
Kana pasina zviziviso zvitsva zvinosvika kusumbu, inobviswa mundangariro uye hashi yayo nemhinduro inonyorerwa Apache Cassandra.
Kana puratifomu inogamuchira kushambadza kutsva, inotanga kuedza kutsvaga sumbu rakafanana pakati pevakatogadzirwa uye kutora mhinduro kubva mairi. Kana pasina boka rakadaro, chikuva chinoenda kuCassandra uye chinotarisa ipapo. Waiwana here? Yakakura, inoshandisa mhinduro kune cluster uye inotumira kuYula. Kune avhareji yezviuru makumi manomwe zvezvisarudzo zvakadaro "zvinodzokororwa" zuva nezuva-70% yezvese.
Summing up
Tave tichishandisa Odnoklassniki moderation platform kwemakore maviri nehafu. Tinoda mhinduro:
- Isu tinogadzirisa otomatiki 94% yeese ads pazuva.
- Mutengo wekuenzanisa ad imwe yakaderedzwa kubva pa2 rubles kusvika ku7 kopecks.
- Kutenda kune yakagadzirira-yakagadzirwa chishandiso, takakanganwa nezve matambudziko ekutarisira mamodhi.
- Takawedzera nhamba yezviziviso zvakagadziridzwa nemaoko ne 2,5 nguva nenhamba yakafanana yevatariri uye bhajeti. Hunhu hwemanyorero modhiyo hwakawedzerawo nekuda kweotomatiki kudzora, uye hunochinja-chinja kutenderera 0,5% yezvikanganiso.
- Isu tinokurumidza kuvhara mhando nyowani dze spam nemasefa.
- Tinokurumidza kubatanidza madhipatimendi matsva kune mwero . Kubva 2017, Yula akawedzera iyo Real Estate, Vacancies uye Auto verticals.
Source: www.habr.com