I ēnei rā, e 100500 ngā akoranga pūtaiao raraunga, ā, kua roa e mōhiotia ana ko te nuinga o te moni ka taea te whiwhi i roto i te pūtaiao raraunga mai i ngā akoranga pūtaiao raraunga (he aha ai te keri ina taea e koe te hoko koko?). Ko te ngoikoretanga matua o ēnei akoranga ko te kore e pā ki te mahi tūturu: kāore he tangata e hoatu ki a koe ngā raraunga ma, kua tukatukahia i roto i te whakatakotoranga e hiahiatia ana. Ā, ina wehe atu koe i ngā akoranga ka tīmata ki te whakaoti rapanga tūturu, ka puta ake he maha ngā āhuatanga rerekē.
Nō reira, kei te tīmata mātou i tētahi raupapa tuhinga, "He aha ngā hapa ka pā ki te Pūtaiao Raraunga," i ahu mai i ngā wheako tūturu o au, o aku hoa mahi, me ētahi atu. Ka tūhuratia e mātou ngā mahi pūtaiao raraunga noa mā te whakamahi i ngā tauira tūturu, ā, ka kite me pēhea te tupu. I tēnei rā, ka tīmata mātou ki te kohikohi raraunga.
Ā, ko te mea tuatahi ka hē te mahi a te tangata ina tīmata rātou ki te mahi me ngā raraunga tūturu ko te kohikohi i aua raraunga e tika ana. Ko te karere matua o tēnei tuhinga:
Ka whakaititia rawatia e mātou te wā, ngā rauemi me te kaha e hiahiatia ana hei kohikohi, hei horoi, hei whakarite hoki i ngā raraunga.
Ā, ko te mea nui, me matapaki tātou i ngā mahi hei ārai i tēnei.
E ai ki ngā whakatau tata, ko te horoi, te whakarerekētanga, te tukatuka raraunga, te hangarau āhuatanga, me ētahi atu, e 80-90% o te wā e pau ana, ā, ko te tātari he 10-20%, ko te nuinga ia o ngā rauemi mātauranga e aro ana ki te tātari anake.
Me titiro tātou ki tētahi rapanga tātari māmā i roto i ngā putanga e toru hei tauira noa, kia kite ai he aha ngā "āhuatanga whakararuraru".
Hei tauira anō, ka whakaarohia e tātou ngā momo rerekētanga ōrite o te mahi kohikohi raraunga me te whakatairite i ngā hapori mō:
- E rua ngā wāhanga iti o Reddit
- Ngā wāhanga e rua o Habr
- E rua ngā rōpū Odnoklassniki
Huarahi ā-tikanga i roto i te ariā
Whakatuwheratia te paetukutuku ka pānui i ngā tauira. Ki te mārama koe ki a rātou, whakaritea he torutoru hāora mō te pānui, he torutoru hāora mō te waehere i ngā tauira, me te patuiro. Tāpirihia he torutoru hāora mō te whakahiato. Tāpirihia he torutoru hāora hei rahui (whakareatia kia rua, ka tāpirihia kia N hāora).
Kaupapa matua: Ko te whakatau tata o te wā e ahu mai ana i ngā whakapae me ngā matapae mō te roa o te wā e hiahiatia ana.
Me tīmata te tātari wā mā te aromatawai i ngā tawhā e whai ake nei mō te mahi ā-tikanga kua whakaahuatia i runga ake nei:
- He aha te rahi o ngā raraunga, ā, e hia hoki te nui o aua raraunga me kohi ā-tinana (*tirohia i raro*).
- Kia pēhea te roa o te kohikohinga mō te rekoata kotahi, ā, kia pēhea te roa e tatari ai koe i mua i te kohikohinga o te tuarua?
- Whakauruhia te tuhi waehere e tiaki ana i te āhua me te tīmata i te tīmatanga anō ina (kaua mēnā) ka hinga ngā mea katoa.
- Whakatauhia mēnā me whakamanatanga mātou, ā, whakamaherehia te wā e pau ana ki te whiwhi urunga ki te API.
- Aromatawaihia te maha o ngā hapa hei pānga o te uauatanga o ngā raraunga i runga i tētahi mahi motuhake: te hanganga, te maha o ngā panonitanga, te aha me te pēhea te tango.
- Hei kite i ngā hapa whatunga me ngā raruraru e pā ana ki ngā whanonga kaupapa rerekē.
- Arotakehia mēnā kei roto i ngā tuhinga ngā mahi e hiahiatia ana, ā, ki te kore, me pēhea, ā, e hia te nui e hiahiatia ana hei whakaoti rapanga.
Ko te mea nui, hei whakatau tata i te wā, me whakapau kaha koe ki te "tirotiro kaha"—koira anake ka rawaka ai tō whakamahere. Nō reira, ahakoa te nui o te wā e akiakihia ana koe ki te kī, "E hia te wā e pau ana ki te kohikohi raraunga?"—whakaritea he wā mō te tātari tuatahi me te whakamārama i tō whakatau tata mā te whakaatu i te rerekētanga o te wā i runga i ngā tawhā tuturu o te mahi.
Nā, ka whakaaturia e mātou ngā tauira motuhake ka rerekē ai aua tawhā.
Kaupapa Matua: Ka ahu mai te aromatawai i tētahi tātaritanga o ngā āhuatanga matua e pā ana ki te rahinga me te uauatanga o te mahi.
He huarahi pai te whakatau tata i runga i te matapae ina he iti noa ngā āhuatanga mahi, ā, he iti noa iho ngā āhuatanga ka pā nui ki te hanganga o te mahi. Heoi, mō ētahi mahi pūtaiao raraunga, ka tino maha rawa atu aua āhuatanga, ā, kāore e ranea tēnei huarahi.
Te whakataurite i ngā hapori Reddit
Me tīmata tātou ki te take māmā rawa (e ai ki te āhua). Ki te pono, he take tino pai tēnei. Me tirotiro tātou i tā tātou rārangi arowhai uaua:
- He API pai, mārama, kua tuhia hoki.
- He tino māmā noa iho, ā, ko te mea nui, ka whiwhi aunoa koe i tētahi tohu.
- He - me te maha o ngā tauira.
- He hapori e tātari ana, e kohikohi ana hoki i ngā raraunga mō Reddit (tae atu ki ngā ataata YouTube e whakamārama ana me pēhea te whakamahi i te takai Python) .
- Ko ngā tikanga e hiahiatia ana e tātou, kei roto pea i te API. Heoi anō, he āhua kiato, he mā hoki te waehere; kei raro nei tētahi tauira o tētahi mahi e kohikohi ana i ngā kōrero i runga i tētahi pou.
def get_comments(submission_id):
reddit = Reddit(check_for_updates=False, user_agent=AGENT)
submission = reddit.submission(id=submission_id)
more_comments = submission.comments.replace_more()
if more_comments:
skipped_comments = sum(x.count for x in more_comments)
logger.debug('Skipped %d MoreComments (%d comments)',
len(more_comments), skipped_comments)
return submission.comments.list()
I tangohia mai i ngā kohinga o ngā taputapu watea mō te takai.
Ahakoa koinei te āhuatanga pai rawa atu, he maha tonu ngā āhuatanga nui hei whakaaroaro māu:
- Ngā herenga API - me tango raraunga mātou i roto i ngā puranga (moe i waenga i ngā tono, me ētahi atu).
- Te wā kohikohi - mō te tātaritanga me te whakatairite katoa, me whakarite he wā nui kia ngokingoki noa te pūngāwerewere i roto i te subreddit.
- Me whakahaere te karetao i runga i tētahi tūmau—kāore e taea te whakahaere noa i runga i tētahi pona, ka maka ki roto i tō peke tuara, ka mahi i tāu mahi. Nō reira, i whakahaerehia e au ngā mea katoa i runga i tētahi VPS. Whakamahia te waehere whakatairanga habrahabr10 hei penapena i tētahi atu 10%.
- Me whai whakaaro ki te uaua o te uru atu ki ētahi raraunga (e kitea ana e ngā kaiwhakahaere, he uaua rawa rānei ki te kohikohi); kāore e taea te kohikohi i ngā raraunga katoa i roto i te wā tika.
- Ngā Hapa Whatunga: He uaua te whatunga.
- He raraunga tūturu tēnei - kāore i te parakore.
Me whai whakaaro anō hoki ki ēnei āhuatanga rerekē i roto i te tukanga whakawhanake. Ko ngā hāora/rā motuhake e whakawhirinaki ana ki te wheako o te kaiwhakawhanake, ki ngā wheako o mua rānei ki te mahi i ngā mahi ōrite. Heoi, e kite ana mātou he mahi hangarau noa iho tēnei, ā, kāore e hiahiatia he kaha tāpiri hei whakaoti - ka taea te whakatau tata, te whakatakoto, me te whakahaere i ngā mea katoa.
Te whakataurite i ngā wāhanga Habr
Me neke atu tātou ki tētahi tauira whakamere ake, ehara i te mea noa iho, arā, te whakataurite i ngā awa me/ngā wāhanga rānei o Habr.
Me tirotiro tā tātou rārangi arowhai uauatanga—kia mārama ai koe ki ia āhuatanga, me rapu noa e koe te raruraru, me whakamātau hoki.
- I te tīmatanga, tērā pea ka whakaaro koe he API kei reira, engari kāore. Ae, he API kei a Habr, engari kāore e taea e ngā kaiwhakamahi te uru atu (kāore rānei e mahi ana).
- Kātahi ka tīmata koe ki te wetewete i te HTML — "ngā tono kawemai", he aha pea ka hē?
- Nā, me pēhea te wetewete? Ko te huarahi māmā rawa atu, ko te huarahi e whakamahia whānuitia ana, ko te whakahou i ngā ID. Kia mōhio koe, ehara tēnei i te huarahi tino whai hua, ā, me whakahaere i ngā take maha—hei tauira, te kiato o ngā ID tūturu i roto i ngā mea katoa kua oti kē te whakarite.
I tangohia mai i tuhinga. - He uaua te kohikohi i ngā raraunga mata ki te HTML mā te tukutuku. Hei tauira, me kohi, me penapena hoki te whakatauranga o tētahi tuhinga: ka tangohia e koe te kaute mai i te HTML, ka whakatau ki te tiaki hei tau mō te tukatuka atu:
1) ka maka hapa te int(score): nā te mea i runga i a Habr he tohu tango, pērā i te aho "–5", he tohu en, ehara i te tohu tango (he mea ohorere, āe?), nō reira i tētahi wā me whakahoki mai e au te parser ki te ora me tēnei whakatikatika kino.
try: score_txt = post.find(class_="score").text.replace(u"–","-").replace(u"+","+") score = int(score_txt) if check_date(date): post_score += scoreKāore pea he rā, he tāpiri, he tango rānei (e kitea ana i runga ake nei me te mahi check_date, i tupu anō hoki tēnei).
2) Ngā pūāhua motuhake kāore anō kia mawhiti - ka tae mai rātou, me noho rite koe.
3) Ka rerekē te hanganga i runga i te momo pou.
4) Tērā pea he **hanganga rerekē** o ngā pou tawhito.
- Ko te tikanga, me whakahaere te whakahaere hapa me ngā mea ka tupu, kāore rānei, ā, kāore e taea te matapae tika he aha ka hē, he aha te hanganga, me te aha ka hē pea i hea—me ngana noa koe ki te whakaaro ki ngā hapa ka tukuna e te kaitātari.
- Kātahi ka mōhio koe me wetewete koe i roto i ētahi miro, ki te kore ka roa te wetewete i roto i tētahi miro neke atu i te 30 hāora (koinei te wā whakahaere o tētahi wetewete miro-kotahi kua mahi kē, e moe ana, ā, kāore e pāngia e tētahi aukati). tuhinga, i arahina tēnei i tētahi wā ki tētahi kaupapa ōrite:
Nā, anei te rārangi arowhai mā te uauatanga:
- Te mahi tahi me te whatunga me te wetewete HTML me te whakahounga me te tatau mā te ID.
- Ngā tuhinga o te hanganga kanorau.
- He maha ngā wāhi ka ngawari te taka o te waehere.
- He mea tika kia tuhia te waehere ||.
- Kāore i te rawaka ngā tuhinga, ngā tauira waehere, me/ranei te hapori.
Ko te wā e matapaetia ana mō tēnei mahi ka 3-5 ngā wā teitei ake i te kohikohi raraunga mai i Reddit.
Te whakataurite i ngā rōpū Odnoklassniki
Me neke atu tātou ki te take hangarau tino whakamere kua whakaahuatia. Ki ahau nei, he mea whakamere nā te mea, i te tuatahi, he ahua noa iho, engari kāore i te tino rite - ina werohia ki te rākau.
Me tīmata tātou me tā tātou rārangi arowhai uauatanga, ā, kia mōhio tātou he maha o ēnei ka tino uaua ake i tō te āhua tuatahi:
- He API kei reira, engari tata kore rawa ngā mahi e tika ana.
- Ko ētahi mahi e hiahiatia ana he tono urunga mā te īmēra, ko te tikanga kāore e tukuna tonutia te urunga.
- He tino kino te tuhinga (me tīmata tātou me te mea he maha ngā kupu Rūhia me te reo Pākehā e whakaranua ana i ngā wāhi katoa, ā, he tino rerekē - i ētahi wā me whakaaro noa koe ki tā te tangata e hiahia ana mai i a koe i tētahi wāhi) ā, heoi anō, kāore i te pai mō te hoahoa mō te tiki raraunga, hei tauira, .
- Me whai wātū i roto i te tuhinga, engari i te ao tūturu, kāore e whakamahia ana - ā, kāore he huarahi hei mārama ki ngā āhuatanga uaua katoa o ngā aratau API engari ko te rapu haere me te tumanako ka mahi tetahi mea.
- Kāore he tauira, kāore hoki he hapori, ko te mea anake hei tautoko i te kohikohi mōhiohio he iti noa iho i roto i te Python (kāore he maha o ngā tauira whakamahinga).
- Ko te Selenium te kōwhiringa tino whai hua, nā te mea he nui ngā raraunga e hiahiatia ana kua raka.
1) Arā, ka whakamanahia mā te kaiwhakamahi rūpahu (ā, ka mahia ā-ringa te rēhitatanga).2) Heoi, kāore he taurangi mō te mahi tika me te mahi auau a Selenium (i roto i te take ok.ru).
3) Kei roto i te paetukutuku Ok.ru ngā hapa JavaScript, ā, i ētahi wā ka rerekē, ka kore e rite te whanonga.
4) Me mahi koe ki te whakarārangi whārangi, te uta i ngā huānga, me ētahi atu.
5) Ko ngā hapa API e whakahokia mai ana e te takai me whakahaere i roto i te huarahi hacked, hei tauira, pēnei i tēnei (he wāhanga o te waehere whakamātautau):
def get_comments(args, context, discussions): pause = 1 if args.extract_comments: all_comments = set() #makes sense to keep track of already processed discussions for discussion in tqdm(discussions): try: comments = get_comments_from_discussion_via_api(context, discussion) except odnoklassniki.api.OdnoklassnikiError as e: if "NOT_FOUND" in str(e): comments = set() else: print(e) bp() pass all_comments |= comments time.sleep(pause) return all_commentsKo taku hapa tino pai ko:
OdnoklassnikiError("Error(code: 'None', description: 'HTTP error', method: 'discussions.getComments', params: …)”)6) Hei whakamutunga, ko te kōwhiringa Selenium + API te mea tino whaitake.
- Me tiaki te āhua me te tīmata anō i te pūnaha, me te whakahaere i ngā hapa maha, tae atu ki ngā whanonga pae tukutuku kāore i te ōrite—ā, he tino uaua ki te whakaaro i ēnei hapa (ki te kore koe he kaituhi wetewete ngaio).
Ko te wā e matapaetia ana mō tēnei mahi ka 3-5 ngā wā teitei ake i te kohikohi raraunga mai i Habr. Ahakoa te mea e whakamahi ana mātou i te huarahi kaha-whakamātautau me te wetewete HTML ki Habr, engari mā te OK ka taea e mātou te whakamahi i te API i ngā wāhi tino nui.
kitenga
Ahakoa te nui o te tono kia whakatauhia e koe te wā "i taua wā tonu" (kei te whakamahere mātou i tēnei rā!) mō tētahi kōwae paipa tukatuka raraunga nui, e kore rawa e taea te whakatau tata i te wā whakahaere, ahakoa te kounga, me te kore e tātari i ngā tawhā mahi.
I roto i te tirohanga whānui, he pai rawa ngā rautaki aromatawai matatau mō ngā mahi hangarau, engari mō ngā mahi he whakamātautau ake, ā, he "auaha" me te tūhura, arā, he uaua ake te matapae, ka puta ake ngā uauatanga, pērā i ngā tauira kua kōrerohia i konei.
He tauira pai noa iho te kohikohi raraunga—he mahi māmā noa iho, he mahi hangarau hoki te nuinga o te wā, engari ko ngā taipitopito te mea nui. Ā, ko tēnei mahi tonu e āhei ai tātou ki te whakaatu i te whānuitanga o ngā āhuatanga ka pā mai pea he hē, me te roa o te wā e pau ai te mahi.
Ki te tirohia ngā āhuatanga o te mahi me te kore he whakamātautau anō, he rite tonu te āhua o Reddit me OK: he API me he takai Python kei roto i a rāua, engari he tino rerekē ngā āhuatanga matua. Ki te tirohia ēnei tawhā, he uaua ake te wetewete a Habr i a OK—engari i roto i te mahi, he tino rerekē, ā, ka taea te whakatau i tēnei mā te whakahaere i ngā whakamātautau ngāwari hei tātari i ngā tawhā mahi.
I roto i aku wheako, ko te huarahi tino whai hua ko te whakatau tata i te wā e hiahiatia ana mō te tātaritanga tuatahi, ngā whakamātautau tuatahi māmā, me te pānui tuhinga—mā tēnei ka taea e koe te whakarato i tētahi whakatau tata tika mō te mahi katoa. I roto i te tikanga agile rongonui, ka tono ahau kia hangaia he tīkiti mō te "whakatau tata tawhā mahi," hei hanga i tētahi whakatau tata mō ngā mea ka taea te whakatutuki i roto i tētahi tere tere, me te whakarato i tētahi whakatau tata tika ake mō ia mahi.
Nō reira, ko te tautohe tino whai hua ko te whakaatu ki tētahi tohunga "ehara i te hangarau" te nui o te wā me ngā rauemi ka rerekē i runga i ngā tawhā kāore anō kia aromatawaitia.
Source: will.com
