Nzira yekugovera mishumo iri nyore kune robhoti. Kunyora bot muPython uye Google BigQuery

Une mabasa anodzokorora zuva nezuva, vhiki nevhiki here? Somuenzaniso, kunyora mishumo. Iwe unokumbira data, iongorore, ione (ita magirafu, machati), wozoitumira kune mukuru wako. Asi ko kana zvese izvi zvaive otomatiki?

Muchidzidzo ichi tichagadzira bot yeTeregiramu inozobatsira otomatiki kutaura. Uye chinhu chinotonhorera ndechekuti chirongwa chose chichava nemitsara makumi mashanu chete yekodhi! Kana iwe uri kugadzira bot yeTeregiramu kekutanga, saka iwe unofanirwawo kuverenga iyi post.

Skillbox inokurudzira: Nzira inoshanda Python developer kubva pakutanga.

Tinoyeuchidza: kune vese vaverengi veHabr - a 10 ruble mutengo paunenge uchinyoresa mune chero Skillbox kosi uchishandisa iyo Habr promo kodhi.

Ngatitangei

Kuisa maraibhurari

Tichashandisa google-cloud-bigquery kuti uwane data kubva kuGoogle BigQuery. kkburemu, numpy и pandas ichakubatsira kuona data rako. python-telegraph-bot ichatumira data rakapedzwa kuTeregiramu.

pip3 isa google-cloud-bigquery matplotlib numpy pandas python-telegram-bot

Purogiramu inonzi Google BigQuery

Kana isu tichida kushandisa sevhisi, tinoda kubatanidza iyo Google BigQuery API. Kuti tiite izvi tinoenda Google Developers Console uye gadzira chirongwa chitsva (kana sarudza chiripo).

Mupaneru yekudzora, sarudza VUMISA APIS UYE MASEVISI uye tsvaga BigQuery API.

Sarudza Bvumira kubatanidza API.

Gadzira kiyi yeakaundi

Handei zvakare Google Developers Console, sarudza iyo Credentials tab, Gadzira zvitupa uye Service account kiyi.

Zvadaro - New service account, uye isa zita muSevhisi account zita ndima.

Kubva pane Role yekudonha-pasi rondedzero, sarudza Chirongwa> Muridzi, wobva Gadzira.

Iyo faira inozotorwa otomatiki inonzi creds.json.

Seta GOOGLE_APPLICATION_CREDENTIALS, uchitsanangura nzira inoenda kucreds.json muchiteshi.

kunze GOOGLE_APPLICATION_CREDENTIALS='[PATH_TO_CREDS.JSON]'

Kana zvese zvikafamba zvakanaka, inguva yekutanga kunyora chirongwa.

Kugadzira application

Pachidzidzo tichashandisa data kubva bigquery-public-data.stackoverflow, kune yedu report tichasarudza nhamba yezvinyorwa zvezuva nezuva.

Zvese zvinhu zvakapusa.

Bvunza tafura -> Ona iyo data -> Sevha iyo yekuona -> Tumira mufananidzo

Ngatigadzirei basa rimwechete rekutsanangura shinda imwe neimwe.

Mubvunzo kune BigQuery

Kutanga tinopinza raibhurari.

kubva kugoogle.cloud import bigquery

Isu tinogadzira basa rinonzi query_to_bigquery, apo parameter iri mubvunzo.

def query_to_bigquery(query):
    client = bigquery.Client()
    query_job = client.query(query)
    result = query_job.result()
    dataframe = result.to_dataframe()
    return dataframe

Iri basa rinodzosa chikumbiro se data furemu.

Kuona data

Kugadzirisa dambudziko iri, sarudza matplotlib.

import matplotlib.pyplot as plt

Tinoda maparamita mashanu, apo x ndiyo x-axis data, x_label ndiro zita reiyo axis, y ndiro y-axis data, y_label ndiro zita reiyo axis, uye zita ndiro zita rekuona kwese.

def visualize_bar_chart(x, x_label, y, y_label, title):
    plt.title(title)
    plt.xlabel(x_label)
    plt.ylabel(y_label)
    index = np.arange(len(x))
    plt.xticks(index, x, fontsize=5, rotation=30)
    plt.bar(index, y)
    return plt

Sevha mufananidzo

Zvino ngatishandisei mabasa maviri kugadzira chitarisiko uye chekuchengetedza.

Isu tichatumira nhamba yezvinyorwa zvinoburitswa zuva nezuva. Kutanga tinonyora chikumbiro.

query = """
        SELECT DATE(creation_date) date, COUNT(*) total_posts
        FROM `bigquery-public-data.stackoverflow.post_history`
        GROUP BY 1
        HAVING date > DATE_SUB('2018-12-02', INTERVAL 14 DAY)
        ORDER BY 1
        """

Mubvunzo unobatsira kuunganidza data kwemavhiki maviri kutanga Zvita 2, 2018.

Isu tinoshandisa zuva iri nekuti 2018-12-02 ndiro data richangoburwa rakarekodwa mu bigquery-public-data.stackoverflow.post_history, mune dzimwe nguva unogona kushandisa CURRENT_DATE() kuwana data idzva.

Fonera iyo query_to_bigquery basa kuti uwane iyo data.

dataframe = query_to_bigquery(mubvunzo)

Tobva tashandisa date data column ye x-axis, uye total_posts column ye y-axis.

x = dataframe['date'].tolist()
y = dataframe['total_posts'].tolist()

Isu tinozviona tichishandisa visualize_bar_chart basa uye chengetedza semufananidzo.

plt = visualize_bar_chart(x=x, x_label='Date', y=y, y_label='Zvose Zvatumirwa', title='Daily Posts')
plt.savefig('viz.png')

Isu tinoputira iyi kodhi mune basa rinonzi get_and_save_image.

def get_and_save_image():
    query = """
            SELECT DATE(creation_date) date, COUNT(*) total_posts
            FROM `bigquery-public-data.stackoverflow.post_history`
            GROUP BY 1
            HAVING date > DATE_SUB('2018-12-02', INTERVAL 14 DAY)
            ORDER BY 1
            """
    dataframe = query_to_bigquery(query)  
    x = dataframe['date'].tolist()
    y = dataframe['total_posts'].tolist()
    plt = visualize_bar_chart(x=x, x_label='Date', y=y, y_label='Total Posts', title='Daily Posts')
    plt.savefig('viz.png')

Tumira mufananidzo

Kuti utumire mushumo kune anogamuchira, unofanirwa kuziva iyo chat_id parameter.

Isu tinoshandisa userinfobot uye nyora / kutanga. Iyo bot inopindura neruzivo rwunodiwa, chat_id iri mundima yeid.

Zvino ngatigadzire iyo send_image basa. Ichashandisa iyo get_and_save_image basa kutora uye kuchengetedza mufananidzo. Uye zvino tinotumira zvese kune iyo chaiyo kuonana.

def send_image(bot, update):
    get_and_save_image()
    chat_id = 'CHAT_ID_RECEIVER'
    bot.send_photo(chat_id=chat_id, photo=open('viz.png','rb'))

Main program

Pakupedzisira, isu tinogadzira rimwe basa, chikuru, kuvhura iyo application. Usakanganwe kuchinja YAKO_TOKEN yebhoti.

Rangarira: chirongwa ichi chinotumira mufananidzo wacho otomatiki panguva yaunotsanangura. Somuenzaniso, tichatumira mushumo naXNUMX:XNUMX mangwanani zuva nezuva.

def main():
    updater = Updater('YOUR_TOKEN')
    updater.job_queue.run_daily(send_image, time=datetime.datetime.strptime('9:00AM', '%I:%M%p').time(), days=(0,1,2,3,4,5,6))
    updater.start_polling()
    updater.idle()
 
if __name__ == '__main__':
    main()

Nekuda kweizvozvo, application yedu ichaita seizvi:

from google.cloud import bigquery
from telegram.ext import Updater
 
import matplotlib.pyplot as plt
import numpy as np
import datetime
 
def query_to_bigquery(query):
    client = bigquery.Client()
    query_job = client.query(query)
    result = query_job.result()
    dataframe = result.to_dataframe()
    return dataframe
 
def visualize_bar_chart(x, x_label, y, y_label, title):
    plt.title(title)
    plt.xlabel(x_label)
    plt.ylabel(y_label)
    index = np.arange(len(x))
    plt.xticks(index, x, fontsize=5, rotation=30)
    plt.bar(index, y)
    return plt
 
def get_and_save_image():
    query = """
            SELECT DATE(creation_date) date, COUNT(*) total_posts
            FROM `bigquery-public-data.stackoverflow.post_history`
            GROUP BY 1
            HAVING date > DATE_SUB('2018-12-02', INTERVAL 14 DAY)
            ORDER BY 1
            """
    dataframe = query_to_bigquery(query)  
    x = dataframe['date'].tolist()
    y = dataframe['total_posts'].tolist()
    plt = visualize_bar_chart(x=x, x_label='Date', y=y, y_label='Total Posts', title='Daily Posts')
    plt.savefig('viz.png')
 
def send_image(bot, update):
    get_and_save_image()
    chat_id = 'CHAT_ID_RECEIVER'
    bot.send_photo(chat_id=chat_id, photo=open('viz.png', 'rb'))
 
def main():
    updater = Updater('YOUR_TOKEN')
    updater.job_queue.run_daily(send_image, time=datetime.datetime.strptime('9:00AM', '%I:%M%p').time(), days=(0,1,2,3,4,5,6))
    updater.start_polling()
    updater.idle()
 
if __name__ == '__main__':
main()

Sevha faira woridaidza main.py.

Isu tinotangisa iyo application nekuisa murairo mune terminal:

python3 main.py

Zvese zvagadzirira. Iye zvino tine robhoti rine mitsara ye50 yekodhi iyo inogadzira mishumo pasina kupindira kwedu.

Ngatitarisei bot kubva panonekusarudza iyo /send command.

Unogona kuwana kodhi yapera pa GitHub yangu.

Skillbox inokurudzira:

• Makore maviri anoshanda kosi "Ini ndiri PRO webhu developer".
• Yepamhepo kosi "C # mugadziri kubva ku0".
• Kosi yegore rimwechete "PHP mugadziri kubva 0 kusvika PRO".

Source: www.habr.com