🥇Video nrog huab khoom ntes ntawm Raspberry Pi

Prologue

Ib daim vis dis aus tam sim no nthuav tawm hauv Is Taws Nem qhia tias Tesla lub autopilot pom txoj kev li cas.

Kuv tau khaus ntev ntev los tshaj tawm cov vis dis aus txhawb nqa nrog lub ntes, thiab nyob rau lub sijhawm.

Qhov teeb meem yog tias kuv xav tshaj tawm cov vis dis aus los ntawm Raspberry, thiab kev ua haujlwm ntawm neural network ntes ntawm nws tawm ntau yam uas xav tau.

Intel Neural Computer Stick

Kuv xav txog cov kev daws teeb meem sib txawv.

В kab lus kawg sim nrog Intel Neural Computer Stick. Lub hardware yog haib, tab sis yuav tsum tau nws tus kheej network hom.

Txawm hais tias Intel muab cov hloov pauv rau lub hauv paus loj, muaj ntau qhov pitfalls.

Piv txwv li, hom ntawv ntawm lub network xav tau yuav tsis sib xws, thiab yog tias nws sib haum, ces qee cov khaubncaws sab nraud povtseg yuav tsis tau txais kev txhawb nqa ntawm lub cuab yeej, thiab yog tias lawv tau txais kev txhawb nqa, ces qhov yuam kev yuav tshwm sim thaum lub sijhawm hloov dua siab tshiab, vim li ntawd. peb tau txais tej yam txawv txawv ntawm qhov tso zis.

Feem ntau, yog tias koj xav tau qee yam kev xav ntawm neural network, ces nws yuav tsis ua haujlwm nrog NCS. Yog li ntawd, kuv txiav txim siab los sim daws qhov teeb meem uas siv cov cuab yeej siv dav thiab siv tau.

Huab

Qhov pom tseeb lwm txoj hauv kev daws teeb meem hauv zos yog mus rau huab.

Npaj-ua kev xaiv - kuv ob lub qhov muag khiav qus.

Txhua tus thawj coj:

... Thiab kaum ob ntawm cov neeg tsis paub tsawg dua.

Xaiv cov ntau yam no tsis yooj yim.

Thiab kuv txiav txim siab tsis xaiv, tab sis los qhwv cov txheej txheem ua haujlwm zoo qub ntawm OpenCV hauv Docker thiab khiav nws hauv huab.

Qhov kom zoo dua ntawm txoj kev no yog kev yoog raws thiab kev tswj hwm - koj tuaj yeem hloov pauv cov neural network, hosting, server - feem ntau, txhua qhov whim.

Neeg rau zaub mov

Cia peb pib nrog tus qauv hauv zos.

Ib txwm kuv siv Flask rau REST API, OpenCV thiab MobileSSD network.

Tau teeb tsa cov qauv tam sim no ntawm Docker, Kuv pom tias OpenCV 4.1.2 tsis ua haujlwm nrog Mobile SSD v1_coco_2018_01_28, thiab kuv yuav tsum rov qab mus rau qhov pov thawj 11/06_2017.

Thaum pib ntawm kev pabcuam, peb thauj cov npe hauv chav kawm thiab network:

def init():
    tf_labels.initLabels(dnn_conf.DNN_LABELS_PATH)
    return cv.dnn.readNetFromTensorflow(dnn_conf.DNN_PATH, dnn_conf.DNN_TXT_PATH)

Nyob rau hauv ib lub zos docker (ntawm lub laptop tsis heev) nws yuav siv sij hawm 0.3 vib nas this, ntawm Raspberry - 3.5.

Cia peb pib qhov kev xam:

def inference(img):
    net.setInput(cv.dnn.blobFromImage(img, 1.0/127.5, (300, 300), (127.5, 127.5, 127.5), swapRB=True, crop=False))
    return net.forward()

Docker - 0.2 sec, Raspberry - 1.7.

Tig tensor tso rau hauv json nyeem tau:

def build_detection(data, thr, rows, cols):
    ret = []
    for detection in data[0,0,:,:]:
        score = float(detection[2])
        if score > thr:
            cls = int(detection[1])
            a = {"class" : cls, "name" : tf_labels.getLabel(cls),  "score" : score}
            a["x"] = int(detection[3] * cols)
            a["y"] = int(detection[4] * rows)
            a["w"] = int(detection[5] * cols ) - a["x"]
            a["h"] = int(detection[6] * rows) - a["y"]
            ret.append(a)
    return ret

Ntxiv export cov haujlwm no ntawm Flask(input yog ib daim duab, cov zis yog cov txiaj ntsig ntawm tus ntes hauv json).

Lwm txoj kev xaiv, nyob rau hauv uas ua hauj lwm ntau yog hloov mus rau tus neeg rau zaub mov: nws nws tus kheej voj voog cov khoom pom thiab xa rov qab cov duab tiav.

Qhov kev xaiv no yog qhov zoo uas peb tsis xav rub opencv rau lub server.

Docker

Peb sau cov duab.

Code yog combed thiab muab tso rau Github, docker yuav coj nws ncaj qha los ntawm qhov ntawd.

Raws li lub platform, peb yuav coj tib Debian Stretch ib yam li ntawm Raspberry - peb yuav tsis sib txawv ntawm cov pov thawj tech pawg.

Koj yuav tsum tau nruab lub raj mis, protobuf, thov, opencv_python, download Mobile SSD, server code los ntawm Github thiab pib lub server.

FROM python:3.7-stretch

RUN pip3 install flask
RUN pip3 install protobuf
RUN pip3 install requests
RUN pip3 install opencv_python

ADD http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_coco_11_06_2017.tar.gz /
RUN tar -xvf /ssd_mobilenet_v1_coco_11_06_2017.tar.gz

ADD https://github.com/tprlab/docker-detect/archive/master.zip /
RUN unzip /master.zip

EXPOSE 80

CMD ["python3", "/docker-detect-master/detect-app/app.py"]

Yooj yim neeg ntes ntes raws li kev thov.

Tshaj tawm rau Docker Hub

Docker registries tau nce ntau ntawm qhov ceev tsis tsawg dua li huab cua ntes.

Yuav kom tsis txhob thab plaub, peb yuav conservatively mus dhau DockerHub.

Sau npe
Nkag mus rau:
docker nkag
Cia peb los nrog lub ntsiab lus lub npe:
docker tag opencv-detect tprlab/opencv-detect-ssd
Upload duab rau lub server:
docker thawb tprlab/opencv-detect-ssd

Peb tso rau hauv huab

Kev xaiv ntawm qhov chaw khiav lub thawv kuj yog qhov dav heev.

Txhua tus neeg ua si loj (Google, Microsoft, Amazon) muab micro-piv txwv pub dawb rau thawj xyoo.
Tom qab sim nrog Microsoft Azure thiab Google Cloud, kuv tau nyob rau tom kawg vim nws tau tawm sai dua.

Kuv tsis tau sau cov lus qhia ntawm no, vim qhov no yog qhov tshwj xeeb heev rau tus kws kho mob xaiv.

Kuv sim cov kev xaiv kho vajtse sib txawv,
Qib qis (sib koom thiab mob siab rau) - 0.4 - 0.5 vib nas this.
Ntau lub tsheb muaj zog - 0.25 - 0.3.
Zoo, txawm tias qhov xwm txheej phem tshaj plaws, qhov kev yeej yog peb zaug, koj tuaj yeem sim.

Video

Peb tso tawm qhov yooj yim OpenCV video streamer ntawm Raspberry, kuaj pom los ntawm Google Huab.
Rau qhov kev sim, ib cov ntaub ntawv video tau siv uas yog ib zaug ua yeeb yaj kiab ntawm kev sib tshuam tsis sib xws.


def handle_frame(frame):
    return detect.detect_draw_img(frame)
       
def generate():
    while True:
        rc, frame = vs.read()
        outFrame = handle_frame(frame)
        if outFrame is None:
            (rc, outFrame) = cv.imencode(".jpg", frame)
        yield(b'--framern' b'Content-Type: image/jpegrnrn' + bytearray(outFrame) + b'rn')

@app.route("/stream")
def video_feed():
    return Response(generate(), mimetype = "multipart/x-mixed-replace; boundary=frame")

Nrog lub ntes peb tau txais tsis ntau tshaj peb thav duab ib ob, txhua yam mus qeeb heev.
Yog tias koj siv lub tshuab muaj zog rau hauv GCloud, koj tuaj yeem ntes 4-5 thav duab ib ob, tab sis qhov sib txawv yuav luag tsis pom ntawm lub qhov muag, nws tseem qeeb.

Cov huab thiab cov nqi tsheb thauj mus los tsis muaj dab tsi ua nrog nws; lub tshuab ntes tau khiav ntawm cov khoom siv zoo tib yam thiab ua haujlwm ntawm qhov nrawm.

Neural Computer Stick

Kuv tsis tuaj yeem tawm tsam thiab khiav qhov ntsuas ntawm NCS.

Qhov ceev ntawm lub ntes tau qeeb me ntsis tshaj 0.1 vib nas this, nyob rau hauv txhua rooj plaub 2-3 lub sij hawm sai tshaj huab cua ntawm lub tshuab tsis muaj zog, piv txwv li 8-9 ntas ib ob.

Qhov sib txawv ntawm cov txiaj ntsig tau piav qhia los ntawm qhov tseeb tias NCS tau khiav Mobile SSD version 2018_01_28.

PS Tsis tas li ntawd, kev sim tau pom tias lub tshuab hluav taws xob muaj zog zoo nrog I7 processor qhia tau zoo dua me ntsis thiab nws tau dhau los ua kom nyem 10 ntas ib ob ntawm nws.

Pawg

Qhov kev sim mus ntxiv thiab kuv tau teeb tsa lub ntes ntawm tsib lub hauv Google Kubernetes.
Cov pods lawv tus kheej tsis muaj zog thiab txhua tus ntawm lawv tsis tuaj yeem ua ntau tshaj 2 thav duab ib ob.
Tab sis yog tias koj khiav ib pawg nrog N nodes thiab parse thav duab hauv N threads, ces nrog tus naj npawb txaus (5) koj tuaj yeem ua tiav qhov xav tau 10 ntas ib ob.

def generate():
    while True:
        rc, frame = vs.read()
        if frame is not None:
            future = executor.submit(handle_frame, (frame.copy()))
            Q.append(future)

        keep_polling = len(Q) > 0
        while(keep_polling):            
            top = Q[0]
            if top.done():
                outFrame = top.result()
                Q.popleft()
                if outFrame:
                    yield(b'--framern' b'Content-Type: image/jpegrnrn' + bytearray(outFrame) + b'rn')
                keep_polling = len(Q) > 0
            else:
                keep_polling = len(Q) >= M

Nov yog qhov tshwm sim:

Tsawg me ntsis nrawm dua nrog NCS, tab sis muaj zog dua li hauv ib qho kwj.

Qhov nce, ntawm chav kawm, tsis yog linear - muaj overlays rau synchronization thiab sib sib zog nqus luam ntawm opencv dluab.

xaus

Zuag qhia tag nrho, qhov kev sim ua rau peb txiav txim siab tias yog tias koj sim, koj tuaj yeem tawm mus nrog huab cua yooj yim.

Tab sis lub desktop muaj zog lossis cov khoom siv hauv zos tso cai rau koj ua tiav cov txiaj ntsig zoo dua, thiab tsis muaj kev dag ntxias.

ua tim khawv

Tau qhov twg los: www.hab.com

Video ntawm Huab Object Detector ntawm Raspberry Pi