Yako yekutanga neural network pane graphics processing unit (GPU). Nhungamiro Yevatangi

Muchinyorwa chino, ini ndichakuudza nzira yekumisikidza muchina wekudzidza nharaunda mumaminetsi makumi matatu, gadzira neural network yekuzivikanwa kwemufananidzo, uye wobva wamhanyisa network imwechete pane graphics processor (GPU).

Kutanga, ngatitsanangure kuti chii chinonzi neural network.

Kwatiri, iyi imhando yemasvomhu, pamwe nesoftware yayo kana hardware embodiment, yakavakirwa pamusimboti wekuronga uye kushanda kwebiological neural network - network yetsinga masero echinhu chipenyu. Iyi pfungwa yakasimuka tichidzidza maitiro anoitika muuropi uye kuedza kutevedzera maitiro aya.

Neural network haina kurongwa nenzira yakajairwa yezwi, ivo vanodzidziswa. Iko kugona kudzidza ndeimwe yemabhenefiti makuru eNeural network pamusoro pechinyakare algorithms. Nehunyanzvi, kudzidza kunosanganisira kutsvaga iwo coefficients ekubatana pakati peeuroni. Munguva yekudzidzira maitiro, iyo neural network inokwanisa kuona kutsamira kwakaomarara pakati pekuisa data uye inobuda data, pamwe nekuita generalization.

Kubva pamaonero ekudzidza kwemuchina, neural network inyaya yakakosha yemaitiro ekucherekedza maitiro, ongororo yerusarura, nzira dzekubatanidza uye dzimwe nzira.

Zvigadzirwa

Kutanga, ngatitarisei midziyo. Tinoda sevha ine Linux inoshanda system yakaiswa pairi. Midziyo inodiwa kushandisa masisitimu ekudzidza muchina ine simba uye, semhedzisiro, inodhura. Kune avo vasina muchina wakanaka padyo, ini ndinokurudzira kutarisisa kune zvinopihwa nevanopa makore. Iwe unogona kuhaya server inodiwa nekukurumidza uye kubhadhara chete yenguva yekushandiswa.

Mune mapurojekiti pazvinofanirwa kugadzira neural network, ini ndinoshandisa maseva eimwe yeRussia cloud provider. Iyo kambani inopa maseva emakore ekurenda zvakanangana nekudzidza muchina ane simba Tesla V100 graphics processors (GPU) kubva kuNVIDIA. Muchidimbu: kushandisa sevha ine GPU kunogona kuita makumi enguva zvakanyanya kushanda (nekukurumidza) kana ichienzaniswa nesevha yemutengo wakafanana inoshandisa CPU (inozivikanwa pakati pekugadzirisa unit) yekuverenga. Izvi zvinowanikwa nekuda kwezvimiro zveGPU zvivakwa, izvo zvinosangana nekuverenga nekukurumidza.

Kuti tiite mienzaniso inotsanangurwa pazasi, takatenga inotevera sevha kwemazuva akati wandei:

• SSD dhisiki 150 GB
• RAM 32 GB
• Tesla V100 16 Gb processor ine 4 cores

Isu takaisa Ubuntu 18.04 pamushini wedu.

Kugadzira zvakatipoteredza

Iye zvino ngatiisei zvese zvinodiwa pakushanda pane server. Sezvo chinyorwa chedu chiri kunyanya kune vanotanga, ini ndichataura nezve mamwe mapoinzi achave anobatsira kwavari.

Basa rakawanda pakumisa nharaunda rinoitwa kuburikidza nemutsara wekuraira. Vazhinji vevashandisi vanoshandisa Windows seOs yavo inoshanda. Iyo yakajairwa koni mune iyi OS inosiya zvakawanda kuti zvidiwe. Naizvozvo, isu tichashandisa chishandiso chakanakira Cmder/. Dhawunirodha iyo mini vhezheni uye mhanya Cmder.exe. Tevere iwe unofanirwa kubatana kune server kuburikidza neSSH:

ssh root@server-ip-or-hostname

Panzvimbo pe server-ip-or-hostname, tsanangura iyo IP kero kana DNS zita reserver yako. Tevere, isa password uye kana kubatana kwakabudirira, isu tinofanirwa kugamuchira meseji yakafanana neiyi.

Welcome to Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-74-generic x86_64)

Mutauro mukuru wekugadzira mhando dzeML iPython. Uye iyo inonyanya kufarirwa chikuva chekushandisa kwayo paLinux ndeye Anaconda.

Ngatiiise pane yedu server.

Isu tinotanga nekuvandudza maneja wepakeji yemuno:

sudo apt-get update

Isa curl (command line utility):

sudo apt-get install curl

Dhaunirodha yazvino vhezheni yeAnaconda Distribution:

cd /tmp
curl –O https://repo.anaconda.com/archive/Anaconda3-2019.10-Linux-x86_64.sh

Ngatitange kuisa:

bash Anaconda3-2019.10-Linux-x86_64.sh

Panguva yekumisikidza, iwe unozokumbirwa kusimbisa chibvumirano cherezinesi. Pakugadziriswa kwakabudirira iwe unofanirwa kuona izvi:

Thank you for installing Anaconda3!

Mazhinji masisitimu ave ikozvino akagadzirwa ekuvandudza ML modhi; isu tinoshanda neanonyanya kufarirwa: PyTorch и tensor flow.

Kushandisa iyo dhizaini inobvumidza iwe kuti uwedzere kukurumidza kwekusimudzira uye shandisa akagadzirira-akagadzirwa maturusi emabasa akajairwa.

Mumuenzaniso uyu tichashanda nePyTorch. Ngatiiise:

conda install pytorch torchvision cudatoolkit=10.1 -c pytorch

Iye zvino tinoda kuvhura Jupyter Notebook, chishandiso chakakurumbira chekusimudzira cheML nyanzvi. Inokubvumira kunyora kodhi uye pakarepo kuona mhedzisiro yekuitwa kwayo. Jupyter Notebook inosanganisirwa neAnaconda uye yakatoiswa pane yedu server. Iwe unofanirwa kubatana nayo kubva kune yedu desktop system.

Kuti tiite izvi, isu tichatanga kuburitsa Jupyter pane server inotsanangura port 8080:

jupyter notebook --no-browser --port=8080 --allow-root

Tevere, kuvhura imwe tebhu mune yedu Cmder koni (yepamusoro menyu - Nyowani console dialog) isu tichabatana kuburikidza nechiteshi 8080 kune server kuburikidza neSSH:

ssh -L 8080:localhost:8080 root@server-ip-or-hostname

Kana isu tikapinda murairo wekutanga, isu tichapihwa zvinongedzo zvekuvhura Jupyter mubrowser yedu:

To access the notebook, open this file in a browser:
        file:///root/.local/share/jupyter/runtime/nbserver-18788-open.html
    Or copy and paste one of these URLs:
        http://localhost:8080/?token=cca0bd0b30857821194b9018a5394a4ed2322236f116d311
     or http://127.0.0.1:8080/?token=cca0bd0b30857821194b9018a5394a4ed2322236f116d311

Ngatishandisei chinongedzo che localhost:8080. Kopa iyo yakazara nzira uye unamate mukero bar yePC yako yemuno browser. Jupyter Notebook ichavhura.

Ngatigadzire bhuku idzva: Nyowani - Notebook - Python 3.

Ngatitarisei kushanda kwakarurama kwezvikamu zvese zvatakaisa. Ngatiisei muenzaniso wePyTorch kodhi muJupyter uye timhanye kuuraya (Mhanya bhatani):

from __future__ import print_function
import torch
x = torch.rand(5, 3)
print(x)

Mhedzisiro yacho inofanira kuva seizvi:

Kana iwe uine mhedzisiro yakafanana, saka isu takagadzirisa zvese nemazvo uye tinogona kutanga kugadzira neural network!

Kugadzira neural network

Isu tichagadzira neural network yekuzivikanwa kwemufananidzo. Ngatitorei izvi sehwaro gwara.

Isu tichashandisa iyo inowanikwa pachena CIFAR10 dhataseti kudzidzisa network. Iine makirasi: "ndege", "motokari", "shiri", "katsi", "nondo", "imbwa", "datya", "bhiza", "ngarava", "rori". Mifananidzo iri muCIFAR10 ndeye 3x32x32, kureva, 3-chiteshi mavara emifananidzo ye32x32 pixels.

Zvebasa, isu tichashandisa iyo package yakagadzirwa nePyTorch yekushanda nemifananidzo - torchvision.

Tichaita nhanho dzinotevera kuti:

• Kurodha uye normalizing kudzidziswa uye bvunzo data seti
• Neural Network Tsanangudzo
• Network kudzidziswa pane data yekudzidziswa
• Network test pane test data
• Ngatidzokorore kudzidziswa uye kuyedzwa tichishandisa GPU

Tichange tichiita kodhi yese pazasi muJupyter Notebook.

Kurodha uye normalizing CIFAR10

Kopa uye mhanyisa inotevera kodhi muJupyter:

import torch
import torchvision
import torchvision.transforms as transforms

transform = transforms.Compose(
    [transforms.ToTensor(),
     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])

trainset = torchvision.datasets.CIFAR10(root='./data', train=True,
                                        download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=4,
                                          shuffle=True, num_workers=2)

testset = torchvision.datasets.CIFAR10(root='./data', train=False,
                                       download=True, transform=transform)
testloader = torch.utils.data.DataLoader(testset, batch_size=4,
                                         shuffle=False, num_workers=2)

classes = ('plane', 'car', 'bird', 'cat',
           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')

Mhinduro inofanira kuva:

Downloading https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz to ./data/cifar-10-python.tar.gz
Extracting ./data/cifar-10-python.tar.gz to ./data
Files already downloaded and verified

Ngatiratidzei mifananidzo yakati wandei yekuyedza:

import matplotlib.pyplot as plt
import numpy as np

# functions to show an image

def imshow(img):
    img = img / 2 + 0.5     # unnormalize
    npimg = img.numpy()
    plt.imshow(np.transpose(npimg, (1, 2, 0)))
    plt.show()

# get some random training images
dataiter = iter(trainloader)
images, labels = dataiter.next()

# show images
imshow(torchvision.utils.make_grid(images))
# print labels
print(' '.join('%5s' % classes[labels[j]] for j in range(4)))

Neural Network Tsanangudzo

Ngatitangei tifunge kuti neural network yekuzivikanwa kwemufananidzo inoshanda sei. Iyi inzira yakapfava yekunongedzera-kunetiweki. Zvinotora data rekuisa, rinopfuura nepakati akati wandei rimwe nerimwe, uyezve pakupedzisira rinoburitsa yakabuda data.

Ngatigadzire network yakafanana munharaunda yedu:

import torch.nn as nn
import torch.nn.functional as F

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(3, 6, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(16 * 5 * 5, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = x.view(-1, 16 * 5 * 5)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x

net = Net()

Isu tinotsanangurawo basa rekurasikirwa uye optimizer

import torch.optim as optim

criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)

Network kudzidziswa pane data yekudzidziswa

Ngatitangei kudzidzisa yedu neural network. Ndokumbira utarise kuti mushure mekumhanyisa iyi kodhi, iwe uchafanirwa kumirira imwe nguva kusvika basa rapera. Zvakanditorera 5 minutes. Zvinotora nguva kudzidzisa network.

 for epoch in range(2):  # loop over the dataset multiple times

    running_loss = 0.0
    for i, data in enumerate(trainloader, 0):
        # get the inputs; data is a list of [inputs, labels]
        inputs, labels = data

        # zero the parameter gradients
        optimizer.zero_grad()

        # forward + backward + optimize
        outputs = net(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()

        # print statistics
        running_loss += loss.item()
        if i % 2000 == 1999:    # print every 2000 mini-batches
            print('[%d, %5d] loss: %.3f' %
                  (epoch + 1, i + 1, running_loss / 2000))
            running_loss = 0.0

print('Finished Training')

Tinowana mhedzisiro inotevera:

Isu tinochengetedza yedu yakadzidziswa modhi:

PATH = './cifar_net.pth'
torch.save(net.state_dict(), PATH)

Network test pane test data

Isu takadzidzisa network tichishandisa seti yedata rekudzidzisa. Asi isu tinofanirwa kutarisa kana network yakadzidza chero chinhu zvachose.

Isu tichayedza izvi nekufanotaura kirasi label iyo neural network inoburitsa uye kuiyedza kuti tione kana chiri chokwadi. Kana kufanotaura kuri kwechokwadi, tinowedzera sampuli kune runyorwa rwekufanotaura kwakaringana.
Ngatiratidze mufananidzo kubva muyedzo set:

dataiter = iter(testloader)
images, labels = dataiter.next()

# print images
imshow(torchvision.utils.make_grid(images))
print('GroundTruth: ', ' '.join('%5s' % classes[labels[j]] for j in range(4)))

Zvino ngatibvunzei neural network kutiudza zviri mumifananidzo iyi:

net = Net()
net.load_state_dict(torch.load(PATH))

outputs = net(images)

_, predicted = torch.max(outputs, 1)

print('Predicted: ', ' '.join('%5s' % classes[predicted[j]]
                              for j in range(4)))

Mhedzisiro yacho inoita kunge yakanaka: network yakanyatso ratidza matatu kubva pamifananidzo mina.

Ngationei kuti network inoita sei padhata rese rese.

correct = 0
total = 0
with torch.no_grad():
    for data in testloader:
        images, labels = data
        outputs = net(images)
        _, predicted = torch.max(outputs.data, 1)
        total += labels.size(0)
        correct += (predicted == labels).sum().item()

print('Accuracy of the network on the 10000 test images: %d %%' % (
    100 * correct / total))

Zvinoita sekunge network inoziva chimwe chinhu uye iri kushanda. Kana akasarudza makirasi chero nguva, iko kurongeka kwaizova 10%.

Zvino ngationei makirasi ayo network inozivisa zvirinani:

class_correct = list(0. for i in range(10))
class_total = list(0. for i in range(10))
with torch.no_grad():
    for data in testloader:
        images, labels = data
        outputs = net(images)
        _, predicted = torch.max(outputs, 1)
        c = (predicted == labels).squeeze()
        for i in range(4):
            label = labels[i]
            class_correct[label] += c[i].item()
            class_total[label] += 1


for i in range(10):
    print('Accuracy of %5s : %2d %%' % (
        classes[i], 100 * class_correct[i] / class_total[i]))

Zvinoita sekuti network ndiyo yakanakisa pakuziva mota nezvikepe: 71% chokwadi.

Saka network iri kushanda. Zvino ngatiedze kuendesa basa rayo kune graphics processor (GPU) uye tione kuti chii chinoshanduka.

Kudzidzisa neural network paGPU

Kutanga, ini ndichatsanangura muchidimbu kuti CUDA chii. CUDA (Compute Unified Device Architecture) ipuratifomu yekombuta yakafanana yakagadziridzwa neNVIDIA kune general computing pane graphics processing units (GPUs). NeCUDA, vanogadzira vanogona kumhanyisa zvinoshamisa mashandisirwo ekombuta nekusimudzira simba reGPUs. Iyi puratifomu yakatoiswa pane yedu server yatakatenga.

Ngatitangei tatsanangura GPU yedu seyekutanga inooneka cuda mudziyo.

device = torch . device ( "cuda:0" if torch . cuda . is_available () else "cpu" )
# Assuming that we are on a CUDA machine, this should print a CUDA device:
print ( device )

Kutumira network kuGPU:

net.to(device)

Tichazofanirawo kutumira zvinopinda uye zvinangwa padanho rega rega kuGPU:

inputs, labels = data[0].to(device), data[1].to(device)

Ngatidzidzisei zvakare network paGPU:

import torch.optim as optim

criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)
for epoch in range(2):  # loop over the dataset multiple times

    running_loss = 0.0
    for i, data in enumerate(trainloader, 0):
        # get the inputs; data is a list of [inputs, labels]
    inputs, labels = data[0].to(device), data[1].to(device)

        # zero the parameter gradients
        optimizer.zero_grad()

        # forward + backward + optimize
        outputs = net(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()

        # print statistics
        running_loss += loss.item()
        if i % 2000 == 1999:    # print every 2000 mini-batches
            print('[%d, %5d] loss: %.3f' %
                  (epoch + 1, i + 1, running_loss / 2000))
            running_loss = 0.0

print('Finished Training')

Panguva ino, kudzidziswa kwetiweki kwakatora anenge maminetsi matatu. Ngatiyeukei kuti nhanho imwe chete pane yakajairwa processor yakatora maminetsi mashanu. Musiyano hauna kukosha, izvi zvinoitika nekuti network yedu haina kukura zvakanyanya. Paunenge uchishandisa mahombe akakura ekudzidziswa, mutsauko uripo pakati pekumhanya kweGPU uye yechinyakare processor uchawedzera.

Zvinoita sezviri zvese. Zvatakakwanisa kuita:

• Takatarisa kuti GPU chii uye takasarudza sevha iyo yakaiswa;
• Isu takagadzira nharaunda yesoftware kugadzira neural network;
• Isu takagadzira neural network yekuzivikanwa kwemufananidzo uye takaidzidzisa;
• Takadzokorora kudzidziswa kwenetiweki tichishandisa GPU uye takagamuchira kuwedzera kwekumhanya.

Ndichafara kupindura mibvunzo mumhinduro.

Source: www.habr.com