🥇UkuQatshelwa koMzobo oKhawulezayo: ukwenza ubuhlobo ne-R, C++ kunye neenethiwekhi ze-neural

Hayi Habr!

Kwikwindla yokugqibela, uKaggle uye wabamba ukhuphiswano lokwahlula imifanekiso ezotywe ngesandla, ukuQatshelwa koMzobo oKhawulezayo weDoodle, apho, phakathi kwezinye, iqela leengcali ze-R zathatha inxaxheba: Artem Klevtsova, Umphathi wePhilippa и UAndrey Ogurtsov. Asizukuluchaza ngokweenkcukacha ukhuphiswano; esele yenziwe upapasho lwakutsha nje.

Ngeli xesha akuzange kusebenze ngokulima iimbasa, kodwa kufunyenwe amava amaninzi axabisekileyo, ngoko ke ndingathanda ukuxelela uluntu malunga nenani lezona zinto zinomdla kwaye ziluncedo kwi-Kagle kunye nomsebenzi wemihla ngemihla. Phakathi kwezihloko ezixutyushwayo: ubomi obunzima ngaphandle I-OpenCV, JSON ucazululo (le mizekelo ivavanya indibaniselwano yekhowudi ye C++ kwizikripthi okanye iipakethe kwi-R usebenzisa Rcpp), i-parameterization yezikripthi kunye ne-dockerization yesisombululo sokugqibela. Yonke ikhowudi evela kumyalezo kwifomu efanelekileyo ukuphunyezwa iyafumaneka iindawo zokugcina.

Iziqulatho:

Layisha ngokufanelekileyo idatha esuka kwi-CSV ukuya kwi-MonetDB
Ukulungiselela iibhetshi
Iiterators zokukhuphela iibhetshi ukusuka kuvimba weenkcukacha
Ukukhetha i-Architecture yeModeli
Iparameterization yombhalo
Ukwenziwa kwee-scripts
Ukusebenzisa ii-GPU ezininzi kwiLifu likaGoogle
Endaweni yesiphelo

1. Layisha ngokufanelekileyo idatha esuka kwi-CSV kwi-database ye-MonetDB

Idatha yolu khuphiswano ayibonelelwanga ngendlela yemifanekiso esele yenziwe, kodwa kwiifayile ze-CSV ze-340 (ifayile enye kwiklasi nganye) equkethe ii-JSON kunye ne-point coordinates. Ngokudibanisa la manqaku ngemigca, sifumana umfanekiso wokugqibela olinganisa iipikseli ezingama-256x256. Kwakhona kwirekhodi nganye kukho ileyibhile ebonisa ukuba umfanekiso uqatshelwe ngokuchanekileyo ngumdidi osetyenziswa ngexesha lokuqokelela idatha, ikhowudi enoonobumba ababini belizwe ahlala kulo umbhali womfanekiso, isazisi esisodwa, isitampu sexesha. kunye negama leklasi elihambelana negama lefayile. Inguqu eyenziwe lula yedatha yasekuqaleni ilinganisa i-7.4 GB kwi-archive kwaye malunga ne-20 GB emva kokukhupha, idatha epheleleyo emva kokukhupha ithatha i-240 GB. Abaququzeleli baqinisekisa ukuba zombini iinguqulelo zivelisa imizobo efanayo, okuthetha ukuba inguqulelo epheleleyo yayingafuneki. Kwimeko nayiphi na imeko, ukugcina imifanekiso yezigidi ezingama-50 kwiifayile zegraphic okanye ngendlela yoluhlu kwakhawuleza kwajongwa njengengenangeniso, kwaye sagqiba ekubeni sidibanise zonke iifayile ze-CSV kwindawo yokugcina. train_simplified.zip kwisiseko sedatha kunye nesizukulwana esilandelayo semifanekiso yobungakanani obufunekayo "kwi-fly" kwibhetshi nganye.

Inkqubo eqinisekisiweyo kakuhle yakhethwa njenge-DBMS I-MonetDB, oko kukuthi ukuphunyezwa kwe-R njengomqulu I-MonetDBLite. Iphakheji ibandakanya inguqu edibeneyo yeseva yedatha kwaye ikuvumela ukuba uthabathe umncedisi ngokuthe ngqo kwiseshoni ye-R kwaye usebenze nayo apho. Ukudala isiseko sedatha kunye nokudibanisa kuyo kwenziwa ngomyalelo omnye:

con <- DBI::dbConnect(drv = MonetDBLite::MonetDBLite(), Sys.getenv("DBDIR"))

Kuya kufuneka senze iitheyibhile ezimbini: enye yazo zonke iinkcukacha, enye inolwazi lwenkonzo malunga neefayile ezikhutshelweyo (iluncedo ukuba kukho into engahambi kakuhle kwaye inkqubo kufuneka iqalise kwakhona emva kokukhuphela iifayile ezininzi):

Ukudala iitafile

if (!DBI::dbExistsTable(con, "doodles")) {
  DBI::dbCreateTable(
    con = con,
    name = "doodles",
    fields = c(
      "countrycode" = "char(2)",
      "drawing" = "text",
      "key_id" = "bigint",
      "recognized" = "bool",
      "timestamp" = "timestamp",
      "word" = "text"
    )
  )
}

if (!DBI::dbExistsTable(con, "upload_log")) {
  DBI::dbCreateTable(
    con = con,
    name = "upload_log",
    fields = c(
      "id" = "serial",
      "file_name" = "text UNIQUE",
      "uploaded" = "bool DEFAULT false"
    )
  )
}

Indlela ekhawulezayo yokulayisha idatha kwisiseko sedatha yayikukukopa ngokuthe ngqo iifayile ze-CSV usebenzisa i-SQL - umyalelo COPY OFFSET 2 INTO tablename FROM path USING DELIMITERS ',','n','"' NULL AS '' BEST EFFORTphi tablename - igama letafile kunye path - indlela eya kwifayile. Ngelixa usebenza nogcino, kwafunyaniswa ukuba ukuphunyezwa eyakhelwe-ngaphakathi unzip kwi-R ayisebenzi ngokuchanekileyo ngenani leefayile ezisuka kwindawo yokugcina, ngoko ke sisebenzise inkqubo unzip (usebenzisa iparameter getOption("unzip")).

Umsebenzi wokubhala kuvimba weenkcukacha

#' @title Извлечение и загрузка файлов
#'
#' @description
#' Извлечение CSV-файлов из ZIP-архива и загрузка их в базу данных
#'
#' @param con Объект подключения к базе данных (класс `MonetDBEmbeddedConnection`).
#' @param tablename Название таблицы в базе данных.
#' @oaram zipfile Путь к ZIP-архиву.
#' @oaram filename Имя файла внури ZIP-архива.
#' @param preprocess Функция предобработки, которая будет применена извлечённому файлу.
#'   Должна принимать один аргумент `data` (объект `data.table`).
#'
#' @return `TRUE`.
#'
upload_file <- function(con, tablename, zipfile, filename, preprocess = NULL) {
  # Проверка аргументов
  checkmate::assert_class(con, "MonetDBEmbeddedConnection")
  checkmate::assert_string(tablename)
  checkmate::assert_string(filename)
  checkmate::assert_true(DBI::dbExistsTable(con, tablename))
  checkmate::assert_file_exists(zipfile, access = "r", extension = "zip")
  checkmate::assert_function(preprocess, args = c("data"), null.ok = TRUE)

  # Извлечение файла
  path <- file.path(tempdir(), filename)
  unzip(zipfile, files = filename, exdir = tempdir(), 
        junkpaths = TRUE, unzip = getOption("unzip"))
  on.exit(unlink(file.path(path)))

  # Применяем функция предобработки
  if (!is.null(preprocess)) {
    .data <- data.table::fread(file = path)
    .data <- preprocess(data = .data)
    data.table::fwrite(x = .data, file = path, append = FALSE)
    rm(.data)
  }

  # Запрос к БД на импорт CSV
  sql <- sprintf(
    "COPY OFFSET 2 INTO %s FROM '%s' USING DELIMITERS ',','n','"' NULL AS '' BEST EFFORT",
    tablename, path
  )
  # Выполнение запроса к БД
  DBI::dbExecute(con, sql)

  # Добавление записи об успешной загрузке в служебную таблицу
  DBI::dbExecute(con, sprintf("INSERT INTO upload_log(file_name, uploaded) VALUES('%s', true)",
                              filename))

  return(invisible(TRUE))
}

Ukuba ufuna ukuguqula itafile ngaphambi kokuba uyibhale kwisiseko sedatha, kwanele ukudlula kwingxabano preprocess umsebenzi oza kuguqula idatha.

Ikhowudi yokulayisha idatha ngokulandelelana kwisiseko sedatha:

Ukubhala idatha kwisiseko sedatha

# Список файлов для записи
files <- unzip(zipfile, list = TRUE)$Name

# Список исключений, если часть файлов уже была загружена
to_skip <- DBI::dbGetQuery(con, "SELECT file_name FROM upload_log")[[1L]]
files <- setdiff(files, to_skip)

if (length(files) > 0L) {
  # Запускаем таймер
  tictoc::tic()
  # Прогресс бар
  pb <- txtProgressBar(min = 0L, max = length(files), style = 3)
  for (i in seq_along(files)) {
    upload_file(con = con, tablename = "doodles", 
                zipfile = zipfile, filename = files[i])
    setTxtProgressBar(pb, i)
  }
  close(pb)
  # Останавливаем таймер
  tictoc::toc()
}

# 526.141 sec elapsed - копирование SSD->SSD
# 558.879 sec elapsed - копирование USB->SSD

Ixesha lokulayisha idatha linokwahluka ngokuxhomekeke kwiimpawu zesantya se-drive esetyenzisiweyo. Kwimeko yethu, ukufunda nokubhala ngaphakathi kwe-SSD okanye kwi-flash drive (ifayile yomthombo) kwi-SSD (DB) ithatha ngaphantsi kwemizuzu eyi-10.

Kuthatha imizuzwana embalwa ukwenza umhlathi oneleyibhile yodidi olupheleleyo kunye nomhlathi wesalathisi (ORDERED INDEX) ngamanani emigca apho imigqaliselo iya kwenziwa iisampulu xa kuyilwa iibhetshi:

Ukudala iiKholamu ezongezelelweyo kunye neSalathiso

message("Generate lables")
invisible(DBI::dbExecute(con, "ALTER TABLE doodles ADD label_int int"))
invisible(DBI::dbExecute(con, "UPDATE doodles SET label_int = dense_rank() OVER (ORDER BY word) - 1"))

message("Generate row numbers")
invisible(DBI::dbExecute(con, "ALTER TABLE doodles ADD id serial"))
invisible(DBI::dbExecute(con, "CREATE ORDERED INDEX doodles_id_ord_idx ON doodles(id)"))

Ukusombulula ingxaki yokudala ibhetshi kubhabho, kufuneka sifikelele esona santya siphezulu sokukhupha imiqolo engacwangciswanga kwitafile. doodles. Kule nto sisebenzise amaqhinga ama-3. Eyokuqala ibikukunciphisa idimensionality yohlobo olugcina i-ID yokujonga. Kwiseti yedatha yokuqala, uhlobo olufunekayo ukugcina i-ID bigint, kodwa inani lemigqaliselo lenza kube lula ukudibanisa iziphawuli zazo, zilingana nenani le-ordinal, kudidi int. Ukukhangela kukhawuleza kakhulu kule meko. Iqhinga lesibini yayikukusebenzisa ORDERED INDEX - sifike kwesi sigqibo ngamandla, sele sihambe kuzo zonke ezikhoyo ukhetho. Eyesithathu ibikukusebenzisa imibuzo eneparameter. Undoqo wendlela kukuphumeza umyalelo kube kanye PREPARE ngokusetyenziswa okulandelayo kwentetho elungisiweyo xa usenza iqela lemibuzo yohlobo olufanayo, kodwa eneneni kukho inzuzo xa kuthelekiswa nenye elula. SELECT kuvele ukuba phakathi koluhlu lwempazamo yeenkcukacha-manani.

Inkqubo yokulayisha idatha ayidli ngaphezu kwe-450 MB ye-RAM. Oko kukuthi, indlela echaziweyo ikuvumela ukuba uhambise iiseti zedatha ezinobunzima bamashumi egigabytes phantse kuyo nayiphi na i-hardware yebhajethi, kubandakanywa nezixhobo zebhodi enye, epholile kakhulu.

Ekuphela kwento eseleyo kukulinganisa isantya sokufumana kwakhona (ngokungacwangciswanga) idatha kwaye uvavanye isikali xa kusenziwa iisampulu zeebhetshi zobukhulu obahlukeneyo:

Umgangatho wesiseko sedata

library(ggplot2)

set.seed(0)
# Подключение к базе данных
con <- DBI::dbConnect(MonetDBLite::MonetDBLite(), Sys.getenv("DBDIR"))

# Функция для подготовки запроса на стороне сервера
prep_sql <- function(batch_size) {
  sql <- sprintf("PREPARE SELECT id FROM doodles WHERE id IN (%s)",
                 paste(rep("?", batch_size), collapse = ","))
  res <- DBI::dbSendQuery(con, sql)
  return(res)
}

# Функция для извлечения данных
fetch_data <- function(rs, batch_size) {
  ids <- sample(seq_len(n), batch_size)
  res <- DBI::dbFetch(DBI::dbBind(rs, as.list(ids)))
  return(res)
}

# Проведение замера
res_bench <- bench::press(
  batch_size = 2^(4:10),
  {
    rs <- prep_sql(batch_size)
    bench::mark(
      fetch_data(rs, batch_size),
      min_iterations = 50L
    )
  }
)
# Параметры бенчмарка
cols <- c("batch_size", "min", "median", "max", "itr/sec", "total_time", "n_itr")
res_bench[, cols]

#   batch_size      min   median      max `itr/sec` total_time n_itr
#        <dbl> <bch:tm> <bch:tm> <bch:tm>     <dbl>   <bch:tm> <int>
# 1         16   23.6ms  54.02ms  93.43ms     18.8        2.6s    49
# 2         32     38ms  84.83ms 151.55ms     11.4       4.29s    49
# 3         64   63.3ms 175.54ms 248.94ms     5.85       8.54s    50
# 4        128   83.2ms 341.52ms 496.24ms     3.00      16.69s    50
# 5        256  232.8ms 653.21ms 847.44ms     1.58      31.66s    50
# 6        512  784.6ms    1.41s    1.98s     0.740       1.1m    49
# 7       1024  681.7ms    2.72s    4.06s     0.377      2.16m    49

ggplot(res_bench, aes(x = factor(batch_size), y = median, group = 1)) +
  geom_point() +
  geom_line() +
  ylab("median time, s") +
  theme_minimal()

DBI::dbDisconnect(con, shutdown = TRUE)

2. Ukulungiselela iibhetshi

Yonke inkqubo yokulungiselela ibhetshi inala manyathelo alandelayo:

Ukwahlulahlula ii-JSON ezininzi eziqulathe iivektha zemitya kunye nolungelelwaniso lwamanqaku.
Ukudweba imigca enemibala esekelwe kukulungelelaniswa kwamanqaku kumfanekiso wobungakanani obufunekayo (umzekelo, 256 × 256 okanye 128 × 128).
Ukuguqula imifanekiso enesiphumo ibe yi-tensor.

Njengenxalenye yokhuphiswano phakathi kwePython kernels, ingxaki yasonjululwa ngokuyintloko kusetyenziswa I-OpenCV. Enye yezona zifaniso zilula nezicacileyo kwi-R zinokujongeka ngolu hlobo:

Ukusebenzisa i-JSON kwi-Tensor Conversion kwi-R

r_process_json_str <- function(json, line.width = 3, 
                               color = TRUE, scale = 1) {
  # Парсинг JSON
  coords <- jsonlite::fromJSON(json, simplifyMatrix = FALSE)
  tmp <- tempfile()
  # Удаляем временный файл по завершению функции
  on.exit(unlink(tmp))
  png(filename = tmp, width = 256 * scale, height = 256 * scale, pointsize = 1)
  # Пустой график
  plot.new()
  # Размер окна графика
  plot.window(xlim = c(256 * scale, 0), ylim = c(256 * scale, 0))
  # Цвета линий
  cols <- if (color) rainbow(length(coords)) else "#000000"
  for (i in seq_along(coords)) {
    lines(x = coords[[i]][[1]] * scale, y = coords[[i]][[2]] * scale, 
          col = cols[i], lwd = line.width)
  }
  dev.off()
  # Преобразование изображения в 3-х мерный массив
  res <- png::readPNG(tmp)
  return(res)
}

r_process_json_vector <- function(x, ...) {
  res <- lapply(x, r_process_json_str, ...)
  # Объединение 3-х мерных массивов картинок в 4-х мерный в тензор
  res <- do.call(abind::abind, c(res, along = 0))
  return(res)
}

Umzobo wenziwa kusetyenziswa izixhobo ezisemgangathweni ze-R kwaye zigcinwe kwi-PNG yethutyana egcinwe kwi-RAM (kwi-Linux, abalawuli bexeshana be-R babekwe kulawulo. /tmp, ifakwe kwi-RAM). Le fayile emva koko ifundwe njenge-dimensional array enamanani ukusuka ku-0 ukuya ku-1. Oku kubalulekile kuba i-BMP eqhelekileyo ingafundwa kwi-array ekrwada enekhowudi zemibala ye-hex.

Masivavanye iziphumo:

zip_file <- file.path("data", "train_simplified.zip")
csv_file <- "cat.csv"
unzip(zip_file, files = csv_file, exdir = tempdir(), 
      junkpaths = TRUE, unzip = getOption("unzip"))
tmp_data <- data.table::fread(file.path(tempdir(), csv_file), sep = ",", 
                              select = "drawing", nrows = 10000)
arr <- r_process_json_str(tmp_data[4, drawing])
dim(arr)
# [1] 256 256   3
plot(magick::image_read(arr))

Ibhetshi ngokwayo iya kwenziwa ngolu hlobo lulandelayo:

res <- r_process_json_vector(tmp_data[1:4, drawing], scale = 0.5)
str(res)
 # num [1:4, 1:128, 1:128, 1:3] 1 1 1 1 1 1 1 1 1 1 ...
 # - attr(*, "dimnames")=List of 4
 #  ..$ : NULL
 #  ..$ : NULL
 #  ..$ : NULL
 #  ..$ : NULL

Oku kuzalisekiswa kubonakala kungafanelekanga kuthi, kuba ukwenziwa kweebhetshi ezinkulu kuthatha ixesha elide ngokungafanelekanga, kwaye sagqiba ekubeni sithathe ithuba lamava oogxa bethu ngokusebenzisa ithala leencwadi elinamandla. I-OpenCV. Ngelo xesha kwakungekho phakheji isele ilungile ye-R (akukho nanye ngoku), ngoko ke ukuphunyezwa okuncinci komsebenzi ofunekayo kwabhalwa kwi-C ++ kunye nokuhlanganiswa kwikhowudi ye-R usebenzisa. Rcpp.

Ukusombulula le ngxaki, ezi phakheji zilandelayo kunye namathala eencwadi asetyenzisiweyo:

I-OpenCV ukusebenza ngemifanekiso kunye nemigca yokuzoba. Kusetyenziswe iilayibrari zesistim esele zifakwe ngaphambili kunye neefayile zeheader, kunye nonxibelelwano oluguquguqukayo.
xtensor ukusebenza nge-multidimensional arrays kunye ne-tensor. Sisebenzise iifayile zeheader ezifakwe kwiphakheji ye-R yegama elifanayo. Ithala leencwadi likuvumela ukuba usebenze ngee-multidimensional arrays, zombini kumqolo omkhulu kunye nolandelelwano olukhulu lwekholamu.
ndjson yokwahlulahlula i-JSON. Eli thala lisetyenziswa kwi xtensor ngokuzenzekelayo ukuba ikhona kwiprojekthi.
RcppTread ukulungiselela ukusetyenzwa kwemisonto emininzi yevektha esuka kwi-JSON. Kusetyenziswe iifayile zeheader ezinikezwe ngulo mqulu. Ukusuka edume kakhulu RcppParallel Iphakheji, phakathi kwezinye izinto, inendawo eyakhelweyo yokuphazamisa i-loop.

Kufanelekile ukuphawula oko xtensor yajika yaba yi-godsend: ukongeza kwinto yokuba inomsebenzi obanzi kunye nokusebenza okuphezulu, abaphuhlisi bayo baye baphendula kwaye baphendula imibuzo ngokukhawuleza nangeenkcukacha. Ngoncedo lwabo, kwakunokwenzeka ukuphumeza ukuguqulwa kweematriki ze-OpenCV kwi-xtensors ye-xtensors, kunye nendlela yokudibanisa i-tensor yomfanekiso we-3-dimensional kwi-tensor ye-4-dimensional ye-dimension echanekileyo (ibhetshi ngokwayo).

Izixhobo zokufunda iRcpp, xtensor kunye neRcppThread

https://thecoatlessprofessor.com/programming/unofficial-rcpp-api-documentation

https://docs.opencv.org/4.0.1/d7/dbd/group__imgproc.html

https://xtensor.readthedocs.io/en/latest/

https://xtensor.readthedocs.io/en/latest/file_loading.html#loading-json-data-into-xtensor

https://cran.r-project.org/web/packages/RcppThread/vignettes/RcppThread-vignette.pdf

Ukuqokelela iifayile ezisebenzisa iifayile zenkqubo kunye nonxulumano oluguquguqukayo kunye namathala eencwadi afakwe kwisixokelelwano, sisebenzise indlela yeplagin ephunyeziweyo kwiphakheji. Rcpp. Ukufumana ngokuzenzekelayo iindlela kunye neeflegi, sisebenzise into eyaziwayo yeLinux pkg-cwangcisa.

Ukuphunyezwa kweplagi ye-Rcpp ekusebenziseni ithala leencwadi le-OpenCV

Rcpp::registerPlugin("opencv", function() {
  # Возможные названия пакета
  pkg_config_name <- c("opencv", "opencv4")
  # Бинарный файл утилиты pkg-config
  pkg_config_bin <- Sys.which("pkg-config")
  # Проврека наличия утилиты в системе
  checkmate::assert_file_exists(pkg_config_bin, access = "x")
  # Проверка наличия файла настроек OpenCV для pkg-config
  check <- sapply(pkg_config_name, 
                  function(pkg) system(paste(pkg_config_bin, pkg)))
  if (all(check != 0)) {
    stop("OpenCV config for the pkg-config not found", call. = FALSE)
  }

  pkg_config_name <- pkg_config_name[check == 0]
  list(env = list(
    PKG_CXXFLAGS = system(paste(pkg_config_bin, "--cflags", pkg_config_name), 
                          intern = TRUE),
    PKG_LIBS = system(paste(pkg_config_bin, "--libs", pkg_config_name), 
                      intern = TRUE)
  ))
})

Njengesiphumo sokusebenza kwe-plugin, ezi xabiso zilandelayo ziya kutshintshwa ngexesha lenkqubo yokudibanisa:

Rcpp:::.plugins$opencv()$env

# $PKG_CXXFLAGS
# [1] "-I/usr/include/opencv"
#
# $PKG_LIBS
# [1] "-lopencv_shape -lopencv_stitching -lopencv_superres -lopencv_videostab -lopencv_aruco -lopencv_bgsegm -lopencv_bioinspired -lopencv_ccalib -lopencv_datasets -lopencv_dpm -lopencv_face -lopencv_freetype -lopencv_fuzzy -lopencv_hdf -lopencv_line_descriptor -lopencv_optflow -lopencv_video -lopencv_plot -lopencv_reg -lopencv_saliency -lopencv_stereo -lopencv_structured_light -lopencv_phase_unwrapping -lopencv_rgbd -lopencv_viz -lopencv_surface_matching -lopencv_text -lopencv_ximgproc -lopencv_calib3d -lopencv_features2d -lopencv_flann -lopencv_xobjdetect -lopencv_objdetect -lopencv_ml -lopencv_xphoto -lopencv_highgui -lopencv_videoio -lopencv_imgcodecs -lopencv_photo -lopencv_imgproc -lopencv_core"

Ikhowudi yokuphunyezwa yokwahlulahlula i-JSON kunye nokuvelisa ibhetshi yokudluliselwa kwimodeli inikwe phantsi kombhobho. Okokuqala, yongeza ulawulo lweprojekthi yendawo ukukhangela iifayile zeheader (ezifunekayo kwi-ndjson):

Sys.setenv("PKG_CXXFLAGS" = paste0("-I", normalizePath(file.path("src"))))

Ukuphunyezwa kwe-JSON ukuya kuguqulelo lwe-tensor kwi-C++

// [[Rcpp::plugins(cpp14)]]
// [[Rcpp::plugins(opencv)]]
// [[Rcpp::depends(xtensor)]]
// [[Rcpp::depends(RcppThread)]]

#include <xtensor/xjson.hpp>
#include <xtensor/xadapt.hpp>
#include <xtensor/xview.hpp>
#include <xtensor-r/rtensor.hpp>
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <Rcpp.h>
#include <RcppThread.h>

// Синонимы для типов
using RcppThread::parallelFor;
using json = nlohmann::json;
using points = xt::xtensor<double,2>;     // Извлечённые из JSON координаты точек
using strokes = std::vector<points>;      // Извлечённые из JSON координаты точек
using xtensor3d = xt::xtensor<double, 3>; // Тензор для хранения матрицы изоображения
using xtensor4d = xt::xtensor<double, 4>; // Тензор для хранения множества изображений
using rtensor3d = xt::rtensor<double, 3>; // Обёртка для экспорта в R
using rtensor4d = xt::rtensor<double, 4>; // Обёртка для экспорта в R

// Статические константы
// Размер изображения в пикселях
const static int SIZE = 256;
// Тип линии
// См. https://en.wikipedia.org/wiki/Pixel_connectivity#2-dimensional
const static int LINE_TYPE = cv::LINE_4;
// Толщина линии в пикселях
const static int LINE_WIDTH = 3;
// Алгоритм ресайза
// https://docs.opencv.org/3.1.0/da/d54/group__imgproc__transform.html#ga5bb5a1fea74ea38e1a5445ca803ff121
const static int RESIZE_TYPE = cv::INTER_LINEAR;

// Шаблон для конвертирования OpenCV-матрицы в тензор
template <typename T, int NCH, typename XT=xt::xtensor<T,3,xt::layout_type::column_major>>
XT to_xt(const cv::Mat_<cv::Vec<T, NCH>>& src) {
  // Размерность целевого тензора
  std::vector<int> shape = {src.rows, src.cols, NCH};
  // Общее количество элементов в массиве
  size_t size = src.total() * NCH;
  // Преобразование cv::Mat в xt::xtensor
  XT res = xt::adapt((T*) src.data, size, xt::no_ownership(), shape);
  return res;
}

// Преобразование JSON в список координат точек
strokes parse_json(const std::string& x) {
  auto j = json::parse(x);
  // Результат парсинга должен быть массивом
  if (!j.is_array()) {
    throw std::runtime_error("'x' must be JSON array.");
  }
  strokes res;
  res.reserve(j.size());
  for (const auto& a: j) {
    // Каждый элемент массива должен быть 2-мерным массивом
    if (!a.is_array() || a.size() != 2) {
      throw std::runtime_error("'x' must include only 2d arrays.");
    }
    // Извлечение вектора точек
    auto p = a.get<points>();
    res.push_back(p);
  }
  return res;
}

// Отрисовка линий
// Цвета HSV
cv::Mat ocv_draw_lines(const strokes& x, bool color = true) {
  // Исходный тип матрицы
  auto stype = color ? CV_8UC3 : CV_8UC1;
  // Итоговый тип матрицы
  auto dtype = color ? CV_32FC3 : CV_32FC1;
  auto bg = color ? cv::Scalar(0, 0, 255) : cv::Scalar(255);
  auto col = color ? cv::Scalar(0, 255, 220) : cv::Scalar(0);
  cv::Mat img = cv::Mat(SIZE, SIZE, stype, bg);
  // Количество линий
  size_t n = x.size();
  for (const auto& s: x) {
    // Количество точек в линии
    size_t n_points = s.shape()[1];
    for (size_t i = 0; i < n_points - 1; ++i) {
      // Точка начала штриха
      cv::Point from(s(0, i), s(1, i));
      // Точка окончания штриха
      cv::Point to(s(0, i + 1), s(1, i + 1));
      // Отрисовка линии
      cv::line(img, from, to, col, LINE_WIDTH, LINE_TYPE);
    }
    if (color) {
      // Меняем цвет линии
      col[0] += 180 / n;
    }
  }
  if (color) {
    // Меняем цветовое представление на RGB
    cv::cvtColor(img, img, cv::COLOR_HSV2RGB);
  }
  // Меняем формат представления на float32 с диапазоном [0, 1]
  img.convertTo(img, dtype, 1 / 255.0);
  return img;
}

// Обработка JSON и получение тензора с данными изображения
xtensor3d process(const std::string& x, double scale = 1.0, bool color = true) {
  auto p = parse_json(x);
  auto img = ocv_draw_lines(p, color);
  if (scale != 1) {
    cv::Mat out;
    cv::resize(img, out, cv::Size(), scale, scale, RESIZE_TYPE);
    cv::swap(img, out);
    out.release();
  }
  xtensor3d arr = color ? to_xt<double,3>(img) : to_xt<double,1>(img);
  return arr;
}

// [[Rcpp::export]]
rtensor3d cpp_process_json_str(const std::string& x, 
                               double scale = 1.0, 
                               bool color = true) {
  xtensor3d res = process(x, scale, color);
  return res;
}

// [[Rcpp::export]]
rtensor4d cpp_process_json_vector(const std::vector<std::string>& x, 
                                  double scale = 1.0, 
                                  bool color = false) {
  size_t n = x.size();
  size_t dim = floor(SIZE * scale);
  size_t channels = color ? 3 : 1;
  xtensor4d res({n, dim, dim, channels});
  parallelFor(0, n, [&x, &res, scale, color](int i) {
    xtensor3d tmp = process(x[i], scale, color);
    auto view = xt::view(res, i, xt::all(), xt::all(), xt::all());
    view = tmp;
  });
  return res;
}

Le khowudi kufuneka ifakwe kwifayile src/cv_xt.cpp kwaye uqokelele kunye nomyalelo Rcpp::sourceCpp(file = "src/cv_xt.cpp", env = .GlobalEnv); nayo iyafuneka emsebenzini nlohmann/json.hpp из indawo yokugcina. Ikhowudi yahlulwe ngokwemisebenzi emininzi:

to_xt - umsebenzi owenziweyo wokuguqula umfanekiso wematrix (cv::Mat) ukuya kwi-tensor xt::xtensor;
parse_json - umsebenzi ucazulula umtya we-JSON, ukhuphe ulungelelwaniso lwamanqaku, uwafake kwi-vector;
ocv_draw_lines - ukusuka kwi-vector yesiphumo samanqaku, idonsa imigca enemibala emininzi;
process - idibanisa le misebenzi ingentla kwaye yongeza ukukwazi ukukala umfanekiso obangelwayo;
cpp_process_json_str - usonga phezu komsebenzi process, ethumela ngaphandle isiphumo kwi-R-object (i-multidimensional array);
cpp_process_json_vector - usonga phezu komsebenzi cpp_process_json_str, ekuvumela ukuba uqhubekisele phambili umtya wevektha kwimowudi enemisonto emininzi.

Ukuzoba imigca enemibala emininzi, imodeli yombala we-HSV yasetyenziswa, ilandelwa kukuguqulwa kwe-RGB. Masivavanye iziphumo:

arr <- cpp_process_json_str(tmp_data[4, drawing])
dim(arr)
# [1] 256 256   3
plot(magick::image_read(arr))

Ukuthelekiswa kwesantya sokuphunyezwa kwi-R kunye ne-C ++

res_bench <- bench::mark(
  r_process_json_str(tmp_data[4, drawing], scale = 0.5),
  cpp_process_json_str(tmp_data[4, drawing], scale = 0.5),
  check = FALSE,
  min_iterations = 100
)
# Параметры бенчмарка
cols <- c("expression", "min", "median", "max", "itr/sec", "total_time", "n_itr")
res_bench[, cols]

#   expression                min     median       max `itr/sec` total_time  n_itr
#   <chr>                <bch:tm>   <bch:tm>  <bch:tm>     <dbl>   <bch:tm>  <int>
# 1 r_process_json_str     3.49ms     3.55ms    4.47ms      273.      490ms    134
# 2 cpp_process_json_str   1.94ms     2.02ms    5.32ms      489.      497ms    243

library(ggplot2)
# Проведение замера
res_bench <- bench::press(
  batch_size = 2^(4:10),
  {
    .data <- tmp_data[sample(seq_len(.N), batch_size), drawing]
    bench::mark(
      r_process_json_vector(.data, scale = 0.5),
      cpp_process_json_vector(.data,  scale = 0.5),
      min_iterations = 50,
      check = FALSE
    )
  }
)

res_bench[, cols]

#    expression   batch_size      min   median      max `itr/sec` total_time n_itr
#    <chr>             <dbl> <bch:tm> <bch:tm> <bch:tm>     <dbl>   <bch:tm> <int>
#  1 r                   16   50.61ms  53.34ms  54.82ms    19.1     471.13ms     9
#  2 cpp                 16    4.46ms   5.39ms   7.78ms   192.      474.09ms    91
#  3 r                   32   105.7ms 109.74ms 212.26ms     7.69        6.5s    50
#  4 cpp                 32    7.76ms  10.97ms  15.23ms    95.6     522.78ms    50
#  5 r                   64  211.41ms 226.18ms 332.65ms     3.85      12.99s    50
#  6 cpp                 64   25.09ms  27.34ms  32.04ms    36.0        1.39s    50
#  7 r                  128   534.5ms 627.92ms 659.08ms     1.61      31.03s    50
#  8 cpp                128   56.37ms  58.46ms  66.03ms    16.9        2.95s    50
#  9 r                  256     1.15s    1.18s    1.29s     0.851     58.78s    50
# 10 cpp                256  114.97ms 117.39ms 130.09ms     8.45       5.92s    50
# 11 r                  512     2.09s    2.15s    2.32s     0.463       1.8m    50
# 12 cpp                512  230.81ms  235.6ms 261.99ms     4.18      11.97s    50
# 13 r                 1024        4s    4.22s     4.4s     0.238       3.5m    50
# 14 cpp               1024  410.48ms 431.43ms 462.44ms     2.33      21.45s    50

ggplot(res_bench, aes(x = factor(batch_size), y = median, 
                      group =  expression, color = expression)) +
  geom_point() +
  geom_line() +
  ylab("median time, s") +
  theme_minimal() +
  scale_color_discrete(name = "", labels = c("cpp", "r")) +
  theme(legend.position = "bottom")

Njengoko ubona, ukunyuka kwesantya kubonakale kubaluleke kakhulu, kwaye akunakwenzeka ukubamba ikhowudi ye-C ++ ngokuhambelana nekhowudi ye-R.

3. Iiterators zokothula iibhetshi kwiziko ledatha

I-R inegama elifanelekileyo lokucubungula idatha ehambelana ne-RAM, ngelixa i-Python ibonakaliswe ngakumbi ngokuphindaphindiweyo kwedatha, ikuvumela ukuba uphumeze ngokulula kunye ngokwemvelo izibalo ezingaphandle kwe-core (ubalo usebenzisa imemori yangaphandle). Umzekelo oqhelekileyo nofanelekileyo kuthi kumxholo wengxaki echaziweyo luthungelwano olunzulu lwe-neural oluqeqeshwe yindlela yokwehla kwe-gradient kunye noqikelelo lwesithambiso kwinyathelo ngalinye kusetyenziswa indawana encinane yokuqatshelwa, okanye ibhetshi encinci.

Izicwangciso zokufunda ezinzulu ezibhalwe kwiPython zineeklasi ezikhethekileyo ezizalisekisa i-iterators ngokusekelwe kwidatha: iitafile, imifanekiso kwiifolda, iifomathi zokubini, njl. Kwi-R sinokuthatha inzuzo yazo zonke iimpawu zethala leencwadi lePython Iikhamera ngeemva zayo ezahlukeneyo usebenzisa umqulu wegama elifanayo, elisebenza phezu kwempahla bhala kwakhona. Le yokugqibela ifanelwe inqaku elide elahlukileyo; ayikuvumeli kuphela ukuba usebenzise ikhowudi yePython ukusuka ku-R, kodwa ikuvumela ukuba udlulise izinto phakathi kweR kunye neeseshoni zePython, wenze ngokuzenzekelayo zonke iinguqu zohlobo oluyimfuneko.

Silahle isidingo sokugcina yonke idatha kwi-RAM ngokusebenzisa i-MonetDBLite, wonke umsebenzi "wenethiwekhi ye-neural" uya kwenziwa yikhowudi yokuqala kwi-Python, kufuneka sibhale i-iterator phezu kwedatha, kuba akukho nto ilungile. kwimeko enjalo nokuba kwi-R okanye kwiPython. Kukho iimfuno ezimbini kuphela kuyo: kufuneka ibuyisele iibhetshi kwi-loop engapheliyo kwaye igcine imeko yayo phakathi kokuphindaphinda (eyokugqibela kwi-R iphunyezwe ngeyona ndlela ilula kusetyenziswa ukuvalwa). Ngaphambili, bekufuneka ukuguqula ngokuthe gca uluhlu lwe-R lube luluhlu oluluthuli ngaphakathi kwi-iterator, kodwa uguqulelo lwangoku lwempahla. Iikhamera uyayenza ngokwakhe.

I-iterator yoqeqesho kunye nedatha yokuqinisekisa ivele yaba ngolu hlobo lulandelayo:

I-Iterator yoqeqesho kunye nedatha yokuqinisekisa

train_generator <- function(db_connection = con,
                            samples_index,
                            num_classes = 340,
                            batch_size = 32,
                            scale = 1,
                            color = FALSE,
                            imagenet_preproc = FALSE) {
  # Проверка аргументов
  checkmate::assert_class(con, "DBIConnection")
  checkmate::assert_integerish(samples_index)
  checkmate::assert_count(num_classes)
  checkmate::assert_count(batch_size)
  checkmate::assert_number(scale, lower = 0.001, upper = 5)
  checkmate::assert_flag(color)
  checkmate::assert_flag(imagenet_preproc)

  # Перемешиваем, чтобы брать и удалять использованные индексы батчей по порядку
  dt <- data.table::data.table(id = sample(samples_index))
  # Проставляем номера батчей
  dt[, batch := (.I - 1L) %/% batch_size + 1L]
  # Оставляем только полные батчи и индексируем
  dt <- dt[, if (.N == batch_size) .SD, keyby = batch]
  # Устанавливаем счётчик
  i <- 1
  # Количество батчей
  max_i <- dt[, max(batch)]

  # Подготовка выражения для выгрузки
  sql <- sprintf(
    "PREPARE SELECT drawing, label_int FROM doodles WHERE id IN (%s)",
    paste(rep("?", batch_size), collapse = ",")
  )
  res <- DBI::dbSendQuery(con, sql)

  # Аналог keras::to_categorical
  to_categorical <- function(x, num) {
    n <- length(x)
    m <- numeric(n * num)
    m[x * n + seq_len(n)] <- 1
    dim(m) <- c(n, num)
    return(m)
  }

  # Замыкание
  function() {
    # Начинаем новую эпоху
    if (i > max_i) {
      dt[, id := sample(id)]
      data.table::setkey(dt, batch)
      # Сбрасываем счётчик
      i <<- 1
      max_i <<- dt[, max(batch)]
    }

    # ID для выгрузки данных
    batch_ind <- dt[batch == i, id]
    # Выгрузка данных
    batch <- DBI::dbFetch(DBI::dbBind(res, as.list(batch_ind)), n = -1)

    # Увеличиваем счётчик
    i <<- i + 1

    # Парсинг JSON и подготовка массива
    batch_x <- cpp_process_json_vector(batch$drawing, scale = scale, color = color)
    if (imagenet_preproc) {
      # Шкалирование c интервала [0, 1] на интервал [-1, 1]
      batch_x <- (batch_x - 0.5) * 2
    }

    batch_y <- to_categorical(batch$label_int, num_classes)
    result <- list(batch_x, batch_y)
    return(result)
  }
}

Umsebenzi uthatha njengegalelo eliguquguqukayo ngodibaniso kwisiseko sedatha, amanani emigca esetyenzisiweyo, inani leeklasi, ubungakanani bebhetshi, isikali (scale = 1 ihambelana nokunikezela imifanekiso yeepikseli ezingama-256x256, scale = 0.5 — 128x128 pixels), isalathisi sombala (color = FALSE ixela unikezelo olungwevu xa lusetyenziswa color = TRUE Ukubetha ngakunye kuzotywe ngombala omtsha) kunye nesalathiso sangaphambili sothungelwano oluqeqeshwe kwangaphambili kwi-imagenet. Le yokugqibela iyafuneka ukuze kunyuswe amaxabiso epixel ukusuka kwixesha [0, 1] ukuya kwixesha [-1, 1], elalisetyenziswa xa kuqeqeshwa abo babonelelweyo. Iikhamera iimodeli.

Umsebenzi wangaphandle uqulathe uhlobo lwempikiswano ekhangelwayo, itafile data.table ngamanani emigca axubene ngokungacwangciswanga ukusuka samples_index kunye neenombolo zebhetshi, ikhawuntara kunye nenani eliphezulu leebhetshi, kunye nenkcazo ye-SQL yokukhulula idatha esuka kwisiseko sedatha. Ukongeza, sichaze i-analogue ekhawulezayo yomsebenzi ngaphakathi keras::to_categorical(). Sisebenzise phantse yonke idatha yoqeqesho, sishiya isiqingatha sepesenti yokuqinisekiswa, ngoko ke ubungakanani be-epoch bukhawulelwe yiparameter. steps_per_epoch xa ebizwa keras::fit_generator(), kunye nemeko if (i > max_i) isebenze kuphela kwisiqinisekiso sokuqinisekisa.

Kumsebenzi wangaphakathi, izalathi zerowu zikhutshelwa kwibhetshi elandelayo, iirekhodi zikhutshiwe kwisiseko sedatha kunye ne-batch counter iyanda, i-JSON yokwahlula (umsebenzi cpp_process_json_vector(), ebhalwe kwi-C ++) kunye nokudala uluhlu oluhambelana nemifanekiso. Emva koko iivektha ezishushu ezineeleyibhile zeklasi zenziwe, uluhlu olunamaxabiso epixel kunye neelebhile zidityaniswe kuluhlu, elixabiso lokubuya. Ukukhawulezisa umsebenzi, sasebenzisa ukudala izalathisi kwiitafile data.table kunye nokuguqulwa ngekhonkco - ngaphandle kwezi "chips" zephakheji idatha yedatha Kunzima kakhulu ukucinga ukusebenza ngokufanelekileyo nangasiphi na isixa esibalulekileyo sedatha kwi-R.

Iziphumo zemilinganiselo yesantya kwilaptop yeCore i5 zezi zilandelayo:

Ibenchmark ye-Iterator

library(Rcpp)
library(keras)
library(ggplot2)

source("utils/rcpp.R")
source("utils/keras_iterator.R")

con <- DBI::dbConnect(drv = MonetDBLite::MonetDBLite(), Sys.getenv("DBDIR"))

ind <- seq_len(DBI::dbGetQuery(con, "SELECT count(*) FROM doodles")[[1L]])
num_classes <- DBI::dbGetQuery(con, "SELECT max(label_int) + 1 FROM doodles")[[1L]]

# Индексы для обучающей выборки
train_ind <- sample(ind, floor(length(ind) * 0.995))
# Индексы для проверочной выборки
val_ind <- ind[-train_ind]
rm(ind)
# Коэффициент масштаба
scale <- 0.5

# Проведение замера
res_bench <- bench::press(
  batch_size = 2^(4:10),
  {
    it1 <- train_generator(
      db_connection = con,
      samples_index = train_ind,
      num_classes = num_classes,
      batch_size = batch_size,
      scale = scale
    )
    bench::mark(
      it1(),
      min_iterations = 50L
    )
  }
)
# Параметры бенчмарка
cols <- c("batch_size", "min", "median", "max", "itr/sec", "total_time", "n_itr")
res_bench[, cols]

#   batch_size      min   median      max `itr/sec` total_time n_itr
#        <dbl> <bch:tm> <bch:tm> <bch:tm>     <dbl>   <bch:tm> <int>
# 1         16     25ms  64.36ms   92.2ms     15.9       3.09s    49
# 2         32   48.4ms 118.13ms 197.24ms     8.17       5.88s    48
# 3         64   69.3ms 117.93ms 181.14ms     8.57       5.83s    50
# 4        128  157.2ms 240.74ms 503.87ms     3.85      12.71s    49
# 5        256  359.3ms 613.52ms 988.73ms     1.54       30.5s    47
# 6        512  884.7ms    1.53s    2.07s     0.674      1.11m    45
# 7       1024     2.7s    3.83s    5.47s     0.261      2.81m    44

ggplot(res_bench, aes(x = factor(batch_size), y = median, group = 1)) +
    geom_point() +
    geom_line() +
    ylab("median time, s") +
    theme_minimal()

DBI::dbDisconnect(con, shutdown = TRUE)

Ukuba unenani elaneleyo le-RAM, unokukhawulezisa ngokukhawuleza ukusebenza kwedatha ngokuyidlulisela kule RAM inye (i-32 GB yanele umsebenzi wethu). Kwi-Linux, isahlulelo sixhonywe ngokungagqibekanga /dev/shm, ithatha ukuya kwisiqingatha somthamo we-RAM. Ungaqaqambisa ngakumbi ngokuhlela /etc/fstabukufumana irekhodi efana tmpfs /dev/shm tmpfs defaults,size=25g 0 0. Qinisekisa ukuba uqalise kwakhona kwaye ukhangele umphumo ngokusebenzisa umyalelo df -h.

I-iterator yedatha yovavanyo ibonakala ilula kakhulu, kuba idatha yovavanyo ingena ngokupheleleyo kwi-RAM:

Iterator yedatha yovavanyo

test_generator <- function(dt,
                           batch_size = 32,
                           scale = 1,
                           color = FALSE,
                           imagenet_preproc = FALSE) {

  # Проверка аргументов
  checkmate::assert_data_table(dt)
  checkmate::assert_count(batch_size)
  checkmate::assert_number(scale, lower = 0.001, upper = 5)
  checkmate::assert_flag(color)
  checkmate::assert_flag(imagenet_preproc)

  # Проставляем номера батчей
  dt[, batch := (.I - 1L) %/% batch_size + 1L]
  data.table::setkey(dt, batch)
  i <- 1
  max_i <- dt[, max(batch)]

  # Замыкание
  function() {
    batch_x <- cpp_process_json_vector(dt[batch == i, drawing], 
                                       scale = scale, color = color)
    if (imagenet_preproc) {
      # Шкалирование c интервала [0, 1] на интервал [-1, 1]
      batch_x <- (batch_x - 0.5) * 2
    }
    result <- list(batch_x)
    i <<- i + 1
    return(result)
  }
}

4. Ukukhethwa kwemodeli yoyilo lwezakhiwo

I-architecture yokuqala esetyenzisiweyo yayi mobilenet v1, iimpawu ekuxutyushwa ngazo kwi oku umyalezo. Ifakwe njengomgangatho Iikhamera kwaye, ngokufanelekileyo, iyafumaneka kwipakethe yegama elifanayo le-R. Kodwa xa uzama ukuyisebenzisa ngemifanekiso yetshaneli enye, kwavela into engaqhelekanga: i-tensor yegalelo kufuneka isoloko inomlinganiselo. (batch, height, width, 3), oko kukuthi, inani lamajelo alinakutshintshwa. Akukho mda unjalo kwiPython, ke siye sakhawuleza sabhala ukuphunyezwa kwethu kolu lwakhiwo, silandela inqaku lokuqala (ngaphandle kokuyeka okukuhlobo lwekeras):

Uyilo lwe-Mobilenet v1

library(keras)

top_3_categorical_accuracy <- custom_metric(
    name = "top_3_categorical_accuracy",
    metric_fn = function(y_true, y_pred) {
         metric_top_k_categorical_accuracy(y_true, y_pred, k = 3)
    }
)

layer_sep_conv_bn <- function(object, 
                              filters,
                              alpha = 1,
                              depth_multiplier = 1,
                              strides = c(2, 2)) {

  # NB! depth_multiplier !=  resolution multiplier
  # https://github.com/keras-team/keras/issues/10349

  layer_depthwise_conv_2d(
    object = object,
    kernel_size = c(3, 3), 
    strides = strides,
    padding = "same",
    depth_multiplier = depth_multiplier
  ) %>%
  layer_batch_normalization() %>% 
  layer_activation_relu() %>%
  layer_conv_2d(
    filters = filters * alpha,
    kernel_size = c(1, 1), 
    strides = c(1, 1)
  ) %>%
  layer_batch_normalization() %>% 
  layer_activation_relu() 
}

get_mobilenet_v1 <- function(input_shape = c(224, 224, 1),
                             num_classes = 340,
                             alpha = 1,
                             depth_multiplier = 1,
                             optimizer = optimizer_adam(lr = 0.002),
                             loss = "categorical_crossentropy",
                             metrics = c("categorical_crossentropy",
                                         top_3_categorical_accuracy)) {

  inputs <- layer_input(shape = input_shape)

  outputs <- inputs %>%
    layer_conv_2d(filters = 32, kernel_size = c(3, 3), strides = c(2, 2), padding = "same") %>%
    layer_batch_normalization() %>% 
    layer_activation_relu() %>%
    layer_sep_conv_bn(filters = 64, strides = c(1, 1)) %>%
    layer_sep_conv_bn(filters = 128, strides = c(2, 2)) %>%
    layer_sep_conv_bn(filters = 128, strides = c(1, 1)) %>%
    layer_sep_conv_bn(filters = 256, strides = c(2, 2)) %>%
    layer_sep_conv_bn(filters = 256, strides = c(1, 1)) %>%
    layer_sep_conv_bn(filters = 512, strides = c(2, 2)) %>%
    layer_sep_conv_bn(filters = 512, strides = c(1, 1)) %>%
    layer_sep_conv_bn(filters = 512, strides = c(1, 1)) %>%
    layer_sep_conv_bn(filters = 512, strides = c(1, 1)) %>%
    layer_sep_conv_bn(filters = 512, strides = c(1, 1)) %>%
    layer_sep_conv_bn(filters = 512, strides = c(1, 1)) %>%
    layer_sep_conv_bn(filters = 1024, strides = c(2, 2)) %>%
    layer_sep_conv_bn(filters = 1024, strides = c(1, 1)) %>%
    layer_global_average_pooling_2d() %>%
    layer_dense(units = num_classes) %>%
    layer_activation_softmax()

    model <- keras_model(
      inputs = inputs,
      outputs = outputs
    )

    model %>% compile(
      optimizer = optimizer,
      loss = loss,
      metrics = metrics
    )

    return(model)
}

Ukungalungi kwale ndlela kuyabonakala. Ndifuna ukuvavanya iimodeli ezininzi, kodwa ngokuchaseneyo, andifuni kuphinda ndibhale uyilo ngalunye ngesandla. Siye savinjwa nethuba lokusebenzisa iintsimbi zemodeli eziqeqeshwe kwangaphambili kwi-imagenet. Njengesiqhelo, ukufunda amaxwebhu kwanceda. Umsebenzi get_config() ikuvumela ukuba ufumane inkcazelo yomzekelo ngendlela elungele ukuhlelwa (base_model_conf$layers - uluhlu oluqhelekileyo lwe-R), kunye nomsebenzi from_config() yenza uguqulelo olubuyela umva kwimodeli yento:

base_model_conf <- get_config(base_model)
base_model_conf$layers[[1]]$config$batch_input_shape[[4]] <- 1L
base_model <- from_config(base_model_conf)

Ngoku akukho nzima ukubhala umsebenzi jikelele ukufumana nayiphi na into enikeziweyo Iikhamera iimodeli ezinobunzima okanye obungenabo obuqeqeshwe kwi-imagenet:

Umsebenzi wokulayisha izakhiwo esele zenziwe

get_model <- function(name = "mobilenet_v2",
                      input_shape = NULL,
                      weights = "imagenet",
                      pooling = "avg",
                      num_classes = NULL,
                      optimizer = keras::optimizer_adam(lr = 0.002),
                      loss = "categorical_crossentropy",
                      metrics = NULL,
                      color = TRUE,
                      compile = FALSE) {
  # Проверка аргументов
  checkmate::assert_string(name)
  checkmate::assert_integerish(input_shape, lower = 1, upper = 256, len = 3)
  checkmate::assert_count(num_classes)
  checkmate::assert_flag(color)
  checkmate::assert_flag(compile)

  # Получаем объект из пакета keras
  model_fun <- get0(paste0("application_", name), envir = asNamespace("keras"))
  # Проверка наличия объекта в пакете
  if (is.null(model_fun)) {
    stop("Model ", shQuote(name), " not found.", call. = FALSE)
  }

  base_model <- model_fun(
    input_shape = input_shape,
    include_top = FALSE,
    weights = weights,
    pooling = pooling
  )

  # Если изображение не цветное, меняем размерность входа
  if (!color) {
    base_model_conf <- keras::get_config(base_model)
    base_model_conf$layers[[1]]$config$batch_input_shape[[4]] <- 1L
    base_model <- keras::from_config(base_model_conf)
  }

  predictions <- keras::get_layer(base_model, "global_average_pooling2d_1")$output
  predictions <- keras::layer_dense(predictions, units = num_classes, activation = "softmax")
  model <- keras::keras_model(
    inputs = base_model$input,
    outputs = predictions
  )

  if (compile) {
    keras::compile(
      object = model,
      optimizer = optimizer,
      loss = loss,
      metrics = metrics
    )
  }

  return(model)
}

Xa usebenzisa imifanekiso yetshaneli enye, akukho zisindo ziqeqeshelwe kwangaphambili. Oku kunokulungiswa: usebenzisa umsebenzi get_weights() Fumana iintsimbi zomzekelo ngokoluhlu lwezintlu ze-R, tshintsha idimension yento yokuqala yolu luhlu (ngokuthatha umbala wetshaneli okanye iavareji zontathu), kwaye emva koko ulayishe iintsimbi kumzekelo kunye nomsebenzi. set_weights(). Asizange songeze lo msebenzi, kuba ngeli nqanaba kwakusele kucacile ukuba kunokuvelisa ngakumbi ukusebenza ngemifanekiso yemibala.

Senze uninzi lweemvavanyo sisebenzisa i-mobilenet versions 1 kunye ne-2, kunye ne-resnet34. Uyilo lwangoku ngakumbi olunje nge-SE-ResNeXt luqhube kakuhle kolu khuphiswano. Ngelishwa, besingenalo ukuphunyezwa esele sikufumene, kwaye asizange sibhale ezethu (kodwa ngokuqinisekileyo siya kubhala).

5. I-Parameterization yeencwadi zeempendulo

Ukwenzela lula, yonke ikhowudi yokuqalisa uqeqesho yayiyilwe njengeskripthi esinye, iparameterized usebenzisa docopt ngolu hlobo:

doc <- '
Usage:
  train_nn.R --help
  train_nn.R --list-models
  train_nn.R [options]

Options:
  -h --help                   Show this message.
  -l --list-models            List available models.
  -m --model=<model>          Neural network model name [default: mobilenet_v2].
  -b --batch-size=<size>      Batch size [default: 32].
  -s --scale-factor=<ratio>   Scale factor [default: 0.5].
  -c --color                  Use color lines [default: FALSE].
  -d --db-dir=<path>          Path to database directory [default: Sys.getenv("db_dir")].
  -r --validate-ratio=<ratio> Validate sample ratio [default: 0.995].
  -n --n-gpu=<number>         Number of GPUs [default: 1].
'
args <- docopt::docopt(doc)

Iphakheji docopt imele ukuphunyezwa http://docopt.org/ kuba R. Ngoncedo lwayo, izikripthi ziqaliswa ngemiyalelo elula njenge Rscript bin/train_nn.R -m resnet50 -c -d /home/andrey/doodle_db okanye ./bin/train_nn.R -m resnet50 -c -d /home/andrey/doodle_db, ukuba ifayile train_nn.R iyaphunyezwa (lo myalelo uzakuqala ukuqeqesha umfuziselo resnet50 kwimifanekiso yemibala emithathu enomlinganiselo we-128x128 pixels, idatabase kufuneka ibekwe kwifolda /home/andrey/doodle_db). Unokongeza isantya sokufunda, uhlobo lwe-optimizer, kunye nayo nayiphi na enye iparameters enokwenziwa ngokwezifiso kuluhlu. Kwinkqubo yokulungiselela ukupapashwa, kwavela ukuba i-architecture mobilenet_v2 ukusuka kuguqulelo lwangoku Iikhamera kusetyenziso lwe-R akakwazi ngenxa yotshintsho olungathathelwa ngqalelo kwiphakheji ye-R, silindele ukuba bayilungise.

Le ndlela yenza ukuba kube nokwenzeka ukukhawulezisa kakhulu imifuniselo eneemodeli ezahlukeneyo xa kuthelekiswa nosungulo lwemveli lwezikripthi kwi-RStudio (siqaphela ipakethe njengenye indlela enokwenzeka. tfrun). Kodwa eyona nzuzo iphambili kukukwazi ukulawula ngokulula ukuqaliswa kwezikripthi kwiDocker okanye ngokulula kwiseva, ngaphandle kokufaka iRStudio kule nto.

6. Ukufakwa kwidokethi yeencwadi zeempendulo

Sisebenzise i-Docker ukuqinisekisa ukuphatheka kokusingqongileyo kwiimodeli zoqeqesho phakathi kwamalungu eqela kunye nokuthunyelwa ngokukhawuleza efini. Ungaqala ukuqhelana nesi sixhobo, esingaqhelekanga kumdwelisi we-R, nge oku uthotho lweempapasho okanye ikhosi yevidiyo.

I-Docker ikuvumela ukuba wenze eyakho imifanekiso ukusuka ekuqaleni kwaye usebenzise eminye imifanekiso njengesiseko sokwenza eyakho. Xa sihlalutya iinketho ezikhoyo, safikelela kwisigqibo sokuba ukufaka iNVIDIA, CUDA + cuDNN abaqhubi kunye namathala eencwadi ePython yinxalenye yomfanekiso obonakalayo, kwaye sagqiba kwelokuba sithathe umfanekiso osemthethweni njengesiseko. tensorflow/tensorflow:1.12.0-gpu, ukongeza iipakethe eziyimfuneko ze-R apho.

Ifayile ye-docker yokugqibela ibonakala ngolu hlobo:

dockerfile

FROM tensorflow/tensorflow:1.12.0-gpu

MAINTAINER Artem Klevtsov <[email protected]>

SHELL ["/bin/bash", "-c"]

ARG LOCALE="en_US.UTF-8"
ARG APT_PKG="libopencv-dev r-base r-base-dev littler"
ARG R_BIN_PKG="futile.logger checkmate data.table rcpp rapidjsonr dbi keras jsonlite curl digest remotes"
ARG R_SRC_PKG="xtensor RcppThread docopt MonetDBLite"
ARG PY_PIP_PKG="keras"
ARG DIRS="/db /app /app/data /app/models /app/logs"

RUN source /etc/os-release && 
    echo "deb https://cloud.r-project.org/bin/linux/ubuntu ${UBUNTU_CODENAME}-cran35/" > /etc/apt/sources.list.d/cran35.list && 
    apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E084DAB9 && 
    add-apt-repository -y ppa:marutter/c2d4u3.5 && 
    add-apt-repository -y ppa:timsc/opencv-3.4 && 
    apt-get update && 
    apt-get install -y locales && 
    locale-gen ${LOCALE} && 
    apt-get install -y --no-install-recommends ${APT_PKG} && 
    ln -s /usr/lib/R/site-library/littler/examples/install.r /usr/local/bin/install.r && 
    ln -s /usr/lib/R/site-library/littler/examples/install2.r /usr/local/bin/install2.r && 
    ln -s /usr/lib/R/site-library/littler/examples/installGithub.r /usr/local/bin/installGithub.r && 
    echo 'options(Ncpus = parallel::detectCores())' >> /etc/R/Rprofile.site && 
    echo 'options(repos = c(CRAN = "https://cloud.r-project.org"))' >> /etc/R/Rprofile.site && 
    apt-get install -y $(printf "r-cran-%s " ${R_BIN_PKG}) && 
    install.r ${R_SRC_PKG} && 
    pip install ${PY_PIP_PKG} && 
    mkdir -p ${DIRS} && 
    chmod 777 ${DIRS} && 
    rm -rf /tmp/downloaded_packages/ /tmp/*.rds && 
    rm -rf /var/lib/apt/lists/*

COPY utils /app/utils
COPY src /app/src
COPY tests /app/tests
COPY bin/*.R /app/

ENV DBDIR="/db"
ENV CUDA_HOME="/usr/local/cuda"
ENV PATH="/app:${PATH}"

WORKDIR /app

VOLUME /db
VOLUME /app

CMD bash

Ukuze kube lula, iipakethe ezisetyenzisiweyo zafakwa kwizinto eziguquguqukayo; ubuninzi bemibhalo ebhaliweyo ikhutshelwa ngaphakathi kwizikhongozeli ngexesha lokudibanisa. Siphinde satshintsha iqokobhe lomyalelo /bin/bash ukuze kube lula ukusebenzisa umxholo /etc/os-release. Oku kuthintele isidingo sokucacisa inguqulo ye-OS kwikhowudi.

Ukongeza, iskripthi esincinci se-bash sabhalwa esikuvumela ukuba uqalise isikhongozeli esinemiyalelo eyahlukeneyo. Umzekelo, ezi zinokuba zizikripthi zokuqeqesha uthungelwano lwe-neural olwalubekwe ngaphambili ngaphakathi kwesikhongozeli, okanye iqokobhe lomyalelo wokulungisa ingxaki kunye nokubeka esweni ukusebenza kwesingxobo:

Iscript sokuvula isikhongozeli

#!/bin/sh

DBDIR=${PWD}/db
LOGSDIR=${PWD}/logs
MODELDIR=${PWD}/models
DATADIR=${PWD}/data
ARGS="--runtime=nvidia --rm -v ${DBDIR}:/db -v ${LOGSDIR}:/app/logs -v ${MODELDIR}:/app/models -v ${DATADIR}:/app/data"

if [ -z "$1" ]; then
    CMD="Rscript /app/train_nn.R"
elif [ "$1" = "bash" ]; then
    ARGS="${ARGS} -ti"
else
    CMD="Rscript /app/train_nn.R $@"
fi

docker run ${ARGS} doodles-tf ${CMD}

Ukuba lo mbhalo we-bash uqhutywa ngaphandle kweeparamitha, iskripthi siya kubizwa ngaphakathi kwesikhongozeli train_nn.R ngamaxabiso angagqibekanga; ukuba ingxabano yesikhundla sokuqala ngu "bash", ngoko ke isikhongozeli siyakuqala ngokusebenzisana neqokobhe lomyalelo. Kuzo zonke ezinye iimeko, amaxabiso eengxoxo zezikhundla athathelwa indawo: CMD="Rscript /app/train_nn.R $@".

Kuyafaneleka ukuba uqaphele ukuba abalawuli abanedatha yomthombo kunye nedathabheyisi, kunye nesikhokelo sokugcina iimodeli eziqeqeshiweyo, zifakwe ngaphakathi kwesikhongozeli ukusuka kwinkqubo yokusingatha, ekuvumela ukuba ufikelele kwiziphumo zeskripthi ngaphandle kokuguqulwa okungafunekiyo.

7. Ukusebenzisa iiGPU ezininzi kwiLifu likaGoogle

Enye yeempawu zokhuphiswano yayiyidatha enengxolo kakhulu (jonga umfanekiso wesihloko, obolekwe [email protected] kwi-ODS slack). Iibhetshi ezinkulu zinceda ukulwa oku, kwaye emva kovavanyo kwiPC ene-GPU eyi-1, sigqibe kwelokuba sizazi iimodeli zoqeqesho kwii-GPU ezininzi efini. ILifu likaGoogle elisetyenzisiweyo (isikhokelo esihle kwiziseko) ngenxa yokhetho olukhulu lolungelelwaniso olukhoyo, amaxabiso afanelekileyo kunye ne-300 yebhonasi. Ngenxa yokubawa, ndayalela umzekelo we-4xV100 kunye ne-SSD kunye netoni ye-RAM, kwaye loo nto yayiyimpazamo enkulu. Umatshini onjalo utya imali ngokukhawuleza; ungahamba uqhekeze ulinge ngaphandle kombhobho oqinisekisiweyo. Ngeenjongo zemfundo, kungcono ukuthatha i-K80. Kodwa isixa esikhulu se-RAM seza luncedo - i-SSD yelifu ayizange ichukumise ngokusebenza kwayo, ngoko ke idatabase yadluliselwa kuyo. dev/shm.

Eyona nto inomdla kakhulu liqhekeza lekhowudi elinoxanduva lokusebenzisa ii-GPU ezininzi. Okokuqala, imodeli yenziwe kwi-CPU isebenzisa umphathi womxholo, njengakwiPython:

with(tensorflow::tf$device("/cpu:0"), {
  model_cpu <- get_model(
    name = model_name,
    input_shape = input_shape,
    weights = weights,
    metrics =(top_3_categorical_accuracy,
    compile = FALSE
  )
})

Emva koko okungagqitywanga (oku kubalulekile) imodeli ikhutshelwa kwinani elinikiweyo le-GPU ekhoyo, kwaye emva kokuba idityanisiwe:

model <- keras::multi_gpu_model(model_cpu, gpus = n_gpu)
keras::compile(
  object = model,
  optimizer = keras::optimizer_adam(lr = 0.0004),
  loss = "categorical_crossentropy",
  metrics = c(top_3_categorical_accuracy)
)

Indlela yakudala yokukhenkceza yonke imigangatho ngaphandle kweyokugqibela, ukuqeqesha umaleko wokugqibela, ukungakhenkcezi kunye nokuqeqesha yonke imodeli yee-GPU ezininzi ayikwazanga kuphunyezwa.

Uqeqesho lwajongwa ngaphandle kokusetyenziswa. tensorboard, sizibekela umda ekurekhodeni iinkuni kunye nokugcina iimodeli ezinamagama anenkcazelo emva kwexesha ngalinye:

Iicallbacks

# Шаблон имени файла лога
log_file_tmpl <- file.path("logs", sprintf(
  "%s_%d_%dch_%s.csv",
  model_name,
  dim_size,
  channels,
  format(Sys.time(), "%Y%m%d%H%M%OS")
))
# Шаблон имени файла модели
model_file_tmpl <- file.path("models", sprintf(
  "%s_%d_%dch_{epoch:02d}_{val_loss:.2f}.h5",
  model_name,
  dim_size,
  channels
))

callbacks_list <- list(
  keras::callback_csv_logger(
    filename = log_file_tmpl
  ),
  keras::callback_early_stopping(
    monitor = "val_loss",
    min_delta = 1e-4,
    patience = 8,
    verbose = 1,
    mode = "min"
  ),
  keras::callback_reduce_lr_on_plateau(
    monitor = "val_loss",
    factor = 0.5, # уменьшаем lr в 2 раза
    patience = 4,
    verbose = 1,
    min_delta = 1e-4,
    mode = "min"
  ),
  keras::callback_model_checkpoint(
    filepath = model_file_tmpl,
    monitor = "val_loss",
    save_best_only = FALSE,
    save_weights_only = FALSE,
    mode = "min"
  )
)

8. Endaweni yesiphelo

Uninzi lweengxaki esiye sadibana nazo azikasonjululwa:

в Iikhamera akukho msebenzi usele ulungiselelwe ukukhangela ngokuzenzekelayo elona zinga lokufunda (analogue lr_finder kwilayibrari fast.ai); Ngomzamo othile, kuyenzeka ukufaka ufezekiso lomntu wesithathu kwi-R, umzekelo, oku;
njengesiphumo senqaku elidlulileyo, kwakungenakwenzeka ukukhetha isantya soqeqesho esichanekileyo xa usebenzisa ii-GPU ezininzi;
kukho ukunqongophala kwe-neural network architectures yanamhlanje, ngakumbi ezo ziqeqeshwe kwangaphambili kwi-imagenet;
akukho mgaqo-nkqubo womjikelo omnye kunye namazinga okufunda acalulayo (i-cosine annealing ibiyisicelo sethu iphunyeziwe, Enkosi skydan).

Zeziphi izinto eziluncedo ezifundwe kolu khuphiswano:

Kwi-hardware yamandla aphantsi ngokwentelekiso, unokusebenza ngokundilisekileyo (amaxesha amaninzi ubukhulu be-RAM) imiqulu yedatha ngaphandle kweentlungu. Ibhegi yeplastikhi idatha yedatha igcina inkumbulo ngenxa yokuguqulwa kwendawo yeetafile, ezinqanda ukuzikopa, kwaye xa zisetyenziswe ngokuchanekileyo, amandla ayo phantse ahlala ebonisa esona santya siphezulu phakathi kwazo zonke izixhobo ezaziwayo kuthi ngeelwimi zokubhala. Ukugcina idatha kwisiseko sedatha kukuvumela, kwiimeko ezininzi, ukuba ungacingi konke malunga nesidingo sokucudisa yonke idatha kwi-RAM.
Imisebenzi ecothayo kwi-R inokutshintshwa ngokukhawuleza kwi-C++ usebenzisa iphakheji Rcpp. Ukuba ukongeza ekusebenziseni RcppTread okanye RcppParallel, sifumana ukuphunyezwa kwe-cross-threaded multi-threaded, ngoko akukho mfuneko yokulinganisa ikhowudi kwinqanaba le-R.
Iphakheji Rcpp ingasetyenziswa ngaphandle kolwazi olunzulu lweC ++, ubuncinci obufunekayo buchazwe apha. Iifayile zeheader zenani leelayibrari ezipholileyo ze-C ezifana xtensor ekhoyo kwi-CRAN, oko kukuthi, isiseko siyasekwa ukwenzela ukuphunyezwa kweeprojekthi ezidibanisa ikhowudi ye-C++ ephezulu esele yenziwe kwi-R. Ukulungeleka okongeziweyo kukuqaqambisa i-syntax kunye ne-static C ++ yokuhlaziya ikhowudi kwi-RStudio.
docopt ikuvumela ukuba usebenzise imibhalo ezimeleyo eneparameters. Oku kukulungele ukusetyenziswa kwiseva ekude, kuquka. phantsi kwe-docker. Kwi-RStudio, akulunganga ukuqhuba iiyure ezininzi zovavanyo ngoqeqesho lweenethiwekhi ze-neural, kwaye ukufaka i-IDE kumncedisi ngokwawo akusoloko kufanelekile.
I-Docker iqinisekisa ukuphatheka kwekhowudi kunye nokuveliswa kwakhona kweziphumo phakathi kwabaphuhlisi abaneenguqulelo ezahlukeneyo ze-OS kunye namathala eencwadi, kunye nokulula kokuphunyezwa kwiiseva. Ungaqalisa wonke umbhobho woqeqesho ngomyalelo nje omnye.
ILifu likaGoogle yindlela elungele uhlahlo lwabiwo-mali yokulinga kwihardware ebizayo, kodwa kufuneka ukhethe uqwalaselo ngononophelo.
Ukulinganisa isantya samaqhekeza ekhowudi nganye luncedo kakhulu, ngakumbi xa udibanisa i-R kunye ne-C ++, kunye nephakheji. ibhentshi -kwaye kulula kakhulu.

Lilonke la mava abe nomvuzo kakhulu kwaye siyaqhubeka nokusebenza ukusombulula eminye yemiba ephakanyisiweyo.

umthombo: www.habr.com

UkuQatshelwa koMzobo oKhawulezayo: indlela yokwenza abahlobo nge-R, C++ kunye neenethiwekhi ze-neural