The article consists of two parts:
- A brief overview of some architectures for network object detection in images and image segmentation, along with the most understandable references I could find. I tried to select video explanations, preferably in Russian.
- The second part attempts to understand the trajectory of neural network architecture development and the technologies based on them.
Figure 1 – Understanding neural network architectures is not simple.
It all started with me creating two demonstration applications for classification and object detection on an Android phone:
- , where data is processed on the server and sent to the phone. Image classification (image classification) of three types of bears: brown, black, and teddy.
- , where data is processed on the phone itself. Object detection of three types: hazelnut, fig, and date.
There is a difference between tasks of image classification, object detection in images, and . Therefore, there was a need to learn which neural network architectures detect objects in images and which can segment them. I found the following examples of architectures with the most understandable references:
- A series of architectures based on R-CNN (Regions with Convolution Neural Networks features): R-CNN, Fast R-CNN, , To detect an object in an image using the Region Proposal Network (RPN) mechanism, limited regions (bounding boxes) are highlighted. Initially, a slower mechanism called Selective Search was used instead of RPN. Then, the highlighted limited regions are fed into a standard neural network for classification. The R-CNN architecture has explicit 'for' loops for iterating over the limited regions, with up to 2000 passes through the internal AlexNet network. Due to the explicit 'for' loops, the image processing speed slows down. The number of explicit loops, passes through the internal neural network, decreases with each new version of the architecture, and dozens of other changes are made to increase speed and to replace the object detection task with object segmentation in Mask R-CNN.
- (You Only Look OYOLO (You Only Look Once) is the first neural network that recognized objects in real time on mobile devices. Its distinctive feature is the ability to distinguish objects in a single pass (only one look is enough). This means that the YOLO architecture does not have explicit 'for' loops, which allows the network to operate quickly. For example, a similar analogy can be found in NumPy, where matrix operations also eliminate explicit 'for' loops implemented at lower levels through the C programming language. YOLO uses a grid of predefined windows. To avoid multiple detections of the same object, the Intersection over Union (IoU) coefficient is applied. IIntersection oover UUnion). This architecture operates across a wide range and possesses high : the model can be trained on photographs but performs well on drawn images.
- (SSingle SShot MultiBox DDetector) — utilizes the most effective 'hacks' from the YOLO architecture (such as non-maximum suppression) and adds new ones to make the neural network operate faster and more accurately. A distinctive feature is the ability to distinguish objects in a single pass using a designated grid of boxes (default box) on an image pyramid. The image pyramid is encoded in convolutional tensors during sequential convolution and pooling operations (max-pooling reduces spatial dimensions). This allows for the detection of both large and small objects in a single network pass.
- MobileSSD (MobileNetV2 + SSD) – a combination of two neural network architectures. The first network operates quickly and increases recognition accuracy. MobileNetV2 is used instead of VGG-16, which was originally applied in . The second network SSD detects the location of objects in the image.
- – a very small yet accurate neural network. It does not solve the object detection task by itself but can be utilized in combination with various architectures and in mobile devices. A distinctive feature is that the data is first compressed into four 1×1 convolutional filters and then expanded into four 1×1 and four 3×3 convolutional filters. One such iteration of data compression and expansion is called a 'Fire Module.'
- (Semantic Image Segmentation with Deep Convolutional Nets) – object segmentation in an image. A distinctive feature of the architecture is the sparse (dilated convolution) convolution, which preserves spatial resolution. This is followed by a post-processing stage using a graphical probabilistic model (conditional random field), which allows for the removal of small noise in segmentation and improves the quality of the segmented image. Behind the daunting name of 'graphical probabilistic model' lies a regular Gaussian filter approximated over five points.
- I tried to understand the device (Single-Shot Refinement Neural Network for Object Detection), but didn't understand much.
- I also looked at how the attention technology works: , , . A distinctive feature of the attention architecture is the automatic highlighting of regions of increased interest in an image (RoI, Regions of Interest) using a neural network called Attention Unit. Regions of increased interest are similar to bounding boxes, but unlike them, they are not fixed to the image and can have blurred edges. Features are then extracted from the regions of increased interest, which are 'fed' into recurrent neural networks with architectures . Recurrent neural networks can analyze the relationships between features in a sequence. Recurrent neural networks were originally used for translating text into other languages and now for translating and .
As I studied these architectures I realized that I understood nothing.. And it's not that my neural network has issues with the attention mechanism. Creating all these architectures resembles some massive hackathon where authors compete in hacks. A hack is a quick solution to a difficult programming problem. That is, there is no visible and understandable logical connection between all these architectures. All that unites them is a set of the most successful hacks that they borrow from one another, plus a common (backpropagation). There is no ! It's unclear what to change and how to optimize the existing achievements.
As a result of the lack of logical connections between hacks, they are extremely difficult to remember and apply in practice. This leads to fragmented knowledge. At best, a few interesting and unexpected moments are remembered, but most of what is understood and misunderstood fades from memory within a few days. It would be good if the architecture's name comes to mind at least a week later. Yet, reading articles and watching overview videos has consumed several hours and even days of work time!
Figure 2 -
Most authors of scientific articles, in my personal opinion, do everything possible to ensure that even this fragmented knowledge is not understood by the reader. But participial constructions in ten-line sentences with formulas pulled 'out of thin air' — that's a topic for a separate article (the problem ).
For this reason, there arose a need to systematize information on neural networks and, thus, improve the quality of understanding and memorization. Therefore, the main focus of analyzing specific technologies and architectures of artificial neural networks became the following task: to determine where it is all heading, rather than the structure of any specific neural network in isolation.
Where it is all heading. Key findings:
- The number of startups in the field of machine learning in the last two years Possible reason: 'neural networks are no longer something new.'
- Anyone can create a working neural network to solve a simple problem. To do this, one can take a ready-made model from the 'model zoo' and train the last layer of the neural network () on prepared data from or from in free .
- Major neural network manufacturers have begun to create 'model zoos.' With their help, a commercial application can be quickly built: for TensorFlow, for PyTorch, for Caffe2, for Chainer and .
- Neural networks operating in real-time (real-time) on mobile devices. From 10 to 50 frames per second.
- The use of neural networks in phones (TF Lite), in browsers (TF.js), and in (IoT, IInternet of Tof Things). Especially in phones, which already support neural networks at the hardware level (neural accelerators).
- "Every device, clothing items, and perhaps even food will have an IP-v6 address and communicate with each other" – .
- The growth in the number of publications on machine learning has begun to (doubling every two years) since 2015. Clearly, neural networks for article analysis are needed.
- The following technologies are gaining popularity:
- PyTorch – popularity is rapidly increasing and seems to be outpacing TensorFlow.
- Automatic hyperparameter tuning AutoML – popularity is growing steadily.
- Gradual decrease in accuracy and increase in computing speed: , algorithms , approximate computations, quantization (when the weights of neural networks are converted into integers and quantized), neural accelerators.
- Translation and .
- Creating , now in real time.
- The core of DL is a lot of data, but collecting and annotating it is not easy. Therefore, automated annotation () for neural networks using neural networks is developing.
- With neural networks, Computer Science has suddenly become an experimental science and has encountered .
- IT money and the popularity of neural networks arose simultaneously when computation became a market value. The economy is transitioning from a gold-backed to a gold-backed computational economy.See my article on and the reason for the emergence of IT money.
Gradually, a new (Machine Learning & Deep Learning) is emerging, which is based on representing a program as a set of trained neural network models.
Figure 3 – ML/DL as a new programming methodology
However, a "theory of neural networks" has yet to emerge, within which one can think and work systematically. What is currently called "theory" are actually experimental, heuristic algorithms.Links to my resources and others:
Newsletter on Data Science. Primarily on image processing. Anyone who wants to receive it should send an email (foobar167<woof-woof>gmail<dot>com). Links to articles and videos are sent as material accumulates.
- Overall
- list of courses and articles Courses and videos for beginners
- "Introduction to Machine Learning and Artificial Neural Networks" .
- , where everyone can find something interesting for themselves.
- I found it extremely useful video channels analyzing scientific papers on Data Science. Find, subscribe to them, and share the links with your colleagues and me as well. Examples:
- aka with step-by-step instructions and open source code.
Thank you for your attention!
Source: habr.com
