Porting Qt to STM32

Hello! We are in the project Embox launched Qt on STM32F7-Discovery and would like to tell you about it. Previously, we discussed how we managed to run OpenCV.

Qt is a cross-platform framework that includes not only graphical components but also things like QtNetwork, a set of classes for working with databases, Qt for Automation (including for implementing IoT), and much more. The Qt team developers anticipated the use of Qt in embedded systems, so the libraries are quite well configurable. However, until recently, few thought about porting Qt to microcontrollers, likely because such a task seems complex—Qt is large, and MCUs are small.

On the other hand, there are currently microcontrollers designed for multimedia that surpass the first Pentiums. About a year ago, a postappeared on the Qt blog. The developers ported Qt to the RTEMS OS and ran widget examples on several boards powered by STM32F7. This caught our interest. It was noticeable, and the developers also wrote about it, that Qt lags on STM32F7-Discovery. We became curious if we could run Qt under Embox, not just to draw a widget but to run an animation.

Qt 4.8 has long been ported to Embox, so we decided to try it. We chose the moveblocks application—a sample of spring-like animation.

Qt moveblocks on QEMU

First, we configure Qt with the minimum set of components required to support animation as much as possible. For this, there is an option “-qconfig minimal,small,medium…”. It connects a configuration file from Qt containing many macros—what to include / what to disable. After this option, we add other flags to the configuration if we want to disable something additional. Here is an example of our configuration.

To make Qt work, a compatibility layer with the OS needs to be added. One way is to implement QPA (Qt Platform Abstraction). We based it on an existing plugin fb_base that comes with Qt, on which the QPA for Linux runs. As a result, a small plugin emboxfb was created that provides Qt with the framebuffer of Embox, and from there it draws without external assistance.

This is what the plugin creation looks like

QEmboxFbIntegration::QEmboxFbIntegration()
    : fontDb(new QGenericUnixFontDatabase())
{
    struct fb_var_screeninfo vinfo;
    struct fb_fix_screeninfo finfo;
    const char *fbPath = "/dev/fb0";

    fbFd = open(fbPath, O_RDWR);
    if (fbFd setPhysicalSize(QSize(fbWidth, fbHeight));
    mScreens.append(mPrimaryScreen);

    this->printFbInfo();
}

This is what the redraw will look like

QRegion QEmboxFbScreen::doRedraw()
{
    QVector rects;
    QRegion touched = QFbScreen::doRedraw();

    DPRINTF("QEmboxFbScreen::doRedrawn");

    if (!compositePainter) {
        compositePainter = new QPainter(mFbScreenImage);
    }

    rects = touched.rects();
    for (int i = 0; i drawImage(rects[i], *mScreenImage, rects[i]);
    }
    return touched;
}

As a result, with compiler optimization for memory size -Os, the library image ended up being 3.5 MB, which certainly does not fit into the main memory of STM32F746. As we mentioned in our other article about OpenCV, this board has:

• 1 MB ROM
• 320 KB RAM
• 8 MB SDRAM
• 16 MB QSPI

Since support for executing code from QSPI was already added for OpenCV, we decided to start by loading the Embox image with Qt into QSPI entirely. And hooray, everything almost immediately launched from QSPI! But, as in the case with OpenCV, it turned out to be too slow.

Therefore, we decided to do it this way — first copying the image to QSPI, then loading it into SDRAM and executing from there. It became a bit faster from SDRAM, but still far from QEMU.

Next, there was an idea to enable floating point — since Qt performs some calculations for square coordinates in the animation. We tried, but here we did not see a visible speedup, although in article the Qt developers claimed that FPU provides a significant speed boost for "dragging animation" on the touchscreen. Perhaps, in moveblocks, there are significantly fewer floating point calculations, and this depends on the specific example.

The most effective solution turned out to be relocating the framebuffer from SDRAM to internal memory. To achieve this, we changed the screen dimensions from 480×272 to 272×272. We also reduced the color depth from A8R8G8B8 to R5G6B5, which cut the size of one pixel from 4 to 2 bytes. This gave us a framebuffer size of 272 * 272 * 2 = 147968 bytes. This resulted in a significant speedup, likely the most noticeable, making the animation almost smooth.

The final optimization involved executing code from Embox out of RAM, while Qt was run from SDRAM. To do this, we initially statically linked Embox together with Qt as usual, but placed the text, rodata, data, and bss segments of the library into QSPI to subsequently copy them into SDRAM.

section (qt_text, SDRAM, QSPI)
phdr	(qt_text, PT_LOAD, FLAGS(5))

section (qt_rodata, SDRAM, QSPI)
phdr	(qt_rodata, PT_LOAD, FLAGS(5))

section (qt_data, SDRAM, QSPI)
phdr	(qt_data, PT_LOAD, FLAGS(6))

section (qt_bss, SDRAM, QSPI)
phdr	(qt_bss, PT_LOAD, FLAGS(6))

By executing code from Embox out of ROM, we also achieved a noticeable speedup. As a result, the animation became quite smooth:

It was towards the end, while preparing the article and trying different configurations of Embox, that we found Qt moveblocks work wonderfully from QSPI with the framebuffer in SDRAM, with the bottleneck being the framebuffer size! Apparently, a 2x speedup was sufficient to overcome the initial 'slideshow' simply by decreasing the framebuffer size. We weren't able to achieve such results by merely relocating Embox code to various fast memory options (the acceleration was about 1.5 times instead of 2).

How to try it yourself

If you have an STM32F7-Discovery, you can run Qt under Embox yourself. You can read about how to do this on our wiki.

Conclusion

In the end, we managed to run Qt! We believe the complexity of the task is somewhat exaggerated. Naturally, one must consider the specifics of microcontrollers and have a general understanding of computing system architecture. The optimization results indicate the well-known fact that the bottleneck in a computing system is not the processor, but memory.

This year, we will be participating in the festival TechTrain. There, we will discuss and demonstrate Qt, OpenCV on microcontrollers, and other achievements in more detail.

Source: habr.com