As part of meeting 0x0A DC7831 On February 16, we presented a report on the basic principles of binary code emulation and our own development - a hardware platform emulator .
In the article, we will describe how to launch the device firmware in the emulator, demonstrate interaction with the debugger, and perform a small dynamic analysis of the firmware.
prehistory
A long time ago in a galaxy far far away
A couple of years ago, in our laboratory, it became necessary to examine the firmware of the device. The firmware was compressed, unpacked by the bootloader. He did this in a very confused way, shifting the data in memory several times. And the firmware itself then actively interacted with the periphery. And all this on the MIPS core.
For objective reasons, the available emulators did not suit us, but we still wanted to run the code. Then we decided to make our own emulator, which will do the minimum and allow you to unpack the main firmware. We tried - it worked. We thought, what if we add peripherals in order to also perform the main firmware. It didn't hurt too much, and it worked. We thought again and decided to make a full-fledged emulator.
The result is a computer emulator .
Why Copycat?
There is a play on words.
- copycat (English, n. [ˈkɒpɪkæt]) - imitator, imitator
- cat (eng., n. [ˈkæt]) - cat, cat - favorite animal of one of the creators of the project
- The letter "K" is from the Kotlin programming language
copycat
When creating the emulator, very specific goals were set:
- the ability to quickly create a new peripheral, module, processor core;
- the ability to assemble a virtual device from various modules;
- the ability to load any binary data (firmware) into the memory of a virtual device;
- the ability to work with snapshots (snapshots of the system state);
- the ability to interact with the emulator through the built-in debugger;
- nice modern language for development.
As a result, Kotlin was chosen for implementation, a bus architecture (this is when modules communicate with each other via virtual data buses), JSON as a device description format, and GDB RSP as a protocol for interacting with a debugger.
Development has been going on for a little over two years and is actively continuing. During this time, MIPS, x86, V850ES, ARM, PowerPC processor cores were implemented.
The project is growing and it's time to present it to the general public. We will do a detailed description of the project later, but for now we will focus on using Kopycat.
For the most impatient - the promo version of the emulator can be downloaded from .
Rhino in the emulator
Recall that earlier for the SMARTRHINO-2018 conference, a Rhinoceros test device was created to teach reverse engineering skills. The process of static firmware analysis has been described in .
Now let's try to add "speakers" and run the firmware in the emulator.
We need:
1) Java 1.8
2) Python and module to use Python inside the emulator. WHL assembly of the Jep module under Windows can be .
For Windows:
1)
2)
For Linux:
1) socat
You can use Eclipse, IDA Pro, or radare2 as a GDB client.
How does it work?
In order to perform firmware in the emulator, you need to "assemble" a virtual device, which is an analogue of a real device.
The real device ("rhinoceros") can be shown in the block diagram:
The emulator has a modular structure and the final virtual device can be described in a JSON file.
JSON for 105 lines
{
"top": true,
// Plugin name should be the same as file name (or full path from library start)
"plugin": "rhino",
// Directory where plugin places
"library": "user",
// Plugin parameters (constructor parameters if jar-plugin version)
"params": [
{ "name": "tty_dbg", "type": "String"},
{ "name": "tty_bt", "type": "String"},
{ "name": "firmware", "type": "String", "default": "NUL"}
],
// Plugin outer ports
"ports": [ ],
// Plugin internal buses
"buses": [
{ "name": "mem", "size": "BUS30" },
{ "name": "nand", "size": "4" },
{ "name": "gpio", "size": "BUS32" }
],
// Plugin internal components
"modules": [
{
"name": "u1_stm32",
"plugin": "STM32F042",
"library": "mcu",
"params": {
"firmware:String": "params.firmware"
}
},
{
"name": "usart_debug",
"plugin": "UartSerialTerminal",
"library": "terminals",
"params": {
"tty": "params.tty_dbg"
}
},
{
"name": "term_bt",
"plugin": "UartSerialTerminal",
"library": "terminals",
"params": {
"tty": "params.tty_bt"
}
},
{
"name": "bluetooth",
"plugin": "BT",
"library": "mcu"
},
{ "name": "led_0", "plugin": "LED", "library": "mcu" },
{ "name": "led_1", "plugin": "LED", "library": "mcu" },
{ "name": "led_2", "plugin": "LED", "library": "mcu" },
{ "name": "led_3", "plugin": "LED", "library": "mcu" },
{ "name": "led_4", "plugin": "LED", "library": "mcu" },
{ "name": "led_5", "plugin": "LED", "library": "mcu" },
{ "name": "led_6", "plugin": "LED", "library": "mcu" },
{ "name": "led_7", "plugin": "LED", "library": "mcu" },
{ "name": "led_8", "plugin": "LED", "library": "mcu" },
{ "name": "led_9", "plugin": "LED", "library": "mcu" },
{ "name": "led_10", "plugin": "LED", "library": "mcu" },
{ "name": "led_11", "plugin": "LED", "library": "mcu" },
{ "name": "led_12", "plugin": "LED", "library": "mcu" },
{ "name": "led_13", "plugin": "LED", "library": "mcu" },
{ "name": "led_14", "plugin": "LED", "library": "mcu" },
{ "name": "led_15", "plugin": "LED", "library": "mcu" }
],
// Plugin connection between components
"connections": [
[ "u1_stm32.ports.usart1_m", "usart_debug.ports.term_s"],
[ "u1_stm32.ports.usart1_s", "usart_debug.ports.term_m"],
[ "u1_stm32.ports.usart2_m", "bluetooth.ports.usart_m"],
[ "u1_stm32.ports.usart2_s", "bluetooth.ports.usart_s"],
[ "bluetooth.ports.bt_s", "term_bt.ports.term_m"],
[ "bluetooth.ports.bt_m", "term_bt.ports.term_s"],
[ "led_0.ports.pin", "u1_stm32.buses.pin_output_a", "0x00"],
[ "led_1.ports.pin", "u1_stm32.buses.pin_output_a", "0x01"],
[ "led_2.ports.pin", "u1_stm32.buses.pin_output_a", "0x02"],
[ "led_3.ports.pin", "u1_stm32.buses.pin_output_a", "0x03"],
[ "led_4.ports.pin", "u1_stm32.buses.pin_output_a", "0x04"],
[ "led_5.ports.pin", "u1_stm32.buses.pin_output_a", "0x05"],
[ "led_6.ports.pin", "u1_stm32.buses.pin_output_a", "0x06"],
[ "led_7.ports.pin", "u1_stm32.buses.pin_output_a", "0x07"],
[ "led_8.ports.pin", "u1_stm32.buses.pin_output_a", "0x08"],
[ "led_9.ports.pin", "u1_stm32.buses.pin_output_a", "0x09"],
[ "led_10.ports.pin", "u1_stm32.buses.pin_output_a", "0x0A"],
[ "led_11.ports.pin", "u1_stm32.buses.pin_output_a", "0x0B"],
[ "led_12.ports.pin", "u1_stm32.buses.pin_output_a", "0x0C"],
[ "led_13.ports.pin", "u1_stm32.buses.pin_output_a", "0x0D"],
[ "led_14.ports.pin", "u1_stm32.buses.pin_output_a", "0x0E"],
[ "led_15.ports.pin", "u1_stm32.buses.pin_output_a", "0x0F"]
]
}Pay attention to the parameter firmware In chapter params is the name of the file that can be uploaded to the virtual device as firmware.
A virtual device and its interaction with the main operating system can be represented as follows:
The current test instance of the emulator implies interaction with the COM ports of the main OS (debug UART and UART for the Bluetooth module). These can be real ports to which devices are connected or virtual COM ports (for this you just need com0com/socat).
There are currently two main ways to interact with the emulator from the outside:
- GDB RSP protocol (respectively, the tools that support this protocol are Eclipse / IDA / radare2);
- emulator internal command line (Argparse or Python).
Virtual COM ports
In order to interact with the UART of the virtual device on the local machine through the terminal, you need to create a pair of associated virtual COM ports. In our case, one port is used by the emulator, and the second by the terminal program (PuTTY or screen):
Using com0com
Virtual COM ports are configured by the setup utility from the com0com package (console version - C:Program Files (x86)com0comsetupс.exe, or GUI version - C:Program Files (x86)com0comsetupg.exe):
You need to check the boxes enable buffer overrun for all created virtual ports, otherwise the emulator will wait for a response from the COM port.
Using socat
On UNIX systems, virtual COM ports are automatically created by the emulator using the socat utility, for this it is enough to specify a prefix in the port name when starting the emulator socat:.
Internal command line interface (Argparse or Python)
Since Kopycat is a console application, the emulator provides two options for the command line interface to interact with its objects and variables: Argparse and Python.
Argparse is the CLI built into Kopycat and is always available to everyone.
The alternative CLI is the Python interpreter. To use it, you need to install the Jep Python module and configure the emulator to work with Python (the Python interpreter installed on the user's main system will be used).
Installing the Jep Python Module
Under Linux, Jep can be installed via pip:
pip install jepTo install Jep on Windows, you must first install the Windows SDK and the corresponding Microsoft Visual Studio. We have made the task a little easier for you and made JEP for current versions of Python for Windows, so the module can be installed from a file:
pip install jep-3.8.2-cp27-cp27m-win_amd64.whlTo verify the installation of Jep, you need to run on the command line:
python -c "import jep"The following message should be received in response:
ImportError: Jep is not supported in standalone Python, it must be embedded in Java.In the emulator batch file for your system (copycat.bat - for Windows, copycat - for Linux) to the list of parameters DEFAULT_JVM_OPTS add an extra parameter Djava.library.path - it must contain the path to the installed Jep module.
The result for Windows should be a line like this:
set DEFAULT_JVM_OPTS="-XX:MaxMetaspaceSize=256m" "-XX:+UseParallelGC" "-XX:SurvivorRatio=6" "-XX:-UseGCOverheadLimit" "-Djava.library.path=C:/Python27/Lib/site-packages/jep"Running Kopycat
The emulator is a console JVM application. The launch is carried out through the command line script of the operating system (sh/cmd).
Command to run under Windows:
binkopycat -g 23946 -n rhino -l user -y library -p firmware=firmwarerhino_pass.bin,tty_dbg=COM26,tty_bt=COM28The command to run under Linux using the socat utility:
./bin/kopycat -g 23946 -n rhino -l user -y library -p firmware=./firmware/rhino_pass.bin, tty_dbg=socat:./COM26,tty_bt=socat:./COM28-g 23646— TCP port that will be opened for access to the GDB server;-n rhino— name of the main module of the system (complete device);-l user— name of the library to search for the main module;-y library— path for searching for modules included in the device;firmwarerhino_pass.bin- path to the firmware file;- COM26 and COM28 are virtual COM ports.
This will result in a prompt Python > (or Argparse >):
18:07:59 INFO [eFactoryBuilder.create ]: Module top successfully created as top
18:07:59 INFO [ Module.initializeAndRes]: Setup core to top.u1_stm32.cortexm0.arm for top
18:07:59 INFO [ Module.initializeAndRes]: Setup debugger to top.u1_stm32.dbg for top
18:07:59 WARN [ Module.initializeAndRes]: Tracer wasn't found in top...
18:07:59 INFO [ Module.initializeAndRes]: Initializing ports and buses...
18:07:59 WARN [ Module.initializePortsA]: ATTENTION: Some ports has warning use printModulesPortsWarnings to see it...
18:07:59 FINE [ ARMv6CPU.reset ]: Set entry point address to 08006A75
18:07:59 INFO [ Module.initializeAndRes]: Module top is successfully initialized and reset as a top cell!
18:07:59 INFO [ Kopycat.open ]: Starting virtualization of board top[rhino] with arm[ARMv6Core]
18:07:59 INFO [ GDBServer.debuggerModule ]: Set new debugger module top.u1_stm32.dbg for GDB_SERVER(port=23946,alive=true)
Python >Interaction with IDA Pro
As a source file for analysis in IDA, to simplify testing, we use the Rhinoceros firmware in the form (meta-information is stored there).
You can also use the main firmware without metainformation.
After launching Kopycat in IDA Pro, in the Debugger menu, go to the item "Switch debugger…"And choose"Remote GDB debugger". Next, configure the connection: menu Debugger-Process options...
Set values:
- Application - any value
- Hostname: 127.0.0.1 (or the IP address of the remote machine where Kopycat is running)
- Port: 23946
Now the debug start button becomes available (F9 key):
We press it - there is a connection to the debugger module in the emulator. IDA switches to debug mode, additional windows become available: information about registers, about the stack.
Now we can use all the standard features of the debugger:
- step by step instructions (step into и step over - keys F7 and F8, respectively);
- start and pause execution;
- creation of breakpoints for both code and data (F2 key).
Connecting to the debugger does not mean running the firmware code. The current position for execution must be the address 0x08006A74 - the beginning of the function Reset_Handler. If you scroll the listing below, you can see the function call main. You can place the cursor on this line (address 0x08006ABE) and perform the operation Run until cursor (F4 key).
Then you can press F7 to enter the function main.
If you run the command Continue process (F9 key), then the “Please wait” window will appear with a single button Suspend:
When you press Suspend the execution of the firmware code is suspended and can be continued from the same address in the code where it was interrupted.
If you continue executing the code, you will see the following lines in terminals connected to virtual COM ports:
The presence of the "state bypass" line indicates that the virtual Bluetooth module has switched to the mode of receiving data from the user's COM port.
Now in the Bluetooth terminal (in the figure - COM29) you can enter commands in accordance with the Rhino protocol. For example, the command “MEOW” will return the string “mur-mur” to the Bluetooth terminal:
Emulate me not completely
When building an emulator, you can choose the level of detail / emulation of a particular device. So, for example, the Bluetooth module can be emulated in different ways:
- the device is fully emulated with a full set of commands;
- AT commands are emulated, and the data stream is received from the COM port of the main system;
- the virtual device provides full data redirection to the real device;
- as a simple stub that always returns "OK".
The current version of the emulator uses the second approach - the virtual Bluetooth module performs configuration, after which it switches to the “proxying” mode of data from the COM port of the main system to the UART port of the emulator.
Consider the possibility of simple code instrumentation in case some part of the periphery is not implemented. For example, if a timer responsible for controlling data transfer to DMA has not been created (the check is performed in the function ws2812b_waitlocated at 0x08006840), then the firmware will always wait for the flag to be reset busylocated at 0x200004C4, which shows the busyness of the DMA data line:
We can get around this situation by "manually" resetting the flag busy immediately after installing it. In IDA Pro, you can create a Python function and call it in a breakpoint, while putting the breakpoint itself in the code after writing the value 1 to the flag busy.
Breakpoint handler
Let's first create a Python function in IDA. Menu File-Script command...
Add a new snippet in the list on the left, give it a name (for example, BPT),
in the text field on the right, enter the function code:
def skip_dma():
print "Skipping wait ws2812..."
value = Byte(0x200004C4)
if value == 1:
PatchDbgByte(0x200004C4, 0)
return False
After that we press Run and close the script window.
Now let's move on to the code at 0x0800688A, set the breakpoint (F2 key), edit it (context menu Edit breakpoint...), don't forget to set the script type to Python:
If the current value of the flag busy equals 1, then the function should be executed skip_dma in the script line:
If you run the firmware for execution, then the triggering of the breakpoint handler code can be seen in IDA in the window output by line Skipping wait ws2812.... Now the firmware will not wait for the flag to be reset busy.
Interaction with the emulator
Emulation for the sake of emulation is unlikely to cause delight and joy. It is much more interesting if the emulator helps the researcher to see the data in memory or establish the interaction of threads.
We will show how to establish the interaction of RTOS tasks in dynamics. You must first pause the execution of the code if it is running. If we go to the function bluetooth_task_entry to the processing branch of the “LED" command (address 0x080057B8), you can see what is first created and then sent to the system queue ledControlQueueHandle some message.
You should set a breakpoint for accessing a variable ledControlQueueHandlelocated at 0x20000624 and continue executing the code:
As a result, it will first stop at the address 0x080057CA before calling the function osMailAlloc, then to the address 0x08005806 before calling the function osMailPut, then after a while - at 0x08005BD4 (before calling the function osMailGet) that belongs to the function leds_task_entry (LED-task), that is, there was a switching of tasks, and now the LED-task has received control.
In this simple way, you can establish how RTOS tasks interact with each other.
Of course, in reality, the interaction of tasks can be more complicated, but with the use of an emulator, tracking this interaction becomes less laborious.
you can watch a short video of launching the emulator and interacting with IDA Pro.
Launch with Radare2
You can not ignore such a versatile tool as Radare2.
To connect to the emulator using r2, the command would look like this:
radare2 -A -a arm -b 16 -d gdb://localhost:23946 rhino_fw42k6.elfLaunch available now (dc) and pause execution (Ctrl+C).
Unfortunately, at the moment there are problems in r2 when working with a hardware gdb server and memory markup, because of this, breakpoints and Steps do not work (command ds). We hope this will be fixed soon.
Launching with Eclipse
One of the options for using the emulator is to debug the firmware of the device being developed. For clarity, we will also use the Rhino firmware. You can download the firmware sources .
As an IDE, we will use Eclipse from the set .
In order for the firmware directly assembled in Eclipse to be loaded into the emulator, you need to add the parameter firmware=null to the emulator start command:
binkopycat -g 23946 -n rhino -l user -y modules -p firmware=null,tty_dbg=COM26,tty_bt=COM28Setting up a debug configuration
In Eclipse select menu Run - Debug Configurations... In the window that opens, in the section GDB Hardware Debugging you need to add a new configuration, and then on the "Main" tab, specify the current project and application for debugging:
On the "Debugger" tab, you must specify the GDB command:
${openstm32_compiler_path}arm-none-eabi-gdb
And also enter the parameters for connecting to the GDB server (host and port):
On the "Startup" tab, you must specify the following parameters:
- turn on the checkbox load image (so that the assembled firmware image is loaded into the emulator);
- turn on the checkbox Load symbols;
- add run command:
set $pc = *0x08000004(put in the PC register the value from the memory at the address0x08000004- the address is stored there ResetHandler).
Note, if you don't want to download the firmware file from Eclipse, then the options load image и Run commands do not need to be specified.
After pressing Debug, you can work in debugger mode:
- stepping through code
- interaction with breakpoints
Note. Eclipse has, hmm... some quirks... and you have to live with them. For example, if the message “No source available for “0x0″” appears when you start the debugger, then execute the Step (F5) command
Instead of a conclusion
Native code emulation is a very interesting thing. For a device developer, it becomes possible to debug firmware without a real device. For a researcher, it is the ability to conduct dynamic code analysis, which is not always possible even with a device.
We want to provide specialists with a tool that would be convenient, moderately simple and would not take much effort and time to set up and launch.
Write in the comments about your experience with hardware emulators. We invite you to discuss and will be happy to answer questions.
Only registered users can participate in the survey. , you are welcome.
What are you using the emulator for?
-
I develop (debug) firmware
-
researching firmware
-
launch games (Dendi, Sega, PSP)
-
something else (write in the comments)
7 users voted. 2 users abstained.
What software do you use to emulate native code?
-
QEMU
-
Unicorn engine
-
Proteus
-
something else (write in the comments)
6 users voted. 2 users abstained.
What would you like to improve in the emulator you are using?
-
want speed
-
I want ease of setup / launch
-
I want more options for interacting with the emulator (API, hooks)
-
everything suits me
-
something else (write in the comments)
8 users voted. 1 user abstained.
Source: habr.com