Ubuntu application layer support¶

Video hardware codec support¶

Mpp is a set of video codec APIs provided by Rockchip for RK3399, and based on mpp, Rockchip provides a set of gstreamer codec plugins. Users can use the gstreamer to do video codec application according to their own needs, or directly call mpp to realize hardware codec acceleration.

The Ubuntu system released by Firefly has provided complete gstremaer and mpp support, and provides a corresponding demo for users to develop reference.

Gstreamer¶

Ubuntu 16.04, gstreamer 1.12 Already installed in the /opt/ directory
Ubuntu 18.04, gstreamer 1.12 Already installed in the system

/usr/local/bin/h264dec.sh Test H264 decoding

/usr/local/bin/h264enc.sh Test H264 coding

Users can refer to these two scripts to configure their own gstreamer application.

Mpp¶

The mpp-related dev packages are already installed on the system.

/opt/mpp/ files are the demo and source files for the mpp codec.

OpenGL-ES¶

RK3399 support OpenGL ES1.1/2.0/3.0/3.1。

The Ubuntu system released by Firefly already provides full OpenGL-ES support. Run glmark2-es2 to test openGL-ES support. If you want to avoid the impact of the screen refresh rate on the test results, you can use the following command test on the serial terminal.

# systemctl stop lightdm
# export DISPLAY=:0
# Xorg &
# glmark2-es2 –off-screen

In the Chromium browser, type: chrome://gpu in the address bar to see hardware acceleration support.

Note:

EGL is an extension of the OpenGL for the x window system on the arm platform, which is equivalent to the glx library under x86.
Since the driver mode setting used by Xorg will load libglx.so by default (disabling glx will cause some applications failing to detect the glx environment), libglx.so will search the dri implementation library in the system. However, rk3399 Xorg 2D acceleration is directly based on DRM, and does not implement dri library, so libglx.so will report the following error during startup.
```
(EE) AIGLX error: dlopen of /usr/lib/aarch64-linux-gnu/dri/rockchip_dri.so failed
```

This has no effect on system operation and does not need to be processed.

Based on the same reason, some applications will report the following errors during startup, without processing, and will not affect the operation of the application.
```
libGL error: unable to load driver: rockchip_dri.so
libGL error: driver pointer missing
libGL error: failed to load driver: rockchip
```
Some versions of Ubuntu released by Firefly have turned off loading libglx.so by default. In some cases, some applications will run the following error.

GdkGLExt-WARNING **: Window system doesn't support OpenGL.

The method of correction is as follows：

Delete /etc/X11/xorg.conf.d/20-modesetting.conf three-line configuration。
```
Section "Module"
     Disable     "glx"
EndSection
```

OpenCL¶

The Ubuntu system released by Firefly has added opencl1.2 support, and can run the system’s built-in clinfo to get the platform opencl related parameters.

firefly@firefly:~$ clinfo
Platform #0
 Name:                                  ARM Platform
 Version:                               OpenCL 1.2 v1.r14p0-01rel0-git(966ed26).f44c85cb3d2ceb87e8be88e7592755c3

 Device #0
   Name:                                Mali-T860
   Type:                                GPU
   Version:                             OpenCL 1.2 v1.r14p0-01rel0-git(966ed26).f44c85cb3d2ceb87e8be88e7592755c3
   Global memory size:                  1 GB 935 MB 460 kB
   Local memory size:                   32 kB
   Max work group size:                 256
   Max work item sizes:                 (256, 256, 256)
…

OpenCV¶

The support of opencv-3.4.5 has been added to the Ubuntu system of Face-RK3399. The test files are /usr/loacl/bin/camera.py and /usr/local/bin/test_opencvrkisp.sh. After the camera device is registered successfully, run ./test_opencv_rkisp.sh to preview the camera.

The content of /usr/loacl/bin/camera.py is as follows:

import numpy as np
import cv2

#preview /dev/video0
#cap = cv2.VideoCapture("rkisp device=/dev/video0 io-mode=1 analyzer=1 enable-3a=1 path-iqf=/etc/cam_iq.xml ! video/x-raw,format=NV12,width=640,height=480, framerate=30/1 ! videoconvert ! appsink", cv2.CAP_GSTREAMER)

#preview /dev/video5
cap = cv2.VideoCapture("rkisp device=/dev/video5 io-mode=1 analyzer=1 enable-3a=1 path-iqf=/etc/cam_iq.xml ! video/x-raw,format=NV12,width=640,height=480, framerate=30/1 ! videoconvert ! appsink", cv2.CAP_GSTREAMER)

if not cap.isOpened():
	print('VideoCapture not opened')
		exit(0)

	while(True):
	    # Capture frame-by-frame
	    ret, frame = cap.read()
	    # Display the resulting frame
	    cv2.imshow('Hello RKISP-OpenCv',frame)

	    if cv2.waitKey(1) & 0xFF == ord('q'):
            break

# When everything done, release the capture
cap.release()
cv2.destroyAllWindows()

The content of /usr/loacl/bin/test_opencv_rkisp.sh is as follows:

#!/bin/sh

export DISPLAY=:0.0
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/aarch64-linux-gnu/gstreamer-1.0

#export GST_DEBUG=ispsrc:5
#export GST_DEBUG_FILE=/tmp/2.txt
echo "Start RKISP Camera Preview!"
#echo 7 > /sys/module/video_rkisp1/parameters/debug
su root -c "python3 /usr/local/bin/camera.py"

Run the script to see the camera image

_images/mipi_csi.png

TensorFlow Lite¶

RK3399 support for neural network GPU acceleration scheme LinuxNN, Firefly released Ubuntu system, has added support for LinuxNN.

Under opt/tensorflowbin/, run test.sh to test the Demo of the MobileNet model image classifier and the target detection of the MobileNet-SSD model.

firefly@firefly:/opt/tensorflowbin$ ./test.sh
Loaded model mobilenet_ssd.tflite
resolved reporter
nn version: 1.0.0
findAvailableDevices
filename:libarmnn-driver.so     d_info:40432     d_reclen:40s
[D][ArmnnDriver]: Register Service: armnn (version: 1.0.0)!
first invoked time: 1919.17 ms
invoked
average time: 108.4 ms
validCount: 26
car     @ (546, 501) (661, 586)
car     @ (1, 549) (51, 618)
person  @ (56, 501) (239, 854)
person  @ (332, 530) (368, 627)
person  @ (391, 541) (434, 652)
person  @ (418, 477) (538, 767)
person  @ (456, 487) (602, 764)
car     @ (589, 523) (858, 687)
person  @ (826, 463) (1034, 873)
bicycle @ (698, 644) (1128, 925)
write out.jpg succ!

Screen rotation¶

The Ubuntu system released by Firefly, if you need to rotate the display direction of the system by default, you can modify the direction of the corresponding display device in /etc/default/xrandr.

firefly@firefly:~$ cat /etc/default/xrandr
#!/bin/sh

# Rotation can be one of 'normal', 'left', 'right' or 'inverted'.

# xrandr --output HDMI-1 --rotate normal
# xrandr --output LVDS-1 --rotate normal
# xrandr --output EDP-1 --rotate normal
# xrandr --output MIPI-1 --rotate normal
# xrandr --output VGA-1 --rotate normal
# xrandr --output DP-1 --rotate normal

For platforms with a touch screen, if you need to rotate the direction of the touch screen, you can modify the SwapAxes, InvertX, InvertY values in /etc/X11/xorg.conf.d/05-gslX680.conf.

firefly@firefly:~$ cat /etc/X11/xorg.conf.d/05-gslX680.conf
Section "InputClass"
        Identifier "gslX680"
        MatchIsTouchscreen "on"
        MatchProduct  "gslX680"
        Driver "evdev"
        Option "SwapAxes" "off"
       # Invert the respective axis.
        Option "InvertX" "off"
        Option "InvertY" "off"
EndSection

Virtual keyboard¶

In the Ubuntu system released by Firefly, you can execute the onboard at the menu to open a virtual keyboard.

_images/onboard.jpg

Sound setting¶

The Firefly-RK3399 board generally has two or more audio devices. Headphone and HDMI are two common audio devices. Below is an example of audio settings for users to refer to.

Specify audio devices in terminal¶

Audio files in the system are in /usr/share/sound/alsa, please check your sound card before play them:

root@firefly:~# aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: rockchiprt5640c [rockchip,rt5640-codec], device 0: ff890000.i2s-rt5640-aif1 rt5640-aif1-0 []
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: rkhdmidpsound [rk-hdmi-dp-sound], device 0: HDMI-DP multicodec-0 []
  Subdevices: 1/1
  Subdevice #0: subdevice #0

And then specify a sound card to play an audio file. Among them, card0 stands for headphone and card1 stands for HDMI. We usually run command aplay -Dhw:0,0 Fornt_Center.wav to play an audio file, but in order to avoid playback failure because the audio files in the system are mono, so you can run as follow:

#choose headphone to play
root@firefly:/usr/share/sounds/alsa# aplay -Dplughw:0,0 Front_Center.wav
Playing WAVE 'Front_Center.wav' : Signed 16 bit Little Endian, Rate 48000 Hz, Mono

#choose HDMI to play
root@firefly:/usr/share/sounds/alsa# aplay -Dplughw:1,0 Front_Center.wav
Playing WAVE 'Front_Center.wav' : Signed 16 bit Little Endian, Rate 48000 Hz, Mono

Select audio device in graphic interface¶

Click on the sound icon, open Sound Setting，then click Configuration, and you can see two audio devices. For example, if HDMI is set to output audio, HDMI’s sound card device is selected as Output, and the other sound card is set as Off. (If HDMI is silent or quiet, try pressing the physical button on the HDMI screen to increase the volume.)

_images/sound_setting.jpg

Start-up applications while booting up¶

In the system, the application program can be set up automatically according to the user’s needs.

Select in turn from the System Menu Bar, Preferences -> Default applications for LXSession, open the setting interface, and you can set the default opening mode of the application in Launching applocations.

_images/autostart_setting.jpg

In the Autostart column, you can select applications that you need to boot. For example, Bluetooth does not start by default. If it needs to start automatically, it can start Bluetooth automatically by ticking the Blueman Applet.

_images/autostart_bluetooth.jpg

ADB debug¶

ADB, called Android Debug Bridge, is a command line debugging tool for Android that can perform various functions such as tracking system logs, uploading and downloading files, and installing applications. Take Firefly-RK3399 as an example, in order to use the ADB tool for debugging on the Ubuntu system, we have transplanted the ADB service on the board. However, since it is not the Android system, many ADB commands such as adb logcat, adb install, etc. are not available, and are only used as ordinary debugging assistant tools to perform operations such as shell interaction, uploading and downloading of files, etc. Similarly, network remote ADB debugging is not available.

Download ADB¶

In the Ubuntu system, run:

sudo apt-get install android-tools-adb

ADB connect¶

After the ADB is installed, the development board and PC are connected with the Micro USB OTG cable. Then check whether there is a device connection by command adb devices :

firefly@Desktop:~$ adb devices
List of devices attached
0123456789ABCDEF	device

When the device is connected successfully, enter the command line mode by entering the command adb shell :

firefly@Desktop:~$ adb shell
#

In this state, the current path of the command line can not be seen, and the Tab key completion function is invalid. It needs to enter a user to operate normally.

firefly@Desktop:~$ adb shell
# su firefly
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

firefly@firefly:/$

It can also enter the normal command line by entering the command adb shell /bin/bash.

Command adb help can print help information. Note that not all commands are available. Help information just for reference only.

Serial port¶

There are two types of serial ports on the development board: the normal ones that are built-in SoC UARTs, and the extended ones that are converted by SPI. The use of the converted serial port is exactly the same as of the normal one, but with different device file naming:

//uart
root@firefly:~# ls /dev/ttyS*
ttyS0  ttyS1  ttyS2  ttyS3

//spi to uart
root@firefly:~# ls /dev/ttysWK*
ttysWK0  ttysWK1  ttysWK2  ttysWK3

Set baud rate¶

Take ttyS4 as an example below. To view the serial port baud rate setting, run:

root@firefly:~# stty -F /dev/ttyS4
speed 9600 baud; line = 0;
-brkint -imaxbel

Set baud rate:

//Set the baud rate to 115200
root@firefly:~# stty -F /dev/ttyS4 ospeed 115200 ispeed 115200 cs8

//Confirm if it has been modified
root@firefly:~# stty -F /dev/ttyS4
speed 115200 baud; line = 0;
-brkint -imaxbel

Turn off echo¶

Echo is useful in an interative serial terminal session but will affect the result of data transfer, for example, in a loopback test. To turn off echo, run:

//Close echo
root@firefly:~# stty -F /dev/ttyS4 -echo -echoe -echok

//Check the configuration of all features and check if it is closed. The "-" means the feature is off
root@firefly:~# stty -F /dev/ttyS4 -a | grep echo
isig icanon iexten -echo -echoe -echok -echonl -noflsh -xcase -tostop -echoprt
echoctl echoke -flusho -extproc

Send and receive raw data¶

If the serial port is not in raw mode, there may be differences between the data sent and received (For example, the driver may do some line end converting, which is no good to binary data). Use stty to set it in raw mode to ensure that the same data is exactly sent and received:

root@firefly:~# stty -F /dev/ttyS4 raw

Operating mode¶

The serial port has two operating modes: interrupt mode and DMA mode.

Interrupt mode¶

By default, the kernel configures the serial port into interrupt mode so there is no need to make any change. In interrupt mode, the transfer latency is low, but it tends to lose data in bulk data transfer. Therefore do not use interrupt mode in that case.

DMA mode¶

The DMA mode is mainly used when transferring a large amount of data. The kernel will provide a buffer space for the serial port to receive data to minimize the packet loss rate of the serial port transmission.

WARNNING: The default size of the cache space is “8K”. If the transfer exceeds the cache size, the packet will be lost. Therefore, if the DMA mode is used, the sender needs to send multiple packets.

DTS setting:

&uart4 {
        status = "okay";
+       dmas = <&dmac_peri 8>, <&dmac_peri 9>;
+       dma-names = "tx", "rx";
};

DMA mode does not increase the transfer rate. Instead, the transfer rate is reduced because of the need for caching, so do not use DMA mode if you do not need to transfer large amounts of data.

Flow control¶

No matter whether it is interrupt mode or DMA mode, data transmission is not foolproof, because DDR, CPU frequency conversion or high occupancy may cause processing data not to be timely when transferring large amounts of data . At this time, it is necessary to use flow control. There are two types of flow control, one is software flow control and the other is hardware flow control. The following only describes the use of hardware flow control.

Hardware support¶

Hardware flow control requires hardware support. The CTX and RTX pins of the development board need to be connected to the device.

WARNNING: Not all serial ports on the development board support hardware flow control. Please confirm the hardware support from the schematic.

DTS setting¶

uart3: serial@ff1b0000 {
        compatible = "rockchip,rk3399-uart", "snps,dw-apb-uart";
        reg = <0x0 0xff1b0000 0x0 0x100>;
        clocks = <&cru SCLK_UART3>, <&cru PCLK_UART3>;
        clock-names = "baudclk", "apb_pclk";
        interrupts = <GIC_SPI 101 IRQ_TYPE_LEVEL_HIGH 0>;
        reg-shift = <2>;
        reg-io-width = <4>;
        pinctrl-names = "default";
+       pinctrl-0 = <&uart3_xfer &uart3_cts &uart3_rts>;
        status = "disabled";
};

WARNNING: Not all serial ports on the development board support hardware flow control. Please confirm the hardware support from the schematic.

Application settings¶

The application also needs to turn on flow control. Here is how to do that with stty:

//Turn on flow control
root@firefly:~# stty -F /dev/ttyS4 crtscts

//Check if the flow control is turned on, the "-" means the function has been turned off.
root@firefly:~# stty -F /dev/ttyS4 -a | grep crtscts
-parenb -parodd -cmspar cs8 hupcl -cstopb cread clocal crtscts

USB Ethernet¶

Kernel setting¶

In the kernel directory, run:

make firefly_linux_defconfig
make menuconfig

And then select in turn, Device Drivers -> USB Support -> USB Gadget Support

Select Ethernet Gadget (with CDC ethernet support) in the USB Gadget Driver. Meanwhile, select RNDIS support.

<*>   USB Gadget Drivers (Ethernet Gadget (with CDC Ethernet support))  --->
        Ethernet Gadget (with CDC Ethernet support)
[*]       RNDIS support (NEW)

And then compile the kernel in the kernel directory.

make rk3399-firefly.img -j12

After compiling, flash the kernel into the development board. For the flashing process, please refer to the wiki tutorial: Flash partition image

IP address setting¶

Connect PC and development board with Type-C data line and execute lsusb command in PC, if you can see the USB Ethernet device, it means that the connection is successful.

firefly@Desktop:~$ lsusb
Bus 002 Device 003: ID 09da:5814 A4Tech Co., Ltd.
Bus 002 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 005: ID 04f2:b2ea Chicony Electronics Co., Ltd Integrated Camera [ThinkPad]
Bus 001 Device 004: ID 0a5c:21e6 Broadcom Corp. BCM20702 Bluetooth 4.0 [ThinkPad]
Bus 001 Device 003: ID 147e:1002 Upek Biometric Touchchip/Touchstrip Fingerprint Sensor
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 003: ID 0525:a4a2 Netchip Technology, Inc. Linux-USB Ethernet/RNDIS Gadget
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

In the output information, ID 0525:a4a2 Netchip Technology, Inc. Linux-USB Ethernet/RNDIS Gadget is the USB Ethernet.

Development board insert wire and can connect external network.

Setting Development Board IP

Enter ifconfig -a, and you can get the information.

root@firefly:~# ifconfig -a

# eth0 is the Cable Network Card
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 168.168.100.48  netmask 255.255.0.0  broadcast 168.168.255.255
        inet6 fe80::1351:ae2f:442e:e436  prefixlen 64  scopeid 0x20<link>
        ether 8a:4f:c3:77:94:ac  txqueuelen 1000  (Ethernet)
        RX packets 9759  bytes 897943 (897.9 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 236  bytes 35172 (35.1 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 42  base 0x8000

        ...

# usb0 is the virtual USB Network Card
usb0: flags=4098<BROADCAST,MULTICAST>  mtu 1500
        ether 4a:81:b1:34:d2:ad  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Then customize an appropriate IP for the usb0 network card. Note that the IP of usb0 and eth0 are not in the same network segment.

ifconfig usb0 192.168.1.101

Setting IP in PC

# First, check the IP of USB virtual network card
firefly@Desktop:~$ ifconfig
enp0s20u2i1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.2.90  netmask 255.255.255.0  broadcast 192.168.2.255
        inet6 fe80::871c:b87e:1327:7fd4  prefixlen 64  scopeid 0x20<link>
        ether 46:fe:6e:97:ee:a6  txqueuelen 1000  (以太网)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1  bytes 54 (54.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
...


# Setting IP address of USB network card and usb0 IP address of development board in the same network segment.
firefly@Desktop:~$ sudo ifconfig enp0s20u2i1 192.168.1.100

#The default gateway is set to the IP of development board usb0
firefly@Desktop:~$ sudo route add default gw 192.168.1.101

After IP setting, PC and development board can ping each other.

PC Internet Access¶

In the development board, turn on the forwarding function of IPv4.

echo 1 > /proc/sys/net/ipv4/ip_forward

If you want to automatically turn on the forwarding function every time you restart the development board, please modify the net.ipv4.ip_forward value of the /etc/sysctl.conf file to 1. And remember to run sysctl -p to make IPv4 forwarding effective.

Then empty the default rules of iptables and add rules for traffic forwarding.

#empty the default rules
iptables -F
iptables -X
iptables -Z

#add rule
iptables -t nat -A POSTROUTING -s 192.168.1.0/24 -o eth0 -j SNAT --to-source 168.168.100.48

Now, the PC can access the net. Please note as follow:

Corresponding to each IP address on the device in the above steps, pay attention to the fact that the USB virtual network card IP and wired network IP on the development board are not in the same network segment.
The gateway of PC USB network card should be set as IP address of virtual USB network card on development board.
The virtual network card IP address, IP forwarding function, traffic forwarding rules and other settings on the development board will be restored after the device is restarted.

External storage device rootfs¶

In addition to using the root filesystem in the internal eMMC, you can also use the root filesystem of the external storage device, such as SD card. The following is an example of Firefly-RK3399 mounting the root filesystem of SD card.

Create partitions on SD card¶

Insert SD card on PC and use gdisk tool to separate out a partition that loads the root filesystem.

Using fdisk -l to get the device name of the SD card and enter the corresponding device with gdisk command, and then run:

sudo gdisk /dev/sdb

GPT fdisk (gdisk) version 1.0.3

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help):

-------------------------------------------------------------------

#enter '?' can print the help information
#enter 'p' to display all partitions of SD card

Command (? for help): p
Disk /dev/sdb: 15278080 sectors, 7.3 GiB
Model: CARD-READER
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 5801AE61-92ED-42FB-A144-E522E8E15827
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 15278046
Partitions will be aligned on 2048-sector boundaries
Total free space is 15278013 sectors (7.3 GiB)

Number  Start (sector)    End (sector)  Size       Code  Name

-------------------------------------------------------------------

#now create a new partition, create the appropriate partition size to the root filesystem according to your actual situation.

Command (? for help): n     #entern 'n' to create a new partition
Partition number (1-128, default 1): 1      #create No.1 partition
First sector (34-15278046, default = 2048) or {+-}size{KMGTP}:   #press enter, use the default value.
Last sector (2048-15278046, default = 15278046) or {+-}size{KMGTP}: +3G     #partition size is 3G
Current type is 'Linux filesystem'
Hex code or GUID (L to show codes, Enter = 8300):    #press enter, use the default value.
Changed type of partition to 'Linux filesystem'

-------------------------------------------------------------------

#then enter 'i' to get the partition's unique GUID, and mark down it.

Command (? for help): i
Using 1
Partition GUID code: 0FC63DAF-8483-4772-8E79-3D69D8477DE4 (Linux filesystem)
Partition unique GUID: 6278AF5D-BC6B-4213-BBF6-47D333B5CB53
First sector: 2048 (at 1024.0 KiB)
Last sector: 6293503 (at 3.0 GiB)
Partition size: 6291456 sectors (3.0 GiB)
Attribute flags: 0000000000000000
Partition name: 'Linux filesystem'

-------------------------------------------------------------------

#enter 'wq' save and exit gdisk

Command (? for help): wq

Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): y
OK; writing new GUID partition table (GPT) to /dev/sdb.
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.

Format the newly created partition.

sudo mkfs.ext4 /dev/sdb1

When the formatting is completed, use the dd command to flash the root filesystem into the newly created partition of the SD card. The production of root filesystem can be referred to Compile Ubuntu rootfs.

#Fill in the root filesystem path and device name according to your situation.
dd if=/rootfs_path/rootfs.img of=/dev/sdb1

Mount root filesystem of SD card¶

Firstly, we need to modify the device tree file, open the corresponding DTS file, and rewrite the root value of the bootargs parameter under the chosen node as the unique GUID recorded before.

#Change the root value to the first 12 digits of the unique GUID.
chosen {
    bootargs = "earlycon=uart8250,mmio32,0xff1a0000 swiotlb=1 console=ttyFIQ0 rw root=PARTUUID=6278AF5D-BC6B  rootfstype=ext4 rootwait";
};

And then compile the kernel and flash it into the development board. After the root filesystem is flashed to SD card, insert SD card into the TF slot of the development board, and the root filesystem can be accessed on boot.

Note:

Before operations, please backing up important files in USB disk or SD card to avoid data lost.
When operating, please confirm the device name corresponding to your SD card.
The value of the root parameter in the DTS file is unique GUID. Just write down the first 12 bits. You can also modify the value according to the help information of gdisk.

Network startup¶

Network startup is to use TFTP to download the kernel and DTB file from the server to the memory of the target machine and mount the network root filesystem to the target machine through NFS server to achieve diskless startup. Following is an example based on Firefly-RK3399 for user reference.

Prerequisite:

Firefly-Rk3399 development board.
Router and network cable.
A server with TFTP and NFS.
A root filesystem.

Server deployment¶

Deploy TFTP on the server

Install TFTP server.

sudo apt-get install tftpd-hpa

Create /tftpboot directory and grant permission:

mkdir /tftpboot
sudo chmod 777 /tftpboot

And then modify TFTP server configuration file /etc/default/tftpd-hpa :

# /etc/default/tftpd-hpa

TFTP_USERNAME="tftp"
TFTP_DIRECTORY="/tftpboot"
TFTP_ADDRESS="0.0.0.0:69"
TDTP_OPTIONS="-c -s -l"

Then restart TFTP server.

sudo service tftpd-hpa restart

Create a new file in /tftpboot to test TFTP:

firefly@Desktop:~$ cd /tftpboot/
firefly@Desktop:/tftpboot$ touch test
firefly@Desktop:/tftpboot$ cd /tmp
firefly@Desktop:/tmp$ tftp 127.0.0.1
tftp> get test
tftp> q

And if you can see the test file in tmp, it means TFTP server is working.

Deloy NFS on the server

Install NFS server:

sudo apt-get install nfs-kernel-server

Create a shared directory:

sudo mkdir /nfs
sudo chmod 777 /nfs
cd /nfs
sudo mkdir rootfs
sudo chmod 777 rootfs

And then copy the root filesystem into the /nfs/rootfs. The production of root filesystem can be referred to Compile Ubuntu rootfs.

Add the shared directory path in /etc/exports:

/nfs/rootfs *(rw,sync,no_root_squash,no_subtree_check)

The shared directory is set according to the user’s actual situation, in which * represents all users’ access.

Restart NFS server:

sudo /etc/init.d/nfs-kernel-server restart

Locally mount the shared directory to test whether the NFS server is available:

sudo mount -t nfs 127.0.0.1:/nfs/rootfs/ /mnt

Consistent content with nfs/rootfs seen in the mnt directory means NFS server is working. And then umount:

sudo umount /mnt

Kernel configuration¶

If you want to mount the network root filesystem, you need to do the relevant configuration in the kernel and modify the configuration in DTS.

Firstly, setting kernel, run make menuconfig in kernel directory, and select the relevant configuration:

[*] Networking support  --->
         Networking options  --->
                [*]   IP: kernel level autoconfiguration
                [*]     IP: DHCP support
                [*]     IP: BOOTP support
                [*]     IP: RARP support


File systems  --->
        [*] Network File Systems  --->
                [*]   Root file system on NFS

Modify the device tree file rk3399-firefly.dts, and rewrite the root value of the bootargs parameter under the chosen:

chosen {
    bootargs = "earlycon=uart8250,mmio32,0xff1a0000 swiotlb=1 console=ttyFIQ0 root=/dev/nfs rootfstype=ext4 rootwait";
};

root=/dev/nfs means mount network root filesystem by NFS.

Compile kernel:

make ARCH=arm64 rk3399-friefly.img -j12

After compile kernel, copy boot/img, rk3399-firefly.dtb to the /tftpboot:

cp boot.img /tftpboot
cp arch/arm64/boot/dts/rockchip/rk3399-firefly.dtb /tftpboot

Detailed instructions can be referred to in the kernel directory: kernel/Documentation/filesystems/nfs/nfsroot.txt

U-Boot setting¶

The target machine plugs in the wire and connects to the server. Then startup and enter U-Boot command line mode and set the following parameters:

=> setenv ipaddr 192.168.31.101     #target machine IP
=> setenv serverip 192.168.31.106   #set serverip as the server IP

#Set up to download the kernel and DTB files from TFTP. Modify the corresponding address according to actual target machine.
=> setenv bootcmd tftpboot 0x0027f800 boot.img \; tftpboot 0x08300000 rk3399-firefly.dtb \; bootm 0x0027f800 - 0x08300000

#Set up mount network root filesystem, ip=<client-ip>:<server-ip>:<gw-ip>:<netmask>:<hostname>:<device>:<autoconf>
=> setenv bootargs root=/dev/nfs rw nfsroot=192.168.31.106:/nfs/rootfs,v3 ip=192.168.31.101:192.168.31.106:192.168.31.1:255.255.255.0::eth0:off

#save the setting
=> saveenv
Saving Environment to MMC...
Writing to MMC(0)... done

#start up the target machine
=> boot
ethernet@fe300000 Waiting for PHY auto negotiation to complete. done
Speed: 100, full duplex
Using ethernet@fe300000 device
TFTP from server 192.168.31.106; our IP address is 192.168.31.101
Filename 'boot.img'.
Load address: 0x27f800
Loading: #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         ##################################################
         475.6 KiB/s
done
Bytes transferred = 20072448 (1324800 hex)
Speed: 100, full duplex
Using ethernet@fe300000 device
TFTP from server 192.168.31.106; our IP address is 192.168.31.101
Filename 'rk3399-firefly.dtb'.
Load address: 0x8300000
Loading: #######
         645.5 KiB/s
done
Bytes transferred = 97212 (17bbc hex)
## Booting Android Image at 0x0027f800 ...
Kernel load addr 0x00280000 size 19377 KiB
## Flattened Device Tree blob at 08300000
   Booting using the fdt blob at 0x8300000
   XIP Kernel Image ... OK
   Loading Device Tree to 0000000073edc000, end 0000000073ef6bbb ... OK
Adding bank: 0x00200000 - 0x08400000 (size: 0x08200000)
Adding bank: 0x0a200000 - 0x80000000 (size: 0x75e00000)
Total: 912260.463 ms

Starting kernel ...

...

You can see in the boot kernel log:

[   12.146297] VFS: Mounted root (nfs filesystem) on device 0:16.

This means that the network root filesystem has been mounted.

Note:

Ensure that TFTP server and NFS server are working.
Make sure that the target machine inserts the network cable first and then boot-up, and in the same LAN with the server. If it is directly connected to the target machine and the server, please use the crossover network cable.
In the kernel configuration, Root file system on NFS depends on IP: kernel level autoconfiguration, please select IP: kernel level autoconfiguration first, and then you can find Root file system on NFS and select it.
When setting up the remote root file system for mounting, nfsroot = 192.168.31.106:/nfs/rootfs,v3, v3 represents the version information of NFS, Please add it to avoid unsuccessful mounting.

Update kernel and U-Boot online¶

This section describes a simple process for online updates. Package the kernel, U-Boot or other files that need to be updated into DEB, and then import them into the local package repository to download and update automatically on the development board.

Prepare DEB installation package¶

Kernel and U-Boot update files are ready: boot.img, trust.img, uboot.img.

DEB is a software package format of Debian Linux. The key to packaging is to create a control file in the DEBIAN directory. Now create DEB working directory:

mkdir deb
cd deb
mkdir firefly-firmware
cd firefly-firmware
mkdir DEBIAN
mkdir -p usr/share/{kernel,uboot}

# put the update files to corresponding directory
mv ~/boot.img ~/deb/firefly-firmware/usr/share/kernel
mv ~/uboot.img ~/deb/firefly-firmware/usr/share/uboot
mv ~/trust.img ~/deb/firefly-firmware/usr/share/uboot

The files stored in the DEBIAN directory are the control files for DEB package installation and the corresponding script files.

Create control files control and script files postinst under the DEBIAN directory, the control file is used to record software package name, version number, platform, dependency information, and other data.

Package: firefly-firmware #directly name
Version: 1.0
Architecture: arm64
Maintainer: neg
Installed-Size: 1
Section: test
Priority: optional
Descriptionon: This is a deb test

The postinst file is as follows:

echo "-----------uboot updating------------"
dd if=/usr/share/uboot/uboot.img of=/dev/disk/by-partlabel/uboot

echo "-----------trust updating------------"
dd if=/usr/share/uboot/trust.img of=/dev/disk/by-partlabel/trust

echo "-----------kernel updating------------"
dd if=/usr/share/kernel/boot.img of=/dev/disk/by-partlabel/boot

Note: postinst is a script that runs after unpacking data, and other correspondingly scripts:

preinst: A script that runs before unpacking data.
prerm: A script that runs before deleting files when uninstalled.
postrm: A script that runs after deleting a file.

Only the preinst script is used here.

Here is the directory tree created:

deb
└── firefly-firmware
    ├── DEBIAN
    │   ├── control
    │   └── postinst
    └── usr
        └── share
            ├── kernel
            │   └── boot.img
            └── uboot
                ├── trust.img
                └── uboot.img

Enter the deb directory and generate the DEB package with the dpkg command:

dpkg -b firefly-firmware firefly-firmware_1.0_arm64.deb

When generating DEB packages, they are usually named according to this specification: package_version-reversion_arch.deb.

Create a local repository¶

Install the packages we need:

sudo apt-get install reprepro gnupg

Then use GnuPG tool to generate a GPG key. After executing the command, follow the prompt to operate.

gpg --gen-key

Run sudo gpg --list-keys, can see the key information:

sudo gpg --list-keys

gpg: WARNING: unsafe ownership on homedir '/home/firefly/.gnupg'
/home/firefly/.gnupg/pubring.kbx
--------------------------------
pub   rsa3072 2019-05-31 [SC] [expires: 2021-05-30]
      BCB65788541D632C057E696B8CBC526C05417B76
	  uid           [ultimate] firefly <firefly@t-chip.com>
	  sub   rsa3072 2019-05-31 [E] [expires: 2021-05-30]

Next, create the package repository:

cd /var/www
mkdir apt
mkdir -p ./apt/incoming
mkdir -p ./apt/conf
mkdir -p ./apt/key

Export the previously generated keys to the repository folder. Please correspond to the user name and mailbox address you created.

gpg --armor --export firefly firefly@t-chip.com > /var/www/apt/key/deb.gpg.key

Create a distributions file in the conf directory, which reads as follows:

Origin: Neg   #your name
Label: Mian     #package repository name
Suite: stable   #(stable or unstable)
Codename: bionic    #codename
Version: 1.0
Architectures: arm64
Components: main    #components name,such as main，universe.
Description: Deb source test
SignWith: BCB65788541D632C057E696B8CBC526C05417B76 #the key you generated

Establishment of repository tree:

reprepro --ask-passphrase -Vb /var/www/apt export

Add firefly-firmware_1.0_arm64.deb into repository:

reprepro --ask-passphrase -Vb /var/www/apt includedeb bionic ~/deb/firefly-firmware_1.0_arm64.deb

View the file in the repository:

root@Desktop:~# reprepro -b /var/www/apt/ list bionic
bionic|main|arm64: firefly-firmware 1.0

If you want to remove the file in the repository, just run:

reprepro --ask-passphrase -Vb /var/www/apt remove bionic firefly-firmware

And then install nginx server:

sudo apt-get install nginx

Modify nginx configuration file /etc/nginx/sites-available/default:

server {
    listen 80 default_server;
    listen [::]:80 default_server;

    root /var/www/apt;

    access_log /var/log/nginx/repo.access.log;
    error_log   /var/log/nginx/repo.error.log;

    location ~ /(db|conf) {
        deny all;
        return 404;
    }
}

Restart nginx server.

sudo service nginx restart

Client update¶

In the client development board, first, add the source of the local package repository and add a new configuration file bionic. list under the directory etc/apt/sources.list.d, which is as follows:

deb http://192.168.31.106 bionic main

IP address is the nginx server address, bionic is the repository codename, main is the components name.

Get and add the GPG key from the server:

wget -O - http://192.168.31.106/key/deb.gpg.key | apt-key add -

Update and install firefly-firmware_1.0_arm64:

root@firefly:/home/firefly# apt-get update
Hit:1 http://192.168.31.106 bionic InRelease
Hit:2 http://wiki.t-firefly.com/firefly-rk3399-repo bionic InRelease
Hit:3 http://ports.ubuntu.com/ubuntu-ports bionic InRelease
Hit:4 http://archive.canonical.com/ubuntu bionic InRelease
Hit:5 http://ports.ubuntu.com/ubuntu-ports bionic-updates InRelease
Hit:6 http://ports.ubuntu.com/ubuntu-ports bionic-backports InRelease
Hit:7 http://ports.ubuntu.com/ubuntu-ports bionic-security InRelease
Reading package lists... Done

root@firefly:/home/firefly# apt-get install firefly-firmware
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following package was automatically installed and is no longer required:
  libllvm6.0
Use 'apt autoremove' to remove it.
The following NEW packages will be installed:
  firefly-firmware
0 upgraded, 1 newly installed, 0 to remove and 3 not upgraded.
Need to get 0 B/6982 kB of archives.
After this operation, 1024 B of additional disk space will be used.
Selecting previously unselected package firefly-firmware.
(Reading database ... 117088 files and directories currently installed.)
Preparing to unpack .../firefly-firmware_1.0_arm64.deb ...
Unpacking firefly-firmware (1.0) ...
Setting up firefly-firmware (1.0) ...
-----------uboot updating------------
8192+0 records in
8192+0 records out
4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.437281 s, 9.6 MB/s
-----------trust updating------------
8192+0 records in
8192+0 records out
4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.565762 s, 7.4 MB/s
-----------kernel updating------------
39752+0 records in
39752+0 records out
20353024 bytes (20 MB, 19 MiB) copied, 0.1702 s, 120 MB/s

You can see that the poinst script in the DEB package is executed during installation. After installation, restart the development board to take effect by loading the new images.

In /usr/share directory, we can see kernel and uboot directory with boot.img, uboot.img, trust.img in it.

Note:

In making DEB package, the directory at the same level as DEBIAN is regarded as the root directory. That is, the files placed in the same directory as DEBIAN, after the DEB package is installed on the client side, can be found in the root directory of the target system.
The files and scripts in DEB package should be adjusted according to your actual situation.
Every time the configuration file is modified in the repository, the repository directory tree should be re-imported.
In nginx server configuration, the root parameter configures the path of the repository. Please modify it according to the actual situation.
When the client adds a new download source file, pay attention to check the correct server IP address, package repository codename and component name. Note that the client needs to connect to the server.
Clients need to add GPG keys with the apt-key add command before getting information about the local repository.

Distributed compiling with Docker¶

distcc is a program that compiles C, C++, Objective-C or Objective-C++ code distributed by several machines on the network. distcc does not require all machines to share file systems, have synchronous clocks, or install the same libraries and header files, as long as the server machine has the appropriate compilation tools. This demo deploys distcc service using Docker to two Firefly-RK3399 development boards (arm64) and one PC (x86_64), which make it possible to utilizing distcc services to accelerate the compilation of Linux kernel on any one of the devices.

Prerequisites¶

Tow Firefly-RK3399 development boards.
Routers and network cable.
PC machine.

Connect both client and server to the same LAN. After the connection, the corresponding IP addresses are:

PC : 192.168.1.100
development board A: 192.168.1.101
development board B: 192.168.1.102

PC Deployment¶

Install Docker with scripts:

wget -qO- https://get.docker.com/ | sh

In order to enable the current ordinary user can execute the docker command, you need to add the current user to the docker group:

sudo groupadd docker            #add the docker group
sudo gpasswd -a $USER docker    #add current user to the docker group
newgrp docker                   #update docker group

Start the Docker server:

sudo service docker start

Create a file Dockerfile_distcc.x86_64, which reads as follows:

FROM ubuntu:bionic
MAINTAINER Firefly <service@t-firefly.com>

ARG DEBIAN_FRONTEND=noninteractive

RUN apt-get update \
	&& apt-get install -y --no-install-recommends --no-install-suggests\
		gcc-aarch64-linux-gnu distcc\
    && apt-get clean \
    && rm -rf /var/lib/apt/lists

RUN ln -s aarch64-linux-gnu-gcc /usr/bin/gcc &&\
	ln -s aarch64-linux-gnu-gcc /usr/bin/cc

EXPOSE 3632

ENTRYPOINT ["/usr/bin/distccd"]
CMD ["--no-detach" , "--log-stderr" , "--log-level", "debug", "--allow" , "0.0.0.0/0"]

Generate a Docker image:

docker build -t distcc_server:x86_64 -f Dockerfile_distcc.x86_64 .

The generated Docker image can be viewed with the command docker images:

docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
distcc_server       x86_64              138c0b7e3801        9 minutes ago       66.1MB

Run a Docker container with the new image, exposing distcc service on TCP port 3632 of the host network:

docker run -d -p 3632:3632 distcc_server:x86_64

List the running containers with the command docker ps:

docker ps
CONTAINER ID    IMAGE                  COMMAND                  CREATED         STATUS          PORTS                    NAMES
fa468d068185    distcc_server:x86_64   "/usr/bin/distccd --…"   9 minutes ago   Up 9 minutes    0.0.0.0:3632->3632/tcp   epic_chatterjee

Development boards deployment¶

The Firefly-RK3399 kernel does not support docker by default and needs to be configured. File /kernel/arch/arm64/configs/firefly_linux_defconfig can be modified with reference to Firefly Github .

After modification, compile and update the kernel to the development board. And then install docker on development board:

sudo apt-get update
wget -qO- https://get.docker.com/ | sh

Add the current user to the docker group:

sudo groupadd docker
sudo gpasswd -a $USER docker
newgrp docker

And then start Docker service:

sudo service docker start

Create a file Dockerfile_distcc.arm64, which reads as follows:

FROM ubuntu:bionic
MAINTAINER Firefly <service@t-firefly.com>

ARG DEBIAN_FRONTEND=noninteractive

RUN apt-get update \
	&& apt-get install -y --no-install-recommends --no-install-suggests\
	gcc distcc\
    && apt-get clean \
    && rm -rf /var/lib/apt/lists

EXPOSE 3632

ENTRYPOINT ["/usr/bin/distccd"]
CMD ["--no-detach" , "--log-stderr" , "--log-level", "debug", "--allow" , "0.0.0.0/0"]

Generate a Docker image:

docker build -t distcc_server:arm64 -f Dockerfile_distcc.arm64 .

Run a container with the new image, exposing distcc service on TCP port 3632 of the host network:

docker run -d -p 3632:3632 distcc_server:arm64

Export the new image with the docker save command:

docker save -o distcc_server.tar distcc_server:arm64

Copy the generated distcc_server.tar file to another development board and import the Docker image file there:

docker load -i distcc_server.tar

Run a container with the imported image:

docker run -d -p 3632:3632 distcc_server:arm64

Note: If you have a Docker Hub account, you can push the image to the remote Docker repository with docker push, and pull the image simply with docker pull on another development board. Check the Docker document if you are not familiar.

Client compiles the kernel with distcc¶

Now that all three machines have deployed a distributed compilation environment, you can choose any of them as the client to invoke all the services. Here we choose the development boards as the client to compile the Linux kernel.

Create a file Dockerfile_compile.arm64, which reads as follows:

FROM ubuntu:bionic
MAINTAINER Firefly <service@t-firefly.com>

ARG DEBIAN_FRONTEND=noninteractive

RUN apt-get update \
	&& apt-get install -y --no-install-recommends --no-install-suggests\
	bc make python sed libssl-dev binutils build-essential distcc\
    liblz4-tool gcc \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists

Generate a Docker image:

docker build -t compile:arm64 -f Dockerfile_compile.arm64 .

Prepare the kernel source code before starting the container. Create a /etc/distcc/hosts file, which lists the IP addresses of all machines that provide distcc services. The content of the /etc/distcc/hosts file is as follows:

# As described in the distcc manpage, this file can be used for a global
# list of available distcc hosts.
#
# The list from this file will only be used, if neither the
# environment variable DISTCC_HOSTS, nor the file $HOME/.distcc/hosts
# contains a valid list of hosts.
#
# Add a list of hostnames in one line, seperated by spaces, here.
192.168.1.100
192.168.1.101
193.168.1.102

To get more accurate results, first clear the cache on the client development board:

echo 3 > /proc/sys/vm/drop_caches

Start a Docker container with the compile:arm64 image, mount the current kernel directory of the host into the mnt directory of the container, mount the etc/distcc/hosts file of the host into the etc/distcc/hosts of the container, and start the container in interactive mode with the parameter it:

docker run -it --rm -v $(pwd):/mnt -v /etc/distcc/hosts:/etc/distcc/hosts compile:arm64 /bin/bash
root@f4415264351b:/# ls
bin  boot  dev  etc  home  lib  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
root@f4415264351b:/# cd mnt/

Enter the mnt directory in the container, and then start compiling the kernel with distcc. Add the time command to view the time-consuming execution of the compilation command. The CC parameter specifies compiling with distcc.

time make ARCH=arm64 rk3399-firefly.img -j32 CC="distcc"

If you use a PC as a client, you need to compile with the following commands:

time make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- rk3399-firefly.img -j32 CC="distcc aarch64-linux-gnu-gcc"

During compilation, you can use the command distccmon-text 1 to view the compilation in a new window in the container:

distccmon-text 1
                    ...
Compile     perf_regs.c                              192.168.1.101[0]
Compile     fork.c                                   192.168.1.101[2]
Compile     dma-mapping.c                            192.168.1.101[3]
Compile     signal32.c                               192.168.1.100[0]
Compile     open.c                                   192.168.1.100[2]
Compile     traps.c                                  192.168.1.100[3]
Compile     arm64ksyms.c                             192.168.1.102[0]
Compile     mempool.c                                192.168.1.102[2]
Compile     filemap.c                                192.168.1.102[3]
Preprocess                                                localhost[0]
Preprocess                                                localhost[1]
Preprocess                                                localhost[2]
                    ...

When the final compilation command is completed, you can see the time taken for compilation:

real    15m44.809s
user    16m0.029s
sys     6m22.317s

Below is the time spent compiling the kernel using a single development board:

real    23m33.002s
user    113m2.615s
sys     9m29.348s

Comparisons show that distributed compilation using distcc can effectively improve the compilation speed.

Note:

Different platforms require different compilers. For example, on the x86_64 platform, the gcc-aarch64-linux-gnu cross-compiler tool needs to be installed. On the arm64 platform, only the gcc compiler tool needs to be installed. Users need to install the right tools in the Dockerfile to match the actual situation.
If the PC is used as the client, the parameter CC= "distcc aarch64-linux-gnu-gcc" should be used to specify compilation with aarch64-linux-gnu-gcc cross-compiler tool when compiling the kernel in the container.