Algorithm Adaptation

This chapter of “Application Development” is mainly aimed at two kinds of users: (1) Users use ready-made third-party model algorithms. At this time, the user does not need to worry about the details of model training, conversion and inference, just complete the deployment of the algorithm. (2) The user has implemented the algorithm and trained the model. Users need to check the section of “self-developed algorithm” and convert their model into the RKNN model available to the NPU in the CAM-CRV1126S2U/CAM-CRV1109S2U device. This chapter introduces the deployment and development of Rainbowsoft and Rockchip algorithms as examples.

Adapting ArcSoft Algorithm

ArcSoft provides the most basic SDK, which we call the pure version here, and also provides a complete face recognition management system. We call it the full version here. The pure version only contains the libraries needed for face recognition, and users can design UI and business logic according to their needs. The full version includes face recognition gate UI and face back-end management application, which greatly facilitates project development.

Full Version

The official firmware has a built-in full version by default, so it can be used directly after booting. The following steps can be used to deploy the full version as well as update the version.

  • Download the ArcFace Linux RV1109 version software package, ArcFaceGo_RV1109_App.zip.

  • Unzip ArcFaceGo_RV1109_App.zip.

  • Ensure that the device is connected to the host and ADB is available.

  • Execute the script to complete the installation and deployment.

chmod 777 firefly.sh
./firefly.sh

  • ADB executes the gate application startup script. At this time, you can see that the program is running when you connect to the screen, but the device is not activated and cannot perform face recognition.

adb shell ./userdata/arc/start_app.sh 90

  • The host computer sets the static IP of the USB network card to 172.16.110.1.

  • Open 172.16.110.2 in the browser to enter the Web.

  • Refer to the section “Device Networking” to complete device networking. If you use offline authorization, you can skip this step.

  • Log in to the documentation in the Web reference package to complete the activation. The specific application and usage method can also refer to the document.

Pure Version

  • Download the ArcFace Linux RV1109 SDK version software package, ArcFace_RV1109_SDK.zip

  • Unzip ArcFace_RV1109_SDK.zip

  • Refer to the section “Device Networking” to complete device networking. If you use offline authorization, you can skip this step.

  • Refer to the software package API documentation to complete device activation and application development. The technical requirements for using the pure version are relatively high, and you need to be familiar with the API calls of Rainbowsoft.

Adapt RK Algorithm

RK independently developed a set of face recognition algorithm called rockface. On the basis of rockface, RK encapsulates camera preview function, voice broadcast, face recognition UI part display function and face registration management function, etc. This set of applications based on rockface is called rkfacial. Here we also refer to rockface as pure version, and rkfacial as full version.

Code Location

First, you need to obtain the SDK. SDK source code acquisition and compilation will be discussed in “Peripheral Adaptation”.

  • sdk/external/rockface

  • sdk/external/rkfacial

  • sdk/app/QFacialGate The final QT application running

Configuration and Compilation

Rockchip has added the code to the buildroot source code compilation, only need to open or close the corresponding configuration when buildroot compiles.

BR2_PACKAGE_QFACIALGATE=y
BR2_PACKAGE_RKFACIAL_USE_WEB_SERVER=y 
BR2_PACKAGE_ROCKFACE_FACE_DETECTION_V3_FAST=y   
BR2_PACKAGE_ROCKFACE_LIVING_DETECTION=y

In addition to the above configuration, there are some dependent configurations. For details, please refer to the configs/firefly_rv1126_rv1109_facial_gate_defconfig configuration file.

QT Development

QFacialGate is the entrance to the program. Because the factory firmware has a built-in Rkfacial trial version, you can directly modify, compile, and replace QFacialGate to complete the pre-development.

  • After completing a buildroot complete compilation, enter the compilation output directory for secondary development of QFacialGate.

#Here is the output path after compiling with the default configuration
cd sdk/buildroot/output/firefly_rv1126_rv1109_facial_gate/build/QFacialGate-1.0/ 
make

  • You can add some of your own controls after successful identification, such as external relays and LED.

# desktopview.cpp -> paintBox()
if(ret) {
        display_paint_box(left, top, right, bottom);
        switch(desktopView->videoItem->facial.state) {
                case USER_STATE_REAL_UNREGISTERED:
                        display_set_color(set_yuv_color(COLOR_B));
                        break;
                case USER_STATE_REAL_REGISTERED_WHITE:
                        display_set_color(set_yuv_color(COLOR_G));
                        break;
                case USER_STATE_REAL_REGISTERED_BLACK:
                        display_set_color(set_yuv_color(COLOR_BK));
                        break;
                default:
                        display_set_color(set_yuv_color(COLOR_R));
                        break;
        }   
# You can add corresponding operations in the recognition registration, unregistered, and blacklists, and recompile and overlay them on the board.
# It is worth noting that the camera preview is not done with QT, but with Rkfacial. 
# All frames and information drawing are realized by configuring the Rkfacial callback function.
# After the modification is completed, you can directly make the compilation. 
# If the final modification wants to be solidified into the buildroot, you need to add the modification to sdk/app/QFacialGate.

  • Deploy through adb after modification

adb /etc/init.d/S06_QFacialGate stop
adb push QFacialGate /
adb shell cp /QFacialGate /usr/bin/QFacialGate
adb /etc/init.d/S06_QFacialGate start

_images/rkfacial.jpg

  • Development Document

QFacialGate introduction:
app\QFacialGate\doc\Rockchip_Instruction_Linux_QFacialGate_EN.pdf
Rkfacial library introduction:
external\rkfacial\doc\Rockchip_Instruction_Rkfacial_EN.pdf
Web back-end development framework:
docs\RV1126_RV1109\ApplicationNote\Rockchip_Developer_Guide_Linux_Application_Framework_EN.
pdf
Web page introduction:
docs\RV1126_RV1109\ApplicationNote\Rockchip_Instructions_Linux_Web_Configuration_EN.pdf