Face recognition API V2.0

DEMO|github

Hardware Interface API

1. IDCard/ICCard

• Connected devices

Start the monitoring service and monitor the card swiping operation, it is recommended to execute in the on Resume() method.

/*
Open the background monitoring service
*/
IDCardUtil.getInstance().bindIDCardService(Context context);

/*
The NFC module is reported in the form of key value
*/
@Override
public boolean dispatchKeyEvent(KeyEvent event) {
if (IDCardUtil.getInstance().handleEvent(event)) {return true;}

return super.dispatchKeyEvent(event);
}

• Check function support

Since the monitoring service is started asynchronously, it is not necessarily accurate to perform function support detection immediately after connecting the device. It is recommended to check the function support again in the monitoring callback of 6 on Machine Connect.

/*
Check if the device supports ID card recognition
boolean: true means support, false means not support
*/
boolean result =IDCardUtil.getInstance().isSupportIDCard();

/*
Check if the device supports ICCard recognition
boolean: true means support, false means not support
*/
boolean result = IDCardUtil.getInstance().isSupportICCard();

• Set swipe mode

Although the ID card module supports both ID card and IC Card, it does not support simultaneous reading of both, and the card reading mode needs to be set as required NFC module only supports ICCard.

/*
According to the specified reading method when swiping the card;
There are two main types：READCARD_MODE_IDENTITY_CARD and READCARD_MODE_IC_CARD；

IDCardConfig.READCARD_MODE_IDENTITY_CARD = 0;  //IDCard
IDCardConfig.READCARD_MODE_IC_CARD = 1; //ICCard
IDCardConfig.READCARD_MODE_IDENTITY_CARD_UUID = 2; //IDCard UUID 
*/
IDCardUtil.getInstance().setModel(int readMode);

• Set the endian mode for reading IC Card

Set the endian mode when reading the IC Card, the default is big endian.

/*
boolean:true Bigendian, false Little endian
*/
IDCardUtil.getInstance().setICCardEndianMode(boolean useBig);

• Set up listening and callback functions

/*
Binding credit card monitoring callback
context 
callback Listen callback
*/
IDCardUtil.getInstance().setIDCardCallBack(IDCardUtil.IDCardCallBack callBack);

//Listen callback
IDCardUtil.IDCardCallBack callBack = new IDCardUtil.IDCardCallBack() {

//The monitoring service is started asynchronously, if a card reading device is found after startup, it will call back on Machine Connect
@Override
public void onMachineConnect() {
    Log.i("firefly", "onMachineConnect ");
}

//Callback when ID card is swiped
@Override
public void onSwipeIDCard(final IDCardBean info) {
    Log.i("firefly",
        "picture:"+               + "\n" +
        "name：" + info.getName() + "\n" +
        "sex：" + info.getSex() + "\n" +
        "nation：" + info.getNation() + "\n" +
        "birthDate：" + info.getBirthDateStr() + "\n" +
        "address：" + info.getAddress() + "\n" +
        "number：" + info.getNum() + "\n" +
        "issue：" + info.getIssue() + "\n" +
        "expiration date：" + info.getCreateTimeStr() + "-" + info.getValidTimeStr() + "\n" +
        "picture:"+               + info.getPhoto() + "\n" );
}

// Execute callback when swiping IC card
@Override
public void onSwipeICCard(final ICCardBean info) {
    Log.i("firefly", "onSwipeICCard IC=" + info.getIcID());
}

// When setting the card reading mode to READCARD_MODE_IDENTITY_CARD_UUID Time, swipe ID card callback
@Override
public void onSwipeIDCardUUID(final String uuid) {
    Log.i("firefly", "onSwipeIDCardUUID uuid=" + uuid);
}
};

• Disconnect

Remove the monitoring callback and stop the monitoring service, it is recommended to execute in the on Stop () method.

IDCardUtil.getInstance().setIDCardCallBack(null);
IDCardUtil.getInstance().unBindIDCardService(Context context);

2. Fill light control

There are 4 types of fill light: infrared fill light, white fill light, red fill light and green fill light.

• Check whether the device supports infrared fill light;

/*
Boolean: true means support, false means not support
*/
HardwareCtrl.isSupportInfraredFillLight();

• Operation of infrared fill light;

/*
isChecked true means open, false close
*/
HardwareCtrl.setInfraredFillLight(isChecked);

• Check whether the device supports the brightness adjustment of the white fill light, and obtain the range of brightness adjustment;

/*
Boolean:true Indicates that infrared fill light is supported, false indicates that it is not supported
*/
HardwareCtrl.isFillLightBrightnessSupport();

/*
Support the maximum value of brightness adjustment;
*/
HardwareCtrl.getFillLightBrightnessMax();

/*
Support the minimum value of brightness adjustment;
*/
HardwareCtrl.getFillLightBrightnessMin();

• White fill light operation;

/*
On/Off;
isChecked true means open and set the maximum value; false close
*/
HardwareCtrl.ctrlLedWhite(isChecked);

/*
On/Off, if brightness adjustment is supported, the corresponding brightness value can be passed in when the light is turned on;
isChecked  true means open and set the maximum value; false close
*/
HardwareCtrl.ctrlLedWhite(isChecked,brightness);

• Red fill light operation;

/*
isChecked true means open and set the maximum value; false close
*/
HardwareCtrl.ctrlLedRed(isChecked);

• Green fill light operation;

/*
isChecked true means open and set the maximum value; false close
*/
HardwareCtrl.ctrlLedGreen(isChecked);

3. Signal control

• “Rs485/232”

485 serial port: /dev/ttyS4; 232 serial port: /dev/ttyS3; you can view the set value changes through cat /dev/ttyS4 or cat /dev/ttyS3;

/*

Get serial number serialPort 
485 serial port: /dev/ttyS4; 232 serial port: /dev/ttyS3
*/
SerialPort serialPort = HardwareCtrl.openSerialPortSignal(new File("/dev/ttyS3"), 19200, new SerialPort.Callback() {

//rs485/232After sending the signal, the return value received
@Override
public void onDataReceived(byte[] bytes, int i) {
    String result = StringUtils.bytesToHexString(bytes, size);
    Log.i("firefly", "result = "+result);
}
});

/*
Send ‘48562311’ signal
*/
HardwareCtrl.sendSerialPortHexMsg(serialPort, "48562311")

/*
Close the serial port (it is recommended to close the serial port when the page exits)
*/
HardwareCtrl.closeSerialPortSignal(serialPort);

• Wiegand signal operation;

/*
Wiegand Input
*/
HardwareCtrl.openRecvMiegandSignal("/dev/wiegand");

/*
Add Wiegand Input listening callback
*/
HardwareCtrl.recvWiegandSignal(new RecvWiegandCallBack() {
@Override
public void recvWiegandMsg(int i) {
    Log.i("firefly", "result = "+i);
}
});

/*
When the page exits, close Wiegand
*/
HardwareCtrl.closeRecvMiegandSignal();

/*
Wiegand 26 Output
Check the set value changes through cat /sys/devices/platform/wiegand-gpio/wiegand26.
*/
HardwareCtrl.sendWiegandSignal("123456789");

/*
Wiegand 34 Output
Check the set value changes through cat /sys/devices/platform/wiegand-gpio/wiegand34.
*/
HardwareCtrl.sendWiegand34Signal("123456789");

• Level signal/relay signal;

/*
D0 level signal isChecked true means open, false close
*/
LevelSignalUtil.sendSignalD0(isChecked);

/*
D1 level signal isChecked true means open, false close
*/
LevelSignalUtil.sendSignalD1(isChecked);

/*
Relay signal isChecked true means open, false closed
*/
RelayUtil.sendRelaySignal(isChecked);

4. QRCode

• QRCode Operation

/*
Check whether the device supports the QR code function;
Boolean: true means support, false means not support
*/
QrCodeUtil.getInstance().isQrCodeSupport();

/*
Turn on QR code scanning
*/
QrCodeUtil.getInstance().init();

/*
Whether to turn on the QR code scanning fill light
QrCodeUtil.LED_STATE_AUTO // Automatic
QrCodeUtil.LED_STATE_ON // ON
QrCodeUtil.LED_STATE_OFF // OFF
*/
QrCodeUtil.getInstance().setLedState(state);

/*
Add QR code monitoring callback
*/
QrCodeUtil.getInstance().setQRCodeCallback(new QrCodeUtil.QRCodeCallback() {
// Callback method when QR code is connected
@Override
public void onConnect() {
Log.i("firefly","QRCode onConnect:");
}

// Content callback method for QR code recognition
@Override
public void onData(final String s) {
Log.i("firefly","QRCode onData:"+s);
}
});

/*
When the page exits, turn off QR code scanning
*/
QrCodeUtil.getInstance().release();

5. body temperature

• Body temperature operation;

/*
Check whether the device supports body temperature;
Boolean: true means support, false means not support
*/
TempatureUtil.getInstance().isSupport();

/*
Turn on body temperature detection
*/
TempatureUtil.getInstance().openDevice();

/*
Add temperature detection monitoring callback
*/
TempatureUtil.getInstance().setTempatureCallback(new TempatureUtil.TempatureCallback() {
// Callback method when body temperature detection is connected
@Override
public void onConnect() {
Log.i("firefly","TempatureCallback onConnect:");
}

// Content callback method for body temperature detection
@Override
public void update(float ambientTempature, float objectTempature) {
Log.i("firefly","TempatureCallback update:ambientTempature="+ambientTempature + "objectTempature="+objectTempature);
}
});

/*
When the page exits, turn off the body temperature function
*/
TempatureUtil.getInstance().closeDevice();

6. Radar

• Radar operation;

/*
Handling radar events, KeyEvent event
In the Android Activity page, public boolean dispatchKeyEvent(KeyEvent event), handle KeyEvent event
*/
RadarUtil.handleEvent(event);

/*
Dispatch Key Event on the Android Activity page
*/
@Override
public boolean dispatchKeyEvent(KeyEvent event) {
int ret = RadarUtil.handleEvent(event);
if (ret == RadarUtil.EVENT_HANDLE_RADAR_IN) {// Object entry
    Log.i("firefly","EVENT_HANDLE_RADAR_IN");
    return true;
} else if (ret == RadarUtil.EVENT_HANDLE_RADAR_OUT) { // Object leaving
    Log.i("firefly","EVENT_HANDLE_RADAR_OUT");
    return true;
} else if (ret == RadarUtil.EVENT_HANDLE_NOTHING_HANDLED) { // No object
    Log.i("firefly","EVENT_HANDLE_NOTHING_HANDLED");
    return true;
}

return super.dispatchKeyEvent(event);
}



// Monitor radar signals
// The radar signal is triggered in the form of KeyEvent and processed using handleEvent (KeyEvent event)
public static final int RadarUtil.EVENT_HANDLE_NOTHING_UNHANDLE = -1; // Non-radar events are not processed
public static final int RadarUtil.EVENT_HANDLE_NOTHING_HANDLED = 0; // Non-radar events processed

// Radar key-value pair
public static final int RadarUtil.KEYCODE_RADAR_IN = 305; // Something enters
public static final int RadarUtil.KEYCODE_RADAR_OUT = 306; // There is an object leaving

// Radar event
public static final int RadarUtil.EVENT_HANDLE_RADAR_IN = 1; // Something enters
public static final int RadarUtil.EVENT_HANDLE_RADAR_OUT = 2; // There is an object leaving

Android Quick start

1. Add dependent libraries

1.Copy the aar package in the faceEngineYtlf/libs/ directory of the Demo project to the libs folder corresponding to the AS project；
2.Copy the ytlf_v2 folder in the faceEngineYtlf/src/main/assets directory of the Demo project to the src/main/assets folder corresponding to the AS project;

2. Add compile information for aar

implementation(name: 'arctern-release', ext: 'aar')
implementation(name: 'iface-release', ext: 'aar')
implementation(name: 'faceEngineYtlfExternal', ext: 'aar')

Add the following code to build.gradle's dependencies

For details, please refer to the technical case Demo project configuration or doc/FireflyApi_instructions.png ；

3. Setting permissions

Add the following permissions in the AndroidManifest.xml file

<uses-permission android:name="android.permission.INTERNET"/>
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>
<uses-feature android:name="android.hardware.camera" android:required="true"/>
<uses-permission android:name="android.permission.CAMERA"/>

4. Defining FaceAPP variables in your program

When using YTLF Face Manager, you need to indicate the root path of the local SD card file storage directory, for example: “/sdcard/firefly/”； When the SDK is started for the first time, it will check whether there are files such as models and license public keys in the local SD card. If not, then by default, the necessary files will be copied from the App assets directory to the rootPath directory；

// Specify the local SD card file storage directory
YTLFFaceManager.getInstance().initPath(String rootPath);

5.Calling Face API interface through YTLFFaceManager

Face recognition

V2.0 adopts the asynchronous processing method of the thread pool to improve the utilization of the system and reduce the response time of services such as face detection and face tracking.

This SDK development guide will guide you how to install and configure the development environment, and how to perform secondary development and system integration by calling the interface (API) provided by the SDK. Users can call the API provided by the SDK according to requirements to achieve the purpose of using services such as face detection / tracking, living body recognition, face recognition, and other services.

1. SDK Access

1.1 SDK Initialize

When using YTLF Face Manager, you need to indicate the root path of the local SD card file storage directory, for example: “/sdcard/firefly/”； When the SDK is started for the first time, it will check whether there are files such as models and license public keys in the local SD card. If not, then by default, the necessary files will be copied from the App assets directory to the rootPath directory；

// Specify the local SD card file storage directory
YTLFFaceManager.getInstance().initPath(String rootPath);

1.2 Activating License

During initialization, it will check whether there is a local key, if not, it will be activated online, and the activation status will be called back after activation. It is divided into synchronous and asynchronous activation methods. ；

// Asynchronous mode
YTLFFaceManager.getInstance().initLicenseByAsync(String apiKey, new InitLicenseListener(){
    @Override
    public void onLicenseSuccess() {
                toast("success");
    }

    @Override
    public void onLicenseFail(final int code,final String msg) {
                toast("failure");
    }
});


// Synchronization mode true means successful activation
boolean result = YTLFFaceManager.getInstance().initLicense(String apiKey);

1.3 Get unique device code

Get the signature of the current device, the unique encoding of the device；

/*
String :Returns the current device's signature, which is the device's current unique code
*/
String signature = YTLFFaceManager.getInstance().getSignature();

2. SDK StartUp

When starting the SDK, it will detect the operating environment and initialize the SDK. There are synchronous and asynchronous startup methods.；

2.1 Start the SDK synchronously

/*
Determine the config_path through FACE_PATH, which is the directory where the Config.json file is stored
eg:  sdcard/firefly/ytlf_v2/config.json"

Int: 0 Means success, 1 means failure
*/
int result = YTLFFaceManager.getInstance().startFaceSDK();

/*
config_json Specify the contents of config.json

Int:0 Means success, 1 means failure
*/
int result = YTLFFaceManager.getInstance().startFaceSDKByConfigJson(String config_json);

2.2 Start SDK asynchronously

To prevent too much time when initializing and starting the SDK, causing the main thread to be blocked, you can use the asynchronous method to start the SDK；

/*
config_json Specify the contents of config.json
runSDKCallback Callback method after asynchronous start
*/
YTLFFaceManager.getInstance().startFaceSDKByAsynchronous(String config_json, new RunSDKCallback(){
@Override
public void onRunSDKListener(int i) {   //Int:0 Means success, 1 means failure

}
});

3. Face real-time data submission

Perform face detection and tracking based on the data passed into the detector, and return the results of face detection and face tracking in real time.Note: The face direction of the data input to the face detector should be positive, that is, the face angle Should be 0 degrees and no other angle； Currently, RGB and IR video streams are used for face detection tracking and related detection of living bodies.When it is not necessary to turn on the IR camera or do not need living body detection, IR data parameters can pass RGB parameters, but not NULL；

//Get video stream from camera callback function, input RGB video stream and IR video stream in real time
YTLFFaceManager.getInstance().doDelivery(ArcternImage img_rgb,  ArcternImage img_ir)

//Set the RGB and IR monitor callback function
YTLFFaceManager.getInstance().setOnDetectCallBack(new DetectCallBack() {
//  RGB:
@Override
public void onDetectListener(ArcternImage arcternImage, ArcternRect[] arcternRects, float[] confidences) {

}

//  IR:
@Override
public void onLivingDetectListener(ArcternImage arcternImage, ArcternRect[] arcternRects, float[] confidences) {

}
});

/*Parameter Description：
arcternImage RGB/IR Face image data detected
arcternRects  RGB/IR Collection of detected face frames
confidences RGB/IR Detect the confidence of each face
*/

4. Face real-time tracking callback

If there is data returned from the callback interface, then the real-time data is being detected and tracked;

//Set real-time face detection tracking callback
YTLFFaceManager.getInstance().setOnTrackCallBack(new TrackCallBack() {

/*Face real-time detection and tracking
arcternImage Face image data detected
trackIds  Face tracking in images ID
arcternRects Detected face position
*/
@Override
public void onTrackListener(ArcternImage arcternImage, long[] trackIds, ArcternRect[] arcternRects) {

}
});

5. Real-time face attribute detection

When performing real-time face detection tracking, add a face attribute detection callback method, which can receive face attributes in real time, including live detection value, quality, face angle, mask, and image color.

//Set real-time face attribute detection callback
YTLFFaceManager.getInstance().setOnAttributeCallBack(new AttributeCallBack() {

/*The face belongs to the monitoring callback:
arcternImage Face image data detected
arcternRects Detected face position
trackIds  Face tracking in images ID
arcternAttribute All attributes of all faces in the image
*/
@Override
public void onAttributeListener(ArcternImage arcternImage, long[] trackIds, ArcternRect[] arcternRects, ArcternAttribute[][] arcternAttributes, int[] landmarks) {
StringBuilder s = new StringBuilder();
for (int i = 0; i < arcternRects.length; i++) {
for (int j = 0; j < arcternAttributes[i].length; j++) {
    ArcternAttribute attr = arcternAttributes[i][j];
    switch (j) {
        case ArcternAttribute.ArcternFaceAttrTypeEnum.POSE_PITCH:
            s.append("Face angle:\n With the x axis as the center, the angle of the face up and down:").append(attr.confidence);
            break;
        case ArcternAttribute.ArcternFaceAttrTypeEnum.POSE_YAW:
            s.append("\n Rotate the face left and right around the y-axis:").append(attr.confidence);
            break;
        case ArcternAttribute.ArcternFaceAttrTypeEnum.POSE_ROLL:
            s.append("\n Taking the center point as the center, the x-y plane rotation angle:").append(attr.confidence);
            break;
            
        case ArcternAttribute.ArcternFaceAttrTypeEnum.QUALITY:
            s.append("\nFace quality:").append(attr.confidence);
            break;
            
        case ArcternAttribute.ArcternFaceAttrTypeEnum.LIVENESS_IR:
            if (attr.label != -1) {
                if (attr.confidence >= 0.5) {
                    s.append("\nBiopsy: live body ").append(attr.confidence);
                } else {
                    s.append("\nBiopsy: non-living ").append(attr.confidence);
                }
            }
            break;
            
        case ArcternAttribute.ArcternFaceAttrTypeEnum.IMAGE_COLOR:
            if (attr.label == ArcternAttribute.LabelFaceImgColor.COLOR) {
                s.append("\nColor picture ").append(attr.confidence);
            } else {
                s.append("\nBlack and white ").append(attr.confidence);
            }
            break;
            
        case ArcternAttribute.ArcternFaceAttrTypeEnum.FACE_MASK:
            if (attr.label == ArcternAttribute.LabelFaceMask.MASK) {
                s.append("\nMask ").append(attr.confidence);
            } else {
                s.append("\nWithout mask ").append(attr.confidence);
            }
            break;
    }
}
}
}
});

6. Face real-time search

Perform a real-time search based on the data passed into the detector, and search the database for an ID with a similarity greater than the highest set similarity. The ID can be used to obtain the recognized face and its related information;

//Set real-time face search callback
YTLFFaceManager.getInstance().setOnSearchCallBack(new SearchCallBack() {


/*Search callback:
arcternImage Face image data detected
trackIds  Face tracking in images ID
arcternRects Detected face position
searchId_list A collection of ids that identify faces in images
*/
@Override
public void onSearchListener(ArcternImage arcternImage, long[] trackId_list, ArcternRect[] arcternRects, long[] searchId_list, int[] landmarks,float[] socre) {
if (searchId_list.length > 0 && searchId_list[0] != -1) {
Person person = DBHelper.get(searchId_list[0]);
if (person != null) {
//The recognized face and related information can be obtained by ID;
}
} else {
// Face does not exist;
}
}

7. Get the landmarks coordinates of the eyes, mouth, nose, etc. of the face

Extract facial feature values ​​based on the Arctern Image of the face, which can be used to obtain the landmarks coordinates of the face, such as eyes, mouth, nose, etc.;

/*
Specify Arctern Image to extract facial feature values, including landmarks
*/
ArcternAttrResult ArcternAttrResult = YTLFFaceManager.getInstance().doFeature(ArcternImage arcternImage))

8. Extract face feature values ​​from specified image files

Extract face feature values ​​based on detected faces, which can be used for face comparison;

/*Specify image files to extract face feature values:
imagePath The map's address
etractCallBack  Face feature extraction callback
*/
YTLFFaceManager.getInstance().doFeature(imagePath, new ExtractCallBack() {

/*
acternImage Extract feature image
bytes  Set of face feature values ​​for multiple faces
arcternRects  Face detection result set
*/
@Override
public void onExtractFeatureListener(ArcternImage arcternImage, byte[][] bytes, ArcternRect[] arcternRects) {

if (features.length > 0) {
debugLog("bitmapFeature.length： " + features[0].length);
} else {
Tools.debugLog("Characteristic value is empty !");
}
});

9. Image Bitmap for feature value extraction

According to the face image Bitmap to extract the face feature value, it can be used for face comparison;

/*Specify face image Bitmap to extract face feature values:
bitmap Picture Bitmap data
etractCallBack  Face feature extraction callback
*/
YTLFFaceManager.getInstance().doFeature(Bitmap bitmap, new ExtractCallBack() {

/*
acternImage Extract feature image
bytes  Set of face feature values ​​for multiple faces
arcternRects  Face detection result set
*/
@Override
public void onExtractFeatureListener(ArcternImage arcternImage, byte[][] bytes, ArcternRect[] arcternRects) {

if (features.length > 0) {
debugLog("bitmapFeature.length： " + features[0].length);
} else {
Tools.debugLog("Characteristic value is empty !");
}
});

10. Face feature comparison

Compare the two feature values ​​to get the similarity between the two face feature values;

/*
feature1 First face feature value
Feature2  Second face feature value

return float :Return the similarity of face comparison
*/
float result = YTLFFaceManager.getInstance().doFeature(byte[] feature1,  byte[] feature2)

11. Face database management

11.1 Face storage

Add the extracted face feature values ​​and related IDs to the SDK face database;

/*
id Face ID
feature  Eigenvalues ​​of faces

0 Means success, 1 means failure
*/
int result = YTLFFaceManager.getInstance().dataBaseAdd(long id, byte[] feature)

11.2 Face library delete

According to the specified ID, delete the face information of the SDK face database;

/*
id Face ID

0 Means success, 1 means failure
*/
int result = YTLFFaceManager.getInstance().dataBaseDelete(long id)

11.3 Face library update

According to the specified ID, update the face information of the SDK face database;

/*
id Face ID
feature  Eigenvalues ​​of faces

0 Means success, 1 means failure
*/
int result = YTLFFaceManager.getInstance().dataBaseUpdate(long id, byte[] feature)

11.4 Add feature values ​​to faces in batches

According to the specified multiple IDs, batch update multiple face information corresponding to the SDK face database;

/*
id Face ID Array
feature  Array of multiple face feature values

0 Means success, 1 means failure
*/
int result = YTLFFaceManager.getInstance().dataBaseAdd(long[] id, byte[][] feature)

11.5 Face data removal

Delete all face information of SDK face database;

/*
0 Means success, 1 means failure
*/
int result = YTLFFaceManager.getInstance().dataBaseClear()