Face recognition SDK

This SDK development guide will guide you how to install and configure the development environment, and how to perform secondary development and system integration by calling the interface (API) provided by the SDK. Users can call the API provided by the SDK according to requirements to achieve the purpose of using services such as face detection / tracking, living body recognition, face recognition, and other services.

1. Return values

public static final int SUCCESS = 0; //execution succeed
public static final int ERROR_INVALID_PARAM = -1; //Illegal parameter
public static final int ERROR_TOO_MANY_REQUESTS = -2; //Too many requests
public static final int ERROR_NOT_EXIST = -3; //does not exist
public static final int ERROR_FAILURE = -4; // Execution failed

2. FaceInfo

public class FaceInfo {
     public Rect mRect;//Face recognition frame
     public FaceAttribute mAttr;//Face attributes
     public FaceQuality mQuality;//Face recognition quality
     public Landmark mLandmark;//Used to store the coordinates of 5 key points, which are left eye, right eye, nose, left side of lips, right side of lips
}

3. FaceAttribute

public class FaceAttribute {
     public int mGender;//Gender 0-> Male, 1->Female
     public int mEmotion;//expression 0->calm 1->glad
     public int mAge;//Age
}

4. FaceQuality

public class FaceQuality {
     public float mScore;//Confidence in Face Quality
     public float mLeftRight;//Roll
     public float mUpDown;//Pitch
     public float mHorizontal;//Yaw
     public float mClarity;//Clarity
     public float mBright;//Bright
}

5. Constructors

static FaceAPP GetInstance()

Function:
Returns the FaceApp object

Parameters:
  void

Returns:
  FaceAPP

Example ：

private FaceAPP face = FaceAPP.GetInstance();

6. Recognize facial features

int Recognize( Image image, float featureArray [][512], int size, List faceinfos, int[] res )

Function :
Identify the facial features in the submitted Image, then compare with the data in the featureArray to find the data with the highest similarity.Returns the index of the array.

Parameters :
  image : Image object holding face image

  featureArray : An array of facial feature values. The length of the facial feature value array is 512.

  size : The length of featureArray

  faceinfos : FaceInfo checklist. FaceInfo is an object used to save the results of operations

  res: Face recognition results.The length of res is always 1.If no face is recognized in the picture,res[0] return ERROR_INVALID_PARAM.If the face in image is not in featureArray,res[0] return ERROR_NOT_EXIST.If res [0]> = 0, the index of the featureArray is returned.

Returns  Execution succeed return SUCCESS
      Execution failed return ERROR_FAILURE

Example ：

float[][] featurelist = new float[][]; //Array storing facial feature values
int size = featurelist.lenth;
int[] ret = new int[1];
byte[] tmpPos = new byte[1024];
FaceAPP.Image image = FaceAPP.GetInstance().new Image();
image.matAddrframe = mRgbaFrame.getNativeObjAddr();
face.Recognize( image, featurelist, size, tmpPos, res );

7. Recognize facial features(According to facial feature values)

int Recognize( float[] feature, float featureArray [][512], int size, float[] high, int[] res )

Function :
According to the input facial feature values, then compare with the data in the featureArray to find the data with the highest similarity.Returns the index of the array。

Parameters :
  feature : Facial feature values

  featureArray : An array of facial feature values. The length of the facial feature value array is 512.

  size : The length of featureArray

  high : Similarity score

  res : Face recognition results.The length of res is always 1.If no face is recognized in the picture,res[0] return ERROR_INVALID_PARAM.If the face in image is not in featureArray, res[0] return ERROR_NOT_EXIST. If res [0]> = 0, the index of the featureArray is returned.

Returns
      Execution succeed return SUCCESS
      Execution failed return ERROR_FAILURE

Example ：

float[][] featurelist = new float[][]; //Array storing facial feature values
int size = featurelist.lenth;
int[] ret = new int[1];
byte[] tmpPos = new byte[1024];
float[] feature;
float[] high = float[1];
FaceAPP.Image image = FaceAPP.GetInstance().new Image();
image.matAddrframe = mRgbaFrame.getNativeObjAddr();
face.Recognize( feature, featurelist, size, high, tmpPos, res );

8. Detect faces

int Detect( Image image, List faceinfos, int[] res )

Function :
Detect faces in submitted pictures

Parameters :
  image : Image object containing a face image

  faceinfos : FaceInfo checklist. FaceInfo is an object used to save the results of operations

  res : Face recognition results.The length of res is always 1.If no face is recognized in the picture,res[0] return ERROR_INVALID_PARAM.

Returns
      Execution succeed return SUCCESS
      Execution failed return ERROR_FAILURE

Example ：

int[] ret = new int[1];
byte[] tmpPos = new byte[1024];//Used to store location information
FaceAPP.Image image = FaceAPP.GetInstance().new Image(); //Init
image.matAddrframe = mRgbaFrame.getNativeObjAddr(); //Set the memory address of the picture to be identified
if( success = face.Detect( image, tmpPos, res ) ){
//to do
};

9. Compare feature data

int Compare( float[] origin, float[] chose, float score )

Function :
Used to compare the similarity of two feature values

Parameters :
  origin : Feature array to be compared

  chose : Feature array for comparison

  score : similarity between origin and chose

Returns
      Execution succeed return SUCCESS
      Execution failed return ERROR_FAILURE

Example ：

float score;
float[] origin = new float[512];
Float[] chose = new float[512];
face.Compare( origin, chose, score );

10. Face extraction with binocular and living body

int GetFeature( Image image, Image grayImage, float[] feature, List faceinfos, int[] res )

Function :
Extracting facial features from data obtained by binocular cameras,and only one face can be extracted.

Parameters:
  image : Image object containing a face image

  grayImage : Image object containing an infrared image

  feature : Stores the facial feature information of the detection results, and returns an empty array without a face

  faceinfos : FaceInfo checklist. FaceInfo is an object used to save the results of operations

  res : Face recognition results.The length of res is always 1.If no face is recognized in the picture,res[0] return ERROR_INVALID_PARAM.

Returns
      SUCCESS : Successfully obtained face information
      ERROR_FAILURE : No face information was obtained

Example ：

float[] feature = new float[512];
int[] ret = new int[1];
byte[] tmpPos = new byte[1024]; //Used to store location information
ret = face.GetFeature( image, grayImage, feature, tmpPos, res);
if( ret == SUCCESS ){
//to do 
}

11. Liveness Detection

int DetectLiveness(Image image, List faceinfos, int[] res)

Function:
Detect whether a face is alive.

Parameters:
  image : Color face pictures, pictures for detection and recognition.

  faceinfos : FaceInfo checklist. FaceInfo is an object used to save the results of operations

  res : Face recognition results.The length of res is always 1.If no face is recognized in the picture,res[0] return ERROR_INVALID_PARAM.

Returns
      SUCCESS : Living
      ERROR_FAILURE : Non-living

Example ：

int[] ret=new int[1];
ret= face.DetectLiveness(image,grayImage ,tmpPos,res);
if(ret== SUCCESS){
//to do 
}

12. Binocular for Liveness detection

int GetDetectLiveness(Image image, Image grayImage, List faceinfos, int[] res)

Function:
Detect whether a face is alive.

Parameters:
  image : Color face pictures, pictures for detection and recognition.

  grayImage :Image object containing an infrared image.

  faceinfos : FaceInfo checklist. FaceInfo is an object used to save the results of operations

      res : Face recognition results.The length of res is always 1.If no face is recognized in the        picture,res[0] return ERROR_INVALID_PARAM.

Returns
      SUCCESS : Living
      ERROR_FAILURE : Non-living

Example ：

int[] ret=new int[1];
ret= face.GetDetectLiveness(image,grayImage ,tmpPos,res);
if(ret== SUCCESS){
//to do 
}

13. Binocular Calibration

int Calibration( Image image, Image grayImage, float[] scale, int[] Rect, int[] res );

Function:
The calibration of the dual camera recognition consisting of infrared light and ordinary light requires one person to stand at the optimal position (0.8-1 meters), and it takes about 20 checks to obtain the face correction parameters. And return the coordinates of the infrared camera display area relative to ordinary light, this area is an effective recognition and living body detection area.

Parameters :
  image : Color face pictures, pictures for detection and recognition.

  grayImage : Image object containing an infrared image

  scale : Returns Face Frame Correction Parameters

  rect : Returns the overlapping area of the infrared image and the color image (the corresponding area of the infrared image in the color image / recommended detection area)

  res : Face recognition results.The length of res is always 1.If no face is recognized in the picture,res[0] return ERROR_INVALID_PARAM.

Returns
      SUCCESS : Calibration succeeded
      ERROR_FAILURE : Calibration failed

Example ：

int[] ret = new int[1];
float[] scale = new float[1];
int[] rect = new int[4];
ret = face.Calibration( image, grayimage, scale, rect, res );
if( ret == SUCCESS ){
//to do 
}

14. Device Activation 1

int AuthorizedDevice( String uidStr, String password, Context activity )

Function:
Device activation

Parameters:
  uidStr : OEMID + contract Id

  password : password

Returns
      0 : Authentication succeeded

Example ：

String oem_id = "1000000000000001";//OEMID
String contract_id = "0001";// contract Id
String password = "0123456789abcdef0123456789abcdef"; //password
String uidStr = oem_id + contract_id;
int res = face.AuthorizedDevice( uidStr, password, LoginActivity.this ); 

15. Device Activation 2

int fireflyInit(Context context, String uidStr, String password)

Function :
Device activation;This interface is a temporary interface for firefly, which can permanently activate the device. This interface may be removed later.If the interface description is not included in subsequent development documents, it has been removed. The development documents are subject to the documents in the open source Demo.

Parameters :
  uidStr : OEMID + contract Id

  password : password

Returns
      0 : Authentication succeeded

Example ：

String oem_id = "1000000000000001";//OEMID
String contract_id = "0001";//contract Id
String password = "0123456789abcdef0123456789abcdef"; //password
String uidStr = oem_id + contract_id;
int res =face.fireflyInit(LoginActivity.this, uidStr, password);

16. Device Activation 3

int AuthorizedDeviceUserPassword(String uidStr, String password, Context context, String userPassword)

Function :
Device activation，This interface is mainly used for the activation method with user password.

Parameters :
  uidStr : OEMID + contract Id

  password : password

  userPassword:user password

Returns
      0 : Authentication succeeded

Example ：

String oem_id ="1000000000000001";//OEMID
String contract_id ="0001";//contract Id
String password = "0123456789abcdef0123456789abcdef";//password
String userPassword ="012345678912";//user password;
String uidStr = oem_id+contract_id;
int res =face. AuthorizedDeviceUserPassword(uidStr,password,
LoginActivity.this,userPassword);

17. Get authentication activation status

int getAuthStatus()

Function:
Get authentication activation status

Parameters:
  void

Returns
      0 : activated

Example ：

int res = face.getAuthStatus(); 

18. Extracting facial features

int GetFeature( Image image, float[] feature, List faceinfos, int[] res )

Function:
Extract feature values of pictures in image,and get only the characteristics of one person in the picture。

Parameters:
  image : Image object containing a face image

  feature : Stores the facial feature information of the detection results, and returns an empty array without a face

  faceinfos : FaceInfo checklist. FaceInfo is an object used to save the results of operations

  res : Face recognition results.The length of res is always 1.If no face is recognized in the picture,res[0] return ERROR_INVALID_PARAM.

Returns
      SUCCESS : Feature extraction successful
      ERROR_FAILURE : No face information was obtained

Example ：

float[] feature = new float[512];
int[] ret = new int[1];
byte[] tmpPos = new byte[1024]; //Used to store location information
ret = face.GetFeature( image, feature, tmpPos, res );
if( ret == SUCCESS ){
//to do 
}

19. Extraction of face features (based on face coordinate information)

int GetFeature( Image image, FaceInfo detectInfo, float[] feature, int[] res )

Function:
Extract feature values of pictures in image,and get only the characteristics of one person in the picture。

Parameters:
  image : Image object containing a face image

  detectInfo : Face information detected for facial feature extraction

  feature : Stores the facial feature information of the detection results, and returns an empty array without a face

  res : Face recognition results.The length of res is always 1.If no face is recognized in the picture,res[0] return ERROR_INVALID_PARAM.

Returns
      SUCCESS : Feature extraction successful
      ERROR_FAILURE : No face information was obtained

Example ：

float[] feature = new float[512];
int[] ret = new int[1];
float[] detectinfo = new float[]{ x0, y0, x1, y1, landmarkx0, landmarky0,
				 landmarkx1, landmarky1, landmarkx2, landmarky2,
				 landmarkx3, landmarky3, landmarkx4, landmarky4 }
byte[] tmpPos = new byte[1024]; //Used to store location information
ret = face.GetFeature( image, detectinfo, feature, res );
if( ret == SUCCESS ){
//to do 
}

20. Extracting facial features (based on picture files)

float[] GetFeature(String path, List faceinfos)

Function:
Get the facial feature value data in the image according to the path of the incoming image file.

Parameters:
  path :The absolute path of the face image file.

  faceinfos : FaceInfo checklist. FaceInfo is an object used to save the results of operations

Returns
      Float[] : Face feature value array

Example ：

int ret = this.FaceGetFeatureFromAddr(addr, feature, mFaceInfos, policy);

if (ret == -1) {
     return null;
} else {
     faceInfos.addAll(Arrays.asList(this.mFaceInfos).subList(0,ret));
     return this.feature;
}

21. Extract key points of the face

int GetLandmark ( Image image, float[] landmark, int[] res )

Function:
Get face key point coordinate information

Parameters:
  Image : Face picture, a picture used to extract key point information of the face, only extract key point information of a person

  Landmark : Used to store the coordinates of 5 key points, which are left eye, right eye, nose, left side of lips, right side of lips

  res : Face recognition results.The length of res is always 1.If no face is recognized in the picture,res[0] return ERROR_INVALID_PARAM.

Returns
      SUCCESS : Successfully obtain coordinate information of face keypoints
      ERROR_FAILURE : No facial keypoint information was obtained

Example ：

float[] landmark = new float[10];
int[] ret = new int[1];
ret = face.GetLandmark( image, landmark, res);
if( ret == SUCCESS ){

}

22. Face quality

FaceInfo getQuality(long matAddrframe)

Function:
Get the face information of the largest face in the image。

Parameters:
  matAddrframe : Image address stored in Mat object

Returns
      FaceInfo : Used to store face information in pictures

Example ：

Faceinfo faceinfo=new Faceinfo();
faceinfo=face.getQuality(matAddrframe);
if(faceinfo!=null){
}

23. Parameters Setting

bool SetParameter( const String[] name, float value[] )

Function:
Set the entered Parameter key and value

Parameters:
  char[] name : Parameters key array

  a : Internal Parameters, set according to the example, please do not modify.

  b : Internal Parameters, set according to the example, please do not modify.

  c : Internal Parameters, set according to the example, please do not modify.

  d : Internal Parameters, set according to the example, please do not modify.

  factor : Detect face magnification. Internal Parameters, set according to the example, please do not modify.

  min_size : Minimum face frame size, recommended range:32-80

  faceclarity : Photo sharpness threshold, recommended range:200-400

  perfoptimize : Whether to optimize the effect , recommended range: 0 or 1

  livenessdetect : Whether live detection , recommended range:0 or 1

  gray2colorscale : Binocular live detection ratio , recommended range: 0.1-0.5

  frame_num : Number of frames,recommended range:20-40

  quality_thresh : Picture quality threshold,recommended range:0.7-0.8

  mode : Operating mode :0-> Entrance 1-> Access control

  facenum : Detects the maximum number of faces and supports detection of up to 3 faces and 1 face.recommended range:1-3

  value[] : Parameters key array

Returns:
  Whether the parameters are set successfully

Example ：

String[] name = { "a", "b","c", "d", "factor", "min_size", "clarity", "perfoptimize", 
		  "livenessdetect", "gray2colorscale", "frame_num", "qualit_thresh",
		  "mode", "facenum" };
double[] value = {0.9, 0.9, 0.9, 0.715, 0.6, 64, 400, 1, 0, 0.5, 20, 0.8, 1, 1 };
face.SetParameter( name, value );

24. Parameters Setting(Mode for modifying parameter length as needed)

bool SetParameters( String[] name, float value[] )

Function:
Set the entered Parameter key and value

Parameters:
  char[] name : Parameters key array(length>=1),See details in 23.

  value[] : Parameters value array(length>=1)

Returns:
  Whether the parameters are set successfully

Example ：

String[] name = { "perfoptimize", "livenessdetect", "frame_num", "quality_thresh", 
		  "mode", "facenum" };
double[] value = { 1, 0, 20, 0.8, 1, 1 };
face.SetParameter( name, value );

25. Get version information

public String GetVersion()

Function:
Get information about the current SDK version

Parameters   void

Returns:
  Return current version information

Example ：

Face.GetVersion();

26. Get Facelib Version

public String GetFacelibVersion()

Function:
Get Facelib Version

Parameters:
  void

Returns:
  Return Facelib version information

Example ：

Face.GetFacelibVersion();

27. Open the face database

int OpenDB()

Function:
Using face database, internal face database can achieve 1-to-N efficient and fast query

Parameters:
  void

Returns
      Execution succeed return SUCCESS
      Execution failed return ERROR_FAILURE

Example ：

if(face.OpenDB() == SUCCESS){
// TODO
}

28. Registered face

int AddDB( float[] feature, string name )

Function:
Registered face to database

Parameters:
  feature : Facial features

  name : Registered name

Returns
      Execution succeed return SUCCESS
      Execution failed return ERROR_FAILURE

Example ：

String name= "test";
int[] res=new int[];
if( Face.GetFeature( image, feature, tmpPos, res ) == SUCCESS ){
	Face.AddDB ( feature, name );
}

29. Save face

int SaveDB()

Function:
Save the results of adding, modifying, and deleting faces to a database file；If it is not called，the modification is invalid after exiting the program.

Parameters:
  void

Returns
      Execution succeed return SUCCESS
      Execution failed return ERROR_FAILURE

Example ：

Face.AddDB(feature,name);
...
//After adding faces, make sure to execute Face.SaveDB () before restarting the device;
Face.SaveDB();

30. Delete face

int DelDB(string name)

Function:
  Delete face

Parameters:
  name : Registered name

Returns
      Execution succeed return SUCCESS
      Execution failed return ERROR_FAILURE

Example ：

String name= "test";
Face.DelDB (name);

31. Delete all registered faces

int DelAllDB()

Function:
Delete all registered faces

Parameters:
  void

Returns
      Execution succeed return SUCCESS
      Execution failed return ERROR_FAILURE

Example ：

Face.DelAllDB ();

32. Querying Face

String QueryDB( float[] feature, float [] score )

Function:
Returns the data that is closest to the input facial features.

Parameters:
  feature : Facial features

  score : Store the similarity score of the queried face data and the input face data

Returns
      Execution succeed return registered name
      Execution failed return unknown

Example ：

float[] score = new float[1];
String name = Face.QueryDB( feature, score );
if( score > thresh_hold ){
// TODO
}

33. Close the face database

int CloseDB()

Function:
Close the face database

Parameters:
  void

Returns
      Execution succeed return SUCCESS
      Execution failed return ERROR_FAILURE

Example ：

if( Face.CloseDB() == SUCCESS ){
// TODO
}

34. Load Quick Compare Function

int FastQueryInit()

Function:
Load Quick Compare Function

Parameters:
  void

Returns
      Execution succeed return SUCCESS
      Execution failed return ERROR_FAILURE

Example ：

if(Face.FastQueryInit () == SUCCESS){
// TODO
}

35. Refresh fast match buffer

int FastQueryFlush(float [] data, int num)

Function:
Refresh fast match buffer

Parameters:
  Data : External feature buffer

  Num : Number of features

Returns
      Execution succeed return SUCCESS
      Execution failed return ERROR_FAILURE

Example ：

if(Face.FastQueryFlush(data,num) == SUCCESS){
// TODO
}

36. Quick Query

int FastQuery(float[] data, float[] feature, float[] scores, int num)

Function:
  Quick query

Parameters:
  Data : External feature buffer

  feature : The feature to query

  scores:Similarity score

  num : Number of features

Returns
      Execution succeed return SUCCESS
      Execution failed return ERROR_FAILURE

Example ：

if(Face.FastQuery(data,feature,num) == SUCCESS){
// TODO
}

37. Get face attributes

int GetFaceAttr( Image image, FaceInfo data, FaceAttribute face_attr, int *res )

Function:
After Detect is executed, the face attributes including age, gender and expression are obtained by detecting the acquired face position and keypoint information.

Parameters:
  image : Image object containing a face image

  data : Detected face position and face keypoint information

  face_attr : Store calculated face attributes

  res : Face recognition results.The length of res is always 1.If no face is recognized in the picture,res[0] return ERROR_INVALID_PARAM.

Returns
      SUCCESS : Obtain face attributes succeed
      ERROR_FAILURE : Obtain face attributes failed

Example ：

if( face.GetFaceAttr( Image, data, attr, res ) == SUCCESS ){
// TODO
}

38. Release face recognition resources

public void Destroy()

Function:
Free resources allocated when initializing and setting Parameters

Parameters:
  void

Returns  void

Example ：

Face.Destroy(); 

39. Sample code

Initialize the binocular camera face recognition demo.

public class MainActivity extends Activity implements CvCameraViewListener2 {

    private FaceAPP face= FaceAPP.GetInstance(); //face as member variable
    .........
    @Override
    protected void onCreate(Bundle savedInstanceState) {
      .........
        String[] name={"a","b","c","d","factor","min_size","clarity","perf-optimize","liveness-detect","gray2color-scale"};
	double[] value={0.9,0.9,0.9,0.715,0.6,64,400,1,0,0.5};
	face.SetParameter(name,value);
	mainLoop = new Thread() { //Face detection needs to run on a child thread

	public void run() {
	    .........
	    float[] feature=new float[512];
	    byte[] tmpPos = new byte[1024];// used to store position information
            switch (mixController.curState){
                 ......
                 case mixController. STATE_IDLE :
                      FaceAPP.Image image= FaceAPP. GetInstance ().new Image();
                      image.matAddrframe=mRgbaFrame.getNativeObjAddr();
                      int[] res=new int[1];
                      int ret;
                      ret= face.GetFeature(image,feature,tmpPos,res);
                      if(ret== SUCCESS){
                            //to do Successfully obtained facial feature values
                       }
            }
        }
    }
}