Face recognition SDK

This SDK development guide will guide you how to install and configure
the development environment, and how to perform secondary development
and system integration by calling the interface (API) provided by the
SDK. Users can call the API provided by the SDK according to
requirements to achieve the purpose of using services such as face
detection / tracking, living body recognition, face recognition, and
other services.

1. Return values

   public static final int SUCCESS = 0; //execution succeed public
   static final int ERROR_INVALID_PARAM = -1; //Illegal parameter
   public static final int ERROR_TOO_MANY_REQUESTS = -2; //Too many
   requests public static final int ERROR_NOT_EXIST = -3; //does not
   exist public static final int ERROR_FAILURE = -4; // Execution
   failed

2. FaceInfo

   public class FaceInfo {
        public Rect mRect;//Face recognition frame
        public FaceAttribute mAttr;//Face attributes
        public FaceQuality mQuality;//Face recognition quality
        public Landmark mLandmark;//Used to store the coordinates of 5 key points, which are left eye, right eye, nose, left side of lips, right side of lips
   }

3. FaceAttribute

   public class FaceAttribute {
        public int mGender;//Gender 0-> Male, 1->Female
        public int mEmotion;//expression 0->calm 1->glad
        public int mAge;//Age
   }

4. FaceQuality

   public class FaceQuality {
        public float mScore;//Confidence in Face Quality
        public float mLeftRight;//Roll
        public float mUpDown;//Pitch
        public float mHorizontal;//Yaw
        public float mClarity;//Clarity
        public float mBright;//Bright
   }

5. Constructors

   static FaceAPP GetInstance()

   Function: Returns the FaceApp object

   Parameters:   void

   Returns:   FaceAPP

Example ：

   private FaceAPP face = FaceAPP.GetInstance();

6. Recognize facial features

   int Recognize( Image image, float featureArray [][512], int size,
   List faceinfos, int[] res )

   Function :  Identify the facial features in the submitted Image,
   then compare with the data in the featureArray to find the data
   with the highest similarity.Returns the index of the array.

   Parameters :   image : Image object holding face image

     featureArray : An array of facial feature values. The length of
   the facial feature value array is 512.

     size : The length of featureArray

     faceinfos : FaceInfo checklist. FaceInfo is an object used to
   save the results of operations

     res: Face recognition results.The length of res is always 1.If no
   face is recognized in the picture,res[0] return
   ERROR_INVALID_PARAM.If the face in image is not in
   featureArray,res[0] return ERROR_NOT_EXIST.If res [0]> = 0, the
   index of the featureArray is returned.

   Returns  Execution succeed return SUCCESS       Execution failed
   return ERROR_FAILURE

Example ：

   float[][] featurelist = new float[][]; //Array storing facial feature values
   int size = featurelist.lenth;
   int[] ret = new int[1];
   byte[] tmpPos = new byte[1024];
   FaceAPP.Image image = FaceAPP.GetInstance().new Image();
   image.matAddrframe = mRgbaFrame.getNativeObjAddr();
   face.Recognize( image, featurelist, size, tmpPos, res );

7. Recognize facial features(According to facial feature values)

   int Recognize( float[] feature, float featureArray [][512], int
   size, float[] high, int[] res )

   Function : According to the input facial feature values, then
   compare with the data in the featureArray to find the data with the
   highest similarity.Returns the index of the array。

   Parameters :   feature : Facial feature values

     featureArray :  An array of facial feature values. The length of
   the facial feature value array is 512.

     size : The length of featureArray

     high : Similarity score

     res : Face recognition results.The length of res is always 1.If
   no face is recognized in the picture,res[0] return
   ERROR_INVALID_PARAM.If the face in image is not in featureArray,
   res[0] return ERROR_NOT_EXIST. If res [0]> = 0, the index of the
   featureArray is returned.

   Returns       Execution succeed return SUCCESS       Execution
   failed return ERROR_FAILURE

Example ：

   float[][] featurelist = new float[][]; //Array storing facial feature values
   int size = featurelist.lenth;
   int[] ret = new int[1];
   byte[] tmpPos = new byte[1024];
   float[] feature;
   float[] high = float[1];
   FaceAPP.Image image = FaceAPP.GetInstance().new Image();
   image.matAddrframe = mRgbaFrame.getNativeObjAddr();
   face.Recognize( feature, featurelist, size, high, tmpPos, res );

8. Detect faces

   int Detect( Image image, List faceinfos, int[] res )

   Function : Detect faces in submitted pictures

   Parameters :   image : Image object containing a face image

     faceinfos :  FaceInfo checklist. FaceInfo is an object used to
   save the results of operations

     res :  Face recognition results.The length of res is always 1.If
   no face is recognized in the picture,res[0] return
   ERROR_INVALID_PARAM.

   Returns       Execution succeed return SUCCESS       Execution
   failed return ERROR_FAILURE

Example ：

   int[] ret = new int[1];
   byte[] tmpPos = new byte[1024];//Used to store location information
   FaceAPP.Image image = FaceAPP.GetInstance().new Image(); //Init
   image.matAddrframe = mRgbaFrame.getNativeObjAddr(); //Set the memory address of the picture to be identified
   if( success = face.Detect( image, tmpPos, res ) ){
   //to do
   };

9. Compare feature data

   int Compare( float[] origin, float[] chose, float score )

   Function :  Used to compare the similarity of two feature values

   Parameters :    origin : Feature array to be compared

     chose : Feature array for comparison

     score : similarity between origin and chose

   Returns       Execution succeed return SUCCESS       Execution
   failed return ERROR_FAILURE

Example ：

   float score;
   float[] origin = new float[512];
   Float[] chose = new float[512];
   face.Compare( origin, chose, score );

10. Face extraction with binocular and living body

   int GetFeature( Image image, Image grayImage, float[] feature, List
   faceinfos, int[] res )

   Function : Extracting facial features from data obtained by
   binocular cameras,and only one face can be extracted.

   Parameters:   image : Image object containing a face image

     grayImage : Image object containing an infrared image

     feature : Stores the facial feature information of the detection
   results, and returns an empty array without a face

     faceinfos : FaceInfo checklist. FaceInfo is an object used to
   save the results of operations

     res : Face recognition results.The length of res is always 1.If
   no face is recognized in the picture,res[0] return
   ERROR_INVALID_PARAM.

   Returns       SUCCESS : Successfully obtained face information
   ERROR_FAILURE : No face information was obtained

Example ：

   float[] feature = new float[512];
   int[] ret = new int[1];
   byte[] tmpPos = new byte[1024]; //Used to store location information
   ret = face.GetFeature( image, grayImage, feature, tmpPos, res);
   if( ret == SUCCESS ){
   //to do 
   }

11. Liveness Detection

   int DetectLiveness(Image image, List faceinfos, int[] res)

   Function: Detect whether a face is alive.

   Parameters:   image : Color face pictures, pictures for detection
   and recognition.

     faceinfos : FaceInfo checklist. FaceInfo is an object used to
   save the results of operations

     res : Face recognition results.The length of res is always 1.If
   no face is recognized in the picture,res[0] return
   ERROR_INVALID_PARAM.

   Returns       SUCCESS : Living       ERROR_FAILURE : Non-living

Example ：

   int[] ret=new int[1];
   ret= face.DetectLiveness(image,grayImage ,tmpPos,res);
   if(ret== SUCCESS){
   //to do 
   }

12. Binocular for Liveness detection

   int GetDetectLiveness(Image image, Image grayImage, List faceinfos,
   int[] res)

   Function: Detect whether a face is alive.

   Parameters:   image : Color face pictures, pictures for detection
   and recognition.

     grayImage :Image object containing an infrared image.

     faceinfos : FaceInfo checklist. FaceInfo is an object used to
   save the results of operations

         res : Face recognition results.The length of res is always
   1.If no face is recognized in the        picture,res[0] return
   ERROR_INVALID_PARAM.

   Returns       SUCCESS : Living       ERROR_FAILURE : Non-living

Example ：

   int[] ret=new int[1];
   ret= face.GetDetectLiveness(image,grayImage ,tmpPos,res);
   if(ret== SUCCESS){
   //to do 
   }

13. Binocular Calibration

   int Calibration( Image image, Image grayImage, float[] scale, int[]
   Rect, int[] res );

   Function: The calibration of the dual camera recognition consisting
   of infrared light and ordinary light requires one person to stand
   at the optimal position (0.8-1 meters), and it takes about 20
   checks to obtain the face correction parameters. And return the
   coordinates of the infrared camera display area relative to
   ordinary light, this area is an effective recognition and living
   body detection area.

   Parameters :   image : Color face pictures, pictures for detection
   and recognition.

     grayImage : Image object containing an infrared image

     scale : Returns Face Frame Correction Parameters

     rect : Returns the overlapping area of the infrared image and the
   color image (the corresponding area of the infrared image in the
   color image / recommended detection area)

     res : Face recognition results.The length of res is always 1.If
   no face is recognized in the picture,res[0] return
   ERROR_INVALID_PARAM.

   Returns       SUCCESS : Calibration succeeded       ERROR_FAILURE :
   Calibration failed

Example ：

   int[] ret = new int[1];
   float[] scale = new float[1];
   int[] rect = new int[4];
   ret = face.Calibration( image, grayimage, scale, rect, res );
   if( ret == SUCCESS ){
   //to do 
   }

14. Device Activation 1

   int AuthorizedDevice( String uidStr, String password, Context
   activity )

   Function:  Device activation

   Parameters:   uidStr : OEMID + contract Id

     password : password

   Returns       0 : Authentication succeeded

Example ：

   String oem_id = "1000000000000001";//OEMID
   String contract_id = "0001";// contract Id
   String password = "0123456789abcdef0123456789abcdef"; //password
   String uidStr = oem_id + contract_id;
   int res = face.AuthorizedDevice( uidStr, password, LoginActivity.this ); 

15. Device Activation 2

   int fireflyInit(Context context, String uidStr, String password)

   Function : Device activation;This interface is a temporary
   interface for firefly, which can permanently activate the device.
   This interface may be removed later.If the interface description is
   not included in subsequent development documents, it has been
   removed. The development documents are subject to the documents in
   the open source Demo.

   Parameters :   uidStr : OEMID + contract Id

     password : password

   Returns       0 : Authentication succeeded

Example ：

   String oem_id = "1000000000000001";//OEMID
   String contract_id = "0001";//contract Id
   String password = "0123456789abcdef0123456789abcdef"; //password
   String uidStr = oem_id + contract_id;
   int res =face.fireflyInit(LoginActivity.this, uidStr, password);

16. Device Activation 3

   int AuthorizedDeviceUserPassword(String uidStr, String password,
   Context context, String userPassword)

   Function : Device activation，This interface is mainly used for the
   activation method with user password.

   Parameters :   uidStr : OEMID + contract Id

     password : password

     userPassword:user password

   Returns       0 : Authentication succeeded

Example ：

   String oem_id ="1000000000000001";//OEMID
   String contract_id ="0001";//contract Id
   String password = "0123456789abcdef0123456789abcdef";//password
   String userPassword ="012345678912";//user password;
   String uidStr = oem_id+contract_id;
   int res =face. AuthorizedDeviceUserPassword(uidStr,password,
   LoginActivity.this,userPassword);

17. Get authentication activation status

   int getAuthStatus()

   Function: Get authentication activation status

   Parameters:   void

   Returns       0 : activated

Example ：

   int res = face.getAuthStatus(); 

18. Extracting facial features

   int GetFeature( Image image, float[] feature, List faceinfos, int[]
   res )

   Function: Extract feature values of pictures in image,and get only
   the characteristics of one person in the picture。

   Parameters:   image : Image object containing a face image

     feature : Stores the facial feature information of the detection
   results, and returns an empty array without a face

     faceinfos : FaceInfo checklist. FaceInfo is an object used to
   save the results of operations

     res : Face recognition results.The length of res is always 1.If
   no face is recognized in the picture,res[0] return
   ERROR_INVALID_PARAM.

   Returns       SUCCESS : Feature extraction successful
   ERROR_FAILURE : No face information was obtained

Example ：

   float[] feature = new float[512];
   int[] ret = new int[1];
   byte[] tmpPos = new byte[1024]; //Used to store location information
   ret = face.GetFeature( image, feature, tmpPos, res );
   if( ret == SUCCESS ){
   //to do 
   }

19. Extraction of face features (based on face coordinate information)

   int GetFeature( Image image, FaceInfo detectInfo, float[] feature,
   int[] res )

   Function:  Extract feature values of pictures in image,and get only
   the characteristics of one person in the picture。

   Parameters:   image : Image object containing a face image

     detectInfo : Face information detected for facial feature
   extraction

     feature : Stores the facial feature information of the detection
   results, and returns an empty array without a face

     res : Face recognition results.The length of res is always 1.If
   no face is recognized in the picture,res[0] return
   ERROR_INVALID_PARAM.

   Returns       SUCCESS : Feature extraction successful
   ERROR_FAILURE :  No face information was obtained

Example ：

   float[] feature = new float[512];
   int[] ret = new int[1];
   float[] detectinfo = new float[]{ x0, y0, x1, y1, landmarkx0, landmarky0,
   				 landmarkx1, landmarky1, landmarkx2, landmarky2,
   				 landmarkx3, landmarky3, landmarkx4, landmarky4 }
   byte[] tmpPos = new byte[1024]; //Used to store location information
   ret = face.GetFeature( image, detectinfo, feature, res );
   if( ret == SUCCESS ){
   //to do 
   }

20. Extracting facial features (based on picture files)

   float[] GetFeature(String path, List faceinfos)

   Function: Get the facial feature value data in the image according
   to the path of the incoming image file.

   Parameters:   path :The absolute path of the face image file.

     faceinfos : FaceInfo checklist. FaceInfo is an object used to
   save the results of operations

   Returns       Float[] : Face feature value array

Example ：

   int ret = this.FaceGetFeatureFromAddr(addr, feature, mFaceInfos, policy);

   if (ret == -1) {
        return null;
   } else {
        faceInfos.addAll(Arrays.asList(this.mFaceInfos).subList(0,ret));
        return this.feature;
   }

21. Extract key points of the face

   int GetLandmark ( Image image, float[] landmark, int[] res )

   Function: Get face key point coordinate information

   Parameters:   Image : Face picture, a picture used to extract key
   point information of the face, only extract key point information
   of a person

     Landmark : Used to store the coordinates of 5 key points, which
   are left eye, right eye, nose, left side of lips, right side of
   lips

     res : Face recognition results.The length of res is always 1.If
   no face is recognized in the picture,res[0] return
   ERROR_INVALID_PARAM.

   Returns       SUCCESS : Successfully obtain coordinate information
   of face keypoints       ERROR_FAILURE : No facial keypoint
   information was obtained

Example ：

   float[] landmark = new float[10];
   int[] ret = new int[1];
   ret = face.GetLandmark( image, landmark, res);
   if( ret == SUCCESS ){

   }

22. Face quality

   FaceInfo getQuality(long matAddrframe)

   Function: Get the face information of the largest face in the image
   。

   Parameters:   matAddrframe : Image address stored in Mat object

   Returns       FaceInfo : Used to store face information in pictures

Example ：

   Faceinfo faceinfo=new Faceinfo();
   faceinfo=face.getQuality(matAddrframe);
   if(faceinfo!=null){
   }

23. Parameters Setting

   bool SetParameter( const String[] name, float value[] )

   Function: Set the entered Parameter key and value

   Parameters:   char[] name : Parameters key array

     a : Internal Parameters, set according to the example, please do
   not modify.

     b : Internal Parameters, set according to the example, please do
   not modify.

     c : Internal Parameters, set according to the example, please do
   not modify.

     d : Internal Parameters, set according to the example, please do
   not modify.

     factor : Detect face magnification. Internal Parameters, set
   according to the example, please do not modify.

     min_size : Minimum face frame size, recommended range:32-80

     faceclarity : Photo sharpness threshold, recommended
   range:200-400

     perfoptimize : Whether to optimize the effect , recommended
   range: 0 or 1

     livenessdetect : Whether live detection , recommended range:0 or
   1

     gray2colorscale : Binocular live detection ratio , recommended
   range: 0.1-0.5

     frame_num : Number of frames,recommended range:20-40

     quality_thresh : Picture quality threshold,recommended
   range:0.7-0.8

     mode : Operating mode :0-> Entrance 1-> Access control

     facenum : Detects the maximum number of faces and supports
   detection of up to 3 faces and 1 face.recommended range:1-3

     value[] : Parameters key array

   Returns:   Whether the parameters are set successfully

Example ：

   String[] name = { "a", "b","c", "d", "factor", "min_size", "clarity", "perfoptimize", 
   		  "livenessdetect", "gray2colorscale", "frame_num", "qualit_thresh",
   		  "mode", "facenum" };
   double[] value = {0.9, 0.9, 0.9, 0.715, 0.6, 64, 400, 1, 0, 0.5, 20, 0.8, 1, 1 };
   face.SetParameter( name, value );

24. Parameters Setting(Mode for modifying parameter length as needed)

   bool SetParameters( String[] name, float value[] )

   Function: Set the entered Parameter key and value

   Parameters:   char[] name : Parameters key array(length>=1),See
   details in 23.

     value[] : Parameters value array(length>=1)

   Returns:   Whether the parameters are set successfully

Example ：

   String[] name = { "perfoptimize", "livenessdetect", "frame_num", "quality_thresh", 
   		  "mode", "facenum" };
   double[] value = { 1, 0, 20, 0.8, 1, 1 };
   face.SetParameter( name, value );

25. Get version information

   public String GetVersion()

   Function: Get information about the current SDK version

   Parameters   void

   Returns:   Return current version information

Example ：

   Face.GetVersion();

26. Get Facelib Version

   public String GetFacelibVersion()

   Function: Get Facelib Version

   Parameters:   void

   Returns:   Return Facelib version information

Example ：

   Face.GetFacelibVersion();

27. Open the face database

   int OpenDB()

   Function: Using face database, internal face database can achieve
   1-to-N efficient and fast query

   Parameters:   void

   Returns       Execution succeed return SUCCESS       Execution
   failed return ERROR_FAILURE

Example ：

   if(face.OpenDB() == SUCCESS){
   // TODO
   }

28. Registered face

   int AddDB( float[] feature, string name )

   Function: Registered face to database

   Parameters:   feature : Facial features

     name : Registered name

   Returns       Execution succeed return SUCCESS       Execution
   failed return ERROR_FAILURE

Example ：

   String name= "test";
   int[] res=new int[];
   if( Face.GetFeature( image, feature, tmpPos, res ) == SUCCESS ){
   	Face.AddDB ( feature, name );
   }

29. Save face

   int SaveDB()

   Function: Save the results of adding, modifying, and deleting faces
   to a database file；If it is not called，the modification is
   invalid after exiting the program.

   Parameters:   void

   Returns       Execution succeed return SUCCESS       Execution
   failed return ERROR_FAILURE

Example ：

   Face.AddDB(feature,name);
   ...
   //After adding faces, make sure to execute Face.SaveDB () before restarting the device;
   Face.SaveDB();

30. Delete face

   int DelDB(string name)

   Function:   Delete face

   Parameters:   name : Registered name

   Returns       Execution succeed return SUCCESS       Execution
   failed return ERROR_FAILURE

Example ：

   String name= "test";
   Face.DelDB (name);

31. Delete all registered faces

   int DelAllDB()

   Function: Delete all registered faces

   Parameters:   void

   Returns       Execution succeed return SUCCESS       Execution
   failed return ERROR_FAILURE

Example ：

   Face.DelAllDB ();

32. Querying Face

   String QueryDB( float[] feature, float [] score )

   Function: Returns the data that is closest to the input facial
   features.

   Parameters:   feature : Facial features

     score : Store the similarity score of the queried face data and
   the input face data

   Returns       Execution succeed return registered name
   Execution failed return unknown

Example ：

   float[] score = new float[1];
   String name = Face.QueryDB( feature, score );
   if( score > thresh_hold ){
   // TODO
   }

33. Close the face database

   int CloseDB()

   Function: Close the face database

   Parameters:   void

   Returns       Execution succeed return SUCCESS       Execution
   failed return ERROR_FAILURE

Example ：

   if( Face.CloseDB() == SUCCESS ){
   // TODO
   }

34. Load Quick Compare Function

   int FastQueryInit()

   Function: Load Quick Compare Function

   Parameters:    void

   Returns       Execution succeed return SUCCESS       Execution
   failed return ERROR_FAILURE

Example ：

   if(Face.FastQueryInit () == SUCCESS){
   // TODO
   }

35. Refresh fast match buffer

   int FastQueryFlush(float [] data, int num)

   Function: Refresh fast match buffer

   Parameters:   Data : External feature buffer

     Num : Number of features

   Returns       Execution succeed return SUCCESS       Execution
   failed return ERROR_FAILURE

Example ：

   if(Face.FastQueryFlush(data,num) == SUCCESS){
   // TODO
   }

36. Quick Query

   int FastQuery(float[] data, float[] feature, float[] scores, int
   num)

   Function:   Quick query

   Parameters:   Data : External feature buffer

     feature : The feature to query

     scores:Similarity score

     num : Number of features

   Returns       Execution succeed return SUCCESS       Execution
   failed return ERROR_FAILURE

Example ：

   if(Face.FastQuery(data,feature,num) == SUCCESS){
   // TODO
   }

37. Get face attributes

   int GetFaceAttr( Image image, FaceInfo data, FaceAttribute
   face_attr, int *res )

   Function: After Detect is executed, the face attributes including
   age, gender and expression are obtained by detecting the acquired
   face position and keypoint information.

   Parameters:   image : Image object containing a face image

     data : Detected face position and face keypoint information

     face_attr : Store calculated face attributes

     res : Face recognition results.The length of res is always 1.If
   no face is recognized in the picture,res[0] return
   ERROR_INVALID_PARAM.

   Returns       SUCCESS : Obtain face attributes succeed
   ERROR_FAILURE : Obtain face attributes failed

Example ：

   if( face.GetFaceAttr( Image, data, attr, res ) == SUCCESS ){
   // TODO
   }

38. Release face recognition resources

   public void Destroy()

   Function: Free resources allocated when initializing and setting
   Parameters

   Parameters:   void

   Returns  void

Example ：

   Face.Destroy(); 

39. Sample code

Initialize the binocular camera face recognition demo.

   public class MainActivity extends Activity implements CvCameraViewListener2 {

       private FaceAPP face= FaceAPP.GetInstance(); //face as member variable
       .........
       @Override
       protected void onCreate(Bundle savedInstanceState) {
         .........
           String[] name={"a","b","c","d","factor","min_size","clarity","perf-optimize","liveness-detect","gray2color-scale"};
   	double[] value={0.9,0.9,0.9,0.715,0.6,64,400,1,0,0.5};
   	face.SetParameter(name,value);
   	mainLoop = new Thread() { //Face detection needs to run on a child thread

   	public void run() {
   	    .........
   	    float[] feature=new float[512];
   	    byte[] tmpPos = new byte[1024];// used to store position information
               switch (mixController.curState){
                    ......
                    case mixController. STATE_IDLE :
                         FaceAPP.Image image= FaceAPP. GetInstance ().new Image();
                         image.matAddrframe=mRgbaFrame.getNativeObjAddr();
                         int[] res=new int[1];
                         int ret;
                         ret= face.GetFeature(image,feature,tmpPos,res);
                         if(ret== SUCCESS){
                               //to do Successfully obtained facial feature values
                          }
               }
           }
       }
   }