AI algorithm deployment¶
AIBOX-1684X has a comprehensive local private deployment capability of AI algorithms, whether it is a novel large language model with high computing requirements, or a classic YOLOv5 object detection model, AIBOX-1684X can be competent.
This chapter mainly introduces the SOPHON-DEMO project. For large model deployment, refer to the LLM deployment chapter.
SOPHON-DEMO project introduction¶
SOPHON-DEMO contains a series of porting routines for mainstream AI algorithms, as well as detailed deployment documentation to make it easy for users to run.
Project repository link: SOPHON-DEMO。
Examples list¶
The examples provided by SOPHON-DEMO are divided into three modules from easy to difficult: tutorial, sample and application.
tutorialmodule stores some examples of basic interfaces.samplemodule stores some serial examples of classic algorithms on SOPHONSDK.applicationmodule stores some typical applications in typical scenarios.
| tutorial | code |
|---|---|
| resize | C++/Python |
| crop | C++/Python |
| crop_and_resize_padding | C++/Python |
| ocv_jpubasic | C++/Python |
| ocv_vidbasic | C++/Python |
| blend | C++/Python |
| stitch | C++/Python |
| contents | category | code | BModel |
|---|---|---|---|
| LPRNet | License Plate Recognition | C++/Python | FP32/FP16/INT8 |
| ResNet | Image classification | C++/Python | FP32/FP16/INT8 |
| RetinaFace | Face detection | C++/Python | FP32 |
| segformer | Semantic segmentation | C++/Python | FP32/FP16 |
| SAM | Semantic segmentation | Python | FP32/FP16 |
| yolact | Instance segmentation | C++/Python | FP32/FP16/INT8 |
| YOLOv8_seg | Instance segmentation | C++/Python | FP32/FP16/INT8 |
| PP-OCR | OCR | C++/Python | FP32/FP16 |
| OpenPose | Keypoint detection | C++/Python | FP32/FP16/INT8 |
| C3D | Video recognition | C++/Python | FP32/FP16/INT8 |
| DeepSORT | Object tracking | C++/Python | FP32/FP16/INT8 |
| ByteTrack | Object tracking | C++/Python | FP32/FP16/INT8 |
| CenterNet | Object Detection + pose estimation | C++/Python | FP32/FP16/INT8 |
| YOLOv5 | Object Detection | C++/Python | FP32/FP16/INT8 |
| YOLOv34 | Object Detection | C++/Python | FP32/INT8 |
| YOLOX | Object Detection | C++/Python | FP32/INT8 |
| SSD | Object Detection | C++/Python | FP32/INT8 |
| YOLOv7 | Object Detection | C++/Python | FP32/FP16/INT8 |
| YOLOv8_det | Object Detection | C++/Python | FP32/FP16/INT8 |
| YOLOv5_opt | Object Detection | C++/Python | FP32/FP16/INT8 |
| ppYOLOv3 | Object Detection | C++/Python | FP32/FP16/INT8 |
| ppYoloe | Object Detection | C++/Python | FP32/FP16 |
| WeNet | Speech Recognition | C++/Python | FP32/FP16 |
| BERT | Language | C++/Python | FP32/FP16 |
| ChatGLM2 | Language | C++/Python | FP16/INT8/INT4 |
| Llama2 | Language | C++ | FP16/INT8/INT4 |
| ChatGLM3 | Language | Python | FP16/INT8/INT4 |
| Qwen | Language | Python | FP16/INT8/INT4 |
| Qwen1_5 | Language | Python | FP16/INT8/INT4 |
| StableDiffusionV1.5 | Image Generation | Python | FP32/FP16 |
| GroundingDINO | MultiModal Object Detection | Python | FP16 |
| application | scenarios | code |
|---|---|---|
| VLPR | Multi-streams Vehicle License Plate Recognition | C++/Python |
| YOLOv5_multi | Multi-streams Object Detection | C++ |
| YOLOv5_multi_QT | Multi-streams Object Detection + QT_HDMI display | C++ |