MIPI CSI Introduction AIO-3399PRO-JD4 Development board with two MIPI interface, MIPI maximum support support 4K photography, and support 1080P 30FPS above video recording. In addition, the development board also supports USB camera. This article takes OV13850 camera as an example to explain the configuration process on the development board. Interface rendering DTS configuration isp0: isp@ff910000 { … status = "okay"; } isp1: isp@ff920000 { … status = "okay"; } Driving instructions The directory of camera codes is as follows: hal3_camera : ├── AAL //Android abstraction layer is responsible for interacting with the framework ├── common //Implementation of common files, such as threads, message processing, log printing, etc │ ├── gcss //xml Analytical correlation │ ├── imageProcess //Image processing, such as scale │ ├── jpeg //jpeg Encoding correlation │ ├── mediacontroller //Media pipeline related │ ├── platformdata hal3 The attributes supported by Hal 3 capability are mainly the attributes obtained from XML │ ├── utils //At present, there is only one error. H, which defines some return values │ └── v4l2dev //It encapsulates some specific IOS that interact with the v4l2 driver ├── etc //Profile directory ├── include //Header file of control loop, buffer & Manager related header file ├── lib //3A engine related Library ├── psl //Physical layer, physical implementation layer, all the implementation logic is basically here │ └── rkisp1 //At present, only rkisp1 is available │ ├── tasks //Basically, only a few notify interface classes and jpegencodetask are used │ └── workers //Data acquisition and processing are all here └── tools //Contains a python script that automatically generates graph setting XML Linux Kernel-4.4:| |-- arch/arm64/boot/dts/rockchip //DTS |-- drivers/phy/rockchip/ //mipi dphy driver |-- phy-rockchip-mipi-rx.c |-- drivers/media| |-- platform/rockchip/isp1 //rkisp1 isp driver |-- capture.c //Including MP / SP configuration and VB2, frame interrupt processing |-- dev.c //Including probe, asynchronous registration, clock, pipeline, IOMMU and media / v4l2 framework |-- isp_params.c //3A related parameter settings |-- isp_stats.c //3A related parameter settings |-- regs.c //Register Related read and write operations |-- rkisp1.c //Corresponding to ISP ﹣ SD entity node, including receiving data from Mipi, and with the function of cross |-- i2c/ |-- ov13850.c CIS(cmos image sensor)driver |-- vm149c.c VCM driver ic driver |-- spi/ rk1608 ap driver |-- rk1608.c register the rk1608 spi device |-- rk1608_dev.c register the /dev/rk_preisp misc device |-- rk1608_dphy.c //Register v4l2 media node and interact with rk1608 and AP Configuration principle The configuration process can be completed by setting camera-related pins and clocks. As can be seen from the schematic diagram of the camera interface below, pins to be configured are: MIPI_PWR, MIPI_PDN0_CAM/MIPI_PDN1_CAM, RST_CAM_0/RST_CAM_1. mipi interface MIPI_PWR = GPIO0_B5; MIPI_PDN0_CAM/MIPI_PDN1_CAM corresponds to GPIO2_A1 / GPIO2_A0 of RK3399Pro; MIPI_RST0/MIPI_RST1 corresponds to GPIO1_C4 of RK3399Pro; In the development board, these three pins are set in kernel/arch/arm64/boot/dts/rockchip/rk3399pro-firefly-usbacm.dtsi. Configuration steps configure driver ov13850 and driver VCM Configure Android Modify kernel/arch/arm64/boot/dts/rockchip/rk3399pro-firefly- usbacm.dtsi to init the camera driver: { vcc_mipi: vcc_mipi { compatible = "regulator-fixed"; enable-active-high; gpio = <&gpio3 29 GPIO_ACTIVE_HIGH>; pinctrl-names = "default"; pinctrl-0 = <&dvp_pwr>; regulator-name = "vcc_mipi"; }; }; ..... &i2c1 { vm149c: vm149c@0c { compatible = "silicon touch,vm149c"; status = "okay"; reg = <0x0c>; rockchip,camera-module-index = <0>; rockchip,camera-module-facing = "back"; }; ov13850b: ov13850b@10 { compatible = "ovti,ov13850"; reg = <0x10>; clocks = <&cru SCLK_CIF_OUT>; clock-names = "xvclk"; avdd-supply = <&vcc_mipi>; /* dvdd-supply = <>; */ /* dovdd-supply = <>; */ /* reset-gpios = <>; */ //mipi-pwr-gpios = <&gpio3 29 GPIO_ACTIVE_HIGH>; reset-gpios = <&gpio2 6 GPIO_ACTIVE_HIGH>; pwdn-gpios = <&gpio3 27 GPIO_ACTIVE_HIGH>; //pinctrl-names = "rockchip,camera_default", "rockchip,camera_sleep"; //pinctrl-0 = <&cam0_default_pins>; //pinctrl-1 = <&cam0_sleep_pins>; pinctrl-names = "rockchip,camera_default"; pinctrl-0 = <&cif_clkout>; firefly,clkout-enabled-index = <0>; rockchip,camera-module-index = <0>; rockchip,camera-module-facing = "back"; rockchip,camera-module-name = "CMK-CT0116"; rockchip,camera-module-lens-name = "Largan-50013A1"; lens-focus = <&vm149c>; port { ucam_out0: endpoint { remote-endpoint = <&mipi_in_ucam0>; data-lanes = <1 2>; }; }; }; vm149c_front: vm149c_front@0c { compatible = "silicon touch,vm149c"; status = "okay"; reg = <0x0c>; rockchip,camera-module-index = <1>; rockchip,camera-module-facing = "front"; }; ov13850f: ov13850f@10 { compatible = "ovti,ov13850"; reg = <0x10>; clocks = <&cru SCLK_CIF_OUT>; clock-names = "xvclk"; avdd-supply = <&vcc_mipi>; //mipi-pwr-gpios = <&gpio3 29 GPIO_ACTIVE_HIGH>; reset-gpios = <&gpio2 1 GPIO_ACTIVE_HIGH>; pwdn-gpios = <&gpio3 28 GPIO_ACTIVE_HIGH>; //pinctrl-names = "rockchip,camera_default", "rockchip,camera_sleep"; //pinctrl-0 = <&cam0_default_pins>; //pinctrl-1 = <&cam0_sleep_pins>; pinctrl-names = "rockchip,camera_default"; pinctrl-0 = <&cif_clkout>; firefly,second-enabled-index = <1>; firefly,clkout-enabled-index = <0>; rockchip,camera-module-index = <1>; rockchip,camera-module-facing = "front"; rockchip,camera-module-name = "CMK-CT0116"; rockchip,camera-module-lens-name = "Largan-50013A1"; lens-focus = <&vm149c_front>; port { ucam_out0: endpoint { remote-endpoint = <&mipi_in_ucam0>; data-lanes = <1 2>; }; }; }; Modify the power on sequence of ov13850 drive kernel/drivers/media/i2c/ov13850.c +static int __ov13850_power_on(struct ov13850 *ov13850); +static void __ov13850_power_off(struct ov13850 *ov13850); static int ov13850_s_power(struct v4l2_subdev *sd, int on) { struct ov13850 *ov13850 = to_ov13850(sd); @@ -1055,6 +1058,7 @@ static int ov13850_s_power(struct v4l2_subdev *sd, int on) goto unlock_and_return; } + __ov13850_power_on(ov13850); ret = ov13850_write_array(ov13850->client, ov13850_global_regs); if (ret) { v4l2_err(sd, "could not set init registers\n"); @@ -1063,9 +1067,20 @@ static int ov13850_s_power(struct v4l2_subdev *sd, int on) } ov13850->power_on = true; + /* export gpio */ + if (!IS_ERR(ov13850->reset_gpio)) + gpiod_export(ov13850->reset_gpio, false); + if (!IS_ERR(ov13850->pwdn_gpio)) + gpiod_export(ov13850->pwdn_gpio, false); } else { pm_runtime_put(&client->dev); + __ov13850_power_off(ov13850); ov13850->power_on = false; + /* unexport gpio */ + if (!IS_ERR(ov13850->reset_gpio)) + gpiod_unexport(ov13850->reset_gpio); + if (!IS_ERR(ov13850->pwdn_gpio)) + gpiod_unexport(ov13850->pwdn_gpio); } According to the sensor requirements, power on configuration of __ov13850_power_on. Android configuration modify hardware/rockchip/camera/etc/camera/camera3_profiles_rk3399pro.xml to register the camera： The main modifications are as follows: Sensor name cameraId="0" //Cameraid is set to 0 after and 1 before. When there is only one camera, it is 0. The name must be consistent with the name of the sensor driver. The currently provided sensor driver format is as follows: name="ov13850" moduleId="m00" By the following command: adb shell cat /sys/class/video4linux/*/name get the names of all v4l2 device nodes, where the form is as follows m00_b_ov13850 1-0010 as sensor node name。 In this command rule, M00 represents moduleid, mainly for matching len and flash , 'B' means the camera direction is post, if it is front, 'f', 'ov13850' means the sensor name, '1-0010' means the I2C address. DEBUG (1) If the rkisp driver is loaded successfully, there will be video and media devices in the /dev/ directory. Possible in system Multiple /dev/video devices. You can query the video nodes registered by rkisp through /sys console:/ # grep '' /sys/class/video4linux/video*/name /sys/class/video4linux/video0/name:rkisp1_mainpath /sys/class/video4linux/video1/name:rkisp1_selfpath /sys/class/video4linux/video2/name:rkisp1_dmapath /sys/class/video4linux/video3/name:rkisp1-statistics /sys/class/video4linux/video4/name:rkisp1-input-params /sys/class/video4linux/video5/name:rkisp1_mainpath /sys/class/video4linux/video6/name:rkisp1_selfpath /sys/class/video4linux/video7/name:rkisp1_dmapath /sys/class/video4linux/video8/name:rkisp1-statistics /sys/class/video4linux/video9/name:rkisp1-input-params (2) Judge whether the camera driver is loaded successfully When all cameras are registered, the kernel will print out the following log. localhost ~ # dmesg | grep Async [ 0.682982] rkisp1: Async subdev notifier completed If find that the kernel does not have the async subdev notifier completed log line, please check the sensor first Whether there are related errors and whether I2C communication is successful. (3) debug upper layer information dumpsys media.camera Under the terminal, you can directly modify /vendor/etc/camera/Camera3_profiles.xml to debug each parameter and restart it to take effect FAQs Unable to open the camera, first determine whether sensor I2C is communicating. If not, check whether MCLK and power supply are normal (power / powerdown / reset / MCLK / i2cbus) respectively Refer to the driver list ov8858.c Gc2145.c Ov7251 tc35874x.c