Developer Guide

  • 2022.3
  • 10/25/2022
  • Public Content
Contents

Sony’s IMX390 MIPI Camera Stream Through a GStreamer* Video Pipeline

This tutorial tells you how to set up and run a GStreamer* video pipeline using Sony’s IMX390 MIPI sensor.
Prerequisites: To enable Sony’s IMX390 MIPI sensor, you must use the Resource Design Center (RDC), have a Corporate Non-Disclosure Agreement (CNDA) in place, and ask for download access.

Step 1: Update the Kernel to Version 5.10.131

The only supported kernel version for EI for AMR is Intel’s Linux* LTS Kernel 5.10.131. Depending on your Ubuntu* 20.04 version, the default kernels are:
  • 5.4 on Ubuntu* 20.04.1
  • 5.8 on Ubuntu* 20.04.2
  • 5.11 on Ubuntu* 20.04.3
  • 5.13 on Ubuntu* 20.04.4
To check your kernel version:
uname -r
Step 1 is only valid for updating from kernel versions 5.4, 5.8, 5.11 and 5.13. If you have a different kernel, go to the Support Forum.
This process can take from 30 minutes to two hours, depending on your system.
  1. Clone Intel’s Linux* LTS kernel 5.10.131 repository from GitHub*.
    git clone https://github.com/intel/linux-intel-lts.git cd linux-intel-lts
  2. Check out to the
    lts-v5.10.131-yocto-220812T072653Z
    branch.
    git checkout lts-v5.10.131-yocto-220812T072653Z
  3. Install the necessary packages and
    gcc
    dependencies.
    sudo apt-get -y install build-essential gcc bc bison flex libssl-dev libncurses5-dev libelf-dev dwarves zstd
  4. Copy the configuration file to your folder, and rename it
    .config
    .
    cp /boot/config-$(uname -r) ./.config
  5. Change these values of the kernel configuration.
    scripts/config --set-str SYSTEM_TRUSTED_KEYS "" scripts/config --set-str CONFIG_SYSTEM_REVOCATION_KEYS ""
  6. Enable the Sony* IMX390-related and TI* TI960-related modules.
    scripts/config --module CONFIG_VIDEO_IMX390 scripts/config --module CONFIG_VIDEO_TI960 scripts/config --module CONFIG_VIDEO_INTEL_IPU6 scripts/config --module CONFIG_VIDEO_AR0234 scripts/config --module CONFIG_PINCTRL_TIGERLAKE scripts/config --enable CONFIG_INTEL_IPU6_TGLRVP_PDATA
  7. Compile the kernel, and make the Debian* kernel packages.
    This kernel compilation step takes a long time to complete:
    • approximately one hour on systems with 32 GB of RAM
    • two to three hours on systems with 8 GB of RAM
    make olddefconfig make -j4 deb-pkg
  8. Install the new Debian* kernel packages.
    cd ../ && sudo dpkg -i linux-*.deb
  9. For
    kernel versions 5.11 and 5.13
    , the newly installed kernel is a lower version than the system kernel, so the system needs to be configured to use it instead of the latest version.
    1. Open the GRUB.
      sudo cp /etc/default/grub /etc/default/grub.bak sudo vi /etc/default/grub
    2. Change the value of
      GRUB_DEFAULT
      from
      GRUB_DEFAULT=0
      to
      GRUB_DEFAULT="Advanced options for Ubuntu>Ubuntu, with Linux 5.10.131"
      .
    3. Update the GRUB.
      cd /tmp sudo update-grub
  10. Reboot your system.
    sync sudo reboot -fn
  11. Check your kernel version after reboot.
    uname -r

Step 2: Install the IPU6 Packages

  1. Download the Tiger Lake IPU6 Packages on the host (it contains the IPU RPM libraries and IPU firmware).
  2. Unzip the archive, and copy the RPM folder to the /tmp folder:
    unzip 645460.zip tar -xf ipu6_rpm_beta.tar.bz2 cp -r rpm /tmp
  3. Run the Docker* image as root:
    xhost + ./run_interactive_docker.sh amr-gstreamer:<TAG> root -e "--volume /sys/kernel/:/sys/kernel:rw --volume /sys/class:/sys/class:rw"
  4. If your network runs behind proxies, export the corresponding proxies in the container.
    export http_proxy="http://<http_proxy>:port" export https_proxy="http://<https_proxy>:port"
  5. Install the RPM package in the Docker* container:
    apt-get update apt-get install rpm
  6. Set isys_freq:
    echo 400 > /sys/kernel/debug/intel-ipu/buttress/isys_freq
  7. Install the IPU6 firmware and other necessary user-space libraries:
    rpm -ivh /tmp/rpm/* --nodeps --force
  8. Prepare the setup:
    export DISPLAY=:0 #If you are on VNC adapt this value to the correct one export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:/usr/lib64/pkgconfig:/usr/lib/pkgconfig export LD_LIBRARY_PATH=/usr/local/lib:/usr/lib64:/usr/lib export GST_PLUGIN_PATH=/usr/lib/gstreamer-1.0 export GST_GL_PLATFORM=egl

Step 3: Run the Sample Application

  1. Run the GStreamer* pipeline:
    gst-launch-1.0 icamerasrc device-name=imx390 printfps=true num-vc=1 ! video/x-raw,format=NV12,width=1920,height=1200 ! videoconvert ! xvimagesink
    Expected output: A video opens, showing images captured with the camera using Sony’s IMX390 MIPI sensor.

Troubleshooting

For general robot issues, go to: Troubleshooting for Robot Tutorials.

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.