Nvidia Jetson: Difference between revisions
(Adds an image to the GPIO programming section.) |
m (→Camera) |
||
(2 intermediate revisions by the same user not shown) | |||
Line 9: | Line 9: | ||
<br /> | <br /> | ||
==== Figuring out the JetPack Version ==== | ==== Figuring out the JetPack Version ==== | ||
The [https://developer.nvidia.com/embedded/jetpack-archive JetPack Archive] shows a list of JetPack download packages with corresponding versions of L4T (Linux for Tegra). | |||
In a running L4T installation, the version can be looked up via the Debian package manager utility: | |||
<pre class="terminal"> | <pre class="terminal"> | ||
dpkg-query --show nvidia-l4t-core | dpkg-query --show nvidia-l4t-core | ||
</pre> | </pre> | ||
<br /> | <br /> | ||
Some JetPack and L4T versions are listed below for convenience. | |||
{| | {| | ||
! style="width:6em; font-weight:bold" | L4T | ! style="width:6em; font-weight:bold; text-align: left" | L4T | ||
! style="width:6em; font-weight:bold" | JetPack | ! style="width:6em; font-weight:bold; text-align: left" | JetPack | ||
! style="width:8em; font-weight:bold" | release date | ! style="width:8em; font-weight:bold; text-align: left" | release date | ||
! style="font-weight:bold" | Ubuntu | ! style="font-weight:bold; text-align: left" | Ubuntu base | ||
! style="font-weight:bold; text-align: left" | Remark | |||
|- | |- | ||
| | | 35.1 || 5.0.2 || 2022-08-15 || 20.04 || 5.x drops support for Jetson Nano | ||
|- | |- | ||
| 32. | | 32.7.1 || 4.6.1 || 2022-02-23 || 18.04 || | ||
|- | |- | ||
| 32. | | 32.5 || 4.5.0 || 2021-01-22 || 18.04 || | ||
|- | |- | ||
| 32. | | 32.4.3 || 4.4.0 || 2020-07-06 || 18.04 || | ||
|- | |- | ||
| 32.2.0 || 4.2.1 || 2019-07-16 || 18.04 | | 32.2.0 || 4.2.1 || 2019-07-16 || 18.04 || | ||
|} | |} | ||
<br /> | <br /> | ||
Line 121: | Line 128: | ||
<br /> | <br /> | ||
=== Camera === | === Camera === | ||
Cameras can be connected to Jetson Nano over USB or over the CSI connector. In case of a CSI camera, the image sensor must be a Sony IMX219, such as the [https://www.raspberrypi.org/products/camera-module-v2/ Raspberry Pi Camera v2]. | Cameras can be connected to Jetson Nano over USB or over the CSI connector. In case of a CSI camera, the image sensor must be a '''Sony IMX219''', such as the [https://www.raspberrypi.org/products/camera-module-v2/ Raspberry Pi Camera v2]. | ||
<br /> | <br /> | ||
Use the '' | Use the ''Video4Linux'' command-line tool '''v4l2-ctl''' to list all video inputs that are available to the system and what type of image frames they provide. | ||
<pre class="terminal"> | <pre class="terminal"> | ||
sudo apt install v4l-utils | sudo apt install v4l-utils | ||
Line 130: | Line 137: | ||
</pre> | </pre> | ||
<br /> | <br /> | ||
You can view the video stream from the CSI camera using [[GStreamer]] | You can view the video stream from the CSI camera using [[GStreamer]] via | ||
<pre class="terminal"> | |||
gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=NV12, framerate=(fraction)30/1' ! nvoverlaysink | |||
</pre> | |||
for a full-screen view, or | |||
<pre class="terminal"> | <pre class="terminal"> | ||
gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM),width=1920, height=1080, framerate=30/1' ! nvvidconv flip-method=0 ! nvegltransform ! nveglglessink -e | gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM), width=1920, height=1080, framerate=30/1' ! nvvidconv flip-method=0 ! nvegltransform ! nveglglessink -e | ||
</pre> | </pre> | ||
for an OpenGL window view. | |||
<br /> | |||
<br /> | |||
''nvarguscamerasrc'' is a Gstreamer plugin for Nvidia's proprietary [https://docs.nvidia.com/jetson/l4t-multimedia/group__LibargusAPI.html Argus Camera API] for Linux on Tegra devices, and ''nveglglessink'' is a better-performing OpenGL sink than ''glimagesink''. Curiously, ''nvarguscamerasrc'' will show up in the GStreamer element list when searching for elements of type ''video'' but not when searching for ''source/video''. | |||
<pre class="terminal"> | <pre class="terminal"> | ||
gst-inspect-1.0 -t video | gst-inspect-1.0 -t video | ||
Line 217: | Line 231: | ||
=== GPIO Programming === | === GPIO Programming === | ||
[[File:Nvidia_Jetson_GPIO.jpg]] | [[File:Nvidia_Jetson_GPIO.jpg | 500px]] | ||
Latest revision as of 2022-12-11T12:40:10
The Jetson Family
Nvidia's Jetson family of products are compact Tegra computers, which feature a large number of parallel computation units that are suitable for hardware-accelerated machine learning applications. The various models differ in their computational performance.
JetPack
JetPack is Nvidia's name for a software bundle that contains development software and a modified version of Ubuntu Linux with a custom kernel for the Tegra hardware, called Linux For Tegra or L4T.
Figuring out the JetPack Version
The JetPack Archive shows a list of JetPack download packages with corresponding versions of L4T (Linux for Tegra).
In a running L4T installation, the version can be looked up via the Debian package manager utility:
dpkg-query --show nvidia-l4t-core
Some JetPack and L4T versions are listed below for convenience.
L4T | JetPack | release date | Ubuntu base | Remark |
---|---|---|---|---|
35.1 | 5.0.2 | 2022-08-15 | 20.04 | 5.x drops support for Jetson Nano |
32.7.1 | 4.6.1 | 2022-02-23 | 18.04 | |
32.5 | 4.5.0 | 2021-01-22 | 18.04 | |
32.4.3 | 4.4.0 | 2020-07-06 | 18.04 | |
32.2.0 | 4.2.1 | 2019-07-16 | 18.04 |
Assigning a Fixed Ethernet IP Address
You can either use the old ifconfig tool
ifconfig eth0 192.168.0.10 netmask 255.255.255.0 up
or the newer ip tool (the example below is incomplete)
ip addr show ip link set eth0 up
To make the IP address permanent, the easy way is to open the System Settings window from the Ubuntu GUI. Then
- open the Network settings,
- select the wired connection from the left hand list,
- click on the "Options..." button in the bottom right corner,
- open the "IPv4 Settings" tab,
- change the value of the "Method:" field to "Manual",
- enter the desired address (192.168.0.10) into the "Addresses" list.
Setting up WiFi
First, make sure that the WLAN hardware (maybe a USB stick) is operational. Check the availability of the wlan0 interface among all network interfaces:
ifconfig
Also, make sure that wpasupplicant is installed.
sudo apt-get install wpasupplicant
Now edit the file /etc/wpa_supplicant.conf to replace the "ssid" and "psk" values in the network example that starts with the line # Simple case: WPA-PSK:
network={ ssid="ssid_name" psk="password" priority=5 }
where ssid_name and password need to be replaced with the WiFi name (SSID) and the password of your router, respectively.
Finally, connect to the router:
sudo killall wpa_supplicant sudo wpa_supplicant -B -i wlan0 -c /etc/wpa_supplicant.conf -D wext sudo dhclient wlan0
Setting up SSH
The developer kit features HDMI and DisplayPort outputs for connecting to a display. In a robot, the control interface for the Jetson Nano will be a terminal session via SSH. To set up the SSH service on the Nano first make sure that the ssh package is installed:
sudo apt install openssh-client openssh-server
You can now start the SSH service like this:
sudo service ssh start
and stop it like this:
sudo service ssh stop
To make the SSH service start automatically at each boot, type
sudo systemctl enable ssh sudo systemctl enable ssh.service sudo systemctl start ssh
To prevent the system from starting the SSH service at each boot, type
sudo systemctl stop ssh sudo systemctl disable ssh sudo systemctl disable ssh.service
SSH Login Over WiFi Connection
The Jetson Nano and the Jetson Xavier developer kit boards come without WiFi connectivity. You can use a USB WiFi adaptor stick (e.g., Edimax ) to add WiFi connectivity. On Ubuntu, you will need to give permission to the system to make the WiFi connection available to all other users, which includes the SSH daemon, in order to enable remote logins over WiFi without requiring an active local login session (via connected peripherals or Ethernet connection).
To give permission,
- open the "System Settings" application,
- go to "Network",
- make sure "Wireless" is selected on the left panel,
- on the right side panel, next to the SSID of the WLAN server, click on the arrow icon,
- click on the "Settings..." button on the bottom right corner,
- in the window that opens, go to the tab "General",
- check the "All users may connect to this network" option
Camera
Cameras can be connected to Jetson Nano over USB or over the CSI connector. In case of a CSI camera, the image sensor must be a Sony IMX219, such as the Raspberry Pi Camera v2.
Use the Video4Linux command-line tool v4l2-ctl to list all video inputs that are available to the system and what type of image frames they provide.
sudo apt install v4l-utils v4l2-ctl --list-devices v4l2-ctl --list-formats
You can view the video stream from the CSI camera using GStreamer via
gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=NV12, framerate=(fraction)30/1' ! nvoverlaysink
for a full-screen view, or
gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM), width=1920, height=1080, framerate=30/1' ! nvvidconv flip-method=0 ! nvegltransform ! nveglglessink -e
for an OpenGL window view.
nvarguscamerasrc is a Gstreamer plugin for Nvidia's proprietary Argus Camera API for Linux on Tegra devices, and nveglglessink is a better-performing OpenGL sink than glimagesink. Curiously, nvarguscamerasrc will show up in the GStreamer element list when searching for elements of type video but not when searching for source/video.
gst-inspect-1.0 -t video
Streaming Images Over The Network
To stream the camera images over the network via UDP, run the following script on the Jetson
#!/bin/bash STREAM_RECEIVER_ADDRESS=$1 STREAM_RECEIVER_PORT=$2 GST_LAUNCH="gst-launch-1.0 -v" GST_DEBUG="--gst-debug=3" INPUT="nvarguscamerasrc ! 'video/x-raw(memory:NVMM),width=1280,height=720,framerate=30/1,format=(string)NV12' ! nvvidconv ! 'video/x-raw(memory:NVMM),format=(string)I420'" # Stream via RTP over UDP STREAM_OUTPUT="omxh264enc bitrate=8000000 ! 'video/x-h264, stream-format=byte-stream' ! h264parse ! rtph264pay config-interval=10 pt=96 ! udpsink host=<host address> port=<port number>" # Show in local GL window VIEW_OUTPUT="nvegltransform ! nveglglessink -e" if [ $# == 0 ]; then eval $GST_LAUNCH $INPUT ! $VIEW_OUTPUT elif [ $# == 2 ]; then eval $GST_LAUNCH $INPUT ! $STREAM_OUTPUT fi
where the client IP address and the client port number need to be supplied as arguments 1 and 2, respectively.
Run the following script on the client machine to receive and display the video stream in a GStreamer pipeline.
UNPACK_RTP_H264_PAYLOAD="'application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=H264, payload=96' ! rtph264depay ! avdec_h264" GST_UDPSRC="gst-launch-1.0 -v --gst-debug=3 udpsrc port=<port number>" VIDEO_SINK="videoconvert ! autovideosink" eval $GST_UDPSRC ! $UNPACK_RTP_H264_PAYLOAD ! $VIDEO_SINK
where you need to replace <port number> with the same port number used by the sender. Check the receiver's firewall settings to make sure the port is not blocked! On macOS you can do so by opening the Firewall settings and first add Terminal to the list of applications for which incoming connections are allowed. Upon launching the script, you will be prompted to allow incoming connections for gst-launch-1.0 as well.
The order in which the Gstreamer pipelines are launched is irrelevant because of the connectionless data transmission via UDP. The receiver will be continuously waiting for incoming UDP packets, and the sender will be sending out packets even if there is no receiver.
Resources
- example GStreamer pipelines (RidgeRun)
Power Modes
Use the nvpmodel command line tool for selecting or configuring the performance level of a Jetson device. You can query the current mode via
sudo nvpmodel -q --verbose
and switch between modes with the "-m" option like this
sudo nvpmodel -m <mode_number>
where <mode_number> is one of the mode numbers (e.g., 0 or 1) which the tool reads from the default configuration file "/etc/nvpmodel.conf" and which you can edit to create a custom mode:
sudo vim /etc/nvpmodel.conf
Disabling USB Autosuspend
If you are operating your Jetson with an attached USB mass storage device you will need to disable USB autosuspend. Otherwise the device will become unavailable after a while, and your applications will report an I/O error.
Edit the file /boot/extlinux/extlinux.conf by adding the following boot argument to the line starting with APPEND:
usbcore.autosuspend=-1
This will disable autosuspend upon the next reboot.
To check that the autosuspend mode changed from active (2) to disabled (-1), enter
cat /sys/module/usbcore/parameters/autosuspend
GPIO Programming
Jetson Nano Pinout
Based on the JetsonHacks pinout table.
Sysfs GPIO | Name | Pin | Pin | Name | Sysfs GPIO | |
---|---|---|---|---|---|---|
3.3 V Power out | 1 | 2 | 5.0 V Power in | |||
I2C_2_SDA | 3 | 4 | 5.0 V Power in | |||
I2C_2_SCL | 5 | 6 | GND | |||
gpio216 | AUDIO_MCLK | 7 | 8 | UART_2_TX | ||
GND | 9 | 10 | UART_2_RX | |||
gpio50 | UART_2_RTS | 11 | 12 | I2S_4_SCLK | gpio79 | |
gpio14 | SPI_2_SCK | 13 | 14 | GND | ||
gpio194 | LCD_TE | 15 | 16 | SPI_2_CS1 | gpio232 | |
3.3 V | 17 | 18 | SPI_2_CS0 | gpio15 | ||
gpio16 | SPI_1_COPI | 19 | 20 | GND | ||
gpio17 | SPI_1_CIPO | 21 | 22 | SPI_2_CIPO | gpio13 | |
gpio18 | SPI_1_SCK | 23 | 24 | SPI_1_CS0 | gpio19 | |
GND | 25 | 26 | SPI_1_CS1 | gpio20 | ||
I2C_1_SDA | 27 | 28 | I2C_1_SCL | |||
gpio149 | CAM_AF_EN | 29 | 30 | GND | ||
gpio200 | GPIO_PZ0 | 31 | 32 | LCD_BL_PWM | gpio168 | |
gpio38 | GPIO_PE6 | 33 | 34 | GND | ||
gpio76 | I2S_4_LRCK | 35 | 36 | UART_2_CTS | gpio51 | |
gpio12 | SPI_2_COPI | 37 | 38 | I2S_4_SDIN | gpio77 | |
GND | 39 | 40 | I2S_4_SDOUT | gpio78 |
Using SYSFS
The old, deprecated way of GPIO programming on Linux is to write to the sysfs virtual files that are managed by the kernel. For GPIO, the Linux kernel provides the /sys/class/gpio virtual directory. GPIO commands, like creating an output pin and writing a value, are performed by writing to or reading from specific files in that directory.
Since sysfs is a filesystem-based interface, GPIO state can be easily manipulated with a shell script. Here is an example bash code for blinking an LED connected to GPIO pin 11:
echo 50 > /sys/class/gpio/export sleep 0.1 echo out > /sys/class/gpio/gpio50/direction sleep 0.1 for n in {0..2} do echo 1 > /sys/class/gpio/gpio50/value sleep 1 echo 0 > /sys/class/gpio/gpio50/value sleep 1 done echo 50 > /sys/class/gpio/unexport
Using Libgpiod
The best way for programming GPIO without having to mess with the inefficient sysfs, is to use the C library libgpiod or a binding to another programming language. First, make sure libgpiod, the C header, and the related command-line tools are installed
sudo apt install gpiod libgpiod-dev
gpiodetect gives us information about the available GPIO banks. Typing
sudo gpiodetect
returns
gpiochip0 [tegra-gpio] (256 lines) gpiochip1 [max77620-gpio] (8 lines)
where gpiochip0 is the GPIO bank exposed by the J41 header.
Drilling down into GPIO bank 0 with gpioinfo
gpioinfo gpiochip0
we can see the pin functions of J41 GPIO bank:
gpiochip0 - 256 lines: line 0: unnamed unused input active-high line 1: unnamed unused input active-high line 2: unnamed "pcie_wake" input active-high [used] line 3: unnamed unused input active-high line 4: unnamed unused input active-high line 5: unnamed unused input active-high line 6: unnamed "vdd-usb-hub-en" output active-high [used] line 7: unnamed unused input active-high line 8: unnamed unused input active-high line 9: unnamed unused input active-high line 10: unnamed unused input active-high line 11: unnamed unused input active-high line 12: "SPI1_MOSI" unused input active-high line 13: "SPI1_MISO" unused input active-high line 14: "SPI1_SCK" unused input active-high line 15: "SPI1_CS0" unused input active-high . . . line 252: unnamed unused input active-high line 253: unnamed unused input active-high line 254: unnamed unused input active-high line 255: unnamed unused input active-high
Using the gpioset shell command we can set the output level of a GPIO pin:
sudo gpioset gpiochip0 50=1
If the LED does not light up, check if line 50 is still being used by sysfs.
Back to programming the GPIO in C++. Create the file libgpiod_test.cpp containing the following C++ code:
#include <gpiod.h> #include <cstdlib> #include <thread> #include <chrono> int main(int numarg, char** args) { gpiod_chip* const gpio0 = gpiod_chip_open_by_name("gpiochip0"); if (gpio0 == NULL) { return EXIT_FAILURE; } gpiod_line* const line50 = gpiod_chip_get_line(gpio0, 50); if (line50 == NULL) { gpiod_chip_close(gpio0); return EXIT_FAILURE; } if (gpiod_line_request_output(line50, "LED", 0) != 0) { gpiod_line_release(line50); gpiod_chip_close(gpio0); return EXIT_FAILURE; } for (int i = 1; i <= 6; i++) { gpiod_line_set_value(line50, i%2); std::this_thread::sleep_for(std::chrono::milliseconds(1000)); } gpiod_line_release(line50); gpiod_chip_close(gpio0); return EXIT_SUCCESS; }
Compile and run.
g++ -o libgpiod_test libgpiod_test.cpp -lgpiod sudo ./libgpiod_test
Debug
To view the state of all the available GPIO pins type
sudo cat /sys/kernel/debug/tegra_gpio
Jetson Model Specifics
Jetson Nano
The Nano is the Jetson with the smallest footprint and lowest performance. It features 128 compute units based on the Maxwell architecture. The developer kit comes with host circuit board that provides connection ports like USB, Ethernet, HDMI and DisplayPort.
Power Supply and OS Installation
The Nano Developer Kit requires a power supply that can deliver 5 V and at least 2 A, either over the micro USB port, or via a standard 5.5/2.1mm barrel power jack, which requires bridging the J48 header pins with a jumper. When operated via the power jack, the micro USB port can be used to install the Nano operating system from an external computer. Otherwise, the OS image needs to be written to a micro-SD card that is inserted into the card slot of the developer kit.
Preconfigured Power Modes
- mode 0 (MAXN)
the default mode with maximum performance and up to 10 W power consumption - mode 1 (5 W)
low-power mode where only two out of four CPU cores are working and the GPU clock frequency is constrained to keep power consumption under 5 W.
MAXN | 5 W | |
---|---|---|
power budget | 10 Watts | 5 Watts |
CPUs in use | 4 | 2 |
CPU max freq. | 1479 MHz | 918 MHz |
GPU max freq. | 921 MHz | 640 MHz |