JetPack

JetPack includes Jetson Linux with bootloader, Linux kernel, Ubuntu desktop environment, and a complete set of libraries for GPU computing, multimedia, graphics, and computer vision acceleration. It also includes examples, documentation, and developer tools for hosts and developer suites, and supports higher-level SDKs such as DeepStream for streaming video analysis, Isaac for robot development, and Riva for conversational AI.

Learn More JetPack 6.0

Install

sudo apt update
sudo apt install nvidia-jetpack

TensorRT

NVIDIA TensorRT is an ecosystem of APIs for high-performance deep learning inference. TensorRT includes an inference runtime and model optimizations that deliver low latency and high throughput for production applications. The TensorRT ecosystem includes TensorRT, TensorRT-LLM, TensorRT Model Optimizer, and TensorRT Cloud. TensorRT Advantage:

  • Speed Up Inference by 36X

  • Optimize Inference Performance

  • Accelerate Every Workload

  • Deploy, Run, and Scale With Triton

Learn More TensorRT

Generative AI

NVIDIA Jetson Orin gives you unparalleled AI compute, large unified memory, and comprehensive software stacks, delivering superior energy efficiency to drive the latest generative AI applications. It delivers fast inference for any gen AI models powered by the transformer architecture, leading the edge performance on MLPerf. The Jetson AI Lab is your gateway to explore tutorials on text generation, text + vision models, image generation, distillation techniques, and access resources to run these models on NVIDIA Jetson Orin. Join the generative AI revolution and start today.

Try it Now

Text Generative: LLM

Model: llama-2-7b-chat.Q4_K_M.gguf

How to try

No.1

AIBOX-Orin NX and PC within the same local area network:

  • On AIBOX-Orin NX to run command: jetson-containers run $(autotag text-generation-webui)

  • On PC to open your browser and access http://<AIBOX-Orin NX IP_ADDRESS>:7860

No.2

  • On AIBOX-Orin NX to run command: jetson-containers run $(autotag text-generation-webui)

  • On AIBOX-Orin NX to open your browser and access http://0.0.0.0:7860

Load Model

On web UI:

  • Click “Model”

  • Click “🔄”

  • Select “TheBloke_Llama-2-7b-Chat-GGUF”

  • Set n-gpu-layers to 128

  • Click “Load”

_images/ai-lab-text-load-model.png

Image Generative: Stable Diffusion

Model: v1-5-pruned-emaonly.safetensors

How to try

No.1

AIBOX-Orin NX and PC within the same local area network:

  • On AIBOX-Orin NX to run command: jetson-containers run $(autotag stable-diffusion-webui)

  • On PC to open your browser and access http://<AIBOX-Orin NX IP_ADDRESS>:7860

No.2

  • On AIBOX-Orin NX to run command: jetson-containers run $(autotag stable-diffusion-webui)

  • On AIBOX-Orin NX to open your browser and access http://0.0.0.0:7860

Load Model

On web UI
_images/ai-lab-image-load-model.png