ARA-240 DNPU with MR-NavQ95 Setup Guide¶

This guide provides the steps to setup and run the ARA-240 DNPU with MR-NavQ95. It provides the steps for:

Hardware setup
Setting up the ARA SDK and eIQ AAF Connector, which is a REST-based server for LLM inference
Downloading and preparing models (CNNs and LLMs)

Prerequisites¶

Hardware:

MR-NavQ95 board
ARA-240 M.2 Module
Fan port to USB adapter
64 GB or greater SD card recommended
Internet access for the board

Software:

imx-6.18.20-2.0.0-navq image

Hardware Setup¶

Connect ARA-240 to top M.2 slot on MR-NavQ95.
The MR-NavQ95 does not have a fan port available for the ARA-240 fans. Currently, recommendation is to use a USB adapter and plug the fans in a separate USB port. Alternatively, one of the CAN ports in the CAN expansion board can be used with an adapter.

Software setup¶

Flash the resulted image to the SD card using uuu normally and boot.
After login, check that the ARA has been successfully initialized by the image by running the following command:
```
sudo chip_info.sh
```
Output should contain a table. Note the firmware version (raw) value.
If the firmware version (raw) is not 131072 or higher (like in the above screenshot where it is 65794), run the following commands to flash the new firmware:
```
sudo program_flash.sh
reboot
```
Run the following command to complete installation of the ARA module and resize the rootfs partition to the entire size of the SD card:
```
sudo /usr/share/rt-sdk-ara240/install.sh
```

Run CNNs (yolov8 or yolox)¶

Note: Internet access is required for the next step.

Run the following command to download the yolov8 or yolox models locally:

sudo fetch_models --repo-id nxp/YOLOv8
sudo fetch_models --repo-id nxp/YOLOX-Detection-Ara240

Use the built-in performance script to check the inference performance of a downloaded modelL
```
sudo run_model_perf.sh
```
For a more complex example, follow the readme at https://github.com/nxp-imx-support/ara2-vision-examples to install and run the vision examples.

Run LLMs¶

Note: Internet access is required for the next steps.

Run the following command to install the eIQ AAF Connector:
```
sudo aaf-connector
```
Run the following command to download the Qwen2.5-7B model from NXP Huggingface. Available models are listed at https://huggingface.co/nxp
```
sudo fetch_models --repo-id nxp/Qwen2.5-7B-Instruct-Ara240
```
Note: Internet access is no longer required from this point on.
Applications use the /var/run/proxy.sock file to communicate with the lower-level ARA-240 runtime stack. Use the following command to change permissions, allowing user-space applications to read and write to the file:
```
sudo chmod a+rw /var/run/proxy.sock
```
Start the eIQ AAF Connector:
```
aaf-connector &
```

Load the downloaded model with the following command:

curl -X 'POST' 'http://127.0.0.1:8000/v1/models' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{"name": "Qwen2.5-7B-Instruct"}'

Test that everything is working with the following inference request:

curl -H 'Content-Type: application/json' \
  -d '{ "model":"Qwen2.5-7B-Instruct","messages":[{"role":"user", "content":"Who are you?"}]}' \
  -X POST 0.0.0.0:8000/v1/chat/completions

Available Models¶

NXP publishes a curated collection of models precompiled for the ARA-240 DNPU (CNNs such as YOLOv8/YOLOX and LLMs such as Qwen2.5) on Hugging Face. Use it to discover which models are available and how to fetch them with fetch_models --repo-id nxp/<model>:

ARA240 model collection: https://huggingface.co/collections/nxp/ara240-models

Further Documentation¶

Ara Software Release Notes (RN00459, Rev 2.1): ara-software-release-notes-rn00459-2.1.pdf
Ara Runtime SDK documentation: https://github.com/nxp-imx/rt-sdk-ara2/tree/main#-getting-started
Ara Vision Examples: https://github.com/nxp-imx-support/ara2-vision-examples
Ara SDK for Model Compilation: https://www.nxp.com/design/design-center/software/embedded-software/ara-software-development-kit:ARA-SDK
eIQ AAF Connector: https://github.com/nxp-imx-support/eiq-aaf-connector