# esp-dl

**Repository Path**: esp-components/esp-dl

## Basic Information

- **Project Name**: esp-dl
- **Description**: Espressif deep-learning library for AIoT applications
- **Primary Language**: C
- **License**: MIT
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 4
- **Forks**: 0
- **Created**: 2022-03-08
- **Last Updated**: 2026-11-04

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# ESP-DL [[中文]](./README_cn.md)

[![Documentation Status](./docs/_static/doc_latest.svg)](https://docs.espressif.com/projects/esp-dl/en/latest/index.html)  [![Component Registry](https://components.espressif.com/components/espressif/esp-dl/badge.svg)](https://components.espressif.com/components/espressif/esp-dl)

ESP-DL is a lightweight and efficient neural network inference framework designed specifically for ESP series chips. With ESP-DL, you can easily and quickly develop AI applications using Espressif's System on Chips (SoCs).

## Overview

ESP-DL offers APIs to load, debug, and run AI models. The framework is easy to use and can be seamlessly integrated with other Espressif SDKs. ESP-PPQ serves as the quantization tool for ESP-DL, capable of quantizing models from ONNX, Pytorch, and TensorFlow, and exporting them into the ESP-DL standard model format.

- **ESP-DL Standard Model Format**: This format is similar to ONNX but uses FlatBuffers instead of Protobuf, making it more lightweight and supporting zero-copy deserialization, with a file extension of `.espdl`.
- **Flexible Mixed Quantization**: ESP-DL supports 8-bit (w8a8), 16-bit (w16a16), and mixed-precision (w8a16) quantization. These schemes can be freely combined within a model to preserve accuracy while maximizing inference efficiency.
- **Efficient Operator Implementation**: ESP-DL efficiently implements common AI operators such as Conv, Gemm, Add, and Mul. [**The list of supported operators**](./operator_support_state.md). For performance benchmark data of some operators on ESP32-S3 and ESP32-P4, please refer to [benchmark_report.md](./benchmark_report.md).
- **Static Memory Planner**: The memory planner automatically allocates different layers to the optimal memory location based on the user-specified internal RAM size, ensuring efficient overall running speed while minimizing memory usage.
- **Dual Core Scheduling**: Automatic dual-core scheduling allows computationally intensive operators to fully utilize the dual-core computing power. Currently, Conv2D and DepthwiseConv2D support dual-core scheduling.
- **8bit LUT Activation**: All activation functions except for ReLU and PReLU are implemented using an 8-bit LUT (Look Up Table) method in ESP-DL to accelerate inference. You can use any activation function, and their computational complexity remains the same.

## News
- [2026/10/15] Optimized a wide range of operators, including Conv, Gemm, MatMul, Resize, Reduce, LUT, Transpose, and Pooling. Models such as MobileNetV2 and YOLO11 achieve more than 3× speedup on ESP32-S3 and more than 1.9× speedup on ESP32-P4. Thanks to [Hosi121](https://github.com/Hosi121), [BoumedineBillal](https://github.com/BoumedineBillal), and other contributors.
- [2026/9/1] Starting from ESP-DL v3.3.11, w8a16 mixed quantization is fully supported.
- [2026/7/30] Added [PP-OCRv6](./models/pp_ocr_v6/) OCR model and [example](./examples/pp_ocr_v6/), recognizing Chinese, English and 46 Latin-script languages on-device on ESP32-P4.
- [2026/6/5] Introduce two skills, [esp32s3-pie-simd](./tools/agents/skills/esp32s3-pie-simd) and [esp32p4-pie-simd](./tools/agents/skills/esp32p4-pie-simd), to help the code agent rewrite the code into a PIE-accelerated version.
- [2026/5/13] Introduced **espdl-quantize skill**, which automatically iterates to search for optimal esp-ppq quantization strategies, reducing manual tuning costs and improving quantized model accuracy. See [how_to_use_espdl_quantize_skill.rst](./docs/en/tutorials/auto_quantization/how_to_use_espdl_quantize_skill.rst).
- [2026/4/23] Support model quantization via AutoQuant tool, which automatically explores optimal quantization strategies and parameters, significantly reducing manual tuning effort. See [how_to_use_AutoQuant.rst](./docs/en/tutorials/auto_quantization/how_to_use_AutoQuant.rst).
- [2026/4/20] Conv and Gemm operators on ESP32-P4 have been upgraded to per-channel quantization, significantly improving model accuracy. **Note**: This requires ESP-PPQ ≥ 1.2.10 and ESP-DL ≥ 3.3.1. The new ESP-DL maintains backward compatibility, and existing per-tensor models are not affected.
- [2026/3/12] [YOLO26](./models/yolo26/) model has been added, thanks to [BoumedineBillal](https://github.com/BoumedineBillal) for the contribution. [Start training and deploying your own YOLO26 model](./examples/tutorial/how_to_quantize_model/quantize_yolo26).
- [2026/3/11] The ESP-DL Agent Tool has been evolved into **espdl-operator skill**, making it easier for AI Agents to automatically implement operators. See [how_to_implement_operators_with_ai_agent.rst](./docs/en/tutorials/how_to_implement_operators_with_ai_agent.rst).
- [2026/2/27] Support quantizing models with TrainedQuantizedThresholdPass. See [quantize_model_with_TQT.rst](./docs/en/tutorials/quantize_model_with_TQT.rst).
- [2026/1/14] Support visualizing `.espdl` models in [netron](https://netron.app/) to facilitate model debugging.

<details>
<summary>Before 2026</summary>

- [2025/11/4] Release [ESP-DL Agent Tool v1.0](./tools/agents/skills/espdl-operator/). This tool is designed for code agent system to assist with implementing and optimizing operators for the ESP-DL framework.
- [2025/10/20] We released ESP-DL v3.2.0 with improved model quantization and deployment capabilities.
- [2025/07/22] Rename the ESP-PPQ package from ppq to esp-ppq to avoid conflicts with ppq, and upgrade all ESP-PPQ dependencies to the latest versions. **Note: In the quantization script, pay attention to modifying the package name.**
- [2025/04/30] We released a new [esp-detection](https://github.com/espressif/esp-detection) project and the `ESPDet-Pico` model, which can easily train and deploy object detection models. [espdet_pico_224_224_cat espdl model](./models/cat_detect/) and [example](./examples/cat_detect/) is a cat detection model trained by esp-detection. Feel free to try it and share your feedback!
- [2025/02/18] We supported yolo11n [espdl model](https://github.com/espressif/esp-dl/tree/master/models/coco_detect) and [example](https://github.com/espressif/esp-dl/tree/master/examples/yolo11_detect).
- [2025/01/09] We updated the schema of espdl model and released ESP-DL v3.1.0. Note: previous models can be load by new schema, but new model is not compatible with previous version.
- [2024/12/20] We released ESP-DL v3.0.0.

</details>

## Getting Started

### Software Requirements

- **ESP-IDF**  

ESP-DL runs based on ESP-IDF. For detailed instructions on how to get ESP-IDF, please see [ESP-IDF Programming Guide](https://idf.espressif.com).

> Please use [ESP-IDF](https://github.com/espressif/esp-idf) `release/v5.3` or above.


- **ESP-PPQ**

ESP-PPQ is a quantization tool based on ppq. If you want to quantize your own model, you can install esp-ppq using the following method. For more installation methods, please refer to the [Getting Started](https://docs.espressif.com/projects/esp-dl/en/latest/getting_started/readme.html#).

```
   pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
   pip install esp-ppq
```

> [!NOTE]
> - The example code installs the Linux PyTorch CPU version. Please install the appropriate PyTorch version based on your actual needs.

### Model Quantization

First, please refer to the [ESP-DL Operator Support State](./operator_support_state.md) to ensure that the operators in your model are already supported.  

ESP-PPQ can directly read ONNX models for quantization. Pytorch and TensorFlow need to be converted to ONNX models first, so make sure your model can be converted to ONNX models. For more details about quantization, please refer to

[how to quantize model](https://docs.espressif.com/projects/esp-dl/en/latest/tutorials/how_to_quantize_model.html)  
[how to quantize MobileNetV2](https://docs.espressif.com/projects/esp-dl/en/latest/tutorials/how_to_deploy_mobilenetv2.html#model-quantization)  
[how to quantize YOLO11n](https://docs.espressif.com/projects/esp-dl/en/latest/tutorials/how_to_deploy_yolo11n.html#model-quantization)  


### Model Deployment
ESP-DL provides a series of API to quickly load and run models.  A typical example is as follows:

[how to load test profile model](https://docs.espressif.com/projects/esp-dl/en/latest/tutorials/how_to_load_test_profile_model.html)  
[how to run model](https://docs.espressif.com/projects/esp-dl/en/latest/tutorials/how_to_run_model.html)  


## ESP-DL Model Zoo

The directory [models](./models/) contains a collection of pre-trained and quantized models optimized for ESP32 series chips. Each model is provided as an ESP-IDF component for easy integration. For more details, please refer to [ESP-DL Model Zoo](./models/README.md).


## Support Operators

If you encounter unsupported operators, please point them out in the [issues](https://github.com/espressif/esp-dl/issues), and we will support them as soon as possible. Contributions to this ESP-DL are also welcomed, please refer to [Creating a New Module (Operator)](https://docs.espressif.com/projects/esp-dl/en/latest/tutorials/how_to_add_a_new_module%28operator%29.html) for more details.

[ESP-DL Operator Support State](./operator_support_state.md)