# detect_tensorrt
**Repository Path**: linClubs/detect_tensorrt
## Basic Information
- **Project Name**: detect_tensorrt
- **Description**: 使用tensorRT加速yolov5-v7版本，可以加速检测。也可以加速分割，采用FP32格式yolov5n在3060TI上1ms(应该是小于1ms)。yolov5s在3060TI上3ms。 yolov5l为14ms; yolov5n-seg在3060上2ms左右, master版本直接cmake编译，不涉及ros
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No
## Statistics
- **Stars**: 0
- **Forks**: 1
- **Created**: 2023-03-02
- **Last Updated**: 2023-08-25
## Categories & Tags
**Categories**: Uncategorized
**Tags**: None
## README
# YOLOv5-TenorRT
# 1 env
+ ubuntu 20.04
+ cuda换成11.3.0
+ cudnn8.6.0
+ TensorRT-8.2.1.8
+ FP16，FP32精度通过，INT8运行未通过
# 2 编译和运行
## 2.1 下载
1. 先将下载yolov5和tensorrtx.git对应的v7.0版本，版本一定要对应上
~~~python
git clone -b v7.0 https://github.com/ultralytics/yolov5.git
git clone -b yolov5-v7.0 https://github.com/wang-xinyu/tensorrtx.git
~~~
2. yolov5-v7.0的环境就不介绍了。
3. tensorrtx编译
只留下tensorrtx中yolov5文件夹，并重命名为tensorrtx，放到原版yolov5同级目录，不移动也行
~~~c
cd tensorrtx
mkdir build
cd build
cmake ..
make -j8
~~~
## 2.2 生成yolov5s.wts
将tensorrtx中`gen_wts.py`文件拷贝到yolov5工程中
并下载yolov5s.pt文件，运行gen_wts.py生成yolov5s.wts
+ 注意权重要与版本对应`v7.0/yolov5s.pt`
+ 分割就下载`v7.0/yolov5s-seg.pt`
~~~c
# 检测
wget https://github.com/ultralytics/yolov5/releases/download/v7.0/yolov5s.pt
# 分割
wget https://github.com/ultralytics/yolov5/releases/download/v7.0/yolov5s.pt
# 生成wts文件
python gen_wts.py -w yolov5s.pt -o yolov5s.wts -t detect
python gen_wts.py -w yolov5s-seg.pt -o yolov5s-seg.wts -t seg
~~~
## 2.3 生成engine
将2.2生成的wts拷贝到../tensorrt/weights/下,运行
会在build下生成yolov5n.engine文件，最后一个参数就是用什么模型就输什么参数，n,s,l,x等
这里用的yolov5n模型，最后一个参数为n
~~~
./yolov5_det -s ../weights/yolov5n.wts yolov5n.engine n
~~~
## 2.4 测试det
~~~
./yolov5_det -d yolov5n.engine ../image_file
~~~
## 2.5 提速FP16与INT8
修改include/config.h中 #define USE_FP32参数
# 3 修改的参数，只用于检测
---
Detection
1. generate .wts from pytorch with .pt, or download .wts from model zoo
```
git clone -b v7.0 https://github.com/ultralytics/yolov5.git
git clone -b yolov5-v7.0 https://github.com/wang-xinyu/tensorrtx.git
cd yolov5/
wget https://github.com/ultralytics/yolov5/releases/download/v7.0/yolov5s.pt
cp [PATH-TO-TENSORRTX]/yolov5/gen_wts.py .
python gen_wts.py -w yolov5s.pt -o yolov5s.wts
# A file 'yolov5s.wts' will be generated.
```
2. build tensorrtx/yolov5 and run
```
cd [PATH-TO-TENSORRTX]/yolov5/
# Update kNumClass in src/config.h if your model is trained on custom dataset
mkdir build
cd build
cp [PATH-TO-ultralytics-yolov5]/yolov5s.wts .
cmake ..
make
./yolov5_det -s [.wts] [.engine] [n/s/m/l/x/n6/s6/m6/l6/x6 or c/c6 gd gw] // serialize model to plan file
./yolov5_det -d [.engine] [image folder] // deserialize and run inference, the images in [image folder] will be processed.
# For example yolov5s
./yolov5_det -s yolov5s.wts yolov5s.engine s
./yolov5_det -d yolov5s.engine ../images
# For example Custom model with depth_multiple=0.17, width_multiple=0.25 in yolov5.yaml
./yolov5_det -s yolov5_custom.wts yolov5.engine c 0.17 0.25
./yolov5_det -d yolov5.engine ../images
```
3. Check the images generated, _zidane.jpg and _bus.jpg
4. Optional, load and run the tensorrt model in Python
```
// Install python-tensorrt, pycuda, etc.
// Ensure the yolov5s.engine and libmyplugins.so have been built
python yolov5_det_trt.py
// Another version of python script, which is using CUDA Python instead of pycuda.
python yolov5_det_trt_cuda_python.py
```
### Classification
```
# Download ImageNet labels
wget https://github.com/joannzhang00/ImageNet-dataset-classes-labels/blob/main/imagenet_classes.txt
# Build and serialize TensorRT engine
./yolov5_cls -s yolov5s-cls.wts yolov5s-cls.engine s
# Run inference
./yolov5_cls -d yolov5s-cls.engine ../images
```
### Instance Segmentation
```
# Build and serialize TensorRT engine
./yolov5_seg -s yolov5s-seg.wts yolov5s-seg.engine s
# Download the labels file
wget -O coco.txt https://raw.githubusercontent.com/amikelive/coco-labels/master/coco-labels-2014_2017.txt
# Run inference with labels file
./yolov5_seg -d yolov5s-seg.engine ../images coco.txt
```
# INT8 Quantization
1. Prepare calibration images, you can randomly select 1000s images from your train set. For coco, you can also download my calibration images `coco_calib` from [GoogleDrive](https://drive.google.com/drive/folders/1s7jE9DtOngZMzJC1uL307J2MiaGwdRSI?usp=sharing) or [BaiduPan](https://pan.baidu.com/s/1GOm_-JobpyLMAqZWCDUhKg) pwd: a9wh
2. unzip it in yolov5/build
3. set the macro `USE_INT8` in src/config.h and make
4. serialize the model and test
## More Information
See the readme in [home page.](https://github.com/wang-xinyu/tensorrtx)