# MNN4NER

**Repository Path**: arcanu/mnn4ner

## Basic Information

- **Project Name**: MNN4NER
- **Description**: MNN是一个轻量级的深度神经网络引擎，支持深度学习的推理与训练。增加ner endpoint  实现在手机 cpu 上进行快速ner 任务
- **Primary Language**: C
- **License**: Apache-2.0
- **Default Branch**: master
- **Homepage**: https://github.com/alibaba/MNN
- **GVP Project**: No

## Statistics

- **Stars**: 1
- **Forks**: 0
- **Created**: 2026-05-21
- **Last Updated**: 2026-06-02

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# MNN-ner
基于MNN 运行微调后NER 模型 ,增加/chat/extract 用来获取ner 信息

MNN(https://github.com/alibaba/MNN/)是一个轻量级的深度神经网络引擎，支持深度学习的推理与训练。适用于服务器/个人电脑/手机/嵌入式各类设备。目前，MNN已经在阿里巴巴的手机淘宝、手机天猫、优酷等30多个App中使用，覆盖直播、短视频、搜索推荐、商品图像搜索、互动营销、权益发放、安全风控等场景。

## 编译
```
apt  install -y cmake libssl-dev  
./build.sh
```

## 模型配置
```
{
    "llm_model": "llm.mnn",
    "llm_weight": "llm.mnn.weight",

    "backend_type": "cpu",
    "thread_num": 4,
    "precision": "low",
    "memory": "low",

    "max_new_tokens": 384,
    "reuse_kv": false,

    "sampler_type": "greedy",
    "temperature": 0.0,
    "top_k": 1,
    "top_p": 1.0,
    "min_p": 0.05,
    "tfs_z": 1.0,
    "typical": 0.95,
    "repetition_penalty": 1.0,
    "presence_penalty": 0.0,
    "frequency_penalty": 0.0,
    "penalty_window": 0,
    "n_gram": 8,
    "ngram_factor": 1.0,
    "tokenizer_file": "tokenizer.mtok",
    "enable_thinking": false
}

```
## 后台脚本
```
# 先杀掉旧进程（假设旧 mls 进程存在）
pkill mls

# 实例 1 绑定 NUMA 0 (核 0-23)
nohup numactl --cpunodebind=0 --membind=0 \
    ./mls serve -c /mnt/disk1/mnn/ner-model/config.json -p 30080 > mls1.log 2>&1 &

# 实例 2 绑定 NUMA 1 (核 24-47)
nohup numactl --cpunodebind=1 --membind=1 \
    ./mls serve -c /mnt/disk1/mnn/ner-model/config.json -p 30081 > mls2.log 2>&1 &

# 实例 3 绑定 NUMA 2 (核 48-71)
nohup numactl --cpunodebind=2 --membind=2 \
    ./mls serve -c /mnt/disk1/mnn/ner-model/config.json -p 30082 > mls3.log 2>&1 &

# 实例 4 绑定 NUMA 3 (核 72-95)
nohup numactl --cpunodebind=3 --membind=3 \
    ./mls serve -c /mnt/disk1/mnn/ner-model/config.json -p 30083 > mls4.log 2>&1 &
```

## nginx 配置
```
upstream chat_backend {
    least_conn;
    keepalive 32;
    keepalive_timeout 60s;
    keepalive_requests 1000;
    server 127.0.0.1:30080 max_fails=3 fail_timeout=30s;
    server 127.0.0.1:30081 max_fails=3 fail_timeout=30s;
    server 127.0.0.1:30082 max_fails=3 fail_timeout=30s;
    server 127.0.0.1:30083 max_fails=3 fail_timeout=30s;   # 新增
}

server {
    listen 8090;
    server_name _;
    
    client_max_body_size 100M;  
    
    location /chat/ {
        proxy_pass http://chat_backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        proxy_connect_timeout 60s;
        proxy_send_timeout 300s;    
        proxy_read_timeout 300s;   
        
        proxy_buffering off;
    }
}
```

## 测试demo 示范
```
curl -N -X POST http://10.22.16.74:8090/chat/extract \
  -H "Content-Type: application/json" \
  -d '{
    "message": "李女士反映：长春市宽城区北人民大街的德克士门店，炸鸡翅未炸熟带血丝。"
  }'

{"address":"长春市宽城区北人民大街"，"entities":["德克士"]，"products":["炸鸡"],"subject":"李女士"}
```

## 模型
基于qwen-0.6B 进行ner 任务微调