# dtest

**Repository Path**: Lamdonn/dtest

## Basic Information

- **Project Name**: dtest
- **Description**: dtest (Dynamic Test) - 轻量级动态 C 语言调试框架，一个极度轻便、极具可移植性的纯 C 语言应用层调试与观测框架。它旨在为不方便连接仿真器，或需要进行高频动态观测与控制的嵌入式系统（及 PC 软件）提供远程交互手段，您可以在不修改或重新编译代码的情况下，在系统运行时动态读取/修改内存变量、配置周期信号上报、甚至直接传入地址调用系统内的任意函数（支持传参）。
- **Primary Language**: C/C++
- **License**: GPL-2.0
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 1
- **Forks**: 0
- **Created**: 2026-06-16
- **Last Updated**: 2026-07-06

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# dtest (Dynamic Test) - Lightweight Dynamic C Debugging Framework

`dtest` is an extremely lightweight, highly portable, pure C application-layer debugging and observation framework. It aims to provide a remote interaction method similar to UDS (Unified Diagnostic Services) for embedded systems (and PC software) that are inconvenient to connect to hardware emulators or require high-frequency dynamic observation and control.

Through `dtest`, you can dynamically read/modify memory variables, configure periodic signal reporting, or even directly pass addresses to invoke any function within the system (with arguments) at runtime, **without modifying or recompiling the code**.

---

## 🌟 Core Features

- **Extremely Lightweight & Zero Dependencies**: Pure C99 implementation, independent of any specific hardware platform or underlying library, deployable on any 32-bit or 64-bit system.
- **UDS-Style Protocol Distribution**: Supports standard Request / Positive Response / Negative Response processing structures, with unified error code (NRC) feedback.
- **Dynamic Memory Read/Write (0x22 / 0x2E)**: Fetch and modify arbitrary memory space data based on physical addresses.
- **Periodic Signal Monitoring (0x2A)**: Dynamically set monitored memory addresses and packet offsets at runtime. The component automatically sends variable status periodically, preventing frequent polling.
- **High-Order Dynamic Function Invocation (0x32)**: Provide the function's memory address and its parameters, and the component will forcefully cast and execute it at runtime, **supporting up to 16 arbitrary type (U8-U64/F32-F64) combined parameter passing and result returning**.
- **Background Periodic Tasks (0x33)**: Support registering any function and its runtime parameters into the background to be executed automatically in a loop at a specified cycle (milliseconds).
- **Command Queue Mechanism**: Separation of reception and execution, allowing safe command reception in high-frequency / hardware interrupts, deferred to the main loop for unified processing.
- **Yielding Memory Control**: No `malloc` is used within the component. All struct memory pools are statically allocated externally and injected by the user, completely eliminating memory fragmentation and illegal overflows.

---

## 🏗️ Framework Architecture & Data Flow

The core design philosophy of `dtest` is **"Data Decoupling and Asynchronous Execution"**. Its execution flow is as follows:

```text
[External Comm Interface (UART/CAN/TCP)]
       │
       ▼ (Interrupt / Receive Callback)
 1. dtest_receive()  ───▶ Pushes command into internal lock-free ring buffer (Non-blocking, fast return)
                                   │
                                   ▼
[System Main Loop / RTOS Task]     │ (When business is idle)
       │                           │
       ▼                           │
 2. dtest_task(delta_ms) ◀─────────┘ Pops queue instruction, parses and dispatches 
       │                             (Also handles periodic signal fetch & background tasks)
       ▼
 3. tx_cb()             ◀─── Assembles positive/negative response frames, calls user-injected TX handle
       │
       ▼
[External Comm Interface (Sent back to Host PC)]
```

## 🚀 Quick Integration & Porting Guide

Porting `dtest` is extremely simple. Just complete the underlying RX/TX connection and memory pool injection.

### 1. Prepare Underlying Transmit Function
Implement a data transmission interface that sends a byte stream to external interfaces:
```c
dtest_tx_status_t my_uart_tx(const uint8_t *data, uint16_t len) {
    // If synchronous blocking send
    // uart_send_blocking(data, len);
    // return DTEST_TX_OK;
    
    // If asynchronous send (e.g., triggering DMA controller)
    if (uart_dma_send(data, len) == OK) {
        return DTEST_TX_PENDING;
    }
    return DTEST_TX_ERROR;
}

// For asynchronous sending, a status check interface can be provided
dtest_tx_status_t my_uart_check_tx(void) {
    if (uart_dma_is_busy()) {
        return DTEST_TX_PENDING;
    }
    return DTEST_TX_OK;
}
```

### 2. Initialize dtest
Inject the transmit callback into the component during system startup:
```c
dtest_config_t config = {0};
config.tx_cb = my_uart_tx;

// Mount memory pools here if advanced features like signals and preset functions are needed:
// config.signal_pool = user_signal_array;
// config.max_signals = 16;

dtest_init(&config);
```

### 3. Receive Data Delivery
When the underlying layer (e.g., UART RX interrupt, network RX task) receives data, feed it directly to `dtest`'s receive queue:
```c
void on_uart_receive(const uint8_t *rx_data, uint16_t len) {
    dtest_receive(rx_data, len); 
}
```

### 4. Place Periodic Task
Periodically call `dtest_task` in your main loop (`while (1)`) or RTOS task:
```c
while (1) {
    // Assume the loop runs every 10ms
    dtest_task(10); 
    delay_ms(10);
}
```

### 5. 🚀 Quick Experience (PC Local Full Simulation)
Using the `USE_VIM_COMM` macro in `main.c`, you can directly experience the full interconnection communication between the target device and GUI on your PC:
1. **Compile Target Device Simulator**: 
   ```bash
   make
   ./built/bin/dtest
   ```
   The simulator will enter a waiting state.
2. **Start Visual Host GUI**: 
   ```bash
   python gui/gui.py
   ```
   After the host GUI starts, it will automatically connect to the simulator pipe. You can freely try sending packets, observing signal waveforms, and calling internal functions in the GUI!

---

## 🖥️ Host GUI Tool Usage Guide

After `gui.py` is started, it provides multiple tabs to implement multi-dimensional debugging functions:

### 🌟 Signal Dashboard (Core)
In the dashboard, you can **intuitively and in real-time** observe any memory variable of the target device, with built-in out-of-bounds alarms.
1. **Load CSV Config**: Click `Load CSV` to import `config/signals.csv`. The configuration table defines variable names, types, endianness, factors, and thresholds.
2. **Send Config to Device**: Click `Send Config to Device`, and the GUI will automatically pack and send it to the device via the `0x2A` command. The device will immediately start periodically reporting (`0xAA`) signals.
3. **Automated Tracking**: The UI table automatically parses and extracts true physical values. When a signal exceeds the safe threshold (Min/Max) defined in the CSV, the background turns red, and errors accumulate.
4. **Waveform Plotting**: Check `Auto-save Log`, and the signal stream will be recorded into a local CSV. Click `Plot Log` to use `matplotlib` to generate interactive, real-time multi-channel **line tracking charts**.

### 🔧 Call Function (Dynamic Function Invocation)
Highly magical advanced feature, suitable for forcibly intervening in the system state at runtime:
1. **Input Arguments**: Supports adding up to 16 parameters, and configuring them individually as `IN`, `OUT` (output pointer, fetching return values), and types `U8-F64`.
2. **Prototype Management**: Save commonly used function parameter lists as Prototypes, supporting one-click saving to `function_proto.json` for reuse.
3. **Execution**: Click `Gen Dynamic Call (0x32)` to send the instruction. After the target function finishes execution, all `OUT` parameters will be extracted and echoed in green text in the Console!

### 📦 Memory & PTask
- **Memory (0x22 / 0x2E)**: Physical address-based raw read/write, supporting endianness conversion, suitable for debugging underlying registers.
- **Periodic Tasks (0x33)**: Mount custom calling logic to continuously run in the background of the target device at a fixed cycle (e.g., 1000ms), without requiring the host PC to stay connected.

---

### 📝 GUI Configuration File Format

The core observation drive of the host GUI relies on configuration files, placed in the `config/` directory by default.

#### 1. Periodic Signal Observation Config Table (`signals.csv`)
This is a standard CSV file. You can manually edit it using Excel or a text editor to guide the Dashboard in fetching, parsing, and evaluating target variables.

| Column Name | Description | Example |
| --- | --- | --- |
| **SignalName** | Signal name, serving as a unique identifier for display and filtering. | `g_system_tick` |
| **Group**      | Signal group name, useful for batch filtering and error stats. | `System` |
| **Address**    | Absolute physical address of the variable in target memory (Hex). | `0x20000004` |
| **Size**       | Actual byte size occupied by the variable in memory (Dec). | `4` |
| **Offset**     | Byte offset in the `0xAA` report payload from the target device. | `0` |
| **Endian**     | Data endianness, enter `Little` or `Big`. | `Little` |
| **Type**       | Base data type (`U8`/`I8`/`U16`/`U32`/`F32`/`F64`/`BYTES`, etc.). | `U32` |
| **Factor**     | Physical value conversion factor (`Phys = Raw * Factor + OffsetVal`). | `1.0` |
| **OffsetVal**  | Physical value conversion offset. | `0` |
| **Min** / **Max**| Safe min/max physical thresholds. Dashboard highlights red if exceeded. | `0` / `255` |
| **Unit**       | Physical unit text for display purposes only. | `ms` |

*(Tip: The `function_proto.json` function prototype configuration file is recommended to be generated directly by saving through the GUI, and generally does not require manual editing.)*

---

## 📖 Protocol Specification (UDS Style)

Frame structure consists of `[Service ID] + [Payload...]`.
Positive response header is `[SID + 0x40]`, and negative response is fixed as `[0x7F] [SID] [NRC]`.

> **Note:** The length of the `[Addr]` field adapts to the pointer size of the target system (`sizeof(void*)`). 4 bytes on a 32-bit MCU, 8 bytes on a 64-bit PC. Data is transmitted in Big-Endian.

### 1. Supported Service IDs (SIDs) Overview

| SID (Hex) | Service Name | Function Description |
| :---: | :--- | :--- |
| `0x22` | **Read Memory** | Direct read of MCU memory data of a specified length based on physical address. |
| `0x2E` | **Write Memory** | Overwrite/modify MCU memory data of a specified length based on physical address. |
| `0x2A` | **Config Signal** | Dynamically set memory observation points. The bottom layer automatically captures snapshots and assembles them into `0xAA` packets for periodic return. |
| `0x2B` | **Read Signals** | Instantly request and return the latest memory status snapshot of all (or single) configured observation signals without waiting for the cycle. |
| `0x31` | **Call Static** | Trigger calling a specific debug function pre-registered via `dtest_register_function` using a short alias ID. |
| `0x32` | **Call Dynamic** | Provide target function physical address and parameter characteristics; forcefully cast and inject arguments at runtime, supporting return value and output pointer extraction. |
| `0x33` | **Config PTask** | Mount a calling request for a parameterized function to the background. The bottom layer automatically wakes and executes it periodically. |

> **Passive Report Message (`0xAA`)**: When periodic signal observation is configured via `0x2A` and enabled, the device automatically outputs data frames starting with `0xAA`. This is a one-way periodic stream, not a request/response model.

---

### 2. Basic Data Types (TypeID) & Direction
In dynamic invocation (`0x32`) and background tasks (`0x33`), parameter types and flow directions must be specified. The protocol combines the **direction flag** (high 2 bits) and **base data type** (low 6 bits) into a `TypeID` byte via bitwise OR (`|`).

| Base Data Type | Value | Description |   | Direction | Value | Description |
| :--- | :---: | :--- | :---: | :--- | :---: | :--- |
| `VOID` | `0x00` | Void type (no arg/return) | | `IN` (Default)| `0x00` | Input arg (passed directly by value) |
| `U8` / `I8` | `0x01`/`0x02` | 8-bit integer | | `OUT` | `0x40` | Output arg (passed by ptr, no initial value) |
| `U16` / `I16`| `0x03`/`0x04` | 16-bit integer | | `INOUT` | `0x80` | Bidirectional arg (passed by ptr, with value) |
| `U32` / `I32`| `0x05`/`0x06` | 32-bit integer | | - | - | *(e.g., `OUT U32` = 0x40 \| 0x05 = `0x45`)* |
| `U64` / `I64`| `0x07`/`0x08` | 64-bit integer | | - | - | - |
| `F32` / `F64`| `0x09`/`0x0A` | Float/Double precision | | - | - | - |
| `BYTES` | `0x0B` | Var-length byte array (`uint8_t*`) | | - | - | - |

> **Packet Assembly Key Points:** 
> 1. **Implicit Pointer Conversion**: Parameters marked as `OUT` and `INOUT` are automatically **converted to pointers (addresses)** during the underlying C call.
> 2. **Result Recovery**: After target function execution, the latest results of these output parameters are extracted and appended right after the function's `RetVal` in the positive response message.
> 3. **Var-length Byte Array**: The payload of a `BYTES` parameter must start with a 2-byte length field `[LenH] [LenL]`.
> 4. **Skipping Payload**: Pure `OUT` parameters do not need initial values, so their value payload field can be omitted in the request message (Note: `OUT BYTES` still requires 2 bytes to indicate allocation size).

---

### 3. Detailed Message Structures

#### 🟢 Read Memory (SID: `0x22`)
| Direction | Byte 0 | Byte 1 ~ `PTR_SIZE` | Payload (Big-Endian) |
| --- | --- | --- | --- |
| **Request** | `0x22` | `Addr` (Start physical address to read) | `Size` (2 bytes, number of continuous bytes to read) |
| **Positive Resp**| `0x62` | `Data...` (Actual memory data stream read) | - |

#### 🟢 Write Memory (SID: `0x2E`)
| Direction | Byte 0 | Byte 1 ~ `PTR_SIZE` | Payload (Big-Endian) |
| --- | --- | --- | --- |
| **Request** | `0x2E` | `Addr` (Start physical address to write) | `Size` (2 bytes) + `Data...` (Continuous write stream of Size length) |
| **Positive Resp**| `0x6E` | - | *(No data, indicates success)* |

#### 🟡 Config Signal (SID: `0x2A`)
| Action | Request Format (Byte 0 = `0x2A`) | Positive Resp (`0x6A`) |
| --- | --- | --- |
| **Add node** (`0x01`) | `[0x2A] [0x01] [Addr] [Size(2 bytes)] [Offset(2 bytes)]` | `[0x6A]` |
| **Clear all** (`0x02`) | `[0x2A] [0x02]` | `[0x6A]` |
| **Start/Stop cycle** (`0x03`)| `[0x2A] [0x03] [CycleMs(2 bytes)]` *(Note: No CycleMs means stop)* | `[0x6A]` |
| **Enable Dyn Length** (`0x04`) | `[0x2A] [0x04] [Enable(1 byte)]` *(1:Dynamic truncation, 0:Fixed length)* | `[0x6A]` |

#### 🟡 Read Signals (SID: `0x2B`)
| Direction | Byte 0 | Byte 1 (Optional) | Payload |
| --- | --- | --- | --- |
| **Request** | `0x2B` | `SigID` (Target pool index. If omitted, reads all) | - |
| **Positive Resp**| `0x6B` | `Payload...` (Assembled signal data snapshot, format identical to `0xAA` auto message)| - |

#### 🟣 Call Static (SID: `0x31`)
| Direction | Byte 0 | Byte 1 | Payload |
| --- | --- | --- | --- |
| **Request** | `0x31` | `FuncID` (Pre-registered short alias) | `[Arg1_Val] [Arg2_Val]...` (Argument data stream) |
| **Positive Resp**| `0x71` | `RetVal...` (Function's direct return value) | `[OutArg_Val]...` (Values of all OUT parameters extracted) |

#### 🟣 Call Dynamic (SID: `0x32`)
| Direction | Byte 0 | Byte 1 ~ `PTR_SIZE` | Byte (PTR+1) | Byte (PTR+2) | Payload |
| --- | --- | --- | --- | --- | --- |
| **Request** | `0x32` | `FuncAddr` (Physical address)| `RetTypeID` | `ArgCnt` | `{ [ArgX_TypeID] [ArgX_Val] }...` |
| **Positive Resp**| `0x72` | `RetVal...` (Function's direct return value) | `[OutArg_Val]...` (Values of all OUT parameters extracted)| - | - |

#### 🟤 Config PTask (SID: `0x33`)
| Action | Request Format (Byte 0 = `0x33`) | Positive Resp (`0x73`) |
| --- | --- | --- |
| **Add/Update** (`0x01`)| `[0x33] [0x01] [FuncAddr] [CycleMs(2B)] [RetTypeID] [ArgCnt] { [Arg_Type] [Arg_Val] }...` | `[0x73]` |
| **Delete single** (`0x02`) | `[0x33] [0x02] [FuncAddr]` | `[0x73]` |
| **Clear all** (`0x03`) | `[0x33] [0x03]` | `[0x73]` |

---

### 4. Negative Response Codes (NRC) Dictionary
When a request format is invalid or conditions are unmet, the device rejects execution and returns `[0x7F] [Requested SID] [NRC]`.
| NRC  | Name | Trigger Scenario |
| ---  | --- | --- |
| `0x10` | General Reject | General rejection (unknown error or transmission locked). |
| `0x12` | SubFunction Not Supported | Unsupported action code, or feature disabled due to zero capacity in configuration pool. |
| `0x13` | Incorrect Message Length | Message truncated, or payload size does not match declared `TypeID` size. |
| `0x31` | Out Of Range | Index out of bounds, read/write size exceeds limit, or signal/task pool is full. |
| `0x33` | Memory Overlap | (Specific) Occurs during `0x2A`: Specified `Offset + Size` exceeds single frame max length. |

## 💻 Quick Experience (PC Simulator Terminal)

The component includes an interactive test terminal `main.c`. You can use it to simulate and test the entire communication and control process in a local PC environment (Windows or Linux).

**Compile & Run**:
```bash
make
./test_dtest
```

## 📊 Resource Footprint & Performance Evaluation

`dtest` is designed for resource-constrained microcontrollers (MCUs), utilizing a **zero dynamic memory allocation** (Zero `malloc`) strategy.

### 1. ROM (Flash) Footprint
- **Minimalist Code**: Core source `dtest.c` is only ~1K lines, zero dependencies (only `<stdint.h>`, `<stdbool.h>`, `<string.h>`).
- **Extremely Low Footprint**: Under GCC `-O2`, full feature binary size is typically between **2KB ~ 4KB**.

### 2. RAM (SRAM) Footprint
RAM footprint is completely controlled by the user. All struct memories are statically allocated externally and injected, with no hidden overhead.
- **Base Context (`dtest_ctx_t`)**: Consists of `tx_buffer` (default 256 bytes) and `cmd_queue` (default 256 bytes × depth). Base footprint ~**550 bytes**.
- **Feature Pool Overhead (Enable as needed)**:
  - **Signal Pool**: ~`16` bytes per slot.
  - **Static Function Pool**: ~`32` bytes per slot.
  - **Periodic Task Pool**: ~`288` bytes per slot (requires backing up all arguments for background execution).

---

## 🔀 RTOS & Bare-metal Deployment Guide

The `dtest` architecture naturally supports the "Single-Producer Single-Consumer" model, ideal for complex interrupt and RTOS environments.

### 1. Receive End (ISR Safe)
`dtest_receive()` uses a lock-free Ring Buffer design with no blocking or loops (just a fast `memcpy`).
- **Deployment**: Highly recommended to place directly in hardware RX ISRs (UART/CAN) or high-priority network RX threads.
- **Concurrency Warning**: If multiple concurrent data sources feed `dtest_receive()`, use a mutex or disable interrupts (`DISABLE_INT()`) to prevent pointer race conditions.

### 2. Execution End (Low Priority Safe)
`dtest_task(delta_ms)` is the brain. Due to its unpredictable execution time (depends on the invoked function), it must **NEVER** run in an interrupt context.
- **Bare-metal**: Inside the `while(1)` super-loop in `main()`.
- **RTOS**: In a low-priority background thread (like IDLE or Debug task) with `osDelay()`.

### 3. Transmit End (Async DMA Supported)
Injected `tx_cb` can be synchronous or asynchronous.
- If using DMA, `tx_cb` returns `DTEST_TX_PENDING`. The state machine automatically locks the internal TX buffer until `tx_check_cb` returns `DTEST_TX_OK`, preventing overwrite trampling.

---

## 🛡️ Security & Production Environment Warning

⚠️ **Extremely Dangerous God Mode**

`dtest` grants external interfaces unlimited access to read/write all physical memory and execute internal functions. Leaving it unrestricted in production poses catastrophic security risks.

### Recommended Protections:
1. **Conditional Compilation**: Completely strip it out in Release builds via macros (`#ifndef NDEBUG`).
2. **Seed & Key Authentication (UDS 0x27)**: Implement a handshake layer in the underlying serial callback. Only forward data to `dtest_receive()` after successful authentication.
3. **Memory Barrier Limitation**: For MPU/MMU systems, restrict the execution permissions of the `dtest_task` thread to prevent accidental writes to protected regions (e.g., Bootloader sectors).

---

## ⚙️ Advanced Macro Configuration & Customization

In `core/dtest.h`, you can fine-tune memory limits based on actual needs:

```c
/* --- Core Configuration --- */
// [Core] Max size of internal RX/TX buffers (bytes). Recommended >=128.
#define DTEST_FRAME_MAX_LEN         256  

// [Signal] Max payload size for periodic snapshot packet (0xAA). Cannot exceed DTEST_FRAME_MAX_LEN.
#define DTEST_SIGNAL_FRAME_LEN      64   

// [Queue] Async receive command queue depth. Increase to resist burst traffic if task scheduling is slow.
#define DTEST_CMD_QUEUE_SIZE        1    
```