# langgraph_bench

**Repository Path**: chenxuqiang2022/langgraph_bench

## Basic Information

- **Project Name**: langgraph_bench
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2026-05-29
- **Last Updated**: 2026-05-29

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# LangGraph Deep Agent Benchmark Tool

A CLI-based performance benchmarking tool for LangGraph Deep Agent. Measures latency, throughput, concurrency, and resource usage with real LLM calls.

## Features

- **Multiple benchmark modes**: Sequential, concurrent, and ramp-up (scalability testing)
- **Real LLM integration**: Supports OpenAI-compatible and Anthropic APIs
- **Comprehensive metrics**: Latency (P50/P95/P99), throughput, CPU/memory usage, tool performance
- **6 built-in tools**: Calculator, web search, file reader, code executor, data analyzer, text summarizer
- **Flexible output**: Terminal (rich formatting), JSON, and CSV reports

## Installation

```bash
pip install -e ".[dev]"
```

## Quick Start

```bash
# Set your API key
export OPENAI_API_KEY="your-key-here"

# Run default benchmark
python main.py

# Run with custom config
python main.py --config configs/stress.yaml

# Override parameters
python main.py --mode concurrent --concurrency 20 --total-requests 200
```

## Benchmark Modes

### Sequential
Execute requests one-by-one for baseline latency measurement.

```bash
python main.py --mode sequential --total-requests 10
```

### Concurrent
Fixed concurrency level to measure steady-state throughput.

```bash
python main.py --mode concurrent --concurrency 10 --total-requests 100
```

### Ramp-Up
Step-wise concurrency increase (1->5->10->25->50) to test scalability.

```bash
python main.py --mode ramp_up
```

## Configuration

Edit `configs/default.yaml` to customize:

- **LLM**: Provider, model, temperature, base_url
- **Agent**: Tool selection, max iterations
- **Benchmark**: Mode, concurrency, request count
- **Prompts**: Custom test prompts
- **Output**: Formats (terminal/json/csv), output directory

## CLI Options

```bash
python main.py --help
```

```
--config PATH          Config file path (default: configs/default.yaml)
--mode MODE            Benchmark mode (sequential|concurrent|ramp_up)
--concurrency N        Concurrency level override
--total-requests N     Total request count
--provider PROVIDER    LLM provider (openai|anthropic)
--model MODEL          Model name
--output-dir DIR       Results output directory
```

## Metrics

| Metric | Description |
|--------|-------------|
| Latency P50/P95/P99 | Percentile response times (ms) |
| Throughput | Requests per second (RPS) |
| CPU Usage | Average and peak CPU utilization |
| Memory Usage | Average and peak RSS memory |
| Token Usage | Input/output token counts |
| Tool Metrics | Per-tool invocation count and latency |

## Output

Results are saved to `./results/` by default:

- **Terminal**: Real-time progress and summary table
- **JSON**: Full metrics with per-request details
- **CSV**: Flattened summary for spreadsheet analysis

## Testing

```bash
pytest tests/ -v
```

## Architecture

```
src/
├── agent/
│   ├── deep_agent.py     # LangGraph ReAct agent
│   └── tools.py          # Built-in tools
├── bench/
│   ├── config.py         # Configuration system
│   ├── metrics.py        # Metrics collection
│   ├── runner.py         # Benchmark execution
│   └── reporter.py       # Result output
└── llm/
    └── providers.py      # LLM adapters
```

## License

MIT