# hadoop-docker

**Repository Path**: zhoulm-data/hadoop-docker

## Basic Information

- **Project Name**: hadoop-docker
- **Description**: 基于Hadoop数仓建设Docker环境搭建
- **Primary Language**: Unknown
- **License**: Apache-2.0
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 2
- **Forks**: 0
- **Created**: 2024-07-10
- **Last Updated**: 2026-03-11

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# 部署应用规划

![](imgs/湖仓架构.png)

## jdk-8u212

## hadoop-3.3.4

```shell
## 构建 hadoop 集群镜像
docker build -t zhoulm-dev/hadoop:3.3.4 . --no-cache
## 启动 hadoop 高可用集群服务
docker-compose -f hadoop.yml up -d

## 构建 hadoop 高可用集群镜像
docker build -t zhoulm-dev/hadoop-ha:3.3.4 . --no-cache
## 启动 hadoop 高可用集群服务
docker-compose -f hadoop-ha.yml up -d
```

## mysql-5.7

```shell
## 构建 mysql 镜像
docker build -t zhoulm-dev/mysql:5.7 . --no-cache
```

## hive-3.1.3
```shell
## 构建 hadoop-hive 集群镜像
docker build -t zhoulm-dev/hadoop-hive:3.3.4-3.1.3 . --no-cache
## 启动 hadoop-hive 集群服务
docker-compose -f hadoop-hive.yml up -d
```

## flink-1.18_2.12

- paimon-1.18-0.8.2
- hudi-1.18-0.15.0
- iceberg-1.18-1.5.2

```shell
## 构建 flink 集群镜像
docker build -t zhoulm-dev/flink:1.18 . --no-cache
## 启动 flink 集群服务
docker-compose -f hadoop-flink.yml up -d
```

## dinky-1.18-1.0.3

```shell
## 构建 flink-dinky 集群镜像
docker build -t zhoulm-dev/flink-dinky:1.18-1.0.3 . --no-cache
## 启动 flink-dinky 集群服务
docker-compose -f flink-dinky.yml up -d
```

## streampark-2.12-2.1.1

```shell
## 构建 flink-streampark 集群镜像
docker build -t zhoulm-dev/flink-streampark:1.18-2.1.1 . --no-cache
## 启动 flink-streampark 集群服务
docker-compose -f flink-streampark.yml up -d
```

## kafka

```shell
## 启动 kafka 服务
docker-compose -f docker-kafka.yaml up -d
```




# 快速部署

```shell
# 上传镜像

docker tag fc4a4847360c registry.cn-hangzhou.aliyuncs.com/zhoulm-dev/mysql:5.7
docker tag 05a764732be0 registry.cn-hangzhou.aliyuncs.com/zhoulm-dev/hadoop:3.3.4
docker tag c24e098768d8 registry.cn-hangzhou.aliyuncs.com/zhoulm-dev/hadoop-hive:3.3.4-3.1.3
docker tag 8a6c1c323b4b registry.cn-hangzhou.aliyuncs.com/zhoulm-dev/hadoop-hive-flink:3.3.4-3.1.3-1.7
docker tag cf02cb4b6d58 registry.cn-hangzhou.aliyuncs.com/zhoulm-dev/hadoop-hive-flink-dinky:3.3.4-3.1.3-1.7-1.0.3

docker push registry.cn-hangzhou.aliyuncs.com/zhoulm-dev/mysql:5.7
docker push registry.cn-hangzhou.aliyuncs.com/zhoulm-dev/hadoop:3.3.4
docker push registry.cn-hangzhou.aliyuncs.com/zhoulm-dev/hadoop-hive:3.3.4-3.1.3
docker push registry.cn-hangzhou.aliyuncs.com/zhoulm-dev/hadoop-hive-flink:3.3.4-3.1.3-1.7
docker push registry.cn-hangzhou.aliyuncs.com/zhoulm-dev/hadoop-hive-flink-dinky:3.3.4-3.1.3-1.7-1.0.3

docker pull registry.cn-hangzhou.aliyuncs.com/zhoulm-dev/mysql:5.7
docker pull registry.cn-hangzhou.aliyuncs.com/zhoulm-dev/hadoop:3.3.4
docker pull registry.cn-hangzhou.aliyuncs.com/zhoulm-dev/hadoop-hive:3.3.4-3.1.3
docker pull registry.cn-hangzhou.aliyuncs.com/zhoulm-dev/hadoop-hive-flink:3.3.4-3.1.3-1.7
docker pull registry.cn-hangzhou.aliyuncs.com/zhoulm-dev/hadoop-hive-flink-dinky:3.3.4-3.1.3-1.7-1.0.3

docker tag fc4a4847360c zhoulm-dev/mysql:5.7
docker tag 05a764732be0 zhoulm-dev/hadoop:3.3.4
docker tag c24e098768d8 zhoulm-dev/hadoop-hive:3.3.4-3.1.3
docker tag 8a6c1c323b4b zhoulm-dev/hadoop-hive-flink:3.3.4-3.1.3-1.7
docker tag cf02cb4b6d58 zhoulm-dev/hadoop-hive-flink-dinky:3.3.4-3.1.3-1.7-1.0.3

```

docker cp jobmanager:/docker-entrypoint.sh .

# 参考资料

[hadoop环境搭建](https://mp.weixin.qq.com/s?__biz=MzI3MDM5NjgwNg==&mid=2247487221&idx=1&sn=4da6d793e24018a3727b71b10a85d2f1&chksm=ead0f21cdda77b0a2bd52d2cbdfb2659a12e0b948306cd56293df4ac6f0c1887a6da16592bf4&cur_album_id=2725790692995776513&scene=189#wechat_redirect)


[hive环境搭建](https://mp.weixin.qq.com/s?__biz=MzI3MDM5NjgwNg==&mid=2247487266&idx=1&sn=adf1e759986020d5fdde1376a24a7e0a&chksm=ead0f3cbdda77add2ebbff2007e44eb9bc357dcb54c778b2d50b3c02961a958315978dc4ae72&scene=178&cur_album_id=2632331943749107715#rd)


[flink环境搭建](https://mp.weixin.qq.com/s?__biz=MzI3MDM5NjgwNg==&mid=2247490819&idx=1&sn=677df7eae9c90e14272f388e84b63bfe&chksm=ead0e1eadda768fc7cf04336a19be3d92b2152e0d1815e939d79092500556fe631eeae5bd098&scene=178&cur_album_id=2632331943749107715#rd)