跳到主要内容

etcd 集群

服务器IP主机角色
192.168.10.31etcd Node01
192.168.10.32etcd Node02
192.168.10.33etcd Node03

etcd Node01​

  1. 下载 etcd 安装包

    wget https://pdpublic.mingdao.com/private-deployment/offline/common/etcd-v3.6.11-linux-amd64.tar.gz

  2. 创建 etcd 运行用户

    useradd -M -s /sbin/nologin etcd

  3. 解压 etcd 安装包并安装二进制

    mkdir -p /usr/local/etcd/bin
    tar -zxf etcd-v3.6.11-linux-amd64.tar.gz
    cp etcd-v3.6.11-linux-amd64/etcd /usr/local/etcd/bin/
    cp etcd-v3.6.11-linux-amd64/etcdctl /usr/local/etcd/bin/
    cp etcd-v3.6.11-linux-amd64/etcdutl /usr/local/etcd/bin/
    chmod +x /usr/local/etcd/bin/etcd*
    chown -R etcd:etcd /usr/local/etcd

  4. 创建数据目录和日志目录

    mkdir -p /data/etcd
    mkdir -p /data/logs/etcd
    chown -R etcd:etcd /data/etcd /data/logs/etcd

  5. 修改 etcd 配置文件

    cat > /usr/local/etcd/etcd.conf.yml <<'EOF'
    name: etcd-01
    data-dir: /data/etcd
    listen-client-urls: http://192.168.10.31:2379,http://127.0.0.1:2379
    advertise-client-urls: http://192.168.10.31:2379
    listen-peer-urls: http://192.168.10.31:2380
    initial-advertise-peer-urls: http://192.168.10.31:2380
    listen-metrics-urls: http://192.168.10.31:2381,http://127.0.0.1:2381
    initial-cluster: etcd-01=http://192.168.10.31:2380,etcd-02=http://192.168.10.32:2380,etcd-03=http://192.168.10.33:2380
    initial-cluster-token: hap-milvus-etcd
    initial-cluster-state: new
    auto-compaction-mode: periodic
    auto-compaction-retention: 10h
    quota-backend-bytes: 8589934592
    snapshot-count: 100000
    max-snapshots: 5
    max-wals: 5
    strict-reconfig-check: true
    pre-vote: true
    enable-v2: false
    metrics: basic
    logger: zap
    log-level: info
    log-outputs:
      - /data/logs/etcd/etcd.log
    enable-log-rotation: true
    log-rotation-config-json: '{"maxsize":100,"maxage":180,"maxbackups":0,"localtime":true,"compress":true}'
    EOF
    • listen-client-urls 和 advertise-client-urls 中的 IP 请替换为实际部署服务器的 IP。

  6. 配置 systemd 管理 etcd-01

    cat > /etc/systemd/system/etcd-01.service <<'EOF'
    [Unit]
    Description=etcd member etcd-01
    Documentation=https://etcd.io/docs/
    After=network-online.target
    Wants=network-online.target
    [Service]
    Type=simple
    User=etcd
    Group=etcd
    ExecStart=/usr/local/etcd/bin/etcd --config-file=/usr/local/etcd/etcd.conf.yml
    Restart=on-failure
    RestartSec=5s
    LimitNOFILE=65536
    LimitNPROC=65536
    OOMScoreAdjust=-999
    [Install]
    WantedBy=multi-user.target
    EOF

  7. 启动 etcd-01 并加入开机自启动

    systemctl daemon-reload
    systemctl start etcd-01
    systemctl enable etcd-01

  8. 服务状态检查

    systemctl status etcd-01 --no-pager

  9. 查看运行日志

    journalctl -u etcd-01 -n 100 -l --no-pager

etcd Node02​

  1. 下载 etcd 安装包

    wget https://pdpublic.mingdao.com/private-deployment/offline/common/etcd-v3.6.11-linux-amd64.tar.gz

  2. 创建 etcd 运行用户

    useradd -M -s /sbin/nologin etcd

  3. 解压 etcd 安装包并安装二进制

    mkdir -p /usr/local/etcd/bin
    tar -zxf etcd-v3.6.11-linux-amd64.tar.gz
    cp etcd-v3.6.11-linux-amd64/etcd /usr/local/etcd/bin/
    cp etcd-v3.6.11-linux-amd64/etcdctl /usr/local/etcd/bin/
    cp etcd-v3.6.11-linux-amd64/etcdutl /usr/local/etcd/bin/
    chmod +x /usr/local/etcd/bin/etcd*
    chown -R etcd:etcd /usr/local/etcd
    • 安装验证
    /usr/local/etcd/bin/etcd --version
    /usr/local/etcd/bin/etcdctl version
    /usr/local/etcd/bin/etcdutl version

  4. 创建数据目录和日志目录

    mkdir -p /data/etcd
    mkdir -p /data/logs/etcd
    chown -R etcd:etcd /data/etcd /data/logs/etcd

  5. 修改 etcd 配置文件

    cat > /usr/local/etcd/etcd.conf.yml <<'EOF'
    name: etcd-02
    data-dir: /data/etcd
    listen-client-urls: http://192.168.10.32:2379,http://127.0.0.1:2379
    advertise-client-urls: http://192.168.10.32:2379
    listen-peer-urls: http://192.168.10.32:2380
    initial-advertise-peer-urls: http://192.168.10.32:2380
    listen-metrics-urls: http://192.168.10.32:2381,http://127.0.0.1:2381
    initial-cluster: etcd-01=http://192.168.10.31:2380,etcd-02=http://192.168.10.32:2380,etcd-03=http://192.168.10.33:2380
    initial-cluster-token: hap-milvus-etcd
    initial-cluster-state: new
    auto-compaction-mode: periodic
    auto-compaction-retention: 10h
    quota-backend-bytes: 8589934592
    snapshot-count: 100000
    max-snapshots: 5
    max-wals: 5
    strict-reconfig-check: true
    pre-vote: true
    enable-v2: false
    metrics: basic
    logger: zap
    log-level: info
    log-outputs:
      - /data/logs/etcd/etcd.log
    enable-log-rotation: true
    log-rotation-config-json: '{"maxsize":100,"maxage":180,"maxbackups":0,"localtime":true,"compress":true}'
    EOF
    • listen-client-urls 和 advertise-client-urls 中的 IP 请替换为实际部署服务器的 IP。

  6. 配置 systemd 管理 etcd-02

    cat > /etc/systemd/system/etcd-02.service <<'EOF'
    [Unit]
    Description=etcd member etcd-02
    Documentation=https://etcd.io/docs/
    After=network-online.target
    Wants=network-online.target
    [Service]
    Type=simple
    User=etcd
    Group=etcd
    ExecStart=/usr/local/etcd/bin/etcd --config-file=/usr/local/etcd/etcd.conf.yml
    Restart=on-failure
    RestartSec=5s
    LimitNOFILE=65536
    LimitNPROC=65536
    OOMScoreAdjust=-999
    [Install]
    WantedBy=multi-user.target
    EOF

  7. 启动 etcd-02 并加入开机自启动

    systemctl daemon-reload
    systemctl start etcd-02
    systemctl enable etcd-02

  8. 服务状态检查

    systemctl status etcd-02 --no-pager

  9. 查看运行日志

    journalctl -u etcd-02 -n 100 -l --no-pager

etcd Node03​

  1. 下载 etcd 安装包

    wget https://pdpublic.mingdao.com/private-deployment/offline/common/etcd-v3.6.11-linux-amd64.tar.gz

  2. 创建 etcd 运行用户

    useradd -M -s /sbin/nologin etcd

  3. 解压 etcd 安装包并安装二进制

    mkdir -p /usr/local/etcd/bin
    tar -zxf etcd-v3.6.11-linux-amd64.tar.gz
    cp etcd-v3.6.11-linux-amd64/etcd /usr/local/etcd/bin/
    cp etcd-v3.6.11-linux-amd64/etcdctl /usr/local/etcd/bin/
    cp etcd-v3.6.11-linux-amd64/etcdutl /usr/local/etcd/bin/
    chmod +x /usr/local/etcd/bin/etcd*
    chown -R etcd:etcd /usr/local/etcd
    • 安装验证
    /usr/local/etcd/bin/etcd --version
    /usr/local/etcd/bin/etcdctl version
    /usr/local/etcd/bin/etcdutl version

  4. 创建数据目录和日志目录

    mkdir -p /data/etcd
    mkdir -p /data/logs/etcd
    chown -R etcd:etcd /data/etcd /data/logs/etcd

  5. 修改 etcd 配置文件

    cat > /usr/local/etcd/etcd.conf.yml <<'EOF'
    name: etcd-03
    data-dir: /data/etcd
    listen-client-urls: http://192.168.10.33:2379,http://127.0.0.1:2379
    advertise-client-urls: http://192.168.10.33:2379
    listen-peer-urls: http://192.168.10.33:2380
    initial-advertise-peer-urls: http://192.168.10.33:2380
    listen-metrics-urls: http://192.168.10.33:2381,http://127.0.0.1:2381
    initial-cluster: etcd-01=http://192.168.10.31:2380,etcd-02=http://192.168.10.32:2380,etcd-03=http://192.168.10.33:2380
    initial-cluster-token: hap-milvus-etcd
    initial-cluster-state: new
    auto-compaction-mode: periodic
    auto-compaction-retention: 10h
    quota-backend-bytes: 8589934592
    snapshot-count: 100000
    max-snapshots: 5
    max-wals: 5
    strict-reconfig-check: true
    pre-vote: true
    enable-v2: false
    metrics: basic
    logger: zap
    log-level: info
    log-outputs:
      - /data/logs/etcd/etcd.log
    enable-log-rotation: true
    log-rotation-config-json: '{"maxsize":100,"maxage":180,"maxbackups":0,"localtime":true,"compress":true}'
    EOF
    • listen-client-urls 和 advertise-client-urls 中的 IP 请替换为实际部署服务器的 IP。

  6. 配置 systemd 管理 etcd-03

    cat > /etc/systemd/system/etcd-03.service <<'EOF'
    [Unit]
    Description=etcd member etcd-03
    Documentation=https://etcd.io/docs/
    After=network-online.target
    Wants=network-online.target
    [Service]
    Type=simple
    User=etcd
    Group=etcd
    ExecStart=/usr/local/etcd/bin/etcd --config-file=/usr/local/etcd/etcd.conf.yml
    Restart=on-failure
    RestartSec=5s
    LimitNOFILE=65536
    LimitNPROC=65536
    OOMScoreAdjust=-999
    [Install]
    WantedBy=multi-user.target
    EOF

  7. 启动 etcd-03 并加入开机自启动

    systemctl daemon-reload
    systemctl start etcd-03
    systemctl enable etcd-03

  8. 服务状态检查

    systemctl status etcd-03 --no-pager

  9. 查看运行日志

    journalctl -u etcd-03 -n 100 -l --no-pager

etcd 集群验证​

  1. 设置集群端点

    export ETCD_ENDPOINTS=http://192.168.10.31:2379,http://192.168.10.32:2379,http://192.168.10.33:2379

  2. 检查集群健康状态

    /usr/local/etcd/bin/etcdctl --endpoints=${ETCD_ENDPOINTS} endpoint health -w table

  3. 查看集群状态

    /usr/local/etcd/bin/etcdctl --endpoints=${ETCD_ENDPOINTS} endpoint status -w table

  4. 查看 Member 列表

    /usr/local/etcd/bin/etcdctl --endpoints=${ETCD_ENDPOINTS} member list -w table

  5. 执行写入、读取与删除测试

    /usr/local/etcd/bin/etcdctl --endpoints=${ETCD_ENDPOINTS} put /etcd-test/hello world
    /usr/local/etcd/bin/etcdctl --endpoints=${ETCD_ENDPOINTS} get /etcd-test/hello
    /usr/local/etcd/bin/etcdctl --endpoints=${ETCD_ENDPOINTS} del /etcd-test/hello

开启认证​

任意一台节点均可执行。

  1. 设置变量

    export ETCD_ENDPOINTS=http://192.168.10.31:2379,http://192.168.10.32:2379,http://192.168.10.33:2379
    export ETCD_ROOT_PASSWORD='请替换成_root_强密码'
    • ETCD_ENDPOINTS 指定集群的三个访问端点，后续所有认证相关命令都通过它操作。
    • ETCD_ROOT_PASSWORD 指定 root 用户密码，正式部署时请替换为强口令。

  2. 确认 root 密码变量已设置

    test -n "$ETCD_ROOT_PASSWORD" && echo "ETCD_ROOT_PASSWORD 已设置" || echo "ETCD_ROOT_PASSWORD 未设置"
    • 检查 root 密码变量是否已设置，避免后续执行时使用空密码。

  3. 创建 root 用户

    /usr/local/etcd/bin/etcdctl --endpoints=${ETCD_ENDPOINTS} user add "root:${ETCD_ROOT_PASSWORD}"
    • user add 用于创建 root 用户并设置密码。

  4. 授予 root 角色

    /usr/local/etcd/bin/etcdctl --endpoints=${ETCD_ENDPOINTS} user grant-role root root
    • user grant-role root root 用于给 root 用户授予 root 角色，确保其拥有管理员权限。

  5. 查看 root 用户信息

    /usr/local/etcd/bin/etcdctl --endpoints=${ETCD_ENDPOINTS} user get root
    • user get root 用于查看 root 用户信息，确认用户已创建成功。

  6. 启用认证功能

    /usr/local/etcd/bin/etcdctl --endpoints=${ETCD_ENDPOINTS} auth enable
    • auth enable 用于开启 etcd 认证功能，开启后访问服务必须携带用户凭据。

  7. 认证验证

    /usr/local/etcd/bin/etcdctl --endpoints=${ETCD_ENDPOINTS} --user="root:${ETCD_ROOT_PASSWORD}" auth status
    /usr/local/etcd/bin/etcdctl --endpoints=${ETCD_ENDPOINTS} --user="root:${ETCD_ROOT_PASSWORD}" endpoint health -w table
    /usr/local/etcd/bin/etcdctl --endpoints=${ETCD_ENDPOINTS} --user="root:${ETCD_ROOT_PASSWORD}" endpoint status -w table
    /usr/local/etcd/bin/etcdctl --endpoints=${ETCD_ENDPOINTS} --user="root:${ETCD_ROOT_PASSWORD}" member list -w table
    • auth status 用于查看当前认证开关状态，确认认证是否已启用。
    • endpoint health -w table 用于检查当前端点健康状态，确认认证开启后服务仍可正常访问。
    • endpoint status -w table 用于查看端点详细状态，确认集群运行正常。
    • member list -w table 用于查看成员列表，确认集群成员信息完整。