Deploy Milvus

Deployment Preparation

Nodes used to run Milvus must first join the Kubernetes cluster, and their status must be Ready.
Before deploying Milvus, prepare the following external dependencies:
- Kafka
- etcd
- S3 object storage (MinIO)
Create the dedicated Milvus bucket mdmilvus in S3 object storage (MinIO)

Milvus uses object storage to store data files. Create a dedicated bucket for it. The following example uses mc to connect to MinIO and create the mdmilvus bucket:
```
mc alias set minio http://MINIO_ENDPOINT MINIO_ACCESS_KEY MINIO_SECRET_KEY
mc mb minio/mdmilvus
mc ls minio
```
- Replace MINIO_ENDPOINT with the actual MinIO endpoint, for example http://10.206.0.6:9000.
- Replace MINIO_ACCESS_KEY and MINIO_SECRET_KEY with the actual access key and secret key.
Check Node Status
```
kubectl get nodes -o wide
```
- Confirm that the nodes planned to run Milvus have joined the Kubernetes cluster.
- Confirm that the node status is Ready.
Add labels and taints to dedicated Milvus worker nodes

First mark the Kubernetes worker nodes used to run Milvus, and use taints to prevent other business Pods from being scheduled to these nodes by mistake.
```
kubectl label nodes <milvus-node-name> hap=milvus
kubectl taint nodes <milvus-node-name> hap=milvus:NoSchedule
```
- hap=milvus is the node label used to identify dedicated Milvus worker nodes.
- hap=milvus:NoSchedule is the node taint used to prevent Pods without the corresponding toleration from being scheduled to these nodes.
- Replace <milvus-node-name> with the actual Milvus worker node name. If there are multiple nodes, add the node label and taint to each node.
Preload images on dedicated Milvus worker nodes

Before installing Milvus Operator and Milvus, prepare the required images on the Kubernetes worker nodes that will run Milvus.
- Servers with Internet Access
- Servers without Internet Access
crictl pull registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-milvus-operator:v1.3.6 crictl pull registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-milvus:v2.6.17
- Milvus image package:
  
  amd64：https://pdpublic.mingdao.com/private-deployment/offline/common/mingdaoyun-milvus-linux-v2.6.17.tar.gz
  
  arm64：https://pdpublic.mingdao.com/private-deployment/offline/common/mingdaoyun-milvus-linux-v2.6.17.tar.gz
- Milvus Operator image package:
  
  amd64：https://pdpublic.mingdao.com/private-deployment/offline/common/mingdaoyun-milvus-operator-linux-v1.3.6.tar.gz
  
  arm64：https://pdpublic.mingdao.com/private-deployment/offline/common/mingdaoyun-milvus-operator-linux-v1.3.6.tar.gz
- The following uses amd64 as an example. Download the image packages, then decompress and import them into the local container runtime:
  
  wget https://pdpublic.mingdao.com/private-deployment/offline/common/mingdaoyun-milvus-linux-v2.6.17.tar.gz wget https://pdpublic.mingdao.com/private-deployment/offline/common/mingdaoyun-milvus-operator-linux-v1.3.6.tar.gz gunzip -d mingdaoyun-milvus-linux-v2.6.17.tar.gz gunzip -d mingdaoyun-milvus-operator-linux-v1.3.6.tar.gz ctr -n k8s.io image import mingdaoyun-milvus-linux-v2.6.17.tar ctr -n k8s.io image import mingdaoyun-milvus-operator-linux-v1.3.6.tar
- registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-milvus-operator:v1.3.6 is used to deploy Milvus Operator.
- registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-milvus:v2.6.17 is used to deploy Milvus cluster workloads.

Deploy Milvus Operator

Deploy Milvus Operator first. The subsequent Milvus custom resource depends on it to create workloads.

For easier maintenance and reuse, store the yaml files used in this section under /data/mingdao/script/kubernetes/milvus/ on Kubernetes Master 01. Run download and deployment commands from this directory afterward.

Prepare milvus-operator-deployment.yaml

Create a directory on Kubernetes Master 01 to store the files, and switch to that directory:

mkdir -p /data/mingdao/script/kubernetes/milvus
cd /data/mingdao/script/kubernetes/milvus

Then download milvus-operator-deployment.yaml to the current directory.

Servers with Internet Access
Servers without Internet Access

wget https://pdpublic.mingdao.com/private-deployment/offline/common/milvus-operator-deployment.yaml

# Download milvus-operator-deployment.yaml in an Internet-connected environment first, then upload it to the current directory.
# Download link:https://pdpublic.mingdao.com/private-deployment/offline/common/milvus-operator-deployment.yaml

Apply the operator resource
```
kubectl apply -f milvus-operator-deployment.yaml
```
- This step deploys the Milvus Operator controller.
- The subsequent Milvus resource can be parsed and created only after the operator is running normally.
Check operator status
```
kubectl get pod -n milvus-operator -o wide
```
- Confirm that the Milvus Operator Pod has started normally.

Deploy Milvus

The following steps are based on milvus.yaml.

For easier maintenance and reuse, store the yaml files used in this section under /data/mingdao/script/kubernetes/milvus/ on Kubernetes Master 01. Run creation and deployment commands from this directory afterward.

Create the Milvus namespace
```
kubectl create namespace milvus
```
- If the namespace already exists, ignore this command.
Create the S3 credential Secret

The Milvus storage dependency uses secretRef: milvus-s3-secret, so create the corresponding Secret in the milvus namespace first.
```
kubectl -n milvus create secret generic milvus-s3-secret \
  --from-literal=accessKey='replace-with-s3-access-key' \
  --from-literal=secretKey='replace-with-s3-secret-key'
```
- accessKey is the object storage access key.
- secretKey is the object storage secret key.

Prepare milvus.yaml

Create milvus.yaml under /data/mingdao/script/kubernetes/milvus/ on Kubernetes Master 01 with the following content:

apiVersion: milvus.io/v1beta1
kind: Milvus
metadata:
  name: hap-rag
  namespace: milvus
  labels:
    app: milvus
spec:
  mode: cluster
  components:
     image: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-milvus:v2.6.17
    imagePullPolicy: IfNotPresent
    nodeSelector:
      hap: milvus
    tolerations:
      - key: "hap"
        operator: "Equal"
        value: "milvus"
        effect: "NoSchedule"
    proxy:
      replicas: 2
      resources:
        requests:
          cpu: "250m"
          memory: "1Gi"
        limits:
          cpu: "2"
          memory: "4Gi"
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 100
              podAffinityTerm:
                topologyKey: kubernetes.io/hostname
                labelSelector:
                  matchLabels:
                    app.kubernetes.io/component: proxy
    mixCoord:
      replicas: 2
      resources:
        requests:
          cpu: "250m"
          memory: "1Gi"
        limits:
          cpu: "4"
          memory: "8Gi"
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 100
              podAffinityTerm:
                topologyKey: kubernetes.io/hostname
                labelSelector:
                  matchLabels:
                    app.kubernetes.io/component: mixcoord
    streamingNode:
      replicas: 2
      resources:
        requests:
          cpu: "250m"
          memory: "1Gi"
        limits:
          cpu: "6"
          memory: "12Gi"
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 100
              podAffinityTerm:
                topologyKey: kubernetes.io/hostname
                labelSelector:
                  matchLabels:
                    app.kubernetes.io/component: streamingnode
    queryNode:
      replicas: 2
      resources:
        requests:
          cpu: "250m"
          memory: "1Gi"
        limits:
          cpu: "8"
          memory: "32Gi"
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 100
              podAffinityTerm:
                topologyKey: kubernetes.io/hostname
                labelSelector:
                  matchLabels:
                    app.kubernetes.io/component: querynode
    dataNode:
      replicas: 2
      resources:
        requests:
          cpu: "250m"
          memory: "1Gi"
        limits:
          cpu: "6"
          memory: "12Gi"
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 100
              podAffinityTerm:
                topologyKey: kubernetes.io/hostname
                labelSelector:
                  matchLabels:
                    app.kubernetes.io/component: datanode
  dependencies:
    msgStreamType: kafka
    kafka:
      external: true
      brokerList: # Modify: Kafka addresses
        - "192.168.10.31:9092"
        - "192.168.10.32:9092"
        - "192.168.10.33:9092"
    etcd:
      external: true
      endpoints: # Modify: etcd addresses
        - "http://192.168.10.31:2379"
        - "http://192.168.10.32:2379"
        - "http://192.168.10.33:2379"
    storage:
      external: true
      type: S3
      endpoint: "cos.ap-nanjing.myqcloud.com:443" # Modify: S3 object storage endpoint
      secretRef: milvus-s3-secret # Modify: S3 Secret name
  config:
    common:
      security:
        authorizationEnabled: true
        defaultRootPassword: "<set_a_strong_password>" # Modify: set the Milvus root user password
    mixCoord:
      enableActiveStandby: true
    msgChannel:
      chanNamePrefix:
        cluster: milvus
    mq:
      type: kafka
    kafka:
      securityProtocol: PLAINTEXT
    etcd:
      rootPath: "milvus"
      metaSubPath: "meta"
      kvSubPath: "kv"
      auth:
        enabled: true
        userName: "root"
        password: "******" # Modify: etcd password
      ssl:
        enabled: false
     minio:
       cloudProvider: aws # Fixed: S3 compatible
       bucketName: "mdmilvus" # Modify as needed: bucket name
       rootPath: "milvus"
       useSSL: true # Modify as needed: whether to access S3 over HTTPS
       useVirtualHost: true

dependencies.kafka.brokerList and dependencies.etcd.endpoints configure the external Kafka and etcd service addresses. Before deployment, confirm that these addresses are reachable from the Milvus worker nodes.
dependencies.storage configures the external S3 object storage. endpoint and secretRef must match the actual environment.
config.common.security.authorizationEnabled enables authentication, and defaultRootPassword must be set to a strong password.
config.etcd.auth.userName and config.etcd.auth.password are used to access the external etcd. Replace them with the actual authentication information.
config.minio.bucketName is currently set to mdmilvus, which specifies the object storage bucket to use. Keep rootPath as milvus. If the object storage endpoint or access method changes, adjust useSSL and useVirtualHost accordingly.

Apply the Milvus resource
```
kubectl apply -f milvus.yaml
```
- This step creates the Milvus cluster for hap-rag through Milvus Operator.
Check Milvus Resource Status
```
kubectl get pods -n milvus -o wide
```
```
kubectl get svc -n milvus
```
- Confirm that each component Pod has been scheduled and is running normally.
- Confirm that the exposed Service has been created.
Access Milvus

After deployment, the Milvus access address is as follows:
```
http://hap-rag-milvus.milvus:19530
```
- This address is accessible only from Pods inside the current Kubernetes cluster.

Connect to the HAP Service

Add the following environment variables to config.yaml of the HAP service:

ENV_RAG_ENABLE: "true"
ENV_MILVUS_URL: "http://hap-rag-milvus.milvus:19530"
ENV_MILVUS_DATABASE: "default"
ENV_MILVUS_USERNAME: "root"
ENV_MILVUS_PASSWORD: "your_password"

Set ENV_MILVUS_PASSWORD to the custom Milvus password configured earlier.
After saving the configuration, restart the HAP microservices for the changes to take effect.

Deployment Preparation​

Deploy Milvus Operator​

Deploy Milvus​

Connect to the HAP Service​

Deployment Preparation

Deploy Milvus Operator

Deploy Milvus

Connect to the HAP Service