根据前文准备好的环境,我们现在来一步步的搭建一个基础的k8s集群
注意,这里的配置信息都是按照我自己的虚拟环境来写的。
把server01作为master节点,server02和server03作为worker节点
各个节点需要配置的服务和命令如下:
master节点上需要部署的服务包括:etcd服务APIServer服务Scheduler服务ControllerManager服务CalicoNode服务kube-proxy服务kubectl命令

worker节点上需要部署的服务包括:CalicoNode服务kubelet服务kube-proxy服务

步骤1:准备文件

安装k8s集群有几种方式可以选择,比如容器化的方式,比如用kubeadmin的方式,这次我们打算尝试的是使用二进制文件的方式。
1. 登录到master虚拟机上(server01),从github上下载安装文件的压缩包,我们使用的是1.13.6版本:

    wget https://github.com/kubernetes/kubernetes/releases/download/v1.13.6/kubernetes.tar.gz
  1. 解压缩

    tar zxvf kubernetes.tar.gz
    
  2. 下载文件,进入刚刚解压好的文件夹

    cd kubernetes
    ./cluster/get-kube-binaries.sh
    

    这个步骤因为涉及到从官网下载文件,由于墙的原因会非常缓慢或者失败,请自行上网解决。

步骤2:master环境部署

  1. etcd部署:
    编写etcd服务的启动配置文件etcd.service,内容如下:

    [Unit]
    Description=Etcd Server
    After=network.target
    After=network-online.target
    Wants=network-online.target
    Documentation=https://github.com/coreos
    
    [Service]
    Type=notify
    WorkingDirectory=/var/lib/etcd/
    ExecStart=/home/anakin/bin/etcd \
    --name=192.168.32.131 \
    --listen-client-urls=http://192.168.32.131:2379,http://127.0.0.1:2379 \
    --advertise-client-urls=http://192.168.32.131:2379 \
    --data-dir=/var/lib/etcd
    Restart=on-failure
    RestartSec=5
    LimitNOFILE=65536
    
    [Install]
    WantedBy=multi-user.target
    

然后执行以下命令:

cp etcd.service /lib/systemd/system/
systemctl enable etcd.service
mkdir -p /var/lib/etcd
service etcd start

如果没有问题的话,etcd服务应该已经跑起来了。

  1. APIServer部署
    编写kube-apiserver.service文件,内容如下:

    [Unit]
    Description=Kubernetes API Server
    Documentation=https://github.com/GoogleCloudPlatform/kubernetes
    After=network.target
    [Service]
    ExecStart=/home/anakin/bin/kube-apiserver \
    --admission-control=NamespaceLifecycle,LimitRanger,DefaultStorageClass,ResourceQuota,NodeRestriction \
    --insecure-bind-address=0.0.0.0 \
    --kubelet-https=false \
    --service-cluster-ip-range=10.68.0.0/16 \
    --service-node-port-range=20000-40000 \
    --etcd-servers=http://192.168.32.131:2379 \
    --enable-swagger-ui=true \
    --allow-privileged=true \
    --audit-log-maxage=30 \
    --audit-log-maxbackup=3 \
    --audit-log-maxsize=100 \
    --audit-log-path=/var/lib/audit.log \
    --event-ttl=1h \
    --v=2
    Restart=on-failure
    RestartSec=5
    Type=notify
    LimitNOFILE=65536
    [Install]
    WantedBy=multi-user.target
    

    然后执行和启动etcd类似的命令

    cp kube-apiserver.service /lib/systemd/system/
    systemctl enable kube-apiserver.service
    service kube-apiserver start
    
  2. ControllerManager部署
    编写kube-controller-manager.service文件,内容如下:

    [Unit]
    Description=Kubernetes Controller Manager
    Documentation=https://github.com/GoogleCloudPlatform/kubernetes
    [Service]
    ExecStart=/home/anakin/bin/kube-controller-manager \
    --address=127.0.0.1 \
    --master=http://127.0.0.1:8080 \
    --allocate-node-cidrs=true \
    --service-cluster-ip-range=10.68.0.0/16 \
    --cluster-cidr=172.20.0.0/16 \
    --cluster-name=kubernetes \
    --leader-elect=true \
    --cluster-signing-cert-file= \
    --cluster-signing-key-file= \
    --v=2
    Restart=on-failure
    RestartSec=5
    [Install]
    WantedBy=multi-user.target
    

    然后,同样的方式启动服务:

    cp kube-controller-manager.service /lib/systemd/system/
    systemctl enable kube-controller-manager.service
    service kube-controller-manager start
    
  3. Scheduler部署
    编写kube-scheduler.service文件,内容如下:

    [Unit]
    Description=Kubernetes Scheduler
    Documentation=https://github.com/GoogleCloudPlatform/kubernetes
    
    [Service]
    ExecStart=/home/anakin/bin/kube-scheduler \
    --address=127.0.0.1 \
    --master=http://127.0.0.1:8080 \
    --leader-elect=true \
    --v=2
    Restart=on-failure
    RestartSec=5
    [Install]
    WantedBy=multi-user.target
    

然后,继续上面的启动方式:

cp kube-scheduler.service /lib/systemd/system/
systemctl enable kube-scheduler.service
service kube-scheduler start
  1. CalicoNode部署 编写kube-calico.service文件,内容如下:

    [Unit]
    Description=calico node
    After=docker.service
    Requires=docker.service
    
    [Service]
    User=root
    PermissionsStartOnly=true
    ExecStart=/usr/bin/docker run --net=host --privileged --name=calico-node \
    -e ETCD_ENDPOINTS=http://192.168.32.131:2379 \
    -e CALICO_LIBNETWORK_ENABLED=true \
    -e CALICO_NETWORKING_BACKEND=bird \
    -e CALICO_DISABLE_FILE_LOGGING=true \
    -e CALICO_IPV4POOL_CIDR=172.20.0.0/16 \
    -e CALICO_IPV4POOL_IPIP=off \
    -e FELIX_DEFAULTENDPOINTTOHOSTACTION=ACCEPT \
    -e FELIX_IPV6SUPPORT=false \
    -e FELIX_LOGSEVERITYSCREEN=info \
    -e FELIX_IPINIPMTU=1440 \
    -e FELIX_HEALTHENABLED=true \
    -e IP=192.168.32.131 \
    -v /var/run/calico:/var/run/calico \
    -v /lib/modules:/lib/modules \
    -v /run/docker/plugins:/run/docker/plugins \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v /var/log/calico:/var/log/calico \
    registry.anakin.sun.com/k8s/calico-node:v2.6.2
    ExecStop=/usr/bin/docker rm -f calico-node
    Restart=always
    RestartSec=10
    
    [Install]
    WantedBy=multi-user.target
    

同样的方式启动:

cp kube-calico.service /lib/systemd/system/
systemctl enable kube-calico.service
service kube-calico start
  1. kubectl命令配置 执行以下命令(root):

    kubectl config set-cluster kubernetes  --server=http://192.168.1.102:8080
    kubectl config set-context kubernetes --cluster=kubernetes
    kubectl config use-context kubernetes
    

    如果有问题,可以手动修改配置文件:~/.kube/config

  2. kube-proxy服务部署 编写kube-proxy.service文件,内容如下:

    [Unit]
    Description=Kubernetes Kube-Proxy Server
    Documentation=https://github.com/GoogleCloudPlatform/kubernetes
    After=network.target
    [Service]
    WorkingDirectory=/var/lib/kube-proxy
    ExecStart=/home/anakin/bin/kube-proxy \
    --bind-address=192.168.32.131 \
    --hostname-override=192.168.32.131 \
    --kubeconfig=/etc/kubernetes/kube-proxy.kubeconfig \
    --logtostderr=true \
    --v=2
    Restart=on-failure
    RestartSec=5
    LimitNOFILE=65536
    
    [Install]
    WantedBy=multi-user.target
    

编写kube-proxy.kubeconfig文件,内容如下:

apiVersion: v1
clusters:
- cluster:
    server: http://192.168.32.131:8080
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
  name: default
current-context: default
kind: Config
preferences: {}
users: []

然后执行如下命令:

mkdir -p /var/lib/kube-proxy
cp kube-proxy.service /lib/systemd/system/
cp kube-proxy.kubeconfig /etc/kubernetes/
systemctl enable kube-proxy.service

OK,至此,master节点应该已经配置完成了。

步骤3:worker环境部署

  1. CalicoNode部署
    参考master部分的内容
  2. kube-proxy服务部署
    参考master部分的内容

  3. kubelet服务配置
    编写kubelet.service文件,内容如下:

    [Unit]
    Description=Kubernetes Kubelet
    Documentation=https://github.com/GoogleCloudPlatform/kubernetes
    After=docker.service
    Requires=docker.service
    
    [Service]
    WorkingDirectory=/var/lib/kubelet
    ExecStart=/home/anakin/bin/kubelet \
    --address=192.168.32.131 \
    --hostname-override=192.168.32.131 \
    --pod-infra-container-image=registry.anakin.sun.com/imooc/pause-amd64:3.0 \
    --kubeconfig=/etc/kubernetes/kubelet.kubeconfig \
    --network-plugin=cni \
    --cni-conf-dir=/etc/cni/net.d \
    --cni-bin-dir=/home/anakin/bin \
    --cluster-dns=10.68.0.2 \
    --cluster-domain=cluster.local. \
    --allow-privileged=true \
    --fail-swap-on=false \
    --logtostderr=true \
    --v=2
    ExecStartPost=/sbin/iptables -A INPUT -s 10.0.0.0/8 -p tcp --dport 4194 -j ACCEPT
    ExecStartPost=/sbin/iptables -A INPUT -s 172.16.0.0/12 -p tcp --dport 4194 -j ACCEPT
    ExecStartPost=/sbin/iptables -A INPUT -s 192.168.0.0/16 -p tcp --dport 4194 -j ACCEPT
    ExecStartPost=/sbin/iptables -A INPUT -p tcp --dport 4194 -j DROP
    Restart=on-failure
    RestartSec=5
    
    [Install]
    WantedBy=multi-user.target
    

编写kubelet.kubeconfig文件,内容如下:

apiVersion: v1
clusters:
- cluster:
    insecure-skip-tls-verify: true
    server: http://192.168.32.131:8080
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: ""
  name: system:node:kube-master
current-context: system:node:kube-master
kind: Config
preferences: {}
users: []

编写10-calico.conf文件,内容如下:

{
    "name": "calico-k8s-network",
    "cniVersion": "0.1.0",
    "type": "calico",
    "etcd_endpoints": "http://192.168.32.131:2379",
    "log_level": "info",
    "ipam": {
        "type": "calico-ipam"
    },
    "kubernetes": {
        "k8s_api_root": "http://192.168.32.131:8080"
    }
}

然后,执行如下命令:

mkdir -p /var/lib/kubelet
mkdir -p /etc/kubernetes
mkdir -p /etc/cni/net.d

cp kubelet.service /lib/systemd/system/
cp kubelet.kubeconfig /etc/kubernetes/
cp 10-calico.conf /etc/cni/net.d/

systemctl enable kubelet.service
service kubelet start

至此,整个集群应该已经搭建好了,如果中间遇到什么问题,可以通过查看系统日志,或者google解决。