Merge pull request #166 from LiuG-lynx/master

install ks on huaweicloud ecs ha
This commit is contained in:
pengfei 2020-09-01 11:42:54 +08:00 committed by GitHub
commit 3bb1042ff2
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
11 changed files with 263 additions and 0 deletions

View File

@ -0,0 +1,263 @@
---
title: "KubeSphere 在华为云 ECS 高可用实例"
keywords: "Kubesphere 安装, 华为云, ECS 高可用性, 高可用性, 负载均衡器"
description: "本教程用于安装高可用性集群"
Weight: 2230
---
由于对于生产环境,我们需要考虑集群的高可用性。教你部署如何在华为云 ECS 实例服务快速部署一套高可用的生产环境
Kubernetes 服务需要做到高可用,需要保证 kube-apiserver 的 HA ,推荐华为云负载均衡器服务.
## 前提条件
- 请遵循该[指南](https://github.com/kubesphere/kubekey),确保您已经知道如何将 KubeSphere 与多节点集群一起安装。有关用于安装的 config.yaml 文件的详细信息。本教程重点介绍配置华为云负载均衡器服务高可用安装。
- 考虑到数据的持久性对于生产环境我们不建议您使用存储OpenEBS建议 NFS GlusterFS 等存储(需要提前安装)。文章为了进行开发和测试,集成的 OpenEBS 直接将 LocalPV 设置为存储服务。
- SSH 可以访问所有节点。
- 所有节点的时间同步。
- Red Hat 在其 Linux 发行版本中包括了SELinux建议关闭 SELinux 或者将 SELinux 的模式切换为 Permissive [宽容]工作模式。
## 创建主机
本示例创建 6 台 Ubuntu 18.04 server 64bit 的云服务器,每台配置为 4 核 8 GB
| 主机IP | 主机名称 | 角色 |
| --- | --- | --- |
|192.168.1.10|master1|master1, etcd|
|192.168.1.11|master2|master2, etcd|
|192.168.1.12|master3|master3, etcd|
|192.168.1.13|node1|node|
|192.168.1.14|node2|node|
|192.168.1.15|node3|node|
> 注意:机器有限,所以把 etcd 放入 master在生产环境建议单独部署 etcd提高稳定性
## 华为云负载均衡器部署
### 创建 VPC
进入到华为云控制, 在左侧列表选择'虚拟私有云' 选择'创建虚拟私有云' 创建VPC,配置如下图
![1-1-创建VPC](/images/docs/huawei-ecs/huawei-VPC-create.png)
### 创建安全组
`访问控制→ 安全组`下,创建一个安全组,设置入方向的规则参考如下:
![2-1-创建安全组](/images/docs/huawei-ecs/huawei-rules-create.png)
> 提示:后端服务器的安全组规则必须放行 100.125.0.0/16 网段,否则会导致健康检查异常,详见 后端服务器配置安全组 。此外,还应放行 192.168.1.0/24 (主机之间的网络需全放行)。
### 创建主机
![3-1-选择主机配置](/images/docs/huawei-ecs/huawei-ECS-basic-settings.png)
在网络配置中,网络选择第一步创建的 VPC 和子网。在安全组中,选择上一步创建的安全组。
![3-2-选择网络配置](/images/docs/huawei-ecs/huawei-ECS-network-settings.png)
### 创建负载均衡器
在左侧栏选择 '弹性负载均衡器',进入后选择 购买弹性负载均衡器
> 以下健康检查结果在部署后才会显示正常,目前状态为异常
#### 内网LB 配置
为所有master 节点 添加后端监听器 ,监听端口为 6443
![4-1-配置内网LB](/images/docs/huawei-ecs/huawei-master-lb-basic-config.png)
![4-2-配置内网LB](/images/docs/huawei-ecs/huawei-master-lb-listeners-config.png)
#### 外网LB 配置
若集群需要配置公网访问,则需要为外网负载均衡器配置一个公网 IP为 所有节点 添加后端监听器,监听端口为 80(测试使用 30880 端口,此处 80 端口也需要在安全组中开放)。
![4-3-配置外网LB](/images/docs/huawei-ecs/huawei-public-lb-basic-config.png)
![4-4-配置外网LB](/images/docs/huawei-ecs/huawei-public-lb-listeners-config.png)
后面配置文件 config.yaml 需要配置 slb 分配的地址
```yaml
controlPlaneEndpoint:
domain: lb.kubesphere.local
address: "192.168.1.8"
port: "6443"
```
### 获取安装程序可执行文件
```bash
#下载 kk installer 至任意一台机器
curl -O -k https://kubernetes.pek3b.qingstor.com/tools/kubekey/kk
chmod +x kk
```
{{< notice tip >}}
您可以使用高级安装来控制自定义参数或创建多节点群集。具体来说,通过指定配置文件来创建集群。
{{</ notice >}}
### 使用 kubekey 部署k8s集群和 KubeSphere 控制台
```bash
# 在当前位置创建配置文件 master-HA.yaml |包含 KubeSphere 的配置文件
./kk create config --with-kubesphere v3.0.0 -f master-HA.yaml
---
# 同时安装存储插件 (支持localVolume、nfsClient、rbd、glusterfs)。您可以指定多个插件并用逗号分隔。请注意,您添加的第一个将是默认存储类。
./kk create config --with-storage localVolume --with-kubesphere v3.0.0 -f master-HA.yaml
```
### 集群配置调整
目前当前集群开启了全量的组件,文末也提供了自定义的方法.可默认为 false
```yaml
apiVersion: kubekey.kubesphere.io/v1alpha1
kind: Cluster
metadata:
name: master-HA
spec:
hosts:
- {name: master1, address: 192.168.1.10, internalAddress: 192.168.1.10, password: yourpassword} # Assume that the default port for SSH is 22, otherwise add the port number after the IP address as above
- {name: master2, address: 192.168.1.11, internalAddress: 192.168.1.11, password: yourpassword} # Assume that the default port for SSH is 22, otherwise add the port number after the IP address as above
- {name: master3, address: 192.168.1.12, internalAddress: 192.168.1.12, password: yourpassword} # Assume that the default port for SSH is 22, otherwise add the port number after the IP address as above
- {name: node1, address: 192.168.1.13, internalAddress: 192.168.1.13, password: yourpassword} # Assume that the default port for SSH is 22, otherwise add the port number after the IP address as above
- {name: node2, address: 192.168.1.14, internalAddress: 192.168.1.14, password: yourpassword} # Assume that the default port for SSH is 22SSH is 22, otherwise add the port number after the IP address as above
- {name: node3, address: 192.168.1.15, internalAddress: 192.168.1.15, password: yourpassword} # Assume that the default port for SSH is 22, otherwise add the port number after the IP address as above
roleGroups:
etcd:
- master[1:3]
master:
- master[1:3]
worker:
- node[1:3]
controlPlaneEndpoint:
domain: lb.kubesphere.local
address: "192.168.1.8"
port: "6443"
kubernetes:
version: v1.17.9
imageRepo: kubesphere
clusterName: cluster.local
masqueradeAll: false # masqueradeAll tells kube-proxy to SNAT everything if using the pure iptables proxy mode. [Default: false]
maxPods: 110 # maxPods is the number of pods that can run on this Kubelet. [Default: 110]
nodeCidrMaskSize: 24 # internal network node size allocation. This is the size allocated to each node on your network. [Default: 24]
proxyMode: ipvs # mode specifies which proxy mode to use. [Default: ipvs]
network:
plugin: calico
calico:
ipipMode: Always # IPIP Mode to use for the IPv4 POOL created at start up. If set to a value other than Never, vxlanMode should be set to "Never". [Always | CrossSubnet | Never] [Default: Always]
vxlanMode: Never # VXLAN Mode to use for the IPv4 POOL created at start up. If set to a value other than Never, ipipMode should be set to "Never". [Always | CrossSubnet | Never] [Default: Never]
vethMTU: 1440 # The maximum transmission unit (MTU) setting determines the largest packet size that can be transmitted through your network. [Default: 1440]
kubePodsCIDR: 10.233.64.0/18
kubeServiceCIDR: 10.233.0.0/18
registry:
registryMirrors: ["https://*.mirror.aliyuncs.com"] # # input your registryMirrors
insecureRegistries: []
privateRegistry: ""
storage:
defaultStorageClass: localVolume
localVolume:
storageClassName: local
---
apiVersion: installer.kubesphere.io/v1alpha1
kind: ClusterConfiguration
metadata:
name: ks-installer
namespace: kubesphere-system
labels:
version: v3.0.0
spec:
local_registry: ""
persistence:
storageClass: ""
authentication:
jwtSecret: ""
etcd:
monitoring: true # Whether to install etcd monitoring dashboard
endpointIps: 192.168.1.10,192.168.1.11,192.168.1.12 # etcd cluster endpointIps
port: 2379 # etcd port
tlsEnable: true
common:
mysqlVolumeSize: 20Gi # MySQL PVC size
minioVolumeSize: 20Gi # Minio PVC size
etcdVolumeSize: 20Gi # etcd PVC size
openldapVolumeSize: 2Gi # openldap PVC size
redisVolumSize: 2Gi # Redis PVC size
es: # Storage backend for logging, tracing, events and auditing.
elasticsearchMasterReplicas: 1 # total number of master nodes, it's not allowed to use even number
elasticsearchDataReplicas: 1 # total number of data nodes
elasticsearchMasterVolumeSize: 4Gi # Volume size of Elasticsearch master nodes
elasticsearchDataVolumeSize: 20Gi # Volume size of Elasticsearch data nodes
logMaxAge: 7 # Log retention time in built-in Elasticsearch, it is 7 days by default.
elkPrefix: logstash # The string making up index names. The index name will be formatted as ks-<elk_prefix>-log
# externalElasticsearchUrl:
# externalElasticsearchPort:
console:
enableMultiLogin: false # enable/disable multiple sing on, it allows an account can be used by different users at the same time.
port: 30880
alerting: # Whether to install KubeSphere alerting system. It enables Users to customize alerting policies to send messages to receivers in time with different time intervals and alerting levels to choose from.
enabled: true
auditing: # Whether to install KubeSphere audit log system. It provides a security-relevant chronological set of recordsrecording the sequence of activities happened in platform, initiated by different tenants.
enabled: true
devops: # Whether to install KubeSphere DevOps System. It provides out-of-box CI/CD system based on Jenkins, and automated workflow tools including Source-to-Image & Binary-to-Image
enabled: true
jenkinsMemoryLim: 2Gi # Jenkins memory limit
jenkinsMemoryReq: 1500Mi # Jenkins memory request
jenkinsVolumeSize: 8Gi # Jenkins volume size
jenkinsJavaOpts_Xms: 512m # The following three fields are JVM parameters
jenkinsJavaOpts_Xmx: 512m
jenkinsJavaOpts_MaxRAM: 2g
events: # Whether to install KubeSphere events system. It provides a graphical web console for Kubernetes Events exporting, filtering and alerting in multi-tenant Kubernetes clusters.
enabled: true
logging: # Whether to install KubeSphere logging system. Flexible logging functions are provided for log query, collection and management in a unified console. Additional log collectors can be added, such as Elasticsearch, Kafka and Fluentd.
enabled: true
logsidecarReplicas: 2
metrics_server: # Whether to install metrics-server. IT enables HPA (Horizontal Pod Autoscaler).
enabled: true
monitoring: #
prometheusReplicas: 1 # Prometheus replicas are responsible for monitoring different segments of data source and provide high availability as well.
prometheusMemoryRequest: 400Mi # Prometheus request memory
prometheusVolumeSize: 20Gi # Prometheus PVC size
alertmanagerReplicas: 1 # AlertManager Replicas
multicluster:
clusterRole: none # host | member | none # You can install a solo cluster, or specify it as the role of host or member cluster
networkpolicy: # Network policies allow network isolation within the same cluster, which means firewalls can be set up between certain instances (Pods).
enabled: true
notification: # It supports notification management in multi-tenant Kubernetes clusters. It allows you to set AlertManager as its sender, and receivers include Email, Wechat Work, and Slack.
enabled: true
openpitrix: # Whether to install KubeSphere Application Store. It provides an application store for Helm-based applications, and offer application lifecycle management
enabled: true
servicemesh: # Whether to install KubeSphere Service Mesh (Istio-based). It provides fine-grained traffic management, observability and tracing, and offer visualization for traffic topology
enabled: true
```
### 执行命令创建集群
```bash
# 指定配置文件创建集群
./kk create cluster --with-kubesphere v3.0.0 -f master-HA.yaml
# 查看 KubeSphere 安装日志 -- 直到出现控制台的访问地址和登陆账号
kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f
```
```bash
#####################################################
### Welcome to KubeSphere! ###
#####################################################
Console: http://192.168.1.10:30880
Account: admin
Password: P@88w0rd
NOTES
1. After logging into the console, please check the
monitoring status of service components in
the "Cluster Management". If any service is not
ready, please wait patiently until all components
are ready.
2. Please modify the default password after login.
#####################################################
https://kubesphere.io 2020-08-28 01:25:54
#####################################################
```
访问公网 IP + Port 为部署后的使用情况,使用默认账号密码 (`admin/P@88w0rd`),文章组件安装为最大化,登陆点击`平台管理>集群管理` 可看到下图安装组件列表和机器情况。
## 如何自定义开启可插拔组件
点击 `集群管理` - `自定义资源CRD` ,在过滤条件框输入 `ClusterConfiguration` ,如图下
![5-1-自定义组件](/images/docs/huawei-ecs/huawei-crds-config.png)
点击 `ClusterConfiguration` 详情,对 `ks-installer` 编辑保存退出即可,组件描述介绍:[文档说明](https://github.com/kubesphere/ks-installer/blob/master/deploy/cluster-configuration.yaml)
![5-2-自定义组件](/images/docs/huawei-ecs/huawei-crds-edit-yaml.png)

Binary file not shown.

After

Width:  |  Height:  |  Size: 252 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 201 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 74 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 136 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 74 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 48 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 124 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 81 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 151 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 126 KiB