diff --git a/README.md b/README.md index 4e18daaf..3ad21b5f 100644 --- a/README.md +++ b/README.md @@ -29,31 +29,34 @@ Use KubeKey in the following three scenarios. * **CentOS/RHEL** *7* * **SUSE Linux Enterprise Server** *15* -> Recommended Linux Kernel Version: `4.15 or later` \ +> Recommended Linux Kernel Version: `4.15 or later` > You can run the `uname -srm` command to check the Linux Kernel Version. -### Kubernetes Versions +### ``Kubernetes Versions -* **v1.17**:   *v1.17.9* -* **v1.18**:   *v1.18.8* * **v1.19**:   *v1.19.9* * **v1.20**:   *v1.20.10* * **v1.21**:   *v1.21.13* * **v1.22**:   *v1.22.12* * **v1.23**:   *v1.23.9* (default) * **v1.24**:   *v1.24.3* -> Looking for more supported versions -> [Kubernetes Versions](./docs/kubernetes-versions.md) -> [K3s Versions](./docs/k3s-versions.md) + +> Looking for more supported versions +> [Kubernetes Versions](./docs/kubernetes-versions.md) +> [K3s Versions](./docs/k3s-versions.md) ### Container Manager + * **Docker** / **containerd** / **CRI-O** / **iSula** + > `Kata Containers` can be set to automatically install and configure runtime class for it when the container manager is containerd or CRI-O. ### Network Plugins + * **Calico** / **Flannel** / **Cilium** / **Kube-OVN** / **Multus-CNI** > Kubekey also supports users to set the network plugin to `none` if there is a requirement for custom network plugin. + ## Requirements and Recommendations * Minimum resource requirements (For Minimal Installation of KubeSphere only)： @@ -69,22 +72,23 @@ Use KubeKey in the following three scenarios. * `sudo`/`curl`/`openssl` should be used in all nodes. * `docker` can be installed by yourself or by KubeKey. * `Red Hat` includes `SELinux` in its `Linux release`. It is recommended to close SELinux or [switch the mode of SELinux](./docs/turn-off-SELinux.md) to `Permissive` -> * It's recommended that Your OS is clean (without any other software installed), otherwise there may be conflicts. + +> * It's recommended that Your OS is clean (without any other software installed), otherwise there may be conflicts. > * A container image mirror (accelerator) is recommended to be prepared if you have trouble downloading images from dockerhub.io. [Configure registry-mirrors for the Docker daemon](https://docs.docker.com/registry/recipes/mirror/#configure-the-docker-daemon). > * KubeKey will install [OpenEBS](https://openebs.io/) to provision LocalPV for development and testing environment by default, this is convenient for new users. For production, please use NFS / Ceph / GlusterFS or commercial products as persistent storage, and install the [relevant client](docs/storage-client.md) in all nodes. -> * If you encounter `Permission denied` when copying, it is recommended to check [SELinux and turn off it](./docs/turn-off-SELinux.md) first +> * If you encounter `Permission denied` when copying, it is recommended to check [SELinux and turn off it](./docs/turn-off-SELinux.md) first * Dependency requirements: -KubeKey can install Kubernetes and KubeSphere together. The dependency that needs to be installed may be different based on the Kubernetes version to be installed. You can refer to the list below to see if you need to install relevant dependencies on your node in advance. +KubeKey can install Kubernetes and KubeSphere together. Some dependencies need to be installed before installing kubernetes after version 1.18. You can refer to the list below to check and install the relevant dependencies on your node in advance. -| | Kubernetes Version ≥ 1.18 | Kubernetes Version < 1.18 | -| ----------- | ------------------------- | ------------------------- | -| `socat` | Required | Optional but recommended | -| `conntrack` | Required | Optional but recommended | -| `ebtables` | Optional but recommended | Optional but recommended | -| `ipset` | Optional but recommended | Optional but recommended | -| `ipvsadm` | Optional but recommended | Optional but recommended | +| | Kubernetes Version ≥ 1.18 | +| ------------- | -------------------------- | +| `socat` | Required | +| `conntrack` | Required | +| `ebtables` | Optional but recommended | +| `ipset` | Optional but recommended | +| `ipvsadm` | Optional but recommended | * Networking and DNS requirements: * Make sure the DNS address in `/etc/resolv.conf` is available. Otherwise, it may cause some issues of DNS in cluster. @@ -93,19 +97,21 @@ KubeKey can install Kubernetes and KubeSphere together. The dependency that need ## Usage ### Get the KubeKey Executable File + * The fastest way to get KubeKey is to use the script: + ``` curl -sfL https://get-kk.kubesphere.io | sh - ``` * Binary downloads of the KubeKey also can be found on the [Releases page](https://github.com/kubesphere/kubekey/releases). Unpack the binary and you are good to go! - * Build Binary from Source Code - ```shell script - git clone https://github.com/kubesphere/kubekey.git - cd kubekey - make kk - ``` + + ```shell + git clone https://github.com/kubesphere/kubekey.git + cd kubekey + make kk + ``` ### Create a Cluster @@ -119,7 +125,7 @@ Quick Start is for `all-in-one` installation which is a good start to get famili > If you have problem to access `https://storage.googleapis.com`, execute first `export KKZONE=cn`. -```shell script +```shell ./kk create cluster [--with-kubernetes version] [--with-kubesphere version] ``` @@ -127,21 +133,19 @@ Quick Start is for `all-in-one` installation which is a good start to get famili * Create a pure Kubernetes cluster with default version (Kubernetes v1.23.7). - ```shell script - ./kk create cluster - ``` - + ```shell + ./kk create cluster + ``` * Create a Kubernetes cluster with a specified version. - ```shell script - ./kk create cluster --with-kubernetes v1.24.1 --container-manager containerd - ``` - + ```shell + ./kk create cluster --with-kubernetes v1.24.1 --container-manager containerd + ``` * Create a Kubernetes cluster with KubeSphere installed. - ```shell script - ./kk create cluster --with-kubesphere v3.2.1 - ``` + ```shell + ./kk create cluster --with-kubesphere v3.2.1 + ``` #### Advanced @@ -151,33 +155,34 @@ You have more control to customize parameters or create a multi-node cluster usi 1. First, create an example configuration file - ```shell script - ./kk create config [--with-kubernetes version] [--with-kubesphere version] [(-f | --filename) path] - ``` + ```shell + ./kk create config [--with-kubernetes version] [--with-kubesphere version] [(-f | --filename) path] + ``` **examples:** * create an example config file with default configurations. You also can specify the file that could be a different filename, or in different folder. - ```shell script - ./kk create config [-f ~/myfolder/abc.yaml] - ``` + ```shell + ./kk create config [-f ~/myfolder/abc.yaml] + ``` * with KubeSphere - ```shell script - ./kk create config --with-kubesphere v3.2.1 - ``` - + ```shell + ./kk create config --with-kubesphere v3.2.1 + ``` 2. Modify the file config-sample.yaml according to your environment + > Note: Since Kubernetes temporarily does not support uppercase NodeName, contains uppercase letters in workerNode`s name will lead to subsequent installation error -> +> > A persistent storage is required in the cluster, when kubesphere will be installed. The local volume is used default. If you want to use other persistent storage, please refer to [addons](./docs/addons.md). + 3. Create a cluster using the configuration file - ```shell script - ./kk create cluster -f config-sample.yaml - ``` + ```shell + ./kk create cluster -f config-sample.yaml + ``` ### Enable Multi-cluster Management @@ -189,12 +194,11 @@ KubeSphere has decoupled some core feature components since v2.1.0. These compon You can enable any of them according to your demands. It is highly recommended that you install these pluggable components to discover the full-stack features and capabilities provided by KubeSphere. Please ensure your machines have sufficient CPU and memory before enabling them. See [Enable Pluggable Components](https://github.com/kubesphere/ks-installer#enable-pluggable-components) for the details. - ### Add Nodes Add new node's information to the cluster config file, then apply the changes. -```shell script +```shell ./kk add nodes -f config-sample.yaml ``` @@ -202,7 +206,7 @@ Add new node's information to the cluster config file, then apply the changes. You can delete the node by the following command，the nodeName that needs to be removed. -```shell script +```shell ./kk delete node -f config-sample.yaml ``` @@ -212,42 +216,52 @@ You can delete the cluster by the following command: * If you started with the quick start (all-in-one): -```shell script +```shell ./kk delete cluster ``` * If you started with the advanced (created with a configuration file): -```shell script +```shell ./kk delete cluster [-f config-sample.yaml] ``` + ### Upgrade Cluster + #### Allinone + Upgrading cluster with a specified version. -```shell script + +```shell ./kk upgrade [--with-kubernetes version] [--with-kubesphere version] ``` + * Support upgrading Kubernetes only. * Support upgrading KubeSphere only. * Support upgrading Kubernetes and KubeSphere. #### Multi-nodes + Upgrading cluster with a specified configuration file. -```shell script + +```shell ./kk upgrade [--with-kubernetes version] [--with-kubesphere version] [(-f | --filename) path] ``` + * If `--with-kubernetes` or `--with-kubesphere` is specified, the configuration file will be also updated. * Use `-f` to specify the configuration file which was generated for cluster creation. > Note: Upgrading multi-nodes cluster need a specified configuration file. If the cluster was installed without kubekey or the configuration file for installation was not found, the configuration file needs to be created by yourself or following command. Getting cluster info and generating kubekey's configuration file (optional). -```shell script + +```shell ./kk create config [--from-cluster] [(-f | --filename) path] [--kubeconfig path] ``` -* `--from-cluster` means fetching cluster's information from an existing cluster. + +* `--from-cluster` means fetching cluster's information from an existing cluster. * `-f` refers to the path where the configuration file is generated. -* `--kubeconfig` refers to the path where the kubeconfig. +* `--kubeconfig` refers to the path where the kubeconfig. * After generating the configuration file, some parameters need to be filled in, such as the ssh information of the nodes. ## Documents @@ -269,9 +283,13 @@ Getting cluster info and generating kubekey's configuration file (optional). ## Contributors ✨ Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)): + + + + @@ -355,6 +373,7 @@ Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/d

pixiake
💻 📖
+ diff --git a/README_zh-CN.md b/README_zh-CN.md index 4253ebcd..3095b48b 100644 --- a/README_zh-CN.md +++ b/README_zh-CN.md @@ -30,19 +30,18 @@ * **CentOS/RHEL** *7* * **SUSE Linux Enterprise Server** *15* -> 建议使用 Linux Kernel 版本: `4.15 or later` \ +> 建议使用 Linux Kernel 版本: `4.15 or later` > 可以通过命令 `uname -srm` 查看 Linux Kernel 版本。 -### Kubernetes 版本 +### ``Kubernetes 版本 -* **v1.17**:   *v1.17.9* -* **v1.18**:   *v1.18.8* * **v1.19**:   *v1.19.9* * **v1.20**:   *v1.20.10* * **v1.21**:   *v1.21.13* * **v1.22**:   *v1.22.12* * **v1.23**:   *v1.23.9* (default) * **v1.24**:   *v1.24.3* + > 查看更多支持的版本[点击这里](./docs/kubernetes-versions.md) ## 要求和建议 @@ -59,7 +58,7 @@ * 所有节点的时间同步。 * `sudo`/`curl`/`openssl` 应在所有节点使用。 * `docker` 可以自己安装，也可以通过 KubeKey 安装。 - * `Red Hat` 在其 `Linux` 发行版本中包括了`SELinux`，建议[关闭SELinux](./docs/turn-off-SELinux_zh-CN.md)或者将[SELinux的模式切换](./docs/turn-off-SELinux_zh-CN.md)为Permissive[宽容]工作模式 + * `Red Hat` 在其 `Linux` 发行版本中包括了 `SELinux`，建议[关闭SELinux](./docs/turn-off-SELinux_zh-CN.md)或者将[SELinux的模式切换](./docs/turn-off-SELinux_zh-CN.md)为Permissive[宽容]工作模式 > * 建议您的操作系统环境足够干净 (不安装任何其他软件)，否则可能会发生冲突。 > * 如果在从 dockerhub.io 下载镜像时遇到问题，建议准备一个容器镜像仓库 (加速器)。[为 Docker 守护程序配置镜像加速](https://docs.docker.com/registry/recipes/mirror/#configure-the-docker-daemon)。 @@ -68,15 +67,15 @@ * 依赖要求: -KubeKey 可以同时安装 Kubernetes 和 KubeSphere。根据 KubeSphere 所安装版本的不同，您所需要安装的依赖可能也不同。请参考以下表格查看您是否需要提前在节点上安装有关的依赖。 +KubeKey 可以同时安装 Kubernetes 和 KubeSphere。在版本1.18之后，安装kubernetes前需要安装一些依赖。你可以参考下面的列表，提前在你的节点上检查并安装相关依赖。 -| | Kubernetes 版本 ≥ 1.18 | Kubernetes 版本 < 1.18 | -| ----------- | ---------------------- | ---------------------- | -| `socat` | 必须安装 | 可选，但推荐安装 | -| `conntrack` | 必须安装 | 可选，但推荐安装 | -| `ebtables` | 可选，但推荐安装 | 可选，但推荐安装 | -| `ipset` | 可选，但推荐安装 | 可选，但推荐安装 | -| `ipvsadm` | 可选，但推荐安装 | 可选，但推荐安装 | +| | Kubernetes 版本 ≥ 1.18 | +| ------------- | ----------------------- | +| `socat` | 必须安装 | +| `conntrack` | 必须安装 | +| `ebtables` | 可选，但推荐安装 | +| `ipset` | 可选，但推荐安装 | +| `ipvsadm` | 可选，但推荐安装 | * 网络和 DNS 要求： * 确保 `/etc/resolv.conf` 中的 DNS 地址可用。否则，可能会导致集群中出现某些 DNS 问题。 @@ -86,17 +85,16 @@ KubeKey 可以同时安装 Kubernetes 和 KubeSphere。根据 KubeSphere 所安 ### 获取安装程序可执行文件 -* 下载KubeKey可执行文件 [Releases page](https://github.com/kubesphere/kubekey/releases) +* 下载KubeKey可执行文件 [Releases page](https://github.com/kubesphere/kubekey/releases) 下载解压后可直接使用。 - * 从源代码生成二进制文件 - ```shell script - git clone https://github.com/kubesphere/kubekey.git - cd kubekey - ./build.sh - ``` + ```shell + git clone https://github.com/kubesphere/kubekey.git + cd kubekey + ./build.sh + ``` > 注意： > @@ -115,7 +113,7 @@ KubeKey 可以同时安装 Kubernetes 和 KubeSphere。根据 KubeSphere 所安 > 如果无法访问 `https://storage.googleapis.com`, 请先执行 `export KKZONE=cn`. -```shell script +```shell ./kk create cluster [--with-kubernetes version] [--with-kubesphere version] ``` @@ -123,26 +121,25 @@ KubeKey 可以同时安装 Kubernetes 和 KubeSphere。根据 KubeSphere 所安 * 使用默认版本创建一个纯 Kubernetes 集群 - ```shell script - ./kk create cluster - ``` - + ```shell + ./kk create cluster + ``` * 创建指定一个（[支持的版本](#KubernetesVersions)）的 Kubernetes 集群 - ```shell script - ./kk create cluster --with-kubernetes v1.19.8 - ``` - + ```shell + ./kk create cluster --with-kubernetes v1.19.8 + ``` * 创建一个部署了 KubeSphere 的 Kubernetes 集群 （例如 `--with-kubesphere v3.1.0`） - ```shell script - ./kk create cluster --with-kubesphere [version] - ``` + ```shell + ./kk create cluster --with-kubesphere [version] + ``` * 创建一个指定的 container runtime 的 Kubernetes 集群（docker, crio, containerd and isula） - ```shell script - ./kk create cluster --container-manager containerd - ``` + ```shell + ./kk create cluster --container-manager containerd + ``` + #### 高级用法 您可以使用高级安装来控制自定义参数或创建多节点集群。具体来说，通过指定配置文件来创建集群。 @@ -151,33 +148,33 @@ KubeKey 可以同时安装 Kubernetes 和 KubeSphere。根据 KubeSphere 所安 1. 首先，创建一个示例配置文件 - ```shell script - ./kk create config [--with-kubernetes version] [--with-kubesphere version] [(-f | --filename) path] - ``` + ```shell + ./kk create config [--with-kubernetes version] [--with-kubesphere version] [(-f | --filename) path] + ``` **例子：** * 使用默认配置创建一个示例配置文件。您也可以指定文件名称或文件所在的文件夹。 - ```shell script - ./kk create config [-f ~/myfolder/config-sample.yaml] - ``` - + ```shell + ./kk create config [-f ~/myfolder/config-sample.yaml] + ``` * 同时安装 KubeSphere - ```shell script - ./kk create config --with-kubesphere - ``` - + ```shell + ./kk create config --with-kubesphere + ``` 2. 根据您的环境修改配置文件 config-sample.yaml + > 注意： 由于 Kubernetes 暂不支持大写 NodeName， worker 节点名中包含大写字母将导致后续安装过程无法正常结束 > > 当指定安装KubeSphere时，要求集群中有可用的持久化存储。默认使用localVolume，如果需要使用其他持久化存储，请参阅 [addons](./docs/addons.md) 配置。 + 3. 使用配置文件创建集群。 - ```shell script - ./kk create cluster -f ~/myfolder/config-sample.yaml - ``` + ```shell + ./kk create cluster -f ~/myfolder/config-sample.yaml + ``` ### 启用多集群管理 @@ -193,14 +190,15 @@ KubeSphere 有多个可插拔功能组件，功能组件的介绍可参考 [配 将新节点的信息添加到集群配置文件，然后应用更改。 -```shell script +```shell ./kk add nodes -f config-sample.yaml ``` + ### 删除节点 通过以下命令删除节点，nodename指需要删除的节点名。 -```shell script +```shell ./kk delete node -f config-sample.yaml ``` @@ -210,30 +208,37 @@ KubeSphere 有多个可插拔功能组件，功能组件的介绍可参考 [配 * 如果您以快速入门（all-in-one）开始： -```shell script +```shell ./kk delete cluster ``` * 如果从高级安装开始（使用配置文件创建的集群）： -```shell script +```shell ./kk delete cluster [-f config-sample.yaml] ``` ### 集群升级 + #### 单节点集群 + 升级集群到指定版本。 -```shell script + +```shell ./kk upgrade [--with-kubernetes version] [--with-kubesphere version] ``` + * `--with-kubernetes` 指定kubernetes目标版本。 * `--with-kubesphere` 指定kubesphere目标版本。 #### 多节点集群 + 通过指定配置文件对集群进行升级。 -```shell script + +```shell ./kk upgrade [--with-kubernetes version] [--with-kubesphere version] [(-f | --filename) path] ``` + * `--with-kubernetes` 指定kubernetes目标版本。 * `--with-kubesphere` 指定kubesphere目标版本。 * `-f` 指定集群安装时创建的配置文件。 @@ -241,12 +246,14 @@ KubeSphere 有多个可插拔功能组件，功能组件的介绍可参考 [配 > 注意: 升级多节点集群需要指定配置文件. 如果集群非kubekey创建，或者创建集群时生成的配置文件丢失，需要重新生成配置文件，或使用以下方法生成。 Getting cluster info and generating kubekey's configuration file (optional). -```shell script + +```shell ./kk create config [--from-cluster] [(-f | --filename) path] [--kubeconfig path] ``` -* `--from-cluster` 根据已存在集群信息生成配置文件. + +* `--from-cluster` 根据已存在集群信息生成配置文件. * `-f` 指定生成配置文件路径. -* `--kubeconfig` 指定集群kubeconfig文件. +* `--kubeconfig` 指定集群kubeconfig文件. * 由于无法全面获取集群配置，生成配置文件后，请根据集群实际信息补全配置文件。 ### 启用 kubectl 自动补全 @@ -255,7 +262,7 @@ KubeKey 不会启用 kubectl 自动补全功能。请参阅下面的指南并将 **先决条件**：确保已安装 `bash-autocompletion` 并可以正常工作。 -```shell script +```shell # 安装 bash-completion apt-get install bash-completion @@ -284,8 +291,11 @@ kubectl completion bash >/etc/bash_completion.d/kubectl 欢迎任何形式的贡献! 感谢这些优秀的贡献者，是他们让我们的项目快速成长。 + + + @@ -369,6 +379,7 @@ kubectl completion bash >/etc/bash_completion.d/kubectl

pixiake
💻 📖
+ diff --git a/docs/ha-mode.md b/docs/ha-mode.md index 2f7583c9..9e245757 100644 --- a/docs/ha-mode.md +++ b/docs/ha-mode.md @@ -1,17 +1,23 @@ # HA mode (internal loadbalancing) -K8s components require a loadbalancer to access the apiservers via a reverse proxy. Kubekey provides an internal ha mode. The way kubekey uses is referred to as localhost loadbalancing. The kubelet of each master node connects the local kube-apiserver, and the kubelet of each worker node connects the kube-apiserver via a local reverse proxy. Based on this, kubekey will deploy a haproxy-based proxy that resides on each worker node as the local reverse proxy. +K8s components require a loadbalancer to access the apiservers via a reverse proxy. Kubekey uses **kube-vip** and **haproxy** to provide internal ha mode. +## haproxy +The way kubekey uses is referred to as localhost loadbalancing. The kubelet of each master node connects the local kube-apiserver, and the kubelet of each worker node connects the kube-apiserver via a local reverse proxy. Based on this, kubekey will deploy a haproxy-based proxy that resides on each worker node as the local reverse proxy. -![Image](img/internalLoadBalancer.png?raw=true) +![Image](img/haproxy.png?raw=true) + +## kube-vip +The load balancing is provided through IPVS (IP Virtual Server) and provides a Layer 4 (TCP-based) round-robin across all of the control plane nodes. By default, the load balancer will listen on the default port of 6443 as the Kubernetes API server. The IPVS virtual server lives in kernel space and doesn't create an "actual" service that listens on port 6443. This allows the kernel to parse packets before they're sent to an actual TCP port. Based on this, kubekey will deploy a static pod that resides on each control-plane node as the internal loadbalancing. + +![Image](img/kube-vip.png?raw=true) ## Usage Modify your configuration file and uncomment the item `internalLoadbalancer`: ```yaml controlPlaneEndpoint: - ##Internal loadbalancer for apiservers - internalLoadbalancer: haproxy + internalLoadbalancer: haproxy #Internal loadbalancer for apiservers. Support: haproxy, kube-vip [Default: ""] - domain: lb.kubesphere.local - address: "" + domain: lb.kubesphere.local + address: "" # The IP address of your load balancer. If you use internalLoadblancer in "kube-vip" mode, a VIP is required here. port: 6443 ``` diff --git a/docs/img/internalLoadBalancer.png b/docs/img/haproxy.png similarity index 100% rename from docs/img/internalLoadBalancer.png rename to docs/img/haproxy.png diff --git a/docs/img/kube-vip.png b/docs/img/kube-vip.png new file mode 100644 index 00000000..9fa796b6 Binary files /dev/null and b/docs/img/kube-vip.png differ