This section explains how to install Kubernetes and KubeSphere in an Internet-accessible environment.

The installation process will use the open-source tool KubeKey. For more information about KubeKey, please visit the GitHub KubeKey repository.

Prerequisites

  • Prepare at least 1 Linux server as a cluster node. In a production environment, to ensure high availability of the cluster, it is recommended to prepare at least 5 Linux servers, with 3 servers as control plane nodes and 2 servers as worker nodes. If you are installing KubeSphere on multiple Linux servers, make sure all servers belong to the same subnet.

  • The operating system and version of the cluster nodes must be Ubuntu 16.04, Ubuntu 18.04, Ubuntu 20.04, Ubuntu 22.04, Debian 9, Debian 10, CentOS 7, CentOS Stream, RHEL 7, RHEL 8, SLES 15, openSUSE Leap 15, or Kylin v10. The operating systems of multiple servers can be different. For support of other operating systems and versions, please consult QingCloud’s official solution experts or delivery service experts.

  • In a production environment, to ensure the cluster has sufficient computing and storage resources, it is recommended that each cluster node be configured with at least 8 CPU cores, 16 GB of memory, and 200 GB of disk space. In addition, it is recommended to mount an additional 200 GB of disk space in the /var/lib/docker (for Docker) or /var/lib/containerd (for containerd) directory of each cluster node for storing container runtime data.

  • In a production environment, it is recommended to configure high availability for the KubeSphere cluster in advance to avoid service interruption in the event of a single control plane node failure. For more information, please refer to the Configure High Availability.

    Note

    If you plan to have multiple control plane nodes, be sure to configure high availability for the cluster in advance.

  • By default, KubeSphere uses the local disk space of the cluster nodes as persistent storage. In a production environment, it is recommended to configure an external storage system as persistent storage in advance. For more information, please refer to the Configure External Persistent Storage.

  • If the cluster nodes do not have a container runtime installed, the installation tool KubeKey will automatically install Docker as the container runtime for each cluster node during the installation process. You can also manually install containerd, CRI-O, or iSula as the container runtime in advance.

    Note

    CRI-O and iSula have not been fully tested for compatibility with KubeSphere, and there may be unknown issues.

  • Make sure the DNS server addresses configured in the /etc/resolv.conf file on all cluster nodes are available. Otherwise, the KubeSphere cluster may encounter domain name resolution issues.

  • Make sure you can use the sudo, curl, and openssl commands on all cluster nodes.

  • Make sure the time is synchronized on all cluster nodes.

Configure Firewall Rules

KubeSphere requires specific ports and protocols for communication between services. If your infrastructure environment has enabled a firewall, you need to open the required ports and protocols in the firewall settings. If your infrastructure environment does not have a firewall enabled, you can skip this step.

The following table lists the ports and protocols that need to be opened in the firewall.

Service Protocol Action Start Port End Port Remarks

ssh

TCP

allow

22

N/A

N/A

etcd

TCP

allow

2379

2380

N/A

apiserver

TCP

allow

6443

N/A

N/A

calico

TCP

allow

9099

9100

N/A

bgp

TCP

allow

179

N/A

N/A

nodeport

TCP

allow

30000

32767

N/A

master

TCP

allow

10250

10258

N/A

worker

TCP

allow

10250

N/A

N/A

dns

TCP

allow

53

N/A

N/A

dns

UDP

allow

53

N/A

N/A

local-registry

TCP

allow

5000

N/A

Required for offline environments

local-apt

TCP

allow

5080

N/A

Required for offline environments

rpcbind

TCP

allow

111

N/A

Required when using NFS as persistent storage

ipip

IPENCAP / IPIP

allow

N/A

N/A

Required when using Calico

Install Dependencies

You need to install socat, conntrack, ebtables, and ipset for all cluster nodes. If the above dependencies already exist on each cluster node, you can skip this step.

On Ubuntu systems, run the following command to install the dependencies on the servers:

sudo apt install socat conntrack ebtables ipset -y

If the cluster nodes use other operating systems, replace apt with the corresponding package management tool for the operating system.

Create a Kubernetes Cluster

  1. If you have restricted access to GitHub/Googleapis, log in to any cluster node and execute the following command to set the download region:

    export KKZONE=cn
  2. Execute the following command to download the latest version of KubeKey:

    curl -sfL https://get-kk.kubesphere.io | sh -

    After downloading, the KubeKey binary file kk will be generated in the current directory.

  3. Execute the following command to add execute permission to the KubeKey binary file kk:

    sudo chmod +x kk
  4. Run the following command to create the installation configuration file config-sample.yaml:

     ./kk create config --with-kubernetes <Kubernetes version>

    Replace <Kubernetes version> with the actual required version, for example v1.27.4. KubeSphere natively supports Kubernetes v1.23~1.32.

    After the command executes, it will generate the installation configuration file config-sample.yaml.

    Note

    After installation, do not delete the installation configuration file config-sample.yaml, as it will still be needed for subsequent operations such as adding nodes. If this file is lost, you will need to recreate the installation configuration file.

  5. Run the following command to edit the installation configuration file config-sample.yaml:

    vi config-sample.yaml

    The following is a part of the configuration file sample. For a complete example, please refer to this file.

    apiVersion: kubekey.kubesphere.io/v1alpha2
    kind: Cluster
    metadata:
      name: sample
    spec:
      hosts:
      - {name: controlplane1, address: 192.168.0.2, internalAddress: 192.168.0.2, port: 23, user: ubuntu, password: Testing123, arch: arm64} # For arm64 nodes, please add the parameter arch: arm64
      - {name: controlplane2, address: 192.168.0.3, internalAddress: 192.168.0.3, user: ubuntu, privateKeyPath: "~/.ssh/id_rsa"}
      - {name: worker1, address: 192.168.0.4, internalAddress: 192.168.0.4, user: ubuntu, password: Testing123}
      - {name: worker2, address: 192.168.0.5, internalAddress: 192.168.0.5, user: ubuntu, password: Testing123}
      - {name: registry, address: 192.168.0.6, internalAddress: 192.168.0.6, user: ubuntu, password: Testing123}
      roleGroups:
        etcd:
        - controlplane1
        - controlplane2
        control-plane:
        - controlplane1
        - controlplane2
        worker:
        - worker1
        - worker2
        # If you want to use kk to automatically deploy the image registry, please set up the registry (it is recommended that the image registry and cluster nodes be deployed separately to reduce mutual influence)
        registry:
        - registry
      controlPlaneEndpoint:
        internalLoadbalancer: haproxy # If you need to deploy a high availability cluster and no load balancer is available, you can enable this parameter to perform load balancing within the cluster.
        domain: lb.kubesphere.local
        address: ""
        port: 6443
      kubernetes:
        version: v1.23.15
        clusterName: cluster.local
      network:
        plugin: calico
        kubePodsCIDR: 10.233.64.0/18
        kubeServiceCIDR: 10.233.0.0/18
        ## multus support. https://github.com/k8snetworkplumbingwg/multus-cni
        enableMultusCNI: false
      registry:
        # If you want to use kk to deploy harbor, you can set this parameter to harbor. If you do not set this parameter and you need to use kk to deploy the image registry, docker registry will be deployed by default.
        # Harbor does not support arm64. This parameter does not need to be configured when deploying in an arm64 environment.
        type: harbor
        # If you use kk to deploy harbor or other registries that require authentication, you need to set the auths of the corresponding registries. If you use kk to deploy the default docker registry, you do not need to configure the auths parameter.
        # Note: If you use kk to deploy harbor, please set the auths parameter after creating the harbor project.
        auths:
          "dockerhub.kubekey.local":
              username: admin # harbor default username
              password: Harbor12345 # harbor default password
              plainHTTP: false # If the registry uses http, please set this parameter to true
        privateRegistry: "dockerhub.kubekey.local/kse"   # Set the private registry address used during cluster deployment.
        registryMirrors: []
        insecureRegistries: []
      addons: []
  6. Set the information of each server under the spec:hosts parameter in config-sample.yaml.

    Parameter Description

    name

    The user-defined server name.

    address

    The SSH login IP address of the server.

    internalAddress

    The internal IP address of the server within the subnet.

    port

    The SSH port number of the server. If using the default port 22, this parameter can be omitted.

    user

    The SSH login username for the server. This user must be the root user or another user with sudo command execution privileges. If using the root user, this parameter can be omitted.

    password

    The SSH login password for the server. If privateKeyPath is already set, this parameter can be omitted.

    privateKeyPath

    The path to the SSH login private key for the server. If password is already set, this parameter can be omitted.

    arch

    The hardware architecture of the server. If the server’s hardware architecture is arm64, set this parameter to arm64; otherwise, do not set it. The installation package by default only supports scenarios where all cluster nodes are of x86_64 or arm64 architecture. If the hardware architecture of cluster nodes is not identical, technical assessment is required for the specific situation. Please consult QingCloud official solution experts or delivery service experts.

  7. Set the server’s role under the spec:roleGroups parameter in config-sample.yaml.

    Parameter Description

    etcd

    Nodes for installing the etcd database. Set the cluster control plane nodes under this parameter.

    control-plane

    Cluster control plane nodes. If you have configured high availability for the cluster, you can set multiple control plane nodes.

    worker

    Cluster worker nodes.

    registry

    Server used to create a private image service. This server will not be used as a cluster node. When installing or upgrading KubeSphere, if the cluster nodes cannot connect to the internet, you need to set the server used to create a private image service under this parameter. In other cases, please comment out this parameter.

  8. If you have multiple control plane nodes, set high availability information under the spec:controlPlaneEndpoint parameter in config-sample.yaml.

    Parameter Description

    internalLoadbalancer

    The type of the local load balancer. If using a local load balancer configuration, set this parameter to haproxy. Otherwise, comment out this parameter.

    domain

    The internal access domain name of the load balancer. Set this parameter to lb.kubesphere.local.

    address

    The IP address of the load balancer. If using a local load balancer configuration, leave this parameter empty; if using a dedicated load balancer, set this parameter to the IP address of the load balancer; if using a general server as a load balancer, set this parameter to the floating IP address of the load balancer.

    port

    The port number the load balancer listens on, which is the port number of the apiserver service. Set this parameter to 6443.

  9. If you need to use external persistence storage, set the external persistence storage information under the spec:addons parameter in config-sample.yaml.

    • If using a cloud storage device, set the following parameters under spec:addons (replace <configuration file path> with the actual path of the storage plugin configuration file):

        - name: csi-qingcloud
          namespace: kube-system
          sources:
            chart:
              name: csi-qingcloud
              repo: https://charts.kubesphere.io/test
              valuesFile: <configuration file path>
    • If using NeonSAN storage, set the following parameters under spec:addons (replace <configuration file path> with the actual path of the storage plugin configuration file):

        - name: csi-neonsan
          namespace: kube-system
          sources:
            chart:
              name: csi-neonsan
              repo: https://charts.kubesphere.io/test
              valuesFile: <configuration file path>
    • If using an NFS storage system, set the following parameters under spec:addons (replace <configuration file path> with the actual path of the storage plugin configuration file):

        - name: nfs-client
          namespace: kube-system
          sources:
            chart:
              name: nfs-client-provisioner
              repo: https://charts.kubesphere.io/main
              valuesFile: <configuration file path>
  10. Run the following command to create the Kubernetes cluster:

     ./kk create cluster -f config-sample.yaml
    Note

    If you want to use OpenEBS LocalPV, you can add the --with-local-storage parameter after the command. If you want to integrate with other storage solutions, configure the relevant storage plugins under the spec:addons parameter in config-sample.yaml or install them after the Kubernetes deployment.

    If you see the following information, it means that the Kubernetes cluster is successfully created.

    Pipeline[CreateclusterPipeline] execute successfully

Install KubeSphere

KubeSphere Core (ks-core) is the core component of KubeSphere, providing the fundamental runtime environment for extensions. After KubeSphere Core is installed, you can access the KubeSphere web console.

  1. On the cluster nodes, run the following command to install KubeSphere Core.

    chart=oci://hub.kubesphere.com.cn/kse/ks-core
    version=1.2.3-20251118
    helm upgrade --install -n kubesphere-system --create-namespace ks-core $chart --debug --wait --version $version --reset-values

    If you have restricted access to Docker Hub, add the following configuration to the above command to modify the image pull address for extensions.

    --set extension.imageRegistry=swr.cn-north-9.myhuaweicloud.com/ks

    If you see the following information, it means that ks-core installation is successful:

    NOTES:
    Thank you for choosing KubeSphere Helm Chart.
    
    Please be patient and wait for several seconds for the KubeSphere deployment to complete.
    
    1. Wait for Deployment Completion
    
        Confirm that all KubeSphere components are running by executing the following command:
    
        kubectl get pods -n kubesphere-system
    
    2. Access the KubeSphere Console
    
        Once the deployment is complete, you can access the KubeSphere console using the following URL:
    
        http://192.168.6.10:30880
    
    3. Login to KubeSphere Console
    
        Use the following credentials to log in:
    
        Account: admin
        Password: P@88w0rd
    
    NOTE: It is highly recommended to change the default password immediately after the first login.
    
    For additional information and details, please visit https://kubesphere.io.
  2. From the successful information, retrieve the Console, Account, and Password parameters to obtain the IP address of the KubeSphere web console, the admin username, and the admin password. Use a web browser to log in to the KubeSphere web console.

    Note

    Depending on your network environment, you may need to configure traffic forwarding rules and open port 30880 in the firewall.

  3. By now, the KubeSphere web console only provides the core functionalities of KubeSphere. To access more features, you need to install extensions from the Extensions Center.

    Before using KubeSphere and its extensions, please first activate KubeSphere and extensions.