We are now ready to create our cluster!

Here’s how to get started:

Option 1: Create A New Cluster

  1. Download the Spotinst’s KOPS binary.
  2. Verify the required scripts from prerequisites are available. If not, download the updated scripts and open the archive file.
  3. Set your required cluster variables in the downloaded 00-env.sh.
  4. Execute 01-create.sh.

 

Option 2: Migrate an existing cluster managed by AWS Auto Scaling Groups to Spotinst Ocean

Migration of existing deployment will consist of two separate layers: master nodes and worker nodes’ Instance Group(s). Begin with the master nodes then import the workers

NOTES:

1. Ensure that you are in the correct cluster context for kubectl.

2. If you are importing a cluster, the cluster name must be the same as the primary instance group name.

3. Before doing a workload migration, ensure that there are no unscheduled pods. If there are unscheduled pods, the migration will stop.

    1. Master nodes: to perform the migration with no downtime, migrate the masters one-by-one. Choose the scenario below that matches the current deployment
      1. Scenario 1: For deployments of master nodes in individual Auto Scaling Groups
        1. For each master ASG:
          1. Drain the node 
            1. kubectl drain <node>
          2. Scale down the Auto Scaling Group by one instance 
            1. Set target, min, max to 0,0,0
          3. Import the Auto Scaling Group into Spotinst Elastigroup. Make sure to specify the following during Elastigroup creation:
            1. The Elastigroup’s name equals by default to the Auto Scaling Group name, don’t change its name. 
            2. Masters are recommended to run as On-Demand instances. 
              • Switch to the General tab, and under Advanced set the Spot Percentage to 0.
        2. Repeat the steps with the remaining master nodes, until all nodes are running in Spotinst, and all masters Auto Scaling Groups were downscaled to zero. 
      2. Scenario 2: If all masters are running in the same Auto Scaling Group 
        1. Drain one master node in the ASG 
          1. kubectl drain <node>
        2. Scale down the Auto Scaling Group by one instance 
          1. Set target, min, max to 1 less than before.
        3. Import the Auto Scaling Group into Spotinst Elastigroup. Make sure to specify the following during Elastigroup creation:
          1. The Elastigroup’s name equals by default to the Auto Scaling Group name, don’t change its name. 
          2. Masters are recommended to run as On-Demand instances. 
            1. Switch to the General tab, and under Advanced set the Spot Percentage to 0.
          3. Initially set target, min, max to 1,1,1.
          4. Verify new node in Elastigroup has joined the K8s cluster
        4. Repeat steps ii 1-2 for the next master node.
        5. Increase the capacity of your master Elastigroup by 1.
        6. Repeat steps 4-5 for remaining master(s).
    2. Enable Spotinst support by toggling the feature flag. On the management station where KOPS commands are run:
      1. export KOPS_FEATURE_FLAGS="+Spotinst,SpotinstOcean"
    3. Make sure that Spotinst environment variables are defined on the KOPS management server:
      1. SPOTINST_TOKEN and SPOTINST_ACCOUNT
    4. Worker Nodes
      1. In the case of multiple worker nodes Instance Groups, set the Ocean default launch specification label on your primary Instance Group. Example:
        1. apiVersion: kops/v1alpha2
          kind: InstanceGroup
          metadata:
           labels:
             kops.k8s.io/cluster: kops.k8s.com
             spotinst.io/ocean-default-launchspec: "true"
           name: nodes
          spec:
          ...
      2. In a case of a single Instance Group there is no need to add any labels.
      3. Run KOPS Update.
        1. KOPS will now identify and create the missing launch specifications on your Ocean Cluster.
      4. Gradually scale down your Auto Scaling Groups
        1. Ocean will automatically provision the required instances to ensure your pods are scheduled.
      5. Clean up (optional):
        1. Delete all Auto Scaling Groups.
        2. Delete all Launch Configurations.

Option 3: Hybrid - Migrate just the worker nodes of an existing cluster managed by AWS Auto Scaling Groups to Spotinst Ocean

This section describes how to perform a hybrid migration. This enables you to migrate only some or all of the worker instance groups, while leaving the master nodes, and any remaining worker node groups to be managed in an AWS ASG. To that end, we will skip migrating the Masters nodes. Proceed with the following steps:

  1. Enable Spotinst support by toggling the feature flag on the management station where KOPS commands are run:
    1. export KOPS_FEATURE_FLAGS="+Spotinst,SpotinstOcean,SpotinstHybrid"
  2. Make sure that Spotinst environment variables are defined on the KOPS management server:
    1. SPOTINST_TOKEN and SPOTINST_ACCOUNT
  3. Worker Nodes migration
    1. In the case of multiple worker nodes Instance Groups:
      1. Set the Ocean default launch specification label  and the spotinst.io/hybrid: "true" label on your primary Instance Group. Example:

        apiVersion: kops/v1alpha2
        kind: InstanceGroup 
        metadata: 
         labels:
           kops.k8s.io/cluster: kops.k8s.com
           spotinst.io/hybrid: "true"
           spotinst.io/ocean-default-launchspec: "true"
         name: nodes 
        spec:
        ...
      2. Label the rest of the instance groups you want to migrate to Ocean with the label spotinst.io/hybrid: "true". Example:
        apiVersion: kops/v1alpha2
        kind: InstanceGroup
        metadata:
          creationTimestamp: null
          labels:
            kops.k8s.io/cluster: kops.ek8s.com
            spotinst.io/hybrid: "true"
          name: nodes
        spec:
        ...
      3. Run KOPS Update
        1. KOPS will now identify and create the Ocean Cluster along with the required launch specifications.
      4. Gradually scale down your Auto Scaling Groups
        1. Ocean will automatically provision the required instances to ensure your pods are scheduled.
    2. In a case of a single Instance Group:
      1. Label the instance group you wish to migrate to Ocean with the label spotinst.io/hybrid: "true". Example:
        apiVersion: kops/v1alpha2
        kind: InstanceGroup
        metadata:
          creationTimestamp: null
          labels:
            kops.k8s.io/cluster: kops.ek8s.com
            spotinst.io/hybrid: "true"
          name: nodes
        spec:
        ...
      2. Run KOPS Update. 
        1. KOPS will now identify and create the Ocean Cluster.
      3. Gradually scale down your Auto Scaling Group
        1. Ocean will automatically provision the required instances to ensure your pods are scheduled.
    3. Clean up (optional):
      1. Delete imported Auto Scaling Groups.
      2. Delete all Launch Configurations.