Upgrading Comet on Kuberntes¶

Upgrading Comet using the Helm Chart¶

Generally-speaking, it is not necessary to upgrade the helm chart in order to upgrade your version of Comet.

All that is required is to update your override values passed to the helm chart to specify the version(s) to which you would like to upgrade.

# ...
frontend:
  frontendVersion: x.x.x
  # ...
backend:
  backendVersion: x.x.x
  # ...
  optimizer:
    # ...
    version: x.x.x
    # ...
# ...

Then run the helm upgrade to apply the change (or run it using your CI/CD tool):

helm upgrade <release-name> comet-ml/comet-ml --version <your current version of the helm chart> -f override-values.yaml

Or, if yuou are using a local copy of the helm chart:

helm upgrade <release-name> . -f override-values.yaml

Using Private Image Registries¶

If you install Comet using a private image registry, make sure that the versions you are specifying have been uploaded to the registry for all the relevant Comet images.

Please reach out to Comet Deployment Support if you need help acquiring these images to upload to your private registry.

Upgrading the Comet ML Helm Chart¶

Normal Upgrade Steps¶

helm upgrade <release-name> comet-ml/comet-ml -f override-values.yaml

Optionally, you can specify the version of the helm chart to upgrade to:

helm upgrade <release-name> comet-ml/comet-ml --version <x.x.x> -f override-values.yaml

From <=4.0.x to >=4.1.x¶

Chart Name changed to 'comet-ml', so update repository pull accordingly.

Due to a change in label selectors, it is not possible to execute a helm upgrade, instead it is necessary to uninstall and reinstall the chart.

Internal Redis¶

A change in how the Internal Redis is installed requires moving some non-default configurations. The following shows to where certain custom settings have been moved.

# Prior to 4.1.0
redis:
  enableInternalRedis: true
  image:
    repoistory: docker.io/redis
    tag: 5.0.7
  mpm:
    redisHost: "redis-internal"
    redisPort: "6379"
    redisToken: "NA"
    backupPath: "/opt/comet-ml/backups/redis/regular"
    maxTotalWorkers: "60"
    maxWaitTimeMs: "5000"
    redisSsl: false
    readTimeoutMs: "2000"
  persistenceVolume:
    storageClassName: "comet-generic"
    redisInternalVolumeSize: "10Gi"
  service:
    type: ClusterIP
    port: 6379
  nodeSelector: {}
  tolerations: []
  affinity: {}

Becomes:

# From 4.1.0
backend:
  mpm:
    # …
    redis:
      backupPath: "/opt/comet-ml/backups/redis/regular"
      maxTotalWorkers: "60"
      maxWaitTimeMs: "5000"
      readTimeoutMs: "2000"
# …
redis:
  enableInternalRedis: true
  redisHost: "redis-internal"
  redisPort: "6379"
  redisToken: "NA"
  redisSSL: false
redisInternal:
  image:
    registry: docker.io
    repoistory: redis
    tag: 5.0.7
  master:
    service:
      type: ClusterIP
      ports:
        redis: 6379
    persistence:
      enabled: true
      storageClass: "comet-generic"
      size: 10Gi
    nodeSelector: {}
    tolerations: []
    affinity: {}

Be sure to compare your override values to the defaults in values.yaml or those emitted from helm show values comet-ml/comet-ml as it is not necessary to override values when not deviating from the default.