External Secrets Managers

Comet supports integration with services like Azure Key Vault and AWS Secrets Manager to enable external management of secrets.

AWS¶

To configure Comet to use a preexisting credential stored in AWS Secrets Manager, some configuration is required on the Comet side.

To configure comet, we'll need to set the following values by running cometctl aio update-config Note that the aws_sm_key and aws_sm_secret expect the string IAM-ROLE as provided. Not a Role ARN.

comet_aws_sm_enabled: true
aws_sm_keys: '["$AWS_SM_SECRET_NAME","$AWS_SM_SECRET_NAME_2"]'
aws_sm_keys_vars: '["COMET_ENV_VAR_IN_ALL_CAPS","COMET_ENV_VAR_IN_ALL_CAPS2"]'
aws_sm_arn: "arn:aws:secretsmanager:{$REGION}:276069367280:secret:/application/elasticache-token-comet-common-redis-cluster-5JwKB5"
aws_sm_region: "us-east-1"
aws_sm_key: "IAM-ROLE"
aws_sm_secret: "IAM-ROLE"

If you are setting these values for a Kubernetes deployment, the values for keys and secret should be set to the following string:

aws_sm_key: "IAM-ROLE-SERVICE-ACCOUNT"
aws_sm_secret: "IAM-ROLE-SERVICE-ACCOUNT"

If necessary you can have the AWS_SM_KEYS value be whatever keyname you already have the credential stored under, only the AWS_SM_KEYS_VARS variables will be required to correspond exactly to those used by Comet.

After you're finished, run cometctl aio update-config.

Optimizer¶

Comet can be configured to source the Redis URL only when stored in AWS Secrets Manager, however, some configuration is required on the Comet side.

To configure Comet, set the following variables in /etc/default/comet-ml-optimizer or your Kubernetes config map:

AWS_SM_OPTIMIZER_REDIS_URL_SECRET_NAME="SecretName"
AWS_SM_OPTIMIZER_KEY="REDIS_URL_KEY"
AWS_SM_OPTIMIZER_REGION="us-east-1"

If you have the REDIS_URL set in /etc/default/comet-ml-optimizer, it must be removed entirely, as it will take precedence over any remotely sourced URL even when set to a blank string.

Azure¶

To configure Comet to use Azure Key Vault as a secrets manager, some configuration is required on the Azure side as well as the Comet side.

Assuming a preexisting VM and Keyvault, you will need to create an App Registration for Azure Active Directory so that Comet can authenticate to your Keyvault and retrieve secrets from it. Take note of the Client ID and create a Secret to authenticate with. Copy this secret and store it, as you will not be able to view the secret after creating it.

Once complete, navigate to your keyvault in the Azure Portal and select Access Policy from the menu.

Click Add Access Policy and select a template, or you may also give permissions manually.

For the principal you will select the App Registration you've just created.

Add the access policy and remember to save your changes.

To configure Comet, we'll need to set the following values by running cometctl aio update-config

comet_azure_sm_enabled: true
azure_key_vault_uri: "https://$REPLACEME.vault.azure.net/"
azure_tenant_id: [`"$AZURE_SUBSCRIPTION"`](https://docs.microsoft.com/en-us/azure/active-directory/fundamentals/active-directory-how-to-find-tenant)
azure_sm_keys: '["$AZURE_KV_SECRET_NAME","$AZURE_KV_SECRET_NAME_2"]'
azure_sm_keys_vars: '["COMET_ENV_VAR_IN_ALL_CAPS","COMET_ENV_VAR_IN_ALL_CAPS2"]'
azure_client_id: "$MY_APP_REGISTRATION_CLIENT_ID"
azure_client_secret: "$MY_APP_REGISTRATION_SECRET"

As Azure Key Vault doesn't allow underscores in key names there's a need to provide a mapping to our internal environment variable names. If necessary you can have the AZURE_SM_KEYS value be whatever keyname you already have the credential stored under, only the AZURE_SM_KEYS_VARS variables will be required to correspond exactly to those used by Comet.

After you're finished, run cometctl aio update-config.

GCP¶