Configuration
This page presents the full range of configuration options for controlling how your data is logged to Comet.
There are additional configuration parameters that are specific to Comet integrations with other tools and frameworks. For details, see the Comet documentation on the relevant integration.
Order of precedence for configuration parameters¶
Comet prioritizes setting configuration parameters in the following order:
- Arguments passed to the Experiment object, for example,
Experiment(api_key="Your API Key", disable_logging=True) - Environment parameters, for example,
export COMET_API_KEY=<Your API Key> - A setting in the
.comet.configfile in your current folder - A setting in the
.comet.configfile in yourHOMEdirectory
In other words, an explicit parameter passed to the Experiment object has top priority, followed by parameters set through environment parameters, followed by a .comet.config file in your current working directory, followed by the .comet.config file in your HOME directory.
Configuration parameters¶
These parameters control behaviors of the SDK that are not related to logging Experiment data.
The table below outlines all the available configuration environment parameters and their corresponding names in the .comet.config file.
| Comet Config Section/Name | Environment Parameter Name | Description |
|---|---|---|
| [comet] api_key | COMET_API_KEY | The API key used for creating Experiments and using the Python API |
| [comet] workspace | COMET_WORKSPACE | Use this workspace when creating new experiments |
| [comet] project_name | COMET_PROJECT_NAME | Use this project when creating new experiments |
| [comet] url_override | COMET_URL_OVERRIDE | URL to upload the experiments to, this needs to be updated for on-premise deployments. Defaults to https://www.comet.com/clientlib/. |
| [comet] offline_directory | COMET_OFFLINE_DIRECTORY | Set the offline directory for OfflineExperiment(). Defaults to .cometml-runs. |
| [comet] git_directory | COMET_GIT_DIRECTORY | Set the directory that contains your git repo, if you are not under it |
| [comet] display_summary_level | COMET_DISPLAY_SUMMARY_LEVEL | Controls the level of detail displayed in the experiment summary. If set to 0, the summary is hidden, but the summary notification is still sent. If set to 2, the experiment summary shows everything that has been logged. Defaults to 1. |
| [comet] rich_output | COMET_RICH_OUTPUT | When enabled, Comet console log messages will be rendered using formatted methods from the Rich library. Defaults to True. |
| [comet] system_cpu_interval | COMET_SYSTEM_CPU_INTERVAL | Controls the interval at which system CPU metrics (CPU utilization and memory utilization) are logged. The value must be in seconds and cannot be less than 1s. Defaults to 60. |
| [comet] system_gpu_interval | COMET_SYSTEM_GPU_INTERVAL | Controls the interval at which system GPU metrics are logged. The value must be in seconds and cannot be less than 1s. Defaults to 60. |
| [comet] system_network_interval | COMET_SYSTEM_NETWORK_INTERVAL | Controls the interval at which system network metrics are logged. The value must be in seconds and cannot be less than 1s. Defaults to 60. |
| [comet] system_disk_interval | COMET_SYSTEM_DISK_INTERVAL | Controls the interval at which system disk metrics are logged. The value must be in seconds and cannot be less than 1s. Defaults to 60. |
| [comet] http_session.retry_total | COMET_HTTP_SESSION_RETRY_TOTAL | Number of times the SDK will try to upload a message before triggering the fallback to offline mechanism. Defaults to 3. |
| [comet] http_session.retry_backoff_factor | COMET_HTTP_SESSION_RETRY_BACKOFF_FACTOR | Controls how much time is applied between two different attempts to send data to Comet. Comet will wait for backoff_factor * (2 ** (attempt_number - 1)) seconds between attemps. Defaults to 2. |
| [comet_fallback_streamer] keep_offline_zip | COMET_FALLBACK_STREAMER_KEEP_OFFLINE_ZIP | If True, save a local copy of experiment data for all training runs, not the just the ones that encountered connectivity issues. Defaults to False. |
| [comet_logging] hide_api_key | COMET_LOGGING_HIDE_API_KEY | If True, redact the Comet API key in logs. Defaults to True. |
| [comet_logging] file | COMET_LOGGING_FILE | Use the given file for storing log messages. This can also contain patterns such as "comet-{project}.log" (See below) |
| [comet_logging] file_level | COMET_LOGGING_FILE_LEVEL | By default, the log outputs will contains INFO and higher level (WARNING and ERROR) log messages. This configuration item can be used to change the level of the logged messages. Defaults to INFO. |
| [comet_logging] file_overwrite | COMET_LOGGING_FILE_OVERWRITE | Overwrites the log file on each run, if True. Defaults to False. |
| [comet_logging] console | COMET_LOGGING_CONSOLE | Set the logging level for console messages (for example, INFO, DEBUG). |
| [comet_logging] metrics_ignore | COMET_LOGGING_METRICS_IGNORE | List of framework:metric names to not log automatically. Defaults to keras:batch_size,keras:batch_batch |
| [comet_logging] parameters_ignore | COMET_LOGGING_PARAMETERS_IGNORE | List of framework:parameter names to not log automatically. Defaults to keras:verbose,keras:do_validation,keras:validation_steps |
| [comet_logging] others_ignore | COMET_LOGGING_OTHERS_IGNORE | List of framework:other names do not log automatically. |
| [comet_timeout] cleaning | COMET_TIMEOUT_CLEANING | How many times (in seconds) does the Python SDK wait until all experiment data has been uploaded. Default is 3600 seconds / 1 hour. |
Experiment configuration parameters¶
In addition to the above, you can also control exactly what each Experiment logs with the following configuration parameters in the section [comet_auto_log].
| Comet Config Section/Name | Environment Parameter Name | Description |
|---|---|---|
| [comet_auto_log] code | COMET_AUTO_LOG_CODE | Auto Log source code for the current Experiment Run. Defaults to True. |
| [comet_auto_log] git_metadata | COMET_AUTO_LOG_GIT_METADATA | Auto Log git metadata for the current Experiment Run. Defaults to True. |
| [comet_auto_log] git_patch | COMET_AUTO_LOG_GIT_PATCH | Auto Log the git patch for current Experiment Run. Set to True to enable. |
| [comet_auto_log] graph | COMET_AUTO_LOG_GRAPH | Auto Log the model graph. This will depend on the framework being used. Set to True to enable. |
| [comet_auto_log] parameters | COMET_AUTO_LOG_PARAMETERS | Auto Log the model parameters. This will depend on the framework being used. Set to True to enable. |
| [comet_auto_log] weights | COMET_AUTO_LOG_WEIGHTS | Auto Log the model weights. This will depend on the framework being used. Set to True to enable. |
| [comet_auto_log] metrics | COMET_AUTO_LOG_METRICS | Auto Log the model metrics. This will depend on the framework being used. Set to True to enable. |
| [comet_auto_log] cli_arguments | COMET_AUTO_LOG_CLI_ARGUMENTS | Auto Log CLI arguments passed to the training script. Set to True to enable. |
| [comet_auto_log] output_logger | COMET_AUTO_LOG_OUTPUT_LOGGER | Auto Log console and Jupyter Notebook standard output and error. Set to True to enable |
| [comet_auto_log] co2 | COMET_AUTO_LOG_CO2 | Auto Log CO2 emissions from the run using the code-carbon package. Set to True to enable |
| [comet_auto_log] env_details | COMET_AUTO_LOG_ENV_DETAILS | Auto Log details about the compute environment. Set to True to enable, set to False to quickly disable all environment details logging. |
| [comet_auto_log] env_gpu | COMET_AUTO_LOG_ENV_GPU | Auto Log the collection of GPU details and system metrics (utilization, memory usage etc..). Logging Environment Details must also be set to True |
| [comet_auto_log] env_host | COMET_AUTO_LOG_ENV_HOST | Auto Log the host information (ip, host name, python version, user etc...). Logging Environment Details must also be set to True |
| [comet_auto_log] env_cpu | COMET_AUTO_LOG_ENV_CPU | Auto Log the collection of CPU details and system metrics (utilization, memory usage etc..). Logging Environment Details must also be set to True |
| [comet_auto_log] env_cpu_per_core | COMET_AUTO_LOG_ENV_CPU_PER_CORE | Auto Log the collection of CPU details and system metrics (utilization etc..) for each CPU core. Logging Environment Details must also be set to True. Defaults to False |
| [comet_auto_log] env_network | COMET_AUTO_LOG_ENV_NETWORK | Auto Log the collection of network system metrics. Logging Environment Details must also be set to True. Defaults to True |
| [comet_auto_log] env_disk | COMET_AUTO_LOG_ENV_DISK | Auto Log the collection of disk system metrics. Logging Environment Details must also be set to True. Defaults to True |
| [comet_auto_log] disable | COMET_AUTO_LOG_DISABLE | Disable all network communication with the Comet backend. It is useful when you want to test to make sure everything is working, without actually logging anything. Defaults to False. |
Environment-only parameters¶
In addition to the configuration parameters, there are a few special-purpose Comet environment-only parameters. These only operate in the environment, and cannot be set in the configuration:
| Environment Parameter | Description |
|---|---|
| COMET_INI | A path to another .comet.config file to take into account |
| COMET_OPTIMIZER_ID | Current optimizer ID; for use with the comet optimize CLI command |
| COMET_OPTIMIZER_PROCESS_ID | Current job number; set when using comet optimize CLI command |
| COMET_OPTIMIZER_PROCESS_JOBS | Total number of parallel jobs; set when using comet optimize CLI command |
| COMET_DISABLE_AUTO_LOGGING | Disable all automatic logging (see next section) |
Configure Comet file logging¶
Comet lets you log the console output and errors to a log file. You can use a number of patterns to name your Comet log file. These are keywords that are surrounded by curly braces.
To set this pattern through an enviroment parameter:
export COMET_LOGGING_FILE={user}.log
To set this pattern through the .comet.config file:
[comet_logging]
file={user}.log
The following table shows the full list of available patterns:
| COMET_LOGGING_FILE Pattern | Example | Expanded Form | Meaning |
|---|---|---|---|
| {user} | comet-{user}.log | comet-jones.log | Uses the OS username |
| {project} | comet-{project}.log | comet-general.log | Uses the COMET_PROJECT_NAME if set, else "general" |
| {pid} | {project}-{pid}.log | general-2635.log | Uses the OS Process ID |
| {datetime} | comet-{datetime}.log | comet-20190814-151135.log | Uses a Year, Month, Day, Hour, Minute, Seconds time format |
Other configuration logging¶
| Configuration item | Comet Config Section/Name | Environment variable | Description |
|---|---|---|---|
| System metrics prefix | [comet] distributed_node_identifier | COMET_DISTRIBUTED_NODE_IDENTIFIER | A prefix to be used on all logged system metrics. |
Feb. 9, 2024