CLI Configuration

This guide explains how to configure the obvyr CLI tool to collect test execution data from your environments.

Configuration Overview

The Obvyr CLI supports multiple configurations per installation, allowing you to track test data for different projects or environments from a single machine. Each configuration is identified by a name and configured with its own API key and settings.

Configuration Methods

The CLI uses a hierarchical configuration system that supports multiple methods, with later methods overriding earlier ones:

1. Default values (built into the agent)
2. Environment variables (from .env file or shell exports)
3. Command-line options (flags passed directly to commands)

Environment Variables

The CLI uses environment variables with a nested naming convention. All configuration variables are prefixed with OBVYR_.

Required Configuration

At minimum, you need:

# Required: CLI configuration with API key
export OBVYR_PROFILES__DEFAULT__API_KEY="your-api-key-here"

# Recommended: User identifier for tracking who ran commands
export OBVYR_CLI_USER="your-username"

Using a .env File

Create a .env file in your project directory or home directory:

# CLI Configuration
OBVYR_PROFILES__DEFAULT__API_KEY=your-api-key-here
OBVYR_CLI_USER=timmah

# Optional: Set request timeout in seconds (default: 10.0)
OBVYR_PROFILES__DEFAULT__TIMEOUT=15.0

# Optional: Add tags to all observations from this CLI
OBVYR_PROFILES__DEFAULT__TAGS=unit-tests,local-dev

Security

Never commit .env files containing API keys to version control. Add .env to your .gitignore file.

Multiple Configuration Profiles

You can configure multiple profiles by using different names instead of DEFAULT:

# Development environment
OBVYR_PROFILES__DEVELOPMENT__API_KEY=dev-api-key-here
OBVYR_PROFILES__DEVELOPMENT__TAGS=development,local

# Staging environment
OBVYR_PROFILES__STAGING__API_KEY=staging-api-key-here
OBVYR_PROFILES__STAGING__TAGS=staging

# Production CI
OBVYR_PROFILES__PRODUCTION__API_KEY=prod-api-key-here
OBVYR_PROFILES__PRODUCTION__TAGS=production,ci

To use a specific configuration profile:

# Use the default configuration
obvyr-cli pytest tests/

# Use a named configuration
obvyr-cli --profile DEVELOPMENT pytest tests/
obvyr-cli --profile STAGING npm test
obvyr -p PRODUCTION pytest tests/

Configuration Options Reference

CLI Settings

Each configuration profile supports the following options:

Variable	Description	Default	Required
API_KEY	Authentication token from Obvyr dashboard	-	Yes
TIMEOUT	Request timeout in seconds	10.0	No
VERIFY_SSL	Enable/disable SSL certificate verification	true	No
TAGS	Comma-separated list of tags to add to observations	[]	No
ATTACHMENT_PATHS	Comma-separated paths to files or directories to attach to observations. Note: ATTACHMENT_PATH (singular) is deprecated but still supported.	None	No

Global Settings

Variable	Description	Example
OBVYR_CLI_USER	Execution context identifier (NOT individual names for data protection)	local-dev, github-ci, automation
OBVYR_LOG_LEVEL	Logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL)	WARNING

Environment-Specific Configuration

Local Development

For local development, create a .env file in your project root:

# .env
OBVYR_CLI_USER=local-dev
OBVYR_PROFILES__DEFAULT__API_KEY=your-dev-cli-key
OBVYR_PROFILES__DEFAULT__TAGS=development,local

Then use the CLI as normal:

obvyr-cli pytest tests/

CI/CD Environments

In CI/CD, set environment variables through your platform's secrets management:

GitHub Actions

env:
  OBVYR_CLI_USER: github-ci
  OBVYR_PROFILES__DEFAULT__API_KEY: ${{ secrets.OBVYR_AGENT_API_KEY }}
  OBVYR_PROFILES__DEFAULT__ATTACHMENT_PATHS: test-results/junit.xml
  OBVYR_PROFILES__DEFAULT__TAGS: ci,github-actions

steps:
  - name: Run tests with Obvyr
    run: obvyr-cli pytest --junitxml=test-results/junit.xml tests/

GitLab CI

variables:
  OBVYR_CLI_USER: gitlab-ci
  OBVYR_PROFILES__DEFAULT__ATTACHMENT_PATHS: test-results/junit.xml
  OBVYR_PROFILES__DEFAULT__TAGS: ci,gitlab

test:
  script:
    - export OBVYR_PROFILES__DEFAULT__API_KEY=$OBVYR_AGENT_API_KEY
    - obvyr-cli pytest --junitxml=test-results/junit.xml tests/

Jenkins

environment {
    OBVYR_CLI_USER = 'jenkins-ci'
    OBVYR_PROFILES__DEFAULT__API_KEY = credentials('obvyr-agent-key')
    OBVYR_PROFILES__DEFAULT__ATTACHMENT_PATHS = 'test-results/junit.xml'
    OBVYR_PROFILES__DEFAULT__TAGS = 'ci,jenkins'
}

steps {
    sh 'obvyr-cli pytest --junitxml=test-results/junit.xml tests/'
}

Attachment Configuration

The CLI can automatically include specific files with your observations, such as JUnit XML test reports or coverage data.

Size Limits

• Individual file limit: 5MB per file
• Total attachment limit: 10MB per observation
• Only text-based files are supported (binary files are automatically excluded)

Attachment Configuration

Specify one or more file paths (comma-separated) to attach:

# Include JUnit XML report with observations
OBVYR_PROFILES__DEFAULT__ATTACHMENT_PATHS=test-results/junit.xml

# Include multiple files (comma-separated)
OBVYR_PROFILES__DEFAULT__ATTACHMENT_PATHS=test-results/junit.xml,coverage.json,test-report.html

# Include files modified within 10 seconds of command completion
OBVYR_PROFILES__DEFAULT__ATTACHMENT_MAX_AGE_SECONDS=10

File Paths Only

Specify individual file paths, not directories. The CLI does not support directory attachments.

File Freshness

Only files modified within ATTACHMENT_MAX_AGE_SECONDS of the command's completion are included. This prevents accidentally attaching stale files from previous runs. The default is 10 seconds.

Graceful Degradation

If attachment files are missing, oversized, or stale, the CLI will log a warning and continue without failing. Your command execution is never affected by attachment issues.

Tagging Strategy

Tags help you filter and organise observations in the Obvyr dashboard. Consider using tags for:

• Environment: development, staging, production
• CI/CD platform: github-actions, gitlab-ci, jenkins
• Test type: unit-tests, integration-tests, e2e-tests
• Team: platform-team, product-team
• Feature flags: feature-new-auth, feature-payments

Example Tagging Configurations

# Development environment
OBVYR_PROFILES__DEFAULT__TAGS=development,local,unit-tests

# CI environment
OBVYR_PROFILES__CI__TAGS=ci,github-actions,integration-tests

# E2E tests in staging
OBVYR_PROFILES__E2E__TAGS=staging,e2e-tests,selenium

Verifying Configuration

List Available Profiles

To see which configuration profiles are configured:

obvyr-cli --list-profiles

Output:

Available profiles:
  - DEFAULT
  - DEVELOPMENT
  - STAGING
  - PRODUCTION

Show CLI Configuration

To view the active configuration for a profile (API keys are hidden):

# Show default configuration
obvyr-cli --show-config

# Show specific configuration profile
obvyr-cli --show-config --profile PRODUCTION

Output:

Active Profile: PRODUCTION

Configuration:
  API_URL: https://api.obvyr.com
  API_KEY: agt_***...***
  TIMEOUT: 10.0
  VERIFY_SSL: true
  ATTACHMENT_PATHS: None
  ATTACHMENT_MAX_AGE_SECONDS: 10
  TAGS: ['production', 'ci']

Troubleshooting

Configuration Not Loading

If your configuration isn't being picked up:

1. Check variable naming: Ensure you're using double underscores (__) as delimiters
2. Verify case sensitivity: Profile names must be uppercase (e.g., DEFAULT, not default)
3. Check .env location: The CLI looks for .env in the current directory
4. Use --show-config: Verify what configuration is actually loaded

Missing API Key Error

ValueError: Configuration 'DEFAULT' not found.

This means no configuration was found. Ensure you've set:

OBVYR_PROFILES__DEFAULT__API_KEY=your-api-key

Configuration Profile Not Found

ValueError: Configuration 'STAGING' not found.

Either:

1. Configure the named profile: OBVYR_PROFILES__STAGING__API_KEY=...
2. Use the default configuration: obvyr-cli pytest tests/ (without --profile)
3. Check spelling and case: profile names are case-sensitive and must be uppercase

Best Practices

1. Use Different Profiles for Different Purposes

# Separate profiles for different test types
OBVYR_PROFILES__UNIT__API_KEY=unit-test-key
OBVYR_PROFILES__INTEGRATION__API_KEY=integration-test-key
OBVYR_PROFILES__E2E__API_KEY=e2e-test-key

2. Use Meaningful Tags

# Good: Descriptive and useful for filtering
OBVYR_PROFILES__DEFAULT__TAGS=development,unit-tests,macos

# Avoid: Too generic or not useful
OBVYR_PROFILES__DEFAULT__TAGS=tests,stuff

3. Set Execution Context Identifier

Use OBVYR_CLI_USER to identify the execution context, not individuals:

# Local development: generic developer context
OBVYR_CLI_USER=local-dev

# CI/CD: platform and environment
OBVYR_CLI_USER=github-ci
OBVYR_CLI_USER=jenkins-prod
OBVYR_CLI_USER=gitlab-staging

# Automation: process identifier
OBVYR_CLI_USER=scheduled-tests
OBVYR_CLI_USER=deploy-validation

Why context over identity:

• Data protection: Avoids collecting personal identifiers
• Privacy compliance: GDPR/regulatory friendly
• Useful insights: Pattern analysis works with "local vs CI", not individual tracking

4. Keep API Keys Secure

• Use CI/CD secret management systems
• Never commit API keys to version control
• Rotate keys regularly through the Obvyr dashboard
• Use different keys for different environments

Next Steps

Now that you've configured the CLI, you're ready to:

• Learn about collecting test data
• Explore CI/CD integration patterns
• View your observations in the Obvyr dashboard