Skip to content

CLI Reference ​

Complete reference guide for the Obvyr CLI tool. For a quick start guide, see Getting Started. For configuration details, see CLI Configuration.

Overview ​

The Obvyr CLI (obvyr-cli) is a Python command-line tool distributed via PyPI that wraps your test commands to capture execution data and parse JUnit XML output for test-level insights. It works by intercepting command execution, collecting metadata, parsing JUnit XML test results, and sending observations to the Obvyr API for pattern analysis.

How it works:

  1. Wrap your test command - obvyr-cli pytest --junitxml=junit.xml tests/
  2. Generate JUnit XML - Your test framework outputs structured test results
  3. Obvyr parses JUnit XML - Extracts individual test results, pass/fail status, execution times
  4. Pattern analysis - Obvyr identifies flaky tests, trends, and reliability metrics

Key characteristics:

  • JUnit XML parsing - Core feature that enables test-level insights
  • Zero workflow disruption - Tests run exactly as before, even if the profile or API is unavailable
  • Language agnostic - Works with any test framework that outputs JUnit XML
  • Multi-profile support - Different configurations for different environments
  • Attachment governance - Automatic file selection with size limits

Installation ​

Installation Methods Comparison ​

MethodBest ForProsConsPrerequisites
uvx (Recommended)CI/CD, trying Obvyr, always-latestNo persistent installation, always latest version, fast, no dependency conflicts, downloads Python 3.14 if you do not have itSlightly longer command syntaxuv installed
pipxLocal development, persistent installIsolated environment, no project conflicts, installed onceRequires manual updatespipx installed, Python 3.14 or later
pipSimple setups, existing pip workflowsNo extra tools, familiarMay conflict with project dependenciesPython 3.14 or later, with pip

Installing Prerequisites ​

macOS ​

bash
# Install uv (for uvx)
brew install uv

# OR install pipx (for pipx)
brew install pipx

Linux ​

bash
# Install uv (for uvx)
curl -LsSf https://astral.sh/uv/install.sh | sh

# OR install pipx (for pipx)
python3 -m pip install --user pipx
python3 -m pipx ensurepath

Windows ​

powershell
# Install uv (for uvx)
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"

# OR install pipx (for pipx)
python -m pip install --user pipx
python -m pipx ensurepath

Installing obvyr-cli ​

Run without persistent installation:

bash
# Verify it works
uvx obvyr-cli --help

# Use directly
uvx obvyr-cli pytest tests/

Advantages:

  • No installation step in CI/CD pipelines
  • Always runs latest version automatically
  • No PATH configuration needed
  • Zero dependency conflicts
  • Downloads a suitable Python (3.14 or later) automatically if you do not have one

When to use:

  • CI/CD pipelines (GitHub Actions, GitLab CI, Jenkins)
  • Trying Obvyr for the first time
  • When you want automatic updates

Option B: pipx ​

Install in isolated environment:

bash
# Install
pipx install obvyr-cli

# Verify
obvyr-cli --help

# Use
obvyr-cli pytest tests/

Requires Python 3.14 or later on your PATH.

Advantages:

  • Installed once, run many times
  • Isolated from project dependencies
  • Standard command syntax

When to use:

  • Local development with frequent usage
  • You prefer persistent installations
  • You want explicit version control

Upgrading:

bash
pipx upgrade obvyr-cli

Uninstalling:

bash
pipx uninstall obvyr-cli

Option C: pip ​

Install in current Python environment:

bash
# Install
pip install obvyr-cli

# Verify
obvyr-cli --help

# Use
obvyr-cli pytest tests/

Requires Python 3.14 or later.

Advantages:

  • No extra tools required
  • Familiar installation method

Disadvantages:

  • May conflict with project dependencies
  • Installs in current environment

When to use:

  • Simple projects without complex dependencies
  • Existing pip-based workflows

Upgrading:

bash
pip install --upgrade obvyr-cli

Uninstalling:

bash
pip uninstall obvyr-cli

Verifying Installation ​

bash
# Check the CLI runs (uvx)
uvx obvyr-cli --help

# Check the CLI runs (pipx/pip)
obvyr-cli --help

Either command prints the usage text and the list of options. To see which version is installed, use pipx list or pip show obvyr-cli.

Platform-Specific Notes ​

macOS ​

  • Install uv or pipx via Homebrew for easiest setup
  • If you use pipx or pip, ensure Python 3.14 or later is installed (python3 --version). uvx downloads it for you
  • May need to add pipx binaries to PATH: export PATH="$PATH:$HOME/.local/bin"

Linux ​

  • Python 3.14 or later required for pipx and pip (uvx downloads it for you)
  • On Debian/Ubuntu, your distribution may not package Python 3.14 yet. Use uvx, or install it with uv python install 3.14
  • pipx may require python3-venv package
  • Add to PATH in ~/.bashrc or ~/.zshrc

Windows ​

  • Python 3.14 or later required for pipx and pip (uvx downloads it for you)
  • Install from python.org or Microsoft Store
  • pipx requires running pipx ensurepath and restarting terminal
  • May need to enable script execution: Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

Command-Line Interface ​

Basic Usage ​

bash
# General syntax
obvyr-cli [OPTIONS] COMMAND [ARGS...]

# With uvx
uvx obvyr-cli [OPTIONS] COMMAND [ARGS...]

Global Options ​

OptionShortDescriptionExample
--profile PROFILE-pUse named configuration profileobvyr-cli -p STAGING pytest
--show-config-Display active configuration (API keys hidden)obvyr-cli --show-config
--list-profiles-List all configured profilesobvyr-cli --list-profiles
--verbose-vEnable verbose logging (debug mode)obvyr-cli -v pytest tests/
--quiet-qEnable quiet mode (errors only)obvyr-cli -q pytest tests/
--dry-run-Run the command, then print exactly what would be sent instead of sending itobvyr-cli --dry-run pytest tests/
--output-mode MODE-Output mode: auto (detects CI, the default), stream (force streaming), or batch (force batch)obvyr-cli --output-mode batch pytest
--no-stream-Disable streaming output (equivalent to --output-mode batch)obvyr-cli --no-stream pytest
--force-stream-Force streaming output (equivalent to --output-mode stream)obvyr-cli --force-stream pytest
--no-color-Do not set FORCE_COLOR for the wrapped commandobvyr-cli --no-color pytest
--help-Show help message and exitobvyr-cli --help

--no-stream and --force-stream cannot be used together. --show-config and --list-profiles only take effect when no command is given.

Put Obvyr's options before the command. Everything from the command onwards is passed to it unchanged, so obvyr-cli -v pytest -p no:cacheprovider turns on Obvyr's verbose logging and gives pytest the -p no:cacheprovider option.

Examples ​

Basic Usage ​

bash
# Wrap pytest
obvyr-cli pytest tests/

# Wrap npm test
obvyr-cli npm test

# Wrap gradle test type checking
obvyr-cli gradle test src/

# Wrap any command
obvyr-cli make test

Using Profiles ​

bash
# Use staging profile
obvyr-cli --profile STAGING pytest tests/

# Short form
obvyr-cli -p PRODUCTION npm test

# Show which profile is active
obvyr-cli --show-config --profile STAGING

Verbose Output ​

bash
# See detailed execution info
obvyr-cli --verbose pytest tests/

# Combine with profile
obvyr-cli -v -p DEV pytest tests/

List Available Profiles ​

bash
# See all configured profiles
obvyr-cli --list-profiles

# Output example:
# Available profiles:
#   - DEFAULT
#   - STAGING
#   - PRODUCTION

Show Configuration ​

bash
# Show active configuration (DEFAULT profile)
obvyr-cli --show-config

# Show specific profile configuration
obvyr-cli --show-config --profile PRODUCTION

# Output example (the API key is never printed):
#
# Profile 'PRODUCTION' configuration:
#
#   API_URL: https://api.obvyr.com
#   ATTACHMENT_PATH: None
#   ATTACHMENT_PATHS: []
#   ATTACHMENT_MAX_AGE_SECONDS: 10
#   TIMEOUT: 10.0
#   VERIFY_SSL: True
#   ALLOW_INSECURE_API_URL: False
#   TAGS: ['production', 'ci']
#   ENV_MODE: allowlist
#   ENV_INCLUDE: []
#   REDACT_PATTERNS: []

Exit Codes ​

The CLI preserves the exit code of the wrapped command:

Exit CodeMeaning
0Command succeeded
1-255Command failed (exit code from wrapped command)
1Configuration error (if command wasn't run)

Example:

bash
# If pytest fails with exit code 1, obvyr-cli also exits with 1
obvyr-cli pytest tests/
echo $?  # Will show 1 if tests failed

This ensures CI/CD pipelines fail correctly when tests fail.

How It Works ​

Command Wrapping Architecture ​

┌─────────────────────────────────────────────┐
│ You run: obvyr-cli pytest tests/           │
└─────────────────┬───────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────────┐
│ 1. CLI loads configuration                  │
│    - Reads OBVYR_* environment variables    │
│    - Validates API key and settings         │
└─────────────────┬───────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────────┐
│ 2. CLI executes wrapped command             │
│    - Runs pytest tests/ as subprocess       │
│    - Captures stdout/stderr in real-time    │
│    - Streams output to your terminal        │
│    - Records execution timing               │
└─────────────────┬───────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────────┐
│ 3. CLI processes output and metadata        │
│    - Parses JUnit XML if present            │
│    - Selects allow-listed env variables     │
│    - Gathers attachment files               │
│    - Records exit code and timing           │
└─────────────────┬───────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────────┐
│ 4. CLI sends observation to Obvyr API       │
│    - Packages all data as JSON              │
│    - Authenticates with API key             │
│    - Sends asynchronously                   │
│    - Returns wrapped command exit code      │
└─────────────────────────────────────────────┘

Data Capture Process ​

What gets captured:

  1. Command Execution

    • Full command string, with secrets in arguments redacted
    • Exit code (success/failure)
    • Start and end timestamps
    • Total execution duration
  2. Output Streams

    • Complete stdout and stderr (interleaved), with secrets redacted before upload
    • Real-time streaming to terminal (no buffering delay)
  3. Environment Context

    • Execution context identifier (OBVYR_CLI_USER - e.g., "local-dev", "github-ci")
    • A short allow-list of CI environment variables (build, run, commit, and branch identifiers)
    • Anything you add with ENV_INCLUDE
  4. Test Framework Metadata

    • JUnit XML parsing (if available)
    • Individual test results (pass/fail/skip)
    • Test execution times
    • Test failure messages
  5. File Attachments (optional)

    • Test reports
    • Coverage data
    • Log files
    • Any configured attachment paths

What does NOT get captured:

  • Environment variables outside the allow-list (unless you add them with ENV_INCLUDE)
  • Variables beginning OBVYR_, including your API key (never sent)
  • Your operating system, Python version, and working directory
  • Files outside configured attachment paths
  • Interactive input/output
  • Binary files (only text files)

Controlling What Is Sent ​

The CLI sends only the data described here. Run with --dry-run at any time to see exactly what would be sent.

Environment variables. By default only these CI identifiers are sent. Every other variable is omitted:

CI, CONTINUOUS_INTEGRATION, GITHUB_ACTIONS, GITHUB_RUN_ID, GITHUB_RUN_NUMBER, GITHUB_WORKFLOW, GITHUB_REPOSITORY, GITHUB_SHA, GITHUB_REF_NAME, GITLAB_CI, CI_PIPELINE_ID, CI_JOB_ID, CI_COMMIT_SHA, CI_COMMIT_REF_NAME, JENKINS_URL, BUILD_NUMBER, BUILDKITE, BUILDKITE_BUILD_NUMBER, CIRCLECI, CIRCLE_BUILD_NUM, CIRCLE_SHA1

Variables beginning with OBVYR_ (including your API key) are never sent, even if you name them in ENV_INCLUDE.

SettingDefaultDescription
ENV_MODEallowlistallowlist sends the variables above plus ENV_INCLUDE. none sends no environment at all.
ENV_INCLUDE[]Comma-separated names of additional variables to send.
REDACT_PATTERNS[]Comma-separated regular expressions to redact in addition to the built-in patterns. A pattern cannot contain a comma; write [,] instead.
bash
export OBVYR_PROFILES__DEFAULT__ENV_INCLUDE="BUILD_FLAVOUR,REGION"
export OBVYR_PROFILES__DEFAULT__REDACT_PATTERNS="ACME-\\d+"
export OBVYR_PROFILES__DEFAULT__ENV_MODE="none"   # send no environment at all

Redaction. Before upload, matches are replaced with [REDACTED] in environment values, command arguments, and command output. The built-in patterns cover:

  • Variables and flags whose name contains token, key, password, passwd, secret, or credential (the value is redacted)
  • Assignments such as password=value or secret: value
  • AWS access key IDs (AKIA…, ASIA…)
  • Bearer tokens

Redaction is best-effort pattern matching, not a guarantee. Use --dry-run to check what your commands produce.

What is not changed:

  • Terminal output is displayed unredacted. Only the uploaded copy is redacted.
  • The wrapped command still receives your full environment.
  • Attachments are uploaded as they are, without redaction. Only attach files that are safe to share.

Dry run. --dry-run runs the command as normal, then prints the exact command.json and output.txt that would be uploaded, and lists the attachments by name and size. It makes no network call and does not need an API key to be valid.

Upgrading. Earlier versions sent the entire environment. If you relied on specific variables appearing in Obvyr, add them to ENV_INCLUDE.

API Communication ​

Request flow:

bash
obvyr-cli pytest tests/
    ↓
CLI → Obvyr API (https://api.obvyr.com)
    ↓
Authentication via API key (JWT token)
    ↓
Observation stored in DynamoDB
    ↓
Available in dashboard immediately

Authentication:

  • API keys are 6-month JWT tokens
  • Sent in Authorization: Bearer <token> header
  • Token includes project_id and agent_id
  • Tokens can be rotated in dashboard

Timeout behaviour:

  • Default timeout: 10 seconds
  • Configurable via OBVYR_PROFILES__<PROFILE>__TIMEOUT
  • If API request times out, CLI still succeeds (command result preserved)

Offline Behaviour ​

What happens if the API is unavailable?

bash
# Your command ALWAYS runs, even if API is down
obvyr-cli pytest tests/

# Scenarios:
# ✅ API available → Observation sent, command runs
# ✅ API timeout → Warning logged, command runs
# ✅ API error → Warning logged, command runs
# ✅ Network down → Warning logged, command runs
# ✅ Profile not found → Warning logged, nothing sent, command runs

The CLI never blocks your workflow. If the API is unreachable, or the profile it needs is not configured, the CLI logs a warning and continues. Your tests run exactly as they would without Obvyr, and the wrapped command's exit code is passed through.

Advanced Usage ​

Multi-Profile Workflows ​

Use different profiles for different environments:

bash
# Local development
export OBVYR_PROFILES__DEV__API_KEY="dev-key"
export OBVYR_PROFILES__DEV__TAGS="development,local"
obvyr-cli --profile DEV pytest tests/

# Staging CI
export OBVYR_PROFILES__STAGING__API_KEY="staging-key"
export OBVYR_PROFILES__STAGING__TAGS="staging,ci"
obvyr-cli --profile STAGING pytest tests/

# Production CI
export OBVYR_PROFILES__PROD__API_KEY="prod-key"
export OBVYR_PROFILES__PROD__TAGS="production,ci"
obvyr-cli --profile PROD pytest tests/

Why use multiple profiles?

  • Track different test types separately (unit, integration, e2e)
  • Separate local vs CI observations
  • Different projects or services
  • Environment-specific configuration

Integration Patterns ​

Install uv before using uvx

uvx ships with uv and is an alias for uv tool run. It avoids a persistent install of obvyr-cli (no pip install obvyr-cli, no virtual environment, and no dependency conflicts with your project), but it does not avoid installing uv. Hosted images such as ubuntu-latest and python:3.12 do not include uv, so each example below installs it first.

GitHub Actions ​

yaml
name: Tests with Obvyr

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.12'

      - name: Install dependencies
        run: pip install -r requirements.txt

      - name: Set up uv
        uses: astral-sh/setup-uv@v6

      - name: Run tests with Obvyr
        env:
          OBVYR_CLI_USER: github-ci
          OBVYR_PROFILES__DEFAULT__API_KEY: ${{ secrets.OBVYR_API_KEY }}
          OBVYR_PROFILES__DEFAULT__TAGS: ci,github-actions
        run: uvx obvyr-cli pytest tests/

GitLab CI ​

yaml
test:
  stage: test
  image: python:3.12
  variables:
    OBVYR_CLI_USER: gitlab-ci
    OBVYR_PROFILES__DEFAULT__TAGS: ci,gitlab
  before_script:
    - pip install uv
  script:
    - pip install -r requirements.txt
    - export OBVYR_PROFILES__DEFAULT__API_KEY=$OBVYR_API_KEY
    - uvx obvyr-cli pytest tests/

Jenkins ​

groovy
pipeline {
    agent any

    environment {
        OBVYR_CLI_USER = 'jenkins-ci'
        OBVYR_PROFILES__DEFAULT__API_KEY = credentials('obvyr-api-key')
        OBVYR_PROFILES__DEFAULT__TAGS = 'ci,jenkins'
    }

    stages {
        stage('Test') {
            steps {
                sh 'pip install -r requirements.txt'
                sh 'pip install uv'
                sh 'uvx obvyr-cli pytest tests/'
            }
        }
    }
}

Pre-commit Hook ​

bash
#!/bin/bash
# .git/hooks/pre-commit

# Run tests with Obvyr before commit
obvyr-cli --profile DEV pytest tests/ || exit 1

Custom Configuration Patterns ​

Per-Repository Configuration ​

Create .env in project root:

bash
# .env (add to .gitignore!)
OBVYR_CLI_USER=local-dev
OBVYR_PROFILES__DEFAULT__API_KEY=your-api-key
OBVYR_PROFILES__DEFAULT__TAGS=my-project,local

Shell Aliases ​

bash
# ~/.bashrc or ~/.zshrc

# Alias for common test command
alias otest="obvyr-cli pytest tests/"

# Alias for verbose testing
alias otestv="obvyr-cli --verbose pytest tests/"

# Alias for staging profile
alias otest-staging="obvyr-cli --profile STAGING pytest tests/"

Makefile Integration ​

makefile
# Makefile

.PHONY: test test-staging test-prod

test:
	obvyr-cli pytest tests/

test-staging:
	obvyr-cli --profile STAGING pytest tests/

test-prod:
	obvyr-cli --profile PROD pytest tests/

test-verbose:
	obvyr-cli --verbose pytest tests/

Performance Considerations ​

CLI overhead:

  • Typical overhead: 100-300ms per command execution
  • Primarily from API request (async, non-blocking)
  • JUnit XML parsing adds ~50ms for large test suites

Minimising overhead:

  1. Use uvx in CI/CD (no installation time)
  2. Ensure good network connectivity to API
  3. Limit attachment file sizes
  4. Use tags instead of excessive metadata

Attachment performance:

  • Files validated before upload (existence, size, freshness)
  • Priority-based selection (JSON > XML > YAML > plain text)
  • 5MB per file limit, 10MB total limit
  • Binary files automatically excluded

Troubleshooting ​

Installation Issues ​

"Command not found: obvyr-cli" (pipx/pip) ​

Cause: Installation directory not in PATH

Solution:

bash
# Check where pipx installs binaries
pipx list

# Add to PATH (add to ~/.bashrc or ~/.zshrc)
export PATH="$PATH:$HOME/.local/bin"

# Reload shell
source ~/.bashrc  # or source ~/.zshrc

"Command not found: uvx" ​

Cause: uv not installed

Solution:

bash
# macOS
brew install uv

# Linux
curl -LsSf https://astral.sh/uv/install.sh | sh

# Windows
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"

"Python version too old" ​

Cause: obvyr-cli requires Python 3.14 or later. pipx and pip use the Python already on your machine.

Solution: run the CLI with uvx, which downloads a suitable Python for you:

bash
uvx obvyr-cli pytest tests/

Or install Python 3.14 or later yourself and keep using pipx or pip:

bash
# Check current version
python3 --version

# Install Python 3.14 (macOS)
brew install python@3.14

# Install Python 3.14 with uv (any platform)
uv python install 3.14

Configuration Issues ​

"Profile 'DEFAULT' not found" ​

What you see: a warning saying nothing is being reported to Obvyr. Your command still runs and its exit code is passed through, but no observation is sent.

Cause: OBVYR_PROFILES__DEFAULT__API_KEY not set

Solution:

bash
# Check if set
echo $OBVYR_PROFILES__DEFAULT__API_KEY

# Set it
export OBVYR_PROFILES__DEFAULT__API_KEY="your-api-key"

# Or create .env file
cat > .env << EOF
OBVYR_PROFILES__DEFAULT__API_KEY=your-api-key
OBVYR_CLI_USER=local-dev
EOF

"Profile 'STAGING' not found" ​

What you see: the same warning, naming the profile you asked for with --profile. Your command still runs, but nothing is sent. (--show-config has nothing to show for a missing profile, so it reports Error: Profile 'STAGING' not found in configuration. instead.)

Cause: Named profile not configured

Solution:

bash
# List available profiles
obvyr-cli --list-profiles

# Either:
# 1. Configure the missing profile
export OBVYR_PROFILES__STAGING__API_KEY="staging-key"

# 2. Use an existing profile
obvyr-cli --profile DEFAULT pytest tests/

# 3. Use default profile (omit --profile flag)
obvyr-cli pytest tests/

"Invalid API key" or 401 Unauthorized ​

Cause: Incorrect or expired API key

Solution:

  1. Verify API key in dashboard (Project → Agent → Settings)
  2. Check for typos in environment variable
  3. Regenerate API key if needed
  4. Ensure no extra whitespace in key
bash
# Verify what's configured
obvyr-cli --show-config

# Check environment variable
echo "$OBVYR_PROFILES__DEFAULT__API_KEY" | cat -A

Runtime Issues ​

"Connection timeout" or "Cannot reach API" ​

Cause: Network connectivity issues

Solution:

bash
# Test connectivity
curl https://api.obvyr.com/health

# Check proxy settings
echo $HTTP_PROXY
echo $HTTPS_PROXY

# If behind proxy, configure
export HTTPS_PROXY=http://proxy.example.com:8080

"SSL certificate verification failed" ​

Cause: Corporate proxy or SSL interception

Solution (temporary):

bash
# Disable SSL verification (NOT recommended for production)
export OBVYR_PROFILES__DEFAULT__VERIFY_SSL=false

Solution (proper):

bash
# Install corporate CA certificate
export REQUESTS_CA_BUNDLE=/path/to/corporate-ca.crt

Tests fail when wrapped but pass normally ​

Cause: Rare output buffering or environment variable conflicts

Solution:

bash
# Run with verbose logging to diagnose
obvyr-cli --verbose pytest tests/

# Check environment variables
obvyr-cli env | grep OBVYR

# Report issue with verbose output

Debug Mode ​

Enable verbose logging for troubleshooting:

bash
# Verbose CLI output
obvyr-cli --verbose pytest tests/

# Set log level via environment variable
export OBVYR_LOG_LEVEL=DEBUG
obvyr-cli pytest tests/

# Combine with profile debugging
obvyr-cli -v --show-config --profile STAGING

What verbose mode shows:

  • Configuration loading details
  • Command execution timing
  • API request/response details
  • Attachment file selection process
  • JUnit XML parsing results

Getting Help ​

If you encounter issues not covered here:

  1. Check configuration: obvyr-cli --show-config
  2. Run with verbose output: obvyr-cli --verbose pytest tests/
  3. Check API status: Visit status.obvyr.com (if available)
  4. Review logs: Look for warning/error messages
  5. Contact support: support@obvyr.com

Include in your support request:

  • CLI version (pipx list or pip show obvyr-cli; for uvx, say that you use uvx and the date you last ran it)
  • Python version (python3 --version)
  • Operating system
  • Installation method (uvx/pipx/pip)
  • Verbose output (--verbose)
  • Configuration (with API key redacted)

Next Steps ​