CLI Reference

Complete reference guide for the Obvyr CLI tool. For a quick start guide, see Getting Started. For configuration details, see CLI Configuration.

Overview

The Obvyr CLI (obvyr-cli) is a Python command-line tool distributed via PyPI that wraps your test commands to capture execution data and parse JUnit XML output for test-level insights. It works by intercepting command execution, collecting metadata, parsing JUnit XML test results, and sending observations to the Obvyr API for pattern analysis.

How it works:

Wrap your test command - obvyr-cli pytest --junitxml=junit.xml tests/
Generate JUnit XML - Your test framework outputs structured test results
Obvyr parses JUnit XML - Extracts individual test results, pass/fail status, execution times
Pattern analysis - Obvyr identifies flaky tests, trends, and reliability metrics

Key characteristics:

JUnit XML parsing - Core feature that enables test-level insights
Zero workflow disruption - Tests run exactly as before
Language agnostic - Works with any test framework that outputs JUnit XML
Multi-profile support - Different configurations for different environments
Attachment governance - Automatic file selection with size limits

Installation

Installation Methods Comparison

Method	Best For	Pros	Cons	Prerequisites
uvx (Recommended)	CI/CD, trying Obvyr, always-latest	No persistent installation, always latest version, fast, no dependency conflicts	Slightly longer command syntax	`uv` installed
pipx	Local development, persistent install	Isolated environment, no project conflicts, installed once	Requires manual updates	`pipx` installed
pip	Simple setups, existing pip workflows	No extra tools, familiar	May conflict with project dependencies	Python + pip

Installing Prerequisites

macOS

bash

# Install uv (for uvx)
brew install uv

# OR install pipx (for pipx)
brew install pipx

Linux

bash

# Install uv (for uvx)
curl -LsSf https://astral.sh/uv/install.sh | sh

# OR install pipx (for pipx)
python3 -m pip install --user pipx
python3 -m pipx ensurepath

Windows

powershell

# Install uv (for uvx)
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"

# OR install pipx (for pipx)
python -m pip install --user pipx
python -m pipx ensurepath

Installing obvyr-cli

Option A: uvx (Recommended)

Run without persistent installation:

bash

# Verify it works
uvx obvyr-cli --version

# Use directly
uvx obvyr-cli pytest tests/

Advantages:

No installation step in CI/CD pipelines
Always runs latest version automatically
No PATH configuration needed
Zero dependency conflicts

When to use:

CI/CD pipelines (GitHub Actions, GitLab CI, Jenkins)
Trying Obvyr for the first time
When you want automatic updates

Option B: pipx

Install in isolated environment:

bash

# Install
pipx install obvyr-cli

# Verify
obvyr-cli --version

# Use
obvyr-cli pytest tests/

Advantages:

Installed once, run many times
Isolated from project dependencies
Standard command syntax

When to use:

Local development with frequent usage
You prefer persistent installations
You want explicit version control

Upgrading:

bash

pipx upgrade obvyr-cli

Uninstalling:

bash

pipx uninstall obvyr-cli

Option C: pip

Install in current Python environment:

bash

# Install
pip install obvyr-cli

# Verify
obvyr-cli --version

# Use
obvyr-cli pytest tests/

Advantages:

No extra tools required
Familiar installation method

Disadvantages:

May conflict with project dependencies
Installs in current environment

When to use:

Simple projects without complex dependencies
Existing pip-based workflows

Upgrading:

bash

pip install --upgrade obvyr-cli

Uninstalling:

bash

pip uninstall obvyr-cli

Verifying Installation

bash

# Check version (uvx)
uvx obvyr-cli --version

# Check version (pipx/pip)
obvyr-cli --version

# Expected output (example)
obvyr-cli version 2.3.0

Platform-Specific Notes

macOS

Install uv or pipx via Homebrew for easiest setup
Ensure Python 3.12+ is installed (python3 --version)
May need to add pipx binaries to PATH: export PATH="$PATH:$HOME/.local/bin"

Linux

Python 3.12+ required
On Debian/Ubuntu: apt install python3.12
pipx may require python3-venv package
Add to PATH in ~/.bashrc or ~/.zshrc

Windows

Python 3.12+ required
Install from python.org or Microsoft Store
pipx requires running pipx ensurepath and restarting terminal
May need to enable script execution: Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

Command-Line Interface

Basic Usage

bash

# General syntax
obvyr-cli [OPTIONS] COMMAND [ARGS...]

# With uvx
uvx obvyr-cli [OPTIONS] COMMAND [ARGS...]

Global Options

Option	Short	Description	Example
`--version`	-	Show CLI version and exit	`obvyr-cli --version`
`--profile PROFILE`	`-p`	Use named configuration profile	`obvyr-cli -p STAGING pytest`
`--show-config`	-	Display active configuration (API keys hidden)	`obvyr-cli --show-config`
`--list-profiles`	-	List all configured profiles	`obvyr-cli --list-profiles`
`--verbose`	`-v`	Enable verbose logging output	`obvyr-cli -v pytest tests/`
`--help`	`-h`	Show help message and exit	`obvyr-cli --help`

Examples

Basic Usage

bash

# Wrap pytest
obvyr-cli pytest tests/

# Wrap npm test
obvyr-cli npm test

# Wrap gradle test type checking
obvyr-cli gradle test src/

# Wrap any command
obvyr-cli make test

Using Profiles

bash

# Use staging profile
obvyr-cli --profile STAGING pytest tests/

# Short form
obvyr-cli -p PRODUCTION npm test

# Show which profile is active
obvyr-cli --show-config --profile STAGING

Verbose Output

bash

# See detailed execution info
obvyr-cli --verbose pytest tests/

# Combine with profile
obvyr-cli -v -p DEV pytest tests/

List Available Profiles

bash

# See all configured profiles
obvyr-cli --list-profiles

# Output example:
# Available profiles:
#   - DEFAULT
#   - STAGING
#   - PRODUCTION

Show Configuration

bash

# Show active configuration (DEFAULT profile)
obvyr-cli --show-config

# Show specific profile configuration
obvyr-cli --show-config --profile PRODUCTION

# Output example:
# Active Profile: PRODUCTION
#
# Configuration:
#   API_URL: https://api.obvyr.com
#   API_KEY: agt_***...***
#   TIMEOUT: 10.0
#   VERIFY_SSL: true
#   ATTACHMENT_PATHS: None
#   TAGS: ['production', 'ci']

Exit Codes

The CLI preserves the exit code of the wrapped command:

Exit Code	Meaning
`0`	Command succeeded
`1-255`	Command failed (exit code from wrapped command)
`1`	Configuration error (if command wasn't run)

Example:

bash

# If pytest fails with exit code 1, obvyr-cli also exits with 1
obvyr-cli pytest tests/
echo $?  # Will show 1 if tests failed

This ensures CI/CD pipelines fail correctly when tests fail.

How It Works

Command Wrapping Architecture

┌─────────────────────────────────────────────┐
│ You run: obvyr-cli pytest tests/           │
└─────────────────┬───────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────────┐
│ 1. CLI loads configuration                  │
│    - Reads OBVYR_* environment variables    │
│    - Validates API key and settings         │
└─────────────────┬───────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────────┐
│ 2. CLI executes wrapped command             │
│    - Runs pytest tests/ as subprocess       │
│    - Captures stdout/stderr in real-time    │
│    - Streams output to your terminal        │
│    - Records execution timing               │
└─────────────────┬───────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────────┐
│ 3. CLI processes output and metadata        │
│    - Parses JUnit XML if present            │
│    - Collects environment variables         │
│    - Gathers attachment files               │
│    - Records exit code and timing           │
└─────────────────┬───────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────────┐
│ 4. CLI sends observation to Obvyr API       │
│    - Packages all data as JSON              │
│    - Authenticates with API key             │
│    - Sends asynchronously                   │
│    - Returns wrapped command exit code      │
└─────────────────────────────────────────────┘

Data Capture Process

What gets captured:

Command Execution
- Full command string
- Exit code (success/failure)
- Start and end timestamps
- Total execution duration
Output Streams
- Complete stdout (standard output)
- Complete stderr (standard error)
- Real-time streaming to terminal (no buffering delay)
Environment Context
- Execution context identifier (OBVYR_CLI_USER - e.g., "local-dev", "github-ci")
- Operating system and version
- Python version
- Working directory
- Environment variables (configurable)
Test Framework Metadata
- JUnit XML parsing (if available)
- Individual test results (pass/fail/skip)
- Test execution times
- Test failure messages
File Attachments (optional)
- Test reports
- Coverage data
- Log files
- Any configured attachment paths

What does NOT get captured:

Sensitive data (unless you explicitly include it)
Files outside configured attachment paths
Interactive input/output
Binary files (only text files)

API Communication

Request flow:

bash

obvyr-cli pytest tests/
    ↓
CLI → Obvyr API (https://api.obvyr.com)
    ↓
Authentication via API key (JWT token)
    ↓
Observation stored in DynamoDB
    ↓
Available in dashboard immediately

Authentication:

API keys are 6-month JWT tokens
Sent in Authorization: Bearer <token> header
Token includes project_id and agent_id
Tokens can be rotated in dashboard

Timeout behaviour:

Default timeout: 10 seconds
Configurable via OBVYR_PROFILES__<PROFILE>__TIMEOUT
If API request times out, CLI still succeeds (command result preserved)

Offline Behaviour

What happens if the API is unavailable?

bash

# Your command ALWAYS runs, even if API is down
obvyr-cli pytest tests/

# Scenarios:
# ✅ API available → Observation sent, command runs
# ✅ API timeout → Warning logged, command runs
# ✅ API error → Warning logged, command runs
# ✅ Network down → Warning logged, command runs

The CLI never blocks your workflow. If the API is unreachable, the CLI logs a warning and continues. Your tests run exactly as they would without Obvyr.

Advanced Usage

Multi-Profile Workflows

Use different profiles for different environments:

bash

# Local development
export OBVYR_PROFILES__DEV__API_KEY="dev-key"
export OBVYR_PROFILES__DEV__TAGS="development,local"
obvyr-cli --profile DEV pytest tests/

# Staging CI
export OBVYR_PROFILES__STAGING__API_KEY="staging-key"
export OBVYR_PROFILES__STAGING__TAGS="staging,ci"
obvyr-cli --profile STAGING pytest tests/

# Production CI
export OBVYR_PROFILES__PROD__API_KEY="prod-key"
export OBVYR_PROFILES__PROD__TAGS="production,ci"
obvyr-cli --profile PROD pytest tests/

Why use multiple profiles?

Track different test types separately (unit, integration, e2e)
Separate local vs CI observations
Different projects or services
Environment-specific configuration

Integration Patterns

GitHub Actions

yaml

name: Tests with Obvyr

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.12'

      - name: Install dependencies
        run: pip install -r requirements.txt

      - name: Run tests with Obvyr
        env:
          OBVYR_CLI_USER: github-ci
          OBVYR_PROFILES__DEFAULT__API_KEY: ${{ secrets.OBVYR_API_KEY }}
          OBVYR_PROFILES__DEFAULT__TAGS: ci,github-actions
        run: uvx obvyr-cli pytest tests/

GitLab CI

yaml

test:
  stage: test
  image: python:3.12
  variables:
    OBVYR_CLI_USER: gitlab-ci
    OBVYR_PROFILES__DEFAULT__TAGS: ci,gitlab
  script:
    - pip install -r requirements.txt
    - export OBVYR_PROFILES__DEFAULT__API_KEY=$OBVYR_API_KEY
    - uvx obvyr-cli pytest tests/

Jenkins

groovy

pipeline {
    agent any

    environment {
        OBVYR_CLI_USER = 'jenkins-ci'
        OBVYR_PROFILES__DEFAULT__API_KEY = credentials('obvyr-api-key')
        OBVYR_PROFILES__DEFAULT__TAGS = 'ci,jenkins'
    }

    stages {
        stage('Test') {
            steps {
                sh 'pip install -r requirements.txt'
                sh 'uvx obvyr-cli pytest tests/'
            }
        }
    }
}

Pre-commit Hook

bash

#!/bin/bash
# .git/hooks/pre-commit

# Run tests with Obvyr before commit
obvyr-cli --profile DEV pytest tests/ || exit 1

Custom Configuration Patterns

Per-Repository Configuration

Create .env in project root:

bash

# .env (add to .gitignore!)
OBVYR_CLI_USER=local-dev
OBVYR_PROFILES__DEFAULT__API_KEY=your-api-key
OBVYR_PROFILES__DEFAULT__TAGS=my-project,local

Shell Aliases

bash

# ~/.bashrc or ~/.zshrc

# Alias for common test command
alias otest="obvyr-cli pytest tests/"

# Alias for verbose testing
alias otestv="obvyr-cli --verbose pytest tests/"

# Alias for staging profile
alias otest-staging="obvyr-cli --profile STAGING pytest tests/"

Makefile Integration

makefile

# Makefile

.PHONY: test test-staging test-prod

test:
	obvyr-cli pytest tests/

test-staging:
	obvyr-cli --profile STAGING pytest tests/

test-prod:
	obvyr-cli --profile PROD pytest tests/

test-verbose:
	obvyr-cli --verbose pytest tests/

Performance Considerations

CLI overhead:

Typical overhead: 100-300ms per command execution
Primarily from API request (async, non-blocking)
JUnit XML parsing adds ~50ms for large test suites

Minimising overhead:

Use uvx in CI/CD (no installation time)
Ensure good network connectivity to API
Limit attachment file sizes
Use tags instead of excessive metadata

Attachment performance:

Files validated before upload (existence, size, freshness)
Priority-based selection (JSON > XML > YAML > plain text)
5MB per file limit, 10MB total limit
Binary files automatically excluded

Troubleshooting

Installation Issues

"Command not found: obvyr-cli" (pipx/pip)

Cause: Installation directory not in PATH

Solution:

bash

# Check where pipx installs binaries
pipx list

# Add to PATH (add to ~/.bashrc or ~/.zshrc)
export PATH="$PATH:$HOME/.local/bin"

# Reload shell
source ~/.bashrc  # or source ~/.zshrc

"Command not found: uvx"

Cause: uv not installed

Solution:

bash

# macOS
brew install uv

# Linux
curl -LsSf https://astral.sh/uv/install.sh | sh

# Windows
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"

"Python version too old"

Cause: obvyr-cli requires Python 3.12+

Solution:

bash

# Check current version
python3 --version

# Install Python 3.12+ (macOS)
brew install python@3.12

# Install Python 3.12+ (Ubuntu/Debian)
sudo apt install python3.12

# Install Python 3.12+ (Windows)
# Download from python.org

Configuration Issues

"Configuration 'DEFAULT' not found"

Cause: OBVYR_PROFILES__DEFAULT__API_KEY not set

Solution:

bash

# Check if set
echo $OBVYR_PROFILES__DEFAULT__API_KEY

# Set it
export OBVYR_PROFILES__DEFAULT__API_KEY="your-api-key"

# Or create .env file
cat > .env << EOF
OBVYR_PROFILES__DEFAULT__API_KEY=your-api-key
OBVYR_CLI_USER=local-dev
EOF

"Configuration 'STAGING' not found"

Cause: Named profile not configured

Solution:

bash

# List available profiles
obvyr-cli --list-profiles

# Either:
# 1. Configure the missing profile
export OBVYR_PROFILES__STAGING__API_KEY="staging-key"

# 2. Use an existing profile
obvyr-cli --profile DEFAULT pytest tests/

# 3. Use default profile (omit --profile flag)
obvyr-cli pytest tests/

"Invalid API key" or 401 Unauthorized

Cause: Incorrect or expired API key

Solution:

Verify API key in dashboard (Project → Agent → Settings)
Check for typos in environment variable
Regenerate API key if needed
Ensure no extra whitespace in key

bash

# Verify what's configured
obvyr-cli --show-config

# Check environment variable
echo "$OBVYR_PROFILES__DEFAULT__API_KEY" | cat -A

Runtime Issues

"Connection timeout" or "Cannot reach API"

Cause: Network connectivity issues

Solution:

bash

# Test connectivity
curl https://api.obvyr.com/health

# Check proxy settings
echo $HTTP_PROXY
echo $HTTPS_PROXY

# If behind proxy, configure
export HTTPS_PROXY=http://proxy.example.com:8080

"SSL certificate verification failed"

Cause: Corporate proxy or SSL interception

Solution (temporary):

bash

# Disable SSL verification (NOT recommended for production)
export OBVYR_PROFILES__DEFAULT__VERIFY_SSL=false

Solution (proper):

bash

# Install corporate CA certificate
export REQUESTS_CA_BUNDLE=/path/to/corporate-ca.crt

Tests fail when wrapped but pass normally

Cause: Rare output buffering or environment variable conflicts

Solution:

bash

# Run with verbose logging to diagnose
obvyr-cli --verbose pytest tests/

# Check environment variables
obvyr-cli env | grep OBVYR

# Report issue with verbose output

Debug Mode

Enable verbose logging for troubleshooting:

bash

# Verbose CLI output
obvyr-cli --verbose pytest tests/

# Set log level via environment variable
export OBVYR_LOG_LEVEL=DEBUG
obvyr-cli pytest tests/

# Combine with profile debugging
obvyr-cli -v --show-config --profile STAGING

What verbose mode shows:

Configuration loading details
Command execution timing
API request/response details
Attachment file selection process
JUnit XML parsing results

Getting Help

If you encounter issues not covered here:

Check configuration: obvyr-cli --show-config
Run with verbose output: obvyr-cli --verbose pytest tests/
Check API status: Visit status.obvyr.com (if available)
Review logs: Look for warning/error messages
Contact support: support@obvyr.com

Include in your support request:

CLI version (obvyr-cli --version)
Python version (python3 --version)
Operating system
Installation method (uvx/pipx/pip)
Verbose output (--verbose)
Configuration (with API key redacted)

Next Steps

Getting Started - Quick 10-minute setup guide
CLI Configuration - Environment variables and advanced configuration
Why Obvyr? - Understand the value proposition
Roadmap - See what's coming next

CLI Reference ​

Overview ​

Installation ​

Installation Methods Comparison ​

Installing Prerequisites ​

macOS ​

Linux ​

Windows ​

Installing obvyr-cli ​

Option A: uvx (Recommended) ​

Option B: pipx ​

Option C: pip ​

Verifying Installation ​

Platform-Specific Notes ​

macOS ​

Linux ​

Windows ​

Command-Line Interface ​

Basic Usage ​

Global Options ​

Examples ​

Basic Usage ​

Using Profiles ​

Verbose Output ​

List Available Profiles ​

Show Configuration ​

Exit Codes ​

How It Works ​

Command Wrapping Architecture ​

Data Capture Process ​

API Communication ​

Offline Behaviour ​

Advanced Usage ​

Multi-Profile Workflows ​

Integration Patterns ​

GitHub Actions ​

GitLab CI ​

Jenkins ​

Pre-commit Hook ​

Custom Configuration Patterns ​

Per-Repository Configuration ​

Shell Aliases ​

Makefile Integration ​

Performance Considerations ​

Troubleshooting ​

Installation Issues ​

"Command not found: obvyr-cli" (pipx/pip) ​

"Command not found: uvx" ​

"Python version too old" ​

Configuration Issues ​

"Configuration 'DEFAULT' not found" ​

"Configuration 'STAGING' not found" ​

"Invalid API key" or 401 Unauthorized ​

Runtime Issues ​

"Connection timeout" or "Cannot reach API" ​

"SSL certificate verification failed" ​

Tests fail when wrapped but pass normally ​

Debug Mode ​

Getting Help ​

Next Steps ​

CLI Reference

Overview

Installation

Installation Methods Comparison

Installing Prerequisites

macOS

Linux

Windows

Installing obvyr-cli

Option A: uvx (Recommended)

Option B: pipx

Option C: pip

Verifying Installation

Platform-Specific Notes

macOS

Linux

Windows

Command-Line Interface

Basic Usage

Global Options

Examples

Basic Usage

Using Profiles

Verbose Output

List Available Profiles

Show Configuration

Exit Codes

How It Works

Command Wrapping Architecture

Data Capture Process

API Communication

Offline Behaviour

Advanced Usage

Multi-Profile Workflows

Integration Patterns

GitHub Actions

GitLab CI

Jenkins

Pre-commit Hook

Custom Configuration Patterns

Per-Repository Configuration

Shell Aliases

Makefile Integration

Performance Considerations

Troubleshooting

Installation Issues

"Command not found: obvyr-cli" (pipx/pip)

"Command not found: uvx"

"Python version too old"

Configuration Issues

"Configuration 'DEFAULT' not found"

"Configuration 'STAGING' not found"

"Invalid API key" or 401 Unauthorized

Runtime Issues

"Connection timeout" or "Cannot reach API"

"SSL certificate verification failed"

Tests fail when wrapped but pass normally

Debug Mode

Getting Help

Next Steps