Introduction to Obvyr

How do you pronounce "Obvyr"?

It's pronounced "ob-veer" (rhymes with "beer", "clear", "here"). It's a trendy 5-letter domain with minimal vowels, related to observability. Yes, we know it's weird. Don't worry, we still use vowels in our code... mostly.

The Testing Confidence Problem

Do your tests actually protect production? Or are you just assuming they do?

Most engineering teams deploy with hope, not evidence:

• Hope that flaky tests aren't hiding real bugs
• Hope that local environment matches CI behaviour
• Hope that your 3,000 tests are all providing value
• Hope that AI-generated code has AI-reliable tests

Obvyr replaces hope with proof.

What Obvyr Does

Obvyr is a testing insights platform that proves test reliability by parsing JUnit XML output from your test runs. By analysing patterns across thousands of test executions, Obvyr reveals:

• Which tests are truly flaky vs. which are genuinely broken
• Test execution patterns across users, environments, and time
• Test-level metrics including pass rates, execution times, and failure trends
• Dashboard insights showing overall test health and reliability

You move from assumption-based testing ("we think our tests are good") to evidence-based testing ("we can prove our tests are reliable").

How Obvyr Organises Your Data

To prove test reliability, Obvyr collects and organises test execution data through a flexible hierarchy designed around how engineering teams actually work:

Organisations

Your organisation account represents your company or team. This is where billing is managed and where you control user access across all your projects.

Why it matters: Multi-tenant isolation ensures your test data is completely separate from other organisations, providing both security and clarity in pattern analysis.

Projects

Projects are logical groupings that make sense for your workflow. You might organise by:

• Codebase (one project per repository) - Compare test patterns across different repositories
• Service (frontend, API, mobile app) - Understand test reliability per service
• Team (platform team, product team) - Track team-specific testing practices
• Environment (staging, production) - Analyse environmental test differences

Why it matters: Flexible project organisation lets you analyse test patterns at the granularity that makes sense for your team, whether that's service-level, team-level, or environment-level insights.

Organise for Insights

There's no single "right" way. Use whatever structure helps you analyse your testing data most effectively. The goal is evidence-based insights, not rigid hierarchy.

CLI Agents

Within each project, you'll create CLI agents to collect data from specific types of testing activity. Each CLI agent has its own API key and wraps different commands to capture execution data.

Why it matters: Separate CLI agents for different test types (unit, integration, e2e) let you analyse patterns specific to each test suite. You can identify which test types are flaky, which catch the most bugs, and which provide the best ROI.

Example setup:

Wyrd Tech (Organisation)
├── Obvyr API (Project)
│   ├── Unit Tests (CLI Agent) - Pytest unit test execution patterns
│   ├── Integration Tests (CLI Agent) - Pytest integration test reliability
│   └── E2E Tests (CLI Agent) - Playwright/Cypress test behaviour
├── Obvyr CLI (Project)
│   ├── Unit Tests (CLI Agent) - Pytest unit test analysis
│   └── Integration Tests (CLI Agent) - Pytest integration test tracking
└── Obvyr UI (Project)
    ├── Unit Tests (CLI Agent) - Vitest unit test patterns
    └── E2E Tests (CLI Agent) - Cypress end-to-end test insights

Observations

Every time you run a command wrapped by the Obvyr CLI, it creates an observation. This captures:

• JUnit XML test results (parsed into individual test metrics)
• Command output (stdout/stderr)
• Execution duration and timing
• User who ran the command
• Environment context and variables

Why it matters: Obvyr parses the JUnit XML from each observation to extract individual test results. Thousands of observations become patterns. Obvyr analyses these patterns to reveal:

• Flaky tests: Tests that fail inconsistently across observations
• Execution patterns: How tests behave across different users and contexts
• Test-level insights: Pass rates, failure trends, and execution time patterns
• Performance trends: Tests getting slower over time

The Obvyr Difference

What Traditional Testing Shows You

• ✅ Test passed (but was it reliable or just lucky?)
• ❌ Test failed (but is it broken or flaky?)
• 📊 85% coverage (but does that coverage catch bugs?)
• ⏱️ 45-minute CI (but which tests provide value?)

What Obvyr Shows You

• ✅ "This test passed in 847/847 executions (100% reliable)"
• ❌ "This test failed in 23/150 executions (15% flaky) - pattern suggests timing issue"
• 📊 "Test execution trends show pass rate declining from 98% to 89% over past month"
• ⏱️ "This test's average execution time increased from 1.2s to 3.5s over 6 months"

The Obvyr Workflow: From Setup to Insights

1. Set Up Your Structure

Create projects in the Obvyr dashboard that match how your team organises testing

Time: 2 minutes Value: Clear data organisation for targeted insights

2. Create CLI Agents

Within each project, create CLI agents for different test types you want to monitor

Time: 3 minutes Value: Separate pattern analysis for unit tests, integration tests, linting, type checking

3. Install and Configure the Obvyr CLI

Install the CLI and configure it with your CLI agent API keys

Time: 2 minutes Value: Start capturing comprehensive test execution data

4. Wrap Your Commands

Replace pytest tests/ with obvyr-cli pytest tests/ (same for any test command)

Time: 1 minute Value: Zero workflow disruption, immediate data collection

5. Analyse the Insights

View patterns, trends, and evidence-based test reliability in the dashboard

Time: Ongoing Value: Prove test reliability, identify flaky tests, optimize CI, prevent incidents

What's Next?

Ready to prove your tests are reliable?

• Why Obvyr? - Understand the full value proposition and what makes Obvyr different
• Problems Solved - See detailed scenarios of specific testing challenges Obvyr solves
• Getting Started - Set up your first project and start collecting evidence in 10 minutes

From Hope to Proof in 10 Minutes

Stop assuming your tests are reliable. Start proving it. Get started now.