Link to this sectionCI/Testing Workflow 🧪#

Continuous Integration (CI) is essential for maintaining high-quality code by catching issues early. This guide covers CI testing and quality checks for Ultralytics projects.

Link to this sectionCI Actions 🔄#

All PRs must pass automated CI checks before merging. Our CI pipeline includes:

Link to this sectionCI Tests#

Primary CI test running unit tests, linting checks, and comprehensive tests.

Link to this sectionDocker Deployment#

Validates deployment using Docker, ensuring Dockerfile and related scripts work correctly.

Link to this sectionBroken Links#

Scans codebase for broken or dead links in markdown and HTML files.

Link to this sectionCodeQL Analysis#

GitHub's semantic analysis tool for finding potential security vulnerabilities and maintaining code quality.

Link to this sectionPyPI Publishing#

Validates project can be packaged and published to PyPI without errors.

Link to this sectionPlatform Testing 🖥️#

Tests run on multiple environments:

• OS: Ubuntu, Windows, macOS
• Python: 3.8, 3.9, 3.10, 3.11, 3.12

Link to this sectionCode Coverage 📊#

We use Codecov to measure and visualize code coverage, providing insights into how well tests exercise the codebase.

Link to this sectionCoverage Integration#

Codecov integration provides:

• Detailed coverage insights
• Coverage comparisons between commits
• Visual overlays on code showing covered lines
• Coverage percentage for the ultralytics package

View full coverage details at codecov.io/github/ultralytics/ultralytics.

Link to this sectionUnderstanding Coverage#

Code coverage shows what percentage of code is executed during tests. High coverage indicates well-tested code but doesn't guarantee absence of bugs. Coverage helps identify untested areas that might be prone to errors.

Link to this sectionRunning Tests Locally 🖥️#

Link to this sectionInstall Development Dependencies#

pip install -e ".[dev]"

Link to this sectionRun All Tests#

pytest tests/

Link to this sectionRun Specific Tests#

# Single file
pytest tests/test_engine.py

# Single test function
pytest tests/test_engine.py::test_train

# Tests matching pattern
pytest -k "export"

# Slow tests only
pytest -m slow

Link to this sectionRun with Coverage#

pytest --cov=ultralytics tests/

Link to this sectionParallel Testing#

# Install pytest-xdist
pip install pytest-xdist

# Run tests in parallel
pytest -n auto

Link to this sectionWriting Tests ✍️#

Link to this sectionTest Structure#

from pathlib import Path

from ultralytics import YOLO

def test_model_export():
    """Test ONNX model export."""
    model = YOLO("yolo26n.pt")
    model.export(format="onnx")
    assert Path("yolo26n.onnx").exists()

Link to this sectionBest Practices#

• Descriptive names: test_export_onnx_format() not test_1()
• Single assertion: Test one thing per function
• Fast tests: Use small models/datasets
• Fixtures: Use pytest fixtures for setup/teardown
• Markers: @pytest.mark.slow for long-running tests

Link to this sectionTest Organization#

tests/
├── test_engine.py      # Training, validation, prediction
├── test_nn.py          # Model architecture
├── test_data.py        # Dataset handling
├── test_utils.py       # Utility functions
└── test_exports.py     # Export formats

Link to this sectionTest Markers#

import pytest

@pytest.mark.slow
def test_full_training():
    """Test full training run (slow)."""
    model = YOLO("yolo26n.pt")
    model.train(data="coco128.yaml", epochs=1)

Link to this sectionCode Quality Checks 🎯#

Link to this sectionFormatting with Ruff#

# Check formatting
ruff check ultralytics/

# Auto-fix issues
ruff check --fix ultralytics/

# Format code
ruff format ultralytics/

Learn more about code standards in our development workflow.

Link to this sectionType Checking#

# Run mypy (where configured)
mypy ultralytics/

Link to this sectionDocstring Formatting#

pip install ultralytics-actions

# Auto-fix
ultralytics-actions-format-python-docstrings .

Link to this sectionCI Troubleshooting 🔧#

Link to this sectionTests Pass Locally But Fail in CI#

Common causes:

• Platform-specific issues: Test on target OS
• Python version differences: Check version compatibility
• Missing dependencies: Verify CI config
• Timing/concurrency issues: Add retries or increase timeouts

Link to this sectionSlow CI Runs#

Solutions:

• Use @pytest.mark.slow for expensive tests
• Mock external dependencies
• Reduce test dataset sizes
• Parallelize with pytest-xdist

Link to this sectionFlaky Tests#

Fixes:

• Add retries for network-dependent tests
• Increase timeouts for slow operations
• Fix race conditions in async code
• Use deterministic random seeds

Link to this sectionPerformance Benchmarks 📈#

CI tracks key metrics:

• Inference speed (FPS)
• Memory usage
• Model size
• Export times

Significant regressions block merging. If metrics change:

1. Verify change is expected
2. Document reason in PR
3. Get approval from maintainers

Link to this sectionCI Status 📋#

Check CI status for all Ultralytics repositories at docs.ultralytics.com/help/CI.

Link to this sectionMain Repository Badges#

Link to this sectionSkipping CI Checks ⚠️#

Add [skip ci] to commit message to skip CI (use sparingly):

git commit -m "Update README [skip ci]"

Only for:

• Documentation-only changes
• Non-code file updates
• Emergency hotfixes (with approval)

Link to this sectionResources 📚#

• Official CI Guide - Complete CI documentation
• Development Workflow - PR process and code standards
• GitHub Actions Docs - CI configuration
• pytest Documentation - Testing framework
• Codecov - Coverage reports