Python Pytest

Integrating Python Pytest based tests with CloudBeat

The steps on this page can be done for you by an AI coding agent: see AI-Assisted Project Setup.

Installation

If you're starting from scratch, create a Python virtual environment inside your project first and activate it.

Linux / macOS

python3 -m venv .venv
source .venv/bin/activate

Windows

python -m venv .venv
.venv\Scripts\activate

Add the CloudBeat modules to your requirements.txt. cloudbeat-selenium is only needed for Selenium (or Appium) based tests:

pytest
cloudbeat-pytest
cloudbeat-selenium
selenium

To install them, use:

pip install -r requirements.txt

cloudbeat-pytest is a Pytest plugin which registers itself automatically: no changes to conftest.py or pytest.ini are needed for the basic reporting. Test cases, their setup and teardown, and failures are reported to CloudBeat.

The plugin is active only when the tests are executed by CloudBeat. When you run pytest locally or in your CI, nothing is reported and nothing changes.

Implementing CloudBeat Reporting

Wrapping the Selenium WebDriver

To see every browser action as a step in the CloudBeat report, with a screenshot when a step fails, wrap the WebDriver in the fixture which creates it. The CloudBeat reporter is available as request.config.cb_reporter while the tests are executed by CloudBeat, and is None otherwise:

import pytest
from selenium import webdriver


@pytest.fixture
def driver(request):
    driver = webdriver.Chrome()

    cb_reporter = getattr(request.config, "cb_reporter", None)
    if cb_reporter is not None:  # executed by CloudBeat
        from cloudbeat_selenium.wrapper import CbSeleniumWrapper
        driver = CbSeleniumWrapper(cb_reporter).wrap(driver)

    yield driver
    driver.quit()

The wrapped object is Selenium's EventFiringWebDriver. All WebDriver methods work as before; code which needs the underlying driver, for example execute_cdp_cmd, can use driver.wrapped_driver.

Using the browser selected in CloudBeat

When CloudBeat executes the tests, the address of the remote WebDriver and the selected browser are available as environment variables. Use them to run on the browser selected in CloudBeat instead of a local one:

import os
import pytest
from selenium import webdriver


@pytest.fixture
def driver(request):
    selenium_url = os.environ.get("CB_SELENIUM_URL")   # CB_APPIUM_URL for mobile tests
    if selenium_url:
        options = webdriver.ChromeOptions()  # match os.environ.get("CB_BROWSER_NAME")
        driver = webdriver.Remote(command_executor=selenium_url, options=options)
    else:
        driver = webdriver.Chrome()

    cb_reporter = getattr(request.config, "cb_reporter", None)
    if cb_reporter is not None:
        from cloudbeat_selenium.wrapper import CbSeleniumWrapper
        driver = CbSeleniumWrapper(cb_reporter).wrap(driver)

    yield driver
    driver.quit()

Named steps

Functions and methods, typically of your page objects, can be reported as named steps with the @cb.step decorator. Placeholders in the step name are filled in from the function arguments, and any steps created inside the function, such as Selenium actions, become its sub-steps:

from cloudbeat_common import cb


class LoginPage:
    @cb.step("Login as {username}")
    def login(self, username, password):
        self.driver.find_element(By.ID, "username").send_keys(username)
        self.driver.find_element(By.ID, "password").send_keys(password)
        self.driver.find_element(By.ID, "submit").click()

A block of code can be reported as a step with the cb.step_context context manager:

with cb.step_context("Verify the order total"):
    assert cart.total() == 42

Both are no-ops when the tests run outside CloudBeat, so they are safe to keep in the code.

Test descriptions

The docstring of a test function is shown as the description of the test case in CloudBeat:

def test_login():
    """A registered user can log in with valid credentials."""
    ...

Running locally

Nothing changes for local runs:

# run everything
pytest

# run parallel tests
pytest -n 4

# run a single test
pytest -v -s tests/test_login.py

Creating a Python Pytest project in CloudBeat

Under Manage, open Projects, click Add Project, choose a name and select Pytest.

Connect your test project to CloudBeat

CloudBeat is a platform for both test reporting and execution. To use CloudBeat, you must deliver your test project files, so that CloudBeat can execute the tests and provide report analytics. You can upload your project as a Zip file, or CloudBeat can synchronize with your test project Git repository.

Git synchronization

In order to sync your test project from Git, select Git Integration and paste the URL of your repository. You can use your username and password, or an access token.

Uploading manually

If you want to upload a local test project, zip your project folder and select Upload.

Make sure you skip the virtual environment folder (.venv), __pycache__, .pytest_cache and all unnecessary files such as results and reports.

How CloudBeat installs and runs your project

• CloudBeat creates a virtual environment and installs your dependencies from the requirements.txt in the project root. If the project has a pyproject.toml, the project itself is installed as well. Make sure cloudbeat-pytest (and cloudbeat-selenium, if used) are listed there.

• Tests are discovered and executed with python -m pytest from the project root, using CloudBeat's own test selection. Options your project needs must therefore be in pytest.ini or pyproject.toml (addopts), not in a shell script.

• CloudBeat agents run on Linux: your tests must run headless and must not depend on Windows paths or tools.

Running your tests

Under Testing, go to Cases and you will see all your tests. Click Run Now to execute a test; the results appear under the Results tab. Expand a result to see all the steps, data, logs and errors, including the Selenium steps and screenshots captured by the wrapped WebDriver.

Running multiple tests

To run several tests together, create a Test Suite under Testing, add the cases, and run the suite. See Creating Tests Suites and Running Tests Suites.