Azure DevOps

Prerequisites

In order to run a test case, we need 3 things: Cloudbeat API Key, test case id and test type.

Cloudbeat API Key

To find your api key, go to Account > Settings > Security

or follow this link: https://app.cloudbeat.io/#/settings/security/

Test Case ID

To find a case id simply open cases and open the case you want to run.

Test Type

You can choose between a case, which is used for a singular test, or suite for multiple tests.

Azure Pipeline

Start by creating a new pipeline in Azure

• Log In and Navigate

  • Go to Azure DevOps, log in, and open your project.

• Go to Pipelines

  • Click on Pipelines in the left-hand menu, then choose New Pipeline.

• Select Source Repository

  • Choose the repository where your code is stored (e.g., Azure Repos or GitHub).

• Choose the Classic Editor

  • When setting up the pipeline, select Use the Classic Editor instead of YAML.

• Configure Pipeline Settings

  • Set the pipeline name (e.g., Cloudbeat-cli Run Test Example).

  • Under Agent Pool, choose Azure Pipelines and select the Agent Specification (e.g., ubuntu-22.04).

• Select the Repository Source:

  • In the pipeline editor, under Get Sources, click the edit icon.

  • Choose GitHub (or another repository type based on your project).

• Authorize the Connection:

  • If GitHub is selected, ensure it’s authorized using an access token. Click Change to modify or reauthorize if necessary.

• Set Repository and Branch:

  • Repository: Select the specific GitHub repository (e.g., oxygenhq/docs).

  • Default Branch: Specify the branch to build from (e.g., refs/heads/master).

• Adjust Source Settings:

  • Clean: Set to false (default) to avoid cleaning the workspace before building. Change to true if a fresh workspace is needed for every run.

  • Tag Sources: Choose whether to tag the repository:

    • Never: Don’t tag builds (default).

    • On Success: Tag only successful builds.

    • Always: Tag every build.

• Report Build Status:

  • Check the box to report the build status back to GitHub.

Define Variables

You can create variables to store data such as API keys and other data to use across the tasks.

Add Agent Job

First we need to add a command which will install the cloudbeat cli

Install Cloudbeat CLI

Add a command line task, name it "Install @cloudbeat/cli"

Add the script:

npm install -g @cloudbeat/cli

Run Test Case

To run our test, we need to add another command line and pass in the test type, id, and api key:

cloudbeat-cli start testType testId --apiKey=apiKey

But since we defined our variables, we can add this to the script:

cloudbeat-cli start $(cbTestType) $(cbTestId) --apiKey=$(cbApiKey)

when the test ends, it will create an JUnit XML results file

Publish Test Results

Lastly we need to publish the results. Add another task called "Publish Test Results":

Run Pipeline

Press on Queue and Run

Inside Cloudbeat, you will see the test case starting

After it's finished, you can see the results by following the link under Publish Test Results: