# Test Studio Tests and Azure DevOps Build Pipeline

<a href="https://www.telerik.com/teststudio" target="_blank">Test Studio</a> tests can be successfully integrated for execution with the Azure DevOps Build pipelines. Below you can read details for the prerequisites and necessary configurations to apply.

## Environmental Prerequisites

The build machine can be any virtual or physical machine to host the Test Studio Runner. Please refer to the <a href="/system-requirements" target="_blank">system requirements</a> for the required machine configuration.

The components to be installed on the machine:

* **Test Studio Run-time Edition** (ArtOfTest.Runner.exe must be available): **mandatory**
* **Internet Browser** (IE latest, Chrome latest and Firefox latest, Edge latest): **mandatory**
* **Visual Studio 2019 Professional**: optional, required only for additional functions like build tasks and other non-directly related to running tests with Test Studio

## Configuring the Azure DevOps Build Pipeline

Below are listed the steps to configure the build pipeline in Azure DevOps.

### Create a new project in Azure DevOps

Start with creating a new project in Azure. The one used in this example is named *Test Studio 1*.

![Create New project](https://www.telerik.com/teststudio/documentation/assets/37a322978ad9ae70ad755c4a0588d965/fig1.png)

### Create a Self-hosted Agent

Depending on whether tests should be running as a part of the app under test deployment to Live/Test environments or as part of daily builds there are **two types of agents**. If tests are supposed to run as a part of the release, the agent executing the tests should be registered in a Deployment group. If tests are supposed to run on demand after each build of the source code, the agent executing the tests should be registered to an Agent pool.<br><br>

The tests should be run using <a href="https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/v2-windows?view=azure-devops" target="_blank">self-hosted Windows agents</a> in order to be able to use the corresponding Test Studio installation.

To create such agent, follow the steps listed below

1. Open the current **Project settings**.

   ![Open the current project settings](https://www.telerik.com/teststudio/documentation/assets/184b04f5817115bcb4b045cc4f701faa/fig2.png)

2. Click on **Pipelines -> Agent Pools**.

   ![Agent Pools](https://www.telerik.com/teststudio/documentation/assets/52f27334fc0e8830216403b2a62294b4/fig3.png)

3. Add a new pool or open the default one.

   ![New Agent Pools](https://www.telerik.com/teststudio/documentation/assets/81033579e7f265534599017f6a1bab5d/fig4.png)

4. Click **New Agent**,

   ![New Agent](https://www.telerik.com/teststudio/documentation/assets/1bb23fe81a755676fabdcf42118b527b/fig5.png)

5. Download the Agent to the corresponding physical or virtual machine as per the installation requirements above.

6. Extract the Agent and run as Administrator.

   ![Start Agent](https://www.telerik.com/teststudio/documentation/assets/96e623b595ec1cbc71ca84c18ab4d8b9/fig6.png)

> **Note** In order to avoid any permissions issues, please make sure that the user configuring the Agent is part of the following Permission Groups: Build Administrators, Release Administrators, Project Administrators, **ProjectName** Team.

### Configure a Self-hosted Agent

1. Enter server URL: i.e. <https://dev.azure.com/OrganizationName>.

2. Create Personal Access Token (PAT) as described <a href="https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/v2-windows?view=azure-devops#permissions" target="_blank">here</a> and save it.

3. Choose PAT for authentication type and paste yours.

4. Choose the Agent pool you created earlier.

5. Choose Agent name, the default name is the machine name on which it is hosted.

6. Choose a work folder or press enter for the default one \_work sub-folder.

7. Select N when asked to enter agent as a service.

8. Enter configure autologon and run agent on startup Y/N is optional.

   ![Configure Agent](https://www.telerik.com/teststudio/documentation/assets/2f101e6b35feb67fa78fb4db945a9885/fig7.png)

9. Upon successful configuration the config console will close, so execute run.cmd in console to trigger the Agent.

   ![Trigger Agent](https://www.telerik.com/teststudio/documentation/assets/9e7e6769a192b290b6b51caad796bd12/fig8.png)

### Run Tests on Agent

1. Go to **Pipelines >> Builds >> Create new**.

   ![Create Pipeline](https://www.telerik.com/teststudio/documentation/assets/7a5ae736f3d4018e5db002e16b07b47f/fig9.png)

2. For this setup choose the option to **Use the classic editor**.

   ![Create Pipeline](https://www.telerik.com/teststudio/documentation/assets/c01d57098dc749b09bce90bffbcf4b93/fig10.png)

3. Choose the repository where your code is and click **Continue**.

   ![Choose repo](https://www.telerik.com/teststudio/documentation/assets/566b4b08efba0a89378714dac161efb7/fig11.png)

4. Choose **Empty job** as a template for the build

   ![Choose empty job](https://www.telerik.com/teststudio/documentation/assets/5dafd01fbce1822954de4b6b1ec6eaf3/fig12.png)

5. Select a pipeline and choose the Agent pool configured earlier.

   ![Select pipeline and agent pool](https://www.telerik.com/teststudio/documentation/assets/691cacf6ef9dfda80dbc33bce321636b/fig13.png)

6. Create a new task on the **Agent -> Utility -> Command line**.

   ![Command line new task](https://www.telerik.com/teststudio/documentation/assets/4911d2653d2363bbb3807421461ad9f0/fig14.png)

7. Insert a command which triggers the test execution via the Test Studio CLI Runner called ArtOfTest.Runner.exe - see the <a href="/features/test-runners/artoftest-runner" target="_blank">CLI runner syntax options here</a>.

> **Note!** The **Publish results** task in the Azure pipeline supports reading results in *junit* format. Test Studio CLI runner provides the option to automatically output the results in that format, so if you intend to use that task **make sure you add the *junit* option in the command**.

The below example executes a test list.

![Command line script](https://www.telerik.com/teststudio/documentation/assets/9e49847717629e126077eaa2c9e8b24c/fig15.png)

8. Select **Continue on error** from the Command line script **Control Options**, if you’d like to complete a results' upload task (see next point) even if the test execution task fails.

   ![Control Options](https://www.telerik.com/teststudio/documentation/assets/0014caa1b2b931a0b0181cd1ad0fdcba/fig16.png)

9. To add a **Publish Test Result** task, select the folder to which the test results are deployed and choose to upload all .xml files \*\*.xml for file search for example.

   ![Publish Test results task](https://www.telerik.com/teststudio/documentation/assets/2e6b63f892bb5b5bc247fcd73e67b325/fig17.png)

Something to consider here is whether results are piled together in this folder, if that’s the case there may be a need to make sure the results folder is empty before each run. If the folder is part of the build folders managed by the Azure agent, then this is not a concern because they are recreated for each run.

![Publish Test results](https://www.telerik.com/teststudio/documentation/assets/52996f740218c295bb99130d00e4cfc2/fig18.png)

### Results from Test Execution

Once the test run finishes, you can see the results in the **Test** tab.

![See test results](https://www.telerik.com/teststudio/documentation/assets/c52b77a4da4a6bbc91afe9410cc682da/fig19.png)
