I believe I got this working by installing a second Hosted agent on the TFS server. I created a new Agent Pool, "Hosted-Interactive" then went through the prompts of the ConfigureAgent.cmd (https://www.visualstudio.com/en-us/docs/build/admin/agents/v1-windows):
- An existing configuration file was detected. This will update the local agent settings. Do you want to also replace the server registration (default is N)? n
- Enter the name for this agent (default is Agent-[SERVER]) Agent-Interactive
- Enter the URL for the Team Foundation Server (default is ) http://[server]:8080/tfs
- Configure this agent against which agent pool? (default pool name is 'default')Hosted-Interactive
- Enter the path of the work folder for this agent (default is 'C:\Agent\_work') d:\agent\_work
- Would you like to install the agent as a Windows Service (Y/N) (default is N) n
"NO" to the final question. Now I have an interactive Agent Pool.
I then had to create another Environment, TEST, in my release to run the powershell script you originally provided above and selected the new Hosted-Interactive Agent Pool to run those tasks. See TEST_Environment.png.
I set this Environment to automatically trigger after the deployment to the DEV environment. As I said, the only task in this environment is to execute the powershell script which executes the tests I have created in Test Studio. Failures or successes in the tests themselves are accurately propagated to the deployment status and it will read Rejected upon any failure.
NOTE 1: I have not found a way to avoid requiring a command prompt open on the TFS server with the c:\agent\vsoagent.exe running. I tried starting it via a Command as the final step of the previous DEV deploy but that did not work. This might prevent this from being a viable solution for you.
NOTE 2: This only runs tests AFTER a deployment. Rolling back a deployment is not possible with this approach.