Configuring Test Studio project to use specific TFS credentials?

[Matthew]

Is there a way to configure a Test Studio project to use certain TFS credentials when it gets the latest tests from source control?

I am running Test Studio tests in a different domain than the TFS server. It appears that that it uses the credentials of the Telerik Test Studio Runtime service account (tst\teleriksvc) when talking to TFS. These credentials, of course, are not valid since the service account is in a different domain than the TFS server.

My project is setup as follows:

• The project logged in TFS using generic TFS credentials (dev\teleriksvc) that we setup for the Telerik service account to use.
• Attached the project to source control.

We plan to execute test lists using CommandLineClient. We tried executing the tests with various configurations, but all of them result in an error saying the credentials are invalid for TFS. From the CMD stack trace:

Telerik.TestStudio.Services.Common.Exceptions.SourceControlNotReachableException: Unable to reach source control server at: http://devtfs.dev.com:8080/tfs
   at Telerik.TestStudio.Shared.Communication.BaseCommunicationLayer.<CheckForKnownResponseCode>d__35.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Telerik.TestStudio.Shared.Communication.BaseCommunicationLayer.<HandleResponse>d__33.MoveNext()
--- End of stack trace from previous location where exception was thrown ---

....

We tried the following:

• Executed the test list with no additional configuration.
• Logged into TFS using dev\teleriksvc in browser. Execute test list.
• Modify the Telerik Scheduling Service account to "dev\teleriksvc" as noted here. I was unable to set updated account and received an error stating: "Error 1057: The account name is invalid or does note exist, or the password is invalid for the account name specified." I'm guessing this is a result of there not being an Active Directory Domain Trust between dev and test domains.
• Modified the project's setting.aiis file in the following way:
  • Set "TfsUserName": "teleriksvc" and "dev\\teleriksvc" and "dev\teleriksvc"
  • Set "TfsPassword": "{our password}"
  • Set "TfsDomain": "dev" and "" (empty)
  • I tried various combinations where the TfsUserName also includes the domain and where the TfsDomain is blank. None of these worked. What is the purpose of these configuration values if they don't work?

I tried running the CommandLineClient after each change noted above, and it resulted in the same stack trace error. If we remove the TFS integration with the project, the test list runs as expected using the CommandLineClient.

Ultimately, we simply want to provide the TFS username/password so that our service account provides those credentials instead of the credentials of the service account. How do we make this happen?

[Elena]

Hi Matthew,

Thank you for the thorough description of the requirements you need to cover and what you have tried so far - this is really helpful for me to understand what is still missing in the configuration in your environment. 

Basically, the Test Studio Scheduling is running as a Windows service and you have correctly identified how you need to impersonate the service and allow it to use specific credentials to access the TFS repository. There is an obstacle, though, and these are the different domains in which the user and the TFS are registered.

I searched the web for any solution on the topic and this seems to require some Active Directory adjustments between the domains to be able to perform cross domain communication. You can refer to a similar topic in a Microsoft forum post here. So, to be able to use the service across different domains, you will need to add trust relationships between these.

Please, let me know if you will need any assistance on the topic or have any concerns on the case. Thank you in advance for your cooperation. 

Regards,
Elena
Progress Telerik

 

[Matthew]

Elena,

Thank you for the thorough information, I appreciate it. What I ended up doing was adding the TFS credentials to the service account's "Windows Credentials" stored in Control Panel > User Accounts > Credential Manager. As a result, whenever the runtime/service account contacts TFS, it'll use these default permissions. I verified this, and it is working fine. (If I remove the Windows Credential, then scheduling a test list throws an error claiming it cannot connect to source control.)

I'm running into another issue though: when I run a test list, it isn't getting the latest test(s) from source control even though the execution server's log claims it is. My expectation is that it will get the latest files for this test list before execution. Please see the attached file. In particular, these lines claim it is getting latest:

[03/02 15:36:16,Telerik.TestStudio.RemoteExecutor.exe(5728:14),Warning] CloudRunnableGUITestBuilder.CreateLocalCacheAsync() : Started getting project from source control
[03/02 15:36:16,Telerik.TestStudio.RemoteExecutor.exe(5728:14),Execution] SourceControlMediator.GetProject() : Getting latest source control version of project ef216904-49f9-4f7e-a36d-28d73f52b4b2
[03/02 15:36:16,Telerik.TestStudio.RemoteExecutor.exe(5728:14),TeamFoundationServer] TFSServer.ConnectToServer() : ------- Function Start -------
[03/02 15:36:16,Telerik.TestStudio.RemoteExecutor.exe(5728:14),TeamFoundationServer] TFSServer.ConnectToServer() : ------- Function End -------
[03/02 15:36:16,Telerik.TestStudio.RemoteExecutor.exe(5728:14),TeamFoundationServer] TFSServer.AttachToRepository() : ------- Function Start -------
[03/02 15:36:16,Telerik.TestStudio.RemoteExecutor.exe(5728:14),TeamFoundationServer] TFSServer.AttachToRepository() : ------- Function End -------
[03/02 15:36:21,Telerik.TestStudio.RemoteExecutor.exe(5728:14),Execution] SourceControlMediator.GetProject() : The project files have been retrieved from source control for ef216904-49f9-4f7e-a36d-28d73f52b4b2
[03/02 15:36:21,Telerik.TestStudio.RemoteExecutor.exe(5728:14),Warning] CloudRunnableGUITestBuilder.CreateLocalCacheAsync() : Finished getting project from source control

 

Notes:

• To test whether the latest is getting pulled from source control, I am modifying one of the test (.tstest) files that the test list is calling.
• The scheduling and execution applications are running on the same server.

[Matthew]

Here is the attached log. I forget to add it to the last post!

Note: I intentionally failed the test. I am simply trying to verify that the latest test files are being pulled down from TFS on execution.

[Elena]

Hello Matthew,

I am pleased to know you managed to sort out the credentials issue. 

The trouble you see now is that you check-in to the TFS repository any test change and run a test list containing this change, using the 'Get Latest from TFS', but you don't see the change during the run. Please, correct me if I have misread any of the details. 

Based on the log file it seems you have closed the executing browser before the test run started - is that correct? And if it is, how do distinguish that the downloaded files from the TFS were not the changed ones? 

Can you, please, elaborate more on the particular changes that you apply to verify if the run is correct? Did you checked-in these before you trigger the run? 

Basically, the Test Studio execution process will connect to the TFS repository and will download the whole project available there. And with these shared details I am not able to identify what the difference, you see, is and what might be causing it. Therefore I will appreciate if you take the time to describe the misbehavior in further details. 

Thank you for your understanding in advance.

Regards,
Elena
Progress Telerik

 

[Matthew]

The trouble you see now is that you check-in to the TFS repository any test change and run a test list containing this change, using the 'Get Latest from TFS', but you don't see the change during the run.

Yes, this is the problem I am encountering.

Based on the log file it seems you have closed the executing browser before the test run started - is that correct? And if it is, how do distinguish that the downloaded files from the TFS were not the changed ones? 

I was under the impression that 'Get Latest From TFS' would retrieve/copy the latest files into the file directory mapped to TFS on the runtime computer. I was closing the browser so that I could check the files themselves for changes.

I let the test run and reviewed the log file. It appears my changes are being ran, but they aren't being copied to the hard drive on that computer. Is that how this feature works?

[Elena]

Hi Matthew,

Thank you for clarifying what your doubts are. To describe how this feature works, I will share some notes about the Test Studio Scheduling setup and how this is designed. 

The Test Studio Scheduling configuration is using the Scheduling and Storage services to control the test execution on multiple machines. The Storage service is responsible for keeping a copy of the project, which can be deployed to any execution machine, and uses the MongoDB as a storage database. 

Then, once the Scheduler prepares a job for execution, the project files, which may vary depending on the tests in the test list, get deployed from the storage database to a temp folder on the selected execution machines and the test list run is then started. The temp folder is being configured in the Execution server setup. 

This is the common case when the 'Get Latest from TFS' option is not enabled.

What is different when 'Get Latest from TFS' is enabled? 

The only difference is that the remote TFS repository replaces the storage database. Which means that the project gets deployed from TFS to the temp folder on the selected remote machine. 

So, if you have applied any changes in the project, checked these into the remote repo, and trigger a run through the Scheduling.CommandLineClient, you should see the changes within the test runs. Though, if you keep a copy of the project on the remote execution machine, this will not be affected by the test list run triggered to use the latest files from TFS. 

I hope this information helps you understand what to expect from the scheduled test list runs. Of course, if you still have any concerns or doubts, please, let me know.

Thank you for your cooperation in this discussion.

Regards,
Elena
Progress Telerik