Hi Geert,
thank you for choosing to evaluate the Test Studio. We always appreciate user feedback and suggestions on how to make our product better so let me go through this stuff one by one:
Test runner
- Test runner on different machine *must* use run as administrator (I get an error), still it continues to run. Quit the app if it must be executed as administrator. Also, *force* it to be started as administrator. - I don't entirely understand what you mean here. Is this in the context of Scheduling? What are you referring to when you say "Test Runner" - would you please give us some details. If you're referring to the ArtOfTest.Runner.exe - then yes - you will need to be logged in as an administrator.
- Configure as scheduler must also be executed as administrator - yes, this is correct. Why do you feel that both of these are a problem? We've very interested in some details on this.
Scheduler
- The create database in the scheduler only works half of the time. In fact, I had to create an empty database and then it could be created. The server was not listed automatically, though other servers (that MS SQL Manager does not recognize) were found.
Would you elaborate on this. In my experience creating the database is not complex. 90% of the time people usually install an SQL server with all the default settings. This means that 90% of the time you just have to type in
(local)\SQLEXPRESS
and check "integrated security" and you will be able do create the database without problems. I say that as someone who closely monitors Test Studio's tech support correspondence and then kind of problems our customers face.
- Start execution server at startup does not work (read: is useless). It must be executed as administrator. - I was not able to reproduce this i.e. it worked when I tried it in my environment. Would you please give me the necessary steps to reproduce the problem. And also the environment and the Test Studio version you're using.
Load test services
- Load test services has no password field, it just shows the password there in clear text. - are you referring to the Services Configuration menu and specifically to the SQL Authentication credentials? Excellent suggestion! I've added this as a bug: http://www.telerik.com/support/pits.aspx#/public/test-studio/13685
500 Telerik points added to your account.
- Load test services has different button sizes. I know this is low prio, but it gives me the feeling the product is not developed with care. - could you attach a screenshot that highlights the buttons in question. Jing is my favorite software for this but a simple Windows screencapture + Paint will work too.
Test studio
- The navigation inside the app is really hard to understand. - would you please elaborate on this. We honestly think our product is very intuitive so we're very interested in hear what you didn't like and how you feel it can be improved.
- Why run as administrator?- Well for us to be able to hook into and control the browser (IE specifically) requires Admin level access. For us to be able to hook into and control the browser, (IE specifically I think) requires Admin level access. So if you want to be able to record or playback your test in IE - Admin security level is required. It's not required for anything else. And it's worth noting that we are evaluating the effort required to record in alternate browsers. Admin level access is required for the Visual Studio plugin.
ScheduleService API
We are trying to automate the scheduler service. What we do is create a client for the web service and schedule a new test for the next minute. Then we wait and we are able to see whether a test succeeds or fails. However, we cannot find any documentation on this service. Also, we cannot find any way how we can automate the retrieval of the log. In case of a failing UI test, we want to create a JIRA issue (we will automate this ourselves because it lacks JIRA support), but we do need the test log. - we have the Scheduling feature which will allow you to do all of this stuff without the need to access the Scheduling services yourself in code. When you schedule a TestList (through Test Studio's GUI) to run on remote machine the results will be automatically fed back into your calendar view after the run has completed. So I don't see why would manually need to write code to get them.
But if you absolutely want to do it - you will have to get the results from the Remote machine. The results are first generated in the Output folder on the remote machine and only then are fed back into the Calendar. The default location for the Output folder is
C:\Temp\Output
Bug tracking is supported out of the box for TFS. But you can make it work with Jira too by writing a custom plugin as described here:
http://www.telerik.com/automated-testing-tools/support/documentation/user-guide/extensions/bug-tracking/custom-plugin.aspx
I hope this helps. When submitting support tickets/forum posts in the future please try to break them down a bit more. What I mean is people don't group together different problems under the same ticket/post. This messes with out ticketing system and virtually guarantees that you'll get a slower response then you would have otherwise.
Thank you for your feedback!
All the best,
Stoich
the Telerik team