I am evaluating the studio to check whether it fits my needs. I have found a *lot* of issues, let me try to sum them up here:
Test runner
- Test runner on different machine *must* use run as administrator (I get an error), still it continues to run. Quit the app if it must be executed as administrator. Also, *force* it to be started as administrator.
- Configure as scheduler must also be executed as administrator.
Scheduler
- The create database in the scheduler only works half of the time. In fact, I had to create an empty database and then it could be created. The server was not listed automatically, though other servers (that MS SQL Manager does not recognize) were found.
- Start execution server at startup does not work (read: is useless). It must be executed as administrator.
Load test services
- Load test services has no password field, it just shows the password there in clear text.
- Load test services has different button sizes. I know this is low prio, but it gives me the feeling the product is not developed with care.
Test studio
- The navigation inside the app is really hard to understand.
- Why run as administrator?
ScheduleService API
We are trying to automate the scheduler service. What we do is create a client for the web service and schedule a new test for the next minute. Then we wait and we are able to see whether a test succeeds or fails. However, we cannot find
any documentation on this service. Also, we cannot find any way how we can automate the retrieval of the log. In case of a failing UI test, we want to create a JIRA issue (we will automate this ourselves because it lacks JIRA support), but we do need the test log.
Can you please provide info on how we can retrieve the test log for a test that is executed on a remote machine? Just like the test studio does.
Thanks.