This is a migrated thread and some comments may be shown as answers.

Display report automation test on Cruise control

38 Answers 495 Views

General Discussions

This is a migrated thread and some comments may be shown as answers.

This question is locked. New answers and comments are not allowed.

Totti asked on 24 Feb 2011, 11:25 AM

Report1.JPG

Report2.JPG

Hello,
I run auromation test from Cruise control by MSTest.exe, then generated result test as results.trx, after that, using plug-in MsTestReport2008.xsl to display result test on Cruise Control. The Report display successfully but the view of report's not quite nice (much information's redundancy). You can see attached report here(2 attached files on single report).
So Could we do this report better ?

Many thanks.

38 Answers, 1 is accepted

answered on 24 Feb 2011, 11:21 PM

Hello Totti,

You are correct, it does not look very good in CruiseControl. How does that same .TRX file look when loaded into Visual Studio? I expect it will look just fine. If so it means that the MsTestReport2008.xsl transform file, which comes with CruiseControl needs to be modified to work with with .trx files that contain WebUI test results.

Since this file is not owned or controlled by Telerik, we're not sure what we can do about it. I have logged feature request 109065 to track this.

All the best,
Cody

the Telerik team

Registration for Q1 2011 What’s New Webinar Week is now open. Mark your calendar for the week starting March 21st and book your seat for a walk through all the exciting stuff we ship with the new release!

answered on 25 Mar 2011, 04:34 AM

TestReport.JPG

Hello,
I have customized xsl file completely and integrate with CruiseControl as its plug in (this xsl file will merge *.trx file of MStest to generate test report).
You can see the test report on Cruise Control in attached file.
What do you think about this ?

answered on 28 Mar 2011, 10:55 AM

TestReportFailure.JPG

Log.JPG

Hi,
- You can see test report in case of existing some fail tests (Attached file)
- I'm having issue with CruiseControl's log file: In the XML log file of CruiseControl , in tag WebAiiTestResult's existing empty namespace : xmlns="", So for this namespace, I 'm impossible to access in child node of WebAiiTestResult (You can see attached). So Do you know how to resolve for this? or How to avoid empty namespace ?
Thanks.

answered on 28 Mar 2011, 06:19 PM

Hi Totti,

Yes, a report with a failure looks great! I've gone ahead and given you 2500 Telerik Points as previously discussed. Can you please attach the updated .xsl file to this forum thread (put it into a .zip file first). Thanks!

Regarding the xmlns issue, there is no way to not have it blank. Microsoft MSTest is setting it that way (rather than CruiseControl). Is there specific information or XML nodes that you are not able to access because of this? It appears in your screenshot you are able to access everything that is necessary.

Kind regards,
Cody
the Telerik team

Vote for the 7 Telerik products at the 2011 Great Indian Developer Awards!

answered on 30 Mar 2011, 11:32 AM

AutomationTestReport.zip

Hi,
- Regarding the xmlns issue, In order to get steps information, I need to access to child node of WebAiiTestResult in XML file but for blank namespace , I cannot declare and use blank namespace in XSL file.
- I also put my xsl file in the forum.

Thanks.

answered on 15 Apr 2011, 12:08 AM

Hello Totti,

I did a little research into this and discovered you can access the nodes that have a default namespace. You need to use an XPath like this:

<xsl:for-each select="./tt:TestRun/tt:Results/art:WebAiiTestResult/WebAiiTestResult/AutomationStepResults/Item">

Note how the last three navigation nodes have no namespace prefix unlike everything before it. This means to use the default/empty namespace.

Regards,
Cody
the Telerik team

Do you want to have your say when we set our development plans? Do you want to know when a feature you care about is added or when a bug fixed? Explore the Telerik Public Issue Tracking system and vote to affect the priority of the items

answered on 14 Nov 2011, 05:34 PM

mstestrepot2008.jpg

I tried using the attached XSL file to display my test results in CruiseControl but am not having much luck. My output is attached.

I am using the VS2010 plugin and the command line to build the trx results file. You can see that the actual Test Studio information does not get displayed.

I have tried accessing the WebAii result elements directly (e.g. <xsl:value-of select="./build/tt:TestRun/tt:TestDefinitions/tt:WebAiiTest/@creation" />) but that was unsuccessful.

Any help would be appreciated.

answered on 24 Nov 2011, 03:37 PM

AutomationTestReportDetail.zip

Hi David,

I'm sorry for the delay getting back to you.

The XSL file you are refering to is created to parse the .trx results generated with Visual Studio 2008 distribution. I found out that there are a lot of changes in the microsoft schema in VS2010. I was able to create a new XSL file according the new schema and now the test results will be displayed as expected.

Please find attached the new AutomationTestReportDetail XSL file and let me know if you need further assistance on this.

Greetings,
Plamen
the Telerik team

Check out Telerik Trainer, the state of the art learning tool for Telerik products.

answered on 29 Nov 2011, 10:04 AM

UnitTestResults.png

TestStudioTestResults.png

AutomationTestReportDetailTStest.zip

Hello David,

The posted xsl file is meant to be used for displaying the results from a Unit test execution. The Unit test contains no steps, so the information about the passed/failed steps will not be displayed(see screenshot).

Please check out this video for an example of Unit test execution.
http://screencast.com/t/rPrUCw1df

In case you are running Test Studio tests(.tstest file), you can use our schema "http://artoftest.com/schemas/WebAiiDesignCanvas/1.0.0" to parse the execution results into CruiseControl.

Here's a video of how to do that:
http://screencast.com/t/nkqPmot1S7xH
Note that when I changed the xsl file in the dashboard.config, I restarted the ccnet Application Pool Tasks from the IIS Manager. This is necessary for the changes to take effect. You can find attached the xsl file I'm using in this example (AutomationTestReportDetailTStest.xsl).

Hope this helps!

Greetings,
Plamen
the Telerik team

Check out Telerik Trainer, the state of the art learning tool for Telerik products.

answered on 19 Dec 2011, 05:51 PM

Plamen,
I am just getting back to this and am still having trouble.

I know I am using the correct XSL file and it is being displayed. My dashboard.config is:

<buildPlugins>

<buildReportBuildPlugin>

<xslFileNames>

<xslFile>xsl\header.xsl</xslFile>

<xslFile>xsl\modifications.xsl</xslFile>

<xslFile>xsl\AutomationTestReportDetailTStest.xsl</xslFile>

<xslFile>xsl\compile-msbuild.xsl</xslFile>

<xslFile>xsl\MsTestSummary2008.xsl</xslFile>

</xslFileNames>

</buildReportBuildPlugin>

<buildLogBuildPlugin />

<!-- <xslReportBuildPlugin description="MSTest 2008 Detail Report" actionName="MSTEST2008DetailReport" xslFileName="xsl\AutomationTestReportDetailTStest.xsl" /> -->

<xslReportBuildPlugin description="MSBuild Report" actionName="MSBuildBuildReport" xslFileName="xsl\msbuild.xsl" />

</buildPlugins

>



I have restarted the ccnet app pool item in IIS to when making the dashboard.config changes.



I am still not seeing test case results.

answered on 21 Dec 2011, 01:20 PM

Hi David,

Your dashboard.config looks correct to me. Could you please provide the ccnet.config file, because most probably the problem is there?

Also, check the "publishers" section of your project configuration, it should point to the same folder where your "resultsfile" is. Here's an example:

<exec>

<executable>C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\mstest.exe</executable>

<baseDirectory>C:\CruiseControl\MSTests\CruiseControlTest\</baseDirectory>

<buildArgs>/testmetadata:CruiseControlTest.vsmdi /resultsfile:CruiseControlTest\testResults.trx</buildArgs>

<buildTimeoutSeconds>60</buildTimeoutSeconds>

</exec>

Publishers:

<publishers>

<merge>

<files>

<file>C:\CruiseControl\MSTests\CruiseControlTest\CruiseControlTest\testResults.trx</file>

</files>

</merge>

<xmllogger />

</publishers>

Note that the path placed in the "publishers" section is the same as the combination of "baseDirectory" plus "resultsfile".

If this is not the case please provide the ccnet.config file, so we can look further into this.

Regards,
Plamen
the Telerik team

Check out Telerik Trainer, the state of the art learning tool for Telerik products.

answered on 07 Aug 2012, 02:48 AM

hi need your help..

what script should i do to remove this error:

The results file "TestResults\testResults.trx" already exists. Please specify a different results file or verify the existing file is no longer needed and delete it


and the list of test are names are not displaying.

answered on 09 Aug 2012, 03:02 AM

unitTest.jpg

Hi Cody,

I made already a script to delete existing .trx file

My another problem was test results name don't appear only the number of test.

Here is my Config:

<cruisecontrol xmlns:cb="urn:ccnet.config.builder">

<project name="ApplicationTestingTP" queue="ApplicationTestingTP" queuePriority="1">

<workingDirectory>C:\TestProject</workingDirectory>

<artifactDirectory>C:\TestProject\ApplicationTestingTP\bin\Debug</artifactDirectory>

<webURL>localhost/ccnet</webURL>

<modificationDelaySeconds>20</modificationDelaySeconds>

<triggers>

<intervalTrigger name="continuous" seconds="300" buildCondition="IfModificationExists" />

</triggers>

<state type="state" directory="C:\ccnetstate" />

<labeller type="iterationlabeller">

<prefix>1.0</prefix>

<duration>1</duration>

<releaseStartDate>2012/7/30</releaseStartDate>

<separator>.</separator>

</labeller>

<tasks>

<msbuild>

<executable>C:\Windows\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe</executable>

<workingDirectory>c:\TestProject</workingDirectory>

<projectFile>ApplicationTesting.sln</projectFile>

<buildArgs>/noconsolelogger /p:Configuration=Debug /v:Minimal</buildArgs>

<targets>Build</targets>

<timeout>900</timeout>

</msbuild>

<exec>

<executable>c:\TestProject\delTestLog.bat</executable>

<baseDirectory>c:\TestProject\TestResults\</baseDirectory>

</exec>

<exec>

<executable>C:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE\mstest.exe</executable>

<baseDirectory>C:\TestProject\</baseDirectory>

<buildArgs>/testmetadata:ApplicationTesting.vsmdi /resultsfile:TestResults\testResults.trx</buildArgs>

<buildTimeoutSeconds>60</buildTimeoutSeconds>

</exec>

</tasks>

<publishers>

<merge>

<files>

<file>C:\TestProject\TestResults\testResults.trx</file>

</files>

</merge>

<xmllogger />

</publishers>

</project>

</cruisecontrol>

Dashboard.config

<?xml version="1.0" encoding="utf-8"?>

<dashboard>

<remoteServices>

<servers>

<server name="local" url="tcp://localhost:21234/CruiseManager.rem" allowForceBuild="true" allowStartStopBuild="true" backwardsCompatible="false" />

</servers>

</remoteServices>

<plugins>

<farmPlugins>

<farmReportFarmPlugin categories="false" refreshInterval="0" successBar="Bottom"/>

<cctrayDownloadPlugin />

<administrationPlugin password="" />

</farmPlugins>

<serverPlugins>

<serverReportServerPlugin />

</serverPlugins>

<projectPlugins>

<projectReportProjectPlugin />

<viewProjectStatusPlugin />

<latestBuildReportProjectPlugin />

<viewAllBuildsProjectPlugin />

</projectPlugins>

<buildPlugins>

<buildReportBuildPlugin>

<xslFileNames>

<xslFile>xsl\header.xsl</xslFile>

<xslFile>xsl\modifications.xsl</xslFile>

<xslFile>xsl\AutomationTestReportDetailTStest.xsl</xslFile>

<xslFile>xsl\compile-msbuild.xsl</xslFile>

<xslFile>xsl\MsTestSummary2010.xsl</xslFile>

</xslFileNames>

</buildReportBuildPlugin>

<buildLogBuildPlugin />

<xslReportBuildPlugin description="MSBuild Report" actionName="MSBuildBuildReport" xslFileName="xsl\msbuild.xsl" />

</buildPlugins>

<securityPlugins>

<simpleSecurity />

</securityPlugins>

</plugins>

</dashboard>

answered on 09 Aug 2012, 08:35 PM

Hello,

I see this section:
<exec>
<executable>c:\TestProject\delTestLog.bat</executable>
<baseDirectory>c:\TestProject\TestResults\</baseDirectory>
</exec>

However you didn't include a copy of delTestLog.bat so I don't know what that does... I can't even confirm calling a .bat file from the <exec> section works in CruiseControl (that's beyond our CC expertise).

If it were me I'd replace the section with this:
<exec>
<executable>del /f /q testResults.trx</executable>
<baseDirectory>c:\TestProject\TestResults\</baseDirectory>
</exec>

Bottom line the .trx file isn't getting deleted as expected prior to running MSTest. You may get better answers contact some CC.NET experts.

Kind regards,
Cody
the Telerik team

Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings

answered on 11 Aug 2012, 01:34 AM

317521-UnitTestResults.png

screenshot2.jpg

Hi Cody,

Please see attached files.

Thanks for your help.

answered on 23 Aug 2012, 02:43 PM

8-23-2012-9-17-54-AM.png

TestStudioMSTest2010Report.zip

Hello Randrew,

I apologize for the delay getting back to you, but I think you're going to like what I came up with for you. Instead of modifying your existing xsl files I created a new custom one just for Test Studio test results. You will now get a report similar to the attached screen shot. The custom XSL file is contained in the attached .zip file. Place it into your folder: C:\Program Files (x86)\CruiseControl.NET\webdashboard\xsl

Next modify the buildReportBuildPlugin section of your dashboard.config file to look like:

<buildReportBuildPlugin>

<xslFileNames>

<xslFile>xsl\header.xsl</xslFile>

<xslFile>xsl\modifications.xsl</xslFile>

<xslFile>xsl\compile-msbuild.xsl</xslFile>

<xslFile>xsl\TestStudioMSTest2010Report.xsl</xslFile>

</xslFileNames>

</buildReportBuildPlugin>

Optionally you can add a custom report entry:
<xslReportBuildPlugin description="Test Studio Report" actionName="TestStudioReport" xslFileName="xsl\TestStudioMSTest2010Report.xsl"></xslReportBuildPlugin>

The only problem I haven't worked out yet is calculating the duration of the test. The total time it took to run all of the tests is not contained in the build log. The start and finish time is there making it possible to calculate and the duration of each individual test is also stored (not currently displayed but could be) just not the grand total. I decided to give you what I have since it's 99% there and gives you the details you're looking for. I'm going to continue working on this and post an update later when I figure out how to calculate the grand total duration in XSL. Kind regards,
Cody
the Telerik team

Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings