Display report automation test on Cruise control

39 posts, 0 answers
  1. Totti
    Totti avatar
    10 posts
    Member since:
    Sep 2010

    Posted 24 Feb 2011 Link to this post

    Hello,
    I run auromation test from Cruise control by MSTest.exe, then generated result test as results.trx, after that, using plug-in MsTestReport2008.xsl to display result test on Cruise Control. The Report display successfully but the view of report's not quite nice (much information's redundancy). You can see attached report here(2 attached files on single report).
    So Could we do this report better ?

    Many thanks.
  2. Cody
    Admin
    Cody avatar
    3360 posts

    Posted 24 Feb 2011 Link to this post

    Hello Totti,

    You are correct, it does not look very good in CruiseControl. How does that same .TRX file look when loaded into Visual Studio? I expect it will look just fine. If so it means that the MsTestReport2008.xsl transform file, which comes with CruiseControl needs to be modified to work with with .trx files that contain WebUI test results.

    Since this file is not owned or controlled by Telerik, we're not sure what we can do about it. I have logged feature request 109065 to track this.

    All the best,
    Cody
    the Telerik team
    Registration for Q1 2011 What’s New Webinar Week is now open. Mark your calendar for the week starting March 21st and book your seat for a walk through all the exciting stuff we ship with the new release!
  3. Totti
    Totti avatar
    10 posts
    Member since:
    Sep 2010

    Posted 25 Feb 2011 Link to this post

    Thank for your answer,
    So I will investigate more and modify either Result file or MsTestReport2008.xsl  file to display report to be good view.
    (I opened result test *.trx file by Visual Studio is OK  good)
  4. Cody
    Admin
    Cody avatar
    3360 posts

    Posted 25 Feb 2011 Link to this post

    Hi Totti,

    That sounds very good. If you do come up with a modified MsTestReport2008.xsl that you are happy with, send it to us like a code sample, grant us permission to freely redistribute to our other customers and then I can grant you 2500 Telerik Points as a reward.

    Best wishes,
    Cody
    the Telerik team
    Registration for Q1 2011 What’s New Webinar Week is now open. Mark your calendar for the week starting March 21st and book your seat for a walk through all the exciting stuff we ship with the new release!
  5. Totti
    Totti avatar
    10 posts
    Member since:
    Sep 2010

    Posted 24 Mar 2011 Link to this post

    Hello,
    I have customized xsl file completely and integrate with CruiseControl as its plug in (this xsl file will merge *.trx file of MStest to generate test report).
    You can see the test report on Cruise Control in attached file.
    What do you think about this ?
  6. Cody
    Admin
    Cody avatar
    3360 posts

    Posted 25 Mar 2011 Link to this post

    Hi Totti,

    That screen shot looks perfect and beautiful! One question, how about one where a test failed? Then I think we can call it work complete.

    All the best,
    Cody
    the Telerik team
  7. Totti
    Totti avatar
    10 posts
    Member since:
    Sep 2010

    Posted 28 Mar 2011 Link to this post

    Hi,
    - You can see test report in case of existing some fail tests (Attached file)
    - I'm having issue with CruiseControl's log file: In the XML log file of CruiseControl , in tag WebAiiTestResult's existing empty namespace : xmlns="", So for this namespace, I 'm impossible to access in child node of  WebAiiTestResult (You can see attached). So Do you know how to resolve for this? or How to avoid empty namespace ?
    Thanks.
  8. Cody
    Admin
    Cody avatar
    3360 posts

    Posted 28 Mar 2011 Link to this post

    Hi Totti,

    Yes, a report with a failure looks great! I've gone ahead and given you 2500 Telerik Points as previously discussed. Can you please attach the updated .xsl file to this forum thread (put it into a .zip file first). Thanks!

    Regarding the xmlns issue, there is no way to not have it blank. Microsoft MSTest is setting it that way (rather than CruiseControl). Is there specific information or XML nodes that you are not able to access because of this? It appears in your screenshot you are able to access everything that is necessary.

    Kind regards,
    Cody
    the Telerik team
  9. Totti
    Totti avatar
    10 posts
    Member since:
    Sep 2010

    Posted 30 Mar 2011 Link to this post

    Hi,
    - Regarding the xmlns issue, In order to get steps information, I need to access to child node of WebAiiTestResult in XML file but for blank namespace , I cannot declare and use blank namespace in XSL file.
    - I also put my xsl file in the forum.

    Thanks.

  10. Cody
    Admin
    Cody avatar
    3360 posts

    Posted 14 Apr 2011 Link to this post

    Hello Totti,

    I did a little research into this and discovered you can access the nodes that have a default namespace. You need to use an XPath like this:

    <xsl:for-each select="./tt:TestRun/tt:Results/art:WebAiiTestResult/WebAiiTestResult/AutomationStepResults/Item">

    Note how the last three navigation nodes have no namespace prefix unlike everything before it. This means to use the default/empty namespace.

    Regards,
    Cody
    the Telerik team
    Do you want to have your say when we set our development plans? Do you want to know when a feature you care about is added or when a bug fixed? Explore the Telerik Public Issue Tracking system and vote to affect the priority of the items
  11. Totti
    Totti avatar
    10 posts
    Member since:
    Sep 2010

    Posted 15 Apr 2011 Link to this post

    Great, many thanks to Cody,
    I have applied your soultion in my report file XSL, It's OK now.

    Thanks and Regards,
    Totti

  12. Cody
    Admin
    Cody avatar
    3360 posts

    Posted 15 Apr 2011 Link to this post

    Hi Totti,

    I am very glad to hear it! Could you post your updated XSL for us?

    Greetings,
    Cody
    the Telerik team
    Do you want to have your say when we set our development plans? Do you want to know when a feature you care about is added or when a bug fixed? Explore the Telerik Public Issue Tracking system and vote to affect the priority of the items
  13. Totti
    Totti avatar
    10 posts
    Member since:
    Sep 2010

    Posted 19 Apr 2011 Link to this post

    Cody,
    You can find updated XSL file that attached in this post
    BTW, I can have more points for updating this file ?

    Thanks
  14. Cody
    Admin
    Cody avatar
    3360 posts

    Posted 19 Apr 2011 Link to this post

    Hello Totti,

    Thank you for submitting the updated .xsl file. Well done! I will speak to your account manager about what may be possible regarding Telerik Points.

    Kind regards,
    Cody
    the Telerik team
    Do you want to have your say when we set our development plans? Do you want to know when a feature you care about is added or when a bug fixed? Explore the Telerik Public Issue Tracking system and vote to affect the priority of the items
  15. David
    David avatar
    45 posts
    Member since:
    Aug 2010

    Posted 14 Nov 2011 Link to this post

    I tried using the attached XSL file to display my test results in CruiseControl but am not having much luck. My output is attached.

    I am using the VS2010 plugin and the command line to build the trx results file. You can see that the actual Test Studio information does not get displayed.

    I have tried accessing the WebAii result elements directly (e.g. <xsl:value-of select="./build/tt:TestRun/tt:TestDefinitions/tt:WebAiiTest/@creation" />) but that was unsuccessful.

    Any help would be appreciated.
  16. David
    David avatar
    45 posts
    Member since:
    Aug 2010

    Posted 14 Nov 2011 Link to this post

    I tried using the attached XSL file to display my test results in CruiseControl but am not having much luck. My output is attached.

    I am using the VS2010 plugin and the command line to build the trx results file. You can see that the actual Test Studio information does not get displayed.

    I have tried accessing the WebAii result elements directly (e.g. <xsl:value-of select="./build/tt:TestRun/tt:TestDefinitions/tt:WebAiiTest/@creation" />) but that was unsuccessful.

    Any help would be appreciated.
  17. Plamen
    Admin
    Plamen avatar
    8 posts

    Posted 24 Nov 2011 Link to this post

    Hi David,

    I'm sorry for the delay getting back to you.

    The XSL file you are refering to is created to parse the .trx results generated with Visual Studio 2008 distribution. I found out that there are a lot of changes in the microsoft schema in VS2010. I was able to create a new XSL file according the new schema and now the test results will be displayed as expected.

    Please find attached the new AutomationTestReportDetail XSL file and let me know if you need further assistance on this.

    Greetings,
    Plamen
    the Telerik team

    Check out Telerik Trainer, the state of the art learning tool for Telerik products.
  18. David
    David avatar
    45 posts
    Member since:
    Aug 2010

    Posted 28 Nov 2011 Link to this post

    I tried the file that Plamen posted and tried a number of different iterations on it (replacing the old .xsl file, using the same name and changing the dashboard.config file) and the page did not render with the test result information. I will try to up the file Plamen sent to get this to work.
  19. Plamen
    Admin
    Plamen avatar
    8 posts

    Posted 29 Nov 2011 Link to this post

    Hello David,

    The posted xsl file is meant to be used for displaying the results from a Unit test execution. The Unit test contains no steps, so the information about the passed/failed steps will not be displayed(see screenshot).

    Please check out this video for an example of Unit test execution.
    http://screencast.com/t/rPrUCw1df

    In case you are running Test Studio tests(.tstest file), you can use our schema "http://artoftest.com/schemas/WebAiiDesignCanvas/1.0.0" to parse the execution results into CruiseControl.

    Here's a video of how to do that:
    http://screencast.com/t/nkqPmot1S7xH 
    Note that when I changed the xsl file in the dashboard.config, I restarted the ccnet Application Pool Tasks from the IIS Manager. This is necessary for the changes to take effect. You can find attached the xsl file I'm using in this example (AutomationTestReportDetailTStest.xsl).

    Hope this helps!

    Greetings,
    Plamen
    the Telerik team

    Check out Telerik Trainer, the state of the art learning tool for Telerik products.
  20. David
    David avatar
    45 posts
    Member since:
    Aug 2010

    Posted 19 Dec 2011 Link to this post

    Plamen,
    I am just getting back to this and am still having trouble.

    I know I am using the correct XSL file and it is being displayed. My dashboard.config is: 
    <buildPlugins>
      <buildReportBuildPlugin>
        <xslFileNames>
          <xslFile>xsl\header.xsl</xslFile>
          <xslFile>xsl\modifications.xsl</xslFile>
          <xslFile>xsl\AutomationTestReportDetailTStest.xsl</xslFile>
          <xslFile>xsl\compile-msbuild.xsl</xslFile>
          <xslFile>xsl\MsTestSummary2008.xsl</xslFile>
        </xslFileNames>
      </buildReportBuildPlugin>
      <buildLogBuildPlugin />
    <!-- <xslReportBuildPlugin description="MSTest 2008 Detail Report" actionName="MSTEST2008DetailReport" xslFileName="xsl\AutomationTestReportDetailTStest.xsl" /> -->
    <xslReportBuildPlugin description="MSBuild Report" actionName="MSBuildBuildReport" xslFileName="xsl\msbuild.xsl" />
    </buildPlugins>

    I have restarted the ccnet app pool item in IIS to when making the dashboard.config changes.

    I am still not seeing test case results. 

  21. Plamen
    Admin
    Plamen avatar
    8 posts

    Posted 21 Dec 2011 Link to this post

    Hi David,

    Your dashboard.config looks correct to me. Could you please provide the ccnet.config file, because most probably the problem is there?

    Also, check the "publishers" section of your project configuration, it should point to the same folder where your "resultsfile" is. Here's an example:
    <exec>
      <!-- Call mstest to run the tests contained in the TestProject -->
      <executable>C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\mstest.exe</executable>
      <baseDirectory>C:\CruiseControl\MSTests\CruiseControlTest\</baseDirectory>       
      <buildArgs>/testmetadata:CruiseControlTest.vsmdi /resultsfile:CruiseControlTest\testResults.trx</buildArgs>      
      <buildTimeoutSeconds>60</buildTimeoutSeconds>
    </exec>

    Publishers:
    <publishers>
      <merge>
        <files>
          <file>C:\CruiseControl\MSTests\CruiseControlTest\CruiseControlTest\testResults.trx</file>
        </files>
      </merge>
      <xmllogger />
    </publishers>

    Note that the path placed in the "publishers" section is the same as the combination of "baseDirectory" plus "resultsfile".

    If this is not the case please provide the ccnet.config file, so we can look further into this.

    Regards,
    Plamen
    the Telerik team

    Check out Telerik Trainer, the state of the art learning tool for Telerik products.
  22. David
    David avatar
    45 posts
    Member since:
    Aug 2010

    Posted 21 Dec 2011 Link to this post

    Plamen,
    I made the appropriate changes to my ccnet.config and the results are showing up.

    Thanks for sticking with me on this issue.
  23. Randrew
    Randrew avatar
    8 posts
    Member since:
    Aug 2012

    Posted 06 Aug 2012 Link to this post

    hi need your help..

    what script should i do to remove this error:
    The results file "TestResults\testResults.trx" already exists. Please specify a different results file or verify the existing file is no longer needed and delete it


    and the list of test are names are not displaying.
  24. Cody
    Admin
    Cody avatar
    3360 posts

    Posted 08 Aug 2012 Link to this post

    Hi David,

    This error message is produced by MSTest (not a Telerik product) when you use a command line like this:

    mstest /testsettings:TestResults\testResults.trx...

    The only way to overcome this error is to delete that file prior to running mstest in your CruiseControl build script. It may be easiest to simply add a "del TestResults\testResults.trx" command right before your MSTest command in the build script.

    Regards,
    Cody
    the Telerik team
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
  25. Randrew
    Randrew avatar
    8 posts
    Member since:
    Aug 2012

    Posted 08 Aug 2012 Link to this post

    Hi Cody,

    I made already a script to delete existing .trx file

    My another problem was test results name don't appear only the number of test.

    Here is my Config:
    <cruisecontrol xmlns:cb="urn:ccnet.config.builder">
      <project name="ApplicationTestingTP" queue="ApplicationTestingTP" queuePriority="1">
        <workingDirectory>C:\TestProject</workingDirectory>
        <artifactDirectory>C:\TestProject\ApplicationTestingTP\bin\Debug</artifactDirectory>
        <webURL>localhost/ccnet</webURL>
        <modificationDelaySeconds>20</modificationDelaySeconds>
        <triggers>
           <intervalTrigger name="continuous" seconds="300" buildCondition="IfModificationExists" />
        </triggers>
        <state type="state" directory="C:\ccnetstate" />
        <labeller type="iterationlabeller">
          <prefix>1.0</prefix>
          <duration>1</duration>
          <releaseStartDate>2012/7/30</releaseStartDate>
          <separator>.</separator>
        </labeller>
     
        <tasks>
          <msbuild>
            <executable>C:\Windows\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe</executable>
            <workingDirectory>c:\TestProject</workingDirectory>
            <projectFile>ApplicationTesting.sln</projectFile>
            <buildArgs>/noconsolelogger /p:Configuration=Debug /v:Minimal</buildArgs>
            <targets>Build</targets>
            <timeout>900</timeout>
          </msbuild>
     
          <exec>
            <executable>c:\TestProject\delTestLog.bat</executable>
            <baseDirectory>c:\TestProject\TestResults\</baseDirectory>
          </exec>
     
          <exec>
            <!-- Call mstest to run the tests contained in the TestProject -->
            <executable>C:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE\mstest.exe</executable>
            <baseDirectory>C:\TestProject\</baseDirectory>
            <buildArgs>/testmetadata:ApplicationTesting.vsmdi /resultsfile:TestResults\testResults.trx</buildArgs>
            <buildTimeoutSeconds>60</buildTimeoutSeconds>
          </exec>
        </tasks>
     
        <publishers>
          <merge>
            <files>
              <file>C:\TestProject\TestResults\testResults.trx</file>
            </files>
          </merge>
          <xmllogger />
        </publishers>
      </project>
     
    </cruisecontrol>
     
    Dashboard.config
     
    <?xml version="1.0" encoding="utf-8"?>
    <dashboard>
      <remoteServices>
        <servers>
          <!-- Update this list to include all the servers you want to connect to. NB - each server name must be unique -->
          <server name="local" url="tcp://localhost:21234/CruiseManager.rem" allowForceBuild="true" allowStartStopBuild="true" backwardsCompatible="false" />
        </servers>
      </remoteServices>
      <plugins>
        <farmPlugins>
          <farmReportFarmPlugin categories="false" refreshInterval="0" successBar="Bottom"/>
          <cctrayDownloadPlugin />
          <administrationPlugin password="" />
        </farmPlugins>
     
        <serverPlugins>
          <serverReportServerPlugin />
        </serverPlugins>
     
        <projectPlugins>
          <projectReportProjectPlugin />
          <viewProjectStatusPlugin />
          <latestBuildReportProjectPlugin />
          <viewAllBuildsProjectPlugin />
        </projectPlugins>
     
        <buildPlugins>
          <buildReportBuildPlugin>
            <xslFileNames>
              <xslFile>xsl\header.xsl</xslFile>
              <xslFile>xsl\modifications.xsl</xslFile>
              <xslFile>xsl\AutomationTestReportDetailTStest.xsl</xslFile>
              <xslFile>xsl\compile-msbuild.xsl</xslFile>
              <xslFile>xsl\MsTestSummary2010.xsl</xslFile>
            </xslFileNames>
          </buildReportBuildPlugin>
          <buildLogBuildPlugin />
          <xslReportBuildPlugin description="MSBuild Report" actionName="MSBuildBuildReport" xslFileName="xsl\msbuild.xsl" />
        </buildPlugins>
     
        <securityPlugins>
          <simpleSecurity />
        </securityPlugins>
      </plugins>
    </dashboard>
  26. Cody
    Admin
    Cody avatar
    3360 posts

    Posted 09 Aug 2012 Link to this post

    Hello,

    I see this section:
    <exec> 
    <executable>c:\TestProject\delTestLog.bat</executable> 
    <baseDirectory>c:\TestProject\TestResults\</baseDirectory> 
     </exec>

    However you didn't include a copy of delTestLog.bat so I don't know what that does... I can't even confirm calling a .bat file from the <exec> section works in CruiseControl (that's beyond our CC expertise).

    If it were me I'd replace the section with this:
    <exec> 
    <executable>del /f /q testResults.trx</executable> 
    <baseDirectory>c:\TestProject\TestResults\</baseDirectory> 
     </exec> 


    Bottom line the .trx file isn't getting deleted as expected prior to running MSTest. You may get better answers contact some CC.NET experts.

    Kind regards,
    Cody
    the Telerik team
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
  27. Cody
    Admin
    Cody avatar
    3360 posts

    Posted 09 Aug 2012 Link to this post

    Hi Randrew,

    My another problem was test results name don't appear only the number of test.

    My apologies, I nearly overlooked this. Can you please clarify this? Can you show me a screen shot of what you are actually seeing compared to what you expected to see?

    Greetings,
    Cody
    the Telerik team
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
  28. Randrew
    Randrew avatar
    8 posts
    Member since:
    Aug 2012

    Posted 10 Aug 2012 Link to this post

    Hi Cody,

    Please see attached files.

    Thanks for your help.
  29. Cody
    Admin
    Cody avatar
    3360 posts

    Posted 15 Aug 2012 Link to this post

    Hello,

    As I recall, CruiseControl uses a .XSLT file for translating the .trx output file into HTML to be displayed by the CruiseControl dashboard. Apparently this .XSLT file needs to be tweaked. If you can locate yours and send it to me, I may be able to figure out what tweaks it needs.

    All the best,
    Cody
    the Telerik team
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
  30. Randrew
    Randrew avatar
    8 posts
    Member since:
    Aug 2012

    Posted 15 Aug 2012 Link to this post

    Hi Cody,

    Attached herewith my xslt files

    thanks
Back to Top
  Cast Your Vote at DevPro Connections! Cast Your Vote at  Win IT Pro Connections!