Ability to pass custom argument through test settings using Art of test runner

[David]

Similar to my other post, I'm working to be able to pass the settings option in the Runner CLI.
It appears to take the full path to a JSON file containing custom settings for the run based on documentation.  Is it possible to add "custom keys" keys in a similar fashion to below?

 public override void OnBeforeTestStarted(BeforeTestStartedArgs args)
        {
            string defaultEnvName = "xxxx";

            string envName = args.Context.Params.ContainsKey("environment") ? args.Context.Params["environment"] : defaultEnvName;

}

[David]

From what I read in old postings when the settings files were xml/config, you could pass in application settings via config files.

Is this still possible through the updated json settings layout? 

[Plamen Mitrev]

Hi David,

You are correct that you can pass a settings JSON file, according to the example for ArtOfTest.Runner.exe. The "settings" parameter will be taken into account by the agent, that executes the command. In this case, you need to make sure that the JSON file is in the correct path on the execution machine and can be accessed by the agent.

The above mentioned approach is the designed way to pass specific settings to a test list, that is part of your continuous integration setup. On the other hand, you can use both OnBeforeTestStarted and OnAfterTestCompleted methods, as well. If you share more details about the test scenario that you want to cover, I will try to suggest the best way for you to do that.

I hope you will find the above information helpful for building the automation setup.

Regards,
Plamen Mitrev
Progress Telerik

 

[David]

Hey Plamen (hope you had a great weekend)

The scenario I'm working to conduct which I believe was capable when settings were xml/app.config based is passing specific app settings (XML example below).  See this post from long ago --> https://www.telerik.com/forums/how-to-combine-my-app-config-with-the-automatically-generated-test-studio-app-config

How can I replicate passing application settings via JSON?  Is it still possible?

Here was the example from XML that was previously used (I'm looking to re-enable this).

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <configSections>
        <section type="ArtOfTest.WebAii.Core.SettingsConfigSectionHandler,ArtOfTest.WebAii" name="WebAii.Settings"/>
        <section type="ArtOfTest.WebAii.Core.WebSettingsConfigSectionHandler,ArtOfTest.WebAii" name="WebAii.Settings.Web"/>
    </configSections>
    <WebAii.Settings.Web defaultBrowser="Chrome"> </WebAii.Settings.Web>
    <appSettings>
        <add key="environment" value="test" />
    </appSettings>
</configuration>

[Plamen Mitrev]

Hi David,

I hope everything is going well for you and you are having pleasant and productive week.

The XML format that you see in that thread was replaced by the JSON format in the documentation. It represents the Test List Settings in Test Studio and you can adjust any of the available options there. Actually, the .aiilist file, that ArtOfTest.Runner.exe executes, comes with the set options, as they were configured for this test list in Test Studio. If everything is set there, you should not need to provide additional settings JSON file.

So, looking at the example, that you shared, I see that you are interested in adjusting the browser and environment for that test list. Those options are as follows.

• To specify the browser, you need to adjust the "DefaultBrowser" value, under "Web" settings. I believe that for Chrome it is "7".
• I assume that with the environment value, you want to specify which application will be tested - test, dev, production, etc. This can be achieve by specifying "BaseUrl" in the settings. You can provide different settings for each test list run of the same test list, to cover all environments. Please keep in mind that if you want to use BaseUrl, your tests should be compatible with this setting and the "Navigate to" steps should not have the full URL (only the part after the Base Url).

Please do not hesitate to contact us again, if you need to follow up on something. I will be happy to advise and assist you further.

Regards,
Plamen Mitrev
Progress Telerik

 

[David]

Hi Plamen,

You're correct on the assumption of what we're trying to do.  Unfortunately using BaseURL doesn't align with our applications for a couple reasons, 1 - we rely on quite a few applications for tests using different URLs and 2-our naming convention doesn't play well with base URL.

Hence why my last few posts have been looking for alternatives, it seems we used to be able to pass appsettings before the migration to json.  Is that no longer a capability?  If no are there any other ways I can pass a custom argument (environment) through test runner?

[Plamen Mitrev]

Hi David,

In this case, when baseURL is not an option for you, I can suggest a different approach to get the environment. Although, I am not sure if that will be the best approach, you can use it as a base to build something that makes more sense in your context.

1. Create a Standalone code file in your Test Studio project.
2. Create static variable with the necessary value that can be accessed by the rest of your scripts. For example the machine name, could be helpful to determine the environment - "System.Environment.MachineName"
3. Whenever you need to change the logic based on the environment (machine name or other value) you can use the previously created variable from the standalone code file to create you condition.

The JSON settings file can only work with the set of options that are listed in the documentation. I hope that the above suggestion will be a good solution for your scenario. If you have any follow up questions or concerns in this regard, please do not hesitate to get back to us.

Thank you for your understanding and cooperation in this discussion.

Regards,
Plamen Mitrev
Progress Telerik

 

[David]

Plamen - It's like you're a fortune teller :) 

That's exactly how our test suite is setup.  Based on what you state, sounds like we won't be able to pass custom app settings through the CLI.

One, potentially poor/hacky option, is to use a field in settings that we don't use (knowing that setting won't impact the test) and pass in the environment that way.  

 

Any other thoughts?

[Elena]

Hi David,

Let me, please, step in this discussion as Plamen is out of the office.

I reviewed the conversation so far and it seems like you have sorted out your doubts - the Test Studio CLI runner can only use its specific options, which will affect the particular run, but not the application under test. 

So, having that said, you can try adjusting the helper class file in a way to be able to pass the necessary variables and parameters to each test. 

The idea you have about modifying the settings file doesn't seem to be an option, because it cannot be accessed from within the test. Also setting any data in it, which is not expected, may cause the CLI runner to not accept it as a valid settings file. 

I hope the information shared so far is beneficial for you. Of course, in case you have any other questions, please, do not hesitate to get back to us. Thank you once again for your cooperation in this discussion. 

Regards,
Elena
Progress Telerik

 

[David]

Elena, thanks for the update!

I have a proof of concept working out for the time being which leverages a settings field that we're not presently using.  So far so good :)

[Elena]

Hello David,

I am pleased to know you managed to build the proof of concept in the desired manner. I will proceed closing this thread as the case seems to be sorted out. Of course, in case you ever need to reopen it, please, do not hesitate to. 

Thank you for your continuous cooperation.

Regards,
Elena
Progress Telerik