Managing Environmental Settings

8 posts, 0 answers
  1. ColinBowern
    ColinBowern avatar
    51 posts
    Member since:
    Aug 2012

    Posted 06 Feb 2013 Link to this post

    I'm looking at the workflow between developer, tester and CI server to come up with an approach for maintaining environmental specific settings (local dev box, dev/test environment, qa environment, production) such as:

    • Base URL
    • Username / Password by Actor / Role
    • Key Data Values 

    I'm wondering how others are tackling this issue?  I see a number of disconnected techniques such as the VS test settings file, data sources, and test list settings.  There seems to be a lack of a universal "project file" across the tool sets which doesn't help either.  What I would like to be able to do is:

    • Developer - selects a "settings/parameters file" in Test Studio Express with environment parameters
    • Tester - selects a "settings/parameters file" in Test Studio Standalone with environment parameters
    • Build - execute runner with a specific "settings/parameters file" that contains environment parameters

    Within the test then I can say "Navigate to {BaseUrl}/Reports", "Login as {Regular User}", etc... Without having to have three different types of files per environment what is the best strategy for managing this type of information?

    UPDATE: I've added an item to the feedback portal to capture this gap in functionality.
  2. Cody
    Admin
    Cody avatar
    3360 posts

    Posted 11 Feb 2013 Link to this post

    Hello Colin,

    My recommendation for your case is to take the Data Driven approach. You can data drive the NavigateTo steps, all the user name and password input, plus your key value pairs (the key being the column name and the value being the data in the row). Put all your environment settings into a spreadsheet, but use different spreadsheets in your different environments. All 3 environments would look for the same named spreadsheet, but give it different content in each environment. Does this approach sound workable to you?

    All the best,
    Cody
    the Telerik team
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
  3. ColinBowern
    ColinBowern avatar
    51 posts
    Member since:
    Aug 2012

    Posted 11 Feb 2013 Link to this post

    Correct me if I'm wrong - the consequence I see with that approach is that I need to connect each test to the data source. With only one data source per test I'm left with a giant spreadsheet containing every possible combination of values if I ever want to use data driven testing properly.  Also this means I have to remember to manage that association across my entire test suite. I understand that it can be done but it needs to scale across a real world test suite not just a hypothetical demo example.  Maybe I'm missing something in how to structure our test suite? (We're new to Test Studio :))
  4. Cody
    Admin
    Cody avatar
    3360 posts

    Posted 15 Feb 2013 Link to this post

    Hello Colin,

    I apologize for the delay getting back to you on this.

    I admit it's difficult to say how hard it would be to implement my suggestion not knowing any real details about your environment and how many variables/settings you need. I can tell you that multiple tests can be bound to the same data source at the same time e.g. you have an excel file "Environment.xlsx" and tests A, B, C can all be bound to this same file all concurrently. Granted you have to put all environment variables into one spread sheet. Usually this is just one row with around 10-20 columns/values.

    Another option would be to use code and read the Windows environment variables and make the test use those. Of course this assumes you set those environment variables in Windows prior to running the test.

    Regards,
    Cody
    the Telerik team
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
  5. ColinBowern
    ColinBowern avatar
    51 posts
    Member since:
    Aug 2012

    Posted 20 May 2013 Link to this post

    Cody - here's what I'm thinking...

    ArtOfTestRunner.exe list=MyTestSet.aiilist variables=LocalDevEnvironment.xml
    ArtOfTestRunner.exe list=MyTestSet.aiilist variables=TestEnvironment.xml

    The variables document is a key-value dictionary made available to all tests.  Maybe there is a global variables properties piece that allows you to define default values to make it easier to run in the IDE?  Or maybe the variable set can be chosen within the IDE (similar to how your platform configuration is configurable in Visual Studio - Debug, Release, etc...).

  6. Cody
    Admin
    Cody avatar
    3360 posts

    Posted 20 May 2013 Link to this post

    Hello Colin,

    We do already support a "settings=" parameter from the command line. It was meant to load the settings that our runner recognizes and would configure the test to use. I am imagining that you could add your own values to the same file, using code to access the data and get it that way.

    Following your approach "variables=LocalDevEnvironment.xml" how is this significantly different than "With only one data source per test I'm left with a giant spreadsheet containing every possible combination of values"?

    Regards,
    Cody
    the Telerik team
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
  7. ColinBowern
    ColinBowern avatar
    51 posts
    Member since:
    Aug 2012

    Posted 20 May 2013 Link to this post

    I'll try adding a few values to that file.  How would that translate to the tester working in the IDE though?

    Using the data source would prevent me from using data-driven testing features which are useful in some of the test scenarios. To me environment parameters which are applied globally to all tests and data sources for data driven testing are two different scenarios.  Yes you could use the data-driven testing features to manage environmental parameters, but it would also increase the maintenance burden when testers use the DDT features.

  8. Cody
    Admin
    Cody avatar
    3360 posts

    Posted 20 May 2013 Link to this post

    Hello Colin,

    How would that translate to the tester working in the IDE though?

    I admit it wouldn't. It's a possible method to use given the features included in the product today.

    To me environment parameters which are applied globally to all tests and data sources for data driven testing are two different scenarios.

    Very good point. I've created a new feature request around this concept. Let me know if I missed anything in the description.

    Kind regards,
    Cody
    the Telerik team
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
Back to Top