Deploy and Test Windows Apps with VSTS Release

A customer wanted to see UI automation testing from a Team Services Build/Release process. They were specifically interested in seeing how we would test windows client applications. Simple enough. I decided to leverage Azure Dev/Test labs for my environment, built a real simple app, and started working on my release definition. I quickly started getting failed releases and muttering to myself, oh yeah, I forgot to configure this…. And configure that. Simple configuration changes, but I don’t do them often enough so don’t remember the exact steps. In this scenario, I ran into four configuration changes I needed to make to the VM.

Since I know I will run into this again I wanted to dig up all the commands necessary to script these changes so I could fully automate my process in the future and share the info since I am sure others will run into the same thing.

The Scenario

I have a windows WPF app that I need to deploy to a windows client vm, install a test agent, then run some UI automation testing. After the automation testing passes, I have an opportunity for manual review of the application.

The Environment

I am using Azure Dev/Test labs to manage my environments;

I do not have a domain controller in the lab, nor is the lab connected to an on prem network. All of my VMs are in a single Vnet.

I am using TFS build/release to build, deploy, test my application;

I didn’t want to expose my dev/test VMs to the internet, except for RDP, so I couldn’t use the hosted build agent. I deployed the Build/Release agent within my Dev/Test Lab Vnet.

I am using Appium,, and the WindowsApplicationDriver,, to test my application.

Necessary Configuration Changes

Here are the three changes I needed to make;

  • Network file sharing needs to be turned on
  • Configure the Build Agent so it can connect to a WinRM HTTP endpoint
  • Install the Windows Application Driver to drive our UI automation
  • Turn on Windows 10 Developer Mode

Network File Sharing

By default, when you spin up a VM in Azure, File Sharing is off. If you need to copy your application to a VM, you will typically use the copy task, which needs File and Print Sharing to be on. The error you get is;

[Error] System.Management.Automation.RuntimeException: Copying failed for resource….

Failed to connect to the path…

Here is a screenshot

The solution, run this command from an Admin PowerShell prompt to turn on Network File Sharing;

netsh advfirewall firewall set rule group=”File and Printer Sharing” new enable=Yes


When you spin up a VM, the WinRM HTTP endpoint is provisioned by default. This is what you need for PowerShell Remoting to work, which many of the Build/Release tasks take advantage of. Problem is that I am not running my VM as a part of a domain, so any attempt to connect to that endpoint fails with the following message (In this case, when I try to deploy the Test Agent);

The full error message reads;

##[error]Error occured on ‘Win10Test:5985’. Details : ‘Connecting to remote server Win10Test failed with the following error message : The WinRM client cannot process the request. If the authentication scheme is different from Kerberos, or if the client computer is not joined to a domain, then HTTPS transport must be used or the destination machine must be added to the TrustedHosts configuration setting. Use winrm.cmd to configure TrustedHosts. Note that computers in the TrustedHosts list might not be authenticated. You can get more information about that by running the following command: winrm help config. For more information, see the about_Remote_Troubleshooting Help topic.’. For troubleshooting, refer

Since my Build/Release agent is in the same Vnet as my target VM, and I only have RDP exposed externally, I chose to solve this problem by adding my target VM to the TrustedHosts list of my Build Server. The script is pretty simple. Open a PowerShell Admin Command window on your build server and enter;

Set-Item WSMan:\localhost\client\trustedhosts -value ‘<ComputerName>’

Again, you want to run this script on your build agent VM. Once done, you will be able to create a PowerShell remote session over an HTTP connection. Make sure you select the HTTP protocol option when configuring your task;

If you want to clean up the trustedhosts list after the release, these commands will remove the entry;

$newvalue = ((get-childitem WSMan:\localhost\client\trustedhosts).value).replace(“<ComputerName>”,””)

Set-Item WSMan:\localhost\client\trustedhosts $newvalue

Install the Windows Application Driver

So I forgot to install the Windows Application Driver on the computer. The error message you get in your release logs is pretty generic;

##[error]System.Exception: Some tests in the test run did not pass, failing the task.

##[error]PowerShell script completed with 1 errors.

It didn’t take me long to figure out I forgot to install the app driver. This is a pretty straight forward, simple set of PowerShell commands to download the app driver and then another command to install the tool;

$url = “

$dest = “c:\appdownload\WinAppDriver.msi”

$client = new-object


Install the App

.\winappdriver.msi /quiet

Developer Mode

To use Appium, I need to leverage the Windows Application Driver; Turns out that the driver requires you to turn on windows 10 developer mode. If you don’t, you will get that same general error in your release logs;

##[error]System.Exception: Some tests in the test run did not pass, failing the task.

##[error]PowerShell script completed with 1 errors.

If you RDP into the target machine while the test is running, you will see the WinAppDriver run, and give you a much more actionable error message which tells you to enable developer mode.

You can easily fix this via the windows GUI, but again, better to script this via PowerShell. I found a thread covering the topic here; Here is the specific code you will need;

# Create AppModelUnlock if it doesn’t exist, required for enabling Developer Mode
$RegistryKeyPath = “HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\AppModelUnlock”
if (-not(Test-Path -Path $RegistryKeyPath)) {
New-Item -Path $RegistryKeyPath -ItemType Directory -Force

# Add registry value to enable Developer Mode
New-ItemProperty -Path $RegistryKeyPath -Name AllowDevelopmentWithoutDevLicense -PropertyType DWORD -Value 1


At this point, all four configuration items can be programmatically done via PowerShell scripts. This is great since I can now add those configuration scripts to my release process, as a Dev/Test lab artifact, or even as part of a PowerShell Desired State Configuration Script. Eventually I could even automate the provisioning and deprovisioning on the VM/Environment so I don’t ever have to wonder if something changed in the environment which affected my tests.

Tagged with:
Posted in Automated Testing, Release Management, Team Services

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: