I have got Octopus deploy run 2 steps. It first deploys my code and its tests onto a remote server. It then runs a powershell script to run an interactive test. The script works fine when run on my local system but throws this error when automatically run on Octopus. How do I resolve this?
Thanks for reaching out. Since Octopus doesn’t run as an interactive process, there are times where it cant execute certain kind of tasks related to Interactive UI testing. We’ve seen this behavior with users that tried to run scripts against interactive IE sessions with no luck.
These parts of the error message lead me to believe this might not work with Octopus
Tu run tests that interact with the desktop, you must setup the test agent to run as interactive process
If you are running the tests as part of your team build, you must also set up the build agent to run a an interactive process
Apparently not only the test agent has to be running in interactive mode(or as a process), but also the system that calls the test agent (in this case, Octopus). You could change the Tentacle service to run in interactive mode, but that would force you to be logged in to the session for it to work.
I’m gonna ask around our devs for a 2nd opinion on this and will get back to you. In the meantime: have you set your test agent to run as a process instead of as a service as recommended on the error message already?
Thanks and sorry for the not so great news.
Yes, the test agent runs as a process, not a service.
Also, it looks like I have made some progress.
I tried using the MSTest.exe instead of vstest.console.exe in my script and it attempts to run the tests but fails, although it runs successfully on my system. (These are interactive tests that start a browser)
But again, this probably is the same issue you mentioned.
I discussed this with one of our devs and he mentioned you should allow the Tentacle service to interact with desktop.
Search on that article for the keyword desktop. Please make sure to not just check the box, but also to make the change on the registry mentioned on the article. Also please keep in mind that this change should be done on the Tentacle service/Tentacle machine, and not on the Octopus Server service/Machine. I’m mentioning this because it wasn’t 100% clear on my previous comment.
Let me know how it goes
I’m trying to allow the service to interact with the desktop as per your linked instructions.
It fails at the first hurdle however, as when I tick the box labeled ‘Allow service to interact with the dekstop’ in the Tentacle service, the Tentacle throws an exception and fails to restart.
If I look in event viewer, the full error is:
2015-07-16 15:31:58.8883 1 FATAL System.IO.IOException: The handle is invalid. at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath) at System.Console.set_Title(String value) at Octopus.Shared.Startup.ConsoleHost.Run(Action`1 start, Action shutdown) in y:\work\refs\heads\master\source\Octopus.Shared\Startup\ConsoleHost.cs:line 26
Thank you for looking into this issue with such interest.
I could not get Octopus to start up those interactive tests, but I did find a workaround due to time constraints. I now get Octopus to create a batch script and run the tests externally. If at a later point Octopus can do that directly, I would definitely switch to that.
Thanks a lot for coming back with your workaround. Hopefully it’ll help other running into this issue.
Could you elaborate on how you got it working eventually as I have the exact same problem.
To run the tests automatically, I created a two-step process.
The first step deploys the project (NuGet package) and the second runs a
batch script with MSTest.
This script creates a .bat file with the dll and MSTest.exe. It also
creates a scheduled task to trigger the test run at that point.
Hope this helps!
I got it working by running the tentacle from the console instead of as a service. It is the same approach Microsoft uses for running CodedUI tests from the build. You just stop (or even disable) the Octopus Tentacle service and start the console from %ProgramFiles%\Octopus\Tentacle\Tentacle.exe.