on one server (and only one of hundreds) we have a tentacle that has several times gotten hung during a deployment. it doesn’t always happen, but has several times now.
when it hangs, it has always been when running a powershell executing database commands. It will have completed some of the them, and then just stops at one point, the same point. It could be that the SQL server is reporting back an error or a message, and the tentacle is not handling it properly.
Once the hang is noticed, I RDP onto the server and stop the tentacle, whereupon the deploy log reports an error:
Instances cannot be resolved and nested lifetimes cannot be created from this LifetimeScope as it has already been disposed.
Server exception:
System.ObjectDisposedException: Instances cannot be resolved and nested lifetimes cannot be created from this LifetimeScope as it has already been disposed.
at Autofac.Core.Lifetime.LifetimeScope.ResolveComponent(IComponentRegistration registration, IEnumerable`1 parameters)
at Autofac.ResolutionExtensions.TryResolveService(IComponentContext context, Service service, IEnumerable`1 parameters, Object& instance)
at Autofac.ResolutionExtensions.ResolveNamed[TService](IComponentContext context, String serviceName, IEnumerable`1 parameters)
at Octopus.Shared.Communications.AutofacServiceFactory.CreateService(String serviceName) in y:\work\refs\heads\master\source\Octopus.Shared\Communications\AutofacServiceFactory.cs:line 19
at Halibut.ServiceModel.ServiceInvoker.Invoke(RequestMessage requestMessage) in y:\work\7ab39c94136bc5c6\source\Halibut\ServiceModel\ServiceInvoker.cs:line 21
at Halibut.Transport.Protocol.MessageExchangeProtocol.InvokeAndWrapAnyExceptions(RequestMessage request, Func`2 incomingRequestProcessor) in y:\work\7ab39c94136bc5c6\source\Halibut\Transport\Protocol\MessageExchangeProtocol.cs:line 139
and in the Logs\OctopusTentacle.txt file, it records this error:
2015-08-19 08:18:45.1380 35 INFO listen://0.0.0.0:10933/ 35 Unhandled error when handling request from client: 216.200.56.133:57523
System.IO.IOException: Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
--- End of inner exception stack trace ---
when I then try to start the tentacle again, it starts, and then stops about a second later. the log records this error:
2015-08-19 08:24:04.3302 6 ERROR A required communications port is already in use. http://g.octopushq.com/HttpPortInUse See: http://g.octopushq.com/HttpPortInUse
System.Net.Sockets.SocketException (0x80004005): Only one usage of each socket address (protocol/network address/port) is normally permitted
at System.Net.Sockets.Socket.DoBind(EndPoint endPointSnapshot, SocketAddress socketAddress)
it seem that when crashing, the Tentacle leaves a process running attached to port 10933, and so the Tentacle fails to restart.
a call to “NetStat -a -n -o” reveals two processes using 10933, one “Listening” and one “Established”. however, when I attempt to use TaskKill on the Pid shown, it says the process is not found. I am forced to reboot the server to get the tentacle running again.
any help would be appreciated. thank you.
