While still on my trial license, I’ve managed to recreate our CI pipeline which consists of 12 environments, 6 machines each. Each test run, the pipeline selects an “available” environment to run tests on. I’ve noticed that the Octopus server service crashes while communication with a tentacle stops working. I’ve found this in the logs:
2014-03-11 15:16:47.6010 ERROR Error posting to: https://172.24.69.100:10933/mx/v1
System.Net.Sockets.SocketException (0x80004005): A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 172.24.69.100:10933
at System.Net.Sockets.TcpClient.Connect(String hostname, Int32 port)
at Pipefish.Transport.SecureTcp.Client.SecureTcpClient.Send(SecureTcpRequest request) in c:\TeamCity\buildAgent\work\cf0b1f41263b24b9\source\Pipefish.Transport.SecureTcp\Client\SecureTcpClient.cs:line 39
at Pipefish.Transport.SecureTcp.MessageExchange.Client.ClientWorker.PerformExchange() in c:\TeamCity\buildAgent\work\cf0b1f41263b24b9\source\Pipefish.Transport.SecureTcp\MessageExchange\Client\ClientWorker.cs:line 298
at Pipefish.Transport.SecureTcp.MessageExchange.Client.ClientWorker.Run() in c:\TeamCity\buildAgent\work\cf0b1f41263b24b9\source\Pipefish.Transport.SecureTcp\MessageExchange\Client\ClientWorker.cs:line 173
2014-03-11 15:17:23.7338 ERROR Error posting to: https://172.24.69.100:10933/mx/v1
System.Net.Sockets.SocketException (0x80004005): A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 172.24.69.100:10933
at System.Net.Sockets.TcpClient.Connect(String hostname, Int32 port)
at Pipefish.Transport.SecureTcp.Client.SecureTcpClient.Send(SecureTcpRequest request) in c:\TeamCity\buildAgent\work\cf0b1f41263b24b9\source\Pipefish.Transport.SecureTcp\Client\SecureTcpClient.cs:line 39
at Pipefish.Transport.SecureTcp.MessageExchange.Client.ClientWorker.PerformExchange() in c:\TeamCity\buildAgent\work\cf0b1f41263b24b9\source\Pipefish.Transport.SecureTcp\MessageExchange\Client\ClientWorker.cs:line 298
at Pipefish.Transport.SecureTcp.MessageExchange.Client.ClientWorker.Run() in c:\TeamCity\buildAgent\work\cf0b1f41263b24b9\source\Pipefish.Transport.SecureTcp\MessageExchange\Client\ClientWorker.cs:line 173
2014-03-11 15:17:59.7881 ERROR Error posting to: https://172.24.69.100:10933/mx/v1
System.Net.Sockets.SocketException (0x80004005): A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 172.24.69.100:10933
at System.Net.Sockets.TcpClient.Connect(String hostname, Int32 port)
at Pipefish.Transport.SecureTcp.Client.SecureTcpClient.Send(SecureTcpRequest request) in c:\TeamCity\buildAgent\work\cf0b1f41263b24b9\source\Pipefish.Transport.SecureTcp\Client\SecureTcpClient.cs:line 39
at Pipefish.Transport.SecureTcp.MessageExchange.Client.ClientWorker.PerformExchange() in c:\TeamCity\buildAgent\work\cf0b1f41263b24b9\source\Pipefish.Transport.SecureTcp\MessageExchange\Client\ClientWorker.cs:line 298
at Pipefish.Transport.SecureTcp.MessageExchange.Client.ClientWorker.Run() in c:\TeamCity\buildAgent\work\cf0b1f41263b24b9\source\Pipefish.Transport.SecureTcp\MessageExchange\Client\ClientWorker.cs:line 173
2014-03-11 15:18:15.8725 ERROR Error posting to: https://172.24.69.100:10933/mx/v1
System.Net.Sockets.SocketException (0x80004005): No connection could be made because the target machine actively refused it 172.24.69.100:10933
at System.Net.Sockets.TcpClient.Connect(String hostname, Int32 port)
at Pipefish.Transport.SecureTcp.Client.SecureTcpClient.Send(SecureTcpRequest request) in c:\TeamCity\buildAgent\work\cf0b1f41263b24b9\source\Pipefish.Transport.SecureTcp\Client\SecureTcpClient.cs:line 39
at Pipefish.Transport.SecureTcp.MessageExchange.Client.ClientWorker.PerformExchange() in c:\TeamCity\buildAgent\work\cf0b1f41263b24b9\source\Pipefish.Transport.SecureTcp\MessageExchange\Client\ClientWorker.cs:line 298
at Pipefish.Transport.SecureTcp.MessageExchange.Client.ClientWorker.Run() in c:\TeamCity\buildAgent\work\cf0b1f41263b24b9\source\Pipefish.Transport.SecureTcp\MessageExchange\Client\ClientWorker.cs:line 173
Seems like this is fairly normal behaviour that shouldn’t crash the service. I then found this:
2014-03-11 09:10:09.1815 ERROR Unhandled error on request: http://172.24.0.46/api/dashboard : The specified cryptographic algorithm is not supported on this platform.
System.PlatformNotSupportedException: The specified cryptographic algorithm is not supported on this platform.
at System.Security.Cryptography.AesCryptoServiceProvider…ctor()
at lambda_method(Closure )
at Raven.Bundles.Encryption.Codec.GetCryptoProvider(Byte[] iv)
at Raven.Bundles.Encryption.Streams.BlockReaderWriter.ReadBlock(Int64 blockNumber)
at Raven.Bundles.Encryption.Streams.SeekableCryptoStream.Read(Byte[] buffer, Int32 bufferOffset, Int32 count)
at Raven.Abstractions.Extensions.StreamExtensions.ReadEntireBlock(Stream stream, Byte[] buffer, Int32 start, Int32 count)
at Lucene.Net.Store.BufferedIndexInput.ReadBytes(Byte[] b, Int32 offset, Int32 len, Boolean useBuffer)
at Lucene.Net.Index.CompoundFileReader.CSIndexInput.ReadInternal(Byte[] b, Int32 offset, Int32 len)
at Lucene.Net.Store.BufferedIndexInput.Refill()
at Lucene.Net.Store.BufferedIndexInput.ReadByte()
at Lucene.Net.Store.IndexInput.ReadInt()
at Lucene.Net.Store.IndexInput.ReadLong()
at Lucene.Net.Index.FieldsReader.Doc(Int32 n, FieldSelector fieldSelector)
at Raven.Database.Indexing.Index.IndexQueryOperation.d__52.MoveNext()
at Raven.Database.Util.ActiveEnumerable1…ctor(IEnumerable1 enumerable)
at Raven.Database.DocumentDatabase.<>c__DisplayClass91.b__88(IStorageActionsAccessor actions)
at Raven.Storage.Esent.TransactionalStorage.ExecuteBatch(Action1 action, EsentTransactionContext transactionContext)
at Raven.Storage.Esent.TransactionalStorage.Batch(Action1 action)
at Raven.Database.DocumentDatabase.Query(String index, IndexQuery query, Action1 headerInfo, Action1 onResult)
at Raven.Client.Embedded.EmbeddedDatabaseCommands.<>c__DisplayClass24.b__1e()
at System.Threading.Tasks.Task.Execute()
Looking at the event viewer I see this service completely crash:
Application: Octopus.Server.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an internal error in the .NET Runtime at IP 000007FC9BD26D2B (000007FC9BBD0000) with exit code 80131506.
Faulting application name: Octopus.Server.exe, version: 2.1.3.1223, time stamp: 0x5302e1ae
Faulting module name: clr.dll, version: 4.0.30319.18449, time stamp: 0x528fdca2
Exception code: 0xc0000005
Fault offset: 0x0000000000156d2b
Faulting process id: 0x63c
Faulting application start time: 0x01cf3d2b6f752f71
Faulting application path: C:\Program Files\Octopus Deploy\Octopus\Octopus.Server.exe
Faulting module path: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll
Report Id: cca22b15-a920-11e3-941e-005056a43871
Faulting package full name:
Faulting package-relative application ID:
It would be great if someone could help me out as the Octopus server crashing mid test run is obviously not good.