Tentacles on 3.3.22 will not start

I just upgraded our Octopus to 3.3.22 and upgraded all of our tentacles, after which things appeared to be operating normally.
There was then a power outage at our hosting site. (yeah, I know, using a hosting site is suppose to avoid that kind of problem.) once the site came back online and our servers restarted, on about a third of them, the tentacles would not start.

this is the error reported:
Error: Error: Unhandled Exception: System.TypeInitializationException: The type initializer for 'Octopus.Tentacle.Properties.OctopusTentacle' threw an exception. ---> System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information. Error: at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module) Error: at System.Reflection.RuntimeAssembly.get_DefinedTypes() Error: at Octopus.Shared.Versioning.SemanticVersionInfo.LoadGitVersionDetails(Assembly assembly) in Y:\Work\refs\tags\3.3.22\source\Octopus.Shared\Versioning\SemanticVersionInfo.cs:line 27 Error: at System.Lazy1.CreateValue()
Error: at System.Lazy1.LazyInitValue() Error: at Octopus.Shared.Versioning.SemanticVersionInfo.get_NuGetVersion() in Y:\Work\refs\tags\3.3.22\source\Octopus.Shared\Versioning\SemanticVersionInfo.cs:line 107 Error: at Octopus.Tentacle.Properties.OctopusTentacle..cctor() in Y:\Work\refs\tags\3.3.22\source\Octopus.Tentacle\Properties\OctopusTentacle.cs:line 13 Error: --- End of inner exception stack trace --- Error: at Octopus.Tentacle.Program..ctor(String[] commandLineArguments) in Y:\Work\refs\tags\3.3.22\source\Octopus.Tentacle\Program.cs:line 20 Error: at Octopus.Tentacle.Program.Main(String[] args) in Y:\Work\refs\tags\3.3.22\source\Octopus.Tentacle\Program.cs:line 31 Error: The previous command returned a non-zero exit code of: -532462766 Error: The command that failed was: "C:\Program Files\Octopus Deploy\Tentacle\Tentacle.exe" service --instance "Tentacle" --start

this happens when trying to start the service, as well as trying to resinstall or uninstall the service.

on these server, we have multiple tentacle instances installed, and the error occurs for all of them. on other servers, also running 3.3.22, the tentacles are running fine, and can be stopped/restarted with no problems.

thanks for you help.

tentacle_error.txt (1 KB)

HI Mike,

Thanks for reaching out. Could you please attempt to start the service once again, and then send us the full Tentacle log so we can see what’s going on? It’ll also be useful to check for event log entries related to this. Send these too if you find any.

Thanks,
Dalmiro

well, I couldn’t leave the tentacles not running, so I use “sc.exe delete OctopusDeploy Tentacle: Foobar” to remove the tentacle from the registry. I then uninstalled the tentacle manager, then installed it again. when it then launched the tentacle manager, it would see the registry entries and rebuild the tentacles, which would then run.

I had to do this for about 20 tentacles.

nothing was recorded to the tentacle log, it never got that far.

I am attaching the content of the 2 entries from the EventLog that were recorded.

tentacle_eventlog.txt (1 KB)

HI Mike,

Thanks for sharing that info. I’m very sad to hear you had to went through that whole crappy outage+manual re-install experience.

Unfortunately those 2 event logs are not that helpful. I think the only way to get to the bottom of this would be to force a similar outage in an isolated machine, and troubleshoot from there. If you are up to that, we’ll be more than glad to help.

Best regards,
Dalmiro