Octopus upgrade bugs

Legacy Uncategorised
1

Hi there

We upgraded Octopus from 3.1.2 to 3.2.10, and we found a few oddities when we went to upgrade the tentacles:

  1. we saw a lot of exceptions in the log for the upgrade task:
System.ObjectDisposedException: Instances cannot be resolved and nested lifetimes cannot be created from this LifetimeScope as it has already been disposed.
   at Autofac.Core.Lifetime.LifetimeScope.ResolveComponent(IComponentRegistration registration, IEnumerable`1 parameters)
   at Autofac.ResolutionExtensions.TryResolveService(IComponentContext context, Service service, IEnumerable`1 parameters, Object& instance)
   at Autofac.ResolutionExtensions.ResolveNamed[TService](IComponentContext context, String serviceName, IEnumerable`1 parameters)
   at Octopus.Shared.Communications.AutofacServiceFactory.CreateService(String serviceName) in Y:\work\refs\tags\3.1.2\source\Octopus.Shared\Communications\AutofacServiceFactory.cs:line 18
   at Halibut.ServiceModel.ServiceInvoker.Invoke(RequestMessage requestMessage) in y:\work\7ab39c94136bc5c6\source\Halibut\ServiceModel\ServiceInvoker.cs:line 21
   at Halibut.Transport.Protocol.MessageExchangeProtocol.InvokeAndWrapAnyExceptions(RequestMessage request, Func`2 incomingRequestProcessor) in y:\work\7ab39c94136bc5c6\source\Halibut\Transport\Protocol\MessageExchangeProtocol.cs:line 139

  1. the agent upgrade as failing, as the service account it was missing read permissions to HLKM\Software\Octopus\Tentacle, however, the upgrade task was reporting successful, even though the agent remained on the old version.

  2. the permissions listed on Installing Tentacles dont quite match the ones listed on Running Tentacle under a specific user account for use in Powershell - it might be worth updating them to match

We have succesfully upgraded all agents now, so we are not blocked, but just reporting these so that the experience can be made better in the future.

cheers,
Matt

2

Hi Matt,

Thanks for getting in touch and sending through your upgrade notes. This is very helpful! I have a few comments and questions below.

  1. We have seen this exception in the past however it doesn’t impact future deployments so it’s safe to ignore. Just to double-check, did this impact any future deployments?

  2. I’d like to get a bit more detail on this issue. Can you confirm if the following is correct?

  • Tentacle upgrade started via Octopus Portal Environments page
  • Upgrade task reported as successful but target tentacle was not upgraded
  • Target tentacle was running under a custom user account

How did you determine that it was missing read permission to missing read permissions to HLKM\Software\Octopus\Tentacle?

  1. I’ve added a task to review/update our tentacle permissions documentation appropriately.

Thanks again

Rob

3

Hi Rob

Apologies for the delay – I’m on hols (in Aus!) at the moment, so email has taken a back seat.

It did not affect any deployments – they all worked successfully.

Yes, the upgrade was started via the octopus portal environments page, yes.
Yes, the task reported successful, but the last status notice said the tentacle was running 3.1.2, not 3.2.10.
Yes, the tentacle was running under a custom domain account.

We found an error in the application event log that said the setup was unable to read the registry key, and another one that reporting the installation failing with an error code (can’t remember what it was though).

Hope this information helps.

Cheers,
Matt

4

Hi Matt,

Happy Holidays to you also! We are running a skeleton crew at the moment, so if this isn’t a super-pressing issue I’ll let Rob Pearson pick it up with you when you both return from holidays.

Thanks!
Mike

5

Hi Matt,

Thanks for following up and reporting this. I reproduced the failed tentacle upgrades issue and I’ve created a github issue to have it fixed. You can follow it’s progress at the following URL.
https://github.com/OctopusDeploy/Issues/issues/2266

Thanks again!

Rob