Accounts w/ Management Certs Stopped Working

Legacy Cloud
1

Hello,

We are on version 2018.7.7 of Octopus Deploy, and we recently upgraded to 2018.7.10. After the upgrade, any and all deployments using an account with management certificate authentication would begin to fail, per the error below.

Hyak.Common.CloudException: ForbiddenError: The server failed to authenticate the request. Verify that the certificate is valid and is associated with this subscription. 
   at Microsoft.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 
   at Microsoft.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccess(Task task) 
   at Microsoft.WindowsAzure.Management.Compute.DeploymentOperationsExtensions.GetBySlot(IDeploymentOperations operations, String serviceName, DeploymentSlot deploymentSlot) 
   at Calamari.Azure.Integration.AzureCloudServiceConfigurationRetriever.GetConfiguration(SubscriptionCloudCredentials credentials, String serviceName, DeploymentSlot slot) 
   at Calamari.Azure.Deployment.Conventions.ConfigureAzureCloudServiceConvention.UpdateConfigurationWithCurrentInstanceCount(XContainer localConfigurationFile, String configurationFileName, VariableDictionary variables) 
   at Calamari.Azure.Deployment.Conventions.ConfigureAzureCloudServiceConvention.Install(RunningDeployment deployment) 
   at Calamari.Deployment.ConventionProcessor.RunInstallConventions() 
   at Calamari.Deployment.ConventionProcessor.RunConventions() 
Running rollback conventions... 
ForbiddenError: The server failed to authenticate the request. Verify that the certificate is valid and is associated with this subscription. 
Hyak.Common.CloudException 
   at Microsoft.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 
   at Microsoft.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccess(Task task) 
   at Microsoft.WindowsAzure.Management.Compute.DeploymentOperationsExtensions.GetBySlot(IDeploymentOperations operations, String serviceName, DeploymentSlot deploymentSlot) 
   at Calamari.Azure.Integration.AzureCloudServiceConfigurationRetriever.GetConfiguration(SubscriptionCloudCredentials credentials, String serviceName, DeploymentSlot slot) 
   at Calamari.Azure.Deployment.Conventions.ConfigureAzureCloudServiceConvention.UpdateConfigurationWithCurrentInstanceCount(XContainer localConfigurationFile, String configurationFileName, VariableDictionary variables) 
   at Calamari.Azure.Deployment.Conventions.ConfigureAzureCloudServiceConvention.Install(RunningDeployment deployment) 
   at Calamari.Deployment.ConventionProcessor.RunInstallConventions() 
   at Calamari.Deployment.ConventionProcessor.RunConventions() 
   at Calamari.Azure.Commands.DeployAzureCloudServiceCommand.Execute(String[] commandLineArguments) 
   at Calamari.Program.Execute(String[] args) 
The remote script failed with exit code 100 
The action Deploy on the Octopus Server failed

We had to rollback to 2018.7.7 to continue using Octopus. This issue occurs on version 2018.7.8, 7.9, and 7.10. The account this is failing on is scoped to one tenant and one environment, and is only available for tenanted deployments. The account is against the AzureUSGovernment azure environment, and the management certificate is still present and valid for those Azure subscriptions affected.

2

Hi Skyler,

Definitely something is broken!
I have raised a bug for this one and you can follow it’s progress on https://github.com/OctopusDeploy/Issues/issues/4813

Thanks
John

3

Hi Skyler,

We have just released version 2018.7.11, can you please update to this version and see if this issue is fixed.
We did a few changes in this version and believe it should address this problem.

Let us know!

Cheers
John

4

After installing 2018.7.11, the issue still persists with the same message.

5

Hi Skyler,

Ok, we need a bit more info.
Can you send me your project deployment process as a json, to do this go to Project|Process and click the three vertical dots beside the “Add Step” button, and there should be a menu “Download as Json”.

Cheers
John

6

elweb-tenant-elweb-process.json (18.3 KB)

7

Hi Skyler,

Are you able to send us the deployment logs ?

Cheers
John

8

ServerTasks-55248.log.txt (18.7 KB)

9

Hi Skyler,

Just a quick one to let you know that we have found the bug in our code and we should have a fix out on Monday.

Cheers
John

10

Hi Skyler,

We have just released version 2018.7.12 that contains the fix for this issue.

Let us know of any other problems.

Cheers
John

11

Looks like that fixed the issue, thanks! This issue started in version 2018.7.8, and was resolved in version 2018.7.12.