Unable to create Azure Service Fabric Target

TommyQuality · August 24, 2018, 3:17pm

Hi,

i am using the cloud based Octopus Deploy, and trying to create a Service Fabric Deployment Target, but it’s failing - it looks like maybe a bug as the on Premises version i have running connects without any problems.

I’ve been using the New-OctopusAzureServiceFabricTarget cmdlet but it throws an error when trying to connect to the target.

11:56:43 Error | An error occurred during this operation. Please check the trace logs for more details.
11:56:43 Error | System.Fabric.FabricException
11:56:43 Error | at System.Fabric.Interop.Utility.WrapNativeSyncInvoke[TResult](Func1 func, String functionTag, String functionArgs) 11:56:43 Error | at System.Fabric.Interop.Utility.RunInMTA(Action action) 11:56:43 Error | at System.Fabric.FabricClient.InitializeFabricClient(SecurityCredentials credentialArg, FabricClientSettings newSettings, String[] hostEndpointsArg) 11:56:43 Error | at Calamari.Azure.HealthChecks.ServiceFabricHealthChecker.ExecuteHealthCheck(CalamariVariableDictionary variables) 11:56:43 Error | at Calamari.Commands.HealthCheckCommand.Execute(String[] commandLineArguments) 11:56:43 Error | at Calamari.Program.Execute(String[] args) 11:56:43 Error | --Inner Exception-- 11:56:43 Error | The credentials supplied to the package were not recognized (Exception from HRESULT: 0x8009030D) 11:56:43 Error | System.Runtime.InteropServices.COMException 11:56:43 Error | at System.Fabric.Interop.NativeClient.IFabricClientSettings2.SetSecurityCredentials(IntPtr credentials) 11:56:43 Error | at System.Fabric.FabricClient.SetSecurityCredentialsInternal(SecurityCredentials credentials) 11:56:43 Error | at System.Fabric.Interop.Utility.<>c__DisplayClass27_0.<WrapNativeSyncInvoke>b__0() 11:56:43 Error | at System.Fabric.Interop.Utility.WrapNativeSyncInvoke[TResult](Func1 func, String functionTag, String functionArgs)

This is having a major impact on deploying to our service fabric instance.

Please help

Thanks

Tommy

MarkSiedle · August 27, 2018, 6:19am

Hi Tommy,

Thanks for getting in touch and sorry to hear you’re having troubles.

We’ve managed to reproduce an issue around SF targets with certificates on Octopus Cloud instances (not your exact error message, but at least a derivative of it).

We believe this issue is happening because the built-in worker that runs in Octopus Cloud instances has restricted permissions. And these lower permissions are stopping Octopus from interacting with the certificate store on the server.

You have a couple of potential workarounds while we investigate the certificate permission issues. We tested both on a Cloud instance to make sure they’re viable.

Option 1:
Switch to use the Azure AD security mode, as this does not rely on any certificates installed on the Octopus Server.

However, this may be daunting to setup if you haven’t explored that world of Azure yet (I pulled my hair out trying to figure out Azure AD, but you may have more luck if you know that world)

Option 2:
Set up an external worker for Octopus to use. This involves setting up a Tentacle on a server you own/control, then your Cloud instance can reach out to this external worker to do the actual work. Your certificate will be installed into this worker’s certificate store, completely separate from your Octopus Cloud instance.

In our case, we setup a Polling Tentacle to our Octopus Cloud instance (confirmed it registered successfully and passed a health check), then configured it as a worker by running the following command on the worker server:

.\Tentacle.exe register-worker --server "https://MY_CLOUD_INSTANCE.octopus.app/" --comms-style TentacleActive --apikey "MY_API_KEY" --workerpool "Default Worker Pool"

Use TentacleActive for Polling Tentacle, TentaclePassive for Listening Tentacles … have a good read of that external worker documentation if you go down this path.

You will also have to install the Service Fabric SDK etc. on the worker (as per the instructions in the callout in the SF documentation), since the worker is now the thing executing the deployment to Azure.

Sorry for the bad news, but hopefully this gives you some options while we look into a fix.

We have created a GitHub issue here that you can track to be notified when a fix is available.

Cheers
Mark

MarkSiedle · August 28, 2018, 3:41am

Hi again Tommy,

One thing we realised today, the issue with cloud-hosted instances is that the built-in worker does not have permissions to write to the LocalMachine certificate store (which is what it uses by default), so what you could try is actually overriding the store location on your Service Fabric target and instead set it to use the CurrentUser store.

E.g. Like this:

Before trying those other workarounds I mentioned yesterday, could you please give that a try and let us know if that solves it?

Cheers
Mark

mambo_chami · February 15, 2021, 12:37pm

Good day @MarkSiedle! I’m having the same problem on my D365FO (on-prem) environment. On my ORCH machine I’m getting the following error

XXX-OR-01
Message Unable to dispatch command id 03ba2823-cc78-4203-8759-b9099a120c15. Retries 10
Detail System.Fabric.FabricException: An error occurred during this operation. Please check the trace logs for more details. —> System.Runtime.InteropServices.COMException: The credentials supplied to the package were not recognized (Exception from HRESULT: 0x8009030D) at System.Fabric.Interop.NativeClient.IFabricServiceManagementClient6.EndResolveServicePartition(IFabricAsyncOperationContext context) at System.Fabric.FabricClient.ServiceManagementClient.ResolveServicePartitionEndWrapper(IFabricAsyncOperationContext context) at System.Fabric.Interop.AsyncCallOutAdapter21.Finish(IFabricAsyncOperationContext context, Boolean expectedCompletedSynchronously) --- End of inner exception stack trace --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Microsoft.ServiceFabric.Services.Client.ServicePartitionResolver.<ResolveHelperAsync>d__29.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Communication.Client.CommunicationClientFactoryBase1.d__24.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.V1.FabricTransport.Client.FabricTransportServiceRemotingClientFactory.<Microsoft-ServiceFabric-Services-Communication-Client-ICommunicationClientFactory-GetClientAsync>d__9.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Communication.Client.ServicePartitionClient1.<GetCommunicationClientAsync>d__28.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Microsoft.ServiceFabric.Services.Communication.Client.ServicePartitionClient1.d__231.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.V1.Client.ServiceRemotingPartitionClient.<InvokeAsync>d__2.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.Builder.ProxyBase.<InvokeAsync>d__15.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.Builder.ProxyBase.<ContinueWithResult>d__161.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at BridgeService.BridgeService.d__19.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at BridgeService.BridgeService.d__18.MoveNext()

I’m not sure how to get out of this? But seems the problem looks like the same.

Any help please?

MarkSiedle · February 15, 2021, 10:57pm

Hi, thanks for getting in touch.

If you’re not using Octopus Cloud and the above details don’t help, could you please submit a new help ticket and include all of the above + details of how you connect to Service Fabric (which auth mode) and any other details about your particular setup (any firewalls, proxies, which version of Service Fabric SDK you have installed etc.) and someone will be able to reach out and investigate.

Thanks
Mark

mambo_chami · February 16, 2021, 7:21am

Ok thanks. @MarkSiedle
Here are the details.

Application : D365FO (on-prem) PU 40

Service Fabric Version : 7.2.445.9590

Local Agent Version : 2.6.0

Windows Firewall : Disabled

Proxy : No Proxy

What I have done :

I have granted FULL CONTROL permission to Network Service, Administrators and svc-LocalAgent$ to the On-PremLocalAgent certificate in all ORCHS.
I have granted FULL CONTROL permission to Network Service, ServiceFabricAdministrators and Administrators to the Star certificate in all ORCHS.
I have granted FULL CONTROL permission to Network Service, Administrators and svc-LocalAgent$ to the Client certificate in all ORCHS.

I have placed all these certs in the Trusted Root Certificate Authorities.

Error Message :

XXX-OR-01 
  Message Unable to dispatch command id bc01b6b4-4cae-4e0b-956a-771bada48d8b. Retries 6 
  Detail System.Fabric.FabricException: An error occurred during this operation. Please check the trace logs for more details. ---> System.Runtime.InteropServices.COMException: The credentials supplied to the package were not recognized (Exception from HRESULT: 0x8009030D) at System.Fabric.Interop.NativeClient.IFabricServiceManagementClient6.EndResolveServicePartition(IFabricAsyncOperationContext context) at System.Fabric.FabricClient.ServiceManagementClient.ResolveServicePartitionEndWrapper(IFabricAsyncOperationContext context) at System.Fabric.Interop.AsyncCallOutAdapter2`1.Finish(IFabricAsyncOperationContext context, Boolean expectedCompletedSynchronously) --- End of inner exception stack trace --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Microsoft.ServiceFabric.Services.Client.ServicePartitionResolver.<ResolveHelperAsync>d__29.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Communication.Client.CommunicationClientFactoryBase`1.<GetClientAsync>d__24.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.V1.FabricTransport.Client.FabricTransportServiceRemotingClientFactory.<Microsoft-ServiceFabric-Services-Communication-Client-ICommunicationClientFactory<Microsoft-ServiceFabric-Services-Remoting-V1-Client-IServiceRemotingClient>-GetClientAsync>d__9.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Communication.Client.ServicePartitionClient`1.<GetCommunicationClientAsync>d__28.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Microsoft.ServiceFabric.Services.Communication.Client.ServicePartitionClient`1.<InvokeWithRetryAsync>d__23`1.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.V1.Client.ServiceRemotingPartitionClient.<InvokeAsync>d__2.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.Builder.ProxyBase.<InvokeAsync>d__15.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.Builder.ProxyBase.<ContinueWithResult>d__16`1.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at BridgeService.BridgeService.<UpdateLcsDeploymentStatus>d__19.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at BridgeService.BridgeService.<DispatchMessage>d__18.MoveNext()

mambo_chami · February 16, 2021, 10:01am

Hie @MarkSiedle, Thanks for the help! I have managed to get out of this by running .\Set-CertificateAcls.ps1 on all ORCHS and subsequently running .\Test-D365FOConfiguration.ps1 which successfully passed.

But I’m now faced with the below error:

 XXX-OR-03 
  Message Unable to dispatch command id ff6fe672-d7e1-461e-a426-f92c58056ccc. Retries 9 
  Detail System.AggregateException: One or more errors occurred. ---> System.Fabric.FabricException: The specified credentials are invalid. ---> System.Runtime.InteropServices.COMException: Exception from HRESULT: 0x80071BF7 at System.Fabric.Interop.NativeClient.IFabricServiceManagementClient6.EndResolveServicePartition(IFabricAsyncOperationContext context) at System.Fabric.FabricClient.ServiceManagementClient.ResolveServicePartitionEndWrapper(IFabricAsyncOperationContext context) at System.Fabric.Interop.AsyncCallOutAdapter2`1.Finish(IFabricAsyncOperationContext context, Boolean expectedCompletedSynchronously) --- End of inner exception stack trace --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Microsoft.ServiceFabric.Services.Client.ServicePartitionResolver.<ResolveHelperAsync>d__29.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Communication.Client.CommunicationClientFactoryBase`1.<GetClientAsync>d__24.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.V1.FabricTransport.Client.FabricTransportServiceRemotingClientFactory.<Microsoft-ServiceFabric-Services-Communication-Client-ICommunicationClientFactory<Microsoft-ServiceFabric-Services-Remoting-V1-Client-IServiceRemotingClient>-GetClientAsync>d__9.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Communication.Client.ServicePartitionClient`1.<GetCommunicationClientAsync>d__28.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Microsoft.ServiceFabric.Services.Communication.Client.ServicePartitionClient`1.<InvokeWithRetryAsync>d__23`1.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.V1.Client.ServiceRemotingPartitionClient.<InvokeAsync>d__2.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.Builder.ProxyBase.<InvokeAsync>d__15.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.Builder.ProxyBase.<ContinueWithResult>d__16`1.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Dynamics.Operations.Local.StateService.StateServiceOperations.<GetTopologyInstanceHealth>d__9.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Dynamics.Operations.Local.StateService.StateServiceListener.<GetTopologyInstanceHealth>d__6.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.V1.Builder.MethodDispatcherBaseWithSerializer.<>c__DisplayClass12_0`1.<ContinueWithResult>b__0(Task`1 t) at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke() at System.Threading.Tasks.Task.Execute() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.V1.Builder.MethodDispatcherBaseWithSerializer.<DispatchAsync>b__2_0(Task`1 t) at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke() at System.Threading.Tasks.Task.Execute() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.V1.Runtime.ServiceRemotingDispatcher.<>c__DisplayClass9_0.<OnDispatch>b__0(Task`1 t) --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Microsoft.ServiceFabric.Services.Remoting.V1.Runtime.ServiceRemotingDispatcher.<>c__DisplayClass9_0.<OnDispatch>b__0(Task`1 t) at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke() at System.Threading.Tasks.Task.Execute() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.Runtime.ServiceRemotingCancellationHelper.<DispatchRequest>d__5`1.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.V1.Runtime.ServiceRemotingDispatcher.<RequestResponseAsync>d__5.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.V1.FabricTransport.Runtime.FabricTransportMessagingHandler.<RequestResponseAsync>d__3.MoveNext() --- End of inner exception stack trace --- at Microsoft.ServiceFabric.Services.Communication.Client.ServicePartitionClient`1.<InvokeWithRetryAsync>d__23`1.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.V1.Client.ServiceRemotingPartitionClient.<InvokeAsync>d__2.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.Builder.ProxyBase.<InvokeAsync>d__15.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.Builder.ProxyBase.<ContinueWithResult>d__16`1.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at BridgeService.BridgeService.<UpdateLcsDeploymentStatus>d__19.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at BridgeService.BridgeService.<DispatchMessage>d__18.MoveNext() ---> (Inner Exception #0) System.Fabric.FabricException: The specified credentials are invalid. ---> System.Runtime.InteropServices.COMException: Exception from HRESULT: 0x80071BF7 at System.Fabric.Interop.NativeClient.IFabricServiceManagementClient6.EndResolveServicePartition(IFabricAsyncOperationContext context) at System.Fabric.FabricClient.ServiceManagementClient.ResolveServicePartitionEndWrapper(IFabricAsyncOperationContext context) at System.Fabric.Interop.AsyncCallOutAdapter2`1.Finish(IFabricAsyncOperationContext context, Boolean expectedCompletedSynchronously) --- End of inner exception stack trace --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Microsoft.ServiceFabric.Services.Client.ServicePartitionResolver.<ResolveHelperAsync>d__29.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Communication.Client.CommunicationClientFactoryBase`1.<GetClientAsync>d__24.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.V1.FabricTransport.Client.FabricTransportServiceRemotingClientFactory.<Microsoft-ServiceFabric-Services-Communication-Client-ICommunicationClientFactory<Microsoft-ServiceFabric-Services-Remoting-V1-Client-IServiceRemotingClient>-GetClientAsync>d__9.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Communication.Client.ServicePartitionClient`1.<GetCommunicationClientAsync>d__28.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Microsoft.ServiceFabric.Services.Communication.Client.ServicePartitionClient`1.<InvokeWithRetryAsync>d__23`1.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.V1.Client.ServiceRemotingPartitionClient.<InvokeAsync>d__2.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.Builder.ProxyBase.<InvokeAsync>d__15.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.Builder.ProxyBase.<ContinueWithResult>d__16`1.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Dynamics.Operations.Local.StateService.StateServiceOperations.<GetTopologyInstanceHealth>d__9.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Dynamics.Operations.Local.StateService.StateServiceListener.<GetTopologyInstanceHealth>d__6.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.V1.Builder.MethodDispatcherBaseWithSerializer.<>c__DisplayClass12_0`1.<ContinueWithResult>b__0(Task`1 t) at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke() at System.Threading.Tasks.Task.Execute() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.V1.Builder.MethodDispatcherBaseWithSerializer.<DispatchAsync>b__2_0(Task`1 t) at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke() at System.Threading.Tasks.Task.Execute() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.V1.Runtime.ServiceRemotingDispatcher.<>c__DisplayClass9_0.<OnDispatch>b__0(Task`1 t) --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Microsoft.ServiceFabric.Services.Remoting.V1.Runtime.ServiceRemotingDispatcher.<>c__DisplayClass9_0.<OnDispatch>b__0(Task`1 t) at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke() at System.Threading.Tasks.Task.Execute() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.Runtime.ServiceRemotingCancellationHelper.<DispatchRequest>d__5`1.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.V1.Runtime.ServiceRemotingDispatcher.<RequestResponseAsync>d__5.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Services.Remoting.V1.FabricTransport.Runtime.FabricTransportMessagingHandler.<RequestResponseAsync>d__3.MoveNext

dane.falvo · February 23, 2021, 5:13am

Hi @mambo_chami,

As MarkSiedle mentioned, can I request you to submit this as a new ticket.

Please also include detials about your Octopus instance. Is it Octopus Cloud or Server and what version are you running.

I will lock this topic, so that you can create a new one.

Regards,

Dane