External Feeds: TeamCity NuGet Authentication issues after HTTPS switch over

Legacy Execution
1

Hi,

We have recently moved TeamCity to HTTPS, I believe the certificate is generated via our Internal CA Server. Prior to the move Octopus Deploy could read the feed with no issues, it uses a user that has been created in TeamCity.

After we switched TC over to HTTPS I change the Nuget Url stored in octopus to HTTPS
https://domain.name:8099/httpAuth/app/nuget/v1/FeedService.svc/

This is the error I receive when Testing the connection in octopus:

NuGet.Protocol.Core.Types.FatalProtocolException: Unable to load the service index for source https://domain.name:8099/httpAuth/app/nuget/v1/FeedService.svc/. ---> System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: The remote server returned an error: (403) Forbidden.
at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)
--- End of inner exception stack trace ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NuGet.Protocol.HttpHandlerResourceV3Provider.CredentialPromptWebRequestHandler.<SendAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at NuGet.Protocol.HttpHandlerResourceV3Provider.CredentialPromptWebRequestHandler.<SendAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NuGet.Protocol.HttpRetryHandler.<SendAsync>d__13.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NuGet.Protocol.HttpSource.<SendWithCredentialSupportAsync>d__28.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NuGet.Protocol.HttpSource.<GetThrottled>d__22.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NuGet.Protocol.HttpSource.<SendAsync>d__21.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NuGet.Protocol.Core.v3.ODataServiceDocumentResourceV2Provider.<CreateODataServiceDocumentResourceV2>d__9.MoveNext()
--- End of inner exception stack trace ---
at NuGet.Protocol.Core.v3.ODataServiceDocumentResourceV2Provider.<CreateODataServiceDocumentResourceV2>d__9.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NuGet.Protocol.Core.v3.ODataServiceDocumentResourceV2Provider.<TryCreate>d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NuGet.Protocol.Core.Types.SourceRepository.<GetResourceAsync>d__11`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NuGet.Protocol.PackageSearchResourceV2FeedProvider.<TryCreate>d__1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NuGet.Protocol.Core.Types.SourceRepository.<GetResourceAsync>d__11`1.MoveNext()

Looks like an authentication issue, although I’ve tried running this with a TC Guest User and updated Octopus to use the Guest Url but I still get the same error. I’ve also tried changing the password of the user and creating a new user.

I’ve switched it back to HTTP and it worked straight away. Octopus and TC are running on the same server. I can navigate to the URL using IE, enter the Octopus TeamCity User’s credentials and this works.

Octopus Version: 3.5.1
TeamCity Version: 9.1.1
TeamCity NuGet.exe: 3.4.3

Thanks,

Simon

2

Update:

I am able to run the following Command from my Workstation and on the server successfully.

Nuget list -source “https://theurlhere”

Tried using Nuget 2.8 & NuGet 3.5.0

3

Hi Simon,

We had an issue where the nuget libraries we use cached credentials, it is possible you are being affected by this. Can you try restarting the Octopus Server service? If this fixes things we resolved this issue in Octopus 3.8.5.

Regards,
Mark

4

I’ve tried restarting the service and I’m still receiving the exact same exception message.

Thanks,

Simon

5

Hi Simon,

I couldn’t reproduce with Octopus 3.5.1 against our TeamCity server, but that’s at v10. I’ll install v9 in the morning my time and do some additional testing.

Would you be able to test the feed from the command line on the Octopus machine with Nuget.exe from https://dist.nuget.org/index.html ? Just something like:

nuget.exe list APackageName -source https://domain.name:8099/httpAuth/app/nuget/v1/FeedService.svc/

We use the official Nuget libs internally which are also used as the basis for Nuget.exe.

Regards,
Mark

6

Unfortunately this didn’t work either.

7

Sorry, I meant to say this did work.

From: Simon Gates
Sent: 28 February 2017 11:31
To: 'Mark Rydstrom’
Subject: RE: External Feeds: TeamCity NuGet Authentication issues after HTTPS switch over. [Problems #51751]

Unfortunately this didn’t work either.

8

Hi Simon,

I’ve done some testing with Teamcity 9.1.1 and Octopus 3.5.1, and while I’m not seeing the 403 Forbidden that you are, I am seeing some unusual behavior (Octo 3.5.1 works fine with Teamcity 10, and current Octo works fine with Teamcity 9, but I don’t get any search results with Teamcity 9/Octo 3.5.1 when testing the feed).

Would you be able to install our current release on a spare dev machine and check if that can talk to your Teamcity feed?

I also had another support call today where it looks like they have a self-signed cert for SSL on Teamcity that isn’t trusted by the Octopus machine. That gives an error about unable to establish a trust relationship, so I doubt that’s the case here, but something to check.

Regards,
Mark

9

Hi Mark,

While I haven’t upgraded Octopus Deploy yet I took this opportunity to upgrade TeamCity as it’s been on our to-do list for a while, it’s now running version 10, unfortunately we still have this problem.

I’ll look into upgrading Octopus Deploy tomorrow.

Thanks,

Simon

10

Hi Mark,

I just wanted to update you, updating to the latest version of Octopus Deploy has solved this problem for us.

Thanks,

Simon