UI Error accessing Templates and Package Library after server upgrade

Octopus Server
, , , ,
1

We recently updated a self-hosted from 3.13.10 (I believe) to 2020.5.0.

Currently, accessing Library->Step Templates the following error is displayed:

Error reading row 4, column 4. Unexpected token while deserializing object: PropertyName. Path ‘Packages[0].AcquisitionLocation’, line 1, position 1026… Compiled reader expression: (DbDataReader reader, DocumentReaderContext context) => { Type deserializeAsType = Octopus.Core.Model.Projects.ActionTemplate context.Column = 0 String temp0 = IIF(reader.IsDBNull(0), null, reader.GetString(0)) context.Column = 1 String temp1 = IIF(reader.IsDBNull(1), null, reader.GetString(1)) context.Column = 2 Int32 temp2 = reader.GetInt32(2) context.Column = 3 String temp3 = IIF(reader.IsDBNull(3), null, reader.GetString(3)) context.Column = 4 ActionTemplate deserializedFromJson = context.DeserializeText(reader, 4, deserializeAsType) context.Column = 5 String temp5 = IIF(reader.IsDBNull(5), null, reader.GetString(5)) context.Column = 6 String temp6 = IIF(reader.IsDBNull(6), null, reader.GetString(6)) ActionTemplate result = deserializedFromJson if (result != null) { result.Id = temp0 result.Name = temp1 result.Version = temp2 result.ActionType = temp3 result.CommunityActionTemplateId = temp5 result.SpaceId = temp6 } result } Unexpected token while deserializing object: PropertyName. Path ‘Packages[0].AcquisitionLocation’, line 1, position 1026.

Similar errors are presented when trying to download a package from the internal package repository.
Using some sleuthing we have identified that Some, not All step templates have the issue buy doing incremental calls to https://deployit/api/Spaces-1/actiontemplates?take=1&skip=1 incrementing the skip to see which ones result in an error and which don’t. We’ve correlated that with the rows from the database in the ActionTemplate which have an ActionType = ‘Octopus.TentaclePackage’
The indicated JSON property of AcquisitionLocation is set to ‘Server’

We have another Octopus instance that was also updated to 2020.5.0, but from a much more recent version that does not have this. So far we can’t see the appreciable difference in the JSON in the ActionTemplate table that could be causing the issue.

Also as Mentioned, we have a same/similar issue when trying to download packages from the package repository:

Error reading row 1, column 4. Unexpected token while deserializing object: PropertyName. Path ‘Steps[11].Actions[0].Packages[0].AcquisitionLocation’, line 1, position 23403…\r\nCompiled reader expression:\r\n\r\n(DbDataReader reader, DocumentReaderContext context) => \r\n{\r\n Type deserializeAsType = Octopus.Core.Model.Projects.DeploymentProcess\r\n context.Column = 0\r\n String temp0 = IIF(reader.IsDBNull(0), null, reader.GetString(0))\r\n context.Column = 1\r\n String temp1 = IIF(reader.IsDBNull(1), null, reader.GetString(1))\r\n context.Column = 2\r\n Boolean temp2 = IIF(reader.IsDBNull(2), default(Boolean), Convert(reader.GetValue(2), Boolean))\r\n context.Column = 3\r\n Int32 temp3 = reader.GetInt32(3)\r\n context.Column = 4\r\n DeploymentProcess deserializedFromJson = context.DeserializeText(reader, 4, deserializeAsType)\r\n context.Column = 5\r\n ReferenceCollection temp5 = Convert(value(Octopus.Core.RelationalStorage.TypeHandlers.UntypedReferenceCollectionTypeHandler).ReadDatabase(reader, 5), ReferenceCollection)\r\n context.Column = 6\r\n String temp6 = IIF(reader.IsDBNull(6), null, reader.GetString(6))\r\n DeploymentProcess result = deserializedFromJson\r\n if (result != null)\r\n {\r\n result.Id = temp0\r\n result.OwnerId = temp1\r\n result.IsFrozen = temp2\r\n result.Version = temp3\r\n result.SpaceId = temp6\r\n }\r\n\r\n result\r\n}\r\n\r\nUnexpected token while deserializing object: PropertyName. Path ‘Steps[11].Actions[0].Packages[0].AcquisitionLocation’, line 1, position 23403.

Interestingly this only applies to packages that existed prior to the update. I manually uploaded a package and was able to download it again without issue. That was a wholey new package, not used in any steps or deployments so may not be a good indication of the core issue.

Any help would be appreciated.

2

Hey Ryan,

Thanks for reaching out.

I’m really sorry to hear you’re having issues after your upgrade.

How long ago did the upgrade occur?
Did you take a backup before you upgraded?
Did you do a straight upgrade from 3 to latest, or did you do it in stages?

Please let me know if you have any questions.

Thanks,
Jeremy

4

Thanks for your response Jeremy,

How long ago did the upgrade occur?

  • Upgraded last night (Wednesday)

Did you take a backup before you upgraded?

  • Yes, we have a DB backup and a snapshot of the server that we can revert to.
  • It’s worth mentioning that we also migrated the DB over to Server 2016 as the old one was on 2012 and not supported by the latest release (this may be the source of the problem, but I repointed the config file to the new ‘restored’ database prior to running the upgrade)

Did you do a straight upgrade from 3 to latest, or did you do it in stages?

5

Hi Ryan,

Thanks for all the information.

Have you done much work or many changes since the upgrade? We have had a few reports of going from 3.x straight to 2020.5 causing various issues. We’ve found that stepping up in chunks has had better results. We’re still trying to figure out if these are outliers or not.

Would it be possible for you to revert and upgrade in smaller increments to get to latest and see if the problem persists? I could try it on my end if you upload a database backup for me to attempt it, but that would be potentially more changes that you guys would be making to your current server that would get lost in the case that you need to revert eventually (by the time it uploads, i download it, upgrade everything, and try to reproduce your error).

Please let me know which direction you’d like to go.

If you want me to try on my end, you can upload the database via our support files.

To do this please do the following:

Log in to octopus.com
Click your profile in the upper right
Click your organization
Go to the support tab
Then upload any required files.​​​

Thanks,
Jeremy

1 Like
7

Hi Ryan,

A colleague of mine working with another incident of this has tested upgrading in chunks and it worked for him.

He upgraded to 2020.2.12, then to 2020.5.0 and it worked for him. The bug has been identified and has been submitted to the engineers, but I’m not sure what that means in the way of people currently experiencing the bug, though.

Is downgrading and upgrading in chunks a viable route for you?

Please let me know.

Thanks,
Jeremy

1 Like
8

Hi Jeremy,
We restored from backup and did incremental updates and that did result in a much happier octopus.
Thank you,

1 Like
9

Hi Ryan,

You’re very welcome! Thanks for letting me know it worked for you. I’m glad you’re in a good state again.

I’ll pass along this information to our engineers as we investigate the upgrade path further.

I hope you have a great weekend.

Best,
Jeremy

1 Like
10

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.