Moving some projects to a new separate Octopus Instance

Legacy Execution
1

Greetings,
as part of a company spinoff activities I need to separate some projects and associated dependencies from an existing Octopus instance to a fresh new one.

I’ve tried to use the migrator tool to export one of the projects (there are 96 in total) and import it into a blank Octopus instance but the transaction is rolling back at the end due to an error.

Can you give me some hints as to where I need to look, the log file tells me this:

2019-07-02 12:00:42.5569 15624 1 ERROR Error importing Project ‘01.CHIL’ (Projects-209):
2019-07-02 12:00:42.5569 15624 1 ERROR Missing dependency Projects-209-2018073104385264

I am not sure what ‘Projects-209-2018073104385264’ is as it doesn’t look like a proper project key.

Here are the commands I used to export and to import:

.\Octopus.Migrator.exe partial-export --clean --directory=c:\export --project=‘01.CHIL’ --password=abc123

.\Octopus.Migrator.exe import --instance=xxxxx --directory=c:\export --password=abc123

Both Octopus Deploy instances run on the same VM so are the same version. MS SQL database also running on the same server.

Any hints for further diagnosis would be highly appreciated.

Best regards,
Robert

3

Hi Robert,

Thanks for getting in touch! I’m sorry to hear you’re hitting this unexpected issue on importing this project. The Missing dependency Projects-209-2018073104385264 seems to imply that the project you’ve exported contains a Deploy a Release step to trigger a deployment of a different project (`Projects-209) - is that assumption correct?

If that’s correct, it looks like you’re probably hitting a current known bug reported here. The workaround to this issue is to additionally export the dependent project (that’s deployed via the Deploy a Release step in your 01.CHIL project) in your partial-export command. You can specify this project in the same command by adding another –project=VALUE option.

If this isn’t applicable to your scenario, could you let me know which Octopus version you’re using to help me reproduce this behavior?

I look forward to hearing back and getting to the bottom of this one!

Best regards,

4

Hi Kenneth,
sorry for missing the obvious information: Octopus Deploy 2018.10.4 LTS.

I have searched for a “Deploy a Release” step in the project’s process and couldn’t find one so it may or not be the bug you mentioned… can I send you the 01.CHIL project process .json file, maybe I’m missing something obvious as I’m not familiar at all with the project in question.

I also tried to find out if Projects-209-2018073104385264 actually exists but it doesn’t (I simply tried the /api URL to fetch it:

{
“ErrorMessage”: “The resource ‘Projects-209-2018073104385264’ was not found.”
}

whereas for Project-209 (that’s our ‘01.CHIL’ project), I obtain:

{
“Id”: “Projects-209”,
“VariableSetId”: “variableset-Projects-209”,
“DeploymentProcessId”: “deploymentprocess-Projects-209”,
“ClonedFromProjectId”: null,
“DiscreteChannelRelease”: false,
“IncludedLibraryVariableSetIds”: [],
“DefaultToSkipIfAlreadyInstalled”: false,
“TenantedDeploymentMode”: “Untenanted”,
“DefaultGuidedFailureMode”: “EnvironmentDefault”,
“VersioningStrategy”: {
“Template”: “#{Octopus.Version.LastMajor}.#{Octopus.Version.LastMinor}.#{Octopus.Version.NextPatch}”,
“DonorPackage”: null,
“DonorPackageStepId”: null
},
“ReleaseCreationStrategy”: {
“ChannelId”: null,
“ReleaseCreationPackage”: null,
“ReleaseCreationPackageStepId”: null
},
“Templates”: [],
“AutoDeployReleaseOverrides”: [],
“Name”: “01.CHIL”,
“Slug”: “01-chil”,
“Description”: “CHIL cloud services”,
“IsDisabled”: false,
“ProjectGroupId”: “ProjectGroups-543”,
“LifecycleId”: “Lifecycles-221”,
“AutoCreateRelease”: false,
“ProjectConnectivityPolicy”: {
“SkipMachineBehavior”: “None”,
“TargetRoles”: [],
“AllowDeploymentsToNoTargets”: false
},
“Links”: {
“Self”: “/api/projects/Projects-209”,
“Releases”: “/api/projects/Projects-209/releases{/version}{?skip,take,searchByVersion}”,
“Channels”: “/api/projects/Projects-209/channels{?skip,take,partialName}”,
“Triggers”: “/api/projects/Projects-209/triggers{?skip,take,partialName,triggerActionType}”,
“ScheduledTriggers”: “/api/projects/Projects-209/scheduledtriggers{?skip,take,partialName}”,
“OrderChannels”: “/api/projects/Projects-209/channels/order”,
“Variables”: “/api/variables/variableset-Projects-209”,
“Progression”: “/api/progression/Projects-209{?aggregate}”,
“DeploymentProcess”: “/api/deploymentprocesses/deploymentprocess-Projects-209”,
“Web”: “/app#/projects/Projects-209”,
“Logo”: “/api/projects/Projects-209/logo?cb=Projects-209-2018073104385264”
}
}

Thanks,
Robert

5

Hi Robert,

Thanks for following up and providing that additional information! Projects-209-2018073104385264 is is your project logo file name (which takes the form of Projects-ID-Date). It looks like the import is failing as the project isn’t able to find the logo file. These files are stored in your Artifacts folder (C:\Octopus in standard installations), so I suspect you’d just need to copy that file to the destination instance’s artifacts folder.

Let me know how you go or if there’s anything else I can assist with. :slight_smile:

Best regards,

Kenny

6

Hi Kenneth,
you got it right! I’m experimenting this on a copy of the production instance and I indeed missed copying those folders to my test instance… I can see that the export operation has copied those logos in the export folder now and that the import operation doesn’t fail anymore.

I do have another import warning message I need to investigate but I’m not sure if you prefer me opening a separate request for support or if you’re ok to piggyback on this same thread.

Thanks!
Robert

7

Hi Robert,

That’s great to hear, thanks for letting me know that fixed the import error. :slight_smile:

You can certainly piggyback on this same thread! I’ll be happy to look into this new warning message you’re hitting.

I look forward to hearing back.

Best regards,

Kenny

8

Hi Kenneth,
so I now tried exporting / importing the whole group of projects (95 of them) and hitting a different error.

I have attached the log file for your reference. EDIT: I can’t attach for some reason, even though the log file is only 2MB…
ImporterLogSnippet.txt (5.6 KB)

Regards,
Robert

9

Hi Robert,

Thanks for keeping in touch! The new error you’re hitting as reported in your logs is indicating that both the source and destination projects have channels with the same name. In this case, I think all you’d need to do is add the --overwrite option to your import command which will force any name-matched documents to be overwritten. (Also to be safe, we always recommend taking a backup of your db in case you need to roll back).

I hope this helps get you going! Let me know if there’s anything at all I can assist with. :slight_smile:

Best regards,

10

Hi Kenneth,
I actually did try with the --overwrite option but it just hits a different problem later in the import process. See attachment log file snippet.

ImporterLogSnippet-2.txt (6.3 KB)

Should I attempt to export / import each project independently? It’s a bit tedious as there are 95 of them but I could script it…

Regards,
Robert

11

Hi Kenneth,
do you need me to provide any additional information to diagnose this? I tried a few times the export/import procedure and I’m ending up with the same error.

Kind regards,
Robert

12

Hi Kenneth, any updates on this? I’m still stuck with the import step…

Regards,
Robert

13

Adding some more information: I have upgraded to 2019.6.5 LTS and the error message is a bit more explicit but still there.

14

Hi Robert,

Thanks for following up, and my sincere apologies about completely missing your earlier replies.

Were you able to try exporting/importing only a single project, and if so was that successful?
What’s the new more explicit error message you’re seeing after the upgrade?

I look forward to hearing back!

Best regards,

Kenny

15

Hi Kenneth,
let me summarize what I tried recently:

  • Export / import of a single project using 2018.10.4 LTS - worked ok

  • Export / import of all projects together fails the during import:

    -> Error 1: Name conflict for the “Default Machine Policy”: I could pass this error by logging into the empty instance and renaming the existing machine policy to “Default Machine Policy - ORIGINAL”

    -> Error 2: Channel name already exists:
    2019-07-30 14:38:28.3591 10216 1 ERROR An unexpected error occurred when processing document with source ID ‘Channels-634’: A channel with this name already exists for this project. Please choose a different name.
    Nevermore.UniqueConstraintViolationException: A channel with this name already exists for this project. Please choose a different name.

I’ve been able to finish the import procedure by using the --overwrite flag but this seems to cause other issues because I’m getting various exceptions when navigating the UI or to project sections and other locations so I’m not confident with using this flag.

Final note, I tried with both the 2019.6.5 LTS as well as the latest version from 2019.7.3 and both fail to import the exported projects. Importer tool error messages are a bit more explicit in the latest versions though so that was helpful (see above).

Any suggestions? I can provide you with the exported data if you wish to have a closer look.

Best regards,
Robert

PS: tomorrow is the Swiss National Day so I won’t be available until Friday. I can send you files though.

16

Hi Robert,

Thanks for following up with that extra bit of info! Great to hear you’ve been able to make some progress.

Just one more quick thing to confirm if you don’t mind; after each of these upgrades, did you re-run the export before attempting to import into the other instance? If not, it may be worth trying to do the export and import from scratch (with no upgrades in between).

I look forward to hearing back!

Best regards,

Kenny

17

Hi Kenneth,
I’m back from holidays today and will try to export / import again as I used the same exported files.

I’ll let you know the outcome shortly.

Regards,
Robert

18

Ok, so here’s the latest result:

  • Exported from 2019.7.12 -> Imported ok
  • Exported from 2019.6.7 LTS -> Import fails.

See attached importer log.

edit: The reason I mention this is because we have to stay on LTS versions so we won’t be able to use 2019.6.7 for the production environment. To me it seems that a defect must’ve been fixed between 2019.6.7 and 2019.7.12 that resolves the issue (or some other change).

Kind regards,
Robert

19

Hi Robert,

Thanks for following up on this, and I’m terribly sorry for the delay in getting back to you here. I somehow missed your most recent replies. I’ve made sure to bookmark this thread to ensure I don’t miss another reply. :slight_smile:

This certainly is strange. It’s blowing up at Cannot insert the value NULL into column 'ProjectGroupId'. I’m not sure what change between these two versions could affect this, but it does seem likely that something is responsible for this change in behavior. The only issue related to importing via the migrator fixed in that timeframe is the following:

It’s possible that this was fixed in conjunction with addressing this bug, but this isn’t really an ideal answer. Can you confirm that the failed import of the export taken from 2019.6.7 was also being imported into an instance running this exact same version?

I look forward to hearing back!

Best regards,

Kenny

20

Hi Kenneth,
yes, I can confirm that the export and import are taken with the same version each time, dropped the databases, etc…

It’s all on the same server so I just export the Default instance and import into another instance.

I re-ran all the operations from scratch just to make sure that I could confirm it works on the latest version and fails on the LTS one.

If you’d like me to provide you with the data set to assist with diagnostics & root cause finding, let me know!

Kind regards,
Robert

21

Hi Robert,

Thanks for following up and confirming that. It might be worth looking at the export taken from your 2019.6.7 LTS version to help us narrow this down. If you’re willing to supply that, feel free to upload it privately and securely to the following link (which is upload only).

https://file.ac/21euCO1oZvAxaocjvJMHlg/

I look forward to hearing back!

Best regards,

Kenny