Corruption after restore from backup

Legacy Uncategorised
1

Hi,

I had a unusual situation where a tentacle got stuck reporting that another task was already in progress. I double checked that no other were tasks were actually running. The tentacle in question was running on the Octopus Server, and this happened after the tentacle process was terminated in task manager.

To try and resolve this I restarted Octopus Server, restarted the Tentacle, deleted and recreated the release, deleted and recreated the project. Ran the Raven repair tool, rebooted the server. Uninstalled the tentacle and re-added. No matter what I did I keep getting the same error. As a last resort I restored from an earlier backup from before the issue arose, and to my annoyance found I had exactly the same issue.

Eventually I cleared this by uninstalling the tentacle, deleting the installation folder, and then reinstalling the tentacle.

The next problem I found is that the restore process has corrupted one of my projects, and it’s lost it’s reference to the project VariableSet. I looked in Raven and found the referenced VariableSet was indeed missing. As the backup didn’t fix the original issue, I tried to restore back to to my original version and found the same corruption still occurred. I was able to fix this by manually crafting a VariableSet document to match the project VariableSet Id.

Any idea how to recover from corruption of the database? The repair tool doesn’t seem to catch all issues. I’m now getting ghost releases and projects which error when clicked with not found messages. Everything was fine prior to restore (apart from the stuck tentacle).

Thanks,
Mike

2

Hi Mike,

Thanks for getting in touch!

I am sorry that you are having so many issues. We do have a fix in the system currently where there was a part of the backup process missing (it will be released soon), however from what you are describing that does not appear to be the issue here.
As for the task in progress, you can end up with ghost tasks when things are terminated and messages cannot get through between the Octopus Server and Tentacle to properly close them down. There is a timeout feature that eventually clears them out.

But now after the restore as you have had so many issues we would suggest that a Skype call is the next step to resolving this.
https://octopusdeploy.acuityscheduling.com/schedule.php
Please pick a time that suits so we can attempt to figure this out and get everything back up and running smoothly.

Vanessa

3

HI sorry for not getting back you. I managed to resolve my issue my manipulating the RavenDB manually.