We are using octopus to deploy a windows application to our internal users (~250 targets). We have it setup to ignore targets that become unavailable or are unavailable at deployment but we are seeing an inconsistent behavior where deployment simply stops and the last machine on the ‘Deploy’ step has an empty output.
Can’t seem to find anything wrong on the tentacle logs for that machine that has ‘failed’ but has no output.
Other machines do fail on the same deployment but the log appears normally and deployment continues as expected.
Any ideas on how to understand what the problem is?
Hi Sebastian,
I’m really sorry for the delay getting back to you. I have been trying unsuccessfully to reproduce the issue you are seeing and got pulled away yesterday for an urgent bug. It’s my mistake, you can usually expect a prompt response.
Attempting to reproduce the issue I created a deployment that runs a script step to 300 Tentacles with the setting “Ignore unavailable machines” turned on. As a part of my script it turns random Tentacles on and off, so some are offline when the script attempts to run on them and some go offline while the script is executing. Unfortunately the deployment finishes every time for me.
There are a couple pieces of information that would help figure this out. Could send the raw task log for the deployment that gets stuck and the Tentacle log for the machine that is stuck. See here for help on getting raw task logs: https://octopus.com/docs/how-to/get-the-raw-output-from-a-task
A massive help would be a process dump from your Octopus Server and the affected Tentacle while the deployment is stuck. There are instruction here for generating a process dump: https://octopus.com/docs/reference/process-dumps#creating-a-process-dump
You can upload the dump files or any file containing sensitive information to: https://file.ac/0YGQA-U5jzM/
Cheers,
Shane
Good morning Shane,
Thanks for the response. I’ve uploaded the logfile from the server on that link you provided. About the crashdump, not sure if that would be useful now as it was several days ago we last attempted to deploy using octopus for that specific application (given it required more human intervention than our current/previous method) but do let me know if that would be at all useful and I’ll go through the steps.
Please let me know if there is anything I can help with in finding the issue as I am keen on using octo for our client deployments as well.
Regards,
Seb.
HI Seb,
Unfortunately the log does not indicate much. Is it always the same machine and project that is stuck? Have you ever had this happen for health checks?
I think the process dump would be most helpful. It will allow us to look at what is running at the time the Tentacle gets stuck. If you wouldn’t mind running through the steps I think it will give us the best shot to diagnose what is going wrong.
Cheers,
Shane
