one of our severs came to a crashing halt during a recent deployment due to phantom tasks making several tentacles get stuck in a wait state, thinking there was a task to be completed when there wasn’t.
our typical way to solve this is to stop the deploy, stop the effected tentacles, clear out the c:\octopus\tentacle\actor folder, restart the tentacles, and restart the deploy at the step where it left off getting stuck.
but this one time, it became difficult, because it appears that there were over 2 million files in the actors folder. why would there be SO MANY files there?
this is happening on several of our servers. sometimes in the c:\octopus\tentacle\actors folder, and sometimes in a named tentacle specific folder, like c:\octopus\CENTRAL\Tentacle\Actors
but we’re seeing over a million files in these folders, sometimes over two million.
Sorry for the delay in getting back to you, it looks like the Tentacle is having issues with accessing the Clock.pfa file, could you please confirm for me if this is the file that is filling up your actors folder?
Also, could you please check the Windows Audit Event log and see if there are any Audit Failures around times that these errors occurred.
the files filling the Actors folder are names Clock.pfs.wrxxxxxxxxxxxxxxxxxxx where the xxxxx’s is a long string of hexidecimal digits
several servers are running mostly fine, but are collecting hundreds-of-thousands or millions of these files. I’m looking around our network too see how widespread this problem is on our servers.
the pattern appears to be that on servers that host sites/services for only one customer, and thus only have one tentacle are fine. servers that host sites/services for multiple customers, and thus has multiple tentacles that are used simultaneously during deployments, have this problem of millions of files in the Actors folder(s).
This problem does go away in 3.0 However, this doesn’t really help you right now though.
We have had another report of this previously and they had a look at permissions of the account the Tentacle runs under, but we haven’t heard back from them if that sorted their issue or not unfortunately.
the problem was caused by the fact that we had multiple tentacle installed on the server to allow parallel deployments, but due to having been installed with an early version of the tentacle installer, all tentacles were pointing to the SAME actors folder. this caused lots of file access problems as each tentacle fought for the same clock files, and others.
we have reconfigured these servers’ tentacles to use separate folders, and the problem has gone away.
However, this doesn’t really help you right now though.