Copy Files to Custom Directory Very Slow

Legacy Uncategorised
1

Version: 2.6.5.1010

We’re having an issue where copying files to a custom installation directory is very slow. Often times over 30 minutes for a 400 MB deployment.

The deployment from Octopus to the Tentacle is fairly quick (Around 4 minutes), however the “Copy files to C:\wwwroot\application\1.0.0.0” step takes forever.

Manually copying the files from ‘C:\Octopus\Applications\Application\1.0.0.0’ to ‘C:\wwwroot\application\1.0.0.0’ happens within a few minutes also, so it doesn’t appear to be an I/O issue.

Any ideas what might be causing this?

Thanks,
Will

2

Hi Will,

Thanks for getting in touch. Could you share us your deployment log so we can see the context of the slowdown?

http://docs.octopusdeploy.com/display/OD/Get+the+raw+output+from+a+task

Thanks,
Dalmiro

3

Attached with sensitive information replaced with generic info.

ServerTasks-8184_log.txt (10 KB)

4

Hi Will,

Your log shows the following:

14:01:53   Verbose  |           Looking for any configuration transformation files
14:01:53   Verbose  |           Looking for appSettings and connectionStrings in any .config files
14:59:36   Verbose  |           Storing a record of the deployment.

The slowdown seems to occur when the feature Configuration Variables starts running, which basically does a recursive search through your 14k files until it finds a .config one, and then it digs into that file to look for “AppSettings” and “connectionStrings”. It seems like this recursive seach is what’s killing your I/O.

Instead of using “Configuration Variables”, could you try to use the “Substitute Variables in files” feature? On this one you can specify the exact path of the files that you want Octopus to pay attention to, which should significantly decrease the overall time of the process.

More info about this feature: http://docs.octopusdeploy.com/display/OD/Substitute+Variables+in+Files

Let me know if that works

Best regards,
Dalmiro

5

Forgot to attach this screenshot. In red you can see the feature you are using right now, and in green the one I’m recommending you to use.

configfeatures.jpg
configfeatures.jpg1222×419 54.3 KB

6

Thanks Dalmiro, I’ll check this out and report back.

7

The slowness is still occuring after disabling the Configuration Variables option.

To me, the issue appears to be during the file copy phase:

                    |           Success: Copy files to C:\wwwroot\Retailer\Website\1.0.0.0
14:01:54   Info     |             Purging the directory 'C:\wwwroot\Retailer\Website\1.0.0.0'
14:01:54   Info     |             Copying package contents to 'C:\wwwroot\Retailer\Website\1.0.0.0'
                    |           
                    |           Success: CustomScriptConvention.PostDeploy.ps1
14:59:35   Verbose  |             Running PowerShell script: C:\wwwroot\Retailer\Website\1.0.0.0\CustomScriptConvention.PostDeploy.ps1
14:59:36   Info     |             Updating physical path for: photo.samsclub.com
                    |             Original Physical Path: C:\wwwroot\Retailer\Website\1.0.0.0
                    |             New Physical Path: C:\wwwroot\Retailer\Website\1.0.0.0
                    |             Confirmation:
                    |             Name             ID   State      Physical Path                       Bindings
                    |             ----             --   -----      -------------                       --------
                    |             www.website.com  1    Started    C:\wwwroot\Retailer\Website\1.0.0.0 http *:80:www.website.com
                    |           
                    |             Storing to variable 'previousVersion': 0.9.0.0
14:59:36   Info     |             Storing to variable 'newVersion': 1.0.0.0
8

Hi Will,

Do you have some sort of anti-virus software that might be trying to scan every file that’s being created/moved to that directory? If you have one, adding C:\wwwroot\* as an exception might be a good thing to test.

Dalmiro