We are continuously deploying every branch of our project on new commits, for testing purposes.
This deployment is done on two servers.
On one of the two servers creating a new site takes a really long time, several minutes.
from log
Slow server:
04:43:59 Info | Site "my-non-existing-site" does not exist, creating...
04:48:00 Info | Name : my-non-existing-site
04:48:00 Info | ID : XXX
04:48:00 Info | State : Started
04:48:00 Info | PhysicalPath : C:\Octopus\Applications\some-path
Note 4 minutes from starting site creating to completion.
Fast server:
04:43:31 Info | Site "my-non-existing-site" does not exist, creating...
04:43:33 Info | Name : my-non-existing-site
04:43:33 Info | ID : XXX
04:43:33 Info | State : Started
04:43:33 Info | PhysicalPath : C:\Octopus\Applications\some-path
The slow server is a less powerful machine than the fast server, but it is only this task that show this kind of slowness.
I believe the calamari script line taking a long time would be this one:
Thanks for getting in touch. That is strange try some testing directly on the machine, and see what you get, here’s a script to create a site and dump out the times. It might also be worthwhile running resource monitor and see what else is going on.
echo "timed run of creating an IIS site"
Import-Module WebAdministration
$siteName = [guid]::NewGuid()
echo "creating IIS:\Sites\$siteName"
New-Item -Path "IIS:\Sites\$siteName" -Type Site -Bindings @{protocol="http";bindingInformation="*:15444:"}
$command = Get-History -Count 1
echo ">>> New-Item ran for $(($command.EndExecutionTime - $command.StartExecutionTime).TotalSeconds) seconds"
Set-ItemProperty -Path "IIS:\Sites\$siteName" -name PhysicalPath -value "C:\Test\$siteName"
$command = Get-History -Count 1
echo ">>> New-ItemProperty ran for $(($command.EndExecutionTime - $command.StartExecutionTime).TotalSeconds) seconds"
Remove-Item -Path "IIS:\Sites\$siteName"
echo "removed site, and done."
What are the machine specs of the fast and slow one, are they physical, virtual, or virtual and shared?
Tried creating a site in UI, no slowness experienced.
Tried running your script with following output:
Name ID State Physical Path Bindings
---- -- ----- ------------- --------
ce158270-a219-4f 10 Started http *:15444:
e5-98b0-56dc094b
33bc
>>> New-Item ran for 3.9252245 seconds
>>> New-ItemProperty ran for 3.9252245 seconds
removed site, and done.
One other difference between the fast and slow server, is that the slow one is a polling tentacle.
Thanks for trying that script out, that’s very strange. The fact that it’s a polling server shouldn’t really have an impact.
A simple next step might to be logged in on that slow machine to watch what it’s doing while the deploy scripts run. If you could have task manager and resource monitor open, look for processes using high CPU first, then disk access. It would be good to compare both the fast server and the slow server as part of this.
Could you also list more differences between the 2 machines, what other software runs on them, does either run anti-virus software, what are their cpu/ram specs, etc, to see if something stands out? Also if you could run the deploys on the slow server a few times to see if it’s consistently slow, or if there’s any variation.
We did a lot of deploys, and we did not properly clean up expired stuff, leaving each server with hundreds of sites, and as the number of sites grew, the time for site creating increased.
Deleting all the unused sites reduced the time to around 2 minutes for complete deployment of new sites on both nodes.