Hi,
We’re deploying raster images for the mapping component of our app using Octopus Deploy, but Octopus is taking a very long time (over 10 hours) to deploy our packages of raster images (they contain over 5 million tile images).
Do you have any suggestions to speed things up? I was thinking of perhaps making the package contain a single zip file, and then a post-deployment step does the unzipping?
Many thanks,
Ant.
Hi @antmeehan
Thanks for getting in touch!
This sounds like an interesting problem to have. Can I ask a few questions so we can work out the best way that we can get these files to disk? Are you able to let me know the (rough) size of the package, and a rough size of each individual file? We would also love to know if it’s faster extracting the files outside of Octopus (as it shouldn’t be).
One option that came up straight away would be to use the Transfer a package step, then you could perform the extraction yourself (either manually or by triggering a script to do the job for you).
If you can give us as much context as possible that would be greatly appreciated!
Thanks Anthony, I look forward to hearing from you shortly,
Regards,
Alex
Hi Alex,
The biggest package is 4.7Gb and has ~5 millions files, with each file only a few hundred bytes each.
Thanks for the tip on the Transfer a package step. I’ll give that a try now. I’ve timed using 7zip to extract the files and it is around 1hr 41 mins on my local machine.
Thanks,
Ant.
Hi Anthony,
Just wanted to check in on how you were going, is the Transfer a package step working out for you? Happy to dig a bit deeper into performance if needed.
I look forward to hearing from you,
Regards,
Alex
Hi Alex,
Yes the Transfer a package worked perfectly. I then added another step to do the extraction using 7zip and then remove the transferred package.
The Octopus.Action[<Step name>].Output.Package.FilePathvariable came in very handy for this process 
Thanks,
Anthony.
Hi Anthony,
That’s great news! Let me know if there is anything else we can assist with.
Happy deployments,
Regards
Alex
Hi Alex,
We have had one issue with these large packages, which is that it seems to fail when trying to calculate a delta for a newer version:
| Running: Building delta for **Package** v2019.1.2 (2.857 GB)
08:35:10 Info | Using package C:\Octopus\Packages\**Package**\**Package**.2019.1.1.zip with hash b04f3ee8f5e43fa3b162981b50bb72fe1acabb33 for creating delta.
08:35:10 Verbose | Building signature file: C:\Octopus\OctopusServer\PackageCache\**Package**.2019.1.1.zip.octosig
08:35:10 Verbose | - Using nearest package: C:\Octopus\Packages\**Package**\**Package**.2019.1.1.zip
08:35:10 Verbose | Building delta file: C:\Octopus\OctopusServer\PackageCache\**Package**.2019.1.1_B04F3EE8_to_2019.1.2_0E99C62D.octodelta
08:35:10 Verbose | - Using package: C:\Octopus\Packages\**Package**\**Package**.2019.1.2.zip.
08:35:10 Verbose | - Using signature: C:\Octopus\OctopusServer\PackageCache\**Package**.2019.1.1.zip.octosig
08:37:09 Warning | The previous command returned a non-zero exit code of: 3
08:37:09 Warning | The command that failed was: "C:\Program Files\Octopus Deploy\Octopus\Octodiff.exe" delta "C:\Octopus\OctopusServer\PackageCache\**Package**.2019.1.1.zip.octosig" "C:\Octopus\Packages\**Package**\**Package**.2019.1.2.zip" "C:\Octopus\OctopusServer\PackageCache\**Package**.2019.1.1_B04F3EE8_to_2019.1.2_0E99C62D.octodelta.partial" --progress
|
Any ideas? Is there a way to turn off using deltas for this project/step?
Thanks,
Anthony.
Hi Anthony,
You can indeed disable Delta on a per project basis, you can find the details on how to do that here.
I suspect that Delta is falling over on the number of files (and there small size) in your package, let me know if you want me to look into this further for you.
Let me know if there is anything else you need,
Regards,
Alex
HI Alex,
That’s for that. I’ll let you know how it goes.
Thanks,
Anthony.