Here are the steps to reproduce:
OS: Windows 10
Notes: Works with a single file upload.
Hi,
Thanks for getting in touch. Just to make sure I understand your scenario correctly, you have multiple Zip files inside the package you are deploying from Octopus? And those files are being correctly uploaded to S3 but if you try to download and extract them they fail/are corrupt?
From what I can see in the screenshot, you have configured both the file pattern for what to upload and also the file pattern for what to perform variable substitution on. I think it may be the latter that is causing the issue, as the variable substitution is designed to find and replace values in a text file it will likely corrupt a Zip file if it changes any content.
Would you be able to try clearing that second field and trying again to see if that resolves it?
Regards
Shannon
Hello Shannon,
Thanks for your response. You got it all correct. I have tried clearing everything from the ‘Variable Substitution file Patterns’ field, but it copied only one of the zip files in the packages. I need to copy all of the zip files to a S3 bucket.
Thanks,
Mishka
Hi Mishka,
Ok, so that stopped it corrupting the file, but there’s still only one file?
Are the files all in the root level of the package? If so then what you have should work, I’d have thought.
It might be worth trying **\ at the start of the path, it uses ant syntax and this will tell if to try all files in all paths.
If that still doesn’t work, could you attach a deployment log and we’ll see if that sheds some light on what’s happening.
Regards
Shannon
Shannon,
It still didn’t work. Here is the deployment log:
Running the step as the AWS user octopus-deploy-service-user
October 26th 2018 09:52:20
Info
Glob pattern ‘*0.1.0+66.zip’ matched 8 files.
October 26th 2018 09:52:20
Warning
No files were found that match the substitution target pattern ‘**’
October 26th 2018 09:52:20
Info
Attempting to upload C:\Octopus\Work\20181026155217-147725-2098\staging\ApprovalEmail.Lambda.0.1.0+66.zip to bucket hg-sre with key release/packages/0.1.0+61-test2/ApprovalEmail.Lambda.zip0.1.0+66/ApprovalEmail.Lambda.0.1.0+66.zip.
October 26th 2018 09:52:20
Info
Saving object version id to variable “Octopus.Action[Upload lambda s3 zip].Output.Files[release/packages/0.1.0+61-test2/ApprovalEmail.Lambda.zip0.1.0+66/ApprovalEmail.Lambda.0.1.0+66.zip]”
October 26th 2018 09:52:20
Info
Attempting to upload C:\Octopus\Work\20181026155217-147725-2098\staging\DeleteSlackNotification.Lambda.0.1.0+66.zip to bucket hg-sre with key release/packages/0.1.0+61-test2/ApprovalEmail.Lambda.zip0.1.0+66/DeleteSlackNotification.Lambda.0.1.0+66.zip.
October 26th 2018 09:52:21
Info
Saving object version id to variable “Octopus.Action[Upload lambda s3 zip].Output.Files[release/packages/0.1.0+61-test2/ApprovalEmail.Lambda.zip0.1.0+66/DeleteSlackNotification.Lambda.0.1.0+66.zip]”
October 26th 2018 09:52:21
Info
Attempting to upload C:\Octopus\Work\20181026155217-147725-2098\staging\InteractiveSlackNotifiction.Lambda.0.1.0+66.zip to bucket hg-sre with key release/packages/0.1.0+61-test2/ApprovalEmail.Lambda.zip0.1.0+66/InteractiveSlackNotifiction.Lambda.0.1.0+66.zip.
October 26th 2018 09:52:22
Info
Saving object version id to variable “Octopus.Action[Upload lambda s3 zip].Output.Files[release/packages/0.1.0+61-test2/ApprovalEmail.Lambda.zip0.1.0+66/InteractiveSlackNotifiction.Lambda.0.1.0+66.zip]”
October 26th 2018 09:52:22
Info
Attempting to upload C:\Octopus\Work\20181026155217-147725-2098\staging\Jira.Lambda.0.1.0+66.zip to bucket hg-sre with key release/packages/0.1.0+61-test2/ApprovalEmail.Lambda.zip0.1.0+66/Jira.Lambda.0.1.0+66.zip.
October 26th 2018 09:52:22
Info
Saving object version id to variable “Octopus.Action[Upload lambda s3 zip].Output.Files[release/packages/0.1.0+61-test2/ApprovalEmail.Lambda.zip0.1.0+66/Jira.Lambda.0.1.0+66.zip]”
October 26th 2018 09:52:22
Info
Attempting to upload C:\Octopus\Work\20181026155217-147725-2098\staging\Octopus.Lambda.0.1.0+66.zip to bucket hg-sre with key release/packages/0.1.0+61-test2/ApprovalEmail.Lambda.zip0.1.0+66/Octopus.Lambda.0.1.0+66.zip.
October 26th 2018 09:52:23
Info
Saving object version id to variable “Octopus.Action[Upload lambda s3 zip].Output.Files[release/packages/0.1.0+61-test2/ApprovalEmail.Lambda.zip0.1.0+66/Octopus.Lambda.0.1.0+66.zip]”
October 26th 2018 09:52:23
Info
Attempting to upload C:\Octopus\Work\20181026155217-147725-2098\staging\ReleaseInfo.Lambda.0.1.0+66.zip to bucket hg-sre with key release/packages/0.1.0+61-test2/ApprovalEmail.Lambda.zip0.1.0+66/ReleaseInfo.Lambda.0.1.0+66.zip.
October 26th 2018 09:52:23
Info
Saving object version id to variable “Octopus.Action[Upload lambda s3 zip].Output.Files[release/packages/0.1.0+61-test2/ApprovalEmail.Lambda.zip0.1.0+66/ReleaseInfo.Lambda.0.1.0+66.zip]”
October 26th 2018 09:52:23
Info
Attempting to upload C:\Octopus\Work\20181026155217-147725-2098\staging\ScheduleSlackNotification.Lambda.0.1.0+66.zip to bucket hg-sre with key release/packages/0.1.0+61-test2/ApprovalEmail.Lambda.zip0.1.0+66/ScheduleSlackNotification.Lambda.0.1.0+66.zip.
October 26th 2018 09:52:24
Info
Saving object version id to variable “Octopus.Action[Upload lambda s3 zip].Output.Files[release/packages/0.1.0+61-test2/ApprovalEmail.Lambda.zip0.1.0+66/ScheduleSlackNotification.Lambda.0.1.0+66.zip]”
October 26th 2018 09:52:24
Info
Attempting to upload C:\Octopus\Work\20181026155217-147725-2098\staging\SimpleSlackNotification.Lambda.0.1.0+66.zip to bucket hg-sre with key release/packages/0.1.0+61-test2/ApprovalEmail.Lambda.zip0.1.0+66/SimpleSlackNotification.Lambda.0.1.0+66.zip.
October 26th 2018 09:52:24
Info
Saving object version id to variable “Octopus.Action[Upload lambda s3 zip].Output.Files[release/packages/0.1.0+61-test2/ApprovalEmail.Lambda.zip0.1.0+66/SimpleSlackNotification.Lambda.0.1.0+66.zip]”
Thanks,
Mishka
Hi Mishka,
It says it matched 8 files, and appears to be trying to copy them to S3. Are they still getting corrupted or is the issue with the file locations or something like that?
The paths like 0.1.0+61-test2/ApprovalEmail.Lambda.zip0.1.0+66/ApprovalEmail.Lambda.0.1.0+66.zip, are they what you’re expecting?
Would you be able to attach some details of the package you’re using and the folder structure you are targeting in S3?
Regards
Shannon
Shannon,
Yes, it matched 8 files, but copied only one file to s3.
If I set File Pattern and Variable Substitution File Patterns both as *#{Octopus.Action.Package.PackageVersion}.zip, it copies all zip files in the package, but I can’t extract them.
If I set File Pattern as *#{Octopus.Action.Package.PackageVersion}.zip and Variable Substitution File Patterns as **, it copies only one file.
My package has multiple zip files with the same version number.
Did you try creating a package with multiple zip files with same string (version in my case) in the naming and tried to upload all to s3?
Thanks,
Mishka
Hi Mishka,
Unfortunately variable substitution won’t work on zip files, so your file pattern is correct if it’s matching 8 files, but the variable substitution pattern needs to be empty if you’re using zip files (as our variable substitution is designed to find and replace values in text files, and, as Shannon mentioned, it will likely corrupt a Zip file if it changes any content).
Based on the last logs you sent through, there’s something wrong with your file versioning which may be causing part of the issue:
Saving object version id to variable "Octopus.Action[Upload lambda s3 zip].Output.Files[release/packages/0.1.0+61-test2/ApprovalEmail.Lambda.zip0.1.0+66/SimpleSlackNotification.Lambda.0.1.0+66.zip]"
Note the ApprovalEmail.Lambda.zip0.1.0+66 is not a valid file extension. That needs to be: ApprovalEmail.Lambda.0.1.0+66.zip doesn’t it?
Also, part of the path is referencing 0.1.0+61-test2 and another part is 0.1.0+66 … shouldn’t these be the same?
Since variable substitution won’t work on zip files within an already-zipped-package, if you need variable substitution on these inner zip files, you’ll need to consider structuring this main package differently.
Hope this helps.
Cheers
Mark
Thanks Mark for your comments. I have decided to copy one zip file at a time.
I have a question and you may help me with this.