Tentacle not Extracting full Package on one Environment

sunil.bindra · June 16, 2020, 8:21am

Hi
I am facing very weird issue with tentacle on one of my environment. I am deploying a package on different Environment. It extracts all teh files and folder on 2 of my environment but on my third environment its not extracting whole package. many files and few folders missing. I have 20+ other applications, whcih are extracting all files on same machine. Dont knwo why its not working for just 2 Apps. All apps are service Fabric Apps. Dont think so its releted to SF apps. Its something with in teh extraction for that particulat package on thsi machine. Even I tried to copy teh pacakage from teh machine on my local and extract it using 7Zip. It extracts all file. here is the log.

Task ID: ServerTasks-121044
Task status: Failed
Task queued: 16 June 2020 08:32:16
Task started: 16 June 2020 08:32:17
Task completed: 16 June 2020 08:32:26
Task duration: 9 seconds
Server version: 2018.9.7+Branch.master.Sha.bc1aa5ccec678935363ecf0dc4c018f3a4bafb82
Server node: BBAMOCTAPPPRD01

                | == Failed: Deploy Messaging.CharlesRiver.CrBroadcastTradeStatusService release 20200616.1.1 to PreProd ==

08:32:17 Verbose | 08:32:17 Verbose | 08:32:17 Verbose | 08:32:17 Verbose | 08:32:17 Verbose | 08:32:17 Verbose | 08:32:17 Verbose | 08:32:26 Fatal |
| == Success: Acquire packages ==
08:32:17 Info 08:32:17 Info 08:32:17 Verbose | 08:32:17 Verbose | 08:32:17 Verbose | 08:32:17 Verbose | 08:32:17 Info 08:32:17 Verbose | 08:32:22 Info 08:32:22 Verbose | |
| Success: |
| Success: 08:32:18 Verbose | 08:32:18 Verbose | 08:32:18 Verbose | 08:32:18 Verbose | 08:32:18 Verbose | 08:32:18 Verbose | 08:32:18 Verbose | 08:32:18 Verbose | 08:32:18 Verbose | 08:32:18 Verbose | 08:32:18 Verbose | 08:32:18 Verbose | 08:32:18 Verbose | 08:32:18 Verbose | 08:32:19 Verbose | 08:32:19 Verbose | 08:32:19 Verbose | 08:32:19 Verbose | 08:32:19 Verbose | 08:32:19 Verbose | 08:32:20 Verbose | 08:32:20 Verbose | 08:32:20 Verbose | 08:32:20 Verbose | 08:32:20 Verbose | 08:32:20 Verbose | 08:32:20 Verbose | 08:32:20 Verbose | 08:32:20 Verbose | 08:32:20 Verbose | 08:32:20 Verbose | 08:32:20 Verbose | 08:32:20 Verbose | 08:32:20 Verbose | 08:32:20 Verbose | 08:32:20 Verbose | 08:32:20 Verbose | 08:32:20 Verbose | 08:32:21 Info | 08:32:21 Info | 08:32:21 Info | 08:32:21 Verbose | 08:32:21 Verbose | 08:32:21 Info | 08:32:21 Info | 08:32:21 Verbose | 08:32:22 Verbose | 08:32:22 Verbose | 08:32:22 Verbose | 08:32:22 Verbose | 08:32:22 Verbose | 08:32:22 Verbose | 08:32:22 Verbose | 08:32:22 Verbose | 08:32:22 Verbose | 08:32:22 Verbose | 08:32:22 Verbose | 08:32:22 Info | 08:32:22 Info | 08:32:22 Info | 08:32:22 Info | 08:32:22 Info | 08:32:22 Info | 08:32:22 Info | 08:32:22 Info | 08:32:22 Info | 08:32:22 Verbose | 08:32:22 Info | |
| == Failed: 08:32:25 Fatal | 08:32:25 Verbose | |
| Failed: 08:32:23 Verbose | 08:32:23 Verbose | | OperatingSystem: | OsBitVersion: x64
| Is64BitProcess: True
| CurrentUser: | MachineName: | ProcessorCount: 2
| CurrentDirectory: | TempDirectory: | HostProcessName: | PID: 2144
08:32:23 Verbose | 08:32:23 Verbose | 08:32:23 Verbose | 08:32:24 Verbose | 08:32:24 Verbose | 08:32:24 Verbose | 08:32:24 Verbose | 08:32:24 Verbose | 08:32:24 Verbose | 08:32:24 Verbose | 08:32:24 Verbose | 08:32:24 Verbose | 08:32:24 Verbose | 08:32:24 Verbose | 08:32:24 Info | 08:32:24 Verbose | 08:32:24 Error | 08:32:24 Error | 08:32:24 Error | 08:32:24 Error | 08:32:24 Error | 08:32:24 Error | 08:32:24 Error | 08:32:24 Error | 08:32:24 Error | 08:32:24 Error | 08:32:24 Verbose | 08:32:24 Verbose |
08:32:24 Verbose |
08:32:24 Verbose |
08:32:25 Error | 08:32:25 Error | 08:32:25 Error | 08:32:25 Error | 08:32:25 Error | 08:32:25 Error | 08:32:25 Error | 08:32:25 Error | 08:32:25 Error | 08:32:25 Error | 08:32:25 Error | 08:32:25 Error | 08:32:25 Verbose | 08:32:25 Verbose | 08:32:25 Verbose | 08:32:25 Fatal | 08:32:25 Fatal | | Step 1: Deployment Start Email does not apply to the current environment, and will not be executed
Step 2: Microsoft Teams - Post start message does not apply to the current environment, and will not be executed
Step 4: Deployment Successful Email does not apply to the current environment, and will not be executed
Step 5: Microsoft Teams - Post success message does not apply to the current environment, and will not be executed
Step 6: Deployment Failed Email does not apply to the current environment, and will not be executed
Step 7: Microsoft Teams - Post failure message does not apply to the current environment, and will not be executed
Guided failure is not enabled for this task
| The deployment failed because one or more steps failed. Please see the deployment log for details.
| Acquiring packages
| Making a list of packages to acquire
No packages are required on the Octopus Server
Delta compression is enabled for package transfers from the Octopus Server to deployment targets
Upload Messaging.CharlesRiver.CrBroadcastTradeStatusService.20200616.1 to PP_SFAPPPRD01_20200615
Checking package cache for package Messaging.CharlesRiver.CrBroadcastTradeStatusService v20200616.1
| Package Messaging.CharlesRiver.CrBroadcastTradeStatusService v20200616.1 was found in cache. No need to download.
Using file: C:\Octopus\Packages\Messaging.CharlesRiver.CrBroadcastTradeStatusService\Messaging.CharlesRiver.CrBroadcastTradeStatusService.20200616.1.nupkg
| All packages have been acquired
Acquire Packages completed
PP_SFAPPPRD01_20200615
Upload package Messaging.CharlesRiver.CrBroadcastTradeStatusService v20200616.1
Starting C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe in working directory ‘C:\Scripts\OctopusTentacle\Work\20200616073216-121044-15’ using ‘Western European (DOS)’ encoding running as ‘BLUEBAY\gMSA_SF_PRD$’ with the same environment variables as the launching process
Octopus Deploy: Calamari version 4.9.18
Environment Information:
OperatingSystem: Microsoft Windows NT 10.0.14393.0
OsBitVersion: x64
Is64BitProcess: True
CurrentUser: BLUEBAY\gMSA_SF_PRD$
MachineName: BBAMSFAPPPRD01
ProcessorCount: 2
CurrentDirectory: C:\Scripts\OctopusTentacle\Work\20200616073216-121044-15
TempDirectory: C:\Users\gMSA_SF_PRD$\AppData\Local\Temp
HostProcessName: Calamari
Package Messaging.CharlesRiver.CrBroadcastTradeStatusService version 20200616.1 hash ca3df3b81157e0726a0388a882e933b3f771679c has not been uploaded.
Finding earlier packages that have been uploaded to this Tentacle.
Found 3 earlier versions of Messaging.CharlesRiver.CrBroadcastTradeStatusService on this Tentacle
- 20200615.3: C:\Scripts\OctopusTentacle\Files\Messaging.CharlesRiver.CrBroadcastTradeStatusService@S20200615.3@4D34BD3C8E3E214BAD3089BFBD83F1D7.nupkg
- 20200615.2: C:\Scripts\OctopusTentacle\Files\Messaging.CharlesRiver.CrBroadcastTradeStatusService@S20200615.2@B7B9957E2F07DD43A6BEF0AF8AA0BF23.nupkg
- 20200615.1: C:\Scripts\OctopusTentacle\Files\Messaging.CharlesRiver.CrBroadcastTradeStatusService@S20200615.1@257FDC8E3E0B19468BA426AA3279A818.nupkg
Process C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe in C:\Scripts\OctopusTentacle\Work\20200616073216-121044-15 exited with code 0
Starting C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe in working directory ‘C:\Scripts\OctopusTentacle\Work\20200616073217-121044-16’ using ‘Western European (DOS)’ encoding running as ‘BLUEBAY\gMSA_SF_PRD$’ with the same environment variables as the launching process
Octopus Deploy: Calamari version 4.9.18
Environment Information:
OperatingSystem: Microsoft Windows NT 10.0.14393.0
OsBitVersion: x64
Is64BitProcess: True
CurrentUser: BLUEBAY\gMSA_SF_PRD$
MachineName: BBAMSFAPPPRD01
ProcessorCount: 2
CurrentDirectory: C:\Scripts\OctopusTentacle\Work\20200616073217-121044-16
TempDirectory: C:\Users\gMSA_SF_PRD$\AppData\Local\Temp
HostProcessName: Calamari
Package Messaging.CharlesRiver.CrBroadcastTradeStatusService version 20200616.1 hash ca3df3b81157e0726a0388a882e933b3f771679c has not been uploaded.
Finding earlier packages that have been uploaded to this Tentacle.
Found 3 earlier versions of Messaging.CharlesRiver.CrBroadcastTradeStatusService on this Tentacle
- 20200615.3: C:\Scripts\OctopusTentacle\Files\Messaging.CharlesRiver.CrBroadcastTradeStatusService@S20200615.3@4D34BD3C8E3E214BAD3089BFBD83F1D7.nupkg
- 20200615.2: C:\Scripts\OctopusTentacle\Files\Messaging.CharlesRiver.CrBroadcastTradeStatusService@S20200615.2@B7B9957E2F07DD43A6BEF0AF8AA0BF23.nupkg
- 20200615.1: C:\Scripts\OctopusTentacle\Files\Messaging.CharlesRiver.CrBroadcastTradeStatusService@S20200615.1@257FDC8E3E0B19468BA426AA3279A818.nupkg
Process C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe in C:\Scripts\OctopusTentacle\Work\20200616073217-121044-16 exited with code 0
Found matching version 20200615.3: C:\Octopus\Packages\Messaging.CharlesRiver.CrBroadcastTradeStatusService\Messaging.CharlesRiver.CrBroadcastTradeStatusService.20200615.3.nupkg
Building delta for Messaging.CharlesRiver.CrBroadcastTradeStatusService v20200616.1 (34.101 MB)
Using package C:\Octopus\Packages\Messaging.CharlesRiver.CrBroadcastTradeStatusService\Messaging.CharlesRiver.CrBroadcastTradeStatusService.20200615.3.nupkg with hash 1bb3f743f5d300ef2e159fd2c1063257f9a9f694 for creating delta.
Signature file Messaging.CharlesRiver.CrBroadcastTradeStatusService.20200615.3.nupkg.octosig already exists, using C:\Octopus\OctopusServer\PackageCache\Messaging.CharlesRiver.CrBroadcastTradeStatusService.20200615.3.nupkg.octosig
Delta file Messaging.CharlesRiver.CrBroadcastTradeStatusService.20200615.3_1BB3F743_to_20200616.1_CA3DF3B8.octodelta already exists, using file C:\Octopus\OctopusServer\PackageCache\Messaging.CharlesRiver.CrBroadcastTradeStatusService.20200615.3_1BB3F743_to_20200616.1_CA3DF3B8.octodelta
Original package was 34.101 MB, delta file is 728.664 KB (97.91% size reduction).
Uploading and applying delta Messaging.CharlesRiver.CrBroadcastTradeStatusService.20200615.3_1BB3F743_to_20200616.1_CA3DF3B8.octodelta
Starting C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe in working directory ‘C:\Scripts\OctopusTentacle\Work\20200616073219-121044-17’ using ‘Western European (DOS)’ encoding running as ‘BLUEBAY\gMSA_SF_PRD$’ with the same environment variables as the launching process
Octopus Deploy: Calamari version 4.9.18
Environment Information:
OperatingSystem: Microsoft Windows NT 10.0.14393.0
OsBitVersion: x64
Is64BitProcess: True
CurrentUser: BLUEBAY\gMSA_SF_PRD$
MachineName: BBAMSFAPPPRD01
ProcessorCount: 2
CurrentDirectory: C:\Scripts\OctopusTentacle\Work\20200616073219-121044-17
TempDirectory: C:\Users\gMSA_SF_PRD$\AppData\Local\Temp
HostProcessName: Calamari
Applying delta to C:\Scripts\OctopusTentacle\Files\Messaging.CharlesRiver.CrBroadcastTradeStatusService@S20200615.3@4D34BD3C8E3E214BAD3089BFBD83F1D7.nupkg with hash 1bb3f743f5d300ef2e159fd2c1063257f9a9f694 and storing as C:\Scripts\OctopusTentacle\Files\Messaging.CharlesRiver.CrBroadcastTradeStatusService@S20200616.1@3A995031EAE3EB4DAE5BBEE61FBD676A.nupkg
Applying delta: 0%
Applying delta: 10%
Applying delta: 40%
Applying delta: 50%
Applying delta: 60%
Applying delta: 70%
Applying delta: 80%
Applying delta: 90%
Process C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe in C:\Scripts\OctopusTentacle\Work\20200616073219-121044-17 exited with code 0
Delta for package Messaging.CharlesRiver.CrBroadcastTradeStatusService v20200616.1 successfully uploaded and applied.
Step 3: Deploy Package ==
The step failed: Activity Deploy Package on PP_SFAPPPRD01_20200615 failed with error ‘The remote script failed with exit code 100’.
Deploy Package completed
PP_SFAPPPRD01_20200615
Octopus Server version: 2018.9.7+Branch.master.Sha.bc1aa5ccec678935363ecf0dc4c018f3a4bafb82
Environment Information:
Microsoft Windows NT 6.3.9600.0
BLUEBAY\svc_OctopusDeployPRD
BBAMOCTAPPPRD01
C:\Windows\system32
C:\Users\svc_OctopusDeployPRD\AppData\Local\Temp
Octopus.Server
Executing Deploy Package (type Deploy a Package) on PP_SFAPPPRD01_20200615
Using Calamari 4.9.18
Starting C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe in working directory ‘C:\Scripts\OctopusTentacle\Work\20200616073221-121044-18’ using ‘Western European (DOS)’ encoding running as ‘BLUEBAY\gMSA_SF_PRD$’ with the same environment variables as the launching process
Octopus Deploy: Calamari version 4.9.18
Environment Information:
OperatingSystem: Microsoft Windows NT 10.0.14393.0
OsBitVersion: x64
Is64BitProcess: True
CurrentUser: BLUEBAY\gMSA_SF_PRD$
MachineName: BBAMSFAPPPRD01
ProcessorCount: 2
CurrentDirectory: C:\Scripts\OctopusTentacle\Work\20200616073221-121044-18
TempDirectory: C:\Users\gMSA_SF_PRD$\AppData\Local\Temp
HostProcessName: Calamari
Deploying package: C:\Scripts\OctopusTentacle\Files\Messaging.CharlesRiver.CrBroadcastTradeStatusService@S20200616.1@3A995031EAE3EB4DAE5BBEE61FBD676A.nupkg
Extracting package to: C:\Scripts\OctopusTentacle\Applications\PreProd\Messaging.CharlesRiver.CrBroadcastTradeStatusService\20200616.1
System.IO.DirectoryNotFoundException: Could not find a part of the path ‘C:\Scripts\OctopusTentacle\Applications\PreProd\Messaging.CharlesRiver.CrBroadcastTradeStatusService\20200616.1\BlueBay.Messaging.CharlesRiver.CrBroadcastTradeStatusServicePkg\Code\BlueBay.Messaging.CharlesRiver.CrBroadcastTradeStatusService.runtimeconfig.json’.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream…ctor(String path, FileMode mode, FileAccess access, FileShare share)
at SharpCompress.Archives.IArchiveEntryExtensions.WriteToFile(IArchiveEntry entry, String destinationFileName, ExtractionOptions options)
at Calamari.Integration.Packages.NuGet.NupkgExtractor.Extract(String packageFile, String directory, Boolean suppressNestedScriptWarning)
at Calamari.Deployment.Conventions.ExtractPackageConvention.Install(RunningDeployment deployment)
at Calamari.Deployment.ConventionProcessor.RunInstallConventions()
at Calamari.Deployment.ConventionProcessor.RunConventions()
Running rollback conventions…
Adding journal entry:
Could not find a part of the path ‘C:\Scripts\OctopusTentacle\Applications\PreProd\Messaging.CharlesRiver.CrBroadcastTradeStatusService\20200616.1\BlueBay.Messaging.CharlesRiver.CrBroadcastTradeStatusServicePkg\Code\BlueBay.Messaging.CharlesRiver.CrBroadcastTradeStatusService.runtimeconfig.json’.
System.IO.DirectoryNotFoundException
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream…ctor(String path, FileMode mode, FileAccess access, FileShare share)
at SharpCompress.Archives.IArchiveEntryExtensions.WriteToFile(IArchiveEntry entry, String destinationFileName, ExtractionOptions options)
at Calamari.Integration.Packages.NuGet.NupkgExtractor.Extract(String packageFile, String directory, Boolean suppressNestedScriptWarning)
at Calamari.Deployment.Conventions.ExtractPackageConvention.Install(RunningDeployment deployment)
at Calamari.Deployment.ConventionProcessor.RunInstallConventions()
at Calamari.Deployment.ConventionProcessor.RunConventions()
at Calamari.Commands.DeployPackageCommand.Execute(String[] commandLineArguments)
at Calamari.Program.Execute(String[] args)
Process C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe in C:\Scripts\OctopusTentacle\Work\20200616073221-121044-18 exited with code 100
Updating manifest with output variables
Updating manifest with action evaluated variables
The remote script failed with exit code 100
The action Deploy Package on PP_SFAPPPRD01_20200615 failed

Thanks

sunil.bindra · June 16, 2020, 12:15pm

Tried Extracting the Pacakge on Machine with 7Zip. It works fine. Extracting all the files. Just failing while Extracting with Octopus Deploy Task

paul.calvert · June 16, 2020, 12:34pm

Hi @sunil.bindra,

Thanks for getting in touch!

I just want to confirm I understand the situation here.
These packages extract on other machines fine?
And other packages extract on these two machines fine?
The issue only occurs with these two specific packages on two specific machines?

The first thing to check would be whether there is any anti-virus running on these machines, if there is, scanning the logs to see if it is placing these files in quarantine.

You could also check the windows event log for similar events.

It would also be worth watching the extraction folder during a deployment to see if these files are extracted and then removed or not.

If this is currently a significant blocker for you, you could use a Transfer a Package and Script step to copy the package and manually extract the contents.

Regards,
Paul

sunil.bindra · June 16, 2020, 1:02pm

Yes Paul

these packages Extracted on Other machines perfectly fine.
Also I have similar type of Packages (Service fabric) around 20 which works fine on this machine as well.
Only 2 projects failing on this machine.
I tried Extracting these packages on teh machine its failing with 7-zip. They are OK.
Nothing in the Event Log related to this.
Cant even see the folder extracted and removed. Just try this.

Thanks

paul.calvert · June 17, 2020, 7:06am

It would be worth running procmon and filter it to monitor the extraction folder. If you then run the deployment this may capture whether the files are being created and what is then happening with them.

system · July 18, 2020, 12:34pm

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.