SSH Target unable to extract files from package

Legacy Execution
1

I am deploying to Ubuntu 16.04 using SSH target.
this VM has mono-complete on it.

tectonic@tectonicsetup:~$ mono --version
Mono JIT compiler version 5.2.0.215 (tarball Mon Aug 14 15:46:12 UTC 2017)
Copyright © 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
TLS: __thread
SIGSEGV: altstack
Notifications: epoll
Architecture: amd64
Disabled: none
Misc: softdebug
LLVM: supported, not enabled.
GC: sgen (concurrent by default)

Here is my octo version

{“Octopus.Version”:“3.7.4”,“System.OSVersion”:“Microsoft Windows NT 6.2.9200.0”,“System.ClrVersion”:“4.0.30319.42000”,“Process.WorkingSetBytes”:300494848,“Process.ThreadCount”:26,“Process.Uptime”:“12:13:57”}

package does upload the the VM but the paths are messed and I cannot read the files.
Instead of saying files name like
/home/user/.octopus/OctopusServer/WorkDir/xxxxxxx/src\s\b\file.txt

See the slashes are wrong.
I cannot read files.

Let me know how I can get past this issue.

2

I can log into Ubuntu VM and run unzip and the files are present and readable. Only the calamari/ssh is unzipping the files and messing up the paths so the files are unreadable.

3

I upgraded to latest version of Octopus and Calamari.
I first tried moonless install and it failed with an error.
I went back to use installing mono-complete and configured SSH target again.

04:50:42 Info | Script start current working directory is /home/tectonic/.octopus/OctopusServer/Work/20170830045039-22438-80
04:50:42 Info | /home/tectonic/.octopus/OctopusServer/Work/20170830045039-22438-80/command.sh
04:50:42 Info | /home/tectonic/.octopus/OctopusServer/Work/20170830045039-22438-80/README.md
04:50:42 Info | /home/tectonic/.octopus/OctopusServer/Work/20170830045039-22438-80/src\sntk8eastnp\vnet.parameters.json
04:50:42 Info | /home/tectonic/.octopus/OctopusServer/Work/20170830045039-22438-80/src\sntk8westnp\vnet.parameters.json
04:50:42 Info | /home/tectonic/.octopus/OctopusServer/Work/20170830045039-22438-80/src\vnetdeploy.json
04:50:42 Info | /home/tectonic/.octopus/OctopusServer/Work/20170830045039-22438-80/variables.json
04:50:42 Info | /home/tectonic/.octopus/OctopusServer/Work/20170830045039-22438-80/src\provisionk8.sh
04:50:42 Info | /home/tectonic/.octopus/OctopusServer/Work/20170830045039-22438-80/Configure.a52-447b-8118-c97dceb0a88a.sh
04:50:42 Info | /home/tectonic/.octopus/OctopusServer/Work/20170830045039-22438-80/Bootstrap.1dc-4598-94be-7ea2400b7a96.src\provisionk8.sh
04:50:42 Info | /home/tectonic/.octopus/OctopusServer/Work/20170830045039-22438-80/src\sntk8eastnp\terraform.tfvars
04:50:42 Info | /home/tectonic/.octopus/OctopusServer/Work/20170830045039-22438-80/src\sntk8westnp\terraform.tfvars
04:50:42 Verbose | Adding journal entry:
04:50:42 Verbose |
04:50:42 Error | cat: ./src/sntk8westnp/terraform.tfvars: No such file or directory

As you can see find command does show the files but the names are messed up. When I try to print the contents of a file I get an error saying no such error or directory

4

Hi,

Thanks for getting in touch! Sorry that you are having issues in deploying to SSH targets.

So that we can investigate what is happening we will need some more information. Firstly, can you please export and attach the full task log for the deployment?

Are you able to advise how your are attempting to deploy your package to this server, if you can confirm if it is one of the built in steps or a custom step/script that would be very much appreciated.

I look forward to resolving this issue with you,

Regards

Alex

5

I am using “Script” setup to invoke the bash script on Ubuntu 16.04 VM.

I will send over the task log shortly.

I want to make sure my email goes directly to you and does not show up in the public forum as it has sensitive information.

Rajinder Singh
Principal Cloud Specialist
Cheval Partners
Cell: 612 802 4156 | Email: rsingh@chevalpartners.commailto:rsingh@chevalpartners.com
Bloghttp://www.rajinders.com/ | LinkedInhttp://www.linkedin.com/in/singhrajinder/ | Twitterhttps://twitter.com/rajsinghper | Webhttp://www.chevalpartners.com/

6

I see that discussion is marked private.
So I have attached the task log.

Thanks for your help.

Rajinder Singh
Principal Cloud Specialist
Cheval Partners
Cell: 612 802 4156 | Email: rsingh@chevalpartners.commailto:rsingh@chevalpartners.com
Bloghttp://www.rajinders.com/ | LinkedInhttp://www.linkedin.com/in/singhrajinder/ | Twitterhttps://twitter.com/rajsinghper | Webhttp://www.chevalpartners.com/

ServerTasks-22450.log.txt (83 KB)

7

Hi Rajinder,

Thank you for sending your logs.

Would you mind providing the script (or an example) where you are attempting to print the contents of a file?

I have tried to reproduce with the following script and my files are output:

src/command.sh:

echo "$(<file.txt)"
echo "$(<../file.txt)"

I am using a script step that runs src/command.sh from a package.

Cheers,
Shane

8

I have attached the entire script.
Look for cat statements.

I am printing all the files names and that is where I see file paths seemed messed up.
I am doing a file exist check that fails.
I am doing cat file name and that is failing as well.
Only difference between your tests and mine is that in my case I am accessing a file that is in a subdirectory.

#!/bin/bash
set -e

clustername=$(get_octopusvariable “clustername”)
clusterversion=$(get_octopusvariable “clusterversion”)
ARM_SUBSCRIPTION_ID=$(get_octopusvariable “ARM_SUBSCRIPTION_ID”)
ARM_CLIENT_ID=$(get_octopusvariable “ARM_CLIENT_ID”)
ARM_CLIENT_SECRET=$(get_octopusvariable “ARM_CLIENT_SECRET”)
ARM_TENANT_ID=$(get_octopusvariable “ARM_TENANT_ID”)
tectonic_admin_password_hash=$(get_octopusvariable “tectonic_admin_password_hash”)

            echo "Script start current working directory is $PWD"

            #for entry in ./*
            #do
            #              echo "$entry"
            #done
            find $(pwd) -name \*.*

            cat ./src/$clustername/terraform.tfvars

            if [ ! -f ./src/$clustername/terraform.tfvars ]; then
                            echo "File does not exist!"
            fi

echo "clustername is $clustername, clusterversion is $clusterversion, ARM_SUBSCRIPTION_ID is $ARM_SUBSCRIPTION_ID, ARM_CLIENT_ID is $ARM_CLIENT_ID "
echo “ARM_TENANT_ID is $ARM_TENANT_ID, tectonic_admin_password_hash is $tectonic_admin_password_hash, ARM_CLIENT_SECRET is $ARM_CLIENT_SECRET”

#update the terraform.tfvars file $tectonic_admin_password_hash

if [ ! -d “/home/tectonic/$clustername” ]; then
echo "cluster $clustername does not exist"
mkdir “/home/tectonic/$clustername”

            #copy all the files from tectonic installer into the cluster directory created above
            cp -r "/home/tectonic/$clusterversion/tectonic" "/home/tectonic/$clustername"


            #cd "/home/tectonic/$clustername/tectonic"

            INSTALLER_PATH=/home/tectonic/$clustername/tectonic/tectonic-installer/linux/installer # Edit the platform name.

            sed "s|<PATH_TO_INSTALLER>|$INSTALLER_PATH|g" /home/tectonic/$clustername/tectonic/terraformrc.example > /home/tectonic/$clustername/tectonic/.terraformrc

            mkdir -p /home/tectonic/$clustername/tectonic/build/$clustername

            echo "export PATH=$PATH:/home/tectonic/$clustername/tectonic/tectonic-installer/linux" > /home/tectonic/$clustername/tectonic/env.sh
            echo "export TERRAFORM_CONFIG=/home/tectonic/$clustername/tectonic/.terraformrc" >> /home/tectonic/$clustername/tectonic/env.sh
            echo "export ARM_SUBSCRIPTION_ID=$ARM_SUBSCRIPTION_ID" >> /home/tectonic/$clustername/tectonic/env.sh
            echo "export ARM_CLIENT_ID=$ARM_CLIENT_ID" >> /home/tectonic/$clustername/tectonic/env.sh
            echo "export ARM_CLIENT_SECRET='$ARM_CLIENT_SECRET'" >> /home/tectonic/$clustername/tectonic/env.sh
            echo "export ARM_TENANT_ID=$ARM_TENANT_ID" >> /home/tectonic/$clustername/tectonic/env.sh
            echo "export CLUSTER=$clustername" >> /home/tectonic/$clustername/tectonic/env.sh

            echo "current working dir is $PWD"

            for entry in ./*
            do
                            echo "$entry"
            done

            if [ ! -f ./src/$clustername/terraform.tfvars ]; then
                            echo "File does not exist!"
            fi

            sed "s|#{tectonic_admin_password_hash}|$tectonic_admin_password_hash|g" ./src/$clustername/terraform.tfvars > /home/tectonic/$clustername/tectonic/build/$clustername/terraform.tfvars

            source /home/tectonic/$clustername/tectonic/env.sh
            cd "/home/tectonic/$clustername/tectonic"
            terraform get platforms/azure

            #cp ~/terraform.tfvars build/$clustername/terraform.tfvars

            terraform plan -var-file=/home/tectonic/$clustername/tectonic/build/$clustername/terraform.tfvars platforms/azure
            if [ $? -ne 0 ]; then
    echo "terraform plan returned error code $?" >&2
            else
                            terraform apply -var-file=/home/tectonic/$clustername/tectonic/build/$clustername/terraform.tfvars platforms/azure
                            if [ $? -ne 0 ]; then
                                            echo "terraform plan returned error code $?" >&2
                            fi
fi

else
echo “cluster $clustername already exists”

            echo "export PATH=$PATH:/home/tectonic/$clustername/tectonic/tectonic-installer/linux" > /home/tectonic/$clustername/tectonic/env.sh
            echo "export TERRAFORM_CONFIG=/home/tectonic/$clustername/tectonic/.terraformrc" >> /home/tectonic/$clustername/tectonic/env.sh
            echo "export ARM_SUBSCRIPTION_ID=$ARM_SUBSCRIPTION_ID" >> /home/tectonic/$clustername/tectonic/env.sh
            echo "export ARM_CLIENT_ID=$ARM_CLIENT_ID" >> /home/tectonic/$clustername/tectonic/env.sh
            echo "export ARM_CLIENT_SECRET='$ARM_CLIENT_SECRET'" >> /home/tectonic/$clustername/tectonic/env.sh
            echo "export ARM_TENANT_ID=$ARM_TENANT_ID" >> /home/tectonic/$clustername/tectonic/env.sh
            echo "export CLUSTER=$clustername" >> /home/tectonic/$clustername/tectonic/env.sh
            sed "s|#{tectonic_admin_password_hash}|$tectonic_admin_password_hash|g" ./src/$clustername/terraform.tfvars > /home/tectonic/$clustername/tectonic/build/$clustername/terraform.tfvars
            source /home/tectonic/$clustername/tectonic/env.sh

            cd "/home/tectonic/$clustername/tectonic"
            terraform plan -var-file=build/$clustername/terraform.tfvars platforms/azure
            terraform apply -var-file=build/$clustername/terraform.tfvars platforms/azure
            #in this case we just need to replace the existing terraform.tfvars file with the new file sent via Octopus Deploy

fi

Rajinder Singh
Principal Cloud Specialist
Cheval Partners
Cell: 612 802 4156 | Email: rsingh@chevalpartners.commailto:rsingh@chevalpartners.com
Bloghttp://www.rajinders.com/ | LinkedInhttp://www.linkedin.com/in/singhrajinder/ | Twitterhttps://twitter.com/rajsinghper | Webhttp://www.chevalpartners.com/

9

Hi Rajinder,

Thanks for your script.

One of the differences I’m noticing in your logs is the current working directory. For my script executions, the working directory is /src. I notice in your variables that the script to execute is src\provisionk8.sh whereas mine is src/command.sh (with a forward slash). I can’t actually get my script to run if i specify it as src\command.sh. I wonder if that is causing the issue?

I’ve attached the package I am using and a screenshot showing the step configuration. Are you able to successfully run the script from that package?

Cheers,
Shane

BashInSubfolder.2.0.5.zip (988 Bytes)

Run_bash_script_setup.png
Run_bash_script_setup.png773×912 21.9 KB

10

In my case the package was created by Atlasssion Bamboo file zip plugin. Bamboo was running on windows server.

I changed the slash to forward slash.
Once I did that I got this error.

Executing script on ‘10.0.0.29’
August 31st 2017 22:13:05
Warning
OctopusPrintVariables is enabled. This should only be used for debugging problems with variables, and then disabled again for normal deployments.
August 31st 2017 22:13:08
Error
Could not find script file: /home/tectonic/.octopus/OctopusServer/Work/20170901031302-22628-1935/src/provisionk8.sh

[cid:image001.png@01D322AC.BCBC4250]

I took command.sh and file.txt from your package and added it to my bitbucket repo.

I got this error
Could not find script file: /home/tectonic/.octopus/OctopusServer/Work/20170901034833-22632-1938/src/command.sh

I logged into the Linux VM and manually extracted the package using unzip command

I do see these files
-rw-rw-r-- 1 tectonic tectonic 8655 Sep 1 03:46 appgatewaydeploy.json
-rw-rw-r-- 1 tectonic tectonic 153 Sep 1 03:46 command.sh
-rw-rw-r-- 1 tectonic tectonic 15 Sep 1 03:46 file.txt
-rw-rw-r-- 1 tectonic tectonic 4888 Sep 1 03:46 provisionk8.sh
drwxrwxr-x 2 tectonic tectonic 4096 Sep 1 03:55 sntk8eastnp
drwxrwxr-x 2 tectonic tectonic 4096 Sep 1 03:55 sntk8westnp
-rw-rw-r-- 1 tectonic tectonic 2655 Sep 1 03:46 vnetdeploy.json

Rajinder Singh
Principal Cloud Specialist
Cheval Partners
Cell: 612 802 4156 | Email: rsingh@chevalpartners.commailto:rsingh@chevalpartners.com
Bloghttp://www.rajinders.com/ | LinkedInhttp://www.linkedin.com/in/singhrajinder/ | Twitterhttps://twitter.com/rajsinghper | Webhttp://www.chevalpartners.com/

image001.png
image001.png1271×1055 126 KB

11

Read my previous email first.

I was able to get the deployment to stop in the middle so I can examine the contents of the Work directory.

tectonic@tectonicsetup:~/.octopus/OctopusServer/Work/20170901040954-22635-1980$ ls -al
total 128
drwxrwxr-x 2 tectonic tectonic 4096 Sep 1 04:09 .
drwxrwxr-x 3 tectonic tectonic 4096 Sep 1 04:09 …
-rw-rw-r-- 1 tectonic tectonic 229 Sep 1 04:09 Bootstrap.931-4b85-8d54-07198accf057.src\provisionk8.sh
-rw-rw-r-- 1 tectonic tectonic 882 Sep 1 04:09 command.sh
-rw-rw-r-- 1 tectonic tectonic 22617 Sep 1 04:09 Configure.fa1-4d9c-9d6a-eaa67becd34b.sh
-rw-rw-r-- 1 tectonic tectonic 648 Sep 1 04:08 README.md
-rw-rw-r-- 1 tectonic tectonic 8655 Sep 1 04:08 src\appgatewaydeploy.json
-rw-rw-r-- 1 tectonic tectonic 153 Sep 1 04:08 src\command.sh
-rw-rw-r-- 1 tectonic tectonic 15 Sep 1 04:08 src\file.txt
-rw-rw-r-- 1 tectonic tectonic 4890 Sep 1 04:09 src\provisionk8.sh
-rw-rw-r-- 1 tectonic tectonic 13966 Sep 1 04:08 src\sntk8eastnp\terraform.tfvars
-rw-rw-r-- 1 tectonic tectonic 808 Sep 1 04:08 src\sntk8eastnp\vnet.parameters.json
-rw-rw-r-- 1 tectonic tectonic 13990 Sep 1 04:08 src\sntk8westnp\terraform.tfvars
-rw-rw-r-- 1 tectonic tectonic 803 Sep 1 04:08 src\sntk8westnp\vnet.parameters.json
-rw-rw-r-- 1 tectonic tectonic 2655 Sep 1 04:08 src\vnetdeploy.json
-rw-rw-r-- 1 tectonic tectonic 8614 Sep 1 04:09 variables.json

See the problem is that when calamari extracts the package it does not create directories. Instead of creating src directory with a file called command.sh it created a file called src\command.sh which is not valid.
If I use src\command.sh the script executes but creates invalid files.

If I use src/command.sh Octo cannot even find the command.sh file.

Rajinder Singh
Principal Cloud Specialist
Cheval Partners
Cell: 612 802 4156 | Email: rsingh@chevalpartners.commailto:rsingh@chevalpartners.com
Bloghttp://www.rajinders.com/ | LinkedInhttp://www.linkedin.com/in/singhrajinder/ | Twitterhttps://twitter.com/rajsinghper | Webhttp://www.chevalpartners.com/

From: Rajinder Singh rsingh@chevalpartners.com
Date: Thursday, August 31, 2017 at 10:59 PM
To: Shane Gill tender2+dd9e92b3ba@tenderapp.com
Subject: Re: SSH Target unable to extract files from package [Problems #57421]

In my case the package was created by Atlasssion Bamboo file zip plugin. Bamboo was running on windows server.

I changed the slash to forward slash.
Once I did that I got this error.

Executing script on ‘10.0.0.29’
August 31st 2017 22:13:05
Warning
OctopusPrintVariables is enabled. This should only be used for debugging problems with variables, and then disabled again for normal deployments.
August 31st 2017 22:13:08
Error
Could not find script file: /home/tectonic/.octopus/OctopusServer/Work/20170901031302-22628-1935/src/provisionk8.sh

[cid:image001.png@01D322AF.A95BC020]

I took command.sh and file.txt from your package and added it to my bitbucket repo.

I got this error
Could not find script file: /home/tectonic/.octopus/OctopusServer/Work/20170901034833-22632-1938/src/command.sh

I logged into the Linux VM and manually extracted the package using unzip command

I do see these files
-rw-rw-r-- 1 tectonic tectonic 8655 Sep 1 03:46 appgatewaydeploy.json
-rw-rw-r-- 1 tectonic tectonic 153 Sep 1 03:46 command.sh
-rw-rw-r-- 1 tectonic tectonic 15 Sep 1 03:46 file.txt
-rw-rw-r-- 1 tectonic tectonic 4888 Sep 1 03:46 provisionk8.sh
drwxrwxr-x 2 tectonic tectonic 4096 Sep 1 03:55 sntk8eastnp
drwxrwxr-x 2 tectonic tectonic 4096 Sep 1 03:55 sntk8westnp
-rw-rw-r-- 1 tectonic tectonic 2655 Sep 1 03:46 vnetdeploy.json

Rajinder Singh
Principal Cloud Specialist
Cheval Partners
Cell: 612 802 4156 | Email: rsingh@chevalpartners.commailto:rsingh@chevalpartners.com
Bloghttp://www.rajinders.com/ | LinkedInhttp://www.linkedin.com/in/singhrajinder/ | Twitterhttps://twitter.com/rajsinghper | Webhttp://www.chevalpartners.com/

image001.png
image001.png1271×1055 171 KB

12

Hi Rajinder,

Thanks for the clarification, I didn’t realise that the contents of your package was being extracted as files with slashes in the names rather than into the correct directories.

Does the package that I sent you extract correctly? Trying to understand if it something about how the package is being assembled that is causing the problem or something about your environment.

I will continue to look into it and let you know what I find.

Cheers,
Shane

13

We were able to fix this issue by building the package in Linux bamboo instead of windows bamboo server.

So we are fine now.

Thanks

Get Outlook for iOShttps://aka.ms/o0ukef

14

Hi Rajinder,

I am glad to hear that you have figured out a workaround.

Would it be possible to get the package that was not working so we can investigate further and fix the issue?

Thanks
Shane