Need consulting about Octopus integration with Jenkins

[at3f1991 (ootoo)]

Hi people, I have a question!

Whenever I want to change code inside an octopus deploy project and rebuild the project, it keeps using the cached package it’s got from Jenkins and it uses cached variables for that. To solve this, I can create a release, but our senior says that it is a bad practice since the versions are going to become different from Jenkins’ package versions. He told me that whenever I want to change the variables I use for our octopus project, I’ll have to rebuild the package from Jenkins manually and then redeploy the project in octopus. And also, he says that we can not do anything about it since the dev team works on these package deployments in life cycles and for that, we can not change our way. I think that it’s not a good practice to manually rebuild in Jenkins and it should be automated done from the octopus side. What do you think?

[Michael_Richardson]

Hi!

I think there may be a couple of questions here. I’ll rephrase, just to make sure I have understood correctly.

1. When you make code changes, which are then build and packaged by Jenkins, they aren’t picked up by existing Octopus releases.

This is true, and is by design. When you create a release in Octopus, the packages are snapshotted to ensure the code you deploy to production is the same as has been deployed to previous environments.

It is possible to get code changes into an existing release. If Jenkins is building the changes as a new package version, then you can edit the release in Octopus and re-select the package version.

If Jenkins is building the changes as the same package version, you can overwrite the existing package when you push from Jenkins, by modifying the Overwrite mode. You can see this field in the Octopus plugin for Jenkins docs.

We generally recommend both these approaches be used only in exceptional circumstances, but they may be valuable to you.

2. When you change variables in Octopus, they aren’t picked up by existing releases

Again this is true and by design. For the same reason as packages, though we do understand that this restriction can be painful.

If I understand correctly, your senior is suggesting you rebuild the package in order to increment the version number, and you can then create a new release in Octopus which will include the modified variables and have the release version in-sync with the package version.

One approach I have seen others take, is to use a 3-part version for the packages (e.g. 1.5.0), and a 4-part version for the Octopus release (e.g. 1.5.0.1). This allows you to create new Octopus releases, without creating new packages, and the understanding is that release x.x.x.y deploys package x.x.x

It is possible to update the variables in an existing release, but this can be dangerous and should be used cautiously.

I hope there’s some information there that is useful. Please ask if we can clarify, or if I’ve misunderstood the question. If you are willing to explain your scenario in more detail (for example what type of variables are you typically modifying; maybe give a real example) then we may be able to offer some more specific options?

[at3f1991 (ootoo)]

Hi Michael!

First off, thank you very much for explaining all of these to me. I really appreciate your help! From what you suggested, I think I’ll have to stick to updating the variable in an existing release. I’m aware that it might be dangerous, but since the packages in Jenkins are in a 4-part version and I’m sure they will not change that this easy, I think this would be my only way to work with the octopus side without being forced to use Jenkins to do a rebuild for every minor change.
The whole goal for this project is to build and deploy azure functions without using visual studio. We are using SVN, Jenkins, and Octopus deploy as tools. We used this link’s guidance to make our zip package and then to upload it to Octopus Built-in Package Repository:
Build and Deploy Azure Functions without using Visual Studio 2017
We use Jenkins to build the .bat script in the link. We made an Azure subscription account on Octopus for the authentication and an Octopus project to use this package. We put a script in one of the steps to create a new resource group and a new function app, and then we use that to deploy our code. The thing is, my supervisor doesn’t say to make a new release every time I change the code on octopus. He wants me to rebuild the package inside the Jenkins so that it would automatically make Octopus redeploy the code inside the project, this time with the updated variables. And from my point of view, all we want to do about this project should be done on the Octopus side, and not by doing some of the jobs on Jenkins and then getting back to change the code from Octopus again. I would love to know your perspective on the best practice in this.

The variables I was talking about, were defined in the script so that we would move them to the project variable part. To give you a better image, here’s a part of the script I have in DEPLOY AN AZURE RESOURCE MANAGER TEMPLATE :

$functionAppName=“func–TestfromNewOctopus”
$location=“West Europe”
$resourceGroupName=“rg-Test”
$newFunctionAppParams = @{
ResourceType = ‘Microsoft.Web/Sites’
ResourceName = $functionAppName
Kind = ‘functionapp’
Location = $location
ResourceGroupName = $resourceGroupName
Properties = @{}
Force = $true
}
$functionApp = New-AzureRmResource @newFunctionAppParams
$functionApp

Cheers!
Atefeh

[Michael_Richardson]

Hey Atefeh,

I apologize for the delay. After reading your response, I think I better understand your scenario now.

It sounds like you are heading towards keeping the deployment-related variables in Octopus, which I would definitely recommend. Things like resource group names, locations, function names, and anything else which is a deployment concern and changes between environments is better modeled in Octopus. Because you’re right, you shouldn’t have to rebuild in order to modify these variables.

Which leaves you with the versioning problem…
If I understand correctly, you want to keep the release number in Octopus and the package version in-sync. Which is a problem, because there are two reasons you may create a new release:

1. The application source code has changed and Jenkins has rebuilt the package
2. You have modified variables in Octopus

There are a few options.

Option 1
One possible approach (the one that we use internally) is to make use of SemVer pre-release versions. For example, if the next release of your application is going to be 3.5.0.0, then every commit gets built by Jenkins as (for example)

• 3.5.0.0-ci.1
• 3.5.0.0-ci.2
• 3.5.0.0-ci.3
• …

And these can be used as the release numbers in Octopus. If you need to change a variable in Octopus, you can manually create the release in Octopus and simply increment the release version, e.g. 3.5.0.0-ci.3.4
The semver ordering rules will handle this nicely. As a side benefit, you can use channels in Octopus to use a different lifecycle for these pre-release versions which doesn’t allow them to be deployed to production.

Then only when you are happy with both the functionality and deployment of a release do you create the release version. This may be when testing is complete, for example. You would then have Jenkins build the package as 3.5.0.0. We do this by tagging the commit, as we use GitVersion.

By using pre-release versions, you may also find you don’t need a 4-part version for your packages.
For us, it also means we don’t “burn” our public release numbers on builds that don’t progress to production.

Option 2
The approach above probably requires some work on your build pipeline to implement. Another option, which again leverages semver components, is when changing variables in Octopus, and you want to create a new release manually, you could simply use the current release number with a metadata component added, for example, if the current release is 3.5.0.1 and you need to modify variables, the new release could be created as 3.5.0.1+001. The new release would contain the same package, 3.5.0.1 as the previous release, but with the updated variables.

Anyone looking at this release number would learn to understand that it contained package version 3.5.0.1

Option 3
A third option is when modifying variables in Octopus, and you want to create a new release with a version that has already been used, to simply delete the existing release.
This will allow you to create the new release manually and re-use the existing version.

I’d be interested to hear your thoughts on these options? Could they work for you?

If I haven’t explained any of those well enough (very possible), then please say

Also, another question: How often do you see think you will modify these variables in Octopus? What causes them to change?

[at3f1991 (ootoo)]

I’d be interested to hear your thoughts on these options? Could they work for you?

Hey Michael,

Thank you very very much for explaining this thoroughly. I really needed to know about your opinion since I’m still new with integrations and I was really confused about the answer that was given to me here. The reason for changing the variables was that there were difficulties with the project’s deployment and I was changing the variables to find out the problem. It was a one-time thing and there was nothing special about that. I was worried about the practice though. I liked all three options you proposed and for us, I think both semantic versioning and the added metadata seem very fair! Thank you so much for the time you took to help me out again!

Cheers!
Atefeh