Publishing build information for a Docker container

s.koshel · June 29, 2020, 9:18am

We use TeamCity, that makes builds and publishes build information to Octopus Cloud.

It works as expected for NuGet packages. But one of our projects consists only of a Docker image and I cannot make build information for it to appear in Octopus. Here are the settings:

The docker image has a version %build.number%
The docker image repository is connected to the Octopus instance as an external feed.
The “OctopusDeploy: Build Information” step goes after the docker push and before the octopus create release steps. It uses the docker image name as Package ID and %build.number% as package version.

What is the way to make it work?

jeremy.miller · June 29, 2020, 5:25pm

Hi Sergei,

Thanks for reaching out. Just to be sure it is working as expected, I was able to do a quick reproduction and was able to get build information to populate for a docker image on an external feed.

Would you be able to provide the commands you are running to push build information, as well as screenshots of:

Your external feed test, searching on the docker image you want to use

Your build information tab

A release referencing the desired docker image (showing the package section).

Could you also please provide the command you’re using to push the build info as well as the build info file?

For reference:
octo build-information --apiKey="API-" --server="[http://octopus01.home.local](http://octopus01.home.local/)" --space="Default" --file="octopus.buildinfo" --package-id="harrisonmeister/rolling-deploy-example" --version="0.0.3"

octopus.buildinfo:

{
   "BuildEnvironment":"TeamCity",
   "Branch":"workitem-tracking",
   "BuildNumber":"4",
   "BuildUrl":"http://teamcity.home.local:8081/viewLog.html?buildId=723",
   "VcsType":"Git",
   "VcsRoot":"git@github.com:OctopusSamples/rolling-deploy-web-example.git",
   "VcsCommitNumber":"30d23e27b04bfe2fa8fbc0e45a86a667d5d6089e",
   "Commits":[
      {
         "Id":"30d23e27b04bfe2fa8fbc0e45a86a667d5d6089e",
         "Comment":"Updating the readme with correct title. Resolves #1"
      }
   ]
}

Some of the above doesnt relate to each other but I wanted to give you examples of all of the info I needed. Please let me know if you have any questions.

Thanks,
Jeremy

s.koshel · June 30, 2020, 2:03pm

Thank you for your reply.

Now I probably see the reason — we don’t have the file with the build info. The “OctopusDeploy: Build information” step in TeamCity does not mention it. But, apparently, it’s required for that step, right? What is the expected location for the build info file on that step? Is there a tool or plugin for TeamCity to generate the Octopus build info file?

jeremy.miller · June 30, 2020, 2:52pm

Hi Sergei,

Teamcity actually builds this for you dynamically. My apologies for not being more clear about where to find it.

The way you will get that information is by first editing the step for build information to show verbose logging like this:

Then once you build, you will see it in the log like this:

You can see the JSON with all the relevant information once you expand the Push Build info section.

Please let me know if you have any other questions.

Thanks,
Jeremy

s.koshel · June 30, 2020, 3:40pm

Here is the octopus.bildinfo file:

And the log for the build information and create release steps:

[17:15:44] : Step 3/4: OctopusDeploy: Build Information (9s)
[17:15:45] : [Step 3/4] Creating /home/smartinmedia/BuildAgent/work/eda48fb1b14ef381/octopus.buildinfo
[17:15:45] : [Step 3/4] Serializing Octopus build information
[17:15:45] : [Step 3/4] Serialized Octopus build information - {
“BuildEnvironment”: “TeamCity”,
“Branch”: “DL-426”,
“BuildNumber”: “0.1.449-DL-426”,
“BuildUrl”: “http://…/viewLog.html?buildId=11157”,
“VcsType”: “Git”,
“VcsRoot”: “https://…/_git/Viewer”,
“VcsCommitNumber”: “52d2f6b320cf199b839fdb2391486c146f5d583e”,
“Commits”: [
{
“Id”: “52d2f6b320cf199b839fdb2391486c146f5d583e”,
“Comment”: “DL-426 Test publishing build info to Octopus\n”
}
]
}
[17:15:45] : [Step 3/4] Wrote /home/smartinmedia/BuildAgent/work/eda48fb1b14ef381/octopus.buildinfo
[17:15:45] : [Step 3/4] ForcePush: false
[17:15:45] : [Step 3/4] OverwriteMode: FailIfExists
[17:15:45] : [Step 3/4] ForcePush: false
[17:15:45] : [Step 3/4] OverwriteMode: FailIfExists
[17:15:45] : [Step 3/4] Octopus Deploy (9s)
[17:15:45] : [Octopus Deploy] Running command: dotnet Octo.dll build-information --server https://home-office.octopus.app --apikey SECRET --package-id smartinmedia.azurecr.io/patho-viewer --version 0.1.449-DL-426 --file /home/smartinmedia/BuildAgent/work/eda48fb1b14ef381/octopus.buildinfo
[17:15:45] : [Octopus Deploy] Pushing build information to Octopus server
[17:15:47] : [Octopus Deploy] Octopus Deploy Command Line Tool, version 6.17.0
[17:15:47] : [Octopus Deploy]
[17:15:48] : [Octopus Deploy] Detected automation environment: “TeamCity/2020.1.1”
[17:15:52] : [Octopus Deploy] Space name unspecified, process will run in the default space context
[17:15:52] : [Octopus Deploy] Handshaking with Octopus Server: https://home-office.octopus.app
[17:15:52] : [Octopus Deploy] Handshake successful. Octopus version: 2020.2.13; API version: 3.0.0
[17:15:53] : [Octopus Deploy] Authenticated as: TeamCity Non-Release Builds (a service account)
[17:15:53] : [Octopus Deploy] Pushing build information for package “smartinmedia.azurecr.io/patho-viewer” version “0.1.449-DL-426”…
[17:15:54] : [Octopus Deploy] Push successful
[17:15:54] : [Octopus Deploy] Octo.exe exit code: 0
[17:15:54] : Step 4/4: OctopusDeploy: Create release (9s)
[17:15:54] : [Step 4/4] Octopus Deploy (8s)
[17:15:54] : [Octopus Deploy] Running command: dotnet Octo.dll create-release --server https://home-office.octopus.app --apikey SECRET --project Viewer --enableservicemessages --version 0.1.449-DL-426 --channel Branch --package smartinmedia.azurecr.io/patho-viewer:0.1.449-DL-426
[17:15:54] : [Octopus Deploy] Creating Octopus Deploy release
[17:15:55] : [Octopus Deploy] Octopus Deploy Command Line Tool, version 6.17.0
[17:15:55] : [Octopus Deploy]
[17:15:55] : [Octopus Deploy] Detected automation environment: “TeamCity/2020.1.1”
[17:15:59] : [Octopus Deploy] Space name unspecified, process will run in the default space context
[17:15:59] : [Octopus Deploy] Handshaking with Octopus Server: https://home-office.octopus.app
[17:15:59] : [Octopus Deploy] Handshake successful. Octopus version: 2020.2.13; API version: 3.0.0
[17:16:00] : [Octopus Deploy] Authenticated as: TeamCity Non-Release Builds (a service account)
[17:16:00] : [Octopus Deploy] Found environments:
[17:16:00] : [Octopus Deploy] This Octopus Server supports channels
[17:16:01] : [Octopus Deploy] Found project: Viewer (Projects-41)
[17:16:01] : [Octopus Deploy] Building release plan for channel ‘Branch’…
[17:16:01] : [Octopus Deploy] Found channel: Branch (Channels-101)
[17:16:01] : [Octopus Deploy] Finding deployment process…
[17:16:02] : [Octopus Deploy] Finding release template…
[17:16:02] : [Octopus Deploy] Using version number provided on command-line: 0.1.449-DL-426
[17:16:02] : [Octopus Deploy] Release plan for Viewer 0.1.449-DL-426
[17:16:02] : [Octopus Deploy] Channel: ‘Branch’
[17:16:02] : [Octopus Deploy]
[17:16:02] : [Octopus Deploy] Creating release…
[17:16:03] : [Octopus Deploy] Release 0.1.449-DL-426 created successfully!
[17:16:03]i: [Octopus Deploy] ##teamcity[setParameter name=‘octo.releaseNumber’ value=‘0.1.449-DL-426’]
[17:16:03] : [Octopus Deploy] Octo.exe exit code: 0

And then there is no build info in the release:

jeremy.miller · June 30, 2020, 3:46pm

Hi Sergei,

Thanks for the information. The teamcity stuff looks okay so far.

That release says theres no packages associated with it. Is that the project where you are deploying the docker image? I believe there should be a package listed there like in my example above.

Also, when you go to build information tab in library, what do you see? Do you see an entry for smartinmedia.azurecr.io/patho-viewer ? Can you send a screenshot of your build info section of the library?

Thanks,

s.koshel · June 30, 2020, 4:01pm

I do not really get, what does it mean to deploy a docker image to a project.
In the Octopus deployment process we use several RAW KUBERNETES YAML steps to deploy the app into a K8s cluster. There is no relation from the docker image to the project, other than the YAML source code.

Here’s the screenshot from the build information tab:

When I click on the last link, the current page gets reloaded.

jeremy.miller · June 30, 2020, 5:24pm

“When I click on the last link, the current page gets reloaded.”

This part is a bug we found a few days back that I am talking to engineering about.

Where are you wanting the build information to show up, in the release page of the project, or somewhere else?

s.koshel · July 1, 2020, 9:09am

I want the build information to appear on the release page and then get promoted to Jira via the corresponding integration. The same way as we have this for other projects (with NuGet packages).

jeremy.miller · July 1, 2020, 2:24pm

Hi Sergei,

In order to get the build information to show up in the release you will need to reference the package in a step somewhere like this:

Even if its in a step that does nothing with the package, this will then add it to your release page and you should have your build information there.

Can you please try that and let me know if that fixes it for you?

Thanks,
Jeremy

s.koshel · July 1, 2020, 2:27pm

I do not see a field for package reference in the DEPLOY RAW KUBERNETES YAML step type.

We specify the package reference when we create a release:

dotnet Octo.dll create-release --server https://server --apikey SECRET --project <Proj> --enableservicemessages --version 0.1.449-DL-426 --channel Branch --package smartinmedia.azurecr.io/patho-viewer:0.1.449-DL-426

Shouldn’t Octopus use the passed package reference?

jeremy.miller · July 1, 2020, 2:29pm

Hi Sergei,

Can you make a dummy script step that does nothing but maybe Write-host “Build Info Added” and add it as a referenced package there to test?

Thanks,
Jeremy

jeremy.miller · July 1, 2020, 3:21pm

Hey Sergei,

I did some tests on my own and I see that when you reference the package in the create-release command, the build info doesnt show up, however the package does show up within my release, just the link between the build info seems to not work. I was doing my testing on 2020.2.13, which version are you currently on?

Can you please try adding the dummy script step and reference the package and manually create the release in the GUI just to see if your build information portion is working and we can dig into the other section after?

Thanks,
Jeremy

s.koshel · July 1, 2020, 4:22pm

I’m also on 2020.2.13.

I’ve added a fake step with a package reference. Now I have the package and the build info in the release. Thank you!

Another thing, that really confuses me, is that the package reference is specified in a deployment step. Does it mean, that during deployment the latest version of the package from the feed will be used? Or the package version is set for the release and this version will be used at any time when the release is deployed? If the package version is linked in the release and cannot be changed, then why a package reference is needed in a build step — wouldn’t it be more clear to use the package from the release?

jeremy.miller · July 1, 2020, 5:30pm

Hi Sergei,

I’m glad to see that it’s working at some capacity so far. Were you able to achieve these results by manually creating the release in the Octopus GUI, or were you able to get these results with the create-release command?

“Does it mean, that during deployment the latest version of the package from the feed will be used?”

The version of the package that will be used is determined by the one you select when creating the release. You are correct that once you choose a package version for a release, whenever that specific release is deployed you will use the same package version and that won’t change.

“If the package version is linked in the release and cannot be changed, then why a package reference is needed in a build step — wouldn’t it be more clear to use the package from the release?”

Could you please clarify this question a bit for me?

Thanks,
Jeremy

s.koshel · July 1, 2020, 6:10pm

Were you able to achieve these results by manually creating the release in the Octopus GUI, or were you able to get these results with the create-release command?

I achieved that via the TeamCity plugin (i.e. dotnet Octo.dll create-release).

Could you please clarify this question a bit for me?

There are multiple places for adding a package reference in a release:

In the release itself
In a build step inside the project

I have to reference the same package in both places. And they both seem mandatory. Wouldn’t it be more convenient to have only the reference in the release? Then I would not need a dummy deployment step.

jeremy.miller · July 1, 2020, 6:48pm

Hi Sergei,

In my testing when trying it with the create-release method, the package would show up, but build info would not populate until I later manually created a release. I need to do more testing on this behavior. Ideally this should work by itself, then you wont need to create the dummy step to get it in your release.

I will do some testing and get back to you. Please feel free to reach out in the mean time with any questions regarding this.

Thanks,
Jeremy

jeremy.miller · July 21, 2020, 5:12pm

Hi Sergei,

I apologize for taking so long to get back to you on this one. I did have some talks with colleagues on this one and they said that it is intended behavior for the package to not show up unless it’s also referenced inside of a step within the project. The --package paramater is only for selecting the version of the package to be used within steps where the package is referenced.

This does seem like a fringe case in which I could see the benefit of not having that requirement, but the intent is to treat docker like any other package in the feed.

Sorry I don’t have better news for you, but hopefully the dummy step is not too cumbersome for you to get build information associated with your releases.

Please reach out if you have any other questions or concerns regarding this.

Thanks,
Jeremy

s.koshel · July 21, 2020, 6:07pm

Hi Jeremy,

Thanks for the update.
A dummy step is fine. It just confused me, that there are two different places for referencing packages.

Do I understand now correctly, that a reference from the process, is, actually, a feed reference, not a package reference? And the package of the specific version is referenced only for a release?

jeremy.miller · July 21, 2020, 6:46pm

Hi Sergei,

Yeah the documentation on the --package parameter could use a little more specificity to say that it’s only for selecting a version of a given package as it can be a bit misleading.

You are correct that the reference within the process steps is just indicating which package ID(or in your case, docker image) should be used, and when creating the release you will specify the version of the package you defined in the process to be used. It’s a two part process.

Please let me know if that answered your question or if I misunderstood the question.

Thanks,
Jeremy