Dynamic EC2 Tentacle from Web Portal

Hi, I was wondering if someone could point me on the right track.

We have an in-house script that allows QA to deploy any build to a fresh Amazon EC2 environment (based off some AMI). I’ve been tasked with finding some way to replicate that functionality, but within the Octopus web portal (they don’t want to maintain extra scripts for QA to run). Additionally, QA needs to be able to change the variables for a potential release.

Can someone help me sketch out the rough workflow? I was thinking something along the lines of:

Have a step template that calls a script that:

Am I on the right track / missing anything?

Hi Jamie,

I hope I understand what you’re trying to achieve sufficiently to be of some help.

Octopus doesn’t currently support dynamically provisioning targets as part of deploying a release (this is something we’re exploring for the future).

So unfortunately you aren’t going to be able to do this as a step-template.

You will need a script that:

  • creates the EC2 tentacle (using the post you linked to as a guide)
  • invokes the Octopus API to register the target (possibly creating the environment as well), giving it the appropriate roles.

Once that is done, managing variables and deploying releases to your environment is Octopus’ bread and butter.

The question is where/how you invoke your script to provision and register the EC2 Tentacle. You could host this within Octopus, by creating a separate project for this purpose. This project would have one or more PowerShell script steps, responsible for performing the provisioning. Alternatively, you could invoke this from your CI server, or anywhere else appropriate. Whatever works best for you.

Another issue then is de-provisioning. Presumably these EC2 instances are relatively short-lived. You will probably want to remove these from Octopus at some point too (this can also be automated via the API).

Possibly this CodeProject article may have some useful information.

I hope this is some help. We’re more than happy to provide whatever assistance we can. You know where to find us.

Regards,
Michael

I guess where I’m stuck is figuring out the best way to allow QA to provision their own machines and tweak the variables as necessary for their particular releases without affecting any other QA person.

It seems like I’d need to run octo.exe with --specificmachines instead of the web portal so the release doesn’t target some other QA machines?

My task is to find a way to do it all in one spot (i.e. the web portal), but if it turns out I can’t then I can go back and say we need to bite the bullet and have some script(s) instead.

I believe you can achieve putting this all into Octopus. Because ultimately, if you can do it via a script, then you can invoke that script via Octopus.
Whether that is the best solution for you, I can’t answer.

How it could work:

Create a project which will be responsible for provisioning your environments, e.g. “Provision EC2 Test Env”.
This project will contain whatever variables are required to create the EC2 environment. This project will have a script step, responsible for invoking the setup script which will perform the EC2 provisioning, and invoke the Octopus API (to create the new machine, etc).

When QA wants a new environment, they modify the required variables, and create a release of the “Provision EC2 Test Env” project, and deploy it. Once that is complete, the EC2 components should be available and registered with Octopus.

As an aside, you can deploy to specific machines via the web portal (click ‘Advanced’ on the deploy release page). But if you are never going to deploy to your EC2 machines as a group, then there probably isn’t any value in putting them in the same Octopus environment. I would consider instead creating an individual Octopus environment for each. You can do this via the API in the setup script mentioned above.

At this point, QA can go to your actual project in Octopus, e.g. “Acme Online”, and deploy the latest release to your newly created environment\machine.

I guess the key concept here is the separation between the provisioning and the deployment. Whether you use a project in Octopus to perform the provisioning, or do it externally (e.g. from TeamCity) comes down to what will work best for you.

I hope this helps.

Regards,
Michael