Octopus+REST+Powershell+JSON = 'Malformed' JSON

Legacy Modelling
1

I just posted this, but when I hit create it said “awaiting moderation” and now I cannot find any evidence of my post…if this is a double post, please delete one of them.

When using the octopus REST powershell script to update a variable, powershell is:

  • Outputt JSON that octopus doesn’t support -> HTTP 500
  • Unable to ‘ingest’ JSON from octopus web client -> locks powershell

Reference script we are using

Here’s what we do:

  • Use the script to ‘ingest’ the variables from octopus
  • this turns it into a powershell custom object
  • modify the variable value
  • convertto-json
  • PUT the JSON
  • Result? Octopus HTTP-500

If you:

  • use the octopus webclient to update that same variable
  • run f12 developer tools in chrome, for e.g.
  • Get the POST body the webclient uses, which is what should be the exact same JSON
  • Compare them, they are different

Attached is the difference.

Notice powershell doesn’t configure the variable inside an array?

If you “convertfrom-json” the octopus working JSON, it locks powershell.

So in summary:

  • Powershell outputs the JSON that octopus won’t accept -> Octopus HTTP-500
  • Octopus outputs JSON that powershell won’t accept -> powershell locks.

Attached is what I’m talking about, and the script we’re using.

I’m going to keep mucking around and see if I can force an array, I think it’s the way I’m doing something but you never know.

Also, both forms of JSON adhere to RFC-4627.

Thanks!

octopus-pics-and-script.zip (86 KB)

2

Hi Wez,

There may have been an error with your original post to do with the attachment you included. We think it is an issue with the forum software we are using and that it could be related to the file type of your attachment. Sorry about that!

With regards to your issue, I think you have encountered one of the interesting quirks of Powershell :slight_smile:

Confusingly, doing round-trip JSON conversion of an array won’t necessarily preserve the original JSON string. The problem seems to originate from the unexpected serialization behaviour of the PSMemberSet type, which is what gets returned when you use ConvertFrom-Json on an array.
Have a look at this post, which discusses the issue in a bit more detail: https://stackoverflow.com/questions/20848507/why-does-powershell-give-different-result-in-one-liner-than-two-liner-when-conve

One simple solution is to serialize (ConvertTo-Json) the SyncRoot object on PsMemberSet, instead of the whole PSMemberSet object. In the script you provided, this means replacing this line:

$variableset.Variables = $newvars

with this one

$variableset.Variables = $newvars.SyncRoot

Another option would be to manipulate the $variableSet.Variables directly, instead of doing extra round-trip serialization of this collection with the $newvars variable. This should bypass any JSON conversion problems entirely.

Let me know if you have any further problems.
Tom

3

Hey Tom,

Thanks for the reply.

Going back through the depths of my memory to last Friday…ha.

The reason I was doing the double-hop conversion is due to my lack of understanding with powershell. If I simply do the invoke-restmethod, that’s gets the data fine…but I can’t modify it (see attached).

However, if I converto-json then convertfrom-json, I can then edit the value.

It’s weird because I can perform a subtraction the value, but can’t assign it something else.

Once I figure that out, the rest should work fine as you said. Cheers.

octo-value-not-work.PNG
octo-value-not-work.PNG737×383 63.9 KB

4

Double post

octo-value-not-work.PNG
octo-value-not-work.PNG737×383 63.9 KB

5

Hey Tom,

Got it :slight_smile: coffee fixes all problems.

What I noticed was that powershell was doing funny things with type, and the type for the variable data was ‘pscustomobject’, which powershell was funnily either interpreting as a single string, or as an actual custom object, depending on how it felt at the time.

$newvars | foreach-object {write-host “hello” $_}

hello {Id=c9464434-0da8-4fe3-7389-34dbe1e5fe1e; Name=KeepTempenv; Value=123123; Scope=; IsSensitive=False; IsEditable=False; Prompt=}

But then if you did some variable assignment, powershell would recognize the individual fields…so this showed it sort of half-worked. From there, I figured if I could convert to an array, that might work.

This worked perfectly (attached).

Once done, the variable updated fine in octo.

Thanks for that, yet again I learned that powershell customobjects are a super big pain in the proverbial to deal with. :).

convert-to-array.PNG

6

Hi Wez,

I’m glad you worked it out! Let me know if you have any other problems :slight_smile:

Tom