Deployment failing due to DTD being prohibited

Legacy Uncategorised
1

As of the recent upgrade to 3.0, we are now getting this error whenever we do our deployment. It happens when we try to do the configuration replacements it would seem.

Deploying package ‘E:_Ops\Octopus\Files\MemberConnect.3.2.0.40.nupkg-1a96cd67-9c1d-4a45-aa48-102d21c30d42’ to machine 'https://qa-entapp1:10933/'
14:34:50Info
Deploying package: E:_Ops\Octopus\Files\MemberConnect.3.2.0.40.nupkg-1a96cd67-9c1d-4a45-aa48-102d21c30d42
14:34:50Info
Using variables from: E:_Ops\Octopus\Work\20150724213448-44\Variables.json
14:34:50Info
Updating appSettings and connectionStrings in: E:\Octopus\QA\MemberConnect\3.2.0.40_3\Web.config
14:34:50Error
System.Xml.XmlException: For security reasons DTD is prohibited in this XML document. To enable DTD processing set the DtdProcessing property on XmlReaderSettings to Parse and pass the settings into XmlReader.Create method.
14:34:50Error
at System.Xml.XmlTextReaderImpl.Throw(Exception e)
14:34:50Error
at System.Xml.XmlTextReaderImpl.ParseDoctypeDecl()
14:34:50Error
at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
14:34:50Error
at System.Xml.Linq.XContainer.ReadContentFrom(XmlReader r)
14:34:50Error
at System.Xml.Linq.XContainer.ReadContentFrom(XmlReader r, LoadOptions o)
14:34:50Error
at System.Xml.Linq.XDocument.Load(XmlReader reader, LoadOptions options)
14:34:50Error
at Calamari.Integration.ConfigurationVariables.ConfigurationVariablesReplacer.ModifyConfigurationFile(String configurationFilePath, VariableDictionary variables) in y:\work\b1fd300731d5f2fc\source\Calamari\Integration\ConfigurationVariables\ConfigurationVariablesReplacer.cs:line 19
14:34:50Error
at Calamari.Deployment.Conventions.ConfigurationVariablesConvention.Install(RunningDeployment deployment) in y:\work\b1fd300731d5f2fc\source\Calamari\Deployment\Conventions\ConfigurationVariablesConvention.cs:line 37
14:34:50Error
at Calamari.Deployment.ConventionProcessor.RunInstallConventions() in y:\work\b1fd300731d5f2fc\source\Calamari\Deployment\ConventionProcessor.cs:line 60
14:34:50Error
at Calamari.Deployment.ConventionProcessor.RunConventions() in y:\work\b1fd300731d5f2fc\source\Calamari\Deployment\ConventionProcessor.cs:line 28
14:34:50Error
Running rollback conventions…
14:34:50Error
System.Xml.XmlException: For security reasons DTD is prohibited in this XML document. To enable DTD processing set the DtdProcessing property on XmlReaderSettings to Parse and pass the settings into XmlReader.Create method.
14:34:50Error
at System.Xml.XmlTextReaderImpl.Throw(Exception e)
14:34:50Error
at System.Xml.XmlTextReaderImpl.ParseDoctypeDecl()
14:34:50Error
at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
14:34:50Error
at System.Xml.Linq.XContainer.ReadContentFrom(XmlReader r)
14:34:50Error
at System.Xml.Linq.XContainer.ReadContentFrom(XmlReader r, LoadOptions o)
14:34:50Error
at System.Xml.Linq.XDocument.Load(XmlReader reader, LoadOptions options)
14:34:50Error
at Calamari.Integration.ConfigurationVariables.ConfigurationVariablesReplacer.ModifyConfigurationFile(String configurationFilePath, VariableDictionary variables) in y:\work\b1fd300731d5f2fc\source\Calamari\Integration\ConfigurationVariables\ConfigurationVariablesReplacer.cs:line 19
14:34:50Error
at Calamari.Deployment.Conventions.ConfigurationVariablesConvention.Install(RunningDeployment deployment) in y:\work\b1fd300731d5f2fc\source\Calamari\Deployment\Conventions\ConfigurationVariablesConvention.cs:line 37
14:34:50Error
at Calamari.Deployment.ConventionProcessor.RunInstallConventions() in y:\work\b1fd300731d5f2fc\source\Calamari\Deployment\ConventionProcessor.cs:line 60
14:34:50Error
at Calamari.Deployment.ConventionProcessor.RunConventions() in y:\work\b1fd300731d5f2fc\source\Calamari\Deployment\ConventionProcessor.cs:line 50
14:34:50Error
at Calamari.Commands.DeployPackageCommand.Execute(String[] commandLineArguments) in y:\work\b1fd300731d5f2fc\source\Calamari\Commands\DeployPackageCommand.cs:line 107
14:34:50Error
at Calamari.Program.Main(String[] args) in y:\work\b1fd300731d5f2fc\source\Calamari\Program.cs:line 18
14:34:51Error
The remote script failed with exit code 100

Has anyone else ever seen this?

2

Hi Shaun,
The error you are seeing sounds to be due to a protective reaction to what is called DTD processing. Based on the risks of DOS attacks referred to as XML Bombs or Billion Laughs attacks, the processing of the entity headers will potentially throw this error to avoid the memory bug. By default in .net 4.0 the XMLReader settings will be set to prohibit and this exception is thrown if not explicitly enabled.
A good summary of this process is described at https://msdn.microsoft.com/en-us/magazine/ee335713.aspx

While I look at a code solution for this (see GitHub Issue #1780), could you please confirm if the xml config undergoing processing has any of the <!DOCTYPE> elements in the top and if so, consider if they can be safely removed.

Let me know if this work around allows you to continue your deployment, and I will keep you updated with the status of this ticket.
Cheers,
Rob

3

Robert,

Thanks for the response. I saw the same things turning up in my searches as well. However, there is no <!DOCTYPE> element in the file.

In the previous version that we were using, we would see this issue on some of our config files (we use strings.config, settings.config and a couple other .config files) which was surprising because they don’t have the <!DOCTYPE> element either. However, it would “fail happy” on those files and continue the deployment, so it was never a real concern.

Now it’s completely stopping the deployment process however, so it’s become more of an issue.

I have attached an example of the file that is throwing the error. You’ll notice the first line is commented out, it was erroring both commented and uncommented, and was an attempt to resolve the issue.

Web.config (14 KB)

4

I am also having this problem and it is holding up deployments. Any word on this?

5

Shaun,
That config does indeed look like its free of <!DOCTYPE> tags and when I try load it, it works fine. Its only when I add the tag to the top do I get the exception thrown as expected.

Could you try run a simple script against the file just to see if there are some environmental factors at work?

       using (var reader = XmlReader.Create(@"C:\temp\Web.config"))
        {
            var doc = XDocument.Load(reader, LoadOptions.PreserveWhitespace);
        }

Are there any other XML files it might be picking up trying to run (although the logs look fairly conclusive that its web.config)?

Conradc,
As this issue was just logged yesterday afternoon the issue the issue will be shortly fixed when possible and this post will be updated as more information is available. Feel free to subscribe to the GH issue to get more updated information.

Thanks guys,
Rob

6

Robert,

We were unable to remove the DOCTYPE declarations at this point, without some other changes. However, we are able to work around this for the time being.

Here are some details that we found before the end of this:

If this error occurs, the deployment fails unless you are in guided failure and can choose to ignore (this is our short term work around until your fix in 3.0.7 is released).

Also, the logging here is misleading. The logging of the filename doesn’t occur until after you do the replacement. So, the logging was in reference to the previous file, not the file that was actually being attempted when the error occurred. If the log (Updating appSettings and connectionStrings in…) were to occur before opening the file, it might be a bit more clear.

Thanks for your help, the suggestion to check the other files is what got us heading in the right direction. Also, we greatly appreciate that this issue is being addressed in a future release.

Have a good day!

7

Hey guys,
A fix for this has gone out in Calamari ( 3.0.1.377 ) which will be bundled in the next Octopus Deploy build (3.0.7) which should go out sometime this afternoon or tomorrow at the latest.
This should allow DTD Processing to take place up to a safe 1 GB memory limit.
Hopefully this gets you back to deploying again. With regards to the delayed logging I’ll take a look at that to ensure the information is more consistent.
Thanks again,
Rob