Octopus sees python info logs as error

[j_steensma]

I am running a python script that has already been deployed to a target server. I’m executing the python script with a Powershell command as under different conditions the script will have different arguments.

In the python script, I’m using the logging module. Why does Octopus see info logging output as an error and how can I get Octopus to see it as just Info logging?

[dane.falvo]

Hi @j_steensma,

Thank you for the information and I’m sorry that you’ve hit this issue.

It seems like this is a fairly common issue with when executing python, java etc via certain shells. It seems that sometimes logs erroneously get submitted to stderr as opposed to stdout.

Would you be able to redirect your stderr to stdout or alternativily to null or a different file?

https://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO-3.html
I think section 3.5 of this guide should work for what you are trying to do.

Regards,

Dane

[j_steensma]

Thanks for the suggestion Dane. It got me thinking if maybe I could change how python output the logs.

Found code by Elias Strehle on Stake Overflow . seems to be working well now.

import logging
import logging.config
import sys

class _ExcludeErrorsFilter(logging.Filter):
    def filter(self, record):
        """Only returns log messages with log level below ERROR (numeric value: 40)."""
        return record.levelno < 40

config = {
    'version': 1,
    'filters': {
        'exclude_errors': {
            '()': _ExcludeErrorsFilter
        }
    },
    'formatters': {
        # Modify log message format here or replace with your custom formatter class
        'my_formatter': {
            'format': '%(asctime)s: %(levelname)-8s: %(message)s'
        }
    },
    'handlers': {
        'console_stderr': {
            # Sends log messages with log level ERROR or higher to stderr
            'class': 'logging.StreamHandler',
            'level': 'ERROR',
            'formatter': 'my_formatter',
            'stream': sys.stderr
        },
        'console_stdout': {
            # Sends log messages with log level lower than ERROR to stdout
            'class': 'logging.StreamHandler',
            'level': 'DEBUG',
            'formatter': 'my_formatter',
            'filters': ['exclude_errors'],
            'stream': sys.stdout
        },
        'file': {
            # Sends all log messages to a file
            'class': 'logging.FileHandler',
            'level': 'DEBUG',
            'formatter': 'my_formatter',
            'filename': 'my.log',
            'encoding': 'utf8'
        }
    },
    'root': {
        # In general, this should be kept at 'NOTSET'.
        # Otherwise it would interfere with the log levels set for each handler.
        'level': 'NOTSET',
        'handlers': ['console_stderr', 'console_stdout', 'file']
    },
}

logging.config.dictConfig(config)

[dane.falvo]

I’m glad you got it working!

Not being strong in Python, I’m going to have to take your word for it, but I really appreciate that you’ve posted the code. Anyone who stumbles upon this ticket will have a great solution at their fingertips.

Happy Deployments!

Dane