Python Logging for Scripts That Run Unattended

A script that runs on a schedule has no one watching it. When it fails, or worse, when it succeeds but processes the wrong data, the only record of what happened is whatever it wrote down along the way. Many scripts write almost nothing, or print messages to a console nobody reads. Python logging, from the standard library, gives you a structured way to record what a script did, with levels for severity, timestamps and destinations such as files or monitoring systems. This guide explains how to set up Python logging for unattended scripts so that failures are visible, investigations are quick and sensitive data stays out of the logs.

Why print statements are not enough

Print statements are fine while you develop a script, but they fall short once it runs unattended. They have no severity, so you cannot separate routine progress from errors. They have no timestamps unless you add them by hand. They go wherever standard output goes, which on a server may be nowhere. And they cannot be turned up or down without editing code. The logging module solves each of these problems, and it is already installed with Python.

Python logging basics

The Python logging HOWTO recommends creating a logger with logging.getLogger(__name__) and calling its debug, info, warning, error and critical methods (Python: Logging HOWTO). Using the module name gives each part of your code its own named logger, so you can tell where each message came from.

The same documentation explains that the default level is WARNING, which means only warnings and more severe events are tracked unless logging is configured otherwise. This surprises many people: info messages simply disappear until you configure a level. Set the level explicitly at the start of your script, usually INFO for production runs and DEBUG when investigating.

Choose levels deliberately

Levels are only useful if they are used consistently. A practical convention for scripts:

• DEBUG: detailed values useful while investigating, such as individual record IDs or raw responses
• INFO: milestones in a normal run, such as start, end, number of records read and written
• WARNING: something unexpected that the script handled, such as a skipped record or a retried request
• ERROR: an operation failed and data may be incomplete, such as a batch that could not be written
• CRITICAL: the script cannot continue, such as missing credentials or an unreachable database

Agree on this convention across your scripts so alerts can rely on it.

Log context, not just messages

A message like "Request failed" is not much help at two in the morning. Include the context needed to act:

• Which run this is, with a run ID or start time
• Which record, file or batch was being processed
• What the script was trying to do, such as "update contact in CRM"
• The error type and message, with the traceback for unexpected exceptions
• Counts at the end of each run: read, written, skipped, failed

Use the logger's exception method inside an except block to record the traceback automatically. Our guide to Python automation scripts explains how logging fits alongside timeouts, retries and safe reruns.

Keep secrets and personal data out

Logs are often stored longer and shared more widely than the data they describe. Never log passwords, API keys, tokens or full request headers. Be careful with personal data such as email addresses, phone numbers and financial details; log record IDs instead, which let you look up details in the source system when needed. Review your log statements as part of code review with this in mind, and remember that DEBUG output often includes raw payloads.

Send logs somewhere durable

Configure handlers to send log records where they will be kept and read. Options include:

• A local file, rotated so it does not grow forever. The logging.handlers module provides RotatingFileHandler, which rotates files by size, and TimedRotatingFileHandler, which rotates by time (Python: logging.handlers)
• Standard output, when the script runs in a container or a scheduler that collects output for you
• A central log service, so logs from many scripts can be searched in one place

Whatever the destination, include timestamps with time zones in the format, so events from different machines can be lined up.

Worked example: a hypothetical logging plan

The table below is a hypothetical logging plan for a nightly script that syncs orders from a spreadsheet export into a CRM. It shows what to log at each level, not a configuration for a specific tool.

EventLevelExample contentRun startedINFORun ID, input file name, row countRow skipped for missing order numberWARNINGRow number, reasonCRM request retriedWARNINGOrder ID, attempt number, status codeCRM write failed after retriesERROROrder ID, error messageCredentials missingCRITICALName of missing setting, not its valueRun finishedINFOCounts of read, written, skipped and failed rows, durationRaw CRM responseDEBUGOnly when investigating, with personal fields removed

Turn errors into alerts

Logs nobody reads do not prevent incidents. Set up alerts for ERROR and CRITICAL messages and for runs that never finish. Also alert on unusual counts, such as a run that read zero rows when it normally reads hundreds. Our post on workflow monitoring explains how to design alerts that catch silent failures without flooding people with noise.

Consider structured log lines

When logs go to a central service, writing each record as a set of named fields, often as one JSON object per line, makes searching and filtering much easier than parsing free text. You can then ask questions such as "all failed CRM writes for order IDs in this batch" without writing fragile text searches. If you adopt this, keep field names consistent across scripts: run ID, record ID, action, outcome and error are a good starting set.

Configure logging once

Set up logging in one place, at the entry point of the script, and let every module use named loggers. Library code should not configure handlers or levels itself; it should only create loggers and log. This keeps behaviour predictable when you reuse modules across scripts. Read the log level from configuration or an environment variable so you can switch to DEBUG for one run without changing code.

Frequently Asked Questions

Why are my info messages not showing?

The Python logging documentation explains that the default level is WARNING, so info and debug messages are ignored until you set a lower level when configuring logging.

Should I log to a file or to standard output?

Log to standard output when a container platform or scheduler collects it for you. Otherwise use a rotating file or send logs to a central service so they are kept and searchable.

What should never appear in logs?

Passwords, API keys, tokens and full request headers. Avoid personal data where you can, and log record IDs instead.

How do I know when a scheduled script fails?

Alert on ERROR and CRITICAL messages, on runs that do not finish and on unusual record counts, rather than relying on someone to read the logs.

Know what your scripts are doing

We set up logging, alerting and run reports for Python automation so failures are found and fixed quickly. See our custom app development services, or talk to us about Python logging.

About the Author

Manav Sharma is the founder of Insyrge and a CRM solutions architect. He works across Zoho, Salesforce, HubSpot, Odoo and more, designing the CRM setups, integrations and automations that keep sales and operations teams working from the same data.