Log Output of Failed Process

[jbdyn]

Hi there 👋,

I started using nox for matrix testing in elva and got a working noxfile.py in no time.
Very nice ❤️

Since there are many sessions, I want to write session-specific logs.

However, there was no way to capture the output of a failed pytest run to see what went wrong.

So, here is my patch to overcome this.

Basically, the output of a failed process is no longer written to sys.stderr solely but instead by the session logger via logger.error.
Before, any output logging was prevented by a raised CommandFailed exception.

This whole story is somewhat related to issue #409.
The difference is that I didn't need to use asyncio in popen.

That is the current state of my noxfile.py

from itertools import product
from typing import Generator, Iterable
import logging
import sys
from pathlib import Path
from datetime import datetime

import nox

def sanitize_path(path):
    for char in ": =().":
        path = path.replace(char, "-")
    return path

BACKEND = "uv|virtualenv"
EDITABLE = ("-e", ".[dev,logo]")
TIMESTAMP = sanitize_path(datetime.now().isoformat(timespec="minutes"))
LOG_PATH = Path(__file__).parent / "logs" / "nox" / TIMESTAMP
LOG_PATH.mkdir(parents=True, exist_ok=True)

PROJECT = nox.project.load_toml("pyproject.toml")

# from classifiers and not `requires-python` entry;
# see https://nox.thea.codes/en/stable/config.html#nox.project.python_versions
PYTHON = nox.project.python_versions(PROJECT)

# versions to test by compatible release;
# check for every version adding new functionality or breaking the API
WEBSOCKETS = ("13.0.0", "13.1.0", "14.0.0", "14.1.0", "14.2.0", "15.0.0")
TEXTUAL = ("1.0.0", "2.0.0", "2.1.0", "3.0.0", "3.1.0", "3.2.0")


def parameters_excluding_last(*params: Iterable[Iterable[str]]) -> Generator[nox.param, None, None]:
    """Genererate the products of parameters except for the last one."""
    latest = tuple(param[-1] for param in params)

    for prod in product(*params):
        if prod != latest:
            yield nox.param(*prod)



def set_log_file(path):
    SESSION_HANDLER = "nox-session"
    logger = logging.getLogger()

    for handler in logger.handlers:
        if handler.name == SESSION_HANDLER:
            logger.removeHandler(handler)

    handler = logging.FileHandler(path)
    handler.name = SESSION_HANDLER
    logger.addHandler(handler)


@nox.session(
    venv_backend=BACKEND,
)
@nox.parametrize(
    # exclude newest since this environment configuration is covered by the `coverage` session below
    ("python", "websockets", "textual"), parameters_excluding_last(PYTHON, WEBSOCKETS, TEXTUAL)
)
def tests(session, websockets, textual):
    LOG_FILE = LOG_PATH / f"{sanitize_path(session.name)}.log"
    set_log_file(LOG_FILE)

    # idempotent
    session.notify("coverage")

    # install from `pyproject.toml`
    session.install(*EDITABLE)

    # overwrite with specific versions;
    # for compatible release specifier spec,
    # see https://packaging.python.org/en/latest/specifications/version-specifiers/#compatible-release
    session.install(
        f"websockets~={websockets}",
        f"textual~={textual}",
    )

    # TODO: run across all tests
    session.run(
        "pytest",
        "--log-file",
        LOG_FILE,
        "--log-file-mode=a",
        "tests/test_component.py",
        silent=True,
    )


@nox.session(
    venv_backend=BACKEND,
)
def coverage(session):
    LOG_FILE = LOG_PATH / f"{sanitize_path(session.name)}.log"
    set_log_file(LOG_FILE)

    # install from `pyproject.toml`;
    # make sure to install the latest possible versions since `uv` won't update otherwise
    if session.venv_backend == "uv":
        session.install("--exact", "--reinstall", *EDITABLE)
    else:
        session.install(*EDITABLE)

    # TODO: run across all tests
    session.run(
        "coverage",
        "run",
        "-m",
        "pytest",
        "--log-file",
        LOG_FILE,
        "--log-file-mode=a",
        "tests/test_component.py",
        silent=False
    )

    # generate reports
    session.run("coverage", "combine", silent=True)
    session.run("coverage", "report", silent=True)
    session.run("coverage", "html", silent=True)

What do you think?

[jbdyn]

Consequently, I would like to name the log file the same as its corresponding environment directory.

Since neither _runner in Session nor _normalize_path are part of the public API, I felt that exposing _runner.envdir in Session as a property would be the least invasive change.

I don't know whether that needs testing, though. 🤔