RuntimeError: Event loop is closed during object finalization after asyncio.run() completes in short-lived scripts

[sergioPereiraBR]

Bug report

Bug description:

Description:

Dear asyncio developers,

We are experiencing a persistent RuntimeError: Event loop is closed that occurs during the finalization phase of objects after asyncio.run() completes in short-lived command-line interface (CLI) scripts. While the exception is "ignored" (it does not crash the program), it pollutes the console output, which is highly undesirable in production environments and negatively impacts user experience.

Problem Context:

Our application is a CLI tool built with Click that performs asynchronous operations (such as database access via SQLAlchemy with aiomysql and asynchronous logging with Loguru). For each asynchronous CLI command, we encapsulate its logic within a dedicated asyncio.run() call, structured as follows:

import asyncio
import click
# ... (other async-aware library imports)

@click.command()
def my_async_command():
    async def _run_command():
        try:
            # Main asynchronous logic (e.g., await db_op(), await log_op())
            print("Executing asynchronous operation...")
            await asyncio.sleep(0.01) # Simulates async work
        finally:
            # Explicit resource release attempts
            # E.g.: await sqlalchemy_engine.dispose()
            # E.g.: logger.shutdown_gracefully()
            print("Resources explicitly released within the same loop.")

    # Executes the async operation and manages its own loop
    asyncio.run(_run_command())
    print("Click command finished.")

# ...

Observed Behavior:

Despite the core logic executing correctly and our implementation of explicit dispose() calls for the SQLAlchemy engine and shutdown() for Loguru (which uses asynchronous queues), the following RuntimeError consistently appears in the console output after the "Click command finished." message and after the resource release messages:

Exception ignored in: <function Connection.__del__ at 0x000001BF6BBA0AE0>
Traceback (most recent call last):
  File "path\to\site-packages\aiomysql\connection.py", line 1131, in __del__
  File "path\to\site-packages\aiomysql\connection.py", line 339, in close
  File "path\to\python\Lib\asyncio\proactor_events.py", line 109, in close
  File "path\to\python\Lib\asyncio\base_events.py", line 761, in call_soon
  File "path\to\python\Lib\asyncio\base_events.py", line 519, in _check_closed
RuntimeError: Event loop is closed

The traceback origin (aiomysql.connection.py) suggests the exception occurs during the finalization (via del method) of aiomysql connection objects. These synchronous destructors attempt to interact with the event loop (e.g., via loop.call_soon), but the loop has already been explicitly closed by asyncio.run().

Impact:

Interface Pollution: The error message is displayed to the end-user, even when the command has completed successfully. This degrades the user experience and gives a false impression of errors.
Monitoring Noise: In CI/CD pipelines or production monitoring systems, these messages can be misinterpreted as failures, leading to unnecessary alerts.
Maintainability: Although "ignored," the persistence of this error raises concerns about resource cleanup and the application's overall robustness.

What we have tried and why it hasn't fully resolved the issue:

1. Explicit Resource Release: We call engine.dispose() (for SQLAlchemy) and logger.complete()/logger.remove() (for Loguru) within the same asynchronous scope (_run_command) before asyncio.run() returns. This ensures that asynchronous operations and connection pools are finalized while the loop is still active.
2. Timing/Garbage Collection Issue: We believe the exception occurs because some low-level connection objects (aiomysql.Connection) persist in memory for a brief period after the engine has been disposed of and the event loop closed. When Python's garbage collector eventually executes the del of these objects, the event loop is no longer available, leading to the RuntimeError. As garbage collection is non-deterministic, it's extremely difficult to control this timing from application code.

Minimal Reproducible Example (MRE):

This MRE simulates the scenario where an asynchronous resource attempts to clean itself up via del after asyncio.run() has closed the event loop:

import asyncio
import gc

class AsyncResourceSimulation:
    def __init__(self):
        print("AsyncResourceSimulation: Instance created.")

    def _cleanup_async_part(self):
        """Simulates an attempt at an asynchronous cleanup operation."""
        print("AsyncResourceSimulation: Attempting asynchronous cleanup operation...")
        try:
            loop = asyncio.get_event_loop()
            if not loop.is_closed():
                # This line simulates the call_soon that aiomysql attempts to make
                loop.call_soon(lambda: print("AsyncResourceSimulation: Async cleanup operation executed."))
            else:
                print("AsyncResourceSimulation: Event loop is closed. Cannot perform async operation.")
                # The actual exception occurs because call_soon raises RuntimeError if the loop is closed
        except RuntimeError as e:
            # This simulates the "Exception ignored" behavior
            print(f"AsyncResourceSimulation: Exception caught in _cleanup_async_part: {e}")


    def __del__(self):
        """Destructor called by garbage collection."""
        print("AsyncResourceSimulation: __del__ called.")
        try:
            # Simulate what aiomysql's __del__ does: try to close the resource
            # which may involve an async operation.
            self._cleanup_async_part()
        except Exception as e:
            # In a real scenario, this exception is "ignored" by the interpreter.
            print(f"AsyncResourceSimulation: Exception caught in __del__ handler: {e}")

async def main_coroutine():
    """Main asynchronous function for our CLI "command"."""
    print("Main Coroutine: Started.")
    # Create an instance of the resource that will be garbage collected
    resource = AsyncResourceSimulation()
    await asyncio.sleep(0.01) # Simulate some async work
    print("Main Coroutine: Completed.")
    # 'resource' is now out of scope, eligible for GC.

if __name__ == "__main__":
    print("Main script: Starting asyncio.run()...")
    asyncio.run(main_coroutine())
    print("Main script: asyncio.run() completed.")
    print("Main script: Forcing garbage collection to trigger __del__ (may or may not be necessary)...")
    gc.collect() # Attempts to force garbage collection to trigger __del__
    print("Main script: Finished.")

Request:

We kindly request an investigation into ways to mitigate or eliminate this RuntimeError during object finalization, especially in short-lived asyncio.run() scenarios. Possible approaches could include:

A mechanism to register "asynchronous finalizers" that are guaranteed to be executed before the event loop is closed by asyncio.run().
Clearer guidance or a recommended design pattern for libraries that require asynchronous cleanup in their destructors.
A way for destructors to detect a "closed" event loop more gracefully, perhaps with a warning instead of a RuntimeError or by conditionally suppressing it internally.
The ability to have completely clean console output is crucial for human-machine interaction in production CLI tools, and resolving this issue would significantly enhance the development and operational experience.

Thank you for your time and dedication to asyncio development.

CPython versions tested on:

3.11

Operating systems tested on:

Windows

[emmatyping]

I am unable to reproduce the failure (even with several attempts) given the example script on Windows with CPython main or 3.11. Can you provide the specific Python version you are using (python -V)?