fix(core): prevent async task garbage collection (RUF006) (#34238)

# PR Title: fix(core): prevent async task garbage collection (RUF006)

## Description
This PR addresses a cryptic issue (flagged by Ruff rule RUF006) where
`asyncio` tasks created via `loop.create_task` could be garbage
collected mid-execution because no strong reference was maintained.

In `libs/core/langchain_core/language_models/llms.py`, the retry
decorator's `_before_sleep` hook creates a fire-and-forget task for
logging/callbacks. If the garbage collector runs before this task
completes, the task may be destroyed, leading to silent failures.

## Changes
- Introduced a module-level set `_background_tasks` to hold strong
references to running tasks.
- Updated `_before_sleep` to add new tasks to this set.
- Added a `done_callback` to remove the task from the set upon
completion, preventing memory leaks.

## Verification
- Verified logic with a standalone script to ensure tasks are
added/removed from the set correctly.
- This is a standard pattern recommended in the Python `asyncio`
documentation.

## Checklist
- [x] I have read the contributing guidelines.
- [x] I have run tests locally (logic verification).

---------

Co-authored-by: Mason Daugherty <[email protected]>
Co-authored-by: Mason Daugherty <[email protected]>
Co-authored-by: Copilot <[email protected]>

Author: 3 people

libs/core/langchain_core/language_models/llms.py

@@ -61,6 +61,8 @@
 
 logger = logging.getLogger(__name__)
 
+_background_tasks: set[asyncio.Task] = set()
+
 
 @functools.lru_cache
 def _log_error_once(msg: str) -> None:
@@ -100,9 +102,9 @@ def _before_sleep(retry_state: RetryCallState) -> None:
                         asyncio.run(coro)
                     else:
                         if loop.is_running():
-                            # TODO: Fix RUF006 - this task should have a reference
-                            #  and be awaited somewhere
-                            loop.create_task(coro)  # noqa: RUF006
+                            task = loop.create_task(coro)
+                            _background_tasks.add(task)
+                            task.add_done_callback(_background_tasks.discard)
                         else:
                             asyncio.run(coro)
                 except Exception as e: