fix: Refactor to reuse Azure credentials and implement async processing

[Ragini-Microsoft]

Purpose

This pull request refactors the main entrypoints of 03_cu_process_data_text.py and 04_cu_process_custom_data.py to use asyncio and shared credentials more consistently, improving resource management and code clarity. The changes ensure that asynchronous operations are properly awaited, credentials are reused and closed correctly, and redundant instantiations are eliminated.

Refactoring for async/await and credential management:

• Wrapped main logic in both scripts (03_cu_process_data_text.py and 04_cu_process_custom_data.py) in an async def main() function and updated the entrypoint to use asyncio.run(main()), ensuring all async operations are properly awaited. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]
• Replaced multiple instantiations of AzureCliCredential and AsyncAzureCliCredential with a single shared instance per script, which is reused for all Azure service clients (DataLake, Search, Content Understanding, etc.). [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11]
• Ensured that credential objects are explicitly closed at the end of each script to prevent resource leaks, with error handling for safe closure. [1] [2]

Improvements to asynchronous agent management:

• Refactored agent creation, usage, and deletion to be fully asynchronous, removing unnecessary nested credential instantiations and ensuring all agent-related async functions are awaited in sequence. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]

These changes make the scripts more robust, maintainable, and efficient in their use of asynchronous programming and Azure credentials.

Does this introduce a breaking change?

• Yes
• No

Golden Path Validation

• I have tested the primary workflows (the "golden path") to ensure they function correctly without errors.

Deployment Validation

• I have validated the deployment process successfully and all services are running as expected with this change.

[Copilot]

Pull request overview

This pull request refactors the two Content Understanding indexing scripts to reuse a single Azure CLI credential per script and to standardize execution around a single async main() entrypoint, reducing repeated auth setup and avoiding nested asyncio.run() calls.

Changes:

• Converted both scripts to async def main() with asyncio.run(main()), and refactored internal flows to use await consistently.
• Reused a shared AzureCliCredential across DataLake/Search/CU/OpenAI token flows and added explicit credential cleanup.
• Updated agent creation / topic-mining flows to run within the async orchestration and reuse shared async credential instances.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
infra/scripts/index_scripts/03_cu_process_data_text.py	Refactors script orchestration to async, reuses a shared Azure CLI credential, and adjusts agent/topic-mining flow accordingly.
infra/scripts/index_scripts/04_cu_process_custom_data.py	Same async + shared-credential refactor for the “custom data” processing path, including search index recreation and agent lifecycle handling.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

🎉 This PR is included in version 3.23.3 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀