adding hybrid_search_alpha=0 condition for hybrid_search enablment by fshabashev · Pull Request #11573 · mindsdb/minds

fshabashev · 2025-09-15T15:29:30Z

Description

Please include a summary of the change and the issue it solves.

Fixes #issue_number

Type of change

(Please delete options that are not relevant)

🐛 Bug fix (non-breaking change which fixes an issue)
⚡ New feature (non-breaking change which adds functionality)
📢 Breaking change (fix or feature that would cause existing functionality not to work as expected)
📄 This change requires a documentation update

Verification Process

To ensure the changes are working as expected:

Test Location: Specify the URL or path for testing.
Verification Steps: Outline the steps or queries needed to validate the change. Include any data, configurations, or actions required to reproduce or see the new functionality.

Additional Media:

I have attached a brief loom video or screenshots showcasing the new functionality or change.

Checklist:

My code follows the style guidelines(PEP 8) of MindsDB.
I have appropriately commented on my code, especially in complex areas.
Necessary documentation updates are either made or tracked in issues.
Relevant unit and integration tests are updated or added.

entelligence-ai-pr-reviews · 2025-09-15T15:30:05Z

🔒 Entelligence AI Vulnerability Scanner

❌ Security analysis failed: Security analysis failed: 404: template 'bp1y9rqvhhvvfpvdc4n5' not found

The base branch was changed.

entelligence-ai-pr-reviews · 2025-10-17T12:09:07Z

🔒 Entelligence AI Vulnerability Scanner

❌ Security analysis failed: Security analysis failed: context deadline exceeded: This error is likely due to exceeding 'timeoutMs' — the total time a long running request (like process or directory watch) can be active. It can be modified by passing 'timeoutMs' when making the request. Use '0' to disable the timeout.

entelligence-ai-pr-reviews · 2025-10-17T13:23:22Z

Review Summary

🏷️ Draft Comments (178)

Skipped posting 178 draft comments that were valid but scored below your review threshold (>=13/15). Feel free to update them here.
.github/workflows/build_deploy_dev.yml (2)
67-89: build_deploy_dev.yml contains repeated checkout and bake steps across jobs, leading to longer CI times and redundant resource usage for every job (checkout and bake can be shared via reusable workflows or composite actions).

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In .github/workflows/build_deploy_dev.yml, lines 67-89, the checkout and docker-bake steps are duplicated across multiple jobs (build, build-cache). This causes redundant resource usage and longer CI times. Refactor the workflow to use a composite action or reusable workflow for these steps, or restructure jobs to share the build context and artifacts, minimizing repeated work.
110-121: The workflow runs all integration tests for every deploy-env in parallel, which can overwhelm shared resources and slow down overall CI if the number of environments grows.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 8/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In .github/workflows/build_deploy_dev.yml, lines 110-121, the integration tests run in parallel for all deploy-envs, which can cause resource contention and slow down CI if the matrix grows. Add a max-parallel value to the matrix strategy to limit concurrent jobs and optimize resource usage.
docs/integrations/data-integrations/databricks.mdx (1)
54-54: http_headers parameter example uses Python list-of-lists syntax, which is invalid JSON and will cause runtime parsing errors if copied as-is.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In docs/integrations/data-integrations/databricks.mdx, line 54, the example for the `http_headers` parameter uses Python-style list-of-lists syntax, which is not valid JSON and will cause runtime errors if users copy it. Please update the example to use valid JSON array-of-arrays syntax and clarify that it should be a JSON string, e.g., `"http_headers": [["Header-1", "value1"], ["Header-2", "value2"]]`.
docs/setup/custom-config.mdx (1)
374-394: config.json example and documentation omit new open_on_start and a2wsgi parameters, which can mislead users and cause startup errors if these are required or expected.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 5/5

Urgency Impact: 2/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In docs/setup/custom-config.mdx, lines 374-394, the example `config.json` and documentation omit the new `open_on_start` parameter in the `gui` section and the `a2wsgi` block in the `api.http` section. This can mislead users and cause startup errors if these parameters are required or expected. Update the example to include `"open_on_start": true` in the `gui` section and add the `"a2wsgi": { "workers": 15, "send_queue_size": 10 }` block to the `api.http` section, matching the formatting of the rest of the example.
mindsdb/__main__.py (2)
0183-0192: Database queries in set_error_model_status_by_pids and set_error_model_status_for_unfinished call db.session.commit() inside a loop, causing N database commits for N records, which severely degrades performance on large datasets.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Optimize the database commit pattern in mindsdb/__main__.py lines 183-192. Currently, `db.session.commit()` is called inside a loop, causing N commits for N records. Refactor so that all updates are performed in the loop, but only a single `db.session.commit()` is called after the loop if any records were updated.
0207-0214: Similarly, set_error_model_status_for_unfinished performs a database commit inside a loop, leading to O(N) commits for N records, which is highly inefficient for large numbers of unfinished models.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 5/5

Urgency Impact: 2/5

Total Score: 10/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor mindsdb/__main__.py lines 207-214 to avoid calling `db.session.commit()` inside the loop. Instead, update all records in the loop and call `db.session.commit()` only once after the loop if any records were changed.
mindsdb/api/a2a/agent.py (2)
47-47: No timeout specified in requests.post in invoke, risking resource exhaustion and degraded performance under network issues.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/a2a/agent.py, line 47, the `invoke` method calls `requests.post` without a timeout, which can cause the application to hang and exhaust resources if the network is slow or unresponsive. Add a reasonable timeout (e.g., 10 seconds) to the `requests.post` call.
44-45: invoke method constructs SQL queries by directly interpolating user input (query), allowing SQL injection and potential unauthorized data access or manipulation.

📊 Impact Scores:

Production Impact: 5/5

Fix Specificity: 2/5

Urgency Impact: 4/5

Total Score: 11/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/a2a/agent.py, lines 44-45, the `invoke` method constructs SQL queries by directly interpolating user input (`query`), which allows SQL injection. Refactor this code to use parameterized queries or a safe query API that prevents injection. Replace the string formatting with a secure parameter passing mechanism, ensuring user input cannot alter the query structure.
mindsdb/api/a2a/common/server/server.py (1)
84-89: user_info is constructed from HTTP headers without validation or sanitization, allowing header injection or spoofing that could lead to privilege escalation or unauthorized access in downstream logic.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 2/5

Urgency Impact: 4/5

Total Score: 10/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/a2a/common/server/server.py, lines 84-89, the `user_info` dictionary is populated directly from HTTP headers without validation, which could allow header spoofing or injection attacks leading to privilege escalation. Refactor this code to sanitize all header values using a helper method (e.g., `_sanitize_header`) that only allows safe characters (alphanumeric, dash, underscore) and strips whitespace. Add this helper method to the class and use it for each header value.
mindsdb/api/a2a/common/server/task_manager.py (2)
133-136: get_push_notification_info may raise KeyError if task_id exists in self.tasks but not in self.push_notification_infos, causing an unhandled exception and possible crash.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 2/5

Urgency Impact: 3/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/a2a/common/server/task_manager.py, lines 133-136, the method `get_push_notification_info` directly accesses `self.push_notification_infos[task_id]`, which can raise a `KeyError` if the key is missing, even if the task exists. Change this to use `.get(task_id)` to avoid unhandled exceptions and return `None` if the notification info is missing.
227-234: The method append_task_history (lines 227-234) creates a full copy of the Task object and slices the history list on every call, which can be expensive for large histories and high-frequency access.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 2/5

Urgency Impact: 2/5

Total Score: 6/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/a2a/common/server/task_manager.py, lines 227-234, the method `append_task_history` creates a full copy of the Task object and slices the history list, which is inefficient for large histories or frequent calls. Refactor this method to avoid unnecessary object copying and only slice the history list in-place, returning the original task object with the updated history.
mindsdb/api/a2a/task_manager.py (1)
205-273: The upsert_task method (lines 205-273) is a large, complex function with deeply nested logic and repeated code for history extraction, making it difficult to maintain and error-prone as the codebase grows.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `upsert_task` method in mindsdb/api/a2a/task_manager.py (lines 205-273) to reduce complexity and code duplication. Extract the repeated history extraction logic into a helper function within the method, and simplify the main logic for better maintainability. Preserve all existing functionality.
mindsdb/api/common/middleware.py (1)
41-44,52-56: TOKENS is a list and both verify_pat and revoke_pat perform O(n) linear scans for every token check or revoke, which will cause significant performance degradation as the number of tokens grows.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 5/5

Urgency Impact: 2/5

Total Score: 10/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/common/middleware.py, lines 41-44 and 52-56, the `TOKENS` list is used for token lookup and removal, resulting in O(n) performance for every check or revoke. Refactor `TOKENS` to be a set instead of a list, and update `generate_pat`, `verify_pat`, and `revoke_pat` to use set operations for O(1) performance. Ensure all code using `TOKENS` is updated accordingly.
mindsdb/api/executor/command_executor.py (1)
227-685: execute_command in ExecuteCommands is a monolithic function with excessive complexity (99 branches, 195 statements), making it hard to maintain and optimize for performance as the codebase grows.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `execute_command` method in mindsdb/api/executor/command_executor.py (lines 227-685). The function is excessively large and complex, with over 99 branches and 195 statements, making it difficult to maintain and optimize. Break it into smaller, well-named methods for each major command type (e.g., SHOW, CREATE, DROP, etc.), and use a dispatch table or clear control flow to route commands. This will improve maintainability, testability, and future performance optimization.
mindsdb/api/executor/datahub/datanodes/project_datanode.py (1)
103-171: query method is excessively complex (C901, PLR0912): too many branches and logic paths, making it hard to maintain and optimize for performance as the codebase grows.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 2/5

Urgency Impact: 2/5

Total Score: 6/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `query` method in mindsdb/api/executor/datahub/datanodes/project_datanode.py (lines 103-171) to reduce complexity and improve maintainability. Move the logic for handling Update, Delete, and Select queries into dedicated private methods (`_handle_update`, `_handle_delete`, `_handle_select`). The main `query` method should delegate to these handlers based on the query type. Preserve all existing logic and ensure functional equivalence.
mindsdb/api/executor/datahub/datanodes/system_tables.py (1)
99-99: for name in inf_schema.tables.keys() (line 99) is O(n) and should be for name in inf_schema.tables for large dicts, to avoid unnecessary key list creation.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 5/5

Urgency Impact: 2/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/executor/datahub/datanodes/system_tables.py, line 99, replace `for name in inf_schema.tables.keys():` with `for name in inf_schema.tables:` to avoid unnecessary list creation and improve performance for large dictionaries.
mindsdb/api/executor/planner/plan_join.py (2)
396-397: self.query_context["binary_ops"] is used in process_table before being initialized, causing a KeyError if get_filters_from_join_conditions is not called first.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/executor/planner/plan_join.py, lines 396-397, the code accesses `self.query_context["binary_ops"]` without ensuring the key exists, which can cause a KeyError. Update the condition to check if "binary_ops" is in `self.query_context` before accessing it.
285-367: plan_join_tables (lines 285-367) is a large, complex function with deeply nested logic, making it difficult to maintain and optimize for performance as join logic grows.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `plan_join_tables` method in mindsdb/api/executor/planner/plan_join.py (lines 285-367). The function is overly large and complex, making it difficult to maintain and optimize. Break it into smaller, well-named helper methods for each major logical block (e.g., subselect replacement, join sequence construction, identifier resolution, and plan step creation). Ensure the refactoring preserves all existing logic and functionality.
mindsdb/api/executor/planner/query_prepare.py (1)
167-366: prepare_select method is excessively large and complex (over 100 lines, >50 statements, >50 branches), making it hard to maintain and optimize, which increases risk of performance and scalability issues as logic grows.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 8/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `prepare_select` method in mindsdb/api/executor/planner/query_prepare.py (lines 167-366). The function is excessively large and complex, with over 100 lines, more than 50 statements, and many branches. Break it into smaller, well-named helper methods to improve maintainability and enable future performance optimizations. Ensure each helper encapsulates a logical sub-task (e.g., predictor discovery, table extraction, column resolution, etc.), and preserve all existing logic and functionality.
mindsdb/api/executor/sql_query/sql_query.py (1)
152-158: BadTableError is raised with a SQL query string containing unsanitized table_name and table_version, allowing error message SQL injection and potential sensitive data exposure if attacker controls these values.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 4/5

Urgency Impact: 3/5

Total Score: 11/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/executor/sql_query/sql_query.py, lines 152-158, the code raises BadTableError with an error message that includes unsanitized user-controlled values (`table_name`, `table_version`) in a SQL query string. This can lead to error message SQL injection and sensitive data exposure. Refactor this block to remove any user-controlled values from the error message and avoid including raw SQL queries. The error message should generically instruct the user to check the model status without echoing input values.
mindsdb/api/executor/sql_query/steps/subselect_step.py (3)
206-209: remove_not_used_conditions may incorrectly remove valid conditions if a column is missing from col_idx but present in lower_col_idx, leading to false positives and incorrect query results.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/executor/sql_query/steps/subselect_step.py, lines 206-209, the code only checks `col_idx` for the presence of a key when removing unused conditions, but should also check `lower_col_idx` to avoid removing valid conditions. Update the condition to `if key not in col_idx and key not in lower_col_idx:`.
92-110: Multiple nested loops and repeated dictionary lookups in column index construction (col_idx, lower_col_idx, tbl_idx) can cause significant performance degradation for large result sets.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Optimize the construction of `col_idx`, `tbl_idx`, and `lower_col_idx` in mindsdb/api/executor/sql_query/steps/subselect_step.py, lines 92-110. Currently, there are multiple loops and repeated dictionary lookups, which can degrade performance for large result sets. Refactor to build all three dictionaries in a single loop, minimizing redundant operations and improving efficiency.
118-191: The check_fields function is highly complex (15 branches, 8 returns), making it difficult to maintain and optimize, and increasing risk of performance regressions as logic grows.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 8/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `check_fields` function in mindsdb/api/executor/sql_query/steps/subselect_step.py (lines 118-191) to reduce its cyclomatic complexity and improve maintainability. The function currently has 15 branches and 8 return statements, making it hard to reason about and optimize. Break it into smaller helper functions for each major case (Function, Variable, Identifier), and simplify the control flow where possible.
mindsdb/api/executor/utilities/functions.py (1)
33-33: requests.get(url) is called without a timeout, which can cause the function to hang indefinitely if the remote server is unresponsive.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/executor/utilities/functions.py, line 33, the call to `requests.get(url)` does not specify a timeout, which can cause the function to hang indefinitely if the remote server does not respond. Add a reasonable timeout (e.g., 30 seconds) to the `requests.get` call to ensure the function fails gracefully if the server is unresponsive.
mindsdb/api/executor/utilities/sql.py (1)
78-87: The try-except inside the for loop in query_df_with_type_infer_fallback (PERF203) causes repeated exception handling overhead for large/frequent queries, which can measurably degrade performance under load.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 2/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/executor/utilities/sql.py, lines 78-87, the try-except block is inside a for loop in `query_df_with_type_infer_fallback`, causing repeated exception handling overhead (PERF203). Refactor this logic to minimize exception handling inside the loop, such as by pre-checking conditions or restructuring the loop to avoid catching exceptions on every iteration. This will improve performance for large or frequent queries.
mindsdb/api/http/gui.py (3)
25-25: requests.get(resource["url"]) is called without a timeout, which can cause the function to hang indefinitely if the server does not respond.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/http/gui.py, line 25, the call to requests.get(resource["url"]) does not specify a timeout, which can cause the function to hang indefinitely if the server is unresponsive. Add a reasonable timeout (e.g., 10 seconds) to this call.
30-32: The get_resources function downloads resources sequentially, causing significant delays when downloading multiple large files; parallelizing with threads would greatly reduce total download time.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 8/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Parallelize resource downloads in mindsdb/api/http/gui.py, lines 30-32. The current code downloads resources sequentially, which is slow for multiple/large files. Replace the for-loop with a ThreadPoolExecutor to download all resources in parallel, raising any exceptions encountered.
22-28: requests.get(resource["url"]) downloads files from a remote S3 bucket without validating the URL or restricting the download path, allowing an attacker to supply a malicious version that could trigger SSRF or overwrite arbitrary files.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 4/5

Urgency Impact: 4/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/http/gui.py, lines 22-28, the code constructs a download URL and file path using the `version` parameter without validation, which could allow SSRF or arbitrary file overwrite if a malicious version is supplied. Add strict validation to ensure `version` only contains alphanumeric, dot, dash, or underscore characters, and reject anything else. Insert this validation at the start of `get_resources`.
mindsdb/api/http/namespaces/agents.py (3)
137-234: put method in AgentResource (lines 137-234) is overly complex with many branches and return statements, making it hard to maintain and error-prone as the codebase grows.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `put` method in `AgentResource` (mindsdb/api/http/namespaces/agents.py, lines 137-234) to reduce cyclomatic complexity and improve maintainability. Break down the method into smaller helper functions for validation, agent existence checks, and update logic. Ensure the refactored code preserves all existing functionality and error handling.
257-316: _completion_event_generator (lines 257-316) is too complex, making it hard to maintain and optimize for streaming large data or error handling at scale.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `_completion_event_generator` function (mindsdb/api/http/namespaces/agents.py, lines 257-316) to reduce cyclomatic complexity and improve maintainability. Extract error handling, chunk processing, and agent parameter setup into helper functions. Ensure the refactored code maintains streaming efficiency and robust error reporting.
102-110: request.json is accessed directly without verifying the request's content type or JSON validity, allowing attackers to trigger exceptions or denial of service by sending malformed or non-JSON payloads.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/http/namespaces/agents.py, lines 102-110, the code accesses request.json directly without checking if the request is valid JSON or has the correct Content-Type. This can allow attackers to send malformed or non-JSON payloads, causing exceptions or denial of service. Update this block to first check request.is_json, then safely parse the JSON with request.get_json(), and handle exceptions by returning a 400 error if the JSON is invalid. Replace all direct uses of request.json in this block with the parsed data.
mindsdb/api/http/namespaces/analysis.py (2)
23-41: The function analyze_df loads the entire DataFrame into memory and calls analyze_dataset(df), which can cause significant memory and CPU usage for large datasets, potentially leading to OOM or severe latency.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/http/namespaces/analysis.py, lines 23-41, the function `analyze_df` processes the entire DataFrame with `analyze_dataset(df)`, which can cause high memory and CPU usage for large datasets. Add a row limit (e.g., 10,000 rows) to the DataFrame before analysis to prevent OOM and severe latency. Only analyze up to the first 10,000 rows if the DataFrame is larger.
50-70: query from user input is passed directly to mysql_proxy.process_query(query) without sufficient sanitization, enabling SQL injection if FakeMysqlProxy does not enforce strict query validation.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 2/5

Urgency Impact: 4/5

Total Score: 10/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/http/namespaces/analysis.py, lines 50-70, user-supplied `query` is passed directly to `mysql_proxy.process_query(query)` without sanitization, creating a SQL injection risk if the proxy does not strictly validate queries. Add input validation to reject queries containing dangerous SQL keywords or patterns before processing.
mindsdb/api/http/namespaces/auth.py (3)
76-84: requests.post call on line 76 lacks a timeout, which can cause the server to hang indefinitely if the auth server is unresponsive.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/http/namespaces/auth.py, lines 76-84, the requests.post call to the auth server does not specify a timeout, which can cause the server to hang indefinitely if the auth server is unresponsive. Add a timeout=5 argument to the requests.post call to ensure the request fails gracefully if the auth server does not respond.
63-84: code parameter from request.args.get("code") is used directly in an OAuth token exchange without validation, allowing an attacker to inject arbitrary or malicious values and potentially manipulate the authentication flow.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 2/5

Urgency Impact: 4/5

Total Score: 10/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/http/namespaces/auth.py, lines 63-84, the `code` parameter from the OAuth callback is used directly in the token exchange without validation. Add input validation to ensure `code` is a non-empty alphanumeric string before using it. If invalid, redirect to `/forbidden`. Apply this fix while preserving the original formatting.
97-104: Sensitive OAuth tokens and usernames are stored in the config without encryption or secure storage, risking exposure if the config is accessed or leaked.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 2/5

Urgency Impact: 4/5

Total Score: 10/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/http/namespaces/auth.py, lines 97-104, sensitive OAuth tokens and usernames are stored in the config in plaintext. Update this to securely encrypt tokens (and optionally usernames) before storing, using a secure encryption utility. Ensure decryption is handled where tokens are read.
mindsdb/api/http/namespaces/chatbots.py (1)
17-91: create_chatbot function is overly complex (11+ branches, 10+ returns), making it hard to maintain and error-prone as the codebase grows.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `create_chatbot` function in mindsdb/api/http/namespaces/chatbots.py (lines 17-91) to reduce cyclomatic complexity and improve maintainability. The function currently has too many branches and return statements, making it hard to follow and error-prone. Break it into smaller helper functions for validation, database handling, and existence checks, and ensure the main function is concise and readable.
mindsdb/api/http/namespaces/config.py (1)
110-196: The put method in the Integration class (lines 110-196) is excessively large and complex, with many branches and statements, making it hard to maintain and error-prone as the system grows.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `put` method in the `Integration` class (mindsdb/api/http/namespaces/config.py, lines 110-196) to reduce its complexity. Break it into smaller, well-named helper methods for file handling, parameter validation, test integration, and integration creation. This will improve maintainability and reduce the risk of bugs as the codebase grows.
mindsdb/api/http/namespaces/file.py (2)
168-168: requests.get(url, stream=True) is called without a timeout, which can cause the server to hang indefinitely if the remote server is unresponsive.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/http/namespaces/file.py, line 168, the code calls `requests.get(url, stream=True)` without a timeout. This can cause the server to hang indefinitely if the remote server is unresponsive. Please add a reasonable timeout (e.g., 30 seconds) to this call: change it to `requests.get(url, stream=True, timeout=30)`.
43-204: put method in File class is extremely large and complex (95 statements, 13 returns, 29 branches), making it hard to maintain and optimize for performance.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 2/5

Urgency Impact: 2/5

Total Score: 6/15

🤖 AI Agent Prompt (Copy & Paste Ready):
The `put` method in `mindsdb/api/http/namespaces/file.py` (lines 43-204) is excessively large and complex, with 95+ statements, 13 return statements, and 29 branches. This makes the code very difficult to maintain, test, and optimize for performance. Please refactor this method by breaking it into smaller, well-named helper functions (e.g., for multipart parsing, URL validation, archive extraction, error handling), and reduce the number of return statements and branches. Ensure each helper function has a single responsibility and the main method orchestrates the flow clearly.
mindsdb/api/http/namespaces/handlers.py (1)
152-157: prepare_formdata writes uploaded files to disk one by one using os.path.join and open, which can be slow and error-prone for large files or many uploads; using Path and streaming would improve scalability and I/O efficiency.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 5/5

Urgency Impact: 2/5

Total Score: 10/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/http/namespaces/handlers.py, lines 152-157, the current code writes uploaded files to disk using os.path.join and open, reading the entire file into memory at once. This is inefficient for large files and can cause high memory usage. Refactor this block to use pathlib.Path for file paths and stream the file in chunks (e.g., 8192 bytes at a time) to reduce memory usage and improve I/O efficiency. Ensure the code preserves the original indentation and updates params[file_name] to the string path.
mindsdb/api/http/namespaces/knowledge_bases.py (2)
169-258: The put method (lines 169-258) is a large, complex function with many responsibilities, making it difficult to maintain and extend as the knowledge base update logic grows.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 5/5

Urgency Impact: 2/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `put` method in mindsdb/api/http/namespaces/knowledge_bases.py (lines 169-258) to reduce complexity and improve maintainability. Extract the dependency setup and the knowledge base update logic into two helper functions (e.g., `_setup_table_dependencies` and `_process_kb_update`). Ensure the main method is concise and delegates responsibilities. Do not change the functional behavior.
189-237: request.json is used directly without validation or sanitization, allowing attackers to inject malicious data that could be processed by downstream logic (e.g., in table.insert_query_result, table.insert_files, or table.insert_web_pages), potentially leading to code execution or data compromise if these methods are not robustly secured.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 3/5

Urgency Impact: 4/5

Total Score: 11/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/http/namespaces/knowledge_bases.py, lines 189-237, user-supplied data from `request.json["knowledge_base"]` is passed directly to methods like `insert_rows`, `insert_files`, `insert_web_pages`, and `insert_query_result` without validation or sanitization. Add strict validation to ensure only expected fields are present, and check that each field is of the correct type (e.g., lists for 'rows', 'files', 'urls'; string for 'query'). Reject any unexpected fields or invalid types with an HTTP 400 error. This prevents malicious input from being processed by downstream logic.
mindsdb/api/http/namespaces/models.py (1)
27-265: The code for checking project existence and model existence is duplicated across multiple endpoints, increasing maintenance burden and risk of inconsistent logic.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/http/namespaces/models.py, lines 27-265, there is significant code duplication for project and model existence checks across multiple endpoints. Refactor by extracting these checks into reusable helper functions or decorators to reduce maintenance burden and ensure consistent error handling.
mindsdb/api/http/namespaces/views.py (2)
28-29: Inefficient use of a for-loop to build all_view_objs can be replaced with a list comprehension for better performance and readability, especially as the number of views grows.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 5/5

Urgency Impact: 1/5

Total Score: 8/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor mindsdb/api/http/namespaces/views.py lines 28-29: Replace the explicit for-loop that builds `all_view_objs` with a list comprehension to improve performance and readability, especially for large numbers of views. The new code should assign `all_view_objs` directly using a list comprehension that extracts the required fields from each view.
45-45, 115-116: query field from user input is passed directly to project.create_view and project.update_view without validation or sanitization, enabling SQL injection if the backend executes raw SQL queries.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 2/5

Urgency Impact: 4/5

Total Score: 10/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/http/namespaces/views.py, lines 45 and 115-116, user-supplied `query` is passed directly to `project.create_view` and `project.update_view` without validation, risking SQL injection if the backend executes raw SQL. Add robust validation or sanitization for the `query` field before using it, and reject queries containing unsafe SQL patterns. Implement a function like `is_safe_sql_query` to enforce this.
mindsdb/api/mcp/__init__.py (1)
113-144: list_databases returns a list of strings on success but a dict on error, causing type inconsistency and potential runtime errors for callers expecting a consistent return type.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 4/5

Urgency Impact: 3/5

Total Score: 11/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/mcp/__init__.py, lines 113-144, the function `list_databases` returns a list of strings on success but a dict on error, causing type inconsistency and potential runtime errors for callers expecting a consistent return type. Refactor `list_databases` so that it always returns a dict, e.g., `{ "type": "table", "data": [...] }` on success and an error dict on failure. Update the function signature and all return statements accordingly.
mindsdb/api/mysql/mysql_proxy/executor/mysql_executor.py (1)
56-59: SQLQuery(self.query, session=self.session, execute=False) and sqlquery.prepare_query() are called on every stmt_prepare, which may re-parse and re-prepare queries unnecessarily for repeated statements, causing significant overhead for high-frequency prepared statements.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Optimize repeated query preparation in mindsdb/api/mysql/mysql_proxy/executor/mysql_executor.py lines 56-59. Currently, SQLQuery and prepare_query are called on every stmt_prepare, causing unnecessary overhead for repeated prepared statements. Refactor to cache the prepared SQLQuery object per unique query (e.g., using self._prepared_sqlquery) and only re-prepare if the query changes. Ensure thread safety if needed.
mindsdb/api/mysql/mysql_proxy/mysql_proxy.py (1)
169-313: The handshake method is extremely complex (17+ branches, 82+ statements), making it hard to maintain and optimize, and increasing risk of performance and correctness issues.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `handshake` method in mindsdb/api/mysql/mysql_proxy/mysql_proxy.py (lines 169-313) to reduce its complexity and improve maintainability. Extract major logic branches (e.g., password resolution, SSL handling) into private helper methods. Ensure all authentication logic and error handling is preserved, but the main method should be under 50 statements and 12 branches. Do not change the method's external interface.
mindsdb/api/mysql/mysql_proxy/utilities/dump.py (1)
396-401: dump_result_set_to_mysql uses try-except inside a loop to infer column sizes, causing significant performance overhead for large result sets.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/mysql/mysql_proxy/utilities/dump.py, lines 396-401, the code uses a try-except block inside a loop to infer column sizes, which causes significant performance overhead for large result sets. Refactor this block to avoid exception handling in the loop by explicitly checking column types before attempting string conversion. Replace the try-except with type checks and only perform the conversion when appropriate.
mindsdb/api/postgres/postgres_proxy/executor/executor.py (1)
110-160: to_postgres_columns (lines 110-160) is a large, complex function with multiple nested conditionals and type checks, making it difficult to maintain and error-prone as the codebase grows.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 5/5

Urgency Impact: 2/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `to_postgres_columns` method in mindsdb/api/postgres/postgres_proxy/executor/executor.py (lines 110-160). The function is too complex and difficult to maintain due to deeply nested type checks and repeated logic. Extract type resolution and name cleaning into helper functions within the method, and simplify the main loop for clarity and maintainability. Preserve all original functionality.
mindsdb/api/postgres/postgres_proxy/postgres_packets/errors.py (1)
1-321: The file contains a massive static dictionary (POSTGRES_ERROR_CODES) with hundreds of string mappings loaded at module import, which can cause significant memory overhead and slow startup in resource-constrained or multi-process environments.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 2/5

Urgency Impact: 2/5

Total Score: 6/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/postgres/postgres_proxy/postgres_packets/errors.py, lines 1-321, the file defines a very large static dictionary (POSTGRES_ERROR_CODES) at module import time, which can cause significant memory overhead and slow startup, especially in resource-constrained or multi-process environments. Refactor the code so that the dictionary is created lazily (on first access) via a function, and only loaded into memory when actually needed. Ensure that POSTGRES_SYNTAX_ERROR_CODE is also initialized lazily. Preserve all existing mappings and code formatting.
mindsdb/api/postgres/postgres_proxy/postgres_packets/postgres_fields.py (1)
5-13: The PostgresField.__init__ constructor has 7 parameters, which increases cognitive load and maintainability burden for future changes or extensions.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the constructor of the PostgresField class in mindsdb/api/postgres/postgres_proxy/postgres_packets/postgres_fields.py (lines 5-13) to use keyword-only arguments. This reduces cognitive load and improves maintainability for future changes. Update the __init__ signature to use *, and ensure all instantiations use keyword arguments.
mindsdb/api/postgres/postgres_proxy/postgres_packets/postgres_message.py (1)
17-23: send_internal in PostgresMessage base class raises NotImplementedError, but send always calls it, causing a runtime exception if a subclass does not override send_internal.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 2/5

Urgency Impact: 3/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/postgres/postgres_proxy/postgres_packets/postgres_message.py, lines 17-23, the base class method `send` always calls `send_internal`, which raises NotImplementedError unless overridden. This causes a runtime exception if a subclass does not implement `send_internal`. Update `send` to check if `send_internal` is implemented in the subclass before calling it, and raise a clear NotImplementedError if not.
mindsdb/api/postgres/postgres_proxy/postgres_packets/postgres_message_formats.py (2)
583-584: In Parse.read, the loop variable i is unused; using for _ in range(self.num_params) is more efficient and idiomatic, reducing confusion and minor overhead in large parameterized queries.

📊 Impact Scores:

Production Impact: 1/5

Fix Specificity: 5/5

Urgency Impact: 1/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/postgres/postgres_proxy/postgres_packets/postgres_message_formats.py lines 583-584, replace the unused loop variable `i` in `for i in range(self.num_params):` with `_` for clarity and minor efficiency, as the index is not used. This improves maintainability and avoids confusion in large parameterized queries.
71-93: AuthenticationClearTextPassword enables clear-text password authentication, exposing user credentials to interception and theft over the network.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 1/5

Urgency Impact: 2/5

Total Score: 5/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/postgres/postgres_proxy/postgres_packets/postgres_message_formats.py, lines 71-93, the `AuthenticationClearTextPassword` class enables clear-text password authentication, which exposes user credentials to interception and theft. Replace the implementation so that its constructor raises a RuntimeError indicating that clear-text authentication is not supported, and remove any code that would allow clear-text password authentication to proceed.
mindsdb/api/postgres/postgres_proxy/postgres_packets/postgres_packets.py (1)
51-58: read_string uses repeated string concatenation in a loop (result = result + b), which is O(n^2) for large strings and can cause significant performance degradation when reading large payloads.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 5/5

Urgency Impact: 2/5

Total Score: 10/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Optimize the `read_string` method in mindsdb/api/postgres/postgres_proxy/postgres_packets/postgres_packets.py (lines 51-58). The current implementation uses repeated string concatenation (`result = result + b`), which is O(n^2) and can cause major performance issues for large strings. Refactor it to use a list to collect bytes and join them at the end for O(n) performance.
mindsdb/api/postgres/postgres_proxy/postgres_proxy.py (3)
332-332: get_encoding may return a bytes object if user_encoding is set, causing .encode() calls to fail with AttributeError.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/postgres/postgres_proxy/postgres_proxy.py, line 332, the `get_encoding` method may return a bytes object if `user_encoding` is set, which will cause `.encode()` calls to fail. Update the method to decode `user_encoding` if it is bytes before returning.
401-427: to_postgres_rows (lines 401-427) is overly complex and inefficient for large result sets, with deeply nested type checks and repeated conversions per cell, causing high CPU usage and poor maintainability.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 8/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor mindsdb/api/postgres/postgres_proxy/postgres_proxy.py lines 401-427 (`to_postgres_rows`). The current implementation is overly complex and inefficient for large result sets, with deeply nested type checks and repeated conversions per cell. Replace it with a single-pass, helper-function-based approach that efficiently handles all types, reduces CPU usage, and improves maintainability. See the suggested code for a performant implementation.
447-460: The main handler loop (main_loop, lines 447-460) processes all messages serially on a single thread, which can cause resource contention and poor scalability under high concurrent client load.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 2/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Review mindsdb/api/postgres/postgres_proxy/postgres_proxy.py lines 447-460 (`main_loop`). The current design processes all client messages serially on a single thread, which can cause resource contention and poor scalability under high concurrent client load. Consider refactoring to support asynchronous or multi-threaded message processing per connection to improve throughput and user experience.
mindsdb/api/postgres/postgres_proxy/utilities/__init__.py (1)
6-6: type(x) == bytes is used for type checking, which fails for subclasses and is not Pythonic; this can cause incorrect behavior if a bytes-like object is passed.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 5/5

Urgency Impact: 2/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/postgres/postgres_proxy/utilities/__init__.py, line 6, replace the type check `if type(x) == bytes:` with `if isinstance(x, bytes):` to ensure correct handling of bytes and bytes-like objects. This prevents incorrect behavior when subclasses of bytes are passed.
mindsdb/api/postgres/start.py (1)
6-6: The start function's verbose argument is unused, which may mislead callers expecting it to affect runtime behavior.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 5/5

Urgency Impact: 2/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/api/postgres/start.py, line 6, the function 'start' defines a 'verbose' argument that is never used, which may mislead callers into thinking it affects runtime behavior. Remove the 'verbose' argument from the function definition.
mindsdb/integrations/handlers/byom_handler/byom_handler.py (1)
423-428: pickle.loads() is used to deserialize model state in ModelWrapperUnsafe without validation, allowing arbitrary code execution if attacker controls model state.

📊 Impact Scores:

Production Impact: 5/5

Fix Specificity: 3/5

Urgency Impact: 4/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/handlers/byom_handler/byom_handler.py, lines 423-428, the code uses `pickle.loads(model_state)` to deserialize model state in ModelWrapperUnsafe without validation, which allows arbitrary code execution if an attacker controls the model state. Replace all uses of `pickle.loads(model_state)` in ModelWrapperUnsafe with a `safe_loads` function that only allows deserialization of dicts, raising an error otherwise. Ensure this is used everywhere model state is loaded with pickle in this class.
mindsdb/integrations/handlers/chromadb_handler/chromadb_handler.py (1)
228-348: select method (lines 228-348) is excessively complex (23 branches, 65 statements), making it hard to maintain and error-prone as the codebase grows.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `select` method in mindsdb/integrations/handlers/chromadb_handler/chromadb_handler.py (lines 228-348). The function is overly complex (23 branches, 65 statements), which significantly impacts maintainability and increases the risk of bugs. Break it into smaller, well-named helper methods for each logical block (e.g., filter parsing, query construction, result post-processing). Ensure the refactor preserves all existing functionality and signatures.
mindsdb/integrations/handlers/druid_handler/druid_handler.py (1)
139-142: native_query fetches all results into memory with fetchall(), which can cause high memory usage or OOM for large result sets.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 2/5

Urgency Impact: 2/5

Total Score: 8/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/handlers/druid_handler/druid_handler.py, lines 139-142, the code uses `cursor.fetchall()` to load all query results into memory at once, which can cause high memory usage or OOM for large result sets. Refactor this to iterate over the cursor and build the result list incrementally, e.g., by appending each row to a list, then constructing the DataFrame from that list. Preserve the original logic and formatting.
mindsdb/integrations/handlers/file_handler/file_handler.py (2)
75-175: The query method (lines 75-175) is excessively complex with many branches and return statements, making it hard to maintain and extend.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 8/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `query` method in mindsdb/integrations/handlers/file_handler/file_handler.py (lines 75-175) to reduce cyclomatic complexity and improve maintainability. Extract major branches (DropTables, Select, Insert) into separate private methods, and minimize the number of return statements and nested conditionals. Ensure the refactored code preserves all existing functionality and error handling.
41-41: Using a mutable default argument (connection_data={}) in __init__ can cause shared state bugs and memory leaks in long-running processes.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/handlers/file_handler/file_handler.py, line 41, replace the mutable default argument `connection_data={}` in the `__init__` method with `connection_data=None`, and add logic inside the method to set it to an empty dict if None. This prevents shared state and memory issues.
mindsdb/integrations/handlers/mysql_handler/mysql_handler.py (1)
85-100: _make_table_response builds a list of Series by iterating over all columns and all rows, resulting in O(n*m) list comprehensions and memory allocations for large result sets.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Optimize the `_make_table_response` function in mindsdb/integrations/handlers/mysql_handler/mysql_handler.py (lines 85-100). The current code builds a list of Series by iterating over all columns and all rows, which is O(n*m) and causes significant memory and CPU overhead for large result sets. Refactor to construct a DataFrame directly from the result list of dicts, then cast columns to the appropriate dtypes in-place. Preserve the original logic for dtype assignment.
mindsdb/integrations/handlers/openai_handler/openai_handler.py (3)
117-119: _check_client_connection may raise a TypeError if e.body is not a dict, causing an unhandled exception instead of a clear error for invalid API keys.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/handlers/openai_handler/openai_handler.py, lines 117-119, the `_check_client_connection` method may raise a `TypeError` if `e.body` is not a dict, resulting in an unhandled exception. Update the exception handling so that if `e.body` is not a dict, it still raises a clear error message instead of failing with a `TypeError`. Ensure the error message is informative and the function never crashes due to an unexpected `e.body` type.
255-453: predict function is excessively large and complex (91+ statements, 35+ branches), making it hard to maintain and optimize, which increases risk of performance regressions and hinders future scalability.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `predict` method in mindsdb/integrations/handlers/openai_handler/openai_handler.py (lines 255-453). The function is excessively large (91+ statements, 35+ branches), making it difficult to maintain and optimize. Break it into smaller, well-named helper methods for each major mode (embedding, image, chat/completion), prompt construction, and post-processing. This will improve maintainability, testability, and reduce risk of performance regressions.
378-389: Repeated use of try/except inside a loop in predict (json_struct prompt construction) causes significant performance overhead for large datasets.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Optimize the json_struct prompt construction loop in mindsdb/integrations/handlers/openai_handler/openai_handler.py (lines 378-389). Avoid using `try/except` inside the loop for every row, as this incurs significant performance overhead on large datasets. Instead, pre-validate or pre-parse the json_struct column outside the loop, and only use try/except for truly exceptional cases.
mindsdb/integrations/handlers/oracle_handler/__init__.py (1)
14-14: type variable assignment on line 14 shadows the Python built-in type, which can cause unexpected runtime errors if type is used later as a function.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/handlers/oracle_handler/__init__.py, line 14, the variable `type` is assigned, which shadows the Python built-in `type` and can cause runtime errors if `type` is used as a function later. Please rename this variable to `handler_type` and update all references accordingly.
mindsdb/integrations/handlers/oracle_handler/oracle_handler.py (2)
73-148: _make_table_response function is overly complex (18+ branches, deep nesting), making it hard to maintain and error-prone for future changes.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 8/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `_make_table_response` function in mindsdb/integrations/handlers/oracle_handler/oracle_handler.py (lines 73-148) to reduce its cyclomatic complexity and improve maintainability. Extract the type mapping logic into a helper function and flatten the branching structure. Ensure the function remains functionally equivalent and preserves all existing logic.
235-236: cur.execute(f"ALTER SESSION SET {key} = {repr(value)}") in connect is vulnerable to SQL injection if key or value are attacker-controlled, allowing arbitrary session changes or code execution.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 2/5

Urgency Impact: 4/5

Total Score: 10/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/handlers/oracle_handler/oracle_handler.py, lines 235-236, the code uses string interpolation to construct an SQL statement for altering session variables: `cur.execute(f"ALTER SESSION SET {key} = {repr(value)}")`. This is vulnerable to SQL injection if `key` or `value` are attacker-controlled. Refactor this to use parameterized queries for the value, and strictly validate or whitelist the `key` to only allow known safe session variable names. Replace the line with: `cur.execute("ALTER SESSION SET " + key + " = :val", {"val": value})` and ensure `key` is validated against a whitelist.
mindsdb/integrations/handlers/pgvector_handler/pgvector_handler.py (2)
673-676: create_index fetches embedding dimension by querying the entire table with LIMIT 1, which can be slow for large tables and is not robust if the table is empty.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 2/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/handlers/pgvector_handler/pgvector_handler.py, lines 673-676, the `create_index` method fetches the embedding dimension by querying the table for a row with `LIMIT 1`, which is inefficient for large tables and fails if the table is empty. Refactor this logic to retrieve the embedding dimension from table metadata (e.g., store at table creation or query information_schema/pg_catalog), and avoid scanning the table. Ensure the method is robust and efficient for large/empty tables.
625-629: update method iterates over DataFrame rows and builds a list of lists for each update, which is inefficient for large batch updates and can cause high memory/CPU usage.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 2/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/handlers/pgvector_handler/pgvector_handler.py, lines 625-629, the `update` method uses a for loop over DataFrame.iterrows() to build update data, which is inefficient for large DataFrames. Refactor this to use DataFrame.itertuples() or vectorized operations, or implement a batch update SQL statement to handle large updates efficiently. Ensure the new approach minimizes memory and CPU usage for large datasets.
mindsdb/integrations/handlers/postgres_handler/postgres_handler.py (3)
305-306: The native_query method loads all query results into memory at once via cur.fetchall(), which can cause high memory usage or OOM for large result sets.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/handlers/postgres_handler/postgres_handler.py, lines 305-306, the code uses `cur.fetchall()` to load all query results into memory, which can cause high memory usage or OOM for large result sets. Refactor this to fetch results in batches (e.g., with `cur.fetchmany(fetch_size)`) and accumulate them, then build the response from the accumulated rows.
111-125: The _make_table_response function creates a list of Series for each column, which is inefficient for large result sets and can be replaced by direct DataFrame construction.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 2/5

Urgency Impact: 2/5

Total Score: 6/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/handlers/postgres_handler/postgres_handler.py, lines 111-125, `_make_table_response` builds a list of Series for each column, which is inefficient for large result sets. Refactor to construct the DataFrame directly from a dict of columns, then cast dtypes as needed, to improve performance and memory usage.
448-468: get_columns constructs SQL with unescaped table_name and schema_name, allowing SQL injection if attacker controls these values.

📊 Impact Scores:

Production Impact: 5/5

Fix Specificity: 3/5

Urgency Impact: 4/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/handlers/postgres_handler/postgres_handler.py, lines 448-468, the `get_columns` method constructs a SQL query by directly interpolating `table_name` and `schema_name` into the query string, which allows SQL injection if these values are attacker-controlled. Refactor this code to use parameterized queries (with placeholders and a params list) instead of string interpolation, and pass the parameters to `native_query`. Ensure the query is safe from injection.
mindsdb/integrations/handlers/snowflake_handler/snowflake_handler.py (3)
72-166: _make_table_response (lines 72-166) is a large, highly complex function with many branches and statements, making it difficult to maintain and error-prone as the codebase grows.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the function `_make_table_response` in mindsdb/integrations/handlers/snowflake_handler/snowflake_handler.py (lines 72-166). The function is overly large and complex, with many branches and type checks, making it hard to maintain. Break it into smaller, well-named helper functions (e.g., for type mapping, datetime handling, etc.) to improve readability and maintainability, while preserving all existing logic and behavior.
519-632: meta_get_column_statistics (lines 519-632) executes a separate large aggregate query for every table, which can cause major performance issues on databases with many tables or large tables.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 2/5

Urgency Impact: 3/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Optimize the `meta_get_column_statistics` method in mindsdb/integrations/handlers/snowflake_handler/snowflake_handler.py (lines 519-632). The current implementation issues a separate aggregate query per table, which can cause severe performance degradation on databases with many or large tables. Consider batching queries, limiting the number of tables processed, or providing a warning/limit for large-scale operations. Ensure the method remains robust for large schemas.
426-443: get_columns uses string interpolation to construct SQL with untrusted table_name, allowing SQL injection and unauthorized data access.

📊 Impact Scores:

Production Impact: 5/5

Fix Specificity: 3/5

Urgency Impact: 4/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/handlers/snowflake_handler/snowflake_handler.py, lines 426-443, the `get_columns` method constructs a SQL query using string interpolation with the `table_name` parameter, which allows SQL injection. Refactor this code to use parameterized queries (bind variables) instead of string interpolation to prevent SQL injection. Ensure the fix preserves all existing logic and functionality.
mindsdb/integrations/handlers/web_handler/urlcrawl_helpers.py (1)
208-288: The recursive function get_all_website_links_recursively (lines 208-288) is deeply nested and highly complex, making it hard to maintain and risking stack overflows or poor performance on large crawls.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the recursive function `get_all_website_links_recursively` in mindsdb/integrations/handlers/web_handler/urlcrawl_helpers.py (lines 208-288) to use an iterative approach (e.g., with a queue) instead of recursion. This will improve maintainability and scalability, preventing stack overflows and making the code easier to follow. Replace the recursive logic with an iterative version that preserves the same crawling and filtering behavior.
mindsdb/integrations/handlers/zendesk_handler/zendesk_tables.py (1)
18-361: Major code duplication: nearly identical logic for select/query handling is repeated across all table classes, increasing maintenance burden and risk of inconsistencies.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor mindsdb/integrations/handlers/zendesk_handler/zendesk_tables.py, lines 18-361. The select/query logic is nearly identical across all table classes, causing major code duplication and maintainability issues. Extract the common logic into a shared private method (e.g., _select_common), parameterized by resource name, and have each select method delegate to it. Ensure all select methods remain functionally equivalent.
mindsdb/integrations/libs/base.py (1)
202-206: table_name used in error logging inside the for future in ... loop may be undefined or incorrect, leading to misleading error messages or exceptions if an error occurs before assignment.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 8/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/libs/base.py, lines 202-206, the variable `table_name` is used in error logging inside the loop over futures, but it may be undefined or refer to the wrong table if an exception occurs before assignment. Update the error and exception logging to avoid referencing `table_name` directly; instead, use a generic message or extract the table name from the future's context if possible. Replace the log messages to not use `table_name` directly.
mindsdb/integrations/libs/llm/utils.py (1)
287-288: batch[messages_col] is accessed before checking if messages_col exists in batch, which can cause a KeyError and mask the intended error message.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 11/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/libs/llm/utils.py, lines 287-288, the code accesses `batch[messages_col]` before checking if `messages_col` exists in `batch`, which can cause a KeyError and mask the intended error message. Please swap the order of the checks so that the existence of `messages_col` is verified before accessing it. The check for `messages_col not in batch` should come before `isinstance(batch[messages_col], list)`.
mindsdb/integrations/libs/ml_handler_process/create_engine_process.py (1)
25-28: result will be None if module.Handler does not have create_engine, causing the function to always return None instead of raising or handling the missing method, which may silently mask integration errors.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 11/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/libs/ml_handler_process/create_engine_process.py, lines 25-28: The function returns None if module.Handler does not have 'create_engine', which can silently mask integration errors. Instead, raise an AttributeError if 'create_engine' is missing. Update the code so that if 'result' is None, an AttributeError is raised with a clear message.
mindsdb/integrations/libs/ml_handler_process/func_call_process.py (1)
23-24: func_call_process swallows all exceptions except NotImplementedError, re-raises them, but this can mask the original stack trace and cause misleading error reporting.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 5/5

Urgency Impact: 2/5

Total Score: 10/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/libs/ml_handler_process/func_call_process.py, lines 23-24, the code uses 'raise e' which can mask the original stack trace. Change it to 'raise' to preserve the original exception context.
mindsdb/integrations/libs/ml_handler_process/learn_process.py (1)
27-161: learn_process function is excessively large and complex (82 statements, 17 branches, 16 complexity), making it hard to maintain and optimize, and increasing risk of performance regressions.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor mindsdb/integrations/libs/ml_handler_process/learn_process.py, lines 27-161. The `learn_process` function is excessively large and complex (82 statements, 17 branches, 16 complexity), making it hard to maintain and optimize. Extract the data fetching logic (the block that handles different `data_integration_ref['type']` cases and builds the DataFrame) into a separate helper function, e.g., `_fetch_training_data`. Replace the original block in `learn_process` with a call to this helper. Ensure the new function is placed above `learn_process` and preserves all original logic and error handling. Do not change the function's external interface.
mindsdb/integrations/libs/ml_handler_process/update_process.py (1)
24-25: update_process swallows all exceptions except NotImplementedError, re-raising them, which can crash the process and prevent proper error handling/logging.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/libs/ml_handler_process/update_process.py, lines 24-25, the code re-raises all exceptions except NotImplementedError, which can crash the process and prevent proper error handling. Change the except block to catch all exceptions and return None instead of re-raising, to ensure the function fails gracefully.
mindsdb/integrations/libs/response.py (1)
107-107: The method self.data_frame.replace([numpy.NaN, pandas.NA], None, inplace=True) (line 107) uses inplace=True, which is discouraged due to inconsistent behavior and can cause unnecessary memory usage for large DataFrames.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 5/5

Urgency Impact: 2/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/libs/response.py, on line 107, replace the use of `inplace=True` in `self.data_frame.replace([numpy.NaN, pandas.NA], None, inplace=True)` with an assignment to avoid inconsistent behavior and unnecessary memory usage for large DataFrames. Change it to: `self.data_frame = self.data_frame.replace([numpy.NaN, pandas.NA], None)`.
mindsdb/integrations/libs/vectordatabase_handler.py (5)
354-354: do_upsert uses df_update.apply(keep_created_at, axis=1) but does not assign the result, so DataFrame rows are not updated, causing stale or missing metadata for updated records (impacts correctness and performance for large upserts).

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/libs/vectordatabase_handler.py, line 354, the code applies `keep_created_at` to `df_update` but does not assign the result, so DataFrame rows are not updated. Update this line to assign the result back to `df_update` to ensure metadata is correctly updated for all rows.
329-333: do_upsert fetches all existing IDs with self.select and then uses df[id_col].isin(existed_ids) for filtering, which is O(n*m) and inefficient for large DataFrames; this can cause major slowdowns on large upserts.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 5/5

Urgency Impact: 2/5

Total Score: 10/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/libs/vectordatabase_handler.py, lines 329-333, `existed_ids` is a list and used with `isin`, which is O(n*m) for large DataFrames. Convert `existed_ids` to a set before using with `isin` to improve lookup performance and scalability.
306-308: do_upsert uses a for-loop with range(len(df)) and df.loc[i, ...] to fill missing IDs, which is slow for large DataFrames; vectorized operations should be used for substantial performance gains.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 5/5

Urgency Impact: 2/5

Total Score: 10/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/libs/vectordatabase_handler.py, lines 306-308, a for-loop is used to fill missing IDs, which is inefficient for large DataFrames. Replace the loop with a vectorized approach using pandas masking and `apply` for better performance.
298-299: do_upsert uses hashlib.md5 for ID generation, which is insecure and slow for large-scale deduplication; use a faster, non-cryptographic hash like xxhash or Python's built-in hash for substantial performance improvement.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 8/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/libs/vectordatabase_handler.py, lines 298-299, `hashlib.md5` is used for ID generation, which is slow and unnecessary for non-cryptographic deduplication. Replace with a faster hash function like `xxhash.xxh64` for better performance on large datasets.
283-368: do_upsert is a large, complex function (13+ branches, multiple responsibilities) that is difficult to maintain and reason about; this increases risk of bugs and hinders future performance optimizations.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/libs/vectordatabase_handler.py, lines 283-368, the `do_upsert` function is overly large and complex, handling ID generation, deduplication, metadata management, and upsert logic in one place. Refactor this function into smaller, well-named helper methods to improve maintainability and enable future performance optimizations.
mindsdb/integrations/utilities/files/file_reader.py (6)
47-48: decode reads the entire file object with file_obj.read(), which will return a str if the file is opened in text mode, but the code assumes it is always bytes, leading to a crash if a text-mode file is passed.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 4/5

Urgency Impact: 3/5

Total Score: 11/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/utilities/files/file_reader.py, lines 47-48, the `decode` function reads the file object assuming it returns bytes, but if a text-mode file is passed, it will return a string, causing downstream decode errors. Add a check to encode to bytes if `byte_str` is a string, so the function always works with bytes.
213-214: is_parquet reads the last 4 bytes with data.read(), which may read more than 4 bytes, causing incorrect signature comparison and false negatives.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 5/5

Urgency Impact: 2/5

Total Score: 10/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/utilities/files/file_reader.py, lines 213-214, the `is_parquet` method reads the last 4 bytes of the file to check the Parquet signature, but uses `data.read()` which reads to EOF, not just 4 bytes. Change `data.read()` to `data.read(4)` to ensure only the last 4 bytes are compared.
269-284: Multiple calls to self.file_obj.seek(0) before every read operation in FileReader methods cause unnecessary I/O overhead, especially for large files or remote file-like objects.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 8/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Reduce unnecessary I/O in mindsdb/integrations/utilities/files/file_reader.py lines 269-284: The `get_contents` method always calls `self.file_obj.seek(0)` before reading, which can be expensive for large or remote files. Refactor to only seek if the file object supports it and avoid redundant seeks if the file pointer is already at the start. Apply similar logic to other methods that repeatedly seek to 0.
358-359: read_json uses json.loads(file_obj.read()) on untrusted input, allowing resource exhaustion or DoS via large or malicious JSON files.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 3/5

Urgency Impact: 3/5

Total Score: 10/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/utilities/files/file_reader.py, lines 358-359, the code reads and parses the entire JSON file without size limits, allowing attackers to upload huge or malicious JSON files that can exhaust memory or crash the service. Add a reasonable read size limit (e.g., 10MB) and handle JSONDecodeError to prevent resource exhaustion and improve error handling.
341-342: read_pdf uses fitz.open(stream=file_obj.read()) without file type validation, allowing attackers to supply non-PDF files and potentially trigger parser exploits.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 3/5

Urgency Impact: 4/5

Total Score: 11/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/utilities/files/file_reader.py, lines 341-342, the code opens any file with fitz (PyMuPDF) without checking if it's a valid PDF. This allows attackers to upload non-PDF files, which could exploit vulnerabilities in the PDF parser. Add a check for the PDF header (e.g., '%PDF-') before opening the file, and raise FileProcessingError if the check fails.
373-374: read_xlsx passes untrusted file objects directly to pd.ExcelFile, risking exploitation of vulnerabilities in the Excel parser by malicious files.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 2/5

Urgency Impact: 3/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/utilities/files/file_reader.py, lines 373-374, the code opens any file with pd.ExcelFile without checking if it's a valid XLSX file. This allows attackers to upload arbitrary files, which could exploit vulnerabilities in the Excel parser. Add a check for the XLSX ZIP header (e.g., 'PK\x03\x04') before opening the file, and raise FileProcessingError if the check fails.
mindsdb/integrations/utilities/rag/rerankers/base_reranker.py (2)
164-182: try-except inside a loop in _backoff_wrapper (lines 164-182) incurs significant performance overhead for large batches, as exception handling is expensive in Python and can slow down high-throughput reranking.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 1/5

Urgency Impact: 2/5

Total Score: 5/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/utilities/rag/rerankers/base_reranker.py, lines 164-182, the `_backoff_wrapper` method uses a `try`-`except` block inside a loop, which causes significant performance overhead for large batches due to expensive exception handling in Python. Refactor this code to minimize the time spent inside the `try` block and avoid unnecessary exception handling within the loop, while preserving the retry/backoff logic.
121-160: Batch size in _rank (lines 121-160) is set to max_concurrent_requests * 2, which can cause memory spikes and resource contention when processing very large datasets, impacting scalability.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 5/5

Urgency Impact: 2/5

Total Score: 10/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/utilities/rag/rerankers/base_reranker.py, lines 121-160, the batch size in `_rank` is set to `max_concurrent_requests * 2`, which can cause memory spikes and resource contention for large datasets. Change the batch size to `max_concurrent_requests` to better control resource usage and improve scalability.
mindsdb/integrations/utilities/rag/rerankers/reranker_compressor.py (1)
22-81: The function acompress_documents is too complex (12 > 10), making it hard to maintain and extend as logic grows.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `acompress_documents` method in mindsdb/integrations/utilities/rag/rerankers/reranker_compressor.py (lines 22-81). The function is too complex (12 > 10 according to linter), making it difficult to maintain. Break it into smaller helper methods for error handling, filtering, and callback management to improve readability and maintainability.
mindsdb/integrations/utilities/rag/retrievers/sql_retriever.py (2)
418-568: _breadth_first_search (lines 418-568) is a very large, deeply nested function with high cyclomatic complexity, making it difficult to maintain and optimize; this impedes future performance improvements and increases risk of subtle inefficiencies.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `_breadth_first_search` method in `mindsdb/integrations/utilities/rag/retrievers/sql_retriever.py` (lines 418-568). The function is excessively large and deeply nested, making it hard to maintain and optimize. Break it into smaller, well-named helper methods for each logical phase (table ranking, column ranking, value ranking, filtering), and reduce cyclomatic complexity. Ensure the refactor preserves all logic and performance characteristics.
186-286: _prepare_value_prompt (lines 186-286) is highly complex with many branches, making it difficult to maintain and optimize for performance as requirements grow.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `_prepare_value_prompt` method in `mindsdb/integrations/utilities/rag/retrievers/sql_retriever.py` (lines 186-286). The function is highly complex with many branches, making it hard to maintain and optimize. Break it into smaller helper functions for each value type and reduce cyclomatic complexity, while preserving all logic and output.
mindsdb/integrations/utilities/rag/splitters/file_splitter.py (2)
44-48: markdown_splitter and html_splitter are instantiated as default values in the dataclass, causing all instances to share the same mutable splitter objects, which can lead to cross-instance state corruption and unpredictable splitting behavior.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 2/5

Urgency Impact: 3/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/integrations/utilities/rag/splitters/file_splitter.py, lines 44-48, the dataclass FileSplitterConfig sets default values for 'markdown_splitter' and 'html_splitter' by directly instantiating MarkdownHeaderTextSplitter and HTMLHeaderTextSplitter. This causes all instances to share the same mutable objects, leading to potential cross-instance state bugs. Change the default values for both fields to None instead of instantiating them directly.
88-118: The split_documents method processes each document sequentially, which can cause significant delays when splitting large numbers of documents or very large files; parallelization could yield substantial throughput improvements.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 2/5

Urgency Impact: 2/5

Total Score: 6/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Parallelize the `split_documents` method in mindsdb/integrations/utilities/rag/splitters/file_splitter.py (lines 88-118). Currently, documents are split sequentially, which is slow for large batches. Refactor to use ThreadPoolExecutor to process documents in parallel, preserving error handling and result aggregation. Ensure the function remains thread-safe and returns the same output structure.
mindsdb/interfaces/agents/agents_controller.py (1)
198-364,365-558: add_agent and update_agent methods are excessively large and complex (over 60 and 75 statements respectively), making them hard to maintain and error-prone as the codebase grows.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `add_agent` (lines 198-364) and `update_agent` (lines 365-558) methods in mindsdb/interfaces/agents/agents_controller.py to reduce their complexity. Each method currently contains too many branches and statements, making them difficult to maintain. Extract logical sub-steps (such as skill validation, parameter processing, and association management) into well-named helper methods within the class. Ensure the refactoring preserves all existing functionality and side effects.
mindsdb/interfaces/agents/litellm_server.py (1)
33-36: Global variables agent_wrapper and mcp_session are used for shared state, which is not thread-safe and can cause race conditions or data corruption under concurrent requests, especially with FastAPI's async server.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 2/5

Urgency Impact: 3/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor mindsdb/interfaces/agents/litellm_server.py lines 33-36 to avoid using global variables (`agent_wrapper`, `mcp_session`) for shared state. Instead, use FastAPI's `app.state` or dependency injection to store and access these objects in a thread-safe manner. Ensure all endpoints and initialization logic are updated to use the new approach, preventing race conditions and data corruption under concurrent requests.
mindsdb/interfaces/agents/mcp_client_agent.py (3)
62-62: asyncio.get_event_loop() will raise RuntimeError in Python 3.10+ if no event loop is set in the current thread, causing agent initialization and completions to fail in new threads or some environments.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/agents/mcp_client_agent.py, on line 62, replace the use of `asyncio.get_event_loop()` with logic that first tries `asyncio.get_running_loop()`, and if that fails, creates and sets a new event loop. This prevents `RuntimeError` in Python 3.10+ when no event loop is set in the current thread. Apply this fix to all similar usages in the file.
120-137: The _langchain_tools_from_skills method synchronously blocks the event loop for MCP connection, which can cause major latency and resource contention if called frequently or in parallel agent instantiations.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 2/5

Urgency Impact: 3/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/agents/mcp_client_agent.py, lines 120-137, refactor `_langchain_tools_from_skills` to use an async implementation (`_langchain_tools_from_skills_async`) and call it with `asyncio.run()`. This avoids blocking the event loop and reduces resource contention when initializing MCP connections, especially under high concurrency.
28-51: query parameter in MCPQueryTool._arun is passed directly to the MCP server without sanitization, enabling SQL injection if user input is not strictly controlled upstream.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 2/5

Urgency Impact: 4/5

Total Score: 10/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/agents/mcp_client_agent.py, lines 28-51, the `MCPQueryTool._arun` method passes the `query` parameter directly to the MCP server, which could allow SQL injection if user input is not strictly controlled. Add input validation to reject queries containing dangerous SQL patterns (such as ';--', '--', '/*', '*/', 'drop ', 'delete ', 'insert ', 'update ', 'alter ', ';', or null bytes) before sending them to the MCP server. Ensure the validation is performed before the call to `self.session.call_tool`.
mindsdb/interfaces/agents/mindsdb_database_agent.py (1)
76-79: get_table_info_no_throw will raise a TypeError if table_names is None because it tries to get its length and iterate over it without checking for None.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/agents/mindsdb_database_agent.py, lines 76-79, the function `get_table_info_no_throw` assumes `table_names` is not None, but its default is None. This will cause a TypeError if called without arguments. Please add a check so that the for-loop is only executed if `table_names` is not None.
mindsdb/interfaces/agents/run_mcp_agent.py (1)
88-201: Function main (lines 88-201) is excessively large and complex (13 branches, 72 statements), making it hard to maintain and reason about, which will significantly hinder future scalability and debugging.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 8/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `main` function in mindsdb/interfaces/agents/run_mcp_agent.py (lines 88-201) to reduce its complexity and size. Extract major logical blocks (argument parsing, agent setup, direct SQL execution, interactive loop, error handling, and cleanup) into well-named helper functions. Ensure each function has a single responsibility and the main function orchestrates them. This will improve maintainability and scalability.
mindsdb/interfaces/chatbot/chatbot_task.py (1)
88-154: The on_message and _on_message methods process and log every message synchronously, which can block the main thread and degrade responsiveness under high message volume.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 3/5

Urgency Impact: 3/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/chatbot/chatbot_task.py, lines 88-154, the `on_message` and `_on_message` methods process and log every message synchronously, which can block the main thread and degrade responsiveness under high message volume. Refactor `on_message` so that message processing (including holding message and response generation) is performed in a background thread, ensuring the main thread remains responsive. Use Python's `threading.Thread` to run the message processing logic asynchronously.
mindsdb/interfaces/chatbot/polling.py (1)
50-85: The MessageCountPolling.run method (lines 50-85) iterates over all chat parameters and all chat IDs on every poll, which can cause O(n*m) performance for large numbers of chats and parameters, leading to high CPU usage and latency as the number of chats grows.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 2/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Optimize the `run` method in `mindsdb/interfaces/chatbot/polling.py` (lines 50-85). The current implementation iterates over all chat parameters and all chat IDs on every poll, resulting in O(n*m) performance for large numbers of chats and parameters. Refactor the method to batch process chat IDs and minimize per-iteration overhead, ideally by grouping chat IDs and reducing the number of nested loops. If possible, batch fetch chat memories to further improve efficiency. Ensure the new implementation maintains the same functionality and error handling.
mindsdb/interfaces/data_catalog/data_catalog_loader.py (1)
148-155,195-202,260-267,313-328: Multiple O(n*m) lookups in _add_column_metadata, _add_column_statistics, _add_primary_keys, and _add_foreign_keys due to repeated next((table.id for table in tables if table.name == ...) and similar for columns; this is a major scalability bottleneck for large catalogs.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Optimize the performance of the following functions in mindsdb/interfaces/data_catalog/data_catalog_loader.py: _add_column_metadata (lines 148-155), _add_column_statistics (lines 195-202), _add_primary_keys (lines 260-267), and _add_foreign_keys (lines 313-328). Replace repeated O(n*m) lookups like `next((table.id for table in tables if table.name == ...))` and similar for columns with precomputed dictionaries for O(1) access. Build lookup maps at the start of each function and use them in the loops. This will significantly improve scalability for large catalogs.
mindsdb/interfaces/database/database.py (1)
64-89: The get_list method appends to result in a loop; using list.extend with a list comprehension for projects and integrations would reduce interpreter overhead and improve performance for large datasets.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 5/5

Urgency Impact: 2/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/database/database.py, lines 64-89, refactor the two for-loops that append to the `result` list in `get_list` to use `list.extend` with list comprehensions. This will reduce interpreter overhead and improve performance for large numbers of projects and integrations. Replace the loops with a single `result.extend([...])` for each.
mindsdb/interfaces/database/integrations.py (1)
283-359: The _get_integration_record_data method (lines 283-359) is highly complex with many branches, making it difficult to maintain and optimize for performance or future scalability.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 8/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `_get_integration_record_data` method in mindsdb/interfaces/database/integrations.py (lines 283-359) to reduce cyclomatic complexity and improve maintainability. Break out major branches (e.g., secret masking, file system operations, handler meta extraction) into helper methods. Ensure the refactored code preserves all existing logic and performance characteristics.
mindsdb/interfaces/database/projects.py (2)
428-447: get_tables calls get_models, get_views, get_agents, and get_knowledge_bases sequentially, each making separate DB queries, causing N+1 query inefficiency and high latency for large projects.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 2/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Optimize the `get_tables` method in mindsdb/interfaces/database/projects.py (lines 428-447) to avoid N+1 query inefficiency. Refactor so that all required data (models, views, agents, knowledge bases) are fetched in as few database queries as possible, ideally batching them within a single session or using eager loading. This will significantly reduce DB roundtrips and improve performance for large projects.
141-223,449-499: combine_view_select and get_columns have high cyclomatic complexity and too many branches, making them hard to maintain and optimize, which impacts long-term scalability and code quality.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `combine_view_select` (lines 141-223) and `get_columns` (lines 449-499) methods in mindsdb/interfaces/database/projects.py to reduce cyclomatic complexity and the number of branches. Break down these functions into smaller, well-named helper methods to improve maintainability and enable future performance optimizations.
mindsdb/interfaces/database/views.py (1)
111-119: The list method (lines 100-119) uses a for-loop to build a list of dicts from a SQLAlchemy query, which can be replaced with a list comprehension for improved performance and readability, especially when handling large datasets.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 5/5

Urgency Impact: 1/5

Total Score: 8/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/database/views.py, lines 111-119, replace the for-loop that appends dicts to the `data` list with a list comprehension to improve performance and readability, especially for large datasets. The new code should build `data` in a single list comprehension.
mindsdb/interfaces/file/file_controller.py (2)
72-74: save_file loads all file metadata and names into memory to check for duplicates, which is inefficient for large numbers of files and can cause high memory and DB usage.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 5/5

Urgency Impact: 2/5

Total Score: 10/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/file/file_controller.py, lines 72-74, `save_file` loads all file metadata and names into memory to check for duplicates, which is inefficient for large numbers of files. Replace this with a direct database query that checks for the existence of the file by name and company_id, to reduce memory and DB usage.
131-131: get_file_pages uses list(tables.values())[0] to get the first value, which creates a full list and is O(n) in memory for large tables, causing unnecessary memory usage.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 5/5

Urgency Impact: 2/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/file/file_controller.py, line 131, `get_file_pages` uses `list(tables.values())[0]` to get the first value, which is inefficient for large tables as it creates a full list in memory. Replace with `next(iter(tables.values()))` to avoid unnecessary memory usage.
mindsdb/interfaces/functions/controller.py (1)
32-32: function_maker's lambda for 4 arguments incorrectly passes arg_2 twice, omitting arg_3, causing wrong argument mapping and potential runtime errors for 4-argument functions.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/functions/controller.py, line 32, the lambda for 4 arguments in `function_maker` incorrectly passes `arg_2` twice instead of `arg_3`. Replace `other_function(arg_0, arg_1, arg_2, arg_2)` with `other_function(arg_0, arg_1, arg_2, arg_3)` to ensure correct argument mapping.
mindsdb/interfaces/functions/to_markdown.py (3)
44-44: requests.get and requests.head are called without a timeout, which can cause the program to hang indefinitely if the remote server is unresponsive.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/functions/to_markdown.py, line 44, the call to `requests.get` does not specify a timeout. This can cause the program to hang indefinitely if the server does not respond. Add a reasonable timeout parameter (e.g., 10 seconds) to the `requests.get` call.
89-109: The recursive parse_element function for XML-to-Markdown conversion can cause stack overflows or severe slowdowns on deeply nested or very large XML files due to unbounded recursion and repeated string concatenation.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 3/5

Urgency Impact: 3/5

Total Score: 10/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the function `parse_element` in mindsdb/interfaces/functions/to_markdown.py (lines 89-109) to use an iterative approach instead of recursion. The current recursive implementation can cause stack overflows or severe slowdowns on deeply nested or very large XML files. Replace it with an iterative version using a stack (e.g., collections.deque) to traverse the XML tree and build the markdown output efficiently. Preserve the markdown formatting and output structure.
110-110: ET.fromstring(file_content.read().decode("utf-8")) uses the standard xml.etree.ElementTree parser, which is vulnerable to XML External Entity (XXE) attacks and other XML-based exploits if untrusted XML is processed.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 3/5

Urgency Impact: 4/5

Total Score: 11/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/functions/to_markdown.py, line 110, the code uses `xml.etree.ElementTree.fromstring` to parse XML, which is vulnerable to XML External Entity (XXE) and other XML-based attacks if untrusted XML is processed. Replace the usage of `xml.etree.ElementTree` with `defusedxml.ElementTree` for secure XML parsing. Update the import and change the parsing call accordingly.
mindsdb/interfaces/jobs/jobs_controller.py (3)
269-284: get_list uses a for-loop to build data instead of a list comprehension, which is inefficient for large job tables and increases memory/CPU usage.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 5/5

Urgency Impact: 1/5

Total Score: 8/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/jobs/jobs_controller.py, lines 269-284, replace the for-loop that appends to `data` with a list comprehension to improve performance when handling large job tables. This reduces memory and CPU overhead. Use the provided commitable suggestion.
441-542: execute_task_local is a very large, complex function (69+ statements, 14+ branches), making it hard to maintain and optimize, increasing risk of performance and logic errors.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 8/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/jobs/jobs_controller.py, lines 441-542, the `execute_task_local` method is excessively large and complex (69+ statements, 14+ branches). Refactor it by extracting logical blocks (such as job condition evaluation, SQL execution, and error handling) into smaller helper methods. This will improve maintainability, testability, and reduce the risk of performance and logic errors.
104-195: query and if_query parameters in JobsController.add are parsed but not sanitized or validated for dangerous SQL, enabling SQL injection if user input is passed directly.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 3/5

Urgency Impact: 4/5

Total Score: 11/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/jobs/jobs_controller.py, lines 104-195, the `add` method parses and stores user-supplied SQL in `query` and `if_query` without sanitization, allowing SQL injection if untrusted input is passed. Add a helper method `_is_safe_sql` to check for dangerous SQL patterns, and call it before parsing each SQL statement in both `query` and `if_query`. Raise a ValueError if unsafe SQL is detected. Ensure the helper is added to the class and used in both loops.
mindsdb/interfaces/jobs/scheduler.py (3)
54-56: stop_thread does not join the worker thread, which may cause resource leaks or unclean shutdown if the thread is still running.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 11/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/jobs/scheduler.py, lines 54-56, the `stop_thread` method only puts an exit message on the queue but does not join the worker thread. This can cause resource leaks or unclean shutdowns if the thread is still running. Please update `stop_thread` to join the thread after sending the exit signal.
105-113: Busy-wait loop in execute_task (while True with blocking q_out.get(timeout=3)) can cause unnecessary CPU usage and delays, especially under high load or with many concurrent tasks.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/jobs/scheduler.py, lines 105-113, the busy-wait loop in `execute_task` uses a short timeout (3s) with `q_out.get(timeout=3)`, causing unnecessary CPU usage and frequent DB writes under load. Increase the timeout to a higher value (e.g., 30s) to reduce resource usage and DB contention, while still updating the status periodically.
72-72: random.randint(1, 10) is used for timing jitter in scheduling (line 72), but Python's random module is not cryptographically secure and can be predicted, enabling timing attacks or job execution prediction in multi-tenant environments.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 8/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/jobs/scheduler.py, line 72, replace the use of `random.randint(1, 10)` with a cryptographically secure random number generator such as `secrets.randbelow(10) + 1` to prevent predictable timing jitter that could be exploited for timing attacks or job execution prediction. Update the import and the sleep line accordingly.
mindsdb/interfaces/knowledge_base/controller.py (3)
712-822: The _adapt_column_names method (lines 712-822) is overly complex, with many branches and statements, making it hard to maintain and optimize for large dataframes.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `_adapt_column_names` method in mindsdb/interfaces/knowledge_base/controller.py (lines 712-822). The function is overly complex, with many branches and statements, making it difficult to maintain and optimize for large dataframes. Break it into smaller helper functions for each logical step (e.g., id column adaptation, content/metadata column mapping, metadata conversion). Ensure the refactored code maintains all current functionality and performance.
614-711: The insert method (lines 614-711) is too complex, with many branches and logic paths, making it difficult to maintain and optimize for high-throughput insert operations.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 8/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `insert` method in mindsdb/interfaces/knowledge_base/controller.py (lines 614-711). The function is too complex, with many branches and logic paths, making it difficult to maintain and optimize for high-throughput insert operations. Split the method into smaller, well-named helper functions for each major step (e.g., document conversion, preprocessing, duplicate filtering, embedding calculation, upsert/insert logic). Ensure the refactored code preserves all current logic and performance.
1049-1210: The add method (lines 1049-1210) in KnowledgeBaseController is excessively complex, with too many branches and statements, making it hard to maintain and error-prone for future scalability.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `add` method in mindsdb/interfaces/knowledge_base/controller.py (lines 1049-1210). The function is excessively complex, with too many branches and statements, making it hard to maintain and error-prone for future scalability. Break it into smaller, well-named helper methods for each major logical section (e.g., parameter validation, preprocessing config handling, embedding model creation, vector DB setup, reranker validation). Ensure the refactored code preserves all existing logic and performance.
mindsdb/interfaces/knowledge_base/evaluate.py (2)
291-291: EvaluateRerank.evaluate and EvaluateDocID.evaluate do not check for required columns in test_data, which can cause runtime KeyError if columns are missing.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/knowledge_base/evaluate.py, lines 291-291, add a check at the start of EvaluateRerank.evaluate to ensure that the input DataFrame contains both 'question' and 'answer' columns. If either is missing, raise a KeyError with a message listing the columns present. This prevents runtime KeyError when accessing missing columns.
47-77: sanitize_json_response uses re.sub to strip markdown code blocks but does not limit the size or depth of the input, allowing attackers to supply extremely large or deeply nested JSON, potentially leading to denial-of-service (DoS) via resource exhaustion.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/knowledge_base/evaluate.py, lines 47-77, the function `sanitize_json_response` does not limit the size or depth of the input, allowing attackers to supply extremely large or deeply nested JSON, potentially leading to denial-of-service (DoS) via resource exhaustion. Update the function to reject responses larger than 100KB and limit the number of parsing attempts to 1000 to prevent resource exhaustion. See the code suggestion for details.
mindsdb/interfaces/knowledge_base/executor.py (1)
274-346: execute_blocks is overly complex (C901/PLR0912, 17+ branches), making it hard to maintain and optimize, and increasing risk of performance bugs in query planning.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 8/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor mindsdb/interfaces/knowledge_base/executor.py lines 274-346: The `execute_blocks` method is too complex (17+ branches, C901/PLR0912), making it hard to maintain and optimize, and increasing risk of performance bugs. Extract the AND/OR logic into private helper methods (`_execute_and_block`, `_execute_or_block`) to reduce cyclomatic complexity and improve maintainability. Ensure the refactor preserves all logic and performance characteristics.
mindsdb/interfaces/knowledge_base/preprocessing/json_chunker.py (1)
376-432: _extract_fields_to_metadata is excessively complex (17+ branches, 60+ lines), making it hard to maintain and error-prone for future changes.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 8/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the function `_extract_fields_to_metadata` in mindsdb/interfaces/knowledge_base/preprocessing/json_chunker.py (lines 376-432) to reduce its cyclomatic complexity and number of branches. The function is currently too complex (17+ branches, 60+ lines), making it hard to maintain. Extract repeated logic into helper functions, minimize nested conditionals, and ensure the function is concise and maintainable while preserving all existing functionality.
mindsdb/interfaces/model/model_controller.py (1)
247-293: prepare_create_statement and related model creation logic do not sanitize or validate statement fields, allowing injection of malicious SQL or code via user-controlled input.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 2/5

Urgency Impact: 4/5

Total Score: 10/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/model/model_controller.py, lines 247-293, the `prepare_create_statement` function and related model creation logic do not sanitize or validate user-controlled fields from `statement`, allowing injection attacks. Add validation to ensure `project_name` and `model_name` are safe identifiers, and sanitize all keys/values in `statement.using` to prevent malicious input.
mindsdb/interfaces/query_context/context_controller.py (3)
555-557: list_queries loads all query records into memory and instantiates a RunningQuery for each, which can cause high memory/CPU usage for large datasets.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 2/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/query_context/context_controller.py, lines 555-557, the `list_queries` method loads all query records into memory and instantiates a `RunningQuery` for each, which can cause high memory and CPU usage for large datasets. Refactor this method to use a generator with `yield_per` to stream results efficiently, reducing memory footprint and improving scalability.
534-540: create_query deletes old queries in a loop, causing N+1 database operations and high DB load for many expired queries.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 2/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/query_context/context_controller.py, lines 534-540, the `create_query` method deletes old queries in a loop, causing N+1 database operations and high DB load for many expired queries. Refactor this to perform bulk deletion of related tasks and queries using a single query for each, significantly reducing database round-trips and improving performance.
240-241: get_state uses pickle.loads(data) to deserialize data from cache without validation, allowing remote code execution if an attacker controls the cache contents.

📊 Impact Scores:

Production Impact: 5/5

Fix Specificity: 2/5

Urgency Impact: 4/5

Total Score: 11/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/query_context/context_controller.py, lines 240-241, the code uses `pickle.loads(data)` to deserialize data from cache, which is unsafe and can lead to remote code execution if the cache is compromised. Update this code to either use a safer serialization format (like JSON, if possible), or at minimum, wrap the deserialization in a try/except block and raise an error if deserialization fails, to prevent code execution from malicious data.
mindsdb/interfaces/skills/custom/text2sql/mindsdb_sql_toolkit.py (1)
21-225: The get_tools method is a single, large function (over 200 lines) that combines SQL and knowledge base tool construction, making it difficult to maintain and extend.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor mindsdb/interfaces/skills/custom/text2sql/mindsdb_sql_toolkit.py, lines 21-225. The `get_tools` method is a large, complex function that combines SQL and knowledge base tool construction, making it hard to maintain and extend. Split it into smaller helper methods (e.g., `_get_sql_tools`, `_get_kb_tools`) for each tool group, and have `get_tools` orchestrate their combination. Preserve all logic and docstrings, but improve maintainability by reducing function size and complexity.
mindsdb/interfaces/skills/skill_tool.py (1)
120-357: _make_text_to_sql_tools (lines 120-357) is a very large, complex function with many branches and responsibilities, making it difficult to maintain and optimize for performance as the codebase grows.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `_make_text_to_sql_tools` method in mindsdb/interfaces/skills/skill_tool.py (lines 120-357). The function is overly large and complex, with many branches and responsibilities (parsing, deduplication, database/knowledge base extraction, toolkit construction). Break it into smaller, well-named helper methods (e.g., for table/KB extraction, deduplication, agent/database struct building) to improve maintainability and enable future performance optimizations.
mindsdb/interfaces/skills/skills_controller.py (1)
72-72: Unnecessary list comprehension in get_skills for project IDs causes O(n) memory use and extra iteration for large project lists.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 5/5

Urgency Impact: 2/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/skills/skills_controller.py, line 72, replace the unnecessary list comprehension 'project_ids = list([p.id for p in projects])' with 'project_ids = [p.id for p in projects]'. This avoids an extra O(n) iteration and memory allocation for large project lists.
mindsdb/interfaces/skills/sql_agent.py (2)
588-588: _get_sample_rows (lines 586-610) constructs SQL queries using direct string interpolation of table and fields, allowing SQL injection if attacker controls these values.

📊 Impact Scores:

Production Impact: 5/5

Fix Specificity: 2/5

Urgency Impact: 4/5

Total Score: 11/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/skills/sql_agent.py, lines 588-588, the SQL query in `_get_sample_rows` is constructed using direct string interpolation of `fields` and `table`, which allows SQL injection if these values are attacker-controlled. Update the code to safely quote and sanitize all identifiers (fields and table) to prevent injection. Only allow valid identifier characters and wrap with backticks.
518-518: get_kb_sample_rows (lines 508-540) constructs SQL queries using direct string interpolation of kb_name, allowing SQL injection if attacker controls this value.

📊 Impact Scores:

Production Impact: 5/5

Fix Specificity: 2/5

Urgency Impact: 4/5

Total Score: 11/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/skills/sql_agent.py, line 518, the SQL query in `get_kb_sample_rows` is constructed using direct string interpolation of `kb_name`, which allows SQL injection if this value is attacker-controlled. Update the code to safely quote and sanitize the table name (kb_name) to prevent injection. Only allow valid identifier characters and wrap with backticks.
mindsdb/interfaces/storage/fs.py (2)
91-91: copy function opens files without context managers, which can cause file descriptor leaks and undefined behavior if exceptions occur.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/storage/fs.py, lines 91-91, the `copy` function opens files without using context managers, which can cause file descriptor leaks and undefined behavior if exceptions occur. Refactor the code so that both `src` and `dst` files are opened using `with` statements (context managers) when calculating their md5 hashes.
91-91: copy function uses hashlib.md5 for file integrity checks, which is insecure and can allow attackers to substitute malicious files with hash collisions.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 11/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/storage/fs.py, line 91, the `copy` function uses `hashlib.md5` to compare file contents, which is insecure due to known hash collisions. Replace all uses of `hashlib.md5` with `hashlib.sha256` for file integrity checks to prevent attackers from exploiting hash collisions.
mindsdb/interfaces/storage/json.py (2)
71-93: __delitem__ and clean methods in both storage classes do not handle commit failures robustly, potentially leaving the database in an inconsistent state if db.session.commit() partially succeeds before an exception.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 11/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/storage/json.py, lines 71-93, the `__delitem__` and `clean` methods catch exceptions on `db.session.commit()` but do not re-raise them, which can mask database errors and leave the system in an inconsistent state. Please update both methods to re-raise the exception after logging and rolling back, so that calling code is properly notified of the failure.
55-61: The get_all_records method loads all records into memory at once, which can cause high memory usage and slowdowns for large datasets.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 8/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/storage/json.py, lines 55-61, the get_all_records method loads all records into memory at once, which can cause high memory usage and slowdowns for large datasets. Refactor get_all_records to return a generator that yields records in batches (e.g., batch_size=1000) using offset/limit pagination, to reduce memory footprint and improve scalability.
mindsdb/interfaces/tabs/tabs_controller.py (1)
135-147: get_all() reads and parses every tab file on every call, causing O(n) file I/O and JSON loads per request; this will scale poorly with large numbers of tabs and frequent access.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 2/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/tabs/tabs_controller.py, lines 135-147, the `get_all()` method reads and parses every tab file on every call, resulting in O(n) file I/O and JSON loads per request. This will scale poorly with large numbers of tabs and frequent access. Refactor `get_all()` to cache the parsed tab data in memory (e.g., with an attribute like `self._tabs_cache`), and invalidate the cache when a tab is added, modified, or deleted. Ensure the cache is used for repeated calls to `get_all()` to avoid unnecessary file I/O and JSON parsing.
mindsdb/interfaces/tasks/task_monitor.py (1)
79-81: check_tasks performs a database query for every active task (db.Tasks.query.get(task_id)), causing N+1 query inefficiency that will severely degrade performance as the number of tasks grows.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 2/5

Urgency Impact: 3/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Optimize the N+1 query pattern in mindsdb/interfaces/tasks/task_monitor.py, lines 79-81. Instead of querying the database for each task with `db.Tasks.query.get(task_id)`, batch fetch all relevant task records in a single query and use them in the loop. Refactor the code to eliminate per-task queries and improve scalability.
mindsdb/interfaces/tasks/task_thread.py (3)
26-26: db.Tasks.query.get(self.task_id) in run() loads the full task record from the database on every thread start, which can be inefficient if called frequently or for many threads; consider bulk-fetching or caching if this is a hot path.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 2/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/tasks/task_thread.py, line 26, the code fetches the task record from the database using `db.Tasks.query.get(self.task_id)` every time a TaskThread is started. If this method is called frequently or for many threads, it can cause significant database load and slowdowns. Consider implementing a caching mechanism or bulk-fetching strategy for task records to reduce database hits and improve performance.
38-48: The run() method contains a large conditional block with repeated instantiation and execution logic for different task types, making it harder to maintain and extend as new types are added.

📊 Impact Scores:

Production Impact: 1/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/tasks/task_thread.py, lines 38-48, the `run()` method uses a large if-elif block to handle different task types, which makes the code harder to maintain and extend. Refactor this block to use a dictionary mapping of object_type to task class, and instantiate/run the appropriate class dynamically. This will improve maintainability and scalability as new task types are added.
52-52: task_record.last_error is set to the full exception traceback, which may expose sensitive internal information to users or logs if this field is later displayed or returned.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 2/5

Urgency Impact: 3/5

Total Score: 8/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/tasks/task_thread.py, line 52, the code sets `task_record.last_error` to the full exception traceback, which can expose sensitive internal details if this field is later shown to users or logs. Change this assignment to only store the exception message (not the full traceback), or a generic error message if the exception is not available. Ensure no sensitive stack trace or internal data is stored in `last_error`.
mindsdb/interfaces/triggers/trigger_task.py (2)
66-66: row.update(key) will raise an exception if key is not a dictionary, causing the trigger to fail for valid events where key is e.g. a string or integer.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/triggers/trigger_task.py, lines 66-66, the code calls `row.update(key)` without checking if `key` is a dictionary. This will raise an exception if `key` is not a dict (e.g., if it's a string or int), causing the trigger to fail. Please update this line to only call `row.update(key)` if `key` is a dictionary, e.g., using `if isinstance(key, dict): row.update(key)`.
33-77: trigger.query_str is parsed and executed with user-controlled data (row, key) injected into the query AST, enabling SQL injection if trigger.query_str or row/key are attacker-controlled.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 3/5

Urgency Impact: 4/5

Total Score: 11/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/interfaces/triggers/trigger_task.py, lines 33-77, the code parses and executes `trigger.query_str` and injects user-controlled data (`row`, `key`) into the query AST, which can enable SQL injection or arbitrary code execution if these values are attacker-controlled. Add strict validation or sanitization for `trigger.query_str` (e.g., only allow whitelisted queries or enforce a strict schema) and for all injected data (`row`, `key`) before parsing or executing the query. If unsafe input is detected, log an error and abort execution.
mindsdb/utilities/api_status.py (2)
40-58: set_api_status is not process-safe: concurrent writes can cause data loss or corruption due to race conditions when reading, modifying, and writing the status file.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 2/5

Urgency Impact: 3/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/utilities/api_status.py, lines 40-58, the `set_api_status` function is not process-safe: concurrent writes can cause data loss or corruption because multiple processes can read, modify, and write the status file simultaneously. Update this function to use file locking (e.g., with `fcntl.flock`) to ensure exclusive access when reading and writing the status file, so that concurrent calls do not corrupt or lose data. Preserve atomic writes and error handling.
28-37: get_api_status reads and parses the status file on every call, causing repeated disk I/O and JSON parsing even for frequent, repeated status checks; this can significantly degrade performance under high-frequency access.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/utilities/api_status.py, lines 28-37, the `get_api_status` function reads and parses the status file from disk on every call, causing repeated disk I/O and JSON parsing. This can significantly degrade performance under high-frequency access. Refactor `get_api_status` to use in-memory caching: cache the parsed status and only reload from disk if the file's modification time has changed. Ensure thread/process safety if needed.
mindsdb/utilities/config.py (1)
222-366: prepare_env_config is a large, complex function (73 statements, 29 branches) that is difficult to maintain and reason about, increasing risk of performance and maintainability issues as config logic grows.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the function `prepare_env_config` in mindsdb/utilities/config.py (lines 222-366). The function is too large and complex (73 statements, 29 branches), making it hard to maintain and reason about. Break it into smaller helper methods for each logical section (e.g., storage, auth, logging, ML queue, reranker, GUI, etc.), and call these helpers from `prepare_env_config`. This will significantly improve maintainability and reduce risk of future performance or logic errors.
mindsdb/utilities/fs.py (1)
105-127: clean_unlinked_process_marks can raise UnboundLocalError if process_id or thread_id are not set due to malformed filename, causing a crash when iterating over unexpected files.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 4/5

Urgency Impact: 3/5

Total Score: 11/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/utilities/fs.py, lines 105-127, the function `clean_unlinked_process_marks` assumes all filenames in the process mark directory are well-formed and split into two integers. If a file with an unexpected name is present, this will cause a ValueError or IndexError, leading to an UnboundLocalError or crash. Update the exception handling to also catch ValueError and IndexError, and use `locals().get` to safely reference `process_id` and `thread_id` in log messages and when appending to `deleted_pids`. Ensure the function does not crash on malformed filenames.
mindsdb/utilities/hooks/profiling.py (2)
52-52: logger.warning("cant get acceess to profiling database") does not log the exception traceback, making debugging connection failures difficult and losing critical error context.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 5/5

Urgency Impact: 2/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/utilities/hooks/profiling.py, line 52, replace the logger.warning call with logger.exception to ensure the exception traceback is logged when profiling database connection fails. This will provide critical error context for debugging. Update the line to: logger.exception("cant get acceess to profiling database")
17-30: The function set_level recursively traverses the entire profiling tree and recalculates values for all nodes on every profiling submission, which can cause significant CPU overhead for large or deeply nested trees.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 2/5

Urgency Impact: 2/5

Total Score: 6/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Optimize the function `set_level` in mindsdb/utilities/hooks/profiling.py (lines 17-30). The current implementation uses recursion to traverse and update the profiling tree, which can cause stack overflows and high CPU usage for large or deeply nested trees. Refactor this function to use an explicit stack (iterative approach) to avoid recursion and improve performance for large datasets. Ensure the logic for updating node values and levels remains functionally equivalent.
mindsdb/utilities/json_encoder.py (1)
10-26: CustomJSONEncoder.default is a large, complex function with many return statements and type checks, making it difficult to maintain and extend, especially as new types are added for serialization.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/utilities/json_encoder.py, lines 10-26, the `CustomJSONEncoder.default` method is a large, complex function with many return statements and type checks. This structure makes it difficult to maintain and extend, especially as new types are added. Refactor this function to reduce complexity, possibly by using a dispatch table or helper methods for type handling, to improve maintainability and scalability.
mindsdb/utilities/langfuse.py (2)
171-185, 186-199, 232-246: self.trace is assumed to be not None in several methods (e.g., start_span, end_span_stream, get_langchain_handler), which will raise AttributeError if setup_trace was never called or failed.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/utilities/langfuse.py, lines 171-185, 186-199, and 232-246, methods like start_span, end_span_stream, and get_langchain_handler assume self.trace is not None, but if setup_trace was never called or failed, self.trace will be None and these will raise AttributeError. Add checks for self.trace being not None (and self.client) at the start of these methods, and return early or None if not set.
266-283: The _get_tool_usage method fetches the entire trace and iterates over all observations every time it is called, which can cause significant performance degradation for traces with many observations.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 5/5

Urgency Impact: 2/5

Total Score: 10/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Optimize the `_get_tool_usage` method in `mindsdb/utilities/langfuse.py` (lines 266-283). The current implementation fetches the entire trace and iterates over all observations every time it is called, which can cause significant performance degradation for traces with many observations. Add a simple caching mechanism (e.g., store the result in `self._cached_tool_usage` and return it if available) to avoid repeated expensive operations within the same object lifecycle.
mindsdb/utilities/log.py (1)
141-146: getLogger() always initializes logging with process_name=None, ignoring the name argument, which can cause log files to be named incorrectly for subprocesses and break per-process logging separation.

📊 Impact Scores:

Production Impact: 3/5

Fix Specificity: 4/5

Urgency Impact: 2/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/utilities/log.py, lines 141-146, the `getLogger()` function always calls `initialize_logging()` with no process name, which breaks per-process log file naming and can cause log file collisions or incorrect log separation in multi-process environments. Update `getLogger()` to accept an optional `process_name` argument, pass it to `initialize_logging(process_name)`, and update the docstring accordingly.
mindsdb/utilities/ml_task_queue/consumer.py (3)
47-47: self._listen_message_threads.remove(current_thread) in _save_thread_link will raise ValueError if current_thread is not in the list, causing a crash and thread leak.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 5/5

Urgency Impact: 3/5

Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/utilities/ml_task_queue/consumer.py, lines 47-47, the code `self._listen_message_threads.remove(current_thread)` can raise a ValueError if the thread is not in the list, causing a crash and possible thread leak. Please update this line to check if `current_thread` is in the list before removing it.
142-213: Major code complexity: _listen method is excessively long and complex (59 statements), making it hard to maintain and reason about, increasing risk of performance and reliability issues.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `_listen` method in `mindsdb/utilities/ml_task_queue/consumer.py` (lines 142-213) to reduce its complexity. Extract the message waiting and message processing logic into separate helper methods (e.g., `_wait_for_message` and `_process_message`). This will make the code more maintainable and easier to reason about. Ensure the refactoring preserves all original functionality and error handling.
216-223: Resource contention: Spawning a new thread for every message in run can exhaust system resources under high load, causing degraded performance and instability.

📊 Impact Scores:

Production Impact: 4/5

Fix Specificity: 4/5

Urgency Impact: 3/5

Total Score: 11/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In `mindsdb/utilities/ml_task_queue/consumer.py`, lines 216-223, update the `run` method to limit the number of concurrent `_listen` threads to the number of CPU cores (or at least 1). This prevents resource exhaustion and improves stability under high load. Track and clean up finished threads in `self._listen_message_threads` before spawning new ones.
mindsdb/utilities/render/sqlalchemy_render.py (1)
179-406,517-654: to_expression and prepare_select are extremely large, complex functions (66/96 branches, 166/96 statements), making them hard to maintain and optimize, and increasing risk of performance bugs.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 3/5

Urgency Impact: 2/5

Total Score: 7/15

🤖 AI Agent Prompt (Copy & Paste Ready):
Refactor the `to_expression` (lines 179-406) and `prepare_select` (lines 517-654) methods in mindsdb/utilities/render/sqlalchemy_render.py. Both functions are extremely large and complex, with excessive branching and statement counts, making them difficult to maintain and optimize. Break them into smaller, well-named helper methods to reduce cyclomatic complexity and improve maintainability, while preserving all existing logic and performance.
mindsdb/utilities/utils.py (1)
32-32: raise ValueError in the except block does not preserve the original exception context, making debugging harder and potentially hiding the root cause.

📊 Impact Scores:

Production Impact: 2/5

Fix Specificity: 5/5

Urgency Impact: 2/5

Total Score: 9/15

🤖 AI Agent Prompt (Copy & Paste Ready):
In mindsdb/utilities/utils.py, line 32, the exception raised in the except block should use `from e` to preserve the original exception context. Please ensure that the raised ValueError includes `from e` to maintain the exception chain for better debugging.

🔍 Comments beyond diff scope (19)

mindsdb/api/executor/datahub/datanodes/project_datanode.py (1)

88-89: predict method raises an exception if model_metadata["update_status"] == "available", which is likely the opposite of intended logic; this prevents predictions on available models.
Category: correctness

mindsdb/api/http/namespaces/config.py (1)

126-130: file.save(file_path) allows arbitrary file writes if attacker controls filename, enabling path traversal and overwriting sensitive files.
Category: security

mindsdb/api/http/namespaces/file.py (1)

101-101: file_object.name is renamed using untrusted user input (data["file"]), allowing path traversal and overwriting arbitrary files on the server.
Category: security

mindsdb/api/http/namespaces/handlers.py (1)

173-173: prepare_formdata previously used AttributeError for missing 'modules' key, but the actual error is KeyError; this would cause a 500 error instead of correct fallback logic.
Category: correctness

mindsdb/api/mysql/mysql_proxy/utilities/dump.py (1)

99-99: _dump_bool returns string values ('1'/'0') instead of integers (1/0), which can cause type mismatches or incorrect MySQL boolean handling.
Category: correctness

mindsdb/api/postgres/postgres_proxy/postgres_packets/postgres_packets.py (1)

101-104: read_startup_message can raise IndexError if the startup message has an odd number of parameters, causing a crash on malformed input.
Category: correctness

mindsdb/api/postgres/postgres_proxy/postgres_proxy.py (1)

193-196: describe with describe_type == b'P' and no message.name will use self.unnamed_statement instead of self.unnamed_portal, causing a KeyError on executor or incorrect field description.
Category: correctness

mindsdb/integrations/handlers/mysql_handler/mysql_handler.py (1)

295-313: get_columns uses string interpolation to inject table_name directly into SQL, allowing SQL injection if table_name is attacker-controlled.
Category: security

mindsdb/integrations/handlers/pgvector_handler/pgvector_handler.py (1)

480-549: hybrid_search constructs SQL queries using direct string interpolation of user-controlled values (e.g., metadata, query, table_name), leading to exploitable SQL injection vulnerabilities.
Category: security

mindsdb/integrations/libs/vectordatabase_handler.py (1)

274-282: set_metadata_cur_time mutates metadata dicts in-place but does not assign the result back to the DataFrame, so changes are not persisted, causing missing or stale metadata timestamps.
Category: correctness

mindsdb/integrations/utilities/handlers/auth_utilities/snowflake/__init__.py (1)

1-1: No significant functional bug detected; the file only re-exports a function and does not contain logic that could cause runtime failures or incorrect results.
Category: correctness

mindsdb/integrations/utilities/rag/rerankers/reranker_compressor.py (1)

58-58: next(d for d in documents if d.page_content == doc) will raise StopIteration if no document matches, causing a runtime exception and crash.
Category: correctness

mindsdb/interfaces/agents/litellm_server.py (1)

131-138: /direct-sql endpoint allows arbitrary SQL queries from user input (request.query) to be executed directly, enabling attackers to perform unauthorized data access or modification (SQL injection risk) if the underlying call_tool('query', ...) does not strictly validate or parameterize queries.
Category: security

mindsdb/interfaces/jobs/jobs_controller.py (1)

415-439: execute_task_local uses string replacement to fill SQL template variables, allowing attackers to inject arbitrary SQL if job definitions contain untrusted input with template markers.
Category: security

mindsdb/interfaces/model/model_controller.py (1)

208-215: rename_model allows renaming to an existing model name, violating uniqueness and causing data corruption or lookup failures.
Category: correctness

mindsdb/interfaces/tasks/task_monitor.py (1)

134-137: stop_task in TaskMonitor does not check if task_id exists in self._active_tasks, which can cause a KeyError and crash if called with a stale or already-removed task id.
Category: correctness

mindsdb/utilities/config.py (1)

111-114: self.user_config is referenced instead of self._user_config, which will raise an AttributeError if user_config property is not set yet.
Category: correctness

mindsdb/utilities/fs.py (1)

178-182: safe_extract prevents path traversal in tar extraction by checking paths, but uses os.path.commonprefix which is not path-aware and can be bypassed, allowing arbitrary file overwrite.
Category: security

mindsdb/utilities/render/sqlalchemy_render.py (1)

0604-0609: prepare_select allows arbitrary SQL text from ast.NativeQuery via sa.text(from_table.query), risking SQL injection if from_table.query is user-controlled.
Category: security

ea-rus · 2025-10-23T05:16:28Z

changes are moved into #11759

adding hybrid_search_alpha=0 condition for hybrid_search enablment

978c10a

fshabashev requested a review from ea-rus September 15, 2025 15:34

fshabashev added the deploy-to-dev label Sep 15, 2025

fshabashev temporarily deployed to dev September 15, 2025 15:37 — with GitHub Actions Inactive

fshabashev temporarily deployed to dev September 15, 2025 15:41 — with GitHub Actions Inactive

dusvyat previously approved these changes Sep 16, 2025

View reviewed changes

ea-rus previously approved these changes Sep 16, 2025

View reviewed changes

fshabashev changed the base branch from main to develop September 16, 2025 15:49

fshabashev requested a review from dusvyat September 16, 2025 16:11

dusvyat approved these changes Sep 16, 2025

View reviewed changes

ea-rus approved these changes Oct 17, 2025

View reviewed changes

Merge branch 'develop' into improve_hybrid_search_interface

838d5f9

ea-rus temporarily deployed to dev October 17, 2025 11:14 — with GitHub Actions Inactive

ea-rus temporarily deployed to dev October 17, 2025 11:17 — with GitHub Actions Inactive

ea-rus mentioned this pull request Oct 17, 2025

Improvements of keyword search #11759

Merged

8 tasks

ea-rus closed this Oct 23, 2025

github-actions Bot locked and limited conversation to collaborators Oct 23, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

adding hybrid_search_alpha=0 condition for hybrid_search enablment#11573

adding hybrid_search_alpha=0 condition for hybrid_search enablment#11573
fshabashev wants to merge 2 commits into
developfrom
improve_hybrid_search_interface

fshabashev commented Sep 15, 2025

Uh oh!

entelligence-ai-pr-reviews Bot commented Sep 15, 2025

Uh oh!

entelligence-ai-pr-reviews Bot commented Oct 17, 2025

Uh oh!

entelligence-ai-pr-reviews Bot commented Oct 17, 2025

Uh oh!

ea-rus commented Oct 23, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

fshabashev commented Sep 15, 2025

Description

Type of change

Verification Process

Additional Media:

Checklist:

Uh oh!

entelligence-ai-pr-reviews Bot commented Sep 15, 2025

Uh oh!

entelligence-ai-pr-reviews Bot commented Oct 17, 2025

Uh oh!

entelligence-ai-pr-reviews Bot commented Oct 17, 2025

Review Summary

Uh oh!

ea-rus commented Oct 23, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants