Custom Code API file persistence issues

[nico525]

What happened?

Hi,

I am running LibreChat with a custom written Code API in python. The API itself is working fine and I can flawlessly generate files and transfer them to LibreChat. However, i recognized that there are issues related to the file size which I can somehow reproduce.

Whenever I ask the chatbot to generate a binary fiel of size X it works roughly until 50-100kb. Bigger files do work well from Code API perspective, but the file doesn't get populated back into LibreChat. I can see the icon of the file, but the download doesn't work as the file transport got corrupted mit transfer.
The Code API responds with the correct headers, file body and status code 200. LibreChat however aborts mid stream. i did add like a lot of debugging logs around

LibreChat/api/server/services/Files/Code/process.js

Line 217 in e8f3be0

const downloadCodeOutputBuffer = async ({

but couldn't really find it out

what's really suspicious is the fact of the fiel size. it works for smaller ones, I could imagine that some wrapper outside (e.g. agent loop?) kills the request

Version Information

librechat v0.8.8

Steps to Reproduce

1. Add Code Execution to your modelspec
2. ask it to generate a 10mb binary file
3. see the file in the response, but no data is stored in the db or container. HTTP Stream is aborted

What browsers are you seeing the problem on?

Firefox

Relevant log output

there are no signs in the log, but the browser request to api/files/code/download/... returns 200 with just parts of the file (e.g. 600kb instead of 5mb)
A download is not triggered in the browser

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

custom written Code API in python.

why not use https://github.com/LibreChat-AI/code-interpreter/ ?

[nico525]

Fair enough, but I cannot use the stack due to internal limitations on the sandbox isolation.
Have you tried generating "big" files through the official code interpreter?

[SerDrozd]

I'd first separate "file generation" from "LibreChat proxying". The fact that /api/files/code/download/... returns 200 with a truncated body is actually useful. The request is reaching the download route and the response has already started.
I'd download the exact same object twice, once directly from the Code API and once through LibreChat, then compare the byte count and SHA256. Also compare the upstream Content-Length with the number of bytes LibreChat actually receives. If the direct download is complete and the proxied one is short, I'd stop looking at DB persistence for now.
One thing I'd instrument is the upstream stream itself. Count bytes on data and log end, aborted, error and close, together with the upstream content-length.
LibreChat currently pipes that upstream readable into the Express response. If the upstream dies after the 200 headers have already been sent, a plain pipe can leave the browser with a partial 200 response and very little information about what actually failed. While debugging I'd use stream.pipeline() so the stream failure is propagated explicitly.
I'd also check the 15 second Axios timeout on the code-output download path. I wouldn't assume it's the cause just because larger files fail, but if your Python endpoint pauses between chunks it would fit the small-file/large-file difference.
On the Python side I'd serve the artifact with FileResponse, or make sure a StreamingResponse keeps the file handle open for the entire iteration. I'd also verify that Content-Length is exactly the number of bytes actually yielded.
If the direct and proxied SHA256 differ, you've narrowed this down to the Code API -> LibreChat transport. If they match, then I'd move further downstream into persistence. That should cut the search space quite a bit.

[nico525]

After all - it turns out to be an issue with the microsoft container apps where the stream suddenly drops when Connection: close header is part of the request: microsoft/azure-container-apps#734
Due to keepAlive: false in the codeServerHttpAgent, this is the standard behavior of node...