Skip to content

feat: Add Gemini Integration in Frontend, implement media processing utilities and Deepgram transcription - #34

Open
pranjalaman wants to merge 1 commit into
JayWebtech:mainfrom
pranjalaman:fix/windows-support-and-gemini-integration
Open

feat: Add Gemini Integration in Frontend, implement media processing utilities and Deepgram transcription#34
pranjalaman wants to merge 1 commit into
JayWebtech:mainfrom
pranjalaman:fix/windows-support-and-gemini-integration

Conversation

@pranjalaman

Copy link
Copy Markdown

What this does

  • Adds Windows support for FFmpeg font paths by implementing proper path escaping (e.g., C\:\\Windows\\Fonts...) to prevent the drawtext filter from crashing during subtitle rendering.
  • Resolves Deepgram 408 Request Timeout (SLOW_UPLOAD) errors by extracting video audio as a highly compressed 64kbps MP3 instead of an uncompressed WAV file, reducing upload sizes by over 90%.
  • Integrates Google Gemini API support into the UI by adding the Gemini Key input to the Onboarding screen and fixing the backend routing to correctly pass Gemini, OpenAI, and OpenRouter keys to the moment detection engine.
  • Updates .gitignore to safely exclude .env.* and .vscode/ directories.

Why

Previously, the app failed to render clips on Windows because FFmpeg misinterpreted Windows drive letters (C:) as filter separators. Additionally, users with longer videos or slower connections experienced upload timeouts to Deepgram because the extracted WAV files were too massive. Lastly, while the backend supported Gemini, the onboarding UI and frontend routing were missing the necessary integration, preventing users from utilizing their Gemini API keys.

Testing

  • Successfully compiled the Tauri backend on Windows using MSVC Build Tools.
  • Ran the app via npm run tauri:dev on a Windows environment.
  • Verified that adding a Gemini API key in the onboarding flow correctly saves and passes the key to the backend during moment detection.
  • Tested audio extraction and observed successfully compressed MP3 files, resulting in instant uploads to Deepgram without timeouts.
  • Tested the "Cut" feature and verified that the final exported video successfully burns in subtitles on Windows without throwing an FFmpeg filtergraph parse error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant