Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions docs/AUTHENTICATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,13 +170,13 @@ curl -X POST https://your-server.com/api/videos/download \
# List active API keys
docker exec youtarr-db mysql -u root -p123qweasd youtarr -e "
SELECT id, name, key_prefix, created_at, last_used_at
FROM ApiKeys
FROM apikeys
WHERE is_active = 1;
"

# Revoke a key by ID
docker exec youtarr-db mysql -u root -p123qweasd youtarr -e "
UPDATE ApiKeys SET is_active = 0 WHERE id = 1;
UPDATE apikeys SET is_active = 0 WHERE id = 1;
"
```

Expand All @@ -186,7 +186,7 @@ For detailed API documentation and examples (bookmarklets, mobile shortcuts, Pyt

### Session Configuration
- **Duration**: 7 days
- **Storage**: Database table `Sessions`
- **Storage**: Database table `sessions`
- **Browser storage**: Token persisted as `authToken` in `localStorage`
- **Client header**: Token forwarded on each API request via `x-access-token`

Expand All @@ -202,7 +202,7 @@ For detailed API documentation and examples (bookmarklets, mobile shortcuts, Pyt
```bash
docker exec youtarr-db mysql -u root -p123qweasd youtarr -e "
SELECT id, session_token, username, expires_at, is_active
FROM Sessions
FROM sessions
WHERE expires_at > NOW()
AND is_active = 1;
"
Expand All @@ -211,7 +211,7 @@ WHERE expires_at > NOW()
#### Clear All Sessions (Force Re-login)
```bash
docker exec youtarr-db mysql -u root -p123qweasd youtarr -e "
DELETE FROM Sessions;
DELETE FROM sessions;
"
```

Expand Down
49 changes: 49 additions & 0 deletions docs/BACKUP_RESTORE.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,12 @@ On Debian/Ubuntu, install jq with: `sudo apt install jq`

## Quick Start

The `backup.sh` and database restore steps below operate on the bundled
`youtarr-db` container. They do not back up or restore an external database
configured through `DB_HOST`. External database users must use their database
provider's backup tools or a database dump connected to the actual external
server, and preserve their local configuration and metadata separately.

### Create a Backup

```bash
Expand Down Expand Up @@ -125,6 +131,49 @@ Skips the confirmation prompt. Use with caution in scripts.

## Migration Scenarios

### Before Updating or Returning to an Earlier Version

Create a backup before updating Youtarr, especially when an update changes the
database schema. Stop Youtarr first so downloads, settings, and metadata do not
change while the backup is taken. For the bundled database:

```bash
./stop.sh
./scripts/backup.sh
```

Wait for the backup to finish successfully before updating. Keep the archive
and record the Youtarr image version you were running. For an external database,
back up the actual external database using your provider's tools or a database
dump; `backup.sh` alone does not cover that database.

The table and column rename migration changes the names expected by older
Youtarr versions. **Changing the Docker image back to an older version alone
does not undo the migration.** To return to the version you backed up:

1. Stop Youtarr and keep it stopped throughout recovery.
2. Restore the backup taken before the update. For the bundled database, use
`./scripts/restore.sh /path/to/pre-update-backup.tar.gz`. For an external
database, restore the pre-update database backup and matching local settings
and metadata using your external backup procedure.
3. After restoring, select the previous image version before starting Youtarr.
For the standard Compose setup, set
`YOUTARR_IMAGE=dialmaster/youtarr:<previous-version-tag>` in `.env`, replacing
the placeholder with the recorded version. Do this after restoration because
`restore.sh` also restores `.env`. Avoid `latest` or `dev` for this recovery.
4. Start Youtarr using the normal startup command for your database setup and
verify your settings and download history.

The bundled database restore replaces the database, including its schema and
`SequelizeMeta` migration history. Starting the newer image against the restored
database would apply the new migrations again.

Restoring returns database state, settings, and backed-up metadata to the backup
time; changes made since then are not retained in the restored database. Video
files are not included in these backups. Files downloaded or deleted after the
backup are not reverted, so the restored history may differ from the files on
disk. Back up the output directory separately if you need to restore those files.

### Moving to a New Computer

1. **On the old computer:**
Expand Down
16 changes: 8 additions & 8 deletions docs/DATABASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,17 +23,17 @@ Youtarr uses MariaDB/MySQL for storing:
| Table | Model | Description |
| :----------------- | :------------- | :-------------------------------- |
| `channels` | `Channel` | YouTube channel information. `m3u_enabled` (boolean, default false): generate a `.m3u` playlist file in the channel folder. `m3u_sort_order` (string, default `oldest_first`): `.m3u` entry order, `oldest_first` or `newest_first`. `auto_removal_protected` (boolean, default false): exclude every video of this channel from auto-removal; only applies while the channel is subscribed (`enabled`), and enabling it clears `auto_removal_keep_recent_count`. `auto_removal_keep_recent_count` (int, nullable): auto-removal always keeps this many of the channel's most recently downloaded videos; mutually exclusive with `auto_removal_protected`, dormant while the channel is unsubscribed. |
| `Videos` | `Video` | Downloaded video metadata. `video_resolution` (VARCHAR(20), nullable): actual pixel dimensions of the downloaded file, e.g. `"1920x1080"`, measured by ffprobe at download time and backfilled by the filesystem rescan; the displayed tier label (e.g. "1080p") is derived client-side (`client/src/utils/videoResolution.ts`), so labeling rules can change without re-probing; NULL = not yet checked or audio-only, `"0x0"` = probe failed (never shown in the UI). |
| `channelvideos` | `ChannelVideo` | Channel <-> video associations. `published_at_source` tracks publishedAt provenance: `exact` (.info.json), `approximate` (yt-dlp flat-playlist date), `estimated` (ordering-only placeholder assigned when YouTube returns a listing with no dates; never displayed), NULL (legacy, treated as approximate) |
| `Jobs` | `Job` | Download job queue. `aux_data` (MEDIUMTEXT, nullable): JSON snapshot of job data other than videos (failed downloads, diagnoses, skip counts, terminated channels), written on job save and merged back on startup load; NULL for jobs recorded before the column existed. |
| `JobVideos` | `JobVideo` | Job <-> video associations |
| `JobVideoDownloads`| `JobVideoDownload`| Download progress tracking |
| `Sessions` | `Session` | User authentication sessions |
| `ApiKeys` | `ApiKey` | API key credentials for external integrations (bookmarklets, shortcuts, automation) |
| `videos` | `Video` | Downloaded video metadata. `video_resolution` (VARCHAR(20), nullable): actual pixel dimensions of the downloaded file, e.g. `"1920x1080"`, measured by ffprobe at download time and backfilled by the filesystem rescan; the displayed tier label (e.g. "1080p") is derived client-side (`client/src/utils/videoResolution.ts`), so labeling rules can change without re-probing; NULL = not yet checked or audio-only, `"0x0"` = probe failed (never shown in the UI). |
| `channelvideos` | `ChannelVideo` | Channel <-> video associations. `published_at_source` tracks `published_at` provenance: `exact` (.info.json), `approximate` (yt-dlp flat-playlist date), `estimated` (ordering-only placeholder assigned when YouTube returns a listing with no dates; never displayed), NULL (legacy, treated as approximate) |
| `jobs` | `Job` | Download job queue. `aux_data` (MEDIUMTEXT, nullable): JSON snapshot of job data other than videos (failed downloads, diagnoses, skip counts, terminated channels), written on job save and merged back on startup load; NULL for jobs recorded before the column existed. |
| `jobvideos` | `JobVideo` | Job <-> video associations |
| `jobvideodownloads`| `JobVideoDownload`| Download progress tracking |
| `sessions` | `Session` | User authentication sessions |
| `apikeys` | `ApiKey` | API key credentials for external integrations (bookmarklets, shortcuts, automation) |
| `playlists` | `Playlist` | Subscribed YouTube playlists with per-playlist sync targets and seeded settings. `auto_download_baseline_at` (DATETIME, nullable): seed-then-track baseline for playlist auto-downloads; NULL until the first auto-download run. `sort_order` (STRING NOT NULL, default `'default'`): saved output order for the `.m3u` file and media server sync; `'reversed'` flips the YouTube playlist order. |
| `playlistvideos` | `PlaylistVideo` | One row per (playlist, video) with the YouTube playlist position |
| `playlist_sync_state` | `PlaylistSyncState` | Per-(playlist, server) sync state: server playlist id, last_synced_at, last_error |
| `subfolders` | `Subfolder` | Durable registry of known subfolder names (id, name unique, createdAt, updatedAt). Backfilled from channels, playlists, and video file paths by the `add-subfolders-table` migration; kept current by register-on-create and register-on-download-override. |
| `subfolders` | `Subfolder` | Durable registry of known subfolder names (id, name unique, created_at, updated_at). Backfilled from channels, playlists, and video file paths by the `add-subfolders-table` migration; kept current by register-on-create and register-on-download-override. |
| `video_watch_status` | `VideoWatchStatus` | Per-video, per-media-server, per-user watch state pulled by the watch status sync. Absence of a row means never synced/unknown, not unwatched. Columns: `video_id`, `server_type` (`plex`/`jellyfin`/`emby`), `server_user_id` (Plex owner is `'1'`), `played`, `play_count`, `position_ms`, `percent_watched`, `last_watched_at`, `last_synced_at`. Unique index on `(video_id, server_type, server_user_id)`. |
| `media_server_users` | `MediaServerUser` | Media-server account directory populated during watch status sync: `server_type`, `server_user_id`, `server_user_name`. Unique index on `(server_type, server_user_id)`. Used to display which users watched a video. |
| `watch_status_sync_cursors` | `WatchStatusSyncCursor` | Durable per-server watch-status sync cursor (unique `server_type`, `cursor` DATETIME). Today only Plex uses it: the newest play-history event scanned, so incremental pulls never permanently skip events. Deleting a row forces a full history re-scan on the next sync. |
Expand Down
2 changes: 1 addition & 1 deletion docs/TROUBLESHOOTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -451,7 +451,7 @@ ERROR 1396 (HY000) at line 21: Operation CREATE USER failed for 'root'@'%'
2. If the error persists, check `docker compose logs youtarr` to see which migration is still failing.
3. Manually reconcile the schema for that migration:
- Connect to MariaDB: `docker compose exec youtarr-db mysql -u root -p youtarr`
- Drop the duplicate column or table mentioned in the error (for example `ALTER TABLE Videos DROP COLUMN media_type;`), **or** restore a known-good backup.
- Drop the duplicate column or table mentioned in the error (for example `ALTER TABLE videos DROP COLUMN media_type;`), **or** restore a known-good backup.
- Exit MySQL and restart the stack.
4. Once the stack is back online, confirm the schema is healthy: `curl http://localhost:3087/api/db-status` reports whether the database connection and schema checks passed, and the startup logs (`docker compose logs youtarr`) show the migration results.

Expand Down
125 changes: 125 additions & 0 deletions migrations/20260830201917-lowercased-table-column-names.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
'use strict';

const { extractTableName, columnExists } = require('./helpers');

// The tableExists function in helpers is case-insensitive so we cannot use it for this migration.
async function tableExists(queryInterface, tableName) {
const tablesRaw = await queryInterface.showAllTables();
return tablesRaw.some((table) => extractTableName(table) === tableName);
}

async function tableAndColumnExist(queryInterface, tableName, columnName) {
return await tableExists(queryInterface, tableName) && await columnExists(queryInterface, tableName, columnName);
};

const TABLES = [
['ApiKeys', 'apikeys'],
['JobVideoDownloads', 'jobvideodownloads'],
['JobVideos', 'jobvideos'],
['Jobs', 'jobs'],
['Sessions', 'sessions'],
['Videos', 'videos'],
];

async function renameTableResumably(queryInterface, original, lowercase, destination) {
// Recognize temporary names left by either an upgrade or a rollback.
// Compare stored names exactly so distinct tables on LCTN=0 are conflicts,
// while LCTN=1 naturally returns only the lowercase stored name.
const candidates = new Set([original, lowercase, `${original}-tmp`, `${lowercase}-tmp`]);
const tables = (await queryInterface.showAllTables()).map(extractTableName);
const matches = tables.filter((name) => candidates.has(name));
if (matches.length !== 1) {
throw new Error(`Cannot rename ${original}: expected one table, found ${matches.length} (${matches.join(', ')})`);
}

const source = matches[0];
if (source === destination) return;

if (source.endsWith('-tmp')) {
await queryInterface.renameTable(source, destination);
} else {
// A case-only rename fails on LCTN=2. Each step commits independently,
// so the temporary name must remain recoverable on the next invocation.
const temporary = `${source}-tmp`;
await queryInterface.renameTable(source, temporary);
await queryInterface.renameTable(temporary, destination);
}
}

async function normalizeTableNames(queryInterface) {
for (const [original, lowercase] of TABLES) {
await renameTableResumably(queryInterface, original, lowercase, lowercase);
}
}

const COLUMNS = [
['channels', 'lastFetchedByTab', 'last_fetched_by_tab'],
['channelvideos', 'publishedAt', 'published_at'],
['jobs', 'jobType', 'job_type'],
['jobs', 'timeCreated', 'time_created'],
['jobs', 'timeInitiated', 'time_initiated'],
['media_server_users', 'createdAt', 'created_at'],
['media_server_users', 'updatedAt', 'updated_at'],
['playlist_sync_state', 'createdAt', 'created_at'],
['playlist_sync_state', 'updatedAt', 'updated_at'],
['playlists', 'createdAt', 'created_at'],
['playlists', 'lastFetched', 'last_fetched'],
['playlists', 'updatedAt', 'updated_at'],
['playlistvideos', 'createdAt', 'created_at'],
['playlistvideos', 'updatedAt', 'updated_at'],
['subfolders', 'createdAt', 'created_at'],
['subfolders', 'updatedAt', 'updated_at'],
['video_watch_status', 'createdAt', 'created_at'],
['video_watch_status', 'updatedAt', 'updated_at'],
['videos', 'audioFilePath', 'audio_file_path'],
['videos', 'audioFileSize', 'audio_file_size'],
['videos', 'filePath', 'file_path'],
['videos', 'fileSize', 'file_size'],
['videos', 'originalDate', 'original_date'],
['videos', 'youTubeChannelName', 'youtube_channel_name'],
['videos', 'youTubeVideoName', 'youtube_video_name'],
['videos', 'youtubeId', 'youtube_id'],
['watch_status_sync_cursors', 'createdAt', 'created_at'],
['watch_status_sync_cursors', 'updatedAt', 'updated_at'],
];

/** @type {import('sequelize-cli').Migration} */
module.exports = {
async up (queryInterface, Sequelize) {
await normalizeTableNames(queryInterface);
for (const [tableName, from, to] of COLUMNS) {
if (await tableAndColumnExist(queryInterface, tableName, from)) {
await queryInterface.renameColumn(tableName, from, to);
}
}

// Sequelize generates invalid SQL for these columns since their default (in the database) is `CURRENT_TIMESTAMP`, see https://github.com/sequelize/sequelize/issues/8868
if (await tableAndColumnExist(queryInterface, 'sessions', 'createdAt')) {
await queryInterface.sequelize.query('ALTER TABLE sessions CHANGE createdAt created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP');
}
if (await tableAndColumnExist(queryInterface, 'sessions', 'updatedAt')) {
await queryInterface.sequelize.query('ALTER TABLE sessions CHANGE updatedAt updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP');
}
},

async down (queryInterface, Sequelize) {
// Recover tables before touching columns, including an interrupted upgrade
// or a rollback that already restored some original table names.
await normalizeTableNames(queryInterface);
if (await tableAndColumnExist(queryInterface, 'sessions', 'created_at')) {
await queryInterface.sequelize.query('ALTER TABLE sessions CHANGE created_at createdAt DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP');
}
if (await tableAndColumnExist(queryInterface, 'sessions', 'updated_at')) {
await queryInterface.sequelize.query('ALTER TABLE sessions CHANGE updated_at updatedAt DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP');
}

for (const [tableName, to, from] of COLUMNS) {
if (await tableAndColumnExist(queryInterface, tableName, from)) {
await queryInterface.renameColumn(tableName, from, to);
}
}
for (const [original, lowercase] of TABLES) {
await renameTableResumably(queryInterface, original, lowercase, original);
}
}
};
Loading
Loading