Skip to content

Latest Version not working with Postgres #463

Closed
Closed
Latest Version not working with Postgres#463
Labels
bug

Description

@safaci2000
safaci2000
opened on Mar 12, 2026

Summary

Unable to take a postgres backup pushing to Google Storage using HMAC keys.

Steps to reproduce

backup-now
2026-03-12.17:31:18 [INFO] ** [db-backup01] Starting Manual Backup for db-backup01
sudo: pg_isready: command not found
2026-03-12.17:31:24 [WARN] ** [01-dashboard-v2-6-0-pgbouncer.postgres-operator.svc__dashboard] Postgres Host 'dashboard-v2-6-0-pgbouncer.postgres-operator.svc' is not accessible, retrying.. (5 seconds so far)
sudo: pg_isready: command not found

What is the expected correct behavior?

Working Backup pushed to GCS/S3.

Relevant logs and/or screenshots

Env below. Redacted some of the more noisy irrelevant entries

env
AWS_CLI_VERSION=1.44.56
CONTAINER_ENABLE_LOGSHIPPING=FALSE
CONTAINER_ENABLE_MESSAGING=TRUE
CONTAINER_ENABLE_MONITORING=FALSE
CONTAINER_ENABLE_SCHEDULING=TRUE
CONTAINER_MESSAGING_BACKEND=msmtp
CONTAINER_MONITORING_BACKEND=zabbix
CONTAINER_SCHEDULING_BACKEND=cron
DB01_BACKUP_BEGIN=0000
DB01_BACKUP_INTERVAL=1440
DB01_BACKUP_LOCATION=S3
DB01_CHECKSUM=SHA1
DB01_CLEANUP_TIME=10080
DB01_COMPRESSION=GZ
DB01_NAME=dashboard
DB01_PASS=XC88mJ8VRTpC2kEDRgcy
DB01_S3_BUCKET=dev-dashboards
DB01_S3_CERT_SKIP_VERIFY=true
DB01_S3_HOST=storage.googleapis.com
DB01_S3_KEY_ID=GOOG1EGG4WAJSS.....
DB01_S3_KEY_SECRET=SECRET_KEY_REDACTED
DB01_S3_PATH=dev-staging/postgres
DB01_S3_PROTOCOL=https
DB01_S3_REGION=us-central1
DB01_SPLIT_DB=FALSE
DB01_TYPE=postgres
DB01_USER=postgres
DBBACKUP_USER=root
DB_HOST=dashboard-v2-6-0-pgbouncer.postgres-operator.svc
DEBUG_MODE=FALSE
DOAS_VERSION=v6.8.2
FLUENTBIT_VERSION=3.1.10
HOME=/root
HOSTNAME=databasebackup-2.6.0-8f7844f8f-25gs8
IMAGE_NAME=tiredofit/db-backup
IMAGE_REPO_URL=https://github.com/tiredofit/docker-db-backup/

MONITOR_PORT=tcp://34.118.230.6:5066
MONITOR_PORT_5066_TCP=tcp://34.118.230.6:5066
MONITOR_PORT_5066_TCP_ADDR=34.118.230.6
MONITOR_PORT_5066_TCP_PORT=5066
MONITOR_PORT_5066_TCP_PROTO=tcp
MONITOR_SERVICE_HOST=34.118.230.6
MONITOR_SERVICE_PORT=5066
MSODBC_VERSION=18.6.1.1-1
MSSQL_VERSION=18.6.1.1-1
MYSQL_REPO_URL=https://github.com/mysql/mysql-server
MYSQL_VERSION=mysql-8.4.8
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
POSTGRES_VERSION=18.3
PS1=[tiredofit/db-backup:4.1.99 \t \w] $
PWD=/
S6_CMD_WAIT_FOR_SERVICES_MAXTIME=0
S6_GLOBAL_PATH=/command:/usr/bin:/bin:/usr/sbin:sbin:/usr/local/bin:/usr/local/sbin
S6_KEEP_ENV=1
S6_OVERLAY_VERSION=3.2.0.2
SHLVL=2
TERM=xterm
TIMEZONE=Etc/GMT
YQ_VERSION=v4.44.2
ZABBIX_VERSION=7.0.8
_=/usr/bin/with-contenv

Environment

Kubernetes Cluster

  • Image version / tag: 4.1.99
  • Host OS: GKE Linux
  • Postgres Version: psql (PostgreSQL) 16.8 - Percona Distribution

I would also add the DB is up and running and supporting a live application so the connection logs are likely invalid or something changed and my config is wrong somehow. The last working version I had running was: 4.1.19.

Activity

  1. safaci2000
    added
    bug
    on Mar 12, 2026
  2. safaci2000
    changed the title [-]Latest Version not working with Postgers[/-] [+]Latest Version not working with Postgres[/+] on Mar 12, 2026

  3. tamimology commented on Mar 13, 2026

    @tamimology
    tamimology
    on Mar 13, 2026

    same for me
    I am running latest postgres v.18 and have the same issue

  4. tiredofit commented on Mar 13, 2026

    @tiredofit
    tiredofit
    on Mar 13, 2026
    Collaborator

    A new 4.1.99 or latest will fix - Didn't include the right binaries in this release. Should be ready within 2hrs.

  5. tlex commented on Mar 13, 2026

    @tlex
    tlex
    on Mar 13, 2026
    Contributor

    The version 4.1.99 doesn't include the binaries.

    My workaround:

    FROM public.ecr.aws/docker/library/postgres:18.3-alpine@sha256:4da1a4828be12604092fa55311276f08f9224a74a62dcb4708bd7439e2a03911 AS pgsql
FROM ghcr.io/tiredofit/docker-db-backup:4.1.99@sha256:d547726ad13079fb1441b1d21f85f043a634cfeda1e013101ca8ba1fc3cf838e

RUN set -ex; \
    apk upgrade --no-cache

COPY --from=pgsql /usr/local/bin/* /usr/local/bin/

  6. tiredofit commented on Mar 13, 2026

    @tiredofit
    tiredofit
    on Mar 13, 2026
    Collaborator

    Clever, repull 4.1.99 or latest and all should be well.

  7. csg33k commented on Mar 13, 2026

    @csg33k
    csg33k
    on Mar 13, 2026

    Hmm, something is still wrong I think.

    I pulled ghcr.io/tiredofit/docker-db-backup:latest just in case it was some weird caching with 4.1.99 since I had pulled that already, I also set the imagePullPolicy: Always as well.

    in K8s:

    ls -ls /usr/local/bin/
total 13960
   40 -rwxr-xr-x 1 root root    38816 Mar 27  2024 envsubst
    4 -rwxr-xr-x 1 root root      962 Mar 12 12:01 logrotate_dbbackup
   40 -rwxr-xr-x 1 root root    38868 Mar 12 12:01 restore
13876 -rwxr-xr-x 1 root root 14206295 Jan 21  2025 yq

    locally: (I never had the image locally, so no cache)

    docker run -it --entrypoint /bin/bash  --rm ghcr.io/tiredofit/docker-db-backup:latest
find / -iname "psql"
5250e23f65bb:/# which mysql
/usr/bin/mysql
5250e23f65bb:/# which psgql
5250e23f65bb:/# which psql

    I would just cut a .100 tag personally, re-using tags causes all kinds of trouble but even if we ignore that I don't think this worked, or maybe I missed something?

  8. tiredofit commented on Mar 13, 2026

    @tiredofit
    tiredofit
    on Mar 13, 2026
    Collaborator

    Ok strange. Will look in AM have come down with an illness and that's not great for troubleshooting :)

  9. tlex commented on Mar 13, 2026

    @tlex
    tlex
    on Mar 13, 2026
    Contributor

    The image on ghcr.io isn't updated, just the one from docker hub:

    $ docker pull ghcr.io/tiredofit/docker-db-backup:4.1.99
4.1.99: Pulling from tiredofit/docker-db-backup
Digest: sha256:d547726ad13079fb1441b1d21f85f043a634cfeda1e013101ca8ba1fc3cf838e
Status: Image is up to date for ghcr.io/tiredofit/docker-db-backup:4.1.99
ghcr.io/tiredofit/docker-db-backup:4.1.99

    but

    $ docker pull tiredofit/db-backup:4.1.99
4.1.99: Pulling from tiredofit/db-backup
Digest: sha256:f58e73758ad83987d2d3696bb04edf2bb9e37af1f9d5d1ae95f60e58ae0b1512
Status: Image is up to date for tiredofit/db-backup:4.1.99
docker.io/tiredofit/db-backup:4.1.99
$
    Edit: indeed, the one from docker hub has the binaries: 
    $ docker run --rm -it --entrypoint "" tiredofit/db-backup:4.1.99 ls -lashi /usr/local/bin
total 32M
32748916    0 drwxr-xr-x 1 root root   50 Mar 13 03:33 .
32748915    0 drwxr-xr-x 1 root root    6 Mar 13 03:33 ..
32727777 120K -rwxr-xr-x 1 root root 118K Mar 13 03:41 clusterdb
32727778 120K -rwxr-xr-x 1 root root 118K Mar 13 03:41 createdb
32727779 120K -rwxr-xr-x 1 root root 119K Mar 13 03:41 createuser
32727780 116K -rwxr-xr-x 1 root root 114K Mar 13 03:41 dropdb
32727781 116K -rwxr-xr-x 1 root root 114K Mar 13 03:41 dropuser
32727782 888K -rwxr-xr-x 1 root root 887K Mar 13 03:41 ecpg
32709532  40K -rwxr-xr-x 1 root root  38K Mar 27  2024 envsubst
32727783 216K -rwxr-xr-x 1 root root 215K Mar 13 03:41 initdb
32748917 4.0K -rwxr-xr-x 1 root root  962 Mar 13 03:33 logrotate_dbbackup
32727784  64K -rwxr-xr-x 1 root root  61K Mar 13 03:41 oid2name
32727785 152K -rwxr-xr-x 1 root root 149K Mar 13 03:41 pg_amcheck
32727786  60K -rwxr-xr-x 1 root root  58K Mar 13 03:41 pg_archivecleanup
32727787 216K -rwxr-xr-x 1 root root 215K Mar 13 03:41 pg_basebackup
32727788  96K -rwxr-xr-x 1 root root  96K Mar 13 03:41 pg_checksums
32727789 188K -rwxr-xr-x 1 root root 187K Mar 13 03:41 pg_combinebackup
32727790  60K -rwxr-xr-x 1 root root  60K Mar 13 03:41 pg_config
32727791  76K -rwxr-xr-x 1 root root  74K Mar 13 03:41 pg_controldata
32727792 152K -rwxr-xr-x 1 root root 152K Mar 13 03:41 pg_createsubscriber
32727793  92K -rwxr-xr-x 1 root root  89K Mar 13 03:41 pg_ctl
32727794 532K -rwxr-xr-x 1 root root 530K Mar 13 03:41 pg_dump
32727795 300K -rwxr-xr-x 1 root root 298K Mar 13 03:41 pg_dumpall
32727796 116K -rwxr-xr-x 1 root root 113K Mar 13 03:41 pg_isready
32727797 128K -rwxr-xr-x 1 root root 127K Mar 13 03:41 pg_receivewal
32727798 128K -rwxr-xr-x 1 root root 126K Mar 13 03:41 pg_recvlogical
32727799  92K -rwxr-xr-x 1 root root  89K Mar 13 03:41 pg_resetwal
32727800 284K -rwxr-xr-x 1 root root 281K Mar 13 03:41 pg_restore
32727801 216K -rwxr-xr-x 1 root root 214K Mar 13 03:41 pg_rewind
32727802  64K -rwxr-xr-x 1 root root  63K Mar 13 03:41 pg_test_fsync
32727803  56K -rwxr-xr-x 1 root root  56K Mar 13 03:41 pg_test_timing
32727804 248K -rwxr-xr-x 1 root root 246K Mar 13 03:41 pg_upgrade
32727805 160K -rwxr-xr-x 1 root root 160K Mar 13 03:41 pg_verifybackup
32727806 140K -rwxr-xr-x 1 root root 138K Mar 13 03:41 pg_waldump
32727807  84K -rwxr-xr-x 1 root root  84K Mar 13 03:41 pg_walsummary
32727808 252K -rwxr-xr-x 1 root root 250K Mar 13 03:41 pgbench
32727809  12M -rwxr-xr-x 1 root root  12M Mar 13 03:41 postgres
32727810 1.1M -rwxr-xr-x 1 root root 1.1M Mar 13 03:41 psql
32727811 124K -rwxr-xr-x 1 root root 123K Mar 13 03:41 reindexdb
32748918  40K -rwxr-xr-x 1 root root  38K Mar 13 03:33 restore
32727812 136K -rwxr-xr-x 1 root root 136K Mar 13 03:41 vacuumdb
32727813  60K -rwxr-xr-x 1 root root  58K Mar 13 03:41 vacuumlo
32709533  14M -rwxr-xr-x 1 root root  14M May  5  2025 yq

  10. tamimology commented on Mar 14, 2026

    @tamimology
    tamimology
    on Mar 14, 2026

    Looks fine for me, except for this error that came up after the latest update last night

    [databases-backup] 2026-03-14T01:17:49.710892821Z 2026-03-14.12:17:49 [INFO] ** [06-sonarr-postgres__sonarr] Next Backup at 2026-03-14 12:17:49 AEDT
[databases-backup] 2026-03-14T01:17:49.916791996Z 2026-03-14.12:17:49 [NOTICE] ** [06-sonarr-postgres__sonarr] Dumping all PostgreSQL databases: 'sonarr' and compressing with 'zstd'
[databases-backup] 2026-03-14T01:17:50.682399527Z 2026-03-14.12:17:50 [INFO] ** [07-seerr-postgres__seerr] Next Backup at 2026-03-15 00:25:00 AEDT
[databases-backup] 2026-03-14T01:17:56.603805531Z 2026-03-14.12:17:56 [INFO] ** [06-sonarr-postgres__sonarr] DB Backup of 'pgsql_all_sonarr-postgres_20260314-121749.sql.zst' completed successfully
[databases-backup] 2026-03-14T01:17:56.632213186Z 2026-03-14.12:17:56 [NOTICE] ** [06-sonarr-postgres__sonarr] Generating SHA1 sum for 'pgsql_all_sonarr-postgres_20260314-121749.sql.zst'
[databases-backup] 2026-03-14T01:17:56.744041682Z 2026-03-14.12:17:56 [NOTICE] ** [06-sonarr-postgres__sonarr] Backup of 'pgsql_all_sonarr-postgres_20260314-121749.sql.zst' created with the size of 543824 bytes
[databases-backup] 2026-03-14T01:17:56.873456459Z ln: failed to create symbolic link '/backup/postgres/sonaarr/latest-pgsql__sonarr-postgres': I/O error
[databases-backup] 2026-03-14T01:17:56.960771999Z 2026-03-14.12:17:56 [NOTICE] ** [06-sonarr-postgres__sonarr] DB Backup for 'all' time taken: Hours: 0 Minutes: 00 Seconds: 07
[databases-backup] 2026-03-14T01:17:56.986145999Z 2026-03-14.12:17:56 [INFO] ** [06-sonarr-postgres__sonarr] Cleaning up old backups on filesystem
[databases-backup] 2026-03-14T01:17:57.018279153Z 2026-03-14.12:17:57 [INFO] ** [06-sonarr-postgres__sonarr] Backup 06 routines finish time: 2026-03-14 12:17:57 AEDT with exit code 0
  11. nfrastack
    locked and limited conversation to collaborators on Sep 4, 2026
  12. tiredofit
    converted this issue into a discussion #504 on Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bug

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions