Skip to content

Fix syscheck queue/restart races and honor FIM frequency under realtime. - #2295

Merged
atomicturtle merged 4 commits into
ossec:mainfrom
atomicturtle:fix/syscheck-mq-restart-frequency
Aug 12, 2026
Merged

Fix syscheck queue/restart races and honor FIM frequency under realtime.#2295
atomicturtle merged 4 commits into
ossec:mainfrom
atomicturtle:fix/syscheck-mq-restart-frequency

Conversation

@atomicturtle

Copy link
Copy Markdown
Member

Bind analysisd MQ early, wake recv on shutdown with PID wait before restart, and use adaptive syscheck idle waits so short is not capped by SYSCHECK_WAIT.

Bind analysisd MQ early, wake recv on shutdown with PID wait before restart,
and use adaptive syscheck idle waits so short <frequency> is not capped by
SYSCHECK_WAIT.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses startup/shutdown race conditions around the analysisd message queue and improves syscheck daemon idling so configured FIM/rootcheck frequencies are honored even under realtime mode.

Changes:

  • Add adaptive idle waiting in syscheckd so short <frequency> values aren’t effectively capped by SYSCHECK_WAIT.
  • Make MQ connection readiness based on actual AF_UNIX connect success (not just socket-path existence) and bind analysisd’s queue earlier during startup.
  • Improve restart/shutdown behavior by waiting for PIDs to exit and waking analysisd’s receive thread during shutdown.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
src/syscheckd/run_check.c Computes an adaptive idle wait so daemon-loop sleep/select respects configured scan cadences.
src/shared/mq_op.c Changes StartMQ(WRITE) to wait for successful connect and updates SendMSG retry/close behavior.
src/os_net/os_net.c Fixes unlink error handling by checking errno correctly.
src/init/ossec-server.sh Adds PID-exit waiting and queue-readiness gating before starting producers.
src/init/ossec-local.sh Same PID-exit waiting and analysis queue readiness gating for local installs.
src/init/ossec-client.sh Adds PID-exit waiting and agent queue readiness gating before starting local producers.
src/analysisd/pipeline.c Adds poll-based recv loop and closes/wakes input queue during shutdown.
src/analysisd/analysisd.c Binds analysisd’s queue earlier (post-chroot) so producers don’t race stale sockets.
Suppressed comments (3)

src/init/ossec-server.sh:350

  • This script uses a /bin/sh shebang, but local is not POSIX and will fail under common /bin/sh implementations. Remove local declarations in wait_for_analysis_queue().
    local elapsed=0
    local max=60
    local qpath="${DIR}/queue/ossec/queue"

src/init/ossec-local.sh:335

  • This script uses a /bin/sh shebang, but local is not POSIX and will fail under common /bin/sh implementations. Remove local declarations in wait_for_analysis_queue().
    local elapsed=0
    local max=60
    local qpath="${DIR}/queue/ossec/queue"

src/init/ossec-client.sh:257

  • This script uses a /bin/sh shebang, but local is not POSIX and will fail under common /bin/sh implementations. Remove local declarations in wait_for_agent_queue().
    local elapsed=0
    local max=60
    local qpath="${DIR}/queue/ossec/queue"

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/shared/mq_op.c
Comment thread src/init/ossec-server.sh Outdated
Comment thread src/analysisd/pipeline.c
Comment thread src/init/ossec-server.sh Outdated

while [ ${elapsed} -lt ${max} ]; do
if [ -S "${qpath}" ] || [ -e "${qpath}" ]; then
python3 - "${qpath}" <<'PY' 2>/dev/null
Comment thread src/init/ossec-local.sh Outdated
Comment thread src/init/ossec-local.sh Outdated
Comment thread src/init/ossec-client.sh Outdated
Comment thread src/init/ossec-client.sh Outdated
Drop non-POSIX local from init helpers, probe queues with python3/python
fail-fast, preserve connect errno in StartMQ, and serialize analysisd
input-queue close against double-close races.
Bind the analysis queue after decoder/rule load so control-script
connect waits do not start producers before recv can drain. Reconnect
logcollector on ignored SendMSG failures after SendMSG closes the fd.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Suppressed comments (5)

src/init/ossec-server.sh:362

  • Hard-requiring python here makes ossec-control start fail on minimal systems even though the daemons themselves are C binaries. Since StartMQ(WRITE) already has its own connect-wait logic, consider degrading gracefully when python isn’t available (warn and continue) instead of aborting startup.
    else
        echo "ERROR: python3 or python required to probe analysis queue at ${qpath}"
        return 1
    fi

src/shared/mq_op.c:117

  • This updated comment documents that SendMSG() may close the queue fd on failure, but a number of existing call sites don’t consistently handle a -1 return (some only log; others retry once and ignore the second result). With a closed fd, later SendMSG() calls can hit EBADF or (worse) send on a reused/unrelated fd number. If this close-on-error behavior is required, the PR should also update all SendMSG() callers to treat -1 as “fd is dead” and reconnect before any reuse (or change the API to take an int *queue so SendMSG can invalidate the caller’s fd).
    /* Five send attempts; after the first failure, sleep 1/3/5/10s
     * between retries (total sleep budget 19s). OS_SOCKTERR is fatal
     * only on the initial attempt; close(queue) so callers that StartMQ
     * again do not leak the dead fd. Callers must not reuse queue after -1
     * without StartMQ (pass-by-value cannot clear the caller's variable).

src/init/ossec-server.sh:312

  • Before sending SIGKILL, re-validate the PID still belongs to an OSSEC process. With the new 45s grace period, a PID can exit and be reused, and kill -9 could hit an unrelated process.

This issue also appears on line 359 of the same file.

        if [ $? = 0 ]; then
            echo "Process ${pid} did not exit; sending SIGKILL .."
            kill -9 ${pid} >/dev/null 2>&1
        fi

src/init/ossec-local.sh:347

  • Hard-requiring python here makes startup fail on systems without python even if the daemons can run. Since producers can already rely on StartMQ(WRITE) retry/connect behavior, consider warning and continuing when python isn’t available rather than aborting.
    else
        echo "ERROR: python3 or python required to probe analysis queue at ${qpath}"
        return 1
    fi

src/init/ossec-client.sh:269

  • This introduces a hard dependency on python for agent startup. On minimal installations without python, ossec-control start will now fail even though the agent daemons are native binaries. Consider warning and continuing when python isn’t present (letting StartMQ retries handle readiness) instead of returning failure here.
    else
        echo "ERROR: python3 or python required to probe agent queue at ${qpath}"
        return 1
    fi

Comment thread src/analysisd/pipeline.c Outdated
Comment on lines +72 to +73
static volatile sig_atomic_t analysisd_shutting_down = 0;
static volatile sig_atomic_t pipeline_m_queue = -1;
sig_atomic_t only covers signal handlers; concurrent pthread access
to analysisd_shutting_down and pipeline_m_queue needs _Atomic stores/loads.
@atomicturtle
atomicturtle merged commit 36eb20a into ossec:main Aug 12, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants