gptel--rewrite-directive-default invoked twice

[sp1ff]

Please update gptel first -- errors are often fixed by the time they're reported.

• I have updated gptel to the latest commit and tested that the issue still exists

Bug Description

During the execution of gptel-rewrite, gptel--rewrite-directive-default is invoked twice. The first time is in the source buffer (i.e. the buffer from which gptel-rewrite is invoked) and the second from some sort of transient buffer, always named "gptel-prompt-nnnnn". The assumption, I guess, is that both invocations will return the same value, but that is not always true (see the example, below). When the two invocations return different values, it is misleading to the user (since a different system prompt is being silently sent).

Backend

None

Steps to Reproduce

Write a little advice to customize the system prompt when working in a particular project:

(defun sp1ff//gptel/first-rewrite-directive (orig-fun &rest args)
  (message "My advice is being invoked in %s, bfn returns %s, and major-mode is %s." (buffer-name) (buffer-file-name) major-mode)
  (let ((bfn (buffer-file-name)))
    (if (and (eq 'rust-mode major-mode) bfn (string-match "mpdpopm" bfn 0 t))
        ;; "Overall, our findings suggest that while adding a persona may lead to
        ;; performance gains in certain settings, the effect of each persona can
        ;; be largely random." --https://arxiv.org/abs/2311.10054
        (concat "Follow the user instructions in refactoring the provided Rust source code.\n"
                "- Generate ONLY Rust code as output, without any explanation or markdown code fences.\n"
                "- Generate code in full, do not abbreviate or omit code.")
      (message "I am deferring to the default implementation #'gptel--rewrite-directive-default'")
      (apply orig-fun args))))

then advise:

(advice-add 'gptel--rewrite-directive-default' :around #'sp1ff//gptel/first-rewrite-directive)

Additional Context

Emacs compiled from source, Arch Linux.

Backtrace

Log Information

My advice is being invoked in messages.rs, bfn returns /home/mgh/code/projects/mpdpopm/mpdpopm/src/messages.rs, and major-mode is rust-mode.
...
My advice is being invoked in  *gptel-prompt*-375581, bfn returns nil, and major-mode is rust-mode.

[karthink]

The directive being called twice is not an issue, the first time is for the preview shown in the minibuffer when typing in the rewrite instructions.

The issue is that the directive is evaluated in a temp buffer instead of the original buffer. A fix is in the works, see the discussion in #1095.

[sp1ff]

I understand that the first time is for the preview; the bug (IMHO) is that you're suggesting to the user that the prompt displayed in the preview is what's going to be sent to the backend, when in reality a completely different prompt could be sent (and was, in my case). Your proposed fix to #1095 would certainly be welcome, but won't address this issue: there's just no way to be sure that the second invocation of the directive will return the same value as the first invocation.

Having invoked the directive once, in the source buffer, and displayed the resulting prompt to the user in the preview (indicating rather strongly that that is what's going to go out on the wire), why not just send that already computed system prompt to gptel-request and save yourself the second invocation?

[karthink]

I disagree, the issue is that the gptel-request API (which gptel-rewrite is a consumer of) is buggy in how it handles function-valued system prompts. Working around the issue downstream is possible, but a bad solution in the long run. gptel-rewrite is not the only feature that is experiencing this bug. I would rather take some time to address the subtleties I mention in #1095 and fix it at the source.

there's just no way to be sure that the second invocation of the directive will return the same value as the first invocation.

Unless two consecutive invocations of the function in the same Emacs context (same buffer state, etc) produce two different system prompt strings, this shouldn't be an issue.

[sp1ff]

there's just no way to be sure that the second invocation of the directive will return the same value as the first invocation.

Unless two consecutive invocations of the function in the same Emacs context (same buffer state, etc) produce two different system prompt strings, this shouldn't be an issue.

But... that's exactly what happened to me, and will happen generally. Emacs Lisp is not a "pure" functional language in that it's functions are allowed to have side effects. If this were Haskell, in which calling a function twice with the same parameters will always give the same result, you would be correct. But this is not Haskell. Two consecutive invocations of a function in the same Emacs context (same buffer state, etc) can absolutely produced two different values; it's literally built into the language:

(defvar n 0 "Global state")
(defun f () (setq n (1+ n)))

No two invocations of f in this example will ever produce the same value.

Look-- it's your package and you're free to address this issue however you see fit. I merely reached-out to let you know I had an issue using it, burning quite a bit of time tracking down why gptel-rewrite wasn't sending the prompt I thought it was.

All the best.

[karthink]

I had an issue using it, burning quite a bit of time tracking down why gptel-rewrite wasn't sending the prompt I thought it was.

Thanks for taking the time.

I've pushed a fix, please update and let me know if it behaves as expected.

To examine the request, you don't need to advise gptel--rewrite-directive-default -- you can use the dry-run option in the gptel-rewrite menu.

But... that's exactly what happened to me, and will happen generally. Emacs Lisp is not a "pure" functional language in that it's functions are allowed to have side effects.

I'm well aware. But this is not what will "happen generally" in this context. What happened to you (and in #1095) is that gptel invisibly swapped out some state that your function depended on, which is now (hopefully) fixed.

No two invocations of f in this example will ever produce the same value.

If your function that produces a system prompt modifies some state that it uses to produce the system prompt, there's not much gptel can do -- there will be too many holes given how Emacs works.