askrene crashes

[michael1011]

In the last 24 hours, our CLN crashed because of two different panics in askrene:

1767724611265   2026-01-06T18:36:51.265Z                ???:0
1767724611265   2026-01-06T18:36:51.265Z        0xffffffffffffffff ???
1767724611265   2026-01-06T18:36:51.265Z                _start+0x24:0
1767724611265   2026-01-06T18:36:51.265Z        0x55fc0c12deb4 ???
1767724611265   2026-01-06T18:36:51.265Z                __libc_start_main+0x8a:0
1767724611265   2026-01-06T18:36:51.265Z        0x7f79666fe28a ???
1767724611265   2026-01-06T18:36:51.265Z                ???:0
1767724611265   2026-01-06T18:36:51.265Z        0x7f79666fe1c9 ???
1767724611265   2026-01-06T18:36:51.265Z                plugins/askrene/askrene.c:1427
1767724611265   2026-01-06T18:36:51.265Z        0x55fc0c131b12 main
1767724611265   2026-01-06T18:36:51.265Z                plugins/libplugin.c:2429
1767724611265   2026-01-06T18:36:51.265Z        0x55fc0c143e6e plugin_main
1767724611265   2026-01-06T18:36:51.265Z                ccan/ccan/io/poll.c:470
1767724611265   2026-01-06T18:36:51.265Z        0x55fc0c1769a0 io_loop
1767724611265   2026-01-06T18:36:51.265Z                ccan/ccan/io/io.c:439
1767724611265   2026-01-06T18:36:51.265Z        0x55fc0c174f92 io_ready
1767724611265   2026-01-06T18:36:51.265Z                ccan/ccan/io/io.c:422
1767724611265   2026-01-06T18:36:51.265Z        0x55fc0c174ed5 do_plan
1767724611265   2026-01-06T18:36:51.265Z                ccan/ccan/io/io.c:60
1767724611265   2026-01-06T18:36:51.265Z        0x55fc0c174a04 next_plan
1767724611265   2026-01-06T18:36:51.265Z                plugins/libplugin.c:1377
1767724611265   2026-01-06T18:36:51.265Z        0x55fc0c1429e0 rpc_conn_read_response
1767724611265   2026-01-06T18:36:51.265Z                plugins/libplugin.c:1073
1767724611265   2026-01-06T18:36:51.265Z        0x55fc0c1428ac handle_rpc_reply
1767724611264   2026-01-06T18:36:51.264Z                plugins/askrene/askrene.c:801
1767724611264   2026-01-06T18:36:51.264Z        0x55fc0c13174c listpeerchannels_done
1767724611264   2026-01-06T18:36:51.264Z                plugins/askrene/askrene.c:669
1767724611264   2026-01-06T18:36:51.264Z        0x55fc0c130f55 do_getroutes
1767724611264   2026-01-06T18:36:51.264Z                plugins/askrene/mcf.c:1636
1767724611264   2026-01-06T18:36:51.264Z        0x55fc0c138bd7 default_routes
1767724611264   2026-01-06T18:36:51.264Z                plugins/askrene/mcf.c:1415
1767724611264   2026-01-06T18:36:51.264Z        0x55fc0c1382ec linear_routes
1767724611264   2026-01-06T18:36:51.264Z                plugins/askrene/refine.c:510
1767724611264   2026-01-06T18:36:51.264Z        0x55fc0c13b110 refine_flows
1767724611264   2026-01-06T18:36:51.264Z                plugins/askrene/refine.c:449
1767724611264   2026-01-06T18:36:51.264Z        0x55fc0c13ac9f increase_flows
1767724611264   2026-01-06T18:36:51.264Z                abort+0xde:0
1767724611264   2026-01-06T18:36:51.264Z        0x7f79666fc8fe ???
1767724611264   2026-01-06T18:36:51.264Z                gsignal+0x1d:0
1767724611264   2026-01-06T18:36:51.264Z        0x7f796671927d ???
1767724611264   2026-01-06T18:36:51.264Z                pthread_kill+0x11c:0
1767724611264   2026-01-06T18:36:51.264Z        0x7f7966772b2c ???
1767724611264   2026-01-06T18:36:51.264Z                ???:0
1767724611264   2026-01-06T18:36:51.264Z        0x7f796671932f ???
1767724611264   2026-01-06T18:36:51.264Z                common/daemon.c:83
1767724611264   2026-01-06T18:36:51.264Z        0x55fc0c145803 crashdump
1767724611264   2026-01-06T18:36:51.264Z                common/daemon.c:38
1767724611264   2026-01-06T18:36:51.264Z        0x55fc0c145767 send_backtrace
1767724611257   2026-01-06T18:36:51.257Z        cln-askrene: FATAL SIGNAL 6 (version 25.12)

and

1767787181708   2026-01-07T11:59:41.708Z                ???:0
1767787181708   2026-01-07T11:59:41.708Z        0xffffffffffffffff ???
1767787181708   2026-01-07T11:59:41.708Z                _start+0x24:0
1767787181708   2026-01-07T11:59:41.708Z        0x55876218aeb4 ???
1767787181708   2026-01-07T11:59:41.708Z                __libc_start_main+0x8a:0
1767787181708   2026-01-07T11:59:41.708Z        0x7f5ac534228a ???
1767787181708   2026-01-07T11:59:41.708Z                ???:0
1767787181708   2026-01-07T11:59:41.708Z        0x7f5ac53421c9 ???
1767787181708   2026-01-07T11:59:41.708Z                plugins/askrene/askrene.c:1427
1767787181708   2026-01-07T11:59:41.708Z        0x55876218eb12 main
1767787181708   2026-01-07T11:59:41.708Z                plugins/libplugin.c:2429
1767787181708   2026-01-07T11:59:41.708Z        0x5587621a0e6e plugin_main
1767787181708   2026-01-07T11:59:41.708Z                ccan/ccan/io/poll.c:470
1767787181708   2026-01-07T11:59:41.708Z        0x5587621d39a0 io_loop
1767787181708   2026-01-07T11:59:41.708Z                ccan/ccan/io/io.c:439
1767787181708   2026-01-07T11:59:41.708Z        0x5587621d1f92 io_ready
1767787181708   2026-01-07T11:59:41.708Z                ccan/ccan/io/io.c:422
1767787181708   2026-01-07T11:59:41.708Z        0x5587621d1ed5 do_plan
1767787181708   2026-01-07T11:59:41.708Z                ccan/ccan/io/io.c:60
1767787181708   2026-01-07T11:59:41.708Z        0x5587621d1a04 next_plan
1767787181708   2026-01-07T11:59:41.708Z                plugins/libplugin.c:1377
1767787181708   2026-01-07T11:59:41.708Z        0x55876219f9e0 rpc_conn_read_response
1767787181708   2026-01-07T11:59:41.708Z                plugins/libplugin.c:1073
1767787181708   2026-01-07T11:59:41.708Z        0x55876219f8ac handle_rpc_reply
1767787181708   2026-01-07T11:59:41.708Z                plugins/askrene/askrene.c:801
1767787181708   2026-01-07T11:59:41.708Z        0x55876218e74c listpeerchannels_done
1767787181708   2026-01-07T11:59:41.708Z                plugins/askrene/askrene.c:664
1767787181708   2026-01-07T11:59:41.708Z        0x55876218e157 do_getroutes
1767787181708   2026-01-07T11:59:41.708Z                plugins/askrene/mcf.c:1649
1767787181708   2026-01-07T11:59:41.708Z        0x558762195c0b single_path_routes
1767787181708   2026-01-07T11:59:41.708Z                plugins/askrene/mcf.c:1404
1767787181708   2026-01-07T11:59:41.708Z        0x558762195397 linear_routes
1767787181708   2026-01-07T11:59:41.708Z                plugins/askrene/mcf.c:1213
1767787181708   2026-01-07T11:59:41.708Z        0x558762194cea single_path_flow
1767787181708   2026-01-07T11:59:41.708Z                plugins/askrene/mcf.c:474
1767787181708   2026-01-07T11:59:41.708Z        0x5587621938ec combine_cost_function
1767787181707   2026-01-07T11:59:41.707Z                __assert_fail+0x46:0
1767787181707   2026-01-07T11:59:41.707Z        0x7f5ac5353516 ???
1767787181707   2026-01-07T11:59:41.707Z                ???:0
1767787181707   2026-01-07T11:59:41.707Z        0x7f5ac534081a ???
1767787181707   2026-01-07T11:59:41.707Z                abort+0xde:0
1767787181707   2026-01-07T11:59:41.707Z        0x7f5ac53408fe ???
1767787181707   2026-01-07T11:59:41.707Z                gsignal+0x1d:0
1767787181707   2026-01-07T11:59:41.707Z        0x7f5ac535d27d ???
1767787181707   2026-01-07T11:59:41.707Z                pthread_kill+0x11c:0
1767787181707   2026-01-07T11:59:41.707Z        0x7f5ac53b6b2c ???
1767787181707   2026-01-07T11:59:41.707Z                ???:0
1767787181707   2026-01-07T11:59:41.707Z        0x7f5ac535d32f ???
1767787181707   2026-01-07T11:59:41.707Z                common/daemon.c:83
1767787181707   2026-01-07T11:59:41.707Z        0x5587621a2803 crashdump
1767787181707   2026-01-07T11:59:41.707Z                common/daemon.c:38
1767787181707   2026-01-07T11:59:41.707Z        0x5587621a2767 send_backtrace
1767787181699   2026-01-07T11:59:41.699Z        cln-askrene: FATAL SIGNAL 6 (version 25.12)
1767787181699   2026-01-07T11:59:41.699Z        cln-askrene: plugins/askrene/mcf.c:474: combine_cost_function: Assertion `fcost != INFINITE' failed.

The second one happened multiple times

[vincenzopalazzo]

What is the version of cln that are you running?

[michael1011]

1767787181699 2026-01-07T11:59:41.699Z cln-askrene: FATAL SIGNAL 6 (version 25.12)

Release v25.12

[michael1011]

Same thing twice today

1767873530867   2026-01-08T11:58:50.867Z        2026-01-08T11:58:50.867Z INFO    plugin-cln-xpay: 115: getroute failed with maxparts=6, so retrying without that restriction
1767873530867   2026-01-08T11:58:50.867Z        2026-01-08T11:58:50.867Z **BROKEN** plugin-cln-askrene: Plugin marked as important, shutting down lightningd!
1767873530867   2026-01-08T11:58:50.867Z        2026-01-08T11:58:50.867Z INFO    plugin-cln-askrene: Killing plugin: exited during normal operation
1767873530747   2026-01-08T11:58:50.747Z        2026-01-08T11:58:50.747005Z TRACE hold::settler: Checking for MPP timeouts
1767873530718   2026-01-08T11:58:50.718Z                ???:0
1767873530718   2026-01-08T11:58:50.718Z        0xffffffffffffffff ???
1767873530718   2026-01-08T11:58:50.718Z                _start+0x24:0
1767873530718   2026-01-08T11:58:50.718Z        0x556c96e1aeb4 ???
1767873530718   2026-01-08T11:58:50.718Z                __libc_start_main+0x8a:0
1767873530718   2026-01-08T11:58:50.718Z        0x7fb1b06a228a ???
1767873530718   2026-01-08T11:58:50.718Z                ???:0
1767873530718   2026-01-08T11:58:50.718Z        0x7fb1b06a21c9 ???
1767873530718   2026-01-08T11:58:50.718Z                plugins/askrene/askrene.c:1427
1767873530718   2026-01-08T11:58:50.718Z        0x556c96e1eb12 main
1767873530718   2026-01-08T11:58:50.718Z                plugins/libplugin.c:2429
1767873530718   2026-01-08T11:58:50.718Z        0x556c96e30e6e plugin_main
1767873530718   2026-01-08T11:58:50.718Z                ccan/ccan/io/poll.c:470
1767873530718   2026-01-08T11:58:50.718Z        0x556c96e639a0 io_loop
1767873530718   2026-01-08T11:58:50.718Z                ccan/ccan/io/io.c:439
1767873530718   2026-01-08T11:58:50.718Z        0x556c96e61f92 io_ready
1767873530718   2026-01-08T11:58:50.718Z                ccan/ccan/io/io.c:422
1767873530718   2026-01-08T11:58:50.718Z        0x556c96e61ed5 do_plan
1767873530718   2026-01-08T11:58:50.718Z                ccan/ccan/io/io.c:60
1767873530718   2026-01-08T11:58:50.718Z        0x556c96e61a04 next_plan
1767873530718   2026-01-08T11:58:50.718Z                plugins/libplugin.c:1377
1767873530718   2026-01-08T11:58:50.718Z        0x556c96e2f9e0 rpc_conn_read_response
1767873530718   2026-01-08T11:58:50.718Z                plugins/libplugin.c:1073
1767873530718   2026-01-08T11:58:50.718Z        0x556c96e2f8ac handle_rpc_reply
1767873530717   2026-01-08T11:58:50.717Z                plugins/askrene/askrene.c:801
1767873530717   2026-01-08T11:58:50.717Z        0x556c96e1e74c listpeerchannels_done
1767873530717   2026-01-08T11:58:50.717Z                plugins/askrene/askrene.c:664
1767873530717   2026-01-08T11:58:50.717Z        0x556c96e1e157 do_getroutes
1767873530717   2026-01-08T11:58:50.717Z                plugins/askrene/mcf.c:1649
1767873530717   2026-01-08T11:58:50.717Z        0x556c96e25c0b single_path_routes
1767873530717   2026-01-08T11:58:50.717Z                plugins/askrene/mcf.c:1404
1767873530717   2026-01-08T11:58:50.717Z        0x556c96e25397 linear_routes
1767873530717   2026-01-08T11:58:50.717Z                plugins/askrene/mcf.c:1213
1767873530717   2026-01-08T11:58:50.717Z        0x556c96e24cea single_path_flow
1767873530717   2026-01-08T11:58:50.717Z                plugins/askrene/mcf.c:474
1767873530717   2026-01-08T11:58:50.717Z        0x556c96e238ec combine_cost_function
1767873530717   2026-01-08T11:58:50.717Z                __assert_fail+0x46:0
1767873530717   2026-01-08T11:58:50.717Z        0x7fb1b06b3516 ???
1767873530717   2026-01-08T11:58:50.717Z                ???:0
1767873530717   2026-01-08T11:58:50.717Z        0x7fb1b06a081a ???
1767873530717   2026-01-08T11:58:50.717Z                abort+0xde:0
1767873530717   2026-01-08T11:58:50.717Z        0x7fb1b06a08fe ???
1767873530717   2026-01-08T11:58:50.717Z                gsignal+0x1d:0
1767873530717   2026-01-08T11:58:50.717Z        0x7fb1b06bd27d ???
1767873530717   2026-01-08T11:58:50.717Z                pthread_kill+0x11c:0
1767873530717   2026-01-08T11:58:50.717Z        0x7fb1b0716b2c ???
1767873530717   2026-01-08T11:58:50.717Z                ???:0
1767873530717   2026-01-08T11:58:50.717Z        0x7fb1b06bd32f ???
1767873530717   2026-01-08T11:58:50.717Z                common/daemon.c:83
1767873530717   2026-01-08T11:58:50.717Z        0x556c96e32803 crashdump
1767873530717   2026-01-08T11:58:50.717Z                common/daemon.c:38
1767873530717   2026-01-08T11:58:50.717Z        0x556c96e32767 send_backtrace
1767873530709   2026-01-08T11:58:50.709Z        cln-askrene: FATAL SIGNAL 6 (version 25.12)
1767873530709   2026-01-08T11:58:50.709Z        cln-askrene: plugins/askrene/mcf.c:474: combine_cost_function: Assertion `fcost != INFINITE' failed.

[michael1011]

Again

1767882305911	2026-01-08T14:25:05.911Z	2026-01-08T14:25:05.911Z **BROKEN** plugin-cln-askrene: Plugin marked as important, shutting down lightningd!
1767882305911	2026-01-08T14:25:05.911Z	2026-01-08T14:25:05.911Z INFO    plugin-cln-askrene: Killing plugin: exited during normal operation
1767882305840	2026-01-08T14:25:05.840Z	2026-01-08T14:25:05.840642Z DEBUG hold::handler: No hold invoice for: 56e06d548dafa4a0183b6f0098a0193cdf9a83e3f4998b17a52541c02a32576a
1767882305767	2026-01-08T14:25:05.767Z		???:0
1767882305767	2026-01-08T14:25:05.767Z	0xffffffffffffffff ???
1767882305767	2026-01-08T14:25:05.767Z		_start+0x24:0
1767882305767	2026-01-08T14:25:05.767Z	0x5645479efeb4 ???
1767882305767	2026-01-08T14:25:05.767Z		__libc_start_main+0x8a:0
1767882305767	2026-01-08T14:25:05.767Z	0x7f377946c28a ???
1767882305767	2026-01-08T14:25:05.767Z		???:0
1767882305767	2026-01-08T14:25:05.767Z	0x7f377946c1c9 ???
1767882305767	2026-01-08T14:25:05.767Z		plugins/askrene/askrene.c:1427
1767882305767	2026-01-08T14:25:05.767Z	0x5645479f3b12 main
1767882305767	2026-01-08T14:25:05.767Z		plugins/libplugin.c:2429
1767882305767	2026-01-08T14:25:05.767Z	0x564547a05e6e plugin_main
1767882305767	2026-01-08T14:25:05.767Z		ccan/ccan/io/poll.c:470
1767882305767	2026-01-08T14:25:05.767Z	0x564547a389a0 io_loop
1767882305767	2026-01-08T14:25:05.767Z		ccan/ccan/io/io.c:439
1767882305767	2026-01-08T14:25:05.767Z	0x564547a36f92 io_ready
1767882305767	2026-01-08T14:25:05.767Z		ccan/ccan/io/io.c:422
1767882305767	2026-01-08T14:25:05.767Z	0x564547a36ed5 do_plan
1767882305767	2026-01-08T14:25:05.767Z		ccan/ccan/io/io.c:60
1767882305767	2026-01-08T14:25:05.767Z	0x564547a36a04 next_plan
1767882305767	2026-01-08T14:25:05.767Z		plugins/libplugin.c:1377
1767882305767	2026-01-08T14:25:05.767Z	0x564547a049e0 rpc_conn_read_response
1767882305767	2026-01-08T14:25:05.767Z		plugins/libplugin.c:1073
1767882305767	2026-01-08T14:25:05.767Z	0x564547a048ac handle_rpc_reply
1767882305766	2026-01-08T14:25:05.766Z		plugins/askrene/askrene.c:801
1767882305766	2026-01-08T14:25:05.766Z	0x5645479f374c listpeerchannels_done
1767882305766	2026-01-08T14:25:05.766Z		plugins/askrene/askrene.c:664
1767882305766	2026-01-08T14:25:05.766Z	0x5645479f3157 do_getroutes
1767882305766	2026-01-08T14:25:05.766Z		plugins/askrene/mcf.c:1649
1767882305766	2026-01-08T14:25:05.766Z	0x5645479fac0b single_path_routes
1767882305766	2026-01-08T14:25:05.766Z		plugins/askrene/mcf.c:1404
1767882305766	2026-01-08T14:25:05.766Z	0x5645479fa397 linear_routes
1767882305766	2026-01-08T14:25:05.766Z		plugins/askrene/mcf.c:1213
1767882305766	2026-01-08T14:25:05.766Z	0x5645479f9cea single_path_flow
1767882305766	2026-01-08T14:25:05.766Z		plugins/askrene/mcf.c:474
1767882305766	2026-01-08T14:25:05.766Z	0x5645479f88ec combine_cost_function
1767882305766	2026-01-08T14:25:05.766Z		__assert_fail+0x46:0
1767882305766	2026-01-08T14:25:05.766Z	0x7f377947d516 ???
1767882305766	2026-01-08T14:25:05.766Z		???:0
1767882305766	2026-01-08T14:25:05.766Z	0x7f377946a81a ???
1767882305766	2026-01-08T14:25:05.766Z		abort+0xde:0
1767882305766	2026-01-08T14:25:05.766Z	0x7f377946a8fe ???
1767882305766	2026-01-08T14:25:05.766Z		gsignal+0x1d:0
1767882305766	2026-01-08T14:25:05.766Z	0x7f377948727d ???
1767882305766	2026-01-08T14:25:05.766Z		pthread_kill+0x11c:0
1767882305766	2026-01-08T14:25:05.766Z	0x7f37794e0b2c ???
1767882305766	2026-01-08T14:25:05.766Z		???:0
1767882305766	2026-01-08T14:25:05.766Z	0x7f377948732f ???
1767882305766	2026-01-08T14:25:05.766Z		common/daemon.c:83
1767882305766	2026-01-08T14:25:05.766Z	0x564547a07803 crashdump
1767882305766	2026-01-08T14:25:05.766Z		common/daemon.c:38
1767882305766	2026-01-08T14:25:05.766Z	0x564547a07767 send_backtrace
1767882305758	2026-01-08T14:25:05.758Z	cln-askrene: FATAL SIGNAL 6 (version 25.12)
1767882305758	2026-01-08T14:25:05.758Z	cln-askrene: plugins/askrene/mcf.c:474: combine_cost_function: Assertion `fcost != INFINITE' failed.

[michael1011]

It's effectively crashing every couple hours by now. The node completely unusable for us