Interactsh Errors

[Sh4d0wHunt3rX]

Describe the bug
In the middle of the scan, bbot returns errors from different modules and it can't complete the scan. Since 3 days ago, couldn't finish any scan on tesla.com

BBOT Command
bbot -f passive subdomain-enum web-thorough cloud-enum -m gowitness fingerprintx wafw00f bypass403 -em smuggler azure_realm, azure_tenant bucket_amazon bucket_azure bucket_digitalocean bucket_file_enum bucket_firebase bucket_google -t tesla.com -om asset_inventory emails json | jq

OS, BBOT Installation Method + Version
OS: Linux Ubuntu, Installation method: pipx install --pip-args '\--pre' bbot, BBOT version: v1.1.6.2772rc

BBOT Config
Default

Screenshots

debug.log

I already reported this problem, just wanted to archive it here too : )

[TheTechromancer]

Thanks @Amiremami, I am working on this one.

[TheTechromancer]

I'm having trouble reproducing this bug. I've run the bbot command twice, and both times it finished in about one hour without any warnings/errors. Unfortunately it's hard to tell from the debug.log alone exactly what went wrong. @Amiremami is it possible you're being blocked/ratelimited by your provider?

[TheTechromancer]

According to debug.log, this was the last web error to happen before the bug triggered:

2024-02-02 10:00:29,840 [TRACE] bbot.core.helpers.web web.py:603 Error with request to URL: https://internetdb.shodan.io/40.99.201.216: Server disconnected without sending a response.
2024-02-02 10:00:29,841 [TRACE] bbot.core.helpers.web web.py:604 Traceback (most recent call last):
  File "/root/.local/pipx/venvs/bbot/lib/python3.10/site-packages/httpx/_transports/default.py", line 67, in map_httpcore_exceptions
    yield
  File "/root/.local/pipx/venvs/bbot/lib/python3.10/site-packages/httpx/_transports/default.py", line 371, in handle_async_request
    resp = await self._pool.handle_async_request(req)
  File "/root/.local/pipx/venvs/bbot/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 268, in handle_async_request
    raise exc
  File "/root/.local/pipx/venvs/bbot/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 251, in handle_async_request
    response = await connection.handle_async_request(request)
  File "/root/.local/pipx/venvs/bbot/lib/python3.10/site-packages/httpcore/_async/connection.py", line 103, in handle_async_request
    return await self._connection.handle_async_request(request)
  File "/root/.local/pipx/venvs/bbot/lib/python3.10/site-packages/httpcore/_async/http11.py", line 133, in handle_async_request
    raise exc
  File "/root/.local/pipx/venvs/bbot/lib/python3.10/site-packages/httpcore/_async/http11.py", line 111, in handle_async_request
    ) = await self._receive_response_headers(**kwargs)
  File "/root/.local/pipx/venvs/bbot/lib/python3.10/site-packages/httpcore/_async/http11.py", line 176, in _receive_response_headers
    event = await self._receive_event(timeout=timeout)
  File "/root/.local/pipx/venvs/bbot/lib/python3.10/site-packages/httpcore/_async/http11.py", line 226, in _receive_event
    raise RemoteProtocolError(msg)
httpcore.RemoteProtocolError: Server disconnected without sending a response.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/root/.local/pipx/venvs/bbot/lib/python3.10/site-packages/bbot/core/helpers/web.py", line 593, in _acatch
    yield
  File "/root/.local/pipx/venvs/bbot/lib/python3.10/site-packages/bbot/core/helpers/web.py", line 228, in request
    response = await client.request(*args, **kwargs)
  File "/root/.local/pipx/venvs/bbot/lib/python3.10/site-packages/bbot/core/helpers/web.py", line 86, in request
    return await super().request(*args, **kwargs)
  File "/root/.local/pipx/venvs/bbot/lib/python3.10/site-packages/httpx/_client.py", line 1559, in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
  File "/root/.local/pipx/venvs/bbot/lib/python3.10/site-packages/httpx/_client.py", line 1646, in send
    response = await self._send_handling_auth(
  File "/root/.local/pipx/venvs/bbot/lib/python3.10/site-packages/httpx/_client.py", line 1674, in _send_handling_auth
    response = await self._send_handling_redirects(
  File "/root/.local/pipx/venvs/bbot/lib/python3.10/site-packages/httpx/_client.py", line 1711, in _send_handling_redirects
    response = await self._send_single_request(request)
  File "/root/.local/pipx/venvs/bbot/lib/python3.10/site-packages/httpx/_client.py", line 1748, in _send_single_request
    response = await transport.handle_async_request(request)
  File "/root/.local/pipx/venvs/bbot/lib/python3.10/site-packages/httpx/_transports/default.py", line 370, in handle_async_request
    with map_httpcore_exceptions():
  File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/root/.local/pipx/venvs/bbot/lib/python3.10/site-packages/httpx/_transports/default.py", line 84, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.RemoteProtocolError: Server disconnected without sending a response.

This might help us reproduce the issue.