Requirement

Just one! Install uv

Usage

OpenAI API: Create .env file with your OpenAI API key. Then run:

cd scripts
./compute.py

Llama.cpp: First download a GGUF file for the model you want to run. Then download and build this fork of Llama.cpp. Then run:

cd llama-cpp/build/bin
./llama-server -m <path to gguf> -ngl 999 -c 1024 --port 8080 -fa

And then, at the same time:

cd scripts
uv run compute.py --model qwen3 --shuffle --maxcount 3425 --model qwen3 --port 8080

Options

• --model NAME: Model name (default: davinci-002)
• --port N: Port for local server
• --maxcount N: Process only first N stories
• --no-resume: Start fresh (ignore checkpoint)
• --no-topic: Disable topic/summary, compare context to no context instead

Analyze Results

cd scripts
./analyze.py ../sequentiality_results_davinci-002_with-topic.csv