random767435 by random767435 · Pull Request #266 · tempestphp/100-million-row-challenge

random767435 · 2026-03-12T07:48:53Z

The unique algorithm 🤘 invented for this challenge:

Any presented URL can be uniquely defined by its 22-character tail. Since the shortest URL (/uses) is 29 characters, the tail allows the URL index to be encoded in a similar hash table.
This makes it possible to process data chunks by reading them from right to left. By doing so, we can first read the date stamp (via a fixed offset), then the URL tail (also via a fixed offset), and then jump to the next line
Additionally both URL index and URL length are encoded in the same hash, and it allows to lookup on the one hash table only (and saves additional time).

As a result, this algorithm is ~20% more efficient than a "standard" left-to-right sliding approach using strpos (because it excludes 100M strpos calls) 😌

random767435 · 2026-03-12T07:49:47Z

/bench

brendt · 2026-03-12T09:07:52Z

Benchmarking complete! Best execution time: 2.542s

Full results:

{
  "results": [
    {
      "command": "cd /Users/brentroose/Dev/100-million-row-challenge/app/Commands/../../.benchmark/pr-266 && php -dmax_execution_time=300 tempest data:parse --input-path=\"/Users/brentroose/Dev/100-million-row-challenge/app/Commands/../../data/real-data.csv\" --output-path=\"/Users/brentroose/Dev/100-million-row-challenge/app/Commands/../../data/real-data-actual.json\"",
      "mean": 2.55149121888,
      "stddev": 0.009021873639487562,
      "median": 2.54806082748,
      "user": 16.11734322,
      "system": 1.7361627600000002,
      "min": 2.54272507748,
      "max": 2.56613370248,
      "times": [
        2.54806082748,
        2.56613370248,
        2.55339636848,
        2.5471401184799998,
        2.54272507748
      ],
      "memory_usage_byte": [
        74219520,
        74219520,
        74219520,
        74219520,
        74219520
      ],
      "exit_codes": [
        0,
        0,
        0,
        0,
        0
      ]
    }
  ]
}

random767435 · 2026-03-12T20:09:46Z

/bench

brendt · 2026-03-12T20:49:25Z

Benchmarking complete! Best execution time: 2.602s

Full results:

{
  "results": [
    {
      "command": "cd /Users/brentroose/Dev/100-million-row-challenge/app/Commands/../../.benchmark/pr-266 && php -dmax_execution_time=300 tempest data:parse --input-path=\"/Users/brentroose/Dev/100-million-row-challenge/app/Commands/../../data/real-data.csv\" --output-path=\"/Users/brentroose/Dev/100-million-row-challenge/app/Commands/../../data/real-data-actual.json\"",
      "mean": 2.6246863872799997,
      "stddev": 0.020150453537264905,
      "median": 2.6253111204799997,
      "user": 16.94691388,
      "system": 1.58959684,
      "min": 2.60268878748,
      "max": 2.64741853748,
      "times": [
        2.60268878748,
        2.6416448284799996,
        2.64741853748,
        2.60636866248,
        2.6253111204799997
      ],
      "memory_usage_byte": [
        74088448,
        74088448,
        74088448,
        74088448,
        74088448
      ],
      "exit_codes": [
        0,
        0,
        0,
        0,
        0
      ]
    }
  ]
}

random767435 · 2026-03-12T21:01:21Z

/bench

random767435 · 2026-03-12T23:02:33Z

/bench

brendt · 2026-03-12T23:40:55Z

Benchmarking complete! Best execution time: 2.449s

Full results:

{
  "results": [
    {
      "command": "cd /Users/brentroose/Dev/100-million-row-challenge/app/Commands/../../.benchmark/pr-266 && php -dmax_execution_time=300 tempest data:parse --input-path=\"/Users/brentroose/Dev/100-million-row-challenge/app/Commands/../../data/real-data.csv\" --output-path=\"/Users/brentroose/Dev/100-million-row-challenge/app/Commands/../../data/real-data-actual.json\"",
      "mean": 2.4623941148799995,
      "stddev": 0.010637497703390642,
      "median": 2.46028953188,
      "user": 16.283930759999997,
      "system": 0.8963642399999999,
      "min": 2.44901794788,
      "max": 2.47798423988,
      "times": [
        2.46595994788,
        2.47798423988,
        2.44901794788,
        2.46028953188,
        2.4587189068799997
      ],
      "memory_usage_byte": [
        74334208,
        74334208,
        74334208,
        74334208,
        74334208
      ],
      "exit_codes": [
        0,
        0,
        0,
        0,
        0
      ]
    }
  ]
}

brendt · 2026-03-12T23:40:58Z

You didn’t optimize. You performed violence (on latency). 🔪
🏆 leaderboard.csv

random767435 · 2026-03-13T06:35:43Z

/bench

brendt · 2026-03-13T09:09:22Z

Benchmarking complete! Best execution time: 2.627s

Full results:

{
  "results": [
    {
      "command": "cd /Users/brentroose/Dev/100-million-row-challenge/app/Commands/../../.benchmark/pr-266 && php -dmax_execution_time=300 tempest data:parse --input-path=\"/Users/brentroose/Dev/100-million-row-challenge/app/Commands/../../data/real-data.csv\" --output-path=\"/Users/brentroose/Dev/100-million-row-challenge/app/Commands/../../data/real-data-actual.json\"",
      "mean": 2.63730512572,
      "stddev": 0.0089991577442048,
      "median": 2.63655013352,
      "user": 16.62120316,
      "system": 1.97577654,
      "min": 2.62739884252,
      "max": 2.65070280052,
      "times": [
        2.63655013352,
        2.65070280052,
        2.63135471752,
        2.64051913452,
        2.62739884252
      ],
      "memory_usage_byte": [
        74088448,
        74137600,
        74137600,
        74137600,
        74285056
      ],
      "exit_codes": [
        0,
        0,
        0,
        0,
        0
      ]
    }
  ]
}

random767435 · 2026-03-13T22:55:56Z

/bench

brendt · 2026-03-13T22:59:55Z

Benchmarking complete! Best execution time: 2.465s

Full results:

{
  "results": [
    {
      "command": "cd /Users/brentroose/Dev/100-million-row-challenge/app/Commands/../../.benchmark/pr-266 && php -dmax_execution_time=300 tempest data:parse --input-path=\"/Users/brentroose/Dev/100-million-row-challenge/app/Commands/../../data/real-data.csv\" --output-path=\"/Users/brentroose/Dev/100-million-row-challenge/app/Commands/../../data/real-data-actual.json\"",
      "mean": 2.4905864510799995,
      "stddev": 0.029308639510552802,
      "median": 2.4910914004799998,
      "user": 16.17893398,
      "system": 0.98564474,
      "min": 2.46529156748,
      "max": 2.53717727648,
      "times": [
        2.46576560948,
        2.46529156748,
        2.53717727648,
        2.49360640148,
        2.4910914004799998
      ],
      "memory_usage_byte": [
        74088448,
        74088448,
        74088448,
        74088448,
        74465280
      ],
      "exit_codes": [
        0,
        0,
        0,
        0,
        0
      ]
    }
  ]
}

random767435 · 2026-03-13T23:00:53Z

/bench

brendt · 2026-03-13T23:03:23Z

Benchmarking complete! Best execution time: 2.448s

Full results:

{
  "results": [
    {
      "command": "cd /Users/brentroose/Dev/100-million-row-challenge/app/Commands/../../.benchmark/pr-266 && php -dmax_execution_time=300 tempest data:parse --input-path=\"/Users/brentroose/Dev/100-million-row-challenge/app/Commands/../../data/real-data.csv\" --output-path=\"/Users/brentroose/Dev/100-million-row-challenge/app/Commands/../../data/real-data-actual.json\"",
      "mean": 2.460128228,
      "stddev": 0.008552351063892696,
      "median": 2.4615851286,
      "user": 16.24836798,
      "system": 0.9338687199999999,
      "min": 2.4487572946,
      "max": 2.4702872526,
      "times": [
        2.4702872526,
        2.4653869196,
        2.4546245445999997,
        2.4615851286,
        2.4487572946
      ],
      "memory_usage_byte": [
        73842688,
        74268672,
        74268672,
        74268672,
        74268672
      ],
      "exit_codes": [
        0,
        0,
        0,
        0,
        0
      ]
    }
  ]
}

random767435 · 2026-03-13T23:04:52Z

/bench

brendt · 2026-03-13T23:14:37Z

Benchmarking complete! Best execution time: 2.455s

Full results:

{
  "results": [
    {
      "command": "cd /Users/brentroose/Dev/100-million-row-challenge/app/Commands/../../.benchmark/pr-266 && php -dmax_execution_time=300 tempest data:parse --input-path=\"/Users/brentroose/Dev/100-million-row-challenge/app/Commands/../../data/real-data.csv\" --output-path=\"/Users/brentroose/Dev/100-million-row-challenge/app/Commands/../../data/real-data-actual.json\"",
      "mean": 2.4686511485600002,
      "stddev": 0.0137146818580164,
      "median": 2.46913488176,
      "user": 16.2275718,
      "system": 1.0150784,
      "min": 2.45503008976,
      "max": 2.48632004876,
      "times": [
        2.47743400676,
        2.48632004876,
        2.45503008976,
        2.45533671576,
        2.46913488176
      ],
      "memory_usage_byte": [
        74301440,
        74301440,
        74301440,
        74301440,
        74301440
      ],
      "exit_codes": [
        0,
        0,
        0,
        0,
        0
      ]
    }
  ]
}

random767435 · 2026-03-13T23:35:58Z

/bench

brendt · 2026-03-14T06:43:12Z

Benchmarking complete! Best execution time: 2.475s

Full results:

{
  "results": [
    {
      "command": "cd /Users/brentroose/Dev/100-million-row-challenge/app/Commands/../../.benchmark/pr-266 && php -dmax_execution_time=300 tempest data:parse --input-path=\"/Users/brentroose/Dev/100-million-row-challenge/app/Commands/../../data/real-data.csv\" --output-path=\"/Users/brentroose/Dev/100-million-row-challenge/app/Commands/../../data/real-data-actual.json\"",
      "mean": 2.48245010106,
      "stddev": 0.007956842258772086,
      "median": 2.47802340886,
      "user": 15.966409559999999,
      "system": 1.1614289799999997,
      "min": 2.47511111786,
      "max": 2.49136340986,
      "times": [
        2.49136340986,
        2.49081436786,
        2.47802340886,
        2.47511111786,
        2.47693820086
      ],
      "memory_usage_byte": [
        79544320,
        79544320,
        79544320,
        79544320,
        79544320
      ],
      "exit_codes": [
        0,
        0,
        0,
        0,
        0
      ]
    }
  ]
}

initial commit

254cf08

github-actions Bot added the bench_needed label Mar 12, 2026

brendt added the verified label Mar 12, 2026

brendt removed bench_needed verified labels Mar 12, 2026

let's see

723f236

github-actions Bot added the bench_needed label Mar 12, 2026

xHeaven added the verified label Mar 12, 2026

brendt removed bench_needed verified labels Mar 12, 2026

another try

db5f595

github-actions Bot added the bench_needed label Mar 12, 2026

another try

5c081b5

xHeaven added the verified label Mar 12, 2026

brendt removed bench_needed verified labels Mar 12, 2026

let's adjust

789f6be