How would you process 100 million rows from a CSV file in the shortest time possible?

I recently found out, through LinkedIn, about an interesting challenge for anyone who enjoys exploring performance, efficiency, and code optimizations.

It's the 100-million-row challenge in PHP, a challenge inspired by similar initiatives in other languages, but adapted for the PHP ecosystem.

🔗 The official challenge repository is available at: tempestphp/100-million-row-challenge.

I decided to take part to explore the problem, study different efficiency approaches, and also contribute to the community.

If you like this kind of problem, consider this an invitation to join in and test your own ideas too. At some point in their career, every developer ends up needing to read, process, and transform large volumes of data, and in those scenarios every implementation detail can make a difference in the final performance.

On top of that, it's a challenge that mixes learning, competition, and fun.

The challenge

💡 The challenge was inspired by the Java community's 1 Billion Row Challenge, but adapted for the PHP ecosystem.

The idea is simple: read a file with 100 million rows, process that information, and generate a final JSON with the aggregated data.

Each row in the file contains a full URL and a timestamp.

The solution must extract the URL path, group the hits by day, and produce a JSON in the following format:

{
  "/blog/11-million-rows-in-seconds": {
    "2025-01-24": 1,
    "2026-01-24": 2
  },
  "/blog/php-enums": {
    "2024-01-24": 1
  }
}

In other words, each page holds a map with the number of hits per date.

Important rules

The challenge has some interesting restrictions:

• The solution must be written in PHP only
• External tools are not allowed
• The script must run entirely within the project
• FFI is not allowed
• JIT is disabled by default

These restrictions make the challenge more interesting, since they force you to focus on:

• file-reading efficiency
• memory usage
• efficient string handling
• data structures

Getting started

To take part, just fork the repository and run the local environment.

# Install dependencies
composer install

# Generate a development dataset (1_000_000 is the default)
php tempest data:generate

# Generate a larger dataset
php tempest data:generate 50_000_000

The real benchmark uses 100 million rows.

After that, just implement your solution in the class:

final class Parser
{
    public function parse(string $inputPath, string $outputPath): void
    {
        throw new Exception('TODO');
    }
}

While developing, you can run the parser with:

php tempest data:parse

And validate the result with:

php tempest data:validate

If validation passes, your implementation is correct.

Running the challenge with Docker (optional)

To standardize the environment and make it easier to run the challenge, I used a simple container with PHP 8.5.

Dockerfile

FROM php:8.5-cli-alpine

WORKDIR /app

RUN apk add --no-cache \
        git \
        icu-dev \
        $PHPIZE_DEPS \
    && docker-php-ext-install intl \
    && docker-php-ext-install pcntl \
    && apk del --no-network $PHPIZE_DEPS \
    && git config --global --add safe.directory /app \
    && { \
        echo "opcache.enable_cli=1"; \
        echo "opcache.jit=0"; \
        echo "opcache.jit_buffer_size=0"; \
    } > /usr/local/etc/php/conf.d/99-disable-jit.ini

COPY --from=composer:latest /usr/bin/composer /usr/bin/composer

ENV COMPOSER_ALLOW_SUPERUSER=1

docker-compose.yml

services:
  app:
    build: .
    working_dir: /app
    volumes:
      - ./:/app
    tty: true

Installing dependencies and running commands:

docker compose run --rm app composer install
docker compose run --rm app php tempest data:generate
docker compose run --rm app php tempest data:parse
docker compose run --rm app php tempest data:validate

Submitting your solution

Submission is done through a Pull Request on the repository. The PR title must be your GitHub username.

After manual validation, the organizers run the benchmark on an official server and record the time on the leaderboard.

You can also request a new run by commenting /bench on the PR.

Categories and ranking

The challenge has two main categories:

• multi-process, where the solution can use multiple processes (for example with pcntl)
• single-thread, where the solution must run using only a single process

The single-thread category is particularly interesting, since it lets you compare algorithm efficiency without relying on multiple processes.

During the challenge, I also contributed by building a ranking site to make it easier to visualize the results and the submitted solutions: PHP 100M Row Challenge Explorer.

The idea was to make it easier to follow the evolution of submissions and explore the community's different implementations.

Later on, that ranking ended up being referenced directly in the challenge's official repository.

Prizes

Winners receive prizes sponsored by JetBrains and Tideways, including PhpStorm and Tideways ElePHPants, JetBrains All Products licenses, and access to Tideways Team.

The challenge is open until March 15, 2026.

Conclusion

This kind of challenge is interesting because it involves processing a considerable amount of data. Situations like this demand attention to several technical aspects, such as I/O efficiency, memory usage, and strategies for handling large-scale processing.

I decided to take part mainly as a technical exercise and also to explore different approaches to the problem.

During the process, I actually ended up using the pcntl extension in practice, something I already knew about but hadn't yet applied for real in a project. It was a great opportunity to experiment with that kind of approach and realize how much using multiple processes can widen the range of possible solutions.

In the end, it's a challenge that demands a lot of attention to detail and a solid knowledge of the language, especially because every millisecond can make a difference in the final result.

Thanks for reading this far. See you in the next one!