Load testing

A ticket shop that could hold 60 buyers

Ticket sales come in bursts. When an event goes on sale, everyone arrives in the same minutes. This shop was fast in normal use and slowed to a crawl at about 60 simultaneous buyers. The goal was 300.

Before and after

Simultaneous buyers
about 400 50 to 70 before
Average page load
about 1.5 s 6 to 10 s before, beyond about 60 users
Slowest database queries
gone 10 to 16 s before
Changes that mattered
2 one index, one library removed

What was found and changed

In order. Every change was followed by a new load test, so each one could be judged by its own result.

  1. Goal

    300 people in the purchase funnel at the same time. That is what the shop needs when a popular event goes on sale.

  2. Baseline

    Failed: About 60 of 300 users handled

    Beyond 50 to 70 simultaneous users the ordering pages slowed down sharply.

    • Target300 users
    • Users handled50 to 70
    • Page loads beyond that6 to 10 s
  3. First guess

    A limit that looked like the cause. The database allowed 200 connections, but in the tests the number never rose above 50. The web server's worker limit (PHP-FPM pm.max_children) was 50, so it was raised to 200.

    The next test showed no change. The limit was not what held the shop back.

  4. Logs

    The slow query log showed the real cause. Under load, queries against the ticket table took 10 to 16 seconds. They held their connections open while they ran, and every page behind them waited.

  5. Change 1

    One database index. An index on the session column of the ticket table removed the slow queries.

    ALTER TABLE ticket
    ADD INDEX idx_ticket_session (session_id),
    ALGORITHM=INPLACE, LOCK=NONE;
    
    ANALYZE TABLE ticket;
  6. Change 2

    An icon library removed. On this site the Font Awesome icon library, under load, made pages load much more slowly and sometimes stopped them loading at all. Only two of its icons were in use, so they were replaced with the shop's own and the library was removed. After that, load times stayed steady at high user counts.

    This problem was in the browser. A request-level test of the JMeter type never loads the library, so it would not have found it.

  7. Final run

    A Wednesday morning, 6:30 to 7:00

    Passed: About 400 users handled

    The target of 300 was passed with moderate server load.

    • Users in the purchase funnelabout 400
    • Average page loadabout 1.5 s
    • Database connections in use5 to 10

The worker limit was left at 200. It matches the database's own limit and does no harm, but it was not what fixed the shop.

Why this needed a load test

  • The limit was invisible. With a few visitors every page was fast. The owner had no way to know the ceiling was 60 buyers until a sale hit it, or a test did.
  • The causes were small. One missing index and one icon library. Neither looks like a risk in a code review.
  • More hardware would not have fixed it. The server had capacity to spare. The slow queries were the bottleneck.
  • The obvious fix was wrong. Raising the worker limit looked right and changed nothing. Only measuring after the change showed that.
  • One problem was in the browser. The icon library failed on the page itself, which a request-level test never loads.

What the server needed

The server's use of resources was followed during the tests. The web application used very little, and the database used most of it.

These are this shop's figures, not a rule for every site.

Server resources used in the load tests
MeasuredResult
300 simultaneous usersabout 3 CPU cores and 10 GB of memory
Database, as a rule of thumbabout 1 CPU core per 100 users
About 400 users after the index5 to 10 database connections at a time

How the work was done

The load tests ran real browsers through the shop's purchase funnel. An AI agent was given the server's log files and the data from the load tests, and it proposed the fixes. People ran the tests, gave the agent its material and applied the changes.

It was an early trial of what is now Supervisor Autotune. Today the agent starts the test and gets the results itself, and you approve each change.

The customer is not named.

How an agent works with SupervisorAbout load testing

Find out how many buyers your shop can hold

Start with your agent

Three steps, and you get 5 free credits when you register, enough for a load test with 50 users. Don't want to use an agent? Use the web app instead.