Performance Testing Fundamentals: Test Types, KPIs & NFRs

Hands-on Lab: Measure percentiles, throughput and Little's law with real traffic

1. Install Python 3.8+ (python3 --version). No libraries are needed.
2. Break one request into phases with curl (Git Bash on Windows works too). Run it 3 times and note how DNS/TLS drop after the first call because of caching:

   curl -o /dev/null -s -w "dns=%{time_namelookup}s connect=%{time_connect}s tls=%{time_appconnect}s ttfb=%{time_starttransfer}s total=%{time_total}s code=%{http_code}\n" https://blazedemo.com/

   Expected: something like dns=0.01s connect=0.2s tls=0.45s ttfb=0.7s total=0.72s code=200. TTFB minus TLS is roughly server processing time.
3. See why averages lie. Open a Python shell and run:

   >>> import statistics, math
   >>> times = [200] * 95 + [5000] * 5        # 100 requests, 5 very slow ones
   >>> statistics.mean(times)
   440
   >>> s = sorted(times)
   >>> s[math.ceil(0.95 * 100) - 1], s[math.ceil(0.99 * 100) - 1]
   (200, 5000)

   The average (440 ms) describes nobody: 95 users saw 200 ms and 5 users waited 5 s. p99 exposes them.
4. Save the following as perf_basics.py and run python3 perf_basics.py:

   # perf_basics.py  -- run: python3 perf_basics.py   (Python 3.8+, no extra packages)
   import math
   import statistics
   import time
   import urllib.request
   from concurrent.futures import ThreadPoolExecutor
   
   URL = "https://blazedemo.com/"
   REQUESTS = 30      # keep it small: this is a shared public demo site
   WORKERS = 5        # 5 concurrent "users" with zero think time
   
   
   def hit(_):
       start = time.perf_counter()
       try:
           with urllib.request.urlopen(URL, timeout=15) as resp:
               resp.read()
               ok = resp.status == 200
       except Exception:
           ok = False
       return (time.perf_counter() - start) * 1000, ok
   
   
   def pct(values, p):
       """Nearest-rank percentile (same idea JMeter/k6 reports use)."""
       s = sorted(values)
       k = max(0, math.ceil(p / 100 * len(s)) - 1)
       return s[k]
   
   
   if __name__ == "__main__":
       t0 = time.perf_counter()
       with ThreadPoolExecutor(max_workers=WORKERS) as pool:
           results = list(pool.map(hit, range(REQUESTS)))
       wall = time.perf_counter() - t0
   
       times = [r[0] for r in results]
       errors = sum(1 for r in results if not r[1])
       avg_s = statistics.mean(times) / 1000
       throughput = len(times) / wall
   
       print(f"samples={len(times)} errors={errors} ({errors / len(times):.1%})")
       print(f"avg={statistics.mean(times):.0f} ms  median={pct(times, 50):.0f} ms")
       print(f"p90={pct(times, 90):.0f} ms  p95={pct(times, 95):.0f} ms  "
             f"p99={pct(times, 99):.0f} ms  max={max(times):.0f} ms")
       print(f"throughput={throughput:.2f} req/s over {wall:.1f} s")
       # Little's law: N = X * R  ->  predicted concurrency should be close to WORKERS
       print(f"Little's law check: X * R = {throughput * avg_s:.2f} (workers = {WORKERS})")

5. Record the output in a table: avg, median, p90, p95, p99, max, error %, throughput. Expected order: median ≤ p90 ≤ p95 ≤ p99 ≤ max, and the Little's law line should print a value close to 5 (the number of workers), because with no think time N = X × R.
6. Change WORKERS to 1 and then 10 (keep REQUESTS at 30). Observe: throughput rises with workers while the server is not saturated; if response time also rises sharply, you are seeing queueing.
7. Using Little's law, calculate the VUs needed for 40 TPS on blazedemo if R = 0.8 s and Z = 4 s (answer: 40 × 4.8 = 192 VUs). Write the calculation in your notes.
8. Write three NFRs for blazedemo (Home, Find Flights, Purchase) in the format shown above. You will reuse them in the capstone.