add live stress harness, app-level admin login rate limit
tests/stress/live_accuracy.mjs: classroom-scale accuracy + latency test that targets the deployed server (single-session, sid=main). Logs in as admin via /admin/login, resets the session, joins N students serially over HTTP, opens N student WebSockets in batches of 8 (250ms apart) plus the instructor WS, then drives every question through the admin "next" command. Each student picks uniformly random A-D, sends the submit, waits for the submit_ack, and records the round-trip latency. After session_ended, the script verifies that every student whose pick == correct got score > 0, every other submission got score == 0, and reports p50/p95/p99 ack latency. First live run: 50 students, 100 submits, 100% acks, 100% accuracy match, p99 555ms (≈intercontinental RTT to HK). tests/stress/live_loop.sh: tmux-friendly loop that runs the live test every 60s and appends a JSONL summary line per cycle to runs/live_summary.jsonl. Mirrors the morning's api_stress run_loop shape so per-cycle aggregates are easy to scrape. app/rate_limit.py: tiny in-memory token bucket. Capacity + refill in tokens/minute, keyed by client IP via X-Forwarded-For (with a fallback to request.client.host). Process-local state — admin login is the only user. POST /admin/login: rate-limited at 10 attempts/minute/IP. Generous for the legit instructor (who succeeds in 1-2 tries) and prohibitive for brute force from a single attacker IP. Student endpoints deliberately NOT rate-limited because campus students share NAT gateways and IP-level limits would false-positive a whole class. The bucket is per-app-instance (instantiated inside the router factory), so test apps each get a fresh one and tests don't poison each other.
This commit is contained in:
72
tests/stress/live_loop.sh
Normal file
72
tests/stress/live_loop.sh
Normal file
@@ -0,0 +1,72 @@
|
||||
#!/usr/bin/env bash
|
||||
# Long-running live accuracy + latency loop.
|
||||
# Each cycle resets the live session, runs the full live_accuracy.mjs
|
||||
# test, parses the summary, and appends a JSON line to runs/live_summary.jsonl.
|
||||
#
|
||||
# Run:
|
||||
# ADMIN_PW=$(cat /tmp/quiz-admin-pw.txt) tmux new -d -s quiz_live \
|
||||
# 'cd /home/ameer/RD/Projects/Apps/quiz/tests/stress && \
|
||||
# ADMIN_PW="$ADMIN_PW" bash live_loop.sh'
|
||||
# Stop:
|
||||
# tmux send -t quiz_live C-c # graceful, then tmux kill-session -t quiz_live
|
||||
#
|
||||
# Tunables:
|
||||
# BASE_URL - default https://quiz.ahkhan.me
|
||||
# N - default 50 students
|
||||
# GAP_S - seconds between cycles (default 60)
|
||||
# ADMIN_PW - required, the live admin password
|
||||
|
||||
set -uo pipefail
|
||||
cd "$(dirname "$0")"
|
||||
|
||||
BASE_URL="${BASE_URL:-https://quiz.ahkhan.me}"
|
||||
N="${N:-50}"
|
||||
GAP_S="${GAP_S:-60}"
|
||||
|
||||
if [ -z "${ADMIN_PW:-}" ]; then
|
||||
echo "ADMIN_PW must be set in env" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
mkdir -p runs
|
||||
SUM="runs/live_summary.jsonl"
|
||||
LOG="runs/live-$(date -u +%Y%m%dT%H%M%SZ).log"
|
||||
|
||||
echo "{\"event\":\"loop_start\",\"ts\":\"$(date -u +%FT%TZ)\",\"target\":\"$BASE_URL\",\"N\":$N,\"gap_s\":$GAP_S,\"log\":\"$LOG\"}" | tee -a "$SUM"
|
||||
|
||||
cycle=0
|
||||
total_pass=0
|
||||
total_fail=0
|
||||
total_acks=0
|
||||
total_submits=0
|
||||
|
||||
trap 'echo "{\"event\":\"loop_stop\",\"ts\":\"$(date -u +%FT%TZ)\",\"cycles\":'$cycle'}" | tee -a "$SUM"; exit 0' INT TERM
|
||||
|
||||
while true; do
|
||||
cycle=$((cycle + 1))
|
||||
ts=$(date -u +%FT%TZ)
|
||||
printf '\n----- live cycle %d (%s) -----\n' "$cycle" "$ts" | tee -a "$LOG"
|
||||
|
||||
out=$(timeout 180 node live_accuracy.mjs "$BASE_URL" "$ADMIN_PW" "$N" 2>&1)
|
||||
ec=$?
|
||||
echo "$out" | tee -a "$LOG" >/dev/null
|
||||
|
||||
pass=$(echo "$out" | sed -n 's/.*Accuracy ok *: \([0-9]*\) \/ \([0-9]*\).*/\1/p')
|
||||
total=$(echo "$out" | sed -n 's/.*Accuracy ok *: \([0-9]*\) \/ \([0-9]*\).*/\2/p')
|
||||
fail=$(echo "$out" | sed -n 's/.*Accuracy fail *: \([0-9]*\)/\1/p')
|
||||
acks=$(echo "$out" | sed -n 's/.*Acks received *: \([0-9]*\) \/.*/\1/p')
|
||||
p50=$(echo "$out" | sed -n 's/.*p50=\([0-9.]*\) .*/\1/p' | tail -1)
|
||||
p95=$(echo "$out" | sed -n 's/.*p95=\([0-9.]*\) .*/\1/p' | tail -1)
|
||||
p99=$(echo "$out" | sed -n 's/.*p99=\([0-9.]*\) .*/\1/p' | tail -1)
|
||||
max=$(echo "$out" | sed -n 's/.*max=\([0-9.]*\)$/\1/p' | tail -1)
|
||||
mean=$(echo "$out" | sed -n 's/.*mean=\([0-9.]*\) .*/\1/p' | tail -1)
|
||||
|
||||
total_pass=$((total_pass + ${pass:-0}))
|
||||
total_fail=$((total_fail + ${fail:-0}))
|
||||
total_acks=$((total_acks + ${acks:-0}))
|
||||
total_submits=$((total_submits + ${total:-0}))
|
||||
|
||||
echo "{\"event\":\"cycle\",\"ts\":\"$ts\",\"cycle\":$cycle,\"exit\":$ec,\"submits\":${total:-0},\"acc_ok\":${pass:-0},\"acc_fail\":${fail:-0},\"acks\":${acks:-0},\"mean_ms\":${mean:-0},\"p50\":${p50:-0},\"p95\":${p95:-0},\"p99\":${p99:-0},\"max\":${max:-0},\"running_pass\":$total_pass,\"running_fail\":$total_fail}" | tee -a "$SUM"
|
||||
|
||||
sleep "$GAP_S"
|
||||
done
|
||||
Reference in New Issue
Block a user