Companion to the 6-month roadmap: an editable financial model (every assumption below is a live input) and the Phase-0 test kit — grading prompts, golden-set template, eval protocol, and a runnable API script — so we can validate quality and cost before writing any product code.
Change anything on the left — unit costs, monthly bill, and the budget below recompute instantly. Defaults: Claude Haiku 4.5 ($1 / $5 per MTok, Aug 2026), Batch API −50%, cached rubric ≈ 0.1× input price, self-hosted Whisper for speech-to-text. Your edits are saved in this browser.
People are ~85% of the program; the API is a rounding error. AI spend ramps with rollout (2% → 10% → 25% → 45% → 70% → 100% of the steady-state bill at your current inputs); the GPU server starts in month 3 with Speaking.
| Month | People | Infra | AI API | Total $ | ₫M |
|---|
Everything needed to run the validation this week: copy the prompt, grade the sample essay, compare against a teacher, repeat ×100. Nothing here requires product engineering.
The system prompt below is the cached prefix — rubric, rules and band anchors stay byte-identical across calls so you pay ~0.1× for them. Only the student's essay travels per request. Fill the three {{placeholders}}, paste 2–3 teacher-graded sample essays where marked, and keep the JSON contract untouched (the app UI will render from it).
You are FLYER's English writing examiner for young learners in Vietnam.
Grade ONE student submission against the rubric below. Accurate, consistent, kind.
TARGET EXAM: {{LEVEL}} // "Cambridge A2 Flyers" | "KET" | "IELTS Writing Task 2"
TASK PROMPT: {{TASK_PROMPT}}
STUDENT AGE BAND: {{AGE_BAND}} // e.g. "8–11"
RUBRIC — score each criterion 0–5:
1. Content — answers the task; ideas developed for the level
2. Organisation — ordering, paragraphing, linking words
3. Language — grammar + vocabulary range and accuracy for the level
4. Mechanics — spelling, punctuation, capitalisation
BAND ANCHORS (keep byte-identical across calls so they cache):
<<PASTE 2–3 TEACHER-GRADED SAMPLE ESSAYS PER BAND HERE>>
RULES:
- Quote errors EXACTLY as written. Never invent an error.
- Max 8 errors, ranked by learning value; merge repeats of the same mistake.
- praise_vi: warm, specific, in Vietnamese, 2 sentences max — praise BEFORE correction.
- Corrections in English; each "why_vi" is ONE short Vietnamese sentence.
- Off-task, empty, not English, or unsafe content → set "flag", leave scores null.
- Output ONLY the JSON object. No markdown, no commentary.
OUTPUT JSON:
{
"flag": null | "off_task" | "empty" | "not_english" | "safety",
"scores": {"content":0-5, "organisation":0-5, "language":0-5, "mechanics":0-5},
"overall_band": <number on the rubric scale>,
"praise_vi": "…",
"errors": [{"quote":"…","type":"grammar|vocab|spelling|punctuation|linking","fix":"…","why_vi":"…"}],
"rewrite_examples": ["<1–2 improved sentences>"],
"next_steps_vi": ["<2 concrete practice tips>"]
}User message per call — nothing else:
STUDENT SUBMISSION:
{{ESSAY}}Sample essay to test with (Flyers level, seeded with typical errors — a good grader should find the tense errors, "more better", and the missing linking words, and still praise the ideas):
My favourite holiday Last summer I go to Da Nang with my family. We stay in a hotel near the beach. Every morning me and my brother swimming in the sea, it was very fun. The food is more better than home, I eat seafood everyday. One day we visit Ba Na Hills, the bridge with two big hand is very beautiful. I take many photo there. I want come back next year because I love this place very much.
Speaking reuses the same shape. Whisper produces the transcript with word timestamps; fluency numbers are computed in code, free (words/min, pauses ≥ 1.5s, filler count) and passed in as data — the LLM interprets, it doesn't measure. Word-level pronunciation flags come from comparing the transcript against the target text (read-aloud tasks) — phoneme scoring is a paid v2 add-on, not needed for launch.
You are FLYER's English speaking examiner for young learners in Vietnam.
Grade ONE recorded answer using the transcript and measured metrics below.
TARGET EXAM: {{LEVEL}} // "Movers Speaking" | "IELTS Speaking Part 2" …
TASK PROMPT: {{TASK_PROMPT}}
RUBRIC — score each criterion 0–5:
1. Pronunciation — intelligibility; words clearly misread/omitted (word level)
2. Fluency — use ONLY the measured metrics; do not re-estimate from text
3. Vocabulary & Grammar — range and accuracy for the level
4. Content — answers the task; ideas developed for the level
BAND ANCHORS: <<PASTE 2–3 GRADED SAMPLE TRANSCRIPTS PER BAND>>
RULES:
- Every error must carry the [start,end] seconds of where it was said,
copied from the transcript timestamps (the app plays that clip back).
- ASR text may itself contain recognition mistakes: if a phrase is implausible
for a child but grammatical, treat it as unclear pronunciation, not grammar.
- praise_vi first, Vietnamese, specific. Output ONLY the JSON.
OUTPUT JSON:
{
"flag": null | "empty" | "not_english" | "safety",
"scores": {"pronunciation":0-5,"fluency":0-5,"vocab_grammar":0-5,"content":0-5},
"overall_band": <number>,
"praise_vi": "…",
"errors": [{"quote":"…","span_sec":[12.4,14.1],"type":"pronunciation|grammar|vocab|fluency","fix":"…","why_vi":"…"}],
"model_answer": "<3–4 sentence answer at target band, child-appropriate>",
"next_steps_vi": ["…","…"]
}User message — transcript with timestamps + metrics JSON:
TRANSCRIPT (word → seconds):
{{TRANSCRIPT_WITH_TIMESTAMPS}}
MEASURED METRICS (computed, trust these):
{"duration_sec": 118, "words_per_min": 74, "pauses_over_1500ms": 6,
"longest_pause_sec": 4.2, "filler_count": 9, "self_corrections": 3}Target: 300 writing + 100 speaking submissions, each graded blind by two teachers. Spread ~20% per band so the AI is tested where it's hardest (middle bands), across Flyers / KET / IELTS. Where the two teachers disagree by more than 1 band, a third adjudicates. This CSV is the whole contract — one row per submission:
id,skill,level,task_id,task_prompt,submission_ref,grader1,g1_content,g1_org,g1_lang,g1_mech,g1_overall,grader2,g2_content,g2_org,g2_lang,g2_mech,g2_overall,adjudicated_overall,notes W-0001,writing,flyers,FT-12,"My favourite holiday",essays/W-0001.txt,Ms.Trang,4,3,3,4,3.5,Mr.Nam,4,3,4,4,3.5,3.5, W-0002,writing,ielts_t2,IE-03,"Some people think…",essays/W-0002.txt,Ms.Trang,5,6,5,6,5.5,Mr.Nam,6,6,5,6,6.0,5.5,"graders split on TR" S-0001,speaking,movers,MS-07,"Describe this picture",audio/S-0001.mp3,Ms.Trang,4,4,3,4,4.0,Mr.Nam,4,3,3,4,3.5,4.0,
Run the prompt (fixed version, logged) over the golden set, join AI scores against adjudicated teacher scores, compute four numbers. Re-run the same script monthly on a 1% random sample of live gradings to catch drift.
If a gate fails: fix in this order — add/replace band-anchor examples (cheapest), tighten rubric wording, escalate model tier for the failing level only. Re-run the full set after every prompt change; never eyeball a few examples and call it fixed.
Minimal harness: cached rubric prefix, one call per essay, JSON out. Grade the golden set with claude-haiku-4-5 (the economics above), and re-grade disagreements with claude-opus-5 to see what the calibration tier buys. On the second call, check usage.cache_read_input_tokens > 0 — if it's 0, the prefix isn't byte-stable and you're paying full price. For >100 essays switch to the Batch API (−50%, results within an hour).
# pip install anthropic · export ANTHROPIC_API_KEY=…
import anthropic, json, pathlib
MODEL_GRADE = "claude-haiku-4-5" # mass tier — the unit costs in this page
MODEL_CALIB = "claude-opus-5" # calibration / appeals tier
RUBRIC_PROMPT = pathlib.Path("writing_rubric_prompt.txt").read_text() # = T1, byte-stable
client = anthropic.Anthropic()
def grade(essay: str, model: str = MODEL_GRADE) -> dict:
msg = client.messages.create(
model=model,
max_tokens=2000,
system=[{
"type": "text",
"text": RUBRIC_PROMPT,
"cache_control": {"type": "ephemeral"}, # rubric+anchors cached ≈0.1×
}],
messages=[{"role": "user", "content": f"STUDENT SUBMISSION:\n{essay}"}],
)
if msg.stop_reason == "refusal": # safety decline → route to teacher
return {"flag": "safety", "scores": None}
raw = msg.content[-1].text
out = json.loads(raw[raw.index("{"): raw.rindex("}") + 1])
out["_usage"] = {"in": msg.usage.input_tokens,
"cached": msg.usage.cache_read_input_tokens,
"out": msg.usage.output_tokens}
return out
if __name__ == "__main__":
essay = pathlib.Path("sample_essay.txt").read_text()
print(json.dumps(grade(essay), ensure_ascii=False, indent=2))
print(json.dumps(grade(essay)["_usage"], indent=2)) # cached > 0 on this 2nd callProduction hardening (later, not for the pilot): validate against a JSON schema and retry once on mismatch; log prompt_version, model, tokens, cost per grading; content-hash dedupe so a resubmitted identical essay returns the stored result.
Set these up in week 1 — they're what makes “free for every learner” safe to promise.