{"prompt":"In your own words, write what YOUR dossier would have caught about the real Uber ATG Tempe 2018 crash: which of your own rows, which real NTSB figure, and the mechanism that links them.","analysis":"My night-time pedestrian miss row (night recall = 0.39, 61% missed) maps directly to the Tempe crash: the NTSB found the system detected the pedestrian at ~5.6 seconds but never resolved her as a \"vehicle\" or \"bicycle\" — it kept cycling classifications, then suppressed automatic braking because it lacked confidence, exactly the classification instability my rare-class cyclist miss row (0.66 recall) flags for underrepresented road users. The mechanism linking them is low-light classification failure — my SOTIF chain identifies that night removes contrast, causing the classifier to oscillate or reject vulnerable road users entirely, precisely what happened when Tempe's system saw a pedestrian but refused to classify her as a person, leaving the car never braking at 39 mph.","signalsDetected":["row","figure","mechanism"],"passed":true,"attempt":1}

{"prompt":"Match each driving condition to its primary-strength sensor and the weak-link sensor that fails there","map":[{"condition":"Pitch-dark rural road","primaryStrength":"Lidar","weakLink":"Camera","correct":true},{"condition":"Dense fog on the motorway","primaryStrength":"Radar","weakLink":"Camera","correct":true},{"condition":"Child behind a parked van","primaryStrength":"Lidar","weakLink":"Occlusion","correct":true},{"condition":"Clear sunny highway","primaryStrength":"Camera","weakLink":"Radar","correct":true}],"passed":true,"attempt":2}

{"checkpointId":"ttcs_iou_classify_run","code":"# Run IoU on this pair, read the number, then classify it (Section 1).\n# Boxes are [x1, y1, x2, y2] in pixels. The object is REAL, there is a ground-truth box.\n\ngt   = [60, 50, 200, 150]   # ground truth: a real object is here\npred = [150, 96, 280, 196]   # the detector's box for this frame\n\ndef iou(a, b):\n    ix1, iy1 = max(a[0], b[0]), max(a[1], b[1])\n    ix2, iy2 = min(a[2], b[2]), min(a[3], b[3])\n    inter = max(0, ix2 - ix1) * max(0, iy2 - iy1)\n    union = (a[2]-a[0])*(a[3]-a[1]) + (b[2]-b[0])*(b[3]-b[1]) - inter\n    return inter / union\n\nscore = iou(gt, pred)\nprint(f\"IoU = {score:.3f}\")\nprint(f\"threshold = 0.5  ->  {'matches' if score >= 0.5 else 'does NOT match'} a ground-truth box\")\nprint(\"So: a real object, no matching box. Your call on the next panel.\")","stdout":"IoU = 0.111\nthreshold = 0.5  ->  does NOT match a ground-truth box\nSo: a real object, no matching box. Your call on the next panel.\n","stderr":"","error":null,"imageCount":0,"durationMs":1,"ranAt":"2026-07-03T22:16:02.167Z"}

{"prompt":"Confirm the IoU threshold at 0.5, compute IoU for the borderline pair, and classify it as hit / miss / phantom; define a false negative as an unmatched real object","iouThreshold":0.5,"pairIoU":0.111,"classification":"hit","falseNegativeDefinition":"a real (ground-truth) object with no predicted box matched at IoU >= 0.5","cellRun":true,"passed":false,"attempt":2}

{"checkpointId":"ttcs_scored_metrics","code":"# Section 2: score the Aurora-7 detector on its detections table.\n# SYNTHETIC, illustrative detections-vs-ground-truth (no raw imagery, no model runs here;\n# proportions grounded in the published KITTI/nuScenes benchmarks).\n# Each row = one scored object, tagged with class + condition, with an outcome:\n#   \"TP\" caught,  \"FN\" a real object MISSED (false negative),  \"FP\" a phantom box.\nimport pandas as pd\n\n# Read the SAME scored detections table Act 3 slices by condition, so Section 2 and\n# Section 3 can never disagree. (Normalise two column names so the scoring below reads\n# cleanly: class -> obj_class, flag -> outcome.)\ndf = pd.read_csv(\"aurora7_detections_kitti.csv\").rename(\n    columns={\"class\": \"obj_class\", \"flag\": \"outcome\"})\nprint(f\"scored objects in table: {len(df)}\")\n\ndef score(d, label):\n    TP = (d.outcome == \"TP\").sum()\n    FP = (d.outcome == \"FP\").sum()\n    FN = (d.outcome == \"FN\").sum()\n    precision = TP / (TP + FP)\n    recall    = TP / (TP + FN)\n    print(f\"{label:18s} TP={TP:4d} FP={FP:3d} FN={FN:3d}  \"\n          f\"precision={precision:.2f}  recall={recall:.2f}\")\n    return recall\n\n# 1) The whole table, the headline (aggregate) numbers.\nagg_recall = score(df, \"ALL (aggregate)\")\n\n# 2) Vulnerable road users only = pedestrians + cyclists. THIS is the safety subset.\nvru = df[df.obj_class.isin([\"Pedestrian\", \"Cyclist\"])]\nvru_recall = score(vru, \"VRU (Ped+Cyc)\")\n\n# 3) The safety number: VRU false-negative RATE = share of real VRUs MISSED.\nvru_fn_rate = 1 - vru_recall\nprint(\"-\" * 60)\nprint(f\"AGGREGATE recall (all classes) : {agg_recall:.2f}   <- looks reassuring\")\nprint(f\"VRU false-negative rate        : {vru_fn_rate:.2f}   <- the safety number\")","stdout":"scored objects in table: 991\nALL (aggregate)    TP= 822 FP= 41 FN=128  precision=0.95  recall=0.87\nVRU (Ped+Cyc)      TP= 392 FP= 35 FN=122  precision=0.92  recall=0.76\n------------------------------------------------------------\nAGGREGATE recall (all classes) : 0.87   <- looks reassuring\nVRU false-negative rate        : 0.24   <- the safety number\n","stderr":"","error":null,"imageCount":0,"durationMs":73,"ranAt":"2026-07-03T22:18:07.076Z"}

{"prompt":"Score the detector: read off precision, aggregate recall, and the VRU false-negative rate (the safety number)","aggregatePrecision":0.95,"aggregateRecall":0.87,"vruRecall":0.76,"vruFalseNegativeRate":0.236,"safetyNumber":"VRU false-negative rate","readOffBand":"20–30%","readOffCorrect":true,"scoringCellRan":true,"passed":true,"attempt":1}

{"prompt":"For each top failure mode write one ISO 21448 SOTIF chain (triggering condition -> functional insufficiency -> hazardous behaviour -> concrete mitigation); a vague mitigation or a SOTIF gap framed as a fault fails","rows":[{"failureMode":"Occluded child at night","triggeringCondition":" Child steps from behind a parked van after dark in a residential area, fully occluded until the last moment.","functionalInsufficiency":"Night + occlusion recall = 0.27 — 73% of these children are missed. Sensors work but are insufficient.","hazardousBehaviour":"No brake, no alert — the car strikes the child at speed.","mitigation":" Daylight-only ODD cap; shadow tracking; two-sensor confirmation; near-field ultrasonics at <30 km/h.\n\n","mitigationIsConcrete":true,"framedAsFault":false},{"failureMode":"Rare-class cyclist miss","triggeringCondition":"Cyclist on road at dawn/dusk/night in mixed traffic; underrepresented in training data.","functionalInsufficiency":"Cyclist recall = 0.66 even in daylight — 34% missed; worse at night.\n\n","hazardousBehaviour":"No reaction — car treats cyclist as clutter and collides or dangerously passes.","mitigation":">10k cyclist training examples; radar micro-Doppler pedal-cadence detection; temporal tracking; daylight + <30 km/h ODD; OTA cyclist model in 60 days.","mitigationIsConcrete":true,"framedAsFault":false}],"mitigations":[" Daylight-only ODD cap; shadow tracking; two-sensor confirmation; near-field ultrasonics at <30 km/h.\n\n",">10k cyclist training examples; radar micro-Doppler pedal-cadence detection; temporal tracking; daylight + <30 km/h ODD; OTA cyclist model in 60 days."],"passed":true,"attempt":1}

{"prompt":"Write the GO / GO-WITH-CONDITIONS / NO-GO ship verdict, each of five fields citing its own dossier section number, internally consistent (no unconditional GO over an unmitigated night miss; no NO-GO if a daylight-restricted release is viable)","verdict":"CONDITIONS","fields":{"definition":"IoU ≥ 0.5 — A detection is a true positive when the predicted box overlaps the ground-truth box by at least 50%. Any real VRU with no match above this threshold is a false negative (missed person).\n\n","metrics":"VRU recall 0.76, FN-rate 24% — The system misses 1 in 4 pedestrians and cyclists. Aggregate recall 0.87 hides this; the safety number is 24% missed VRUs.","worst":"Night + occluded = 0.27 — 73% of occluded children at night are missed. This is the most dangerous failure mode in the dataset.","mitigation":"Daylight-only ODD cap — Disable system at night via time-of-day geofence. Also: shadow tracking, two-sensor confirmation, near-field ultrasonics at <30 km/h, and OTA cyclist model within 60 days.","ask":"Approve daylight-only operation with these limits: sunrise-to-sunset only, ≤30 km/h urban, ≤80 km/h highway, cyclist OTA within 60 days. No night-time operation. No unrestricted release."},"citedSections":["Detection Definition","Metrics","Worst Failure Mode","Mitigation","The Ask"],"internallyConsistent":true,"passed":true,"attempt":2}

{"prompt":"Teach it back in plain words: why is recall on vulnerable road users, not accuracy, the metric that governs safety for a pedestrian detector? Anchor it to a real number you have seen.","teachback":"The asymmetry is this: false positives and false negatives have completely different weights in safety, but accuracy treats them as equal errors. A phantom brake (false positive) startles the driver and may cause a rear-end collision—but a missed pedestrian (false negative) means the car never brakes and the person dies. Those two errors are not the same.\n\nAccuracy hides this because it counts every correct \"empty road\" frame as a win—even if the system sees no one. On a typical drive, 99% of frames have no pedestrian, so a detector that never sees anyone is 99% accurate. That's what accuracy hides: it rewards ignoring people.\n\nRecall flips the question: \"Of all real pedestrians, how many did we actually catch?\" It doesn't care about empty frames—it only cares about the people the system was supposed to see. My night recall is 0.39—61% of real pedestrians are invisible to the system at night. That is the safety number.","signalsDetected":["mechanism","number"],"passed":true,"attempt":2}

{"prompt":"Slice VRU recall per condition, then triage the failure modes by (condition likelihood x miss impact); do not bury a rare-but-catastrophic VRU miss in green","aggregateVruRecall":0.76,"perConditionRecall":[{"condition":"Daylight","recall":0.85},{"condition":"Clear","recall":0.83},{"condition":"Common class","recall":0.79},{"condition":"Rare class (cyclist)","recall":0.66},{"condition":"Occluded","recall":0.45},{"condition":"Night","recall":0.39},{"condition":"Night + occluded","recall":0.27}],"worstCondition":"Night + occluded","worstConditionRecall":0.27,"triage":[{"failureMode":"Night-time pedestrian miss","recallSlice":"night 0.39","likelihood":"Common","impact":"Catastrophic","severityBand":"red"},{"failureMode":"Occluded child at night","recallSlice":"night+occluded 0.27","likelihood":"Occasional","impact":"Catastrophic","severityBand":"red"},{"failureMode":"Rare-class cyclist miss","recallSlice":"rare 0.66","likelihood":"Occasional","impact":"Catastrophic","severityBand":"red"},{"failureMode":"Occluded pedestrian (daytime)","recallSlice":"occluded 0.45","likelihood":"Occasional","impact":"Catastrophic","severityBand":"red"},{"failureMode":"Night glare phantom brake","recallSlice":"precision dip","likelihood":"Occasional","impact":"Serious","severityBand":"amber"},{"failureMode":"Clear-daylight car miss","recallSlice":"day 0.99","likelihood":"Rare","impact":"Minor","severityBand":"green"},{"failureMode":"Dusk / low-sun pedestrian","recallSlice":"transition","likelihood":"Occasional","impact":"Catastrophic","severityBand":"red"}],"topBandFailureModes":["Night-time pedestrian miss","Occluded child at night","Rare-class cyclist miss","Occluded pedestrian (daytime)","Night glare phantom brake","Dusk / low-sun pedestrian"],"passed":true,"attempt":2}

{"checkpointId":"example_iou_demo","code":"# WORKED EXAMPLE: Intersection-over-Union for one clean box pair.\n# Boxes are [x1, y1, x2, y2] in pixels. (Next slide: you do this on a tougher pair.)\n\ngt   = [60, 50, 200, 150]   # ground-truth box: where the object really is\npred = [68, 56, 206, 156]   # the detector's predicted box (a near-perfect fit)\n\ndef iou(a, b):\n    # 1) the SHARED rectangle (intersection): the overlap of the two boxes\n    ix1, iy1 = max(a[0], b[0]), max(a[1], b[1])\n    ix2, iy2 = min(a[2], b[2]), min(a[3], b[3])\n    inter = max(0, ix2 - ix1) * max(0, iy2 - iy1)\n\n    # 2) the COMBINED area (union): both boxes together, counted once\n    area_a = (a[2] - a[0]) * (a[3] - a[1])\n    area_b = (b[2] - b[0]) * (b[3] - b[1])\n    union = area_a + area_b - inter\n\n    return inter / union, inter, union\n\nscore, inter, union = iou(gt, pred)\nTHRESHOLD = 0.5\nverdict = \"HIT  (counts as a detection)\" if score >= THRESHOLD else \"MISS (no match)\"\n\nprint(f\"shared area    = {inter:6.0f} px\")\nprint(f\"combined area  = {union:6.0f} px\")\nprint(f\"IoU            = {score:6.3f}   (shared / combined)\")\nprint(f\"threshold      = {THRESHOLD}\")\nprint(f\"VERDICT        = {verdict}\")","stdout":"shared area    =  12408 px\ncombined area  =  15392 px\nIoU            =  0.806   (shared / combined)\nthreshold      = 0.5\nVERDICT        = HIT  (counts as a detection)\n","stderr":"","error":null,"imageCount":0,"durationMs":1,"ranAt":"2026-07-03T22:53:20.486Z"}

{"checkpointId":"example_metrics_demo","code":"# WORKED EXAMPLE: score a tiny 8-row detections-vs-ground-truth split.\n# (Next slide: you do this on the detector's full table.)\nimport pandas as pd\n\n# Each row = one scored object. outcome is:\n#   \"TP\" a real object the detector caught,\n#   \"FN\" a real object it MISSED (a false negative),\n#   \"FP\" a phantom box over empty road (a false positive).\ndemo = pd.DataFrame([\n    (\"Pedestrian\", \"TP\"),   # 3 real pedestrians, 2 caught, 1 missed\n    (\"Pedestrian\", \"TP\"),\n    (\"Pedestrian\", \"FN\"),\n    (\"Cyclist\",    \"TP\"),   # 2 real cyclists, 1 caught, 1 missed\n    (\"Cyclist\",    \"FN\"),\n    (\"Car\",        \"TP\"),   # 2 real cars, both caught\n    (\"Car\",        \"TP\"),\n    (\"Car\",        \"FP\"),   # 1 phantom car (braked for nothing)\n], columns=[\"obj_class\", \"outcome\"])\n\n# 1) Confusion counts over the whole split.\nTP = (demo.outcome == \"TP\").sum()\nFP = (demo.outcome == \"FP\").sum()\nFN = (demo.outcome == \"FN\").sum()\nprint(f\"TP = {TP}   FP = {FP}   FN = {FN}\")\n\n# 2) Precision = of everything flagged, how much was real.\nprecision = TP / (TP + FP)\n# 3) Recall = of everything really there, how much we caught.\nrecall = TP / (TP + FN)\nprint(f\"precision = TP/(TP+FP) = {TP}/{TP+FP} = {precision:.2f}\")\nprint(f\"recall    = TP/(TP+FN) = {TP}/{TP+FN} = {recall:.2f}\")\n\n# 4) The safety number: recall on vulnerable road users (pedestrians + cyclists),\n#    then the false-negative RATE = the share of real VRUs we MISSED.\nvru = demo[demo.obj_class.isin([\"Pedestrian\", \"Cyclist\"])]\nvru_TP = (vru.outcome == \"TP\").sum()\nvru_FN = (vru.outcome == \"FN\").sum()\nvru_recall  = vru_TP / (vru_TP + vru_FN)\nvru_fn_rate = 1 - vru_recall\nprint(f\"VRU recall  = {vru_TP}/{vru_TP+vru_FN} = {vru_recall:.2f}\")\nprint(f\"VRU FN-rate = 1 - VRU recall = {vru_fn_rate:.2f}   <- the safety number\")","stdout":"TP = 5   FP = 1   FN = 2\nprecision = TP/(TP+FP) = 5/6 = 0.83\nrecall    = TP/(TP+FN) = 5/7 = 0.71\nVRU recall  = 3/5 = 0.60\nVRU FN-rate = 1 - VRU recall = 0.40   <- the safety number\n","stderr":"","error":null,"imageCount":0,"durationMs":17,"ranAt":"2026-07-03T22:54:45.324Z"}