AIDDR midterm study guide
Lectures on risk definitions and tabular data, from the PDFs. The exam is about interpreting a result in light of possible AI risk. You are expected to work the in-class exercises, not to recite the slides.
Risk = likelihood of harm × consequence (impact / severity).
Everything else is a tool you use while looking at a chart. Bayes’ rule, F1, and the names of hypothesis tests can stay on a scratch sheet. A number with no exposure, no severity, and no statement of what it fails to prove is not an answer.
What a full answer does
You will be handed a table, a p-value, a model printout, or a short scenario. Walk it in this order. Skip a step and the risk claim is unfinished.
1. Hazard
A hazard is a state or event that can lead to harm: injury, death, or loss. “The model is an LLM” is not a hazard. “The vehicle stops in a travel lane during pickup” is. “The classifier misses a fatal crash” is. Connectionist systems add hazards that a pump-failure checklist will miss: bad or incomplete labels, training data that was itself generated by a model, environments the data never saw, a world model that is missing a sensor or a feature, too little testing, and a person who trusts the output. After a failure, the list shifts: lack of self-assessment, and human overconfidence.
The empty-room exercise is the pattern. Asked for a room with no elephant, the model draws an elephant and reports success. The hazard is an inadequate world model plus no check that the constraint was met. If a person still acts on that output, the hazard is overconfidence, not only the picture.
2. Consequence
Consequence is often a judgment. Say whose harm you are scoring and why you did not pick the cell next to it. The same event can be a delay for traffic and a severe outcome for a blocked ambulance. Score those as separate hazards. Do not average them into one cell.
3. Likelihood, and the basis
Record how you got the probability, not only the probability. Raw counts are not a rate. Waymo’s 53 crashes are not a worse record than Cruise’s 27, because Waymo drove about 5.5 million miles and Cruise about 1.5 million. The odds ratios on that slide are 4:1 (interval 3–5) and 8:1 (interval 5–11). NVIDIA’s 59:1 comes from one crash in 7,169 miles, and the interval runs from 8 to 421. The point estimate is not the risk. A one-event testing permit is not comparable to a commercial fleet.
Raw crash count
Y axis: number of crashes. Waymo looks worst.
Crashes per million miles
Y axis: crashes ÷ millions of miles (53 / 5.5, 27 / 1.5, 1 / 0.0072). Same data, opposite ranking.
Odds ratio with its interval
Dot is the point estimate, bar is the interval. NVIDIA’s bar spans almost two orders of magnitude, so its 59 cannot rank it against the others. Source: lecture slide on AV crash comparisons.
The flight example is the same move. 0.07 fatalities per billion passenger-miles, times the miles one person actually flies, is a personal risk. The slide treats a daily 500-mile flyer as about 1 in 85,000 over a year. Multiply rate by exposure, and write down the exposure you used.
4. Place it on the chart
On the class matrix, multiply the probability row (1 rare through 5 almost certain) by the impact column (1 insignificant through 5 severe). The product is the cell score. You do not need the color words memorized if you can multiply and then read the cell.
Class risk matrix. Score = probability level × impact level. Darker cells are higher scores. Source: lecture slides on qualitative risk.
The DoD chart is a different numbering. Rank 1 is the worst cell (almost certain × disastrous), and “incredible” is left blank. Do not mix the two grids. The DoD question on the slide is the one that matters for new AI: a system with no track record has no honest frequency, so a precise rank is a costume on a guess.
5. Say what the result does not prove
This is the part the exam is built around. Match the limit to the result you were given.
| If you are looking at | You may say | You may not say |
|---|---|---|
| A p-value of 0.054 | The data do not clear a 0.05 bar. With a handful of points the test has little power. | There is no difference, or the risk went away. |
| A huge confidence interval | The rate is poorly known. One crash on a few thousand miles cannot rank a fleet. | The company with the largest ratio is the most dangerous. |
| Accuracy of 89% on a deep tree | The tree fits this sample. A 3-level prune that falls to about 58% is the interpretable version. | The system is safe, or the 17-level tree found the real causes. |
| A non-significant logistic model | Area, time, and vehicle count do not clearly separate long brick events at this cutoff. | Those factors do not matter in the world. The cutoff itself may be the problem. |
| A stakeholder word such as “possible” | People map the same word onto very different probabilities. | You have a likelihood. Ask for a rate per mile or per trip. |
| A famous, vivid incident | Perceived risk and measured risk diverge. Shark attacks feel more likely than falling airplane parts; parts kill about 30 times more people in the US. | The viral case is the rate. |
Where the system sits before you score a cell
The bounding chart has two axes: deterministic to non-deterministic, and low to high safety criticality. Deterministic high-criticality systems (aircraft, fixed vehicle path plans) can be tested against a spec. Non-deterministic high-criticality systems cannot. That corner holds computer vision in surgery and in vehicles, and LLM chatbots in 911 centers. Medical-record summaries, probabilistic planners, and predictive maintenance sit in the middle band. Shopping recommendations, vacuum routes, and game vision are low criticality: a wrong answer is a product failure, not the same hazard analysis.
Certification gets harder as you move right and up, because you cannot list the outputs. N-gram counts already explode (hundreds of types at 2-grams, hundreds of thousands at 5-grams). A decoder that samples the next token from the last tokens is not a lookup table. “We tested the average case” is a weak safety argument in that corner.
ISO 26262 HARA, mandatory for cars, scores severity, exposure, and controllability. C1 is easily controllable, C2 normally controllable, C3 difficult, C4 uncontrollable. A human who cannot take over turns a moderate malfunction into a high risk even when the model’s accuracy looks fine. Uncertainty about the model is an argument about C3 and C4, not only about the error rate.
FMEA rows assume a function is lost: the pump fails to operate, the local effect in that flight phase is none, severity is IV. A vision system that labels a bus as a train has not “failed off.” It has degraded. Write the mode (wrong class), the mechanism (data, features, or a missing check), and the effect (a bad driving or dispatch decision). The Snohomish point on the slide: the useful next step is that FMEA, not only a story about the crash.
Fault trees start from the bad state and work down. AND means every cause is required. OR means any one is enough. Probabilistic risk assessment then puts numbers on those causes. The numbers help when the data exist. They mislead when the system is new, because a prior dressed as a probability looks like evidence.
Fundamental-problem slide: model “facts” are cited from the open web, with Reddit, Wikipedia, and YouTube at the top of one citation study, and a resume can smuggle an instruction the screen ignores. Provenance of the training distribution is part of the risk of trusting an output.
PUDO exercise
Pickup and drop-off complaints for self-driving cars. The data source on the slide is California PUC quarterly reports. Eight quarters, Q3 2024 through Q2 2026. Use complaints per trip or per mile, not the complaint count.
Step 1. Hazards
Stay concrete. A bad curb stop can block a bike lane, a travel lane, or a crosswalk. A passenger can step out into moving traffic. An immobilized vehicle can block an emergency response. A rider can be stranded. Each of those is its own hazard. “PUDO is in the news” is the motivation, not the hazard.
Step 2. Consequences
Most of these are minor to significant: delay, a near-miss, a blocked curb. A person struck while the vehicle is stopped in the wrong place, or an ambulance that cannot pass, is major or severe. Say that the score is a judgment. The COVID chart on the definition slide is the reminder that the same hazard does not have one consequence for every person exposed.
Step 3. Likelihood, rising or falling
Split the eight quarters into Y1 (Q3 2024–Q2 2025) and Y2 (Q3 2025–Q2 2026). Compute complaints divided by trips, and keep miles in the sentence if you have them. A rising count with a rising fleet can be a falling rate. State the direction of the rate, then pick the matrix row from that rate, not from how often the story appears.
The homework chart of complaints per trip starts near 4.5, drops to about 1.5 by the third point, and then sits between about 1.5 and 1.9. The one-sample t-test against a mean of 1.9 is t(3) = 3.1, p = 0.054. The slide calls this a non-significant large difference. With df = 3 you have four points of information. You cannot claim the last point is a new regime. You also cannot claim the early drop “proved” a permanent fix. The last point lives in the same band as the previous few.
Y axis: complaints per trip. X axis: reporting period. Values are approximate, redrawn from the shape described on the homework chart (about 4.5, down to about 1.5, then flat between 1.5 and 1.9). Not exact data.
Which test, and why
Plot first. The shape of the data decides the test. If the histogram looks like a bell, the tests that compare averages (t-tests) work. If it is lopsided with a long tail, a few extreme values drag the average around, so use a test that ranks the values instead.
Bell-shaped: use a t-test
X axis: trips per day (thousands). Y axis: number of days. Illustrative. Mean and median sit together in the middle.
Skewed: use a rank test
X axis: brick duration (minutes). Y axis: number of events. Illustrative, shaped like the FOIA histogram. The long tail pulls the mean well above the typical event.
| Bell-shaped numbers | Skewed, separate groups | Skewed, same things measured again | |
|---|---|---|---|
| Two groups | Two-sample t-test | Mann-Whitney | Wilcoxon signed-rank |
| Three or more groups | (ANOVA, not on the slide) | Kruskal-Wallis | Friedman |
| One group vs. a fixed number | One-sample t-test | — | — |
If the parametric and Mann-Whitney answers disagree, trust the test that matches the histogram. Then ask an LLM the same question and treat a confident disagreement as a reason to recheck the exposure and the distribution. The model is not a second sample.
Frequentist and Bayesian, as interpretations
Frequentist
The true complaint rate is a fixed number. A confidence interval and a p-value say how compatible the data are with a claim such as “Y1 and Y2 have the same rate.” The p-value is how surprising the result is if that claim is true. It is not the probability that the claim is true, and it is not the size of the harm.
Type I error (α) is rejecting a true “no difference” claim: a false alarm. Type II (β) is missing a real difference. Power is 1 − β. A non-significant result on four points is often a power problem, not a finding of safety.
Bayesian
The rate is a distribution you update. The disease slide is the calculation to be able to redo. Prior P(disease) = 0.001. Test is positive. True positive rate 0.99. False positive rate 0.05.
Numerator: 0.99 × 0.001 = 0.00099. Denominator: 0.00099 + 0.05 × 0.999 = 0.05094. Posterior ≈ 0.019. People guess “about 99%” because they ignore the base rate. P(disease | positive) is not P(positive | disease).
Everyone who tests positive, out of 100,000 people screened (derived from the slide’s rates). Only 99 of the 5,094 positives are sick, about 2%.
Drawback, especially for a new AI system: the prior is a judgment, and with little data the prior is the answer. A precise posterior can hide that. More data helps both approaches. No track record helps neither.
Make three histograms: complaints, trips, and complaints per trip. Symmetric and bell-shaped supports a t-test. A spike at zero or a long right tail does not. Counts of uncommon events often look Poisson. A ratio of two noisy counts is often skewed even when you hoped dividing would calm it down. Name the shape, then pick the test. Do not name a distribution from the variable’s reputation.
Perceived likelihood is not a row on the matrix
On the subjective-probability chart, “possible” stretches from near 0 to near 1. “Almost certain” clusters high and still has spread. Words from a stakeholder are not likelihoods. The intuition checks make the same point with base rates: lung cancer deaths are about 140,000 a year in the US and vehicle deaths about 32,000; lightning kills more Americans in a typical year than tornadoes. A vivid AV video will be scored “almost certain” unless you bring a denominator.
X axis: approximate US deaths per year. Source: intuition-check slide.
AV bricking exercise
An immobilized automated vehicle. Data are a San Francisco FOIA log, February–October 2025. The questions in red on the slide are the exam questions: how long events last, whether duration is getting better or worse, whether some areas or times are worse, and where the public risks sit on the matrix.
Read the histograms before any model
Duration is right-skewed. Almost all of the mass is in the first bin, with a thin tail that still reaches out toward 600–700. The typical event is the short one. The mean is pulled by the tail, so report a median, and treat the tail as its own hazard: a vehicle that sits for a very long time. The later logistic cutoffs are 20 minutes and 40 minutes. Changing that cutoff changes the label, which changes the model.
Time of day is a fraction of the day, and 0.55 is noon. The histogram rises through the afternoon and peaks before evening. That is a count of events, not a rate per mile driven at that hour.
Area codes on the slide: 1 Downtown / Tenderloin / Civic Center, 2 Financial District / Embarcadero, 3 Chinatown / North Beach / Nob Hill, 4 SoMa / Mission Bay, 5 Mission, 6 Castro / Noe / Bernal, 7 Hayes Valley / Haight, 8 Richmond / Presidio, 9 Sunset, 10 Bayview / Hunters Point / Excelsior. Piers 35 and 38 were folded into Embarcadero. Code 4 is the tallest bar. Without miles or trips in that neighborhood, a tall bar can mean the fleet spends more time there.
A hazard taxonomy, then one cell each
Build the list from the log and the rest of the FOIA material, not from a generic AV list. Separate at least: blocked traffic, blocked emergency access, conflict with people walking or cycling around the dead vehicle, a stranded passenger, and a second vehicle hitting the stopped one. Delay is often likely and minor. A secondary crash is less likely and much more severe. They do not share a cell. Say whether duration is improving only after you look across months and account for how many vehicles were operating. One histogram cannot answer “better or worse.”
Which model matches which question
| Question | Model on the slide | Assumption you must check |
|---|---|---|
| What predicts a continuous duration? | Multiple regression | More than one continuous predictor, and the variables are roughly normal. The duration histogram violates that. Transform it or do not trust the coefficients. |
| Is this event a long brick, yes or no? | Logistic regression | You need a 0/1 label. A 20-minute rule and a 40-minute rule are stakeholder choices. Some tools accept continuous features; time and area still need a principled category. |
| Are there natural groups in duration or time? | Clustering | Unsupervised. Explained variance says how cleanly the groups separate. It does not say the cutoff is a safe operating limit. |
| Which factors split long vs short, in order? | Decision tree | Binary outcome. Ordinal, continuous, and nominal inputs can mix. A deep tree will memorize a small FOIA file. |
How to read the logistic printout
The 20-minute model has chi-square about 2.93 on 3 degrees of freedom, p about 0.40. The 40-minute model is about 3.21, p about 0.36. Neither model beats a null model of “no relationship.” Area and time coefficients are small and nowhere near significant. The multi-vehicle coefficient is the largest in both fits. At 40 minutes its p-value is about 0.09 and its odds ratio is a bit above 2, but the interval still includes 1. An interval that includes 1 means the data still allow “no increase.”
Y axis: p-value. Every bar is above the 0.05 line, so nothing clears significance. Source: logistic regression printouts on the bricking slides.
Clustering explained variance is about 94% for one cut and 75% for another. Use that to talk about separation of durations. Do not use it to declare a neighborhood unsafe. If stakeholders pick 20 minutes because that is when an intersection becomes intolerable, say that the safety threshold and the statistical cluster are different decisions.
How to read the decision tree
The in-class tree for a 20-minute target splits first on number of vehicles. A later Claude fit ranked time at about 48%, area at about 41%, and vehicles at about 10%, on a 12-level tree with about 84% accuracy. Another full tree on the slide goes to 17 levels and about 89% accuracy, with 62 leaves. Pruned to depth 3 it has 6 leaves and about 58% accuracy. The drop is the finding. The deep tree fit this file. The small tree is what you could explain, and it is only modestly better than a coin flip. “Vehicles should matter” was a hypothesis. In these fits it matters less than time and place.
Accuracy by tree depth
Y axis: accuracy (%). Deeper trees score higher on this file because they memorize it. The depth-12 tree is a separate fit.
Feature importance, depth-12 fit
X axis: share of the tree’s splitting importance (%). Source: Claude-built tree on the bricking slide.
The slide says not to use Copilot for the tree. The picture is an unreadable bush. A model that emits a 17-level structure and an accuracy in the high 80s has not certified the vehicle. It has produced another output you would need to audit. Prefer the pruned tree, and say what accuracy you gave up for that audit.
Bias in this specific task: a human chose the minute cutoff and the neighborhood bins. The FOIA set is one city and eight months, and only events someone logged. Piers were remapped. The duration distribution is skewed, so a mean or a deep tree will chase outliers. Mitigation is the same list as the data-bias slide: provenance, balance, what was dropped or imputed, and a written record of the cutoff. Good cleaning is one of the stronger risk controls in the course, because it stops a model from laundering a logging artifact into a safety claim.
Reading bias, missingness, and a confusion matrix
Three biases overlap. Human bias is a person’s choice (labels, cutoffs, which error they care about). Systemic bias is institutional or historical, already in the file before you model it. Statistical bias is the gap between a true parameter and the one you estimated: train and test histograms that do not match, a variance you shrank by imputing the mean, a coefficient you cannot interpret because two features move together.
Invalid moves in the modeling workflow
The workflow slide marks these as invalid, against a list of subjective choices that are allowed if you document them. On an exam, naming the invalid move is the risk interpretation.
| Invalid choice | Why it distorts the risk |
|---|---|
| Leaving bad entries (negative ages, duplicate rows) | The fit describes the logging error. |
| Imputing, encoding, or oversampling before the train/test split | Information from the test rows leaks into training. Accuracy is no longer an honest check. |
| Mean imputation when missingness is not completely random | You pull the distribution toward a value the missing rows may not have. |
| Keeping a feature that is not a legitimate predictor | A proxy (a demographic stand-in, a post-outcome field) invents a cause. |
| Scoring the model on the training rows | You measure memorization. The deep bricking tree is the picture of this. |
Missing-data exercise
Target on the slide: fatalities or serious injuries. Predictors are the other columns. Histogram every column before you repair anything, and mark outliers. Then, for the MAR and MCAR files, do both treatments: fill with the mean or median, and drop incomplete rows. Histogram again. The treatment that moves the shape is the more biased one, even if its accuracy looks better.
X axis: travel speed (mph). Y axis: number of crash records. Illustrative. Mean fill stacks every blank into one bar, which squeezes the spread and hides the fast crashes that may be the ones missing.
| Pattern | Meaning | What goes wrong if you ignore it |
|---|---|---|
| MCAR | Missingness is unrelated to the data. | Complete-case analysis mostly costs you sample size and power. |
| MAR | Missingness depends on columns you can see. | Dropping rows drops a predictable slice of the population. Mean fill still flattens variance. |
| MNAR | Missingness depends on the missing value itself. Severe crashes not written down. | Both deletion and mean fill invent a safer dataset than the road. No standard fill fixes this. Say so. |
Build a logistic regression on each repaired file and compare accuracy and F1, not accuracy alone. There is no universally correct fill. There is a large penalty for the worse one. Feature deletion is called out on the slide as its own treatment: dropping a column because it is messy can drop the hazard you were asked to study. Last-observation-carried-forward is a time-series convenience and still biases non-MCAR data. Model-based and multiple imputation can represent uncertainty better and still fail under MNAR, and they cost more computation.
Feature exercise
Question first: which factors contribute most to fatalities and severe injuries on rural roads. Start from first principles (speed, light, road type, restraint, impairment) and a simple filter, and stop at about ten features. The heuristic is at least 10 observations per feature, and 30 is better. A kitchen-sink model on a small crash file will look accurate and will not answer “what contributes most.”
Multicollinearity is the hard constraint. Two features that rise together make the coefficients swap credit, so you cannot rank causes. The numerical cutoff for “too correlated” is a judgment. Missingness still matters here: a feature that is missing exactly when the crash is severe is not a candidate for a clean coefficient. If performance is poor, go back to the histogram, the fill, and the feature list before you reach for a more complex model. Document every drop.
| Selector | Examples | Tradeoff |
|---|---|---|
| Wrapper | SFS, RFE | Often predicts well. Unstable, and expensive. |
| Filter | SelectKBest, MRMR | Stable and cheap. Can miss a combination that only works together. |
| Embedded | L1 penalty | The model itself shrinks features. Not available for every algorithm. |
Accuracy is the wrong headline when harm is rare
For a fatal or serious injury, a false negative is the model saying “not severe” when it was. A false positive is a false alarm. Precision is the share of alarms that were real: TP / (TP + FP). Recall is the share of real events you caught: TP / (TP + FN). Accuracy counts true negatives in the numerator, so a model that never predicts a fatality can score very high on a rare event.
Y axis: score (%). Illustrative: 1,000 crashes, 20 fatal. Model B has 15 true positives, 5 misses, and 60 false alarms. Model A wins on accuracy and misses every fatality.
F1 = 2 × precision × recall / (precision + recall). Use it when you need one number that notices both missed harms and false alarms. On this course’s safety questions, lead with recall and then F1. Quote accuracy only to show how it can flatter a model that misses the hazard. The class feature exercise asks you to post accuracy, precision, recall, and F1 for the ten-feature model. Bring all four, and say which error is the dangerous one.
What you write down so someone else can judge the risk
Model card
Six sections: model details, architecture, training data and methodology, performance metrics, biases and limitations, ethical considerations. It reports how the model behaves across groups and where it is meant to be used. The slide asks whether this is standard practice. It is a disclosure, not a certificate that the output is safe.
System card and checklists
A system card adds harm-category scores and benchmark tables. A high score on a knowledge benchmark does not move a 911 chatbot out of the high-criticality, non-deterministic corner. The bias checklist separates historical bias in the labels, population mismatch between training and use, temporal drift, and the fallacy of collapsing different time periods into one row.
State the research question. Read where the data came from, including the column legend. Histogram and mark outliers. Decide the missing values with the histogram as the screen, because you will not be told MCAR versus MNAR in the field. Select features, check multicollinearity, and accept that you may loop. If the model is weak, undo an earlier choice before adding complexity. Write the decisions down.
Practice prompts
Close the notes and answer in five sentences: hazard, consequence, likelihood with its denominator, matrix cell, and the claim you refuse to make.
- The complaints-per-trip series falls from about 4.5 to about 1.5 and then stays flat. The test of the last point is p = 0.054. Is PUDO risk falling, and where do you put a blocked-lane hazard?
- One company has 53 crashes and another has 1. The second has the worse odds ratio and an interval hundreds of points wide. Which comparison is the risk comparison?
- A 40-minute bricking model is non-significant, and the multi-vehicle interval includes 1. A cluster cut explains 94% of duration variance. What can you tell the city?
- Fatalities are rare. Model A has higher accuracy. Model B has higher recall and a lower F1 gap between precision and recall. Which one did you fail if a severe crash was labeled safe?
- Severe-crash rows are blank more often than minor-crash rows. You fill the blanks with the column mean and accuracy rises. What pattern of missingness is that, and why is the new accuracy a worse safety argument?
Source: AIDDR intro / risk lectures and Tabular Data I–II PDFs. PowerPoint copies were not used.