I pre-registered the threshold, then missed it. Here is the number anyway.

01The result, stated plainly

The model beat a category-median baseline by 9.8% RMSE. The pre-registered bar was 20%. The RMSE ratio was 0.902 with a 95% confidence interval of [0.861, 0.949].

That interval matters twice. It excludes 1.0, so ingredient labels genuinely do carry information about acrylamide. It also excludes 0.80, so that information is decisively less than what I committed in advance to calling useful. This is not an underpowered result. It is a clear negative one, and those are different things that get conflated constantly.

The thresholds, metrics, and failure rule were committed before any modelling. The test wave was hashed before it was seen. No model was refit after the test was unsealed. Every one of those constraints exists to stop me from doing what everyone does when a result disappoints: quietly moving the bar.

02Why it could never have worked well

FDA measured many products more than once. The pooled within-product standard deviation across those replicates is 0.223 in log10 ppb, roughly a factor of 1.7 between two samples of the same product off the same shelf.

No model built from labels can predict that, because the label is identical across replicates. Against a category-median residual SD of 0.474, this caps the maximum achievable RMSE reduction at roughly 50% for any label-based approach, mine or anyone else's.

Measuring the noise floor before interpreting the result changed what the failure means. Without it, 9.8% reads as a weak model. With it, 9.8% reads as most of what a label can possibly tell you, which is a statement about labels rather than about my regression.

03The finding I was not looking for

A baseline fitted on the 2011 wave scored much worse on 2015 than a 2015 oracle did. My first reading was reformulation drift: products changing over four years. That reading was wrong, and it was wrong in a useful way.

Of products appearing in both waves with uncensored measurements, the mean within-product shift is +0.023 log10 with a spread of 0.233, which is indistinguishable from zero and identical to pure replicate noise. No product detectably changed. What changed was which foods FDA put under each category name. In Infant Foods the median fell by 1.56 log10 because the sampled items moved from teething biscuits and baked grahams to jarred fruit and vegetable purees.

The generalisable claim is bigger than acrylamide: food category labels bundle heterogeneous processes. A statistic computed over one sample of a category transfers poorly to another sample of the same category, with no underlying change in any product. That limits category-based inference regardless of what model you fit on top of it.

04The bug that would have invented a result

The first record-linkage implementation used token_set_ratio and returned a 97.4% match rate, which should have been implausible on its face. It was. That scorer returns 100 whenever one token set is a subset of the other, so every product matched its own brand page: "Ore Ida Waffle Fries" scored a perfect 100 against "Waffle".

The diagnostic that catches it is counting distinct matched identifiers rather than trusting the score. 991 products had collapsed onto 436 codes, a 2.27x pile-up, crowding onto tokens like "Chips" and "Great Value". A second tell: 97.1% of matches had a gap of exactly zero between the best and second-best candidate.

After fixing it, the honest join rate against Open Food Facts is roughly 35 to 45%. I report that as a finding rather than a footnote: the two largest public datasets in this domain do not reliably join, which constrains anyone attempting product-level contaminant research at scale.

I also checked whether the matching filter had quietly selected for predictability, because that would have softened my own threshold. Matched and unmatched products differ in level and category mix, but their within-category residual SD is identical (0.474 versus 0.473). The bar was not made easier.

05Why publish a failure at all

Because the alternative is the thing that makes most published results untrustworthy. If I only ship studies that clear their own bar, my repository stops being evidence and becomes a highlight reel, and nobody can tell which of my successes survived a fair test.

There is also a concrete consumer of this result. Food-scanning apps assign a product a score from a barcode, a label, or a photograph. This study measures how much of a process contaminant such inputs can recover: about a tenth of the variation, against a floor set by variation the label cannot see at all. Two bags of the same chips differ by roughly 1.7x, so any interface showing one number per SKU is presenting sampling noise as a property of the product. And a photograph contains no molecular information, so it recovers the label at best, which makes 10% an upper bound on that approach rather than a target for it.

What I would change: the fried-versus-baked contrast, the only genuine process variable in the data, exists in 28 test rows and zero training rows. I should have checked the design matrix for that before committing the feature set, because a variable that cannot appear in training cannot support inference no matter how mechanistically obvious it is.