Who Should Be Generated? Justifying Demographic Targets in Open-Ended Generation
A research paper formalizes the missing-target problem for demographic-value-unspecified generation, decomposing target construction into four commitments: evaluative object, prior admissibility, allocation, and operationalization. It admits a geographic prior under a geographic-membership interpretation and an occupational prior under an incumbency interpretation requiring an independently defended objective. Instantiated in AP-Bench, distribution divergence from geography-derived targets ranges from 0.508 to 0.606 on a 0–1 scale.
The paper addresses fairness evaluation in open-ended generation where prompts leave demographic realization to the model. It argues that existing group fairness definitions assume sensitive attributes are given on the input side, while generative audits examine output-side demographic composition against typically supplied targets. The work formalizes the missing-target problem and proposes a framework for constructing target distributions, demonstrating substantial divergence from geography-derived targets in AP-Bench.
The framework decomposes target construction into four commitments: evaluative object, prior admissibility, allocation, and operationalization. It distinguishes between geographic-membership and incumbency interpretations for priors, with the latter requiring an independently defended objective such as workforce-composition fidelity. The AP-Bench instantiation reveals distribution divergence of 0.508–0.606, indicating significant misalignment between model outputs and geography-derived demographic targets.
This research highlights a gap in current generative AI evaluation practices, where demographic targets are often assumed rather than justified. It suggests that model developers and auditors need to explicitly define and defend the demographic distributions they use for fairness assessments, potentially impacting how companies report and mitigate bias in generative models.
For AI companies, this work provides a structured approach to justifying demographic targets in fairness evaluations, which could reduce legal and reputational risks. It may also influence product design by encouraging more transparent and defensible demographic representation in generative outputs.
Observable next signals include adoption of the proposed framework in fairness auditing tools, updates to benchmark suites like AP-Bench, and increased scrutiny from regulators on how demographic targets are chosen in AI evaluations. Further research may explore alternative priors and their implications for different use cases.