Disagreement In, Disagreement Out? Uneven Downstream Effects of Harm Classifier Differences: "Harm classifiers are machine learning models that are primarily used in three settings: content moderation, where they scale up the detection of harmful text; intervention selection, where they identify the most effective alignment interventions; and preference alignment, where they help train models to be harmless. Prior work has established that popular harm classifiers disagree substantially with each other in terms of which pieces of text are harmful. However, it is not yet known whether these differences actually matter in the settings where harm classifiers are used. We find that for content moderation, some pairs of harm classifiers can disagree in a majority of cases and half the pairs disagree on at least 27% of text; and for intervention selection, five of six alignment interventions rank first under at least one of ten harm classifiers, meaning that the choice of harm classifier can determine which intervention appears best. On the other hand, for preference alignment, we find that harm classifiers have little effect on the aggregate harmfulness of models aligned on their preference signal. Even when two harm classifiers disagree on 55% of texts, the responses of the associated pair of aligned models differ in average harm rates by only 3 percentage points. However, these models do not behave identically: if harm classifiers are more likely to flag text containing content features like political content, the associated aligned models are also less likely to generate that content feature. Our results establish that the choice of harm classifier, which is often seen as an inconsequential choice of convenience, is in fact a critical design decision that can have dramatic effects in some settings (like content moderation), but negligible impacts in others (like harm reduction in preference alignment)." — Vyoma Raman

The Pathetic Designer: How AI Erodes Design Intention, and How Designers Can Push Back: "Generative AI now takes part in every step of design work, and it is becoming hard to tell whose intention stands behind a design. Drawing on interviews with eighteen students in an interactive device design course, I show how design intent erodes and how students push back. Some polish the signs of design work into cosmetics; others recapture intent by repairing, steering and guarding their ideas. Borrowing Lessig's "pathetic dot," I map the forces that press on a designer: policy, tool architecture, grade rivalry and norms. I then turn to professionals. In an experiment with 141 UX designers, adding measures of indecent persuasion to standard UX evaluation data nearly doubled designers' rejection of dark patterns and changed how they justified their decisions. Both studies ask one question: how do we know what counts as good design?" — Hauke Sandhaus