Gender and racial bias issues in a commercial "tone of voice" analysis system.

Social Feedback Speech Technologies (SFST) are programs and devices, often "AI"-powered, that claim to provide users with feedback about how their speech sounds to other humans. To date, academic research has not focused on how such systems perform for a variety of speakers. In 2020, Amazon released a wearable called Halo, touting its fitness and sleep tracking, as well as its ability to evaluate the wearer's voice to help them "understand how they sound to others". The band presents its wearer with 'Positivity' and 'Energy' scores, as well as qualitative evaluations of the voice: adjectives such as 'confident', 'hesitant', 'calm', etc. This study evaluates how Halo performs for American English speakers of different races and genders. We recorded Black and white men and women reading three passages aloud and played them back to the same Halo device in identical positions. We then obtained Halo's Energy and Positivity scores (out of 100), as well as the device's qualitative descriptors of 'tone of voice' for each subject. We subsequently analyzed effects of different acoustic properties, as well as speaker race/gender and the interaction, for how the device scores 'tone of voice'. Overall, Halo's Energy ratings and qualitative descriptors are biased against women and Black speakers. Halo's Positivity scores appear to be based on lexical sentiment analysis and therefore do not vary substantially by speaker. We conclude by discussing the expanding role of SFSTs and their potential harms related to the reinforcement of existing societal and algorithmic biases against marginalized speakers.