Featured Post from Emory University, Goizueta Business School

Is AI Censoring Us?

Aug 7, 2023

5 min

Ramnath K Chellappa


Artificial intelligence has been hogging headlines around the world in recent months. In late March 2023, an unprecedented coalition of tech CEOs signed an open letter calling for a moratorium on AI training. The race to empower powerful artificial minds should be paused, argued signatories (including Elon Musk) to give humanity time to review and reassess the potential risks of developing “human-competitive intelligence”–intelligence that “no one–not even their creators–can understand, predict, or reliably control.”


Concerns about the unchecked rise of AI are not new, and global media is increasingly sounding the alarm, citing concerns that range from invasion of privacy to an existential threat to human existence.


Weighing in on this with compelling new evidence around the “unintended consequences” of AI is research by Goizueta’s Ramnath Chellappa and Information Systems PhD candidate, Jonathan Gomez Martinez.


Uncovering the Threat


Their paper, Content Moderation and AI: Impact on Minority Communities, takes a hard look at how the use of AI in social media could disadvantage LGBTQ+ users. And what they find is worrying.


Chellappa, who is Goizueta Foundation Term Professor of Information Systems & Operations Management, explains that he and Gomez Martinez homed in on Twitter to explore how unchecked artificial language moderation might (mistakenly) censor the use of “otherwise toxic” language by failing to understand the context or nuanced use of the LGBTQ+ lexicon. Examples of this include “reclaimed language”—verbiage that would be a slur in other contexts—but is reclaimed and prosocial if used by the originally targeted community.


Their paper, Content Moderation and AI: Impact on Minority Communities, takes a hard look at how the use of AI in social media could disadvantage LGBTQ+ users. And what they find is worrying.



Chellappa, who is Goizueta Foundation Term Professor of Information Systems & Operations Management, explains that he and Gomez Martinez homed in on Twitter to explore how unchecked artificial language moderation might (mistakenly) censor the use of “otherwise toxic” language by failing to understand the context or nuanced use of the LGBTQ+ lexicon. Examples of this include “reclaimed language”—verbiage that would be a slur in other contexts—but is reclaimed and prosocial if used by the originally targeted community.


“This is a community that has ‘reclaimed’ certain words and expressions that might be considered offensive in other contexts. Terms like ‘queer’ are used within the community both in jest and as a marker of identity and belonging. But if used by those outside the community, this kind of language could be deemed inflammatory or offensive.”



Gomez Martinez adds: “We wanted to measure the extent to which AI’s lack of a nuanced understanding of what is ‘acceptable’ affects minority users’ online interactions. As humans, we understand that marginalized communities have long used ‘reclaimed words’ both in jest and as a kind of rallying cry. Our intuition was that the machine simply wouldn’t understand this without context—context that is more immediately apparent to people.”


Determining the Impact of AI-Based Moderation


To test this, he and Chellappa looked at data from social media behemoth, Twitter. During the pandemic in 2020, the platform made a significant shift to AI-based content moderation to accommodate stay-at-home measures. Data from Twitter’s proprietary Academic Research API afforded Gomez Martinez and Chellappa access to a complete listing of historical tweets and replies before, during and after this period. Together they analyzed a total of 3.8 million interactions (1.8 million tweets and 2.0 million replies) from a panel of 2,751 users, of which 1,224 self-identified as LGBTQ+ in their Twitter bios. Their study ran over four months, from January to May 2020, before, during and after the switch to machine-based moderation.


Using the same tools that Twitter moderators deploy to moderate interactions, Gomez Martinez and Chellappa were able to measure any increase or decrease in pro-social, in-group teasing and toxic language among LGBTQ+ users: terms such as “bitch” or “queer,” which research shows to be a form of ritualized insults—dubbed “reading” by the community—which can appear inappropriate or incoherent to outsiders, says Chellappa.


“Analyzing the language, we find a notable reduction in the use of terms that could be considered toxic. When the AI moderation is in effect, you see these users’ language become more vanilla,” he adds. Quantifiably so, in fact.


Chellappa and Martinez find a 27 percent reduction in the use of reclaimed language among LGBTQ+ users. And while that doesn’t sound like much, it’s significant for the community, says Gomez Martinez.



Using in-language and reading each other is one way for this marginalized group to create a sense of community and social status. Not just that, we know from research that LGBTQ+ people use slurs and insults as a way of preparing themselves emotionally and psychologically for hostile interaction with heterosexual individuals. This kind of teasing and playing helps build resilience, so any reduction in it is significant.”

Jonathan Gomez Martinez



Good Intentions May Breed Unexpected Consequences

So what does this mean for social media, for the LGBTQ+ community or any marginalized group for that matter, that might be prone to automated censorship? And how does any of this play out in the context of broader concerns around AI?


For Chellappa and Gomez Martinez, there is a major hazard in granting technology any degree of control over how human beings interact. And it’s rooted in the mismatch between good intentions and unexpected consequences. Their paper, one of the first to dig into the impact of AI on actual business and society, lays bare some of the real-world impact AI has already had on marginalized people. While this study looks at the LGBTQ+ community, it could equally apply to any group that is prone to bias or exclusion—racial minorities or any other underrepresented demographic.



“Wherever you have user-generated content, you are likely to find communities with their own, unique way of interacting. We looked at LGBTQ+ Twitter users, but you could also look at the African American community, for instance.”

Ramnath K. Chellapa


At a time when social media platforms have become almost newslike in their influence, this is a concern. On the one hand, censoring certain demographics might earn Twitter et al an unwanted reputation for being anti-LGBTQ+ or racist, he adds. But there are even bigger stakes here than bad publicity.


“Twitter has long aspired to be a kind of global town square,” says Gomez Martinez. “But you end up pretty far from that scenario if only some voices are truly heard, or if you start reinforcing biases because you are using a time-saving technology that is not equipped yet to understand the complexity and nuance of human interaction.”


AI isn’t there yet, say Chellappa and Gomez Martinez. And they caution against using AI indiscriminately to expedite or streamline processes that impact human communication and interchange. If we don’t keep track of it, their research shows that AI has the potential to start dictating and moving people into normative behavior—effectively homogenizing us. And that’s a problem.


Looking to know more? Ramnath Chellappa is available to speak with media. Simply click on his icon now to arrange an interview today.

Connect with:
Ramnath K Chellappa

Ramnath K Chellappa

Professor of Information Systems & Operations Management; Academic Director, MS in Business Analytics
Pricing Digital Product Pricing and PiracyEconomics of PrivacyElectronic Markets and US Airline PricingManagement of Information Security and Privacy
Powered by

You might also like...

Check out some other posts from Emory University, Goizueta Business School

Why Negative Campaign Ads Work: David Schweidel on the Psychology Driving This Election Cycle featured image

2 min

Why Negative Campaign Ads Work: David Schweidel on the Psychology Driving This Election Cycle

As the 2026 Senate races heat up, negative campaign ads are once again dominating the airwaves. David Schweidel, Professor of Marketing and the Roberto C. Goizueta Professor in Business Technology at Emory's Goizueta Business School, has researched political advertising for years and is currently tracking the 2026 Senate races. Asked why negative campaigns tend to outperform positive ones, Schweidel points to what sticks with voters: "It's those negative messages. It's those attack messages," often fear- or anger-based, that he says are "more arousing to us" and "tends to move the needle more so than positive advertising." Where an ad comes from matters too. Schweidel's research looks at whether messaging originates from the candidate directly or from third parties like PACs or political parties, and he's found that candidate-sourced messaging tends to be more believable, "coming from a human brand," in his words, rather than an unfamiliar political organization. His current research pushes this further, into how political advertising shapes what AI chatbots tell voters. Schweidel notes that where news coverage and social media once drove poll movement, more voters are now turning to AI chatbots for candidate information. Using Maine Senate candidate Graham Platner as an example, he explains that recent news coverage and online conversation about a candidate gets absorbed by these chatbots, ultimately shaping what's presented to a voter asking about that candidate. For campaigns and advertisers, Schweidel frames this as a new channel to understand, similar to how companies already monitor social media conversation, and predicts political campaigns will start actively tracking how their candidates are portrayed in AI responses, the same way many companies now treat AI presence the way they once treated search engine optimization: "What a lot of companies are trying to come up with now is what is the playbook to do the same thing for AI." Dr. Schweidel is an expert in marketing technology, AI, social media, political marketing, and customer analytics. He holds a PhD in Marketing from the Wharton School of the University of Pennsylvania and is the author of Social Media Intelligence and Profiting from the Data Economy. His research has appeared in the Journal of Marketing, Journal of Marketing Research, Marketing Science, and Management Science, and he has been recognized as a Marketing Science Institute Young Scholar and named to Poets & Quants' "Top 40 Under 40." Dr. Schweidel is available to discuss: Why are negative campaign ads more effective than positive ads? Why do negative emotions drive people to vote, donate, and campaign, more than positive emotions? The connection between AI and campaign ads How organizations make explicit decisions to exploit these trends Click on the connect button in his profile below. 

World Cup 2026: The Business Behind the Game featured image

1 min

World Cup 2026: The Business Behind the Game

As the 2026 FIFA World Cup unfolds across North America, Emory University’s Goizueta Business School experts are available to help media explore the business stories behind the world’s biggest sporting event, from the economics of hosting and ticket pricing to global sponsorship, player brands and the psychology of fandom. Goizueta’s World Cup 2026 Business Hub brings together faculty who can provide timely, research-backed commentary on the commercial, cultural and consumer forces shaping the tournament as it moves from match to match, city to city and story to story. Featured Topics The Economics of Hosting Infrastructure investment, tourism revenue, real estate, local labor markets and the broader financial impact of hosting World Cup matches. The Science of Fandom What drives global fan devotion, audience loyalty and engagement across stadiums, broadcasts and digital platforms. Ticket Pricing and Demand Dynamic pricing, hospitality packages, travel costs and how extraordinary demand shapes the fan experience at major global events. Brand Strategy and Global Sponsorship How companies evaluate World Cup sponsorships, build global campaigns and measure the return on major sports partnerships. The Rise of the Player Brand How star footballers build, extend and monetize personal brands that reach far beyond the pitch. Media can visit Goizueta’s World Cup 2026 Business Hub to explore available experts and connect directly with the right source for their story.

+1
Expert Insights: Want More Engagement? Eliminate the Barriers. featured image

8 min

Expert Insights: Want More Engagement? Eliminate the Barriers.

Anyone born in the 70’s or earlier will probably remember it well. Time was when playing any kind of video game meant physically disporting yourself to the local arcade—a twilight zone of flashing neon, electronic beeps and bops, and the clink of quarters hitting the slot. As technology advanced, the videogame came to you. Home consoles and TV stations rigged with joysticks duly became the mainstay of gaming. The Atari 2600 brought the arcade experience into dens all over the US; Pac-Man, Space Invaders, and Asteroids now at the fingertips of a generation of games who no longer needed to leave home to play. Fast forward to the era of smart phones and hi-tech, and gaming has evolved again. Today, Fortnite, Minecraft, and The Legend of Zelda can accompany you pretty much anywhere—onto a train or a bus, into the canteen at work or school, or under the covers at 2am. In our always-on, on-demand world, video gaming increasingly meets players where they are; a play-anywhere, digital user experience that empowers individuals to engage with their game of choice wherever they are, whenever it suits, and via whatever platform they prefer, desktop or mobile. For users, the benefits seem clear. But what about game producers? As availability expands to new channels and platforms, how does it change user behavior? Does it deepen engagement or does cross-platform continuity simply end up redistributing play—the addition of each new platform shifting players away from, and effectively cannibalizing, existing channels? It’s a conundrum, and not just for video game producers. Retailers, bankers, insurance firms, media, and hospitality providers—anyone with an online-first approach looking to meet their customers wherever they are—should also be cognizant of the potential downsides of channel expansion in the digital space. Weighing in here is research by Professor of Marketing and expert in the intersection of sports and cultural analytics and marketing Michael Lewis. Together with Wooyong Jo of Purdue, Lewis looks at the impact of omni-channel strategy on videogames—a proxy, he says, for other sectors and industries. What they find is critical for marketers and decision-makers in any context or business setting. Increasing the digital touchpoints between your product and customers does impact behavior—but the net results are overwhelmingly positive. Video game players play more, they spend more frequently, and they integrate gameplay more deeply into their everyday lives. In other words, the investment pays off. And the dividends in customer engagement are serious. Switching to the Switch To unpack all of this, Lewis and Jo partnered with a large US video game publisher to analyze player-level behavioral data for one its major titles in the Multiplayer Online Battle Arena, or MOBA genre. Players form teams and compete to destroy opposing team’s bases, selecting a character from a set of 100+ options. Revenue for the publisher comes from a “freemium” business model—users can make voluntary purchases to unlock new characters or buy cosmetic enhancements. These purchases are geared toward enhancing the gaming experience but don’t affect competitive outcomes, making them a critical measure of engagement. In 2019, the game was released for the Nintendo Switch, which can be docked in home consoles but is most commonly used as a mobile, hand-held device. PC players were given the option to download this new version and continue gameplay seamlessly using their existing accounts. Analyzing player behavior before and after the adoption of the new Switch platform, Lewis and Jo were able to zoom in on some critical measures of user engagement including game usage or the total number of matches played, in-game spending—what, when and how much players spent—and player inactivity or churn. “We were able to really get into player behavior over time, and what happens when you introduce the Switch option and remove the constraints of having to play in one place—the home or gaming PC,” says Lewis. “What happens when you make it possible for players to access the game they love while they’re commuting or on their lunchbreak?” Plenty, it turns out. Mobile access: gameplay, spending and churn Crunching the data, Lewis and Jo find that mobile access dramatically increases gameplay. Players who adopted the Switch version played approximately 31% more games than before—a dramatic uptick that underscores how flexibility gains translate into new opportunities to play and engage. And that’s not all. Lewis and Jo also find that gameplay becomes less concentrated within narrow windows—after school or work, say—and is now more spread out across the day, the result of the “ubiquity effect,” says Lewis. “Take away the constraints of having to be in a fixed location and you see players adding additional play sessions. Interestingly though, we don’t find any adverse effect on PC gaming. Players are simply playing more, and playing longer, rather than replacing PC time.” Then there’s in-game purchasing. MOBA-type games typically give players the option to voluntarily buy modifications for characters, known as “skins.” These skins are cosmetic enhancements: new armor, costumes, skill animations or effects. Crucially, these kinds of purchases don’t advance players to new levels of success in the game. Instead, they are used for personalization—to demonstrate status or to celebrate an in-game event. Lewis and Jo find that mobile adopters make more frequent in-game purchases. While the overall total doesn’t increase materially, these players are spending small amounts, more often—almost 7% more frequently than before. This makes intuitive sense, says Lewis. If players are logging in more often, they have more opportunities to feel inspired to want to spend on skins. But there’s another factor that may be at work. “With this kind of in-game purchasing, it’s likely that a lot of it is about credibility. When you buy a skin or a character pack, it’s like you have more aura within the game; you want to signal something to other players and let yourself be known. And this is more than just monetary, it’s about a deeper kind of engagement,” says Lewis. “It’s possible that as mobile access makes the game more of a frequent companion, as the rate of play increases, there’s this effect that players fall deeper into the community—their engagement deepens even more.” Interestingly, the shift to mobile access had the most significant impact precisely on those players whose pre-Switch in-game purchasing was lowest. These users, who were arguably most likely to disengage and drift away from the game, became significantly more active once the hand-held option became available. “If you have players spending less and less inside the game, the intuition is that these are the customers you are most at risk of losing,” says Lewis. “Bringing in the Switch has seen these customers—those more prone to churn—actively reengage with the game, maybe because they have greater propensity for the mobile version.” Either way, this should be a particularly interesting finding for marketers, he adds; retaining existing users is typically cheaper than attracting new ones. “The evidence suggests that mobile access can serve not only as a growth strategy, but also a defensive one if it helps keep marginal users engaged; those who might otherwise have detached from the product altogether.” Help Them Switch So far, so encouraging. There is one potential downside to porting a game or online product to a new channel, however, and that is usability. Lewis and Jo find that players who switched between platforms experience a slight, initial decline in in-game performance—likely because of differences in the control systems between devices. Players who’ve been using keyboard and mouse controls may need time to adapt to hand-held controllers. To mitigate this, he and Jo suggest that producers could offer tutorials or introductory gameplay modes that accelerate the learning curve as users adjust to the new interface. In most cases, usability should be factored in as an additional, hidden cost, when developers and organizations are contemplating investing in more online customer touchpoints. “Expanding your online channels will always have some cost. Taking a game from one platform and porting it to another one isn’t free, so you will want to anticipate the hurdles, even as you weigh up the clear benefits,” says Lewis. “The key is to make sure you protect your users. With things like video games, you want to think about how to guide or upskill your players, maybe have them play bots at first to ramp up their capabilities. Whenever you create a new channel that has a different operating system from the user’s perspective, you’re probably going to want to provide some aid to your fan community.” The benefits of omni-channel access should always be weighted against the costs involved, counsels Lewis. Even so, today’s competitive pressures—the seemingly inexorable march of technological innovation and evolving user expectations—are likely to make platform expansion unavoidable for most online businesses. In the world of video gaming, as major franchises release new products across multiple platforms, and player preferences become more sophisticated, companies may simply have to adopt similar strategies to remain competitive. “As everyone else invests in the same new technologies, you almost have to do the same—just as a matter of doing business,” says Lewis. “If you are launching a video game, you’ve got to compete with whatever Call of Duty or Grand Theft Auto are doing. You can’t just tell your players they can only engage on one platform. The competition is continuously raising the stakes just in terms of the bare minimum.” Building Fandom: the Connective Cultural Tissue More broadly, Lewis and Jo’s findings speak to how human beings form communities of shared passion around business entities and, perhaps more compellingly, around cultural phenomena: video games, for sure, but also sports teams, music, films, comic books, fashion, and more. Understanding the mechanisms that drive and deepen engagement sheds more light on what Lewis calls the “connective cultural tissue of fandom: ”the powerful social bonds, camaraderie, and shared identity that connect people to cultural entities and to each other. Fandom, he argues, is the “key to our world.” Understanding fan behavior is critical to understanding how it is that games, brands, sporting teams, or politics forge communities built on shared passion. “Whatever your organization or business is, you are going to be interested in driving passion. You want people to engage and love what you do. What we’re looking at in this study is a building block towards understanding how cultural entities fit into consumers’ lives, and how eliminating barriers helps to expand communities and drive relationships—extending reach and engagement by weaving cultural experiences more deeply into everyday life.” The real challenge in front of organizations, be they video game producers or online retailers, says Lewis, is to give their product the kind of “cultural meaning” that creates fans—and not just users. “When you think about the behavior of fans, the level of passion and engagement that exists around cultural phenomena—whatever they are from video games to FIFA, the English Football League to the Super Bowl, Taylor Swift to the Republican Party—that’s where you see the passion that really drives the world. And that to me, is critical in understanding how business works, how societies function, and how our world evolves.”

View all posts
Powered by