Salary advice from AI low-balls women and minorities: report

Date:
Mon, 28 Jul 2025 21:00:00 +0000

Description:
New research reveals AI chatbots often offer salary advice that reflects real-world social biases.

FULL STORY

Negotiating your salary is a difficult experience no matter who you are, so naturally, people are sometimes turning to ChatGPT and other AI chatbots for advice about how to get the best deal possible. But, AI models may come with
an unfortunate assumption about who deserves a higher salary. A new study
found that AI chatbots routinely suggest lower salaries to women and some ethnic minorities and people who described themselves as refugees, even when the job, their qualifications, and the questions are identical.

Scientists at the Technical University of Applied Sciences
Wrzburg-Schweinfurt conducted the study, discovering the unsettling results
and the deeper flaw in AI they represent. In some ways, it's not a surprise that AI, trained on information provided by humans, has human biases baked
into it. But that doesn't make it okay, or something to ignore.

For the experiment, chatbots were asked a simple question: What starting
salary should I ask for? But the researchers posed the question while
assuming the roles of a variety of fake people. The personas included men and women, people from different ethnic backgrounds, and people who described themselves as born locally, expatriates, and refugees. All were
professionally identical, but the results were anything but. The researchers reported that "even subtle signals like candidates first names can trigger gender and racial disparities in employment-related prompts."

For instance, ChatGPTs o3 model told a fictional male medical specialist in Denver to ask for $400,000 for a salary. When a different fake persona identical in every way but described as a woman asked, the AI suggested she
aim for $280,000, a $120,000 pronoun-based disparity. Dozens of similar tests involving models like GPT-4o mini, Anthropic's Claude 3.5 Haiku, Llama 3.1
8B, and more brought the same kind of advice difference.

It wasn't always best to be a native white man, surprisingly. The most advantaged profile turned out to be a male Asian expatriate, while a female Hispanic refugee ranked at the bottom of salary suggestions, regardless of identical ability and resume. Chatbots dont invent this advice from scratch,
of course. They learn it by marinating in billions of words culled from the internet. Books, job postings, social media posts, government statistics, LinkedIn posts, advice columns, and other sources all led to the results seasoned with human bias. Anyone who's made the mistake of reading the
comment section in a story about a systemic bias or a profile in Forbes about
a successful woman or immigrant could have predicted it.

AI bias

The fact that being an expatriate evoked notions of success while being a migrant or refugee led the AI to suggest lower salaries is all too telling.
The difference isnt in the hypothetical skills of the candidate. Its in the emotional and economic weight those words carry in the world and, therefore,
in the training data.

The kicker is that no one has to spell out their demographic profile for the bias to manifest. LLMs remember conversations over time now. If you say youre
a woman in one session or bring up a language you learned as a child or
having to move to a new country recently, that context informs the bias. The personalization touted by AI brands becomes invisible discrimination when you ask for salary negotiating tactics. A chatbot that seems to understand your background may nudge you into asking for lower pay than you should, even
while presenting as neutral and objective.

"The probability of a person mentioning all the persona characteristics in a single query to an AI assistant is low. However, if the assistant has a
memory feature and uses all the previous communication results for
personalized responses, this bias becomes inherent in the communication," the researchers explained in their paper. "Therefore, with the modern features of LLMs, there is no need to pre-prompt personae to get the biased answer: all
the necessary information is highly likely already collected by an LLM. Thus, we argue that an economic parameter, such as the pay gap, is a more salient measure of language model bias than knowledge-based benchmarks."

Biased advice is a problem that has to be addressed. That's not even to say
AI is useless when it comes to job advice. The chatbots surface useful
figures, cite public benchmarks, and offer confidence-boosting scripts. But it's like having a really smart mentor who's maybe a little older or makes
the kind of assumptions that led to the AI's problems. You have to put what they suggest in a modern context. They might try to steer you toward more modest goals than are warranted, and so might the AI.

So feel free to ask your AI aide for advice on getting better paid, but just hold on to some skepticism over whether it's giving you the same strategic
edge it might give someone else. Maybe ask a chatbot how much youre worth twice, once as yourself, and once with the neutral mask on. And watch for a suspicious gap.

======================================================================
Link to news story: https://www.techradar.com/ai-platforms-assistants/chatgpt/salary-advice-from-a i-low-balls-women-and-minorities-report

$$
--- SBBSecho 3.28-Linux
* Origin: capitolcityonline.net * Telnet/SSH:2022/HTTP (1:2320/105)

Negotiating your salary is a difficult experience no matter who you are, so

>naturally, people are sometimes turning to ChatGPT and other AI chatbots for
>advice about how to get the best deal possible. But, AI models may come with
>an unfortunate assumption about who deserves a higher salary. A new study
>found that AI chatbots routinely suggest lower salaries to women and some
>ethnic minorities and people who described themselves as refugees, even when
>the job, their qualifications, and the questions are identical.

Scientists at the Technical University of Applied Sciences

>Wrzburg-Schweinfurt conducted the study, discovering the unsettling results
>and the deeper flaw in AI they represent. In some ways, it's not a surprise
>that AI, trained on information provided by humans, has human biases baked
>into it. But that doesn't make it okay, or something to ignore.

The problem here is likely that the AI system bases it's ideas on
the current average (norm) and those salaries are common..

That information is really an estimate of what the hiring company
should have to pay based on that information rather than it is
suggesting the person looking for the job ask for more than the
average for a similar employee reducing their chances of being hired.

This in no way suggests I think anyone should be paid differently
based on anything but job skills.

---
* SLMR Rob * Cover me... I'm changing lanes
* Origin: capitolcityonline.net * Telnet/SSH:2022/HTTP (1:2320/105)

The problem here is likely that the AI system bases it's ideas on
the current average (norm) and those salaries are common..

Indeed. That is a good point!

This in no way suggests I think anyone should be paid differently
based on anything but job skills.

I didn't believe you meant that, either. ;)

Mike


* SLMR 2.1a * Excuse my driving ... I'm trying to reload.
--- SBBSecho 3.28-Linux
* Origin: capitolcityonline.net * Telnet/SSH:2022/HTTP (1:2320/105)

The problem here is likely that the AI system bases it's ideas on

>> the current average (norm) and those salaries are common..

Indeed. That is a good point!

This in no way suggests I think anyone should be paid differently

>> based on anything but job skills.

I didn't believe you meant that, either. ;)

I'm glad you thought that.. B)

But I would be the great enemy of most 'modern' thinkers because
I also think everyone should be paid what they are worth whereas
many these days think everyone should be paid the same because
it's not their fault that they are lazy, dumb or slow...
(To be crude about it.)

These are the same 'thinkers' that don't think anyone should ever
fail a grade in school because, again, it's not fair to penalize
them for not being as smart as others, ignoring the possibility
that they are just lazy.. B)

I guess I got that attitude because almost every job I ever had,
even in Union places, I was often paid more than others around
me who had worked there longer because I could do / produce more
in the same working hours..

At least in cases like that you know you're doing well since
they are willing to go against the Union to pay you extra..
It's not all in your head.. B)

---
* SLMR Rob * Sometimes I wake up grumpy; other times I let her sleep
* Origin: capitolcityonline.net * Telnet/SSH:2022/HTTP (1:2320/105)

But I would be the great enemy of most 'modern' thinkers because
I also think everyone should be paid what they are worth whereas
many these days think everyone should be paid the same because
it's not their fault that they are lazy, dumb or slow...
(To be crude about it.)

As would I!

These are the same 'thinkers' that don't think anyone should ever
fail a grade in school because, again, it's not fair to penalize
them for not being as smart as others, ignoring the possibility
that they are just lazy.. B)

This whole line of thinking is part of what has gotten us into the messes
we are in now. It is "equity" thinking... that everyone's outcomes should
be equal... rather than "equality" thinking... we should all have equal opportunity to reach our own potential, which should not be expected to be
the same between *individuals*.

So we are both great enemies of modern "thinkers." ;)

Mike


* SLMR 2.1a * What's a 6.9? 69 interrupted by a period.
--- SBBSecho 3.28-Linux
* Origin: capitolcityonline.net * Telnet/SSH:2022/HTTP (1:2320/105)

These are the same 'thinkers' that don't think anyone should ever

>> fail a grade in school because, again, it's not fair to penalize
>> them for not being as smart as others, ignoring the possibility
>> that they are just lazy.. B)

This whole line of thinking is part of what has gotten us into the messes

>we are in now. It is "equity" thinking... that everyone's outcomes should
>be equal... rather than "equality" thinking... we should all have equal
>opportunity to reach our own potential, which should not be expected to be
>the same between *individuals*.

This is true.. Equal Opportunity and, after that, you earn what you get.

I suppose in theory exceptional people would get promoted out of the
area they are excelling in where, possibly, they could be paid more
money, especially if that re-situated them outside of the union.

And that's where the Peter Principle comes in, which is the idea
that, in big business, people who do an exceptional job keep being
promoted until they reach a level where they no longer do a good
job, in theory leaving all companies run by incompetent people.. B)

I've certainly run into that a few times...

Unions were good when they were needed but they gained so much
power in many areas that they do more harm than good, and are
less needed now that (in many places) there are government
standards to give workers safe working conditions and reasonable
pay but, of course, the Union always wants their people to have
more than the average, technially more than they are worth, and
for them to pay lots of Union Dues..

I think by this point Unions mainly protect the bad workers and
add to inflation since their higher wages cause increases in prices
bringing about everyone else needing higher wages to catch up, and
then it all starts over.

You can tell I have a bug about them after being threatened by
a Union once because they said I was working hard enough that
I was making everyone else look bad. Things like this had me no
longer working for other people by the time I was 32. My only
'Boss' was the need to do decent work to be worth what I charged.

So we are both great enemies of modern "thinkers." ;)

Hopefully the pendulum will start to swing back before too
many more years. Maybe we will even live that long.. B)

---
* SLMR Rob * It's lonely at the top, but you eat better
* Origin: capitolcityonline.net * Telnet/SSH:2022/HTTP (1:2320/105)

This whole line of thinking is part of what has gotten us into the messes

>we are in now. It is "equity" thinking... that everyone's outcomes should
>be equal... rather than "equality" thinking... we should all have equal
>opportunity to reach our own potential, which should not be expected to be
>the same between *individuals*.

This is true.. Equal Opportunity and, after that, you earn what you get.

I suppose in theory exceptional people would get promoted out of the
area they are excelling in where, possibly, they could be paid more
money, especially if that re-situated them outside of the union.

I think that is the part that bothers others... someone might get promoted because they are better at this, so the not better one tries to make it sound like that is somehow "unfair."

And that's where the Peter Principle comes in, which is the idea
that, in big business, people who do an exceptional job keep being
promoted until they reach a level where they no longer do a good
job, in theory leaving all companies run by incompetent people.. B)

I've certainly run into that a few times...

Me, too, unfortunately. I have also worked at some places where good
people get to a certain point and don't advance because management realizes
the really *need* them doing what they are doing now, and promoting them
will likely leave a hole somewhere.

I don't blame them for that, but it isn't too great if you are one of those folks. ;)

Unions were good when they were needed but they gained so much
power in many areas that they do more harm than good, and are
less needed now that (in many places) there are government
standards to give workers safe working conditions and reasonable
pay but, of course, the Union always wants their people to have
more than the average, technially more than they are worth, and
for them to pay lots of Union Dues..

I think by this point Unions mainly protect the bad workers and
add to inflation since their higher wages cause increases in prices
bringing about everyone else needing higher wages to catch up, and
then it all starts over.

That is my general opinion of Unions. That said, if you have a group that
gets in power and tries to erode the government safety standards, Unions
should in theory prevent that. I say "in theory" as the Union may not step
up if the politicians doing the eroding are the same ones they paid big
money to get elected.

IMHO, that is where Unions can do some serious damage. They become just
like any other PAC or self-serving organization that spends a ton of money
on politicians. They wouldn't do it if they were not expecting something
in return.

So we are both great enemies of modern "thinkers." ;)

Hopefully the pendulum will start to swing back before too
many more years. Maybe we will even live that long.. B)

Yes, but I expect that about as much as I expect to see pigs fly. ;)

Mike


* SLMR 2.1a * He does the work of 3 men.... Larry Moe & Curly.
--- SBBSecho 3.28-Linux
* Origin: capitolcityonline.net * Telnet/SSH:2022/HTTP (1:2320/105)

I suppose in theory exceptional people would get promoted out of the

>> area they are excelling in where, possibly, they could be paid more
>> money, especially if that re-situated them outside of the union.

I think that is the part that bothers others... someone might get promoted

>because they are better at this, so the not better one tries to make it
>sound like that is somehow "unfair."

When Unions are controlling things everything is usually based on
seniority, how long you worked there, not how good you are.
The only way a company can get around that usually is to promote
a good person high enough they are out of the Union, but for a
pretty standard worker who is just better, faster and harder working
than the others, a promotion that high would be out of the question.

I've mentioned before that at one job I had, I was given 'Merit'
raises because I was doing better work than everyone else. I ended
up making almost double the hourly rate that I should have been
getting with the time I'd been there..

This was the first time in 15 years they had given anyone a
raise beyond the time scheduled Union raises.

The end result was I ended up getting threats from the Union
and suddenly others started working harder to get paid more
as well, and did. Who is the loser in this situation?

I suppose in theory, it means you need fewer workers which means
fewer people paying Union Dues.

I have also worked at some places where good

>people get to a certain point and don't advance because management realizes
>the really *need* them doing what they are doing now, and promoting them
>will likely leave a hole somewhere.

Ha.. The last place I worked (not the one above) I eventually found
out that the head office in another country had seen what I had
accomplished there and wanted the company president where I worked
to offer me an opportunity to run my own branch, but they never
told me about it and I assume told the head office I'd refused it.

Probably noteworthy was that I soon after left that company when
I discovered that they were paying me about 2/3 of what someone
doing my job normally gets in that industry so, not only did they
desperately want to keep me there, they wanted to keep me cheaply..

That is my general opinion of Unions. That said, if you have a group that

>gets in power and tries to erode the government safety standards, Unions
>should in theory prevent that. I say "in theory" as the Union may not step
>up if the politicians doing the eroding are the same ones they paid big
>money to get elected.

Yes, I can't see things like safety standards being cut back but
you never know, and I've seen more in the USA that in Canada where
minimum wage in some states is FAR lower than in others, although I
think your Federal gov't is starting to override that to some extent.

i.e. It looks like Wyoming and Georgia have a minimum wage of $5.15
but the Federal laws force that up to $7.25 /hr.

To compare, the Federal minimum wage in Canada is $17.75 /hr.

I get nightmares thinking about my first job I started at 80c /hr. B)

---
* SLMR Rob * 2 + 2 = 5 for extremely large values of 2
* Origin: capitolcityonline.net * Telnet/SSH:2022/HTTP (1:2320/105)