came from men. The algorithm»learned« that being male correlated with success and began penalising re­sumes containing the word»women«. Amazon had to scrap the project. Fast forward to 2025 and we find the gender bias in AI-driven recruitment platforms remains a persistent problem, as algorithms trained on historical hiring data often replicate discriminatory patterns, prioritising male-coded language in résumés(Bhatia et al. 2024), ranking women lower for technical roles, or filtering out candidates based on gaps in employment that in fact reflect caregiving responsibilities rather than compe­tence. The European Institute for Gender Equality (EIGE) report(2021) on platform-work and AI in the la ­bour market finds gendered patterns of risk in which women are more likely to be found in less secure, ­lower-paid or platform-type work. To make this more tangible, think of training an algo­rithm as teaching a child. If you show the child only one kind of book – say, fairy tales in which only princes become heroes – the child will»learn« that only men can be brave. That’s how bias takes root: not through malicious intent, but through skewed learning. Next come data cleaning and labelling , the stage at which information is tidied and annotated so the algo­rithm can make sense of it. In facial recognition sys­tems, for example, developers label thousands of faces with identities. If the dataset includes mainly white fac­es, as was the case in early commercial systems, the model will struggle to recognise darker skin tones. ­This is why, as Joy Buolamwini’s MIT research(2024) showed, commercial facial recognition systems misi­dentified Black women up to 35 per cent of the time, while error rates for white men were below 1 per cent. A report from the EU Agency for Fundamental Rights (2023) reveals that people of African descent still face widespread discrimination across Europe, and warns that the absence of reliable race and ethnicity data is surreptitiously baking bias into the very algorithms and data-cleaning systems shaping everyday decisions. What is not measured, or is measured poorly, tends to be misrepresented or ignored by the model. Finally, the algorithm is trained , fed data until it can make reliable predictions. Developers test it using new data, refine it and deploy it in real-world systems. The output of the training is the model that will make, or heavily influence, concrete decisions about jobs, pay, promotion prospects or access to social services. A mod­el is a mathematical representation of patterns in data, and the choice of which model to use(for example, a de­cision tree, regression model or neural network) depends on the type of problem being solved, the nature and size of the available data, and the trade-offs between accura­cy, interpretability and computational complexity. 4 Friedrich-Ebert-Stiftung e.V. Each stage involves choices: what to optimise for(accu­racy, speed, profit), what to ignore(outliers, minority data), and when to stop training. These seemingly tech­nical decisions are, in fact, profoundly ethical and politi­cal. They determine whose errors are tolerated, whose interests are prioritised, and whose experience is treated as statistical noise. For workers, this is the first critical moment at which social conflict and inequality can be­come silently translated into code. To better understand where these risks emerge, it is use­ful to look more closely at how AI and algorithmic sys­tems are built. In practice, their development can be un­derstood in terms of a four-stage process : problem defi­nition, data preparation, model training, and validation and deployment. (i) Definition – Framing the Problem Every algorithm begins with a question: What are we trying to predict or optimise? For instance, Uber’s dis­patch and pricing algorithms optimise for efficiency, in the sense of getting a passenger to a driver as quickly as possible. But they do not necessarily optimise for equity, safety or driver well-being. That framing shapes everything that follows. The same is true in workplaces. A retailer might intro­duce a scheduling tool to minimise labour costs per hour, or an HR department might deploy a scoring mod­el to rank candidates by predicted performance. If the objective is defined purely in terms of speed or cost, the system will not»see« stability of income, work–life bal­ance or non-discrimination as goals worth protecting. Those values have to be deliberately built into the defi­nition of the problem. (ii) Data preparation – feeding the system Algorithms are data-hungry. They consume massive da­tasets, including photos, resumes, transaction records. But data is never neutral. Who collected it? When? With what tools? Under what conditions? And whose experi­ences were left out altogether? In small towns and villag­es, for example, limited infrastructure and unequal ac­cess to digital technologies mean that entire populations may be underrepresented or remain invisible in datasets built on city-centric sources. When such data is used for training, the resulting models inherit and amplify these blind spots. If training data for a welfare risk-scoring system or an employability algorithm over-represent certain neigh­bourhoods, age groups or migrant communities in»risk« categories, those associations can become self-reinforc­ing. Choices made during data cleaning and labelling – what is treated as an error, what is dropped as an»out-