AI should not decide who gets a social grant: lessons from a South African court case

South Africa's High Court has ruled that the automated system used to vet applicants for the Social Relief of Distress grant was unconstitutional, after digital-only checks helped cut the number of recipients from 16 million to 5.6 million between 2020 and 2022. The court found the process, which relied on automated bank and database checks, breached the rights to equality and social security, partly because it drew on outdated and error-ridden data. Government has appealed to the Supreme Court of Appeal. The outcome will matter well beyond welfare, as departments including Home Affairs and the police expand their own use of automated decision-making systems..By Mark Richard Gaffley*.South Africa introduced a digital social assistance programme in 2020 to identify who was eligible for its Social Relief of Distress grant. The country’s High Court later found that the automated vetting system it relied on was unconstitutional and invalid.The case was brought by the Institute for Economic Justice, a South African think tank, and #PayTheGrants, a civil society movement that advocates for economic justice. The government appealed the decision and the case will be heard by the Supreme Court of Appeal. Legal scholar Mark Gaffley explains the significance of the ruling.How does the system work and why’s there a court case about it?The South African Social Security Agency launched the Social Relief of Distress grant in May 2020. It was intended to provide temporary relief for the most destitute – that is almost one-third of South Africans.To qualify for the R370-a-month (US$23) grant (almost 50% less than the food poverty line), recipients had to receive no income or any other grant. Government estimated that the target group was approximately 18.3 million people.Though the grant was digital-first from the start, new regulations passed in April 2022 shifted its administration from under the Disaster Management Act to the Social Assistance Act. These regulations limited applications to online-only and in many cases treated verdicts given by automated decision-making systems as definitive. This stringent application process made the grant the first large-scale fully digital social grant programme of its kind in the country. It worked by verifying applicants’ income through automated bank account checks and government database records.The updated digital application process led to a drastic reduction of successful applicants. At inception 16 million people benefited from the grant. By 2022 this had fallen to 5.6 million. Because of this, the government was also able to cut the grant budget by billions of rands.In July 2023, the Institute for Economic Justice initiated court proceedings to contest the grant process. They argued that the steps taken by government to digitise the grant process did not adhere to obligations under the country’s constitution. The constitution allows for the progressive realisation of access to social assistance – meaning that socioeconomic rights cannot be realised immediately but over time.The institute argued that the regulations and the application procedure resulted in the irrational, arbitrary and unfair denial of social assistance to millions of people legally entitled to receive it.In January 2025, the High Court agreed. It declared certain of the grant’s regulations unconstitutional and invalid. This included those regulations authorising automated bank verification and database checks.The government appealed the ruling and the matter is set for hearing in the country’s Supreme Court of Appeal.What are automated decision making systems and how are they related to AI?Automated digital decision-making processes rely on algorithmic reasoning or machine learning to come to conclusions, based on the datasets made available to them. Historically these decisions would have been made by humans.The decision by government to use these automated digital processes raises broader concerns, such as making decision processes opaque. In addition, for unsuccessful applicants, there were limited avenues for redress or challenge as the appeal process was also online-only.A large number of artificial intelligence (AI) systems that are relied on for decision-making are likely to incorporate automated digital decision-making processes. This means that the risks identified by this case could materialise in a wide range of scenarios involving the use of AI.The case highlighted that automation was done in such a way that the process excluded the people it was intended to support.It laid bare the discriminatory and exclusionary effects of deploying automated processes in the administration of public sector services.The case stresses the need for greater awareness of the discriminatory effects of relying on automated digital systems. Discriminatory treatment may become more widespread as the uptake of AI and digital automation systems increases. Digital systems are also often seen as neutral, but they can reinforce the disadvantages that vulnerable people already experience.Why is this such an important issue?The case has surfaced three significant AI-related issues.First, it carries implications for the constitutionality protected rights of millions of South Africans whose livelihoods were subjected to the algorithmic whims of automated processes making eligibility determinations.The High Court found that the online-only application process affected:the right to equalitythe right to social securityinterpretation of the bill of rights – the part of the constitution that contains the fundamental rights of all people in the country the state is required to uphold.Second, the case calls for immense caution when using automated digital decision-making processes. They are not inherently unlawful. But if decisions rely on their reasoning, they should be monitored to ensure adherence to constitutional requirements.For example, one criterion for determining if an applicant received an income was whether they had received any bank deposit. These deposits could have been temporary loans, or money received on behalf of someone else. This shouldn’t have been perceived as regular income.Third, the databases the automated systems relied on were error-ridden and outdated. This likely had the effect of indicating employment when this was not the case.Automated digital systems can only be as reliable as the data they rely on. With the grant process, this meant that they may have excluded millions of unemployed people entitled to receive the grant.If the government’s appeal is granted, it may establish legal precedent that, unless taken to the Constitutional Court, prioritises digital efficiencies over constitutional safeguards.In what circumstances can AI tools like this be used for these kinds of purposes?The short answer to this question may be “how long is a piece of string?”.Already, in the public sector, the Department of Education, Department of Home Affairs and the South African Police Service are all at various stages of integrating automated digital processes into their operations..Read more:.Social grants no longer guarantee ANC votes in 2024 election.Careful scrutiny is needed as to whether the use of the automated systems threatens human rights and public interest. It is not too far-fetched to imagine a scenario where an education technology solution starts excluding certain learners because their responses are too abstract to be reconciled with its training data.These same risks apply to the private sector where company operations may be more shielded from public scrutiny. This danger has already become apparent in the plethora of AI-enabled CCTV surveillance cameras operating in the country..*Mark Gaffley is the Director of Legal & Operations at the Global Center on AI Governance, which has been admitted as amicus curiae in the matter being heard at the Supreme Court of Appeal..This article was first published by The Conversation and is republished by permission..Sign up for your early morning brew of the BizNews Insider to keep you up to speed with the content that matters. The newsletter will land in your inbox every morning on weekdays. Register here. Support South Africa’s bastion of independent journalism, offering balanced insights on investments, business, and the political economy, by joining BizNews Premium. Register here.If you prefer WhatsApp for updates, sign up to the BizNews channel here.

AI should not decide who gets a social grant: lessons from a South African court case

Related Stories

DA leader Mark Burke's former company Kastelo targeted by SARB on crypto deals

Bheki Mahlobo: SA economy's dirty picture - Mandela's warning, ignored

Kallie Kriel: AfriForum’s legal eagles to swoop on “Land Grab” Malema

TCS editors: How to tell when you're being fed a narrative by the media