Whose responsibility is to to make sure that workers are paid their holiday pay? Not government it would seem, nor employers, nor payroll providers. Since a 2010 law change estimates indicate that more than 700,000 New Zealanders who work shifts or irregular hours may have been underpaid by more than $2bn. Sometimes the hours and rates were just set and forgotten. But often the algorithms encoded into the payroll software were not fit for purpose.
What’s wrong with that?
An expensive exercise is now being undertaken to make belated repayments to make good some of the losses. Some underpayments are as high as $1500. Someone in government could have provided the code to describe what happened in payroll systems to pay holiday pay correctly. Payroll providers could have sought information or written code to do the job properly. Employers could have sought clarification. But we have begun to trust the code written into tech systems. In the old days IT systems provided a model of reality; now the systems are reality. There are reports from the system – salary and holiday reports capture a moment in time – but there is no shadow paper version of a payroll system or a book with the rules encoded in its operation. For years this unwillingness to capture the intent of the legislation in payroll systems has been overlooked but more importantly the preference for light handed regulation has meant that there is no appetite in government to hold service deliverers to account and to see that those with responsibility to ensure technology is working properly actually does what is promised. The impacts of this go well beyond payroll.
Is there a larger problem?
Increasingly we are getting services and goods and access based on algorithms. If we don’t see the rules encoded into how the algorithms work we are depending on a black box. This is exactly what happened with the payroll problem.
Why this is important?
The instances of use of algorithms to provide goods and services (and pay) exists already and can only grow. The following are examples:
The government’s social investment approach is predicated on identifying people at risk because of their data profile from data sources across agencies. Essentially they are selected for special attention for “wrap around services” by algorithm irrespective of whether they want them or need them. The flag referendum was the first time STV voting has been used in central government at an election time. The STV counting algorithm has been encoded into software which counts the votes but who has seen and checked the algorithm. Pharmac is looking at algorithms and profiling to optimise the delivery of medicines.
In NZ deputy prime Minister Bill English has said that the use of data to deliver public services represents a ‘renewal of the social licence’ and the government has commissioned recent research on this issue. It’s a bit late though. As Val Morse’s presentation to Information, Ethics and the Public Good last year showed the personal data horse has already bolted. It’s going to be hard to retrofit good ethics onto a system whose values are predicated on getting people off benefits, delivering more and more through privatisation, cutting the cost of delivering public services let alone using data from across government to second guess who actually needs social support.
In the UK the anti-radicalisation strategy uses a check-list of psychology research to categorise young people as a terrorist risk. This is an approach which has been heavily criticised by 140 academics and the programme has resulted in 4000 people being assessed – some as young as nine years old. The details of the coding are secret.
In the private sector Pro-Publica’s
Who is checking the quality of the code on these initiatives? They could easily be as problematic as the holiday pay issue. In some instances the use of the algorithms relies on extensive use of information about us to determine whether we fit the criteria. Both the use of data and the resulting uncertainties in the algorithms have been the subject of well informed criticism.
Where might a solution lie ?
This is a growing area of democratic and citizenship deficit. Without the opportunity to open the code our interactions with and services from public and private agencies we have no idea whether what we are getting is correct or fair or why we have been chosen or excluded by the criteria.
Algorithms used for making decisions on the provision or with-holding of services, and the data models and data sources they use should be open for inspection. This should happen irrespective of the service delivery origin and whether or not we pay for it or whether it is delivered through public or private sectors.
Possible approaches
Code should be lodged with a government agency and a community agency fit for purpose – possibly fyi.org.nz, Internet NZ , the Open Source Society or Consumer NZ. The community agency should be funded to carry out this role. Code should be able to be run in a pilot system to demonstrate outcomes.
Radio New Zealand Holiday pay errors could top $2b
Werewolf 09/2015 Big data big problems
The Guardian Academics criticise prevent anti-radicalisation strategy
Pro-publica Breaking the black box what facebook knows about you?
Valerie Morse What if? (video 19 minutes)
Data Futures. The sharing of personal health data
Latest Comments