Courts software putting Justice at risk
His Majesty's Courts and Tribunals service (HMCTS) is a pillar of the UK's justice system, handling legal cases involving decisions as consequential as:
Should children, facing possible harm, be removed from their parents and taken into care?
Should a person retain access to the benefits system?
Following a person's death, who are the rightful beneficiaries to their estate?
These decisions are informed by information held in case management software, the quality and reliability of which is therefore paramount.
Yet, amidst an £816m transformation programme, the software underpinning the above is demonstrably unreliable, with an established pattern of serious malfunction.
Thousands of cases have been affected by data loss, including documents from child protection cases and swathes of personal data from Probate, Benefit Appeals and other types of case.
In the best case such problems are noticed and manually remediated, in a painstaking process that can result in significant delays.
In the worst case they may result in inaccurate information being put before a Judge - raising the risk of miscarriages of justice stemming from the department's own IT systems.
2016
The HMCTS reform programme was announced
"...to make sure that our justice system continues to lead the world..."
The core of this modernisation drive was the digitisation of existing paper and courtroom legal processes to enable them to take place mostly or entirely online:
"We will provide online access by developing a single online system for starting and managing cases across the Crime, Civil, Family and Tribunal (CFT) jurisdictions"
This article focuses on the £816m CFT portion of the programme responsible for Civil Money Claims, Asylum applications, Benefit Appeals, Family law, Divorce, Probate and Employment Tribunals (The Crime portion of the programme has been covered elsewhere)
Case management
From a technical perspective the requirements of a case management application, such as used for managing Probate or Divorce cases, are very mundane.
Staff and citizens enter information on web pages. Other staff and citizens view it. Emails and letters are sent and received as cases progress towards a conclusion.
Demand is predictable; we know approximately how many people are doing to die and get divorced each year and the organisation will never need to scale to hundreds of millions of users.
This is not to trivialise the difficulty of doing this well. Legal processes can involve lots of edge case complexity and successful solutions must model these processes faithfully whilst remaining usable, accessible and effective for all concerned.
Yet despite the technical simplicity, what has been delivered by the CFT programme is drastically reduced in scope and plagued by chronic usability, accessibility, performance and reliability problems.
Led by technology
Instead of technology serving the organisation, the reverse seems to be happening.
The programme operates by making technical decisions about how everything must be built, then proceeds to fit the organisation's requirements around these decisions.
Regardless of user need, staff user interfaces must be built using an in-house (Excel based!) UI ‘microservice’ (and will, therefore, be inadequate, unusable, untestable and in breach of legal accessibility requirements).
Regardless of business need, cases must be persisted in an in-house, shared JSON blob 'database microservice' (lacking any DML, integrity constraints and at high risk of data corruption).
Regardless of practicality architectural designs must be adhered to (so designing a form on a webpage involves a distributed system of 20+ ‘microservices’)
Technical design comes first and all other considerations are secondary.
And so it is that service design, UX, accessibility, performance, maintainability, productivity, automated testing - up to and including reliability as a system of record - are heavily compromised or abandoned entirely.
"The reform programme... was driven by the need to address a proliferation of 'disparate', 'complex and inefficient' case management systems'“
Yet it is striking how closely the programme has replicated the very problems it was created to address, which are clearly a symptom of a deeper organisational dysfunction.
Waterfall
At HMCTS waterfall is more than just a methodology, it is the founding principle around which the organisation is structured.
Architects translate requirements into intricately detailed designs and hand them over to developers to implement.
Developers implement these designs and hand them over to others to be tested, before the the whole lot is 'delivered' to yet another team and becomes someone else's problem.
A consistent theme is the misalignment between those who take a given set of decisions and those who will be responsible for the consequences.
Architects can create extraordinarily complex designs, but others will be responsible for building, testing, integrating and maintaining them.
Developers can rack up mountains of technical debt, but it will be another team picking up the bill further down the road.
Virtually everybody faces an array of people and processes holding a veto over their work whilst having no responsibility for getting it done.
There are many dedicated staff at HMCTS but, amid such a structure, it is close to impossible for any individual to meaningfully influence outcomes.
Legacy
That a case management system could lose information at scale for several years, without adequate diagnosis and remediation, is illustrative of the technical legacy the programme leaves.
Having deployed a technology budget a large multiple of that required, the excess has gone into recreating poor replicas of existing open source software - and the opportunity costs of using these extraordinarily ineffective in-house tools.
By failing to consider testability bugs are likely to get through, and by neglecting data integrity they are likely to have a dangerous impact when they do.
Much of this risk averse organisation’s activity now sits on technical foundations so flawed that a small change to the ‘ui service’ risks introducing an immediate data corruption bug across any or all of their case management applications.
Reform
The programme's headline premise of creating a 'single online system' set its focus on creating technology as an end in itself.
More appropriate would be to develop the organisation's capability to apply technology effectively, rather than viewing it as a capital asset to be designed, built and left in a static state.
If the capability and culture are there then the technology will follow, but turning things around requires experienced technical leadership with a mandate that extends to reshaping the structure of the organisation itself - including procurement and HR.
"Our times – with the advent of the internet and an explosion in new technology – provide the opportunity for radical change” [2016 reform programme announcement]
The internet was decades old when this statement was made - why had we not seen this radical change already?