1        Evaluation of judicial systems and accountability

1. Civil justice systems may be designed in any number of ways. Fair trial guarantees through providing for minimum standards, sufficient to secure effective and practical access to courts and judgment, do not require each state to ensure they have the same court structures, legal representatives and forms of representation, and court procedures. As the European Court of Human Rights put it in respect of the fair trial guarantee under Article 6(1) of the European Convention on Human Rights, and the point is applicable to other such fair trial guarantees, ‘is for […] States to organize their legal systems in such a way that their courts can guarantee to everyone the right to a final decision within a reasonable time […]’^[1]. The extent to which any specific state’s civil justice system met such standards was historically something that was not scrutinised systematically. With a greater focus on the need to implement fair trial guarantees, not least due to the supra-national oversight of bodies such as the European Court of Human Rights and the Inter-American Court of Human Rights as well as bodies such as the Council of Europe’s Venice Commission, structured evaluation of civil justice systems has developed during the 21st century.
2. Evaluation has not, however, only focused on the assessment of whether fair trial guarantees have or are being met. They have also served wider purposes. Linked to the assessment of fair trial guarantees, evaluation has also formed part of procedural reform programmes. It has, for instance, formed an important part of the assessment of reform’s necessity and implementation. It has also, looking wider still, formed part of multi-national efforts by the United Nations and World Bank to improve the operation of civil justice systems as part of efforts to improve the rule of law and, through that, economic development. Underpinning each of these developments it is possible to see the promotion of various aspects of system accountability. Evaluation, particularly when it is ongoing, helps maintain public, democratic accountability of justice systems, for instance by scrutinising their operation, timeliness, and cost-effectiveness, and the utility and success of system-reform. It is a form of international accountability, not least where it forms part of multi-national initiatives to improve the rule of law. It also forms part of explanatory accountability by the system itself, where self-evaluation is made public, eg, it demonstrates to the public how the system operates. The starting point is to outline the three broad types of evaluation study that can be identified: national; regional; and multi-national.
3. National studies take any of three different forms: ones aimed at ongoing evaluation of the system; ones aimed at promoting reform; and ones that scrutinised implementation of reform. The most routine form of evaluation is ongoing evaluation. It has a long provenance. Bentham in his critique of 19th century procedure argued that civil justice systems should be fully open to inspection. This not only meant their being fully compliant with the principle of publicity, but that all information concerning their operation should be available for inspection, scrutiny and evaluation. In order to reduce litigation costs and delay, civil justice needed to be subject to ‘continual testing’^[2]. As Draper summarises his approach, which is developed in Bentham’s work on Scotch Reform,

[…] Bentham’s method consisted from top to bottom in the empirical gathering of information, primarily in tables, covering all relevant details from the number of cases embarked upon, to time taken in hearings and judgements, and the degree of movement of cases between courts in cases of appeal. Statistical comparisons between system and system, court and court and ultimately, between judge and judge, would provide the basis for a thoroughly empirical analysis and critique of the procedures developed.^[3]

4. Such openness would serve two purposes. It would enable individual litigants to judge whether to litigate or continue litigation. They would thus be able to determine, for Bentham, whether proceeding to litigate would be likely to maximise their overall utility, consistently with the principle of utility that underpinned his approach to civil procedure^[4], amongst other things. It would also enable society more generally to determine if civil justice was capable of and was maximising utility for the population as a whole. Removed from his utilitarianism, the continual testing by the public and policy-makers would ensure that steps could be taken to readily identify and rectify problems within the system. Continual evaluation was the basis on which continual improvement and implementation of best practice could be applied.
5. Bentham’s approach is not one that has been implemented generally. Some systems engage in a degree of openness, which permits evaluation. Annual court reports are issued in, for instance, the United Kingdom.^[5] They are issued voluntarily. Similar reports are also published in, for instance, Germany^[6], New Zealand,^[7] Singapore^[8], and South Africa^[9]. By way of contrast, reports on the operation of courts in the United States of America^[10], are published annually further to a statutory requirement to do so.^[11] These reports typically contain information, including tabular information consistent with Bentham’s suggestions, concerning case numbers, case flow, time or delay to resolution, albeit not to the degree of detail that he proposed. They also contain information on the number of justices, registry and court administration numbers and costs.
6. The availability of annually published information ought to provide a foundation for effective, continuous assessment of the operation of civil justice systems. Rarely does it appear to do so effectively. All too often they have the appearance of being simply reportage. On the contrary, many systems that publish such information base system-reform on more detailed, specially-commissioned evaluation or reform reports. This is particularly prevalent in common law jurisdictions, where such reports form a regular feature of their justice systems. They are so regular that it could properly be said that where their recommendations are implemented they rarely achieve their goals; those generally being to reduce procedural inefficiency, cost and delay. Nowhere is this starker than England and Wales which has engaged in such reform on average once a decade since the middle of the 18th century.^[12] Similar approaches can be seen, for instance, in Australia,^[13] Canada,^[14] Ireland,^[15] Japan,^[16] New Zealand,^[17] Hong Kong,^[18] and Singapore^[19]. These types of studies vary in the extent to which they draw on empirical evidence. The significant methodology approach adopted is to canvas views, via consultation papers, from court users, ie, professional court users. Information is not generally obtained from litigants themselves or the wider public. They do not, therefore adopt the methodology common, to independently commissioned or carried out legal needs surveys.^[20]  Where empirical data is obtained, again, there is a focus on it being drawn from the legal profession. To a certain degree therefore it is self-selecting, with biases creeping into the studies depending on who responds. Results may, for instance, be skewed where one particular constituency of professional users and repeat litigants responds disproportionality to others.^[21]
7. An example of evaluation by way of implementation report is that carried out in Hong Kong. During the first decade of the 21st century, Hong Kong’s civil justice system was subject to detailed scrutiny, which resulted in the introduction of reforms to its civil procedure rules in 2009.^[22] As with other reforms, the intention behind them was to ‘improve access to justice at reasonable cost and speed’.^[23] A central weakness of procedural reform, specifically its implementation, has historically been a lack of scrutiny of the reform process. Not only has procedural reform typically been carried out without detailed scrutiny of the causes of the procedure’s problems, as is typified by the reform process in England and Wales,^[24] but there is no evaluation of the reform’s implementation. Reforms in Hong Kong mark a significant departure from this tradition. A reform monitoring committee was established to scrutinise the effect the 2009 reforms had. In particular, the committee was required to assess and publish details of its evidence-base and assessment, of the effect the reforms had on the cost and speed of litigation.^[25] Subsequent to the Hong Kong study monitoring, approaches have expanded further back in the reform process. In England and Wales, for instance, reforms were tested via pilot studies before they were either introduced or abandoned prior to implementation. An example of the former were the pilot studies carried out to test the introduction of, for instance, a concurrent expert evidence process^[26], while the latter is exemplified by an examination of the introduction of a continuous digital procedure, to replace discrete, concentrated hearings, within specialist civil tribunals^[27].
8. The benefits of the two approaches can readily be summarised. Evaluating potential reforms enables an assessment of whether the proposed reform will achieve its stated objective and also, crucially, whether it will create unintended adverse consequences. Medium to long-term post-implementation evaluation should, as was intended to be the case in Hong Kong, enable those administrating the civil justice system to determine if, and if so the extent to which, reform achieves its aims. Perhaps more importantly, medium to long-term monitoring and evaluation of reform can form the basis of ongoing evaluation of the operation of the justice system. Such an approach can bring to light where new problems arise, ie, it can help to identify new causes, or the re-emergence of old causes, of procedural cost and delay. Taken together these two forms of national evaluation can help promote effective access to justice.  
9. Moving beyond national forms of evaluation there are regional ones. The two leading examples of this can be observed in Europe and South America. Both the European Commission for the Efficiency of Justice (CEPEJ)^[28], which was established by the Council of Europe, and the European Union through its ‘Justice Scoreboard’^[29] provide European regional analyses. Corporación Latinobarómetro, which is based in Chile, provides a comparable South American regional analysis through its annual surveys.^[30] Of the three, CEPEJ provides the most detailed approach. Its most recent report from 2020 drew on data submitted for 45 countries, including those outside the Council of Europe, eg, Israel and Morocco. It also obtained data for all four legal jurisdictions within the United Kingdom, thus giving it data from 48 jurisdictions.^[31] It draws on data submitted by national correspondents from each jurisdiction, which it then scrutinises and verifies through ongoing communication with those correspondents. All its data, going back to 2010, is available online, enabling detailed longitudinal analysis.^[32] It provides a range of qualitative data, such as what training judges have undergone and the nature of such training, eg, ethics training, and case management training. Quantitative data, eg, the number of judges, whether they are lay or professional, within the civil justice, are also collected. Data on court budgets, fees, access to legal aid, court use of IT, court organisation, and the use and results of national court satisfaction surveys are collected and analysed. Perhaps most importantly, court efficiency and quality data are scrutinised. The EU’s Justice Scoreboard to a broad degree replicates the approach taken by CEPEJ. It draws on the CEPEJ’s data, while also drawing on a wide range of additional sources, such as EU member state judiciaries, EU and national bar associations and various EU-related networks.^[33] The main difference between it and CEPEJ, other than the range of countries it considers, is that it focuses on three areas: Efficiency in justice systems; quality in justice systems; and, judicial independence.^[34]
10. While national and regional studies have a number of features in common, not least their focus on the efficiency and quality of civil justice systems, multi-national studies tend to have a wider range of foci depending on which body is responsible for issuing them. There is the World Justice Project’s Rule of Law Index. Its reports draw on information from 139 countries. Information is obtained from individual surveys and from lawyers and experts. Its focus is to provide an analysis of the perceived adherence to the rule of law by those countries it surveys. It thus ranges much wider in its analysis than national and regional studies. Within its analysis, it provides an assessment of the operation of each national civil justice surveyed. It does so by reference to seven metrics, which mirror the three metrics used by the EU in its Justice Scoreboard. It assesses justice systems by reference to: accessibility and affordability; the extent to which they are free from discrimination; the extent to which they are free from corruption; the extent to which they are free from improper governmental influence; the extent to which they are free from unreasonable delay; the extent to which they have effective enforcement; and, the extent to which they have impartial and effective ADR processes.^[35] Secondly, there are the World Bank’s Doing Business reports.^[36] These also focus on the rule of law, although they do so from the perspective of the ease of doing business in those countries it surveys. One factor that underpins ease of doing business, and hence is a key element of the development of strong economies, is the quality of the civil justice system and particularly ease of access to it and its effectiveness. The effectiveness of such systems is the basis on which contracts can be enforced, thus it is strongly conducive to a healthy economy. The extent to which contracts can be enforced, and thus through that markets to work effectively and property rights and their transfer can be facilitated, equally depends upon adherence to the rule of law. Hence the World Bank’s reports complement the World Justice Project’s Rule of Law Index, albeit they do so from an economic and business perspective.
11. The three forms of evaluation, national, regional and multi-national can be considered from a number of different perspectives: aim; methodology; audience; utility. By looking through these differing lenses it is possible to identify a number of commonalities across them, not least their focus on efficiency as a key performance indicator for the effectiveness of civil justice systems.
12. Starting though with aims, it is possible to identify a clear commonality across all the studies. Each broadly construed is focused on improving access to justice and through that the rule of law. This is explicitly the intention of the multi-national studies. Their evaluative method focuses directly on the issue of whether, and if so to what extent, civil justice systems promote the rule of law through providing accessible civil justice systems. The regional and national studies take a more oblique approach. CEPEJ, for instance, by focusing on issues such as timeliness to disposition, i.e., the length of time a dispute takes to be resolved by a court from the point in time when proceedings commence, assesses accessibility in terms of the adverse effect of delay. A timely system is one that is able to deliver justice, and enable the enforcement of rights, at a time when a judgment is most effective. The approach taken to this by both the multi-national and regional in this regard is intended to operate on two levels.
13. First, and in this, they share a common approach with national studies, they are intended to highlight the success or otherwise of national systems to those national systems. They make available data and analyses that highlight the features of those national civil justice systems and the results, at a macro level, that they produce for litigants. As such they form a body of evidence that national court administrators and policy-makers can rely upon to consider where defects arise in their system and what reforms are needed. They thus provide a form of ongoing evaluation for justice systems. From a national perspective, this can play an important role, particularly where there is no nationally-based form of ongoing evaluation. Such studies thus can fill national information deficits. Secondly, they can also provide a form of soft power incentive to national systems to reform. They do this by providing comparative studies. CEPEJ does so across, as noted above, 48 jurisdictions. The EU over its member states. The World Justice Project and World Bank do so for 139 countries and 190 economies respectively. By providing a comparative analysis, national systems can be incentivised to improve the operation and accessibility of the civil justice systems. Given the importance of a strong commitment to the rule of law as a basis for developing economic growth, a point implicit in the UN’s Sustainable Development Goals,^[37] is that the multi-national studies emphasize both the commitment to the rule of law and the relationship between an effective civil justice system and economic growth they provide a persuasive basis for countries to carry out access to justice-based reforms.
14. Focusing on timeliness to disposition highlights the importance that efficiency plays in system evaluation. The Australian Productivity Commission, for instance, focused on the need to improve systemic efficiency in its examination of the operation of the Australian Federal Justice System. Efficiency, as a means to deliver justice, is equally a focus of the regional and national studies. Care needs to be taken here, however. Efficient at its most basic focuses simply on the speed with which proceedings move from commencement to judgment. As Rijavec rightly noted, ‘Efficiency is an important part of any system of adjudication; it is certainly not (however) the only relevant value.’^[38] Efficiency ought properly to be considered an important value guiding the operation and assessment of civil justice systems, but it is not an independent one. It is value that is a means to an end and not an end in itself. Efficiency is needed to ensure that the time taken to determine a dispute does not lead to a mis-decision, ie, delay-induced adjudicative error. Nor must delay caused by inefficiency undermine effective enforcement. There is a danger, particularly in national forms of evaluation, that the focus on efficiency may be one that sees it as an end in itself. It may also, where it does not do so, see it as a means to an end other than the delivery of justice. Efficiency is often linked with cost-effectiveness. A focus on improving efficiency could potentially become a focus on cost-reduction, which if not properly viewed within the context of increasing access to justice, enhancing adherence to the rule of law or enhancing the environment for economic development, could undermine the ability of civil justice systems to deliver justice. It is thus important for all forms of evaluation to ensure that evaluation is not simply quantitative where efficiency is concerned, but rather takes account of a wide range of assessment criteria, including the ability of justice systems to secure fair processes, accurate results and effective enforcement.
15. A further consideration needs to be noted where efficiency is concerned. Inefficiency is arguably also a benefit to civil justice systems. This may seem counter-intuitive or simply wrong. However, when considered by reference to, on the one hand, the need to secure adjudicative accuracy and, on the other hand, consensual settlement, its relevance and importance becomes apparent. In so far as the former is concerned, it is often the case that time is needed to ensure that sufficient evidence is obtained by the parties and the court to enable them to effectively scrutinise and challenge it. Time is also needed to marshal witnesses, both lay and expert and to research and develop legal arguments. There is, in all systems, a need for a certain amount of ‘delay’, which could be construed when looked at out-of-context as inefficiency. Too simple and straightforward a focus on efficiency may have an adverse effect on such necessary delay, which would then undermine the civil justice system’s ability to do justice. In so far as the latter is concerned, a certain amount of delay in disposition may be necessary to promote consensual settlement. A certain amount of delay caused by some forms of procedure may also help promote settlement. In this regard, it is noteworthy that the EU Justice Scoreboard considers the effectiveness of ADR systems.
16. What is more typical, and this can be seen from reform evaluations such as the Jackson Costs Review from 2009 in England and Wales, is that there is a focus on reducing delay by streamlining aspects of procedure without any consideration of the role that the time and effort taken further to those processes may be conducive to settlement. That report, for instance, looked at how to reduce the cost and time related to the evidential disclosure process.^[39] It sought to reduce both. No consideration was given, however, to the effect that those processes had, in the form they then had, on the promotion of settlement. The focus was on the reduction of disclosure in order to reduce its cost and delay. Detailed evidential disclosure can, however, help litigants properly assess the merits of their and their opponent’s cases. It is part of the tapestry of settlement consideration. Reducing the amount of evidence disclosed reduces their ability to assess their prospects and thus could lead to a reduction in the efficacy of negotiation, mediation or early neutral evaluation. Too much focus on efficiency, or the reduction in cost, without a clear focus on other process values (such as settlement in this context but equally procedural equality, the ability to participate effectively, and the ability to secure quality in decision-making) may lead civil justice reformers to take steps that undermine these values in the pursuit of a narrowly-defined idea of efficiency.
17. The focus of the three forms of evaluation highlights another broad issue: engagement with civil society. The main target of the various forms of evaluation is specialists. While the evaluation reports are widely and publicly available – not least now through publication on the Internet – they are aimed at clearly defined audiences. National reform and implementation reports are self-evidently targeted at policy-makers, government and the courts. They may be based on consultation, from which their data and assessments may, partially at least, be derived, but even they are implicitly targeted at specialist audiences. Legal professionals, bar associations, bulk court users (insurance associations, for instance), specialist justice reform bodies form the majority of respondents. Members of the public and wider civil society organisations generally either do not respond or form a small minority of respondents. Where an assessment of the operation of civil justice systems is concerned this poses a problem. Reform-focused evaluation is, as noted above, generally concerned with improving access to justice. Yet, while it considers operational and performance data and the views, i.e., qualitative data, from professional court users, this form of evaluation tends to fall short on in obtaining and analysing data from members of the public.^[40] Consequently, qualitative assessments are likely to fail to properly reflect the needs of such users.
18. More broadly, and this applies to all three forms of evaluation report, they focus on the operation of the civil justice system from a court-based perspective. Their focus, whether it be access to justice, the rule of law, or effective support for business and economic growth, is one that looks to access to and the operation of the courts and judgment. This means that they fail to address wider issues. Most importantly, they fail to address, as Eisenberg noted, ‘the primary events in daily life that generate civil justice issues’.^[41] Questions of efficiency might, for instance, be considered more effectively if systemic efficiency of the courts was considered by reference to the reserve army of disputes that never reach the courts due to individual citizens failing to approach the courts. Questions of access to justice might be answered differently if more evidence were obtained on the development and formation of disputes and why so few are transformed into legal proceedings.^[42] Questions of the rule of law might also be answered differently if evaluation also considered, for instance, systemic social exclusion from civil justice systems.^[43] In each of these three cases, by adopting a narrow scope of study, evaluation reports may fail to identify significant social issues that have an impact on the operation, design and delivery of civil justice systems. The methodologies adopted by the three forms of evaluation study do not take proper account of these wider issues. Their results may therefore present a false picture, eg, of the level of compliance with the rule of law by not taking account of levels of self-help or the impact of de facto exclusion from access to justice. Their results may also present a false basis on which the necessity and nature of reform is evaluated. Evaluation, while therefore a positive development, needs to go further to be properly effective. It should, at the least, incorporate effective legal need studies into its programme of assessment, so that greater and more effective public and civic engagement can be incorporated.^[44]
19. Finally, the prospect exists that national and regional evaluation studies may, in the near future, be able to take advantage of the general trend in digitisation of court processes. Where, for instance, digitisation results in the transfer of proceedings and their management to digital court issue and case tracking and management systems, the possibility will exist for national courts to extract a greater range of data from cases than in the past. Such data will also be more easily extracted, and where necessary subject to anonymisation, to enable court’s operation and accessibility to be subject to greater scrutiny and evaluation than is currently the case. It may, for instance, be possible to identify with greater precision the timeliness of proceedings, as well as of all proceedings across all courts. Litigant characteristics could be tracked to feed into analyses of access to justice. It may, for instance, be possible to ascertain whether different groups within society secure differential access, whether their case outcomes differ from other litigant groups, and whether therefore changes in procedure need to be adopted to secure fair and effective access across society. The potential ease with which such data could be made available could also help promote greater transparency in the ongoing operation of civil courts, thus making ongoing evaluation more effective and potentially the norm in all national systems. This in turn could promote greater and more detailed regional and multi-national evaluation. Each of these developments could help enhance the accountability of civil justice systems, while forming a more effective basis for their reform consistently with a range of fair trial values.

Abbreviations and Acronyms

ACCP	Code of Civil Procedure (Argentina)
ACHPR	African Court on Human and Peoples’ Rights
ADR	Alternative Dispute Resolution
ALI	American Law Institute
ANCCPC	Argentine National Civil and Commercial Procedural Code (Argentina)
Art	Article/Articles
ATCCP	Code of Civil Procedure (Austria)
BGH	Bundesgerichtshof (Federal Court of Justice) [Germany]
BID	Banco Interamericano de Desarrollo (Inter-American Development Bank)
CEPEJ	Conseil de l'Europe Commission européenne pour l’efficacité de la justice (Council of Europe European Commission for the efficiency of justice)
cf	confer (compare)
ch	chapter
CIDH	Corte Interamericana de Derechos Humanos (Interamerican Court of Human Rights)
CJEU	Court of Justice of the European Union
EBRD	European Bank for Reconstruction and Development
ECLI	European Case Law Identifier
ECtHR	European Court of Human Rights
ed	editor/editors
edn	edition/editions
eg	exempli gratia (for example)
ELI	European Law Institute
etc	et cetera
EU	European Union
EUR	Euro
ff	following
fn	footnote (external, ie, in other chapters or in citations)
GCCP	Code of Civil Procedure (Germany)
GDPR	General Data Protection Regulation (EU)
ibid	ibidem (in the same place)
ICPR	Civil Procedure Regulations (Israel)
ICT	Information and Communication Technologies
ie	id est (that is)
IIDP	Instituto Iberoamericano de Derecho Procesal (Iberoamerican Institute of Procedural Law)
JCCP	Code of Civil Procedure (Japan)
JPY	Japanese Yen
n	footnote (internal, ie, within the same chapter)
no	number/numbers
para	paragraph/paragraphs
PD	Practice Direction
PDPACP	Pre-Action Conduct and Protocols
pt	part
RSC Order	Rules of the Supreme Court (UK)
SCC	Supreme Court Canada
Sec	Section/Sections
supp	supplement/supplements
TCCP	Code of Civil Procedure (Turkey)
trans/tr	translated, translation/translator
UK	United Kingdom
UKCPR	Civil Procedure Rules (UK)
UNIDROIT	Institut international pour l'unification du droit privé (International Institute for the Unification of Private Law)
UP	University Press
US / USA	United States of America
USD	United States Dollar
USFRCP	Federal Rules of Civil Procedure (US)
v	versus
vol	volume/volumes
WB	World Bank
***	***

---

Cases

International/Supranational

European Court of Human Rights

Caillot v France, Case 36932/97 (ECHR) 4 June 1999 [ECLI:CE:ECHR:1999:0604JUD003693‌297]

Frydlender v France, Case 30979/96 (ECHR), Judgment 27 June 2000 [ECLI:CE:ECHR:2000:0627JUD00309796]

---

Bibliography

Bentham J, ‘Scotch Reform’ in J Bowring (ed), The Works of Jeremy Bentham, Vol V (Edinburgh, 1843).

Draper A, ‘”Corruptions In the Administration of Justice”: Bentham’s Critique of Civil Procedure, 1806-1811’ (2004) 7(1) Journal of Bentham Studies 1.

Eisenberg T, ‘The Need for a National Civil Justice Survey of Incidence and Claiming Behavior’ in S Estreicher & J Radice (ed), Beyond Elite Law – Access to Civil Justice in America (CUP 2016) 53.

Engel D, The Myth of the Litigious Society (University of Chicago Press 2016).

Genn H, Judging Civil Justice (CUP 2010).

Genn H, Paths to Justice (Hart 1999).

J Kakalik et al, Just Speedy, and inexpensive? An Evaluation of Case Management under the Civil Justice Reform Act (Rand 1996) https://www.rand.org/pubs/monograph_reports‌/MR800.html accessed 3 February 2025.

Meggitt G, ‘Civil Justice Reform in Hong Kong - Its Progress and Its Future’ (2008) 38 Hong Kong Law Journal 89.

Nottage L, ‘Civil Procedure Reforms in Japan: The Latest Round’ (2005) 22 Ritsumeikan Law Review 81.

Rijavec V, ‘Efficiency versus Fairness in Civil Procedure in Slovenia’ in C H van Rhee & A Uzelac (ed), Truth and Efficiency in Civil Litigation (Intersentia 2012) 283.

Sorabji J, English Civil Justice after the Woolf and Jackson Reforms (Cambridge University Press 2014).

---