Hicks: The radical improvement of data will transform public sector reform

The comparatively easier issue is developing and accessing more reliable and granular data (evidence-based policy development). The harder issue is the extent to which whether governments will use the data to inform policies and programs, compared to political and short-term considerations (policy-based evidence development).

I am less optimistic than Hicks:

Recent Policy Options articles have reflected on the need for public sector reform to deal with “wicked problems” related to decentralization, outsourcing, lack of public trust, lack of openness and the politicization of policy advice. Donald Savoieargues that the public service, with its obsolete policies and processes, has come so irreparably off its moorings that only an independent royal commission can fix it. Others, including Daniel Caron, Evert Lindquist and Robert Shepherd, call for internal reform processes.

An obvious question is why experiences over the past 50 or so years with both royal commissions and internal reforms have had such poor results. One factor has been the lack of the kind of reliable, granular evidence that would be needed to assess and learn from past initiatives that would also be needed to ensure that new directions can be managed in a transparent, accountable manner. As Caron, Lindquist and Shepherd say, institutional learning must be based on ongoing evidence and much of that needed evidence has, until now, not existed.

However, that is changing. As I demonstrate in a recent article, a radical improvement in evidence has become possible as a result of the availability of rich sources of new information, new ways of combining data from multiple sources in ways that protect privacy and the development of new predictive analytic tools based on big data. Much of this new evidence will be based on calculations using detailed administrative files. Ground-breaking information from other sources are on the horizon. For example, the use of data from personal monitoring devices from volunteers (with guaranteed anonymity) will be possible in the near future.

Another factor contributing to the failure of past reforms has been a tendency to examine the way in which services are administered separately from the actual content of those services. Many issues of public trust, morale, transparency, efficiency and service quality can only be addressed though deep changes in outdated policies and programs. They cannot be addressed by changes in the way programs are administered by the public service.

A third factor is that most reforms have taken place within existing programming and jurisdictional silos. This has made things worse by reinforcing overly fragmented programming, especially in the social policy area. Social policy in Canada is too often characterized by confusing sets of income supports and tax measures administered by both orders of government as well as overlapping but uncoordinated services provided by health care, educational and community organizations.

If the goal is to help people improve the quality of their lives and to increase trust between individuals and their governments, reforms need to be based on accountable partnerships that provide integrated services and supports based on an individual’s needs and aspirations. In practice, that hasn’t happened, largely because there’s been no source of routine evidence about which mixes of intervention are likely to work best for people with diverse characteristics in different circumstances.

However, that could change soon. Formerly wicked problems will be much easier to solve. Progress will soon make it possible to produce the kind of evidence that will allow us to learn from past experience and will support the accountability regimes needed to sustain reform efforts. That same evidence will also support major reform in the way policies are formulated, and the way in which programs are designed and delivered.

The new evidence will allow powerful new kinds of causal analysis. For example, it will become possible to make comparisons of the previous and subsequent characteristics and experiences of the participants in particular programs with those of non-participants with similar characteristics and in similar situations. This information would allow analysis of which programs (and combinations of programs) are working best, for whom and in what circumstances.

Using predictive analytic techniques, information on “what is likely to work best” will be available instantly – at the time when decisions are being made on choices of interventions. It will allow programs to continually improve and evolve based on automatic feedback loops showing what has been working best for people, including those who now feel alienated by standard, often fragmented programs that do not reflect their individual circumstances.

Over time, the new information is likely to reshape the way in which people are matched with programs. It would support the growth of an independent case management function where individuals with the greatest needs are referred to the service most likely to be effective (or, often, a combination of services and income supports) by counsellors who have access to evidence on what is likely to work best, regardless of who offers the service. The same information would allow other individuals, acting on their own, to learn what services exist and which are likely to work best for them given their needs and interests.

Development has reached a point where Statistics Canada is considering plans to use these new techniques to radically improve its social statistics. It is the only agency with the mandate and technical capacity to ensure the privacy and confidentiality of the source data and to ensure the quality of the resulting statistics.

The next logical step would be for it to gradually and cautiously enter into partnership arrangements with agencies in different jurisdictions and in different policy domains. Those agencies who wish to be early reformers can experiment with ways of using the anonymous “what works” evidence using the big data that is securely held within Statistics Canada.

It will take time to develop a fully mature system based on the new analytics. Progress will be slow and uneven across different policy domains. Issues related to public acceptability will be paramount. However, much of the needed background development work has already taken place. If the demand is there, significant progress is possible in many areas within five to 10 years.

To be successful, a public sector reform initiative should have a broad mandate that will result in the creation of the inter-agency partnerships needed to test out the new “what is likely to work best” evidence in practical program and policy applications. It should also propose ways of systematically applying lessons learned to mainstream programming.

Source: Hicks: The radical improvement of data will transform public sector reform

To truly modernize social programs, it will take big data and analytics

I may be biased given some of the obstacles I faced when working on “citizen-centred service” in the early days of Service Canada, where even conceptual work regarding integrating disparate programs and services from a pathfinder perspective faced resistance.

The complexity and coordination required, the organizational and even program stovepipes, and the sheer difficulty in developing and implementing such a change agenda make me a sceptic. After all, the government wasn’t even able to integrate pay services for its own employees with Phoenix, and has had less visible problems and challenges with Shared Services Canada.

But of course, better use of big data, and better capacity to analyse the data, offer considerable potential to assess the effectiveness of current programs, identify gaps and improve outcomes:

Finding better ways of wiring for e-government is important and necessary. Nobody would disagree with the need for better computing, electronic communications and information management.

However, digital improvements will bring about only modest gains if they are applied to programs that, at core, are based on pre-computer technologies, as is the case with most of today’s social and health programs. Transformative changes in program objectives and designs based on big data and micro-analytic tools must be brought into the picture. We need to create a trove of “what works” data that will lead to individually tailored social programming.

Most of today’s social programs were designed decades ago and, reflecting the limits of the technology of the day, provide eligible individuals with standard services, products or income supports that are designed to address specific problems.

For example, an unemployed person might be assigned to a training course with a fixed duration and curriculum. A low-income senior will be provided with a top-up pension in an amount that is predetermined based on the individual’s annual income in the previous year. Someone diagnosed with a particular disease will receive a prescription for specific drugs. These benefits are provided by a variety of independent programs funded by different orders of government — and are often delivered by staff in the social work, education or health disciplines. It is difficult to coordinate or even communicate across these programs, which is why they are often referred to as program silos. Individuals who receive benefits are seen as recipients, clients, patients or students, not as citizens or partners.

It’s a reasonably efficient system that works reasonably well for most people most of the time. On balance, the results are positive, near the average of other OECD countries.

However, the system is seriously showing its age.

The underlying weakness shows up most starkly in the way the system deals with the most vulnerable. People who are most in need of health and social services or income support often face multiple obstacles in life. They might lack several types of skills, have inadequate housing and poor jobs, have differing degrees of family support and financial assets, and face a variety of health, disability and addiction issues. People with multiple needs can face an almost impenetrable array of separate programs, each with different terms and conditions and offering solutions that are partial at best. Even with the help of experts and case managers, it is often impossible to create sensible combined packages of benefits to meet individual needs.

For decades, service providers and groups representing the vulnerable have pointed out the problem of trying to shoehorn people into this complex system of fractured supports and benefits.

And for many years, policy documents relating to education, health and social policy have called for a more holistic approach, with benefits directly tailored to the diverse needs of individuals. These have been referred to as student-centred, citizen-centric and, more recently, individually driven approaches. In health, related aspirations are often referred to as precision medicine or personalized medicine, where medical treatments, practices or products are tailored to the individual patient.

However, the called-for changes have not occurred. Experiments, demonstrations and other initiatives that have attempted to cut across the boundaries of the program silos have proven difficult to sustain and have typically had little impact on the design of mainstream programming.

There are good reasons for the lack of success in moving outside traditional silos:

  • Traditional programming makes it relatively easy to provide ongoing funding, to ensure ministerial accountability and to provide the high professional standards that can ensure, for example, the quality of health and educational interventions.
  • No organization has a mandate to develop interventions that cross these traditional program boundaries.
  • The empirical data needed to assess the effectiveness of tailor-made, holistic interventions are underdeveloped and are certainly not yet strong enough to create the needed accountability arrangements. Strong accountability regimes — the monitoring and evaluation activities that ensure that money is spent effectively, transparently and in line with intended objectives — are essential if reform is to be sustained.

But the solution is on the near horizon: big data and predictive analytics. They offer the opportunity for all citizens to become real partners in the design and implementation of the social and health programs that affect their lives. They can provide transformative gains, particularly for people who are most vulnerable.

This technology is in use in other applications and can be applied to social policy. I discussed it in an IRPP essay I wrote in 2015, The Enabling Society. At its core, individuals would have access to information at the very time they need to make big social and health decisions, to make well-informed choices about which combination of training, social services, housing, income supports and health interventions is likely to work best for them. This information would be calculated from large data sets that record the experience of people who have been in similar circumstances and had similar aspirations in the past. This technology produces information that allows all dimensions of the system to work in harmony:

  • The “what works” information would also be available to case workers, teachers, health professionals and other front-line staff so they can become partners in helping individuals put together flexible packages of interventions that are most likely to meet an individual’s particular needs and aspirations, including benefits provided by programs originating in different disciplines and orders of government.
  • The same information would provide the designers and administrators of the many independent traditional programs with the tools to make improvements steadily and automatically over time based on feedback loops that routinely describe which features of the program are working best and for whom — and at what cost.
  • The same information would also support rigorous accountability regimes both for existing program silos and for the flexible arrangements that provide individually tailored packages of interventions.

Such a system would result in huge gains on multiple fronts: in individual and social well-being, in effectiveness, in reduced cost, in the openness and accountability of public programs and in the ability of different orders of government to work together more harmoniously and in a way that treats citizens as main partners in shaping and delivering social programs.

A radically different approach along these lines, one that so dramatically changes the relationship between government and citizen, obviously cannot be attained overnight.

We should start small, in areas where mechanisms already exist to allow cooperation across jurisdictional and program borders and where the needed “what works” information is already well developed. There are a number of possible starting points.

For example, Employment and Social Development Canada could work with one or more provinces in introducing “what works” information into the daily operation of training and other employment programs on an experimental or demonstration basis under the authority of existing labour market agreements, which provide federal funding to support provincial and territorial employability initiatives. These agreements already allow considerable flexibility in the funding and development of innovative employment programs. As well, the needed “what works” data have already been developed and are already routinely used in the evaluation of these projects.

Once their practicality and effectiveness have been clearly demonstrated, these small initiatives could extend naturally and gradually to other areas and would, eventually, become the normal way we do business.

At the same time, the government of Canada should undertake a large-scale exercise to develop big data from administrative sources, such as anonymized information from tax files, employment insurance records and provincial training and social assistance files. It should also develop the associated analytic tools that will allow us to use these rich data to better understand individual behaviour and the kinds of social interventions that are likely to work best at the level of particular individuals.

Such fundamental but gradual changes in the purpose and design of programs need to go hand in hand with the deep reforms in digital processes that have been discussed in this Policy Options series, including a stronger capacity throughout all parts of government in the use of computers, electronic communications and information management. Process reforms could, in some cases, increase efficiency and improve service delivery and customer satisfaction. They could also provide somewhat better information about what programs and benefits are available and allow greater access to administrative data that have been collected. However, if such reforms are made in isolation, divorced from deep “what works” changes in program goals and designs, they risk creating expectations for change that cannot be met.

Those in the e-government community are not directly responsible for changing basic social policy directions or reforming the structure of social programs, but they can nevertheless play a pivotal role in the development and use of big data and predictive analytics. This might at minimum involve active support for reforms along the lines laid out in a paper by the Experts Panel on Income Security of the Council on Aging of Ottawa that describes the kind of micro-level data and microanalytic tools that are needed. Such support, along with process reform, could go a long way in finally enabling the real transition to the digital world.

Source: To truly modernize social programs, it will take big data and analytics