The world is a bit of a hot mess right now. Oligarchic and narcissistic global leaders acting with impunity and with total disregard for the lives of people, coupled with a complete upheaval of geopolitical norms that has disrupted our traditional diplomatic excellence and the presence of ever more complex supply chains and linked economies. All making the path to prosperity ever harder to find. The need for agile government has never been greater.

A few weeks ago, I joined a panel at Global Innovation Forum, discussing Agile Government. One of those titles that seems to get people all riled up – ‘What do you mean by Agile?? So, for the avoidance of doubt, I’m thinking about this in two ways.

Agile government (big A): the concepts of iterative not incremental, of test and learn and of structured experimentation built into the fabric of its operating structures. And (small a) agile government: the ability for government to react to changes in the world around it, both domestic and international, its ease of cushioning repeated blows and bouncing back with strategy and delivery that improves outcomes for citizens.

Both concepts are linked, but not the same. The behaviours demonstrated in the Agile will enable the latter but are not a pre-requisite for success. Similarly, the structures and mindset that help being agile are not intrinsic for well-run Agile teams.

What we are really talking about here is friction, or at least government’s ability to reduce it. Friction exists everywhere, in planning, policy making, business cases and funding, design, delivery and of course the parliamentary process itself, where the fundamental systems are designed to promote friction through debate. Friction can be positive, it can create healthy tensions, surface unknown needs and ultimately improve the quality and salience of those things that affect people’s lives. But unintentional friction, created through poor system and service design can have the opposite effect. Many years ago, working in the Prime Minister’s office under the coalition government, it was described to me as the ‘iron lung of government closing in around us, suffocating great ideas and innovative thinking’. This might sound excessive, and there is undoubtedly great work going on, but across the public sector it fundamentally remains too hard to get things done.

The iron lung of government closing in around us, suffocating great ideas and innovative thinking.

So a few reflections:

Operating Structures

Government hasn’t evolved its fundamental operating structure since 1854 when the Northcote-Trevelyan Report created the politically neutral Civil Service we have today. There have been attempts, nibbling at the edges: in 1918 the Haldane report reorganised departments around functions, in 1988 the Thatcher era efficiency unit hived off executive functions into agencies and then there was Devolution under New Labour. But these didn’t really have a core impact on the Machinery of Government. If this was a private sector company, its lack of progressive thinking would have put it out of business a long time ago.

Departments still operate in silos, recognised by the current government’s attempt to pursue Missions. The Mandelson / Robbins saga may be a Whitehall bubble issue to many, but it underlines a constant tension that sits between the Civil Service and the Government of the day which boils down to a lack of trust. The web of boards with their biblical printed papers curated by teams of private secretaries and business managers, pored over by leaders so understandably removed from the actual delivery of what they are discussing due to scale and scope. There have been incredibly successful radical moments, none more so in recent years than the obvious GDS / GOV.UK example, but ministerial accountability, departmental silos, a generalist civil service – the fundamental Westminster model has not been genuinely transformed since the Victorian era. It is therefore unsurprising that government behaves and reacts exactly at the speed it is intended to – framed against the time it takes to get a letter from one point to another.

Mark Sedwill, Cabinet Secretary from 2018-2020 was a great thinker on how to make government zing, although struggled to enact much of his vision. His central thesis was around looking at when government worked best. Short answer, in crises. All the factionalism falls away, people become focused, more collaborative, more outcome driven. He said his dream was to have the whole of government operating, in some way at least, in this fashion all the time. This happened, to a degree, in COVID and Brexit, but this took its toll – much of the Civil Service is now yearning for a return to stability, strategy and permanence.

Co-creation and structured experimentation

UK policy follows a model:

Consultation → Legislation → Rollout → Discover failings → Review

It is essentially a waterfall methodology applied to some of the most complex social systems imaginable. But there is another way, by behaving like a product, with staged releases, real-world feedback loops and iterative improvement, Parliament could vote on full enactment. There are a few precedents here. In 2010 the Behavioural Insights Team pioneered Randomised Control Trials, testing interventions before rollout. The model worked but remained a niche capability rather than standard practice. In 2016 the FCA introduced Regulatory sandboxes allowing fintech firms to test regulated products with real customers in a controlled environment. And mentioned previously, GDS adopting discovery, alpha phasing to service delivery, although this didn’t cover policy.

There is now an opportunity, with the proliferation of just-in-time data and AI-assisted analysis, to make this a reality. But there are barriers – there remain significant gaps between policy and operations (not everywhere by any stretch). Centralising DDaT teams away from policy teams is designed to maintain control and prevent the emergence of shadow tech, but in truth it simply creates silos. A low-risk appetite for negative feedback because of working in a live environment prevents iteration at pace. Fiscal pressures and the misunderstanding by leaders of the efficiencies of an experimental model. The desire to announce shiny things in a packaged big bang moment. These all converge, creating a barrier to co-creation and experimentation. It’s just not that exciting announcing a low budget research project.

Money means Power

The funding mechanisms for projects and programmes are designed to give greater control to the centre. By centre, I mean both the Cabinet Office and the Treasury, but also a Ministerial Department as a parent to an Arm’s length body. Here there exists a constant friction, between the needs of the project and that of its financial oversight. Spend is secured through business cases based on a manifesto commitment, hopefully founded on insight or research and approved against an outcome that has often already by set. These documents are linear and static, even when they are not supposed to be. But delivery needs to be flexible, agile, based on experimentation. There are plenty of examples where this exists, but these are too often reliant on personal relationships that cut across departmental boundaries. In too many cases it is too risky to deviate from the line-of-sight control imposed by the centre, particularly where the road might lead away from a manifesto commitment.

Governments still struggle with getting the balance right between autonomy and accountability, devolution and control, risk and reward. A concept to test would be better classifying funding against a specific risk profile, no different to an investment portfolio. This would help back innovative projects and ideas built around agility and experimentation where there is a known problem, but no stated solution.

Can we move the dial?

There is plenty more that I haven’t mentioned here, and there are brilliant case studies where teams of Civil Servants have successfully broken the mould. But for all the efforts of a great number of people, it feels like Agile Government is getting further away from us, not closer. The world is self-disorganising into a new operating model faster than we can evolve ourselves.

But the technologies and methodologies available to us today are not just productivity tools. They make iterative, experimental government structurally possible in a way it never has been before. These alone cannot overcome culture, funding models and accountability structures that were designed for a different era. That requires political will, institutional courage and a willingness to be judged on outcomes rather than announcements.

So here is the ask – not just to those in government, but to all of us advisers, consultants, technologists and academics who surround it. Stop designing for the system as it is. Start designing for the system it needs to become. Push back, ask why and don’t be afraid to challenge systems and processes that feel antiquated. The iron lung is real – but it is not invincible. And in a world that is genuinely a hot mess, we cannot afford to keep suffocating the ideas that might just help.