Structuring Problems for Effective Policy Evaluation

Every good policy begins with a good question. Before governments design schemes, allocate budgets, or roll out reforms, someone has to figure out what the problem actually is – and that is rarely as obvious as it sounds. A scheme meant to reduce child malnutrition can fail not because the food is bad, but because the problem was framed as a food shortage when it was really about water, sanitation, and maternal health. This is the work of problem structuring: the careful, often underrated craft of framing public issues so that policies can be evaluated, adjusted, and improved over time.

What problem structuring really means 🔗

Problem structuring is the process of defining and organising a policy issue so it can be meaningfully analysed, acted upon, and later judged against real-world outcomes. According to policy scholar William N. Dunn, problem structuring uses systematic procedures to handle problems that are ill-defined, ill-structured, or wicked. In other words, it is the toolkit analysts reach for when an issue is messy, contested, and resistant to a single neat solution.

The distinction between defining and structuring a problem matters. Dunn explains that defining a problem means identifying the elements that belong inside its boundaries, while structuring means arranging those elements in a particular way – much like the difference between a pile of stones and a pyramid built from them. A policy team working on urban air quality might identify the same ingredients as everyone else (vehicles, industry, construction dust, stubble burning), but how those elements are arranged in the problem frame determines which interventions get priority.

Why it sits at the heart of the policy cycle 🔗

The policy cycle is usually taught as a sequence: agenda setting, formulation, adoption, implementation, and evaluation. But as the UK Magenta Book reminds practitioners, this one-directional relationship rarely holds in practice; the process is iterative, and evidence gathered during implementation keeps feeding back into how the problem itself is understood. Problem structuring, then, is not a one-off activity at the start of the cycle. It is a recurring discipline that allows policymakers to revisit and refine their understanding of the issue as fresh data, political signals, and ground realities emerge.

Wicked problems and why framing is hard 🔗

Many of the toughest public challenges belong to a category called wicked problems. These are issues with incomplete, contradictory, and shifting requirements – problems that seem to resist any clean solution. Delhi-NCR’s air pollution is a textbook case. It is multi-causal and interdependent, requiring coordinated long-term governance rather than episodic crisis response. Vehicular emissions, construction dust, industrial output, and seasonal stubble burning all interact; a policy that targets only one driver at a time will almost always underperform.

Similarly, the COVID-19 pandemic exposed how quickly a crisis can force problem definitions to evolve. Problem identification, formulation, adoption, and implementation unfolded almost simultaneously as the situation escalated, and continuous evaluation led to real-time policy adjustments. Without rigorous problem structuring, governments would have been left chasing symptoms – hospital bed shortages one week, vaccine hesitancy the next – instead of building a coherent response.

Structured, moderately structured, and unstructured problems 🔗

Not every policy issue is wicked. Scholars like Robert Hoppe argue that problems sit on a spectrum, and good governance means sincerely trying to move unstructured problems into more structured territory, because moderately structured and structured problems lend themselves more easily to organisational policymaking and implementation. A problem is structured when there is broad agreement on values and on the evidence needed to address it. It is unstructured when actors disagree about both. Problem structuring is how analysts negotiate that middle ground.

The building blocks of effective problem structuring 🔗

Turning a messy situation into something policymakers can act on requires a few core ingredients. Each of these inputs shapes whether evaluation later on will be meaningful or merely ceremonial.

Adequate and credible information 🔗

Data is the backbone. Analysts draw on administrative records, surveys, research studies, expert interviews, and stakeholder feedback to understand root causes and test the feasibility of alternatives. When the evidence base is thin or biased, the problem frame inherits those distortions. India’s National Family Health Survey, for instance, has repeatedly reshaped how the malnutrition problem is structured – showing that stunting is not merely about calories but about the first 1,000 days of a child’s life.

Feedback mechanisms 🔗

Feedback keeps a policy honest. The policy process is iterative, and feedback from public opinion, media reports, and evaluation studies helps policymakers make necessary adjustments so the policy remains relevant and effective. Public consultations, grievance portals, social audits under MGNREGA, and parliamentary standing committee reports all serve this function. Without them, a policy ossifies even as the problem shifts beneath it.

Resources that match ambition 🔗

A well-framed problem is useless if the money, people, and technology needed to investigate and act on it are missing. This is why ambitious schemes often underperform in their early years – the administrative machinery simply cannot keep up with the scale of the problem definition. Policymakers must weigh what existing institutions can realistically deliver against what the problem demands.

Political will 🔗

Problem definitions are, in the end, political acts. Hoppe notes that problem definition requires a political decision to stop thinking and shift to public action, and that sooner or later the intellectual debate on problem structuring is cut off by politicians who take responsibility for choosing a particular frame and its solutions. Without political backing, even the most rigorous analysis goes nowhere.

Problem structuring inside the Indian policy machinery 🔗

Institutions like NITI Aayog have made problem structuring and evaluation more central to the policy process. The Development Monitoring and Evaluation Office within NITI Aayog assesses the performance of line ministries based on outputs and outcomes across central and centrally sponsored schemes. This kind of institutional feedback allows problem frames to be updated against evidence rather than kept frozen in their original form.

A case in point: Aspirational Districts 🔗

The Aspirational Districts Programme is a good example of problem structuring done thoughtfully. Covering 250 million people across 112 of the country’s least developed districts, the programme is structured around competition among districts, convergence of schemes, and collaboration across government agencies. Instead of treating “backwardness” as a single national problem, it decomposes the issue into district-level indicators across health, nutrition, education, and infrastructure – a classic move in problem structuring, where a large, unstructured problem is broken into smaller, more structured sub-problems that can actually be tackled.

When problem framing goes wrong 🔗

The Right to Education Act illustrates what happens when problem structuring is incomplete at the design stage. Its implementation faced challenges related to infrastructure gaps, teacher shortages, and coordination issues between central and state education authorities. The original problem frame focused heavily on access and enrolment. Quality, teacher capacity, and learning outcomes were under-weighted, and subsequent evaluations have pushed the system to restructure the problem around learning rather than attendance alone.

The risk of solving the wrong problem 🔗

Perhaps the most consequential error in policy analysis is what Dunn calls a Type III error – solving the wrong problem precisely. Successful problem structuring involves understanding stakeholder perspectives and avoiding Type III errors, which occur when the wrong problem is formulated in the first place. A highway safety policy that frames speeding as a driver behaviour issue will produce very different interventions from one that frames it as a road design issue, and each will generate very different evaluation metrics. The danger is that by the time evaluation reveals the mistake, years of budgets and political capital have already been spent.

Methods that help analysts structure problems 🔗

Analysts use a range of techniques to avoid these traps. Approaches like Strategic Options Development and Analysis, soft systems methodology, system dynamics, and decision conferencing help groups of stakeholders with different perspectives build a shared model of the situation. These methods matter because problem structuring is rarely an individual exercise. It is a group effort, and the quality of the frame depends on whose voices are in the room.

Making evaluation meaningful 🔗

Evaluation is only as good as the problem frame it inherits. If the indicators chosen at the structuring stage miss the real drivers of an issue, the evaluation will tell a reassuring story that bears little relationship to ground reality. This is why experienced practitioners argue that evidence gathering and preparation for evaluation should be going on in different ways across all phases of the policy cycle, with quick feedback loops rather than a single assessment at the end. Embedding evaluation thinking into problem structuring from day one means that by the time the formal review arrives, the data, indicators, and baselines are already in place.

Common obstacles on the ground 🔗

Even with the right frameworks, problem structuring runs into real constraints. Complexity and uncertainty make some issues genuinely hard to pin down. Resource limitations can throttle data collection. And political or bureaucratic resistance – vested interests, turf wars, departmental inertia – can push analysts toward safe, familiar frames rather than accurate but inconvenient ones. Overcoming these hurdles requires coalition-building, transparent methods, and a willingness to revisit assumptions when the evidence demands it.

Why this matters for anyone studying public policy 🔗

Problem structuring is often treated as a preliminary step, something to get out of the way before the “real” work of formulation and implementation begins. That view gets it backwards. The frame chosen at the start shapes every downstream decision: which ministries are involved, what budget is approved, what counts as success, and whether evaluation will reveal useful lessons or merely validate what was already assumed. Seasoned policy designers are advised to spend up to a third or half of their analysis and design time finding the right problem – a striking allocation that signals just how much rides on getting this stage right.

For students and practitioners alike, the takeaway is simple. Before asking “what should the government do?”, ask “what exactly is the problem, and who says so?” That second question, answered carefully, is what separates policies that quietly succeed from those that loudly fail.

What do you think? When a flagship government scheme underdelivers, do you think the bigger failure usually lies in how the problem was framed at the start, or in how it was implemented on the ground? And how much say should citizens have in defining what counts as a public problem in the first place?

References

1. https://www.ippapublicpolicy.org/file/paper/5aeff35b03d17.pdf
2. https://www.epatee-toolbox.eu/evaluation-principles-and-methods/process-of-evaluation/evaluation-and-the-policy-cycle/
3. https://laex.in/daily-mains-question/delhis-air-pollution-a-wicked-problem-explained/
4. https://www.tandfonline.com/doi/full/10.1080/25741292.2018.1427419
5. https://polsci.institute/perspectives-public-administration/stages-of-public-policy-process/
6. https://www.chandlerinstitute.org/governancematters/niti-aayog-the-role-of-a-policy-thinktank
7. https://policycommons.net/artifacts/3814373/problem-structuring-in-public-policy-analysis/4620289/
8. https://chaturvedimayank.wordpress.com/2021/07/14/what-are-problem-structuring-methods/