On the value of open data

by Kent Aitken

---

I spent the first week of October not going to the International Open Data Conference in Madrid. We lucked out, hosting it in Ottawa last year - it's the annual meet-up, stock-take, and direction-setting for a potent concept in modern governance (that is, not a new concept, but modern from an implementation point of view). Namely, that governments should be releasing the research, data, and briefing documents that underpin public decisions. When we specifically refer to open data, it's about the datasets - rows and columns of values - that governments collect, create, and draw on, and releasing them in machine-readable formats.That is, a stack of paper doesn't work; people outside government get to work with, crunch, and analyze the data too.

Alex Howard, a stalwart of the open data scene who works with the Sunlight Foundation, laid out a fantastic overview of both the conference and the state of the open data "movement." I have one point of disagreement, though. Howard writes:

"the deadline for more evidence is getting close. Politicians will always question transparency, which puts a premium on demonstrating why it matters in terms that the public understands and can apply in their lives." 

I instead see two possible routes for the future of open data. One is the value route, where governments firmly decide that it’s worth the cost and effort (through research, case studies, surveys, and notoriously hard-to-pin-down usage data). The second route would be governments start to talk about it like Access to Information (ATI) laws or, in Canada, Official Languages. The framing here becomes about obligation, expectation, and legal duty - not a cost to be debated. We don't concern ourselves with the pros and cons of releasing things in both official languages; we do it because it's just what we do.

And yes, there's a cost to putting open data on that same plane of existence, and it involves re-tooling decades of information management systems to get there. In the meantime, however, we have the most expensive model: internal information architectures, open data registries, and Access to Information flows. We'll always have all three, but right now many documents will exist in each system, where the long-term for open data is minimizing the duplication and triplication. 

Right now the lifecycle for records includes internal use followed by eventual release, sometimes being requested via ATI in between. Once documents and data start going straight to the open-to-the-public storage solution, a couple things will happen:

1. The costs will go down
2. Analysts in other parts of that same government will be able to find and use that data and information sooner (including knowing that it exists in the first place)
3. When people work in government want to work with those outside, they can simply link to the already-open data they're working with (this is one of the ways open data overlaps with open dialogue and citizen engagement)

The transparency antidote to risk

by Kent Aitken

---

A couple years ago my team worked on a project that got some negative attention. I'll go light on the details, but we took some criticism and my teammates got worked up, looking for ways to defend the outcomes. I had the opposite response: now we had some ammunition to do things better in the future.

Across the public sector, the meme seems to be that people are scared of transparency (e.g., open data excuse bingo). Which is, at first blush, understandable: it might mean more scrutiny, more criticism, a loss of control over the conversation.

However, there's another meme that bureaucracies are risk-adverse and that they stick to the status quo. Transparency about public programs and services reveals the risk in the way things are now, and gives decision-makers an even playing field to make decisions about change.

Here's an example. Edmonton has had a public dashboard of public policies and services for years: transit ridership, 311 call response times, growth in small and medium businesses, etc. And some are in the yellow and red, not meeting the benchmarks set.

No one signs up for public service to deliver a service that doesn't meet the public's needs. Having the performance freely available on a struggling program means that the program managers will have to explain why things are the way they are, but it also means that the program will receive the support it needs to improve - or change. It's short term stress for the sake of being a part of something worthwhile in the long run.

And the public gets a better shot of getting what they need, sooner - both in terms of information and public outcomes. Where there's little transparency about current performance, potential changes from the status quo get disproportionately scrutinized. And given that even long-running activities - e.g., how governments have been doing IT - are still experiments with uncertain long-term outcomes, we need to put those activities on an equal footing with their alternatives.

How We Govern Ourselves

by Kent Aitken

---

Today I'm just going to point you towards CBC Ideas: a 54 minute podcast on whether citizens' brains are built for democracy. The episode centers around the PhD work of David Moscrop, but covers many fascinating bases: rationality versus emotion, how we communicate, cameras in Parliament, and how different social mixes can massively influence people's approaches (and groups' outcomes).

Ideas from the Trenches - Too Dumb for Democracy

It's great food for thought on how we govern ourselves. If you get inspired and would like a slightly deeper dive, you can find some counterpoints to the views expressed (report-length, both):

Direct Democracy Works

This highlights possible differences between issues-based and representative-based votes and points to various direct democracy macro success stories.

Why Critics of Transparency are Wrong 

Different angle, but the premise they question - that citizens' views about what they want are inaccurate - is similar. They also specifically take on pros and cons of cameras in governing bodies. 

Get Meta on Engagement

by Kent Aitken

---

This is a Story as an Introduction, and it is as Long as the Rest of the Post

For a while I've held the opinion that public officials should be blogging or writing Op-Eds about decision rationale and key strategies. This is for several reasons: to get feedback with which to make decisions, to expose unforeseen adverse effects, but perhaps most importantly, to get past mere information availability and into actual transparency. Both public officials and citizens would be better off receiving information with both context and some level of thoughtful presentation.

Which I why I applaud initiatives like St. John's Councillor Dave Lane's blog. I can't speak for him as a city Councillor, but I'm a big fan of his recent 566-word post entitled Yes, I voted for a propane tank. Spoiler alert: it explains why he voted to allow a large propane tank to be built on the waterfront. Propane tanks aren't usually popular, but in context, his decision seems incredibly reasonable.

St. John's harbour. It's lovely, and you should go. Don't miss Raymond's, it's epic.

Get Meta

But that's all an aside. In his post from the day before that, Lane hit on a great lesson for stakeholder engagement. The city is launching a public engagement initiative, and they're starting with the question:

"How would you like the City of St. John's to engage?"

Surprisingly, an oft-overlooked idea in engagement activities, and great as either a starting point or a periodic gut-check: asking your stakeholders for input on your relationship with them. Don't just ask, "What do you want us to do?", but "How do you want us to ask you 'What do you want us to do?' "

It's simply a periodic testing of the assumptions on which your processes rest. And it's useful. Get meta. 

Photo by Aconcagua (Own work) [GFDL (http://www.gnu.org/copyleft/fdl.html) or CC-BY-SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons

Trust in the Age of Information

by Kent Aitken

---

Last week Bill Eggers from Deloitte was in Ottawa, explaining his view of The Solution Revolution: the idea that for-profit businesses and social enterprises will solve societal needs before government can, in creative, interesting ways. The question post-presentation was "How can government help this ecosystem of solutions?" One of the themes we hit on in response was that supporting such counter-intuitive approaches is a hard road. The Triple Bottom Line There's a common case study in business literature - whether it's about innovation or marketing - about Gilette's "safety razor." This is the standard now: you buy a handle once, and expensive disposable blades frequently. But when introduced, it was a disruptive innovation, undermining barbershops' monopoly on shaving. From the marketing standpoint, it's a example of knowing what your customers want. As it turned out, they didn't want a steady hand with a straight-razor in a nice barbershop, they actually just wanted a clean-shaven face. Or as Harvard's Ted Levitt put it: "No one ever buys a shovel. What they're buying is a hole." But for governments, this level of disruption and slight counter-intuition doesn't go far enough. Government isn't looking for a shovel or a hole. It's more like government wants a vibrant environment in which kids can grow up. The hole is just for a post, which supports a volleyball net, which isn't even really for volleyball at all - in the long run, it's actually just an interaction device for the neighbouring kids, to build friendships and community. So the savvy marketer, when approached by someone looking to buy a shovel, might then say "Are you sure you don't want a soccer league, instead?" With so many stakeholders, this actually is the level of complexity governments are experiencing in their outcomes, whether or not they actively pursue it. Or even know it. It's the idea of the Triple Bottom Line, taken to extremes. Former Clerk of the Privy Council Jocelyn Bourgon provides case studies from this future: what she describes as a New Synthesis of Public Administration. There is a fascinating common thread running through each case study. Not only do the programs she describes solve the short-term problem at hand, they actually work to obsoletize the government's involvement in that solution in the long run by building communities around the problem. That would be part of my True Bottom Line for Government. (Another would be organizational learning being a core component of any approach to solving problems. For another day.) The Sound Bite World The problem is that this level of nuance is really - and understandably - hard to explain to taxpayers. An example: a few months ago a House Committee in the US released a report detailing $50M in spending on conferences for the IRS between 2010 and 2012. Outrage, predictably, ensued. It seems extravagant until you take the step of dividing that figure by the three-year horizon and by the organization's 106,000 employees. All of a sudden they're spending about $157 per person per year on conferences. Which seems borderline paltry, especially if you compare that to what they spend on each of those employees. To boot, the longer and more accurate story about conferences is that their key benefit is the interaction and relationship building between colleagues and industry partners, far in excess of the value of simply having ideas changing hands and minds. Or for an analogous example, as Clay Shirky put it yesterday at the Code for America Summit: "The product of a hackathon isn't running code. It's the social capital people created among the people in the room." Or what about building civic awareness by gamifying recycling with rewards, which Eggers explained has raised recycling rates from 30% to 90% in some U.S. neighbourhoods? These are harder ideas to sell. Why hasn't the information age changed this? Trust in the Age of Information Even fifty years ago, the machinery of government would have been largely opaque to many Canadians. But as the availability of information has increased, Canadians' trust in government has not. Today, I would posit, we're at an awkward point in this evolution where we have information without understanding; availability without transparency; content without context*. But this situation may be a side-effect of The Connected World: that of radio, television, photojournalism, and globalization, which is giving way to a new one. Which dotted line path will it be, as even more information becomes available? What happens to the trend in public trust in The Hyperconnected World? As the age of Open Government unfolds? Are we going to see an acceleration, or a slowing and reversing? I don't know, and I think much hinges on it. But if government lacks the relationships, willpower, and evidence to pursue messy, complex outcomes - to return to my original analogy, communities, rather than holes - The Hyperconnected World will be characterized by a lot of getting the wrong things done, with stellar efficiency. * Another link to come out of the Code for America Summit yesterday: Transparency is Not Enough, by Danah Boyd. Suddenly I feel bad for having partially rehashed a three-year old argument (I wrote this over the weekend (and to be fair, this is actually somewhat of a synthesis of a handful of themes I've already gotten in to)), but considering that we still haven't moved past weak transparency yet, I'm okay with it.

The Collaborator’s Dilemma

I've been thinking a lot lately about collaboration and Game Theory. More specifically I've been examining and re-examining the Prisoner’s Dilemma in hopes of learning more about how transparency affects collaboration, and I think I may be on to something.


Primer

Game theory attempts to mathematically capture behaviour in strategic situations (games). The prisoner's dilemma is a fundamental problem in game theory and demonstrates why two people might not cooperate even if it is in their best interest to do so.

From Wikipedia:

Two suspects are arrested by the police. The police have insufficient evidence for a conviction, and, having separated the prisoners, visit each of them to offer the same deal. If one testifies for the prosecution against the other (defects) and the other remains silent (cooperates), the defector goes free and the silent accomplice receives the full 10-year sentence. If both remain silent, both prisoners are sentenced to only six months in jail for a minor charge. If each betrays the other, each receives a five-year sentence. Each prisoner must choose to betray the other or to remain silent. Each one is assured that the other would not know about the betrayal before the end of the investigation. How should the prisoners act?

... In this game, as in most game theory, the only concern of each individual player (prisoner) is maximizing his or her own payoff, without any concern for the other player's payoff. The unique equilibrium for this game is a Pareto-suboptimal solution, that is, rational choice leads the two players to both play defect, even though each player's individual reward would be greater if they both played cooperatively.

In the classic form of this game, cooperating is strictly dominated by defecting, so that the only possible equilibrium for the game is for all players to defect. No matter what the other player does, one player will always gain a greater payoff by playing defect. Since in any situation playing defect is more beneficial than cooperating, all rational players will play defect, all things being equal...

The classical prisoner's dilemma can be summarized thus (click to enlarge):

The Collaboration Payoff Matrix

If we mash up the payoff matrix above to focus on collaboration between two parties (as opposed to prisoners) and apply it to time spent working on a project (as opposed to jail time) it would look something like this (click to enlarge):




The New Dilemma

At the heart of the prisoner's dilemma is the inability of the prisoners to communicate with one another. It makes information sharing difficult and verification impossible. The uncertainty creates tension which in turn creates incentives for defection. If the two parties could simply bridge the physical distance between them, they could communicate, share information, be assured that the other party is keeping their word and collectively achieve the optimum outcome (no jail time).

Similarly, at the heart of the collaborator's dilemma is an inability to communicate with one another. However, would-be collaborators aren't dealing with physical distance imposed on them by police interrogators but rather geographical distance, language barriers, organizational boundaries, information asymmetry and an opacity that generally defines the culture of bureaucracy. The result is often the same, opacity-creating tension, in turn creating incentives for free-loading (defection) as neither side can verify the actions of or the information put forward by the other. If bureaucrats can bridge the information divide between them they could have more confidence in the system as a whole and would be more likely to collectively achieve the optimum outcome (faster production) and perhaps start to more effectively undermine the longstanding culture of knowledge as power.

Defection or in this case, free-loading, becomes next to impossible in a system where people can't hide in the shadows, where information can't be hived off for personal gain, or where strategic relationships can't be easily exploited. Transparency just may be the answer to many of our organization's problems; it may also create new ones. But all in all, what I find fascinating is that as our public institutions open up, the systematic pressure towards sub-optimal outcomes (defection) seemingly decreases. This creates room for new incentives to start to take root, and I think we are all in agreement that we could use some new incentives.


Still Work to Do

The trouble is that we aren't there yet, we are still a system in transition, which means that would-be collaborators can be easily taken advantage of by old school bureaucrats. The environment we find ourselves in is one where collaboration, especially in the face of traditional defectors, requires great courage. It requires even greater courage when those traditional defectors occupy higher positions than the would be collaborators within the hierarchy.

Do you have the courage it takes to collaborate or are you just another prisoner?

[Image credit: pasukaru76]