Redux: Visualizing the Entire TBS Policy Suite

by Nick Charney

---

Last week I shared a visualization of the Treasury Board Policy Suite that I put together (See: The Web of Rules: Visualizing the Entire Treasury Board Policy Suite). I didn't have the time to write it up properly but in the spirit of the Internet I thought I'd just throw it out there and do some spaghetti testing. The visualization garnered some significant attention and a bunch of questions from people trying to parse what it all means. This week I wanted to share the next iteration of the visualization (v.2), explain how I put it together and elaborate on some of my key takeaways and next steps.

Version 2: TBS Policy Suite

But first, as a reminder, this is how the TBS website characterizes the policy suite:

... and here is what it actually looks like. You will note that this version has a swanky legend and that the size of a particular instrument is now reflective of the number of connections it has to other instruments (tip: click the three dots in the centre of the visualization to expand the viewing pane, or click here to open it in another tab).



How I put it together

First, I signed up for a free Kumu.io account.

Second, I painstakingly clicked on each of the 273 policy instruments listed on the TBS website and manually connected each instrument to the other instruments listed under the 'related instruments' section on the right hand side of the page.

Third, I made sure to colour code each type of instrument (e.g. framework, policy, standard, directive and guideline) differently.

Fourth, I let Kumu do the rest. Where the instruments are located on the map and the distance between them is a function of the software itself.  While there are obviously some ways to tweak proximity and weighting, I haven't manipulated the data in any way that would unduly shift or bias it's emphasis. The folks at Kumu have been working with me to improve the overall quality of the visualization and I can assure you they have no bias one way or another about what it shows, only that it shows well.

Caveats

I've gotten a couple of comments about how some of the instruments have been moved elsewhere (e.g. National Joint Council) but my approach has been to work strictly with the information posted on the TBS website. I had to draw some boundaries around the scope of the work, those are them.

While I've managed to export, clean, import and rebuild the visualization in a way that improves its overall quality, there may still be some elements of human error. If you see something that's amiss, please let me know and I will look into it.


Key takeaways

The policy suite 'as is' bears no resemblance to the official artist rendering. The former lacks the sense of balance, coherence and wrap around support that the latter somehow conveys so clearly (and incorrectly). I don't have an explanation for this and I doubt that there is anything sinister at play here. It's as likely to be the result of best laid plans being twisted over time, sprinkled with with extenuating circumstances, mandate creep and a culture that papers over problems with more paper.

Given that we now have a clearer picture of what's actually at play within the policy suite, it's far less surprising that our culture is what it is. The most prominent instruments are (in order, based on the number of connections):

1. Policy on Government Security (88 references)
2. Policy on Internal Control (71 references)
3. Policy on Management of Information Technology (59 references)
4. Policy on Privacy Protection (56 references)
5. Policy on Information Management (55 references)

That said, any attempt to improve the web of rules needs to not only consider the number of relationships between instruments but the complexity of the interrelated contents within them. One of the distinct advantages of this visualization is that it makes a strong case for a complete policy reset; it can help illustrate that incremental reductions to the web of rules can take us only so far. It can be used to model the magnitude of the actual effects of reductions on the web itself. Besides, while de-linking one instrument from another or removing an instrument altogether may reduce the overall complexity of the web, it will accomplish little if we simply give into the temptation to shift its contents into another instrument. Let's avoid the policy shell game.

The model also helps drive home the case for mandatory reviews of policy instruments at regular intervals to ensure that they reflect the realities of the context within which they are being applied and not only the realities within which they were originally written. Any new approach needs to be cradle-to-cradle.


Next Steps

1. Constant improvement: if you see something that can improve my work, let me know.
2. Collaboration: I'll be at CodeFest in a couple of weeks sporting this visualization and the dataset I built to produce it.
3. Education: I'll be meeting with some of the officials at Treasury Board to walk them through the model and methodology; and hopefully whatever else we build at CodeFest.
4. Historical analysis: I'm contemplating working on a companion piece that shows how the suite has evolved over time. 
5. Positive Spillover: I'm looking for other applications of my methodology.

If you are interested in any of the above, please let me know. I'm heading on vacation for the week. I'll still be plugged in but corresponding at a far more casual pace.

Cheers