Mikey: Thank you for your feedback and kind endorsements. I am particularly supportive of your penultimate paragraph and the unintended side-effects that almost invariably occur when performance measurements (metrics) are instituted. I am fond of another adage that applies here, "Be careful not to reward A while hoping for B!" On the other hand, the collection and dissemination of qualitative feedback, especially when the objective is purely improvement toward excellence, can be immensely informative and valuable. If the SCI's will is that I remain involved with this effort, that would be my overriding goal in developing a WG Self-Assessment instrument. Regards, Ken From: owner-gnso-improvem-impl-sc@icann.org [mailto:owner-gnso-improvem-impl-sc@icann.org] On Behalf Of Mike O'Connor Sent: Thursday, May 23, 2013 8:25 AM To: Ken Bour Cc: gnso-improvem-impl-sc@icann.org; Julie Hedlund; Hoggarth, Robert (ICANN); 'Marika Konings' Subject: Re: [gnso-improvem-impl-sc] WG Self-Assessments hi Kan, thanks for joining us, and thanks for your thoughts on moving forward. as i mentioned on the call, i'm quite keen on this approach. one of the things i've been fussing about recently is the need to beef up the resources and attention devoted to working groups (the bottom of the bottom-up process). i think most of you have seen my rant about this topic in one place or another already, but here's a link to it to get it into the SCI conversation. http://bar.com/w one of my strongest interests is trying to figure out ways to build a deeper "bench" of working group participants and leaders. i think that a good self-assessment process for WGs could provide a lot of help in showing people what goes on in WGs, where we need to beef up the preparation new-participants get, what kinds of coaching and assistance Chairs could use, etc. the "art" in all this is to avoid accidentally creating structures that drive bad behavior. the old adage "that which gets measured gets done" applies here -- and we've all seen the strange effects that can inadvertently fall out of ill-chosen metrics. for example, there's currently a lot of interest in driving "on-time" into the WG process. a good metric in many cases -- but tricky to apply correctly in a consensus-based decision-making process. that's another reason i like where this is going. i can't think of a better person to work on this effort than you, Ken. it's great to have the combination of your process-improvement expertise and your deep understanding of what ICANN, the GNSO and the PDP process are all about. i'm really looking forward to working with you on this. mikey On May 22, 2013, at 5:31 PM, Ken Bour <ken.bour@verizon.net> wrote: Dear SCI Members: I appreciated the opportunity to attend your teleconference this afternoon and to present an alternative approach to the "survey" you have been contemplating. As requested, this memo is a summary of the points I outlined for your consideration. Background As some of you may recall, I was the original drafter of the WG Guidelines and Charter Template and provided Staff support to the Working Group Work Team during its first year or so. One of the concepts we engineered into the framework was that Chartering Organizations would ask each WG to perform a self-assessment at the end of its life-cycle. The idea was to promote a critical examination of the processes/procedures such that the feedback could be incorporated into a continuous improvement of the WG Guidelines. After consulting with Julie Hedlund and Rob Hoggarth, I reviewed the current version of the WG Guidelines (ANNEX 1 of the GOP) and noticed that, indeed, vestiges of the self-assessment concept are still present in the document. In particular, please see the following sections: Section 5.0 Products & Outputs The products and outputs of a Working Group may be prescribed by the Charter such as a report, recommendations, guidelines, self-assessment or defined by the process under which the WG operates (e.g., Policy Development Process). . Self-Assessment Template (TBD) 6.2 Working Group Charter Template 6.2.4.4 Closure and Working Group Self-Assessment This section of the Charter should describe any instructions for WG final closure including any feedback and/or self-assessment that is requested by the Chartering organization. This section might also indicate if there is any specific format, template, or prescribed manner in which the feedback is to be provided. I have been away from this subject for more than a year, but it appears as though no Chartering Organization has yet asked a WG to generate a self-assessment. Perhaps one reason is that the original template work was never completed. One option that the SCI may wish to consider is to suspend the development of a one-time online survey and, instead, convert the questions (after some refinement) into a semi-permanent "WG Self-Assessment Template," which could be incorporated into the WG Guidelines (as was originally intended) and completed by all WGs (individually and/or collectively) as part of their closure process. I see several advantages to this approach: 1) Feedback would come from actual and recent WG participants (targeted audience). 2) The information collected should be fresh given that the group recently completed its work (salience). 3) If the self-assessment template is reviewed (Chair checklist item?) with team members the start of deliberations (revealing questions that will be asked at the end) and something occurs that uncovers a gap or error in the guidelines, the WG could note it for later inclusion in the self-assessment. 4) Unlike a static survey, incorporating a self-assessment instrument into each WG's process provides a dynamic catalyst for continuous improvement. 5) If the Chartering Organization (e.g., GNSO Council) determines, based upon feedback from one or more WG self-assessments, that the guidelines (or even the self-assessment template itself) need to be amended for any reason, it can direct Staff or another community team to address any deficiencies or issues uncovered. Thinking about this concept further and picking up on Mikey's comment about the learning objectives, my recommendation would be to broaden the self-assessment to ask not just about the WG Guidelines document, but the quality and effectiveness of other important success factors. If the SCI concurs, I would be willing to assemble an initial draft of a WG Self-Assessment Template that would attempt to assess the usefulness and effectiveness of: 1) Support Infrastructure . charter, procedures, tools/templates, and mechanics supporting the WG's operations; 2) WG Processes/Operations . leadership, norms, decision-making (consensus), and outputs. That sounds like a lot of content, but I believe any resulting template should be designed so that it is (a) simple/straightforward to complete (perhaps employing the ICANN Wiki capability?) and (b) respectful of respondents' time (length). I look forward to your comments via the SCI email list. Respectfully, Ken Bour PHONE: 651-647-6109, FAX: 866-280-2356, WEB: www.haven2.com, HANDLE: OConnorStP (ID for Twitter, Facebook, LinkedIn, etc.)