Mikey: Based upon the way I have designed the WG Self-Assessment, the PROCESS dimension is intended to refer to the WG’s internal operations (norms, logistics, decision-making, etc.). In that framework, I would consider the PDP to be an INPUT to the WG (imposed methodology) and, based upon your earlier comments, I attempted to cover it generically in the 1st question of Section II. If a WG member found that the PDP (or any other requirement/constraint) was detrimental to the team’s ability to accomplish its mission, he/she could address it in Section II of the questionnaire. I would recommend that the WG Self-Assessment instrument not be a means for evaluating the PDP (per se) for three reasons: 1) The WG Guidelines and Charter Template (source documents) do not specifically integrate the PDP within the WG process scope; rather, the PDP is specified as part of the ICANN Bylaws. 2) Not all WGs deal with PDP issues, which would mean that any such questions would have to be skipped for some percentage of respondents. The WG Guidelines mention the PDP, but only as an example of the type of methodology that may be imposed upon a WG chartered to address a domain name policy issue. 3) The PDP is a complex, multi-step process in its own right and probably should be evaluated exhaustively, but separately, via survey or other method. On the last point, we were looking for a set of questions that a WG might address collectively vs. individually. Perhaps a separate questionnaire could be developed that would delve deeply into the PDP methodology itself and could be administered to all or a random sample of appropriate WGs. Ken From: owner-gnso-improvem-impl-sc@icann.org [mailto:owner-gnso-improvem-impl-sc@icann.org] On Behalf Of Mike O'Connor Sent: Friday, June 14, 2013 11:12 AM To: gnso-improvem-impl-sc@icann.org Cc: Hoggarth, Robert (ICANN); Ken Bour Subject: Re: [gnso-improvem-impl-sc] Draft v2-WG Self-Assessment Questionnaire hi Ken, see? sometimes i'm not so lightning fast in my replies. this is looking really good. i've got just a little bit i'd like to see us work into the "Processes" section. is there a way to get a question or two in there that gives participants a chance to talk about the PDP process? we've got decision-making methodology (consensus) in there already, maybe this question/LO is closely related to that? looking back at the projects that the SCI is working on (method to suspend/end a WG is a good example), i think some feedback on the *structure* of the PDP would be helpful. for example, we have a lot of comment-periods (and subsequent reviews of those comments) built into the PDP right now. Marika laid all those end to end one time and came up with a minimum time to get through a PDP that's pretty long. it would be nice to start getting feedback from WG participants as to whether they felt that those were all needed and whether they were helpful to the work of the WG. i'm seeing a transition in the way that WG's review those comments. at first, the review felt like a burden that we had to get through because it was a requirement imposed on us by the PD. more recently those comment-reviews have been a really good source of discussion-points and preliminary language that we've woven into initial and final reports. if we stick with that example, there's a process-definition dimension (have we got the right number of comment-periods?) and a process-effectiveness dimension (did the WG effectively make use of those comments in doing their analysis?). both are important. we don't want to change a good process that's being badly carried out, in that case we want to improve the effectiveness of the WG participants. we DO want to review a bad process even though the WG has implemented it well if that badly-defined process is hurting effectiveness and timeliness. one reason i picked that particular example is because there's a lot of pressure to become "more agile" in the PDP right now. what this often turns into is a shorthand for "do it faster!" and, since the comment/review cycles are fixed/required that sometimes means that the only place to shorten the process is the deliberative portion of the WG's work. it would be nice to be able to get some data from participants that might give an early indication that something needs to change there. that analysis would also be helpful in the "let's not break/bypass the bottom-up process" and "policy vs implementation" conversations that are going on at the moment. thanks Ken -- really like where this is headed. mikey On Jun 12, 2013, at 10:42 AM, Ken Bour <ken.bour@verizon.net> wrote: SCI Members: I have been thinking about the topics raised by Mikey and Avri in their first impressions of the Draft Self-Assessment Questionnaire. Something was definitely missing from the original formulation, which led to an “Aha!” moment spurring me to create a second design which, I hope, addresses a few of the comments expressed thus far. I have reconstituted the questionnaire (still five sections but renamed), reorganized some of the original questions, and added a few new ones. In order not to lose track of the first iteration, I made a completely new page and will henceforth house all questionnaire versions under a new heading: Questionnaire Drafts/Versions ( <https://community.icann.org/x/ai5-Ag> https://community.icann.org/x/ai5-Ag). In this new Draft v2 iteration ( <https://community.icann.org/x/bC5-Ag> https://community.icann.org/x/bC5-Ag), I am attempting to take into consideration the dimension raised by Mikey that was only partially accommodated in Draft v1. To evaluate any dynamic system, we could subdivide it into three basic or core components: Inputs → Processes → Outputs. In the first version, I captured many of the processes, the outputs, but only a few of the inputs, namely, team member representativeness, tools, and outside experts. I did not ask about the other critical inputs that impinge upon the success of a WG, e.g., its charter/mission (including time or other constraints) and team memberexpertise. That led to a reconceptualization of the external resources questions into three buckets: administrative, technical, and human. This Draft v2 also shifts the rating scale to 7 points and adds “Background Contributor” to the Role list as suggested by Avri. I thank Mikey and Avri for their ideas/suggestions and look forward to additional feedback from the team… Ken PHONE: 651-647-6109, FAX: 866-280-2356, WEB: www.haven2.com, HANDLE: OConnorStP (ID for Twitter, Facebook, LinkedIn, etc.)

I would agree with Ron. The new PDP is still too fresh as to start developing questions. The experience is with the WGs. If they point to questionnable rules then we should pick it up and try to find answers. I think a more general self-assessment questionnaire should be applicable. Best regards Wolf-Ulrich From: Ron Andruff Sent: Tuesday, June 18, 2013 10:06 PM To: 'Ken Bour' ; 'Mike O'Connor' ; gnso-improvem-impl-sc@icann.org Cc: 'Hoggarth, Robert (ICANN)' Subject: RE: [gnso-improvem-impl-sc] Draft v2-WG Self-Assessment Questionnaire Dear Ken, Thank you for the excellent efforts. In following your various email exchanges with Committee members I noted (from below) the following: 3. The PDP is a complex, multi-step process in its own right and probably should be evaluated exhaustively, but separately, via survey or other method. On the last point, we were looking for a set of questions that a WG might address collectively vs. individually. Perhaps a separate questionnaire could be developed that would delve deeply into the PDP methodology itself and could be administered to all or a random sample of appropriate WGs. Most of us on this list will agree with you that PDPs are complex. For my part, I would like us to keep our eye on the ball vis-à-vis establishing an assessment questionnaire that is more one-size-fits-all as opposed to different questionnaires for different purposes. Better to try to compare apples to apples as best we can. PDP methodology is pushing the limits, in my view, in terms of what we are looking for now. My two cents (as a member, rather than Chair). Thanks, RA Ron Andruff RNA Partners www.rnapartners.com From: owner-gnso-improvem-impl-sc@icann.org [mailto:owner-gnso-improvem-impl-sc@icann.org] On Behalf Of Ken Bour Sent: Tuesday, June 18, 2013 15:05 To: 'Mike O'Connor'; gnso-improvem-impl-sc@icann.org Cc: 'Hoggarth, Robert (ICANN)' Subject: RE: [gnso-improvem-impl-sc] Draft v2-WG Self-Assessment Questionnaire Mikey: Based upon the way I have designed the WG Self-Assessment, the PROCESS dimension is intended to refer to the WG’s internal operations (norms, logistics, decision-making, etc.). In that framework, I would consider the PDP to be an INPUT to the WG (imposed methodology) and, based upon your earlier comments, I attempted to cover it generically in the 1st question of Section II. If a WG member found that the PDP (or any other requirement/constraint) was detrimental to the team’s ability to accomplish its mission, he/she could address it in Section II of the questionnaire. I would recommend that the WG Self-Assessment instrument not be a means for evaluating the PDP (per se) for three reasons: 1) The WG Guidelines and Charter Template (source documents) do not specifically integrate the PDP within the WG process scope; rather, the PDP is specified as part of the ICANN Bylaws. 2) Not all WGs deal with PDP issues, which would mean that any such questions would have to be skipped for some percentage of respondents. The WG Guidelines mention the PDP, but only as an example of the type of methodology that may be imposed upon a WG chartered to address a domain name policy issue. 3) The PDP is a complex, multi-step process in its own right and probably should be evaluated exhaustively, but separately, via survey or other method. On the last point, we were looking for a set of questions that a WG might address collectively vs. individually. Perhaps a separate questionnaire could be developed that would delve deeply into the PDP methodology itself and could be administered to all or a random sample of appropriate WGs. Ken From: owner-gnso-improvem-impl-sc@icann.org [mailto:owner-gnso-improvem-impl-sc@icann.org] On Behalf Of Mike O'Connor Sent: Friday, June 14, 2013 11:12 AM To: gnso-improvem-impl-sc@icann.org Cc: Hoggarth, Robert (ICANN); Ken Bour Subject: Re: [gnso-improvem-impl-sc] Draft v2-WG Self-Assessment Questionnaire hi Ken, see? sometimes i'm not so lightning fast in my replies. this is looking really good. i've got just a little bit i'd like to see us work into the "Processes" section. is there a way to get a question or two in there that gives participants a chance to talk about the PDP process? we've got decision-making methodology (consensus) in there already, maybe this question/LO is closely related to that? looking back at the projects that the SCI is working on (method to suspend/end a WG is a good example), i think some feedback on the *structure* of the PDP would be helpful. for example, we have a lot of comment-periods (and subsequent reviews of those comments) built into the PDP right now. Marika laid all those end to end one time and came up with a minimum time to get through a PDP that's pretty long. it would be nice to start getting feedback from WG participants as to whether they felt that those were all needed and whether they were helpful to the work of the WG. i'm seeing a transition in the way that WG's review those comments. at first, the review felt like a burden that we had to get through because it was a requirement imposed on us by the PD. more recently those comment-reviews have been a really good source of discussion-points and preliminary language that we've woven into initial and final reports. if we stick with that example, there's a process-definition dimension (have we got the right number of comment-periods?) and a process-effectiveness dimension (did the WG effectively make use of those comments in doing their analysis?). both are important. we don't want to change a good process that's being badly carried out, in that case we want to improve the effectiveness of the WG participants. we DO want to review a bad process even though the WG has implemented it well if that badly-defined process is hurting effectiveness and timeliness. one reason i picked that particular example is because there's a lot of pressure to become "more agile" in the PDP right now. what this often turns into is a shorthand for "do it faster!" and, since the comment/review cycles are fixed/required that sometimes means that the only place to shorten the process is the deliberative portion of the WG's work. it would be nice to be able to get some data from participants that might give an early indication that something needs to change there. that analysis would also be helpful in the "let's not break/bypass the bottom-up process" and "policy vs implementation" conversations that are going on at the moment. thanks Ken -- really like where this is headed. mikey On Jun 12, 2013, at 10:42 AM, Ken Bour <ken.bour@verizon.net> wrote: SCI Members: I have been thinking about the topics raised by Mikey and Avri in their first impressions of the Draft Self-Assessment Questionnaire. Something was definitely missing from the original formulation, which led to an “Aha!” moment spurring me to create a second design which, I hope, addresses a few of the comments expressed thus far. I have reconstituted the questionnaire (still five sections but renamed), reorganized some of the original questions, and added a few new ones. In order not to lose track of the first iteration, I made a completely new page and will henceforth house all questionnaire versions under a new heading: Questionnaire Drafts/Versions (https://community.icann.org/x/ai5-Ag). In this new Draft v2 iteration (https://community.icann.org/x/bC5-Ag), I am attempting to take into consideration the dimension raised by Mikey that was only partially accommodated in Draft v1. To evaluate any dynamic system, we could subdivide it into three basic or core components: Inputs → Processes → Outputs. In the first version, I captured many of the processes, the outputs, but only a few of the inputs, namely, team member representativeness, tools, and outside experts. I did not ask about the other critical inputs that impinge upon the success of a WG, e.g., its charter/mission (including time or other constraints) and team memberexpertise. That led to a reconceptualization of the external resources questions into three buckets: administrative, technical, and human. This Draft v2 also shifts the rating scale to 7 points and adds “Background Contributor” to the Role list as suggested by Avri. I thank Mikey and Avri for their ideas/suggestions and look forward to additional feedback from the team… Ken PHONE: 651-647-6109, FAX: 866-280-2356, WEB: www.haven2.com, HANDLE: OConnorStP (ID for Twitter, Facebook, LinkedIn, etc.)

Let me try to recall the pioneers�� thinking was or how I��m interpreting it. 1.. aren't we supposed to be addressing fine tuning of of the new PDP that came out of the committees that preceded us? No. We don��t invent new rules even less important ones. 2. wouldn't it be helpful to gather some insights about rough edges of the PDP that could use improvement? I think this mandate should be given by the council to a special team (maybe the ��old�� PDP WG) 3. wouldn't the pioneers who have gone through that process be a good group to ask? Agreed (see 2.) 4. wouldn't this questionnaire be a good place to do that? Yes but under a council mandate. I could agree to all of your questions, Mikey, to move things towards improvement. But the SCI hasn��t the know-how to develop PDP recommendations �C also some members are extremely knoledgeable. Best regards Wolf-Ulrich From: Mike O'Connor Sent: Thursday, June 20, 2013 2:30 PM To: WUKnoben Cc: Ron Andruff ; 'Ken Bour' ; gnso-improvem-impl-sc@icann.org ; 'Hoggarth, Robert (ICANN)' Subject: Re: [gnso-improvem-impl-sc] Draft v2-WG Self-Assessment Questionnaire hi all, i don't have super-strong feelings about this. but�� i am still trying to figure out what our mission is. aren't we supposed to be addressing fine tuning of of the new PDP that came out of the committees that preceded us? wouldn't it be helpful to gather some insights about rough edges of the PDP that could use improvement? wouldn't the pioneers who have gone through that process be a good group to ask? wouldn't this questionnaire be a good place to do that? confusedly yours, mikey On Jun 19, 2013, at 6:50 AM, WUKnoben <wolf-ulrich.knoben@t-online.de> wrote: I would agree with Ron. The new PDP is still too fresh as to start developing questions. The experience is with the WGs. If they point to questionnable rules then we should pick it up and try to find answers. I think a more general self-assessment questionnaire should be applicable. Best regards Wolf-Ulrich From: Ron Andruff Sent: Tuesday, June 18, 2013 10:06 PM To: 'Ken Bour' ; 'Mike O'Connor' ; gnso-improvem-impl-sc@icann.org Cc: 'Hoggarth, Robert (ICANN)' Subject: RE: [gnso-improvem-impl-sc] Draft v2-WG Self-Assessment Questionnaire Dear Ken, Thank you for the excellent efforts. In following your various email exchanges with Committee members I noted (from below) the following: 3. The PDP is a complex, multi-step process in its own right and probably should be evaluated exhaustively, but separately, via survey or other method. On the last point, we were looking for a set of questions that a WG might address collectively vs. individually. Perhaps a separate questionnaire could be developed that would delve deeply into the PDP methodology itself and could be administered to all or a random sample of appropriate WGs. Most of us on this list will agree with you that PDPs are complex. For my part, I would like us to keep our eye on the ball vis-��-vis establishing an assessment questionnaire that is more one-size-fits-all as opposed to different questionnaires for different purposes. Better to try to compare apples to apples as best we can. PDP methodology is pushing the limits, in my view, in terms of what we are looking for now. My two cents (as a member, rather than Chair). Thanks, RA Ron Andruff RNA Partners www.rnapartners.com From: owner-gnso-improvem-impl-sc@icann.org [mailto:owner-gnso-improvem-impl-sc@icann.org] On Behalf Of Ken Bour Sent: Tuesday, June 18, 2013 15:05 To: 'Mike O'Connor'; gnso-improvem-impl-sc@icann.org Cc: 'Hoggarth, Robert (ICANN)' Subject: RE: [gnso-improvem-impl-sc] Draft v2-WG Self-Assessment Questionnaire Mikey: Based upon the way I have designed the WG Self-Assessment, the PROCESS dimension is intended to refer to the WG��s internal operations (norms, logistics, decision-making, etc.). In that framework, I would consider the PDP to be an INPUT to the WG (imposed methodology) and, based upon your earlier comments, I attempted to cover it generically in the 1st question of Section II. If a WG member found that the PDP (or any other requirement/constraint) was detrimental to the team��s ability to accomplish its mission, he/she could address it in Section II of the questionnaire. I would recommend that the WG Self-Assessment instrument not be a means for evaluating the PDP (per se) for three reasons: 1) The WG Guidelines and Charter Template (source documents) do not specifically integrate the PDP within the WG process scope; rather, the PDP is specified as part of the ICANN Bylaws. 2) Not all WGs deal with PDP issues, which would mean that any such questions would have to be skipped for some percentage of respondents. The WG Guidelines mention the PDP, but only as an example of the type of methodology that may be imposed upon a WG chartered to address a domain name policy issue. 3) The PDP is a complex, multi-step process in its own right and probably should be evaluated exhaustively, but separately, via survey or other method. On the last point, we were looking for a set of questions that a WG might address collectively vs. individually. Perhaps a separate questionnaire could be developed that would delve deeply into the PDP methodology itself and could be administered to all or a random sample of appropriate WGs. Ken From: owner-gnso-improvem-impl-sc@icann.org [mailto:owner-gnso-improvem-impl-sc@icann.org] On Behalf Of Mike O'Connor Sent: Friday, June 14, 2013 11:12 AM To: gnso-improvem-impl-sc@icann.org Cc: Hoggarth, Robert (ICANN); Ken Bour Subject: Re: [gnso-improvem-impl-sc] Draft v2-WG Self-Assessment Questionnaire hi Ken, see? sometimes i'm not so lightning fast in my replies. this is looking really good. i've got just a little bit i'd like to see us work into the "Processes" section. is there a way to get a question or two in there that gives participants a chance to talk about the PDP process? we've got decision-making methodology (consensus) in there already, maybe this question/LO is closely related to that? looking back at the projects that the SCI is working on (method to suspend/end a WG is a good example), i think some feedback on the *structure* of the PDP would be helpful. for example, we have a lot of comment-periods (and subsequent reviews of those comments) built into the PDP right now. Marika laid all those end to end one time and came up with a minimum time to get through a PDP that's pretty long. it would be nice to start getting feedback from WG participants as to whether they felt that those were all needed and whether they were helpful to the work of the WG. i'm seeing a transition in the way that WG's review those comments. at first, the review felt like a burden that we had to get through because it was a requirement imposed on us by the PD. more recently those comment-reviews have been a really good source of discussion-points and preliminary language that we've woven into initial and final reports. if we stick with that example, there's a process-definition dimension (have we got the right number of comment-periods?) and a process-effectiveness dimension (did the WG effectively make use of those comments in doing their analysis?). both are important. we don't want to change a good process that's being badly carried out, in that case we want to improve the effectiveness of the WG participants. we DO want to review a bad process even though the WG has implemented it well if that badly-defined process is hurting effectiveness and timeliness. one reason i picked that particular example is because there's a lot of pressure to become "more agile" in the PDP right now. what this often turns into is a shorthand for "do it faster!" and, since the comment/review cycles are fixed/required that sometimes means that the only place to shorten the process is the deliberative portion of the WG's work. it would be nice to be able to get some data from participants that might give an early indication that something needs to change there. that analysis would also be helpful in the "let's not break/bypass the bottom-up process" and "policy vs implementation" conversations that are going on at the moment. thanks Ken -- really like where this is headed. mikey On Jun 12, 2013, at 10:42 AM, Ken Bour <ken.bour@verizon.net> wrote: SCI Members: I have been thinking about the topics raised by Mikey and Avri in their first impressions of the Draft Self-Assessment Questionnaire. Something was definitely missing from the original formulation, which led to an ��Aha!�� moment spurring me to create a second design which, I hope, addresses a few of the comments expressed thus far. I have reconstituted the questionnaire (still five sections but renamed), reorganized some of the original questions, and added a few new ones. In order not to lose track of the first iteration, I made a completely new page and will henceforth house all questionnaire versions under a new heading: Questionnaire Drafts/Versions (https://community.icann.org/x/ai5-Ag). In this new Draft v2 iteration (https://community.icann.org/x/bC5-Ag), I am attempting to take into consideration the dimension raised by Mikey that was only partially accommodated in Draft v1. To evaluate any dynamic system, we could subdivide it into three basic or core components: Inputs �� Processes �� Outputs. In the first version, I captured many of the processes, the outputs, but only a few of the inputs, namely, team member representativeness, tools, and outside experts. I did not ask about the other critical inputs that impinge upon the success of a WG, e.g., its charter/mission (including time or other constraints) and team memberexpertise. That led to a reconceptualization of the external resources questions into three buckets: administrative, technical, and human. This Draft v2 also shifts the rating scale to 7 points and adds ��Background Contributor�� to the Role list as suggested by Avri. I thank Mikey and Avri for their ideas/suggestions and look forward to additional feedback from the team�� Ken PHONE: 651-647-6109, FAX: 866-280-2356, WEB: www.haven2.com, HANDLE: OConnorStP (ID for Twitter, Facebook, LinkedIn, etc.) PHONE: 651-647-6109, FAX: 866-280-2356, WEB: www.haven2.com, HANDLE: OConnorStP (ID for Twitter, Facebook, LinkedIn, etc.)

Hi, In many ways, I actually agree with Mikey and beleive that especially area 2 is within the mandate we currently have. I even beleive the SCI has the ability to recommend changes to the PDP process if it encountered problems in any of the issues sent to it or found something in the periodic reviews of the various mechanisms we have never yet done. But, be that as it may, I wanted to let you know that the ATRT2 has made a decision to put out a RFP for a Independent Consultant to do a study inot the multistakeholder effectiveness of the GNSO PDP. A draft of the proposal* can be found at: http://mm.icann.org/pipermail/atrt2/2013/000482.html At today's ATRT2 meeting we discussed some few changes to the wording, but I beleive we will be putting this out to bid soon. avri * No, I did not initiate this proposal. That was Alan Greenberg, but many of the members of the ATRT2 thought this was something that needed to be done. I did not think it was time for this yet, as the new PDP methodology is still young. On 20 Jun 2013, at 11:00, WUKnoben wrote:

Let me try to recall the pioneers’ thinking was or how I’m interpreting it. • aren't we supposed to be addressing fine tuning of of the new PDP that came out of the committees that preceded us? No. We don’t invent new rules even less important ones.

2. wouldn't it be helpful to gather some insights about rough edges of the PDP that could use improvement?

I think this mandate should be given by the council to a special team (maybe the “old” PDP WG)

3. wouldn't the pioneers who have gone through that process be a good group to ask?

Agreed (see 2.)

4. wouldn't this questionnaire be a good place to do that?

Yes but under a council mandate.

I could agree to all of your questions, Mikey, to move things towards improvement. But the SCI hasn’t the know-how to develop PDP recommendations – also some members are extremely knoledgeable.

Best regards

Wolf-Ulrich

From: Mike O'Connor Sent: Thursday, June 20, 2013 2:30 PM To: WUKnoben Cc: Ron Andruff ; 'Ken Bour' ; gnso-improvem-impl-sc@icann.org ; 'Hoggarth, Robert (ICANN)' Subject: Re: [gnso-improvem-impl-sc] Draft v2-WG Self-Assessment Questionnaire

hi all,

i don't have super-strong feelings about this. but…

i am still trying to figure out what our mission is. aren't we supposed to be addressing fine tuning of of the new PDP that came out of the committees that preceded us? wouldn't it be helpful to gather some insights about rough edges of the PDP that could use improvement? wouldn't the pioneers who have gone through that process be a good group to ask? wouldn't this questionnaire be a good place to do that?

confusedly yours,

mikey

On Jun 19, 2013, at 6:50 AM, WUKnoben <wolf-ulrich.knoben@t-online.de> wrote:

...

I would agree with Ron.

The new PDP is still too fresh as to start developing questions. The experience is with the WGs. If they point to questionnable rules then we should pick it up and try to find answers. I think a more general self-assessment questionnaire should be applicable.

Best regards

Wolf-Ulrich

From: Ron Andruff Sent: Tuesday, June 18, 2013 10:06 PM To: 'Ken Bour' ; 'Mike O'Connor' ; gnso-improvem-impl-sc@icann.org Cc: 'Hoggarth, Robert (ICANN)' Subject: RE: [gnso-improvem-impl-sc] Draft v2-WG Self-Assessment Questionnaire

Dear Ken,

Thank you for the excellent efforts. In following your various email exchanges with Committee members I noted (from below) the following:

3. The PDP is a complex, multi-step process in its own right and probably should be evaluated exhaustively, but separately, via survey or other method.

On the last point, we were looking for a set of questions that a WG might address collectively vs. individually. Perhaps a separate questionnaire could be developed that would delve deeply into the PDP methodology itself and could be administered to all or a random sample of appropriate WGs.

Most of us on this list will agree with you that PDPs are complex. For my part, I would like us to keep our eye on the ball vis-à-vis establishing an assessment questionnaire that is more one-size-fits-all as opposed to different questionnaires for different purposes. Better to try to compare apples to apples as best we can.

PDP methodology is pushing the limits, in my view, in terms of what we are looking for now.

My two cents (as a member, rather than Chair).

Thanks,

RA

Ron Andruff RNA Partners www.rnapartners.com

From: owner-gnso-improvem-impl-sc@icann.org [mailto:owner-gnso-improvem-impl-sc@icann.org] On Behalf Of Ken Bour Sent: Tuesday, June 18, 2013 15:05 To: 'Mike O'Connor'; gnso-improvem-impl-sc@icann.org Cc: 'Hoggarth, Robert (ICANN)' Subject: RE: [gnso-improvem-impl-sc] Draft v2-WG Self-Assessment Questionnaire

Mikey:

Based upon the way I have designed the WG Self-Assessment, the PROCESS dimension is intended to refer to the WG’s internal operations (norms, logistics, decision-making, etc.). In that framework, I would consider the PDP to be an INPUT to the WG (imposed methodology) and, based upon your earlier comments, I attempted to cover it generically in the 1st question of Section II. If a WG member found that the PDP (or any other requirement/constraint) was detrimental to the team’s ability to accomplish its mission, he/she could address it in Section II of the questionnaire.

I would recommend that the WG Self-Assessment instrument not be a means for evaluating the PDP (per se) for three reasons:

1) The WG Guidelines and Charter Template (source documents) do not specifically integrate the PDP within the WG process scope; rather, the PDP is specified as part of the ICANN Bylaws.

2) Not all WGs deal with PDP issues, which would mean that any such questions would have to be skipped for some percentage of respondents. The WG Guidelines mention the PDP, but only as an example of the type of methodology that may be imposed upon a WG chartered to address a domain name policy issue.

3) The PDP is a complex, multi-step process in its own right and probably should be evaluated exhaustively, but separately, via survey or other method.

On the last point, we were looking for a set of questions that a WG might address collectively vs. individually. Perhaps a separate questionnaire could be developed that would delve deeply into the PDP methodology itself and could be administered to all or a random sample of appropriate WGs.

Ken

From: owner-gnso-improvem-impl-sc@icann.org [mailto:owner-gnso-improvem-impl-sc@icann.org] On Behalf Of Mike O'Connor Sent: Friday, June 14, 2013 11:12 AM To: gnso-improvem-impl-sc@icann.org Cc: Hoggarth, Robert (ICANN); Ken Bour Subject: Re: [gnso-improvem-impl-sc] Draft v2-WG Self-Assessment Questionnaire

hi Ken,

see? sometimes i'm not so lightning fast in my replies.

this is looking really good. i've got just a little bit i'd like to see us work into the "Processes" section. is there a way to get a question or two in there that gives participants a chance to talk about the PDP process? we've got decision-making methodology (consensus) in there already, maybe this question/LO is closely related to that? looking back at the projects that the SCI is working on (method to suspend/end a WG is a good example), i think some feedback on the *structure* of the PDP would be helpful.

for example, we have a lot of comment-periods (and subsequent reviews of those comments) built into the PDP right now. Marika laid all those end to end one time and came up with a minimum time to get through a PDP that's pretty long. it would be nice to start getting feedback from WG participants as to whether they felt that those were all needed and whether they were helpful to the work of the WG. i'm seeing a transition in the way that WG's review those comments. at first, the review felt like a burden that we had to get through because it was a requirement imposed on us by the PD. more recently those comment-reviews have been a really good source of discussion-points and preliminary language that we've woven into initial and final reports.

if we stick with that example, there's a process-definition dimension (have we got the right number of comment-periods?) and a process-effectiveness dimension (did the WG effectively make use of those comments in doing their analysis?). both are important. we don't want to change a good process that's being badly carried out, in that case we want to improve the effectiveness of the WG participants. we DO want to review a bad process even though the WG has implemented it well if that badly-defined process is hurting effectiveness and timeliness.

one reason i picked that particular example is because there's a lot of pressure to become "more agile" in the PDP right now. what this often turns into is a shorthand for "do it faster!" and, since the comment/review cycles are fixed/required that sometimes means that the only place to shorten the process is the deliberative portion of the WG's work. it would be nice to be able to get some data from participants that might give an early indication that something needs to change there. that analysis would also be helpful in the "let's not break/bypass the bottom-up process" and "policy vs implementation" conversations that are going on at the moment.

thanks Ken -- really like where this is headed.

mikey

On Jun 12, 2013, at 10:42 AM, Ken Bour <ken.bour@verizon.net> wrote:

SCI Members:

I have been thinking about the topics raised by Mikey and Avri in their first impressions of the Draft Self-Assessment Questionnaire. Something was definitely missing from the original formulation, which led to an “Aha!” moment spurring me to create a second design which, I hope, addresses a few of the comments expressed thus far.

I have reconstituted the questionnaire (still five sections but renamed), reorganized some of the original questions, and added a few new ones. In order not to lose track of the first iteration, I made a completely new page and will henceforth house all questionnaire versions under a new heading: Questionnaire Drafts/Versions (https://community.icann.org/x/ai5-Ag).

In this new Draft v2 iteration (https://community.icann.org/x/bC5-Ag), I am attempting to take into consideration the dimension raised by Mikey that was only partially accommodated in Draft v1. To evaluate any dynamic system, we could subdivide it into three basic or core components: Inputs → Processes → Outputs. In the first version, I captured many of the processes, the outputs, but only a few of the inputs, namely, team member representativeness, tools, and outside experts. I did not ask about the other critical inputs that impinge upon the success of a WG, e.g., its charter/mission (including time or other constraints) and team memberexpertise. That led to a reconceptualization of the external resources questions into three buckets: administrative, technical, and human.

This Draft v2 also shifts the rating scale to 7 points and adds “Background Contributor” to the Role list as suggested by Avri.

I thank Mikey and Avri for their ideas/suggestions and look forward to additional feedback from the team…

Ken

PHONE: 651-647-6109, FAX: 866-280-2356, WEB: www.haven2.com, HANDLE: OConnorStP (ID for Twitter, Facebook, LinkedIn, etc.)

PHONE: 651-647-6109, FAX: 866-280-2356, WEB: www.haven2.com, HANDLE: OConnorStP (ID for Twitter, Facebook, LinkedIn, etc.)

Now I am really confused about the RFP. I thought the SCI was supposed to review the new PDP process as an "Improvement Implementation issue" after sufficient time has passed and that there was a feeling that sufficient time has not passed. Anne Anne E. Aikman-Scalese Of Counsel Lewis and Roca LLP • Suite 700 One South Church Avenue • Tucson, Arizona 85701-1611 Tel (520) 629-4428 • Fax (520) 879-4725 AAikman@LRLaw.com • www.LewisandRoca.com/Aikman P Please consider the environment before printing this e-mail. This e-mail contains legally privileged and confidential information intended only for the individual or entity named within the message. If the reader of this message is not the intended recipient, or the agent responsible to deliver it to the intended recipient, you are hereby notified that any review, dissemination, distribution or copying of this communication is prohibited. If this communication was received in error, please notify us by reply e-mail and delete the original message. -----Original Message----- From: owner-gnso-improvem-impl-sc@icann.org [mailto:owner-gnso-improvem-impl-sc@icann.org] On Behalf Of Mike O'Connor Sent: Thursday, June 20, 2013 2:01 PM To: gnso-improvem-impl-sc@icann.org Cc: Ken Bour; Hoggarth, Robert (ICANN) Subject: Re: [gnso-improvem-impl-sc] Draft v2-WG Self-Assessment Questionnaire let me take another run at my questions. i think by doing that, i can tidy things up a bit. starting off with our *current* charter: "The SCI will be responsible for reviewing and assessing the effective functioning of recommendations provided by the Operational Steering Committee (OSC) and Policy Process Steering Committee (PPSC) and approved by the GNSO Council. my question 1 is too broad -- we're supposed to "review and assess", not "fine tune" the way i posed the question. so what if we narrow the question to 1 aren't we supposed to review and assess the new PDP that came out of the committees that preceded us? i bet Wolf-Urich will now nod. then, my next question would become 2 wouldn't it be helpful to gather some insights about the rough edges of the new PDP that we're supposed to review and assess? again, i bet that would cheer Wolf-Urich up a bit. my third and fourth questions could then be reworked like this 3 wouldn't the pioneers who have participated as members of Working Groups operating under the revised process be a good group to ask? 4 wouldn't this questionnaire be a good place to do that? how does that go down? better? Avri, that RFP from the ATRT2 is fascinating -- and offers some collaboration and coordination opportunities with the upcoming PDP on Policy and Implementation. surely there's a way that the two efforts can leverage each other. should we drag that idea into the drafting team for that WG? thanks for sharing all that. really good stuff. mikey On Jun 20, 2013, at 3:33 PM, Avri Doria <avri@acm.org> wrote:

Hi,

In many ways, I actually agree with Mikey and beleive that especially area 2 is within the mandate we currently have. I even beleive the SCI has the ability to recommend changes to the PDP process if it encountered problems in any of the issues sent to it or found something in the periodic reviews of the various mechanisms we have never yet done.

But, be that as it may, I wanted to let you know that the ATRT2 has made a decision to put out a RFP for a Independent Consultant to do a study inot the multistakeholder effectiveness of the GNSO PDP.

A draft of the proposal* can be found at: http://mm.icann.org/pipermail/atrt2/2013/000482.html

At today's ATRT2 meeting we discussed some few changes to the wording, but I beleive we will be putting this out to bid soon.

avri

* No, I did not initiate this proposal. That was Alan Greenberg, but many of the members of the ATRT2 thought this was something that needed to be done. I did not think it was time for this yet, as the new PDP methodology is still young.

On 20 Jun 2013, at 11:00, WUKnoben wrote:

...

Let me try to recall the pioneers¡¯ thinking was or how I¡¯m interpreting it. 6¦1 aren't we supposed to be addressing fine tuning of of the new PDP that came out of the committees that preceded us? No. We don¡¯t invent new rules even less important ones.

2. wouldn't it be helpful to gather some insights about rough edges of the PDP that could use improvement?

I think this mandate should be given by the council to a special team (maybe the ¡°old¡± PDP WG)

3. wouldn't the pioneers who have gone through that process be a good group to ask?

Agreed (see 2.)

4. wouldn't this questionnaire be a good place to do that?

Yes but under a council mandate.

I could agree to all of your questions, Mikey, to move things towards improvement. But the SCI hasn¡¯t the know-how to develop PDP recommendations ¨C also some members are extremely knoledgeable.

Best regards

Wolf-Ulrich

From: Mike O'Connor Sent: Thursday, June 20, 2013 2:30 PM To: WUKnoben Cc: Ron Andruff ; 'Ken Bour' ; gnso-improvem-impl-sc@icann.org ; 'Hoggarth, Robert (ICANN)' Subject: Re: [gnso-improvem-impl-sc] Draft v2-WG Self-Assessment Questionnaire

hi all,

i don't have super-strong feelings about this. but¡­

i am still trying to figure out what our mission is. aren't we supposed to be addressing fine tuning of of the new PDP that came out of the committees that preceded us? wouldn't it be helpful to gather some insights about rough edges of the PDP that could use improvement? wouldn't the pioneers who have gone through that process be a good group to ask? wouldn't this questionnaire be a good place to do that?

confusedly yours,

mikey

On Jun 19, 2013, at 6:50 AM, WUKnoben <wolf-ulrich.knoben@t-online.de> wrote:

...

I would agree with Ron.

The new PDP is still too fresh as to start developing questions. The experience is with the WGs. If they point to questionnable rules then we should pick it up and try to find answers. I think a more general self-assessment questionnaire should be applicable.

Best regards

Wolf-Ulrich

From: Ron Andruff Sent: Tuesday, June 18, 2013 10:06 PM To: 'Ken Bour' ; 'Mike O'Connor' ; gnso-improvem-impl-sc@icann.org Cc: 'Hoggarth, Robert (ICANN)' Subject: RE: [gnso-improvem-impl-sc] Draft v2-WG Self-Assessment Questionnaire

Dear Ken,

Thank you for the excellent efforts. In following your various email exchanges with Committee members I noted (from below) the following:

3. The PDP is a complex, multi-step process in its own right and probably should be evaluated exhaustively, but separately, via survey or other method.

On the last point, we were looking for a set of questions that a WG might address collectively vs. individually. Perhaps a separate questionnaire could be developed that would delve deeply into the PDP methodology itself and could be administered to all or a random sample of appropriate WGs.

Most of us on this list will agree with you that PDPs are complex. For my part, I would like us to keep our eye on the ball vis-¨¤-vis establishing an assessment questionnaire that is more one-size-fits-all as opposed to different questionnaires for different purposes. Better to try to compare apples to apples as best we can.

PDP methodology is pushing the limits, in my view, in terms of what we are looking for now.

My two cents (as a member, rather than Chair).

Thanks,

RA

Ron Andruff RNA Partners www.rnapartners.com

From: owner-gnso-improvem-impl-sc@icann.org [mailto:owner-gnso-improvem-impl-sc@icann.org] On Behalf Of Ken Bour Sent: Tuesday, June 18, 2013 15:05 To: 'Mike O'Connor'; gnso-improvem-impl-sc@icann.org Cc: 'Hoggarth, Robert (ICANN)' Subject: RE: [gnso-improvem-impl-sc] Draft v2-WG Self-Assessment Questionnaire

Mikey:

Based upon the way I have designed the WG Self-Assessment, the PROCESS dimension is intended to refer to the WG¡¯s internal operations (norms, logistics, decision-making, etc.). In that framework, I would consider the PDP to be an INPUT to the WG (imposed methodology) and, based upon your earlier comments, I attempted to cover it generically in the 1st question of Section II. If a WG member found that the PDP (or any other requirement/constraint) was detrimental to the team¡¯s ability to accomplish its mission, he/she could address it in Section II of the questionnaire.

I would recommend that the WG Self-Assessment instrument not be a means for evaluating the PDP (per se) for three reasons:

1) The WG Guidelines and Charter Template (source documents) do not specifically integrate the PDP within the WG process scope; rather, the PDP is specified as part of the ICANN Bylaws.

2) Not all WGs deal with PDP issues, which would mean that any such questions would have to be skipped for some percentage of respondents. The WG Guidelines mention the PDP, but only as an example of the type of methodology that may be imposed upon a WG chartered to address a domain name policy issue.

3) The PDP is a complex, multi-step process in its own right and probably should be evaluated exhaustively, but separately, via survey or other method.

On the last point, we were looking for a set of questions that a WG might address collectively vs. individually. Perhaps a separate questionnaire could be developed that would delve deeply into the PDP methodology itself and could be administered to all or a random sample of appropriate WGs.

Ken

From: owner-gnso-improvem-impl-sc@icann.org [mailto:owner-gnso-improvem-impl-sc@icann.org] On Behalf Of Mike O'Connor Sent: Friday, June 14, 2013 11:12 AM To: gnso-improvem-impl-sc@icann.org Cc: Hoggarth, Robert (ICANN); Ken Bour Subject: Re: [gnso-improvem-impl-sc] Draft v2-WG Self-Assessment Questionnaire

hi Ken,

see? sometimes i'm not so lightning fast in my replies.

this is looking really good. i've got just a little bit i'd like to see us work into the "Processes" section. is there a way to get a question or two in there that gives participants a chance to talk about the PDP process? we've got decision-making methodology (consensus) in there already, maybe this question/LO is closely related to that? looking back at the projects that the SCI is working on (method to suspend/end a WG is a good example), i think some feedback on the *structure* of the PDP would be helpful.

for example, we have a lot of comment-periods (and subsequent reviews of those comments) built into the PDP right now. Marika laid all those end to end one time and came up with a minimum time to get through a PDP that's pretty long. it would be nice to start getting feedback from WG participants as to whether they felt that those were all needed and whether they were helpful to the work of the WG. i'm seeing a transition in the way that WG's review those comments. at first, the review felt like a burden that we had to get through because it was a requirement imposed on us by the PD. more recently those comment-reviews have been a really good source of discussion-points and preliminary language that we've woven into initial and final reports.

if we stick with that example, there's a process-definition dimension (have we got the right number of comment-periods?) and a process-effectiveness dimension (did the WG effectively make use of those comments in doing their analysis?). both are important. we don't want to change a good process that's being badly carried out, in that case we want to improve the effectiveness of the WG participants. we DO want to review a bad process even though the WG has implemented it well if that badly-defined process is hurting effectiveness and timeliness.

one reason i picked that particular example is because there's a lot of pressure to become "more agile" in the PDP right now. what this often turns into is a shorthand for "do it faster!" and, since the comment/review cycles are fixed/required that sometimes means that the only place to shorten the process is the deliberative portion of the WG's work. it would be nice to be able to get some data from participants that might give an early indication that something needs to change there. that analysis would also be helpful in the "let's not break/bypass the bottom-up process" and "policy vs implementation" conversations that are going on at the moment.

thanks Ken -- really like where this is headed.

mikey

On Jun 12, 2013, at 10:42 AM, Ken Bour <ken.bour@verizon.net> wrote:

SCI Members:

I have been thinking about the topics raised by Mikey and Avri in their first impressions of the Draft Self-Assessment Questionnaire. Something was definitely missing from the original formulation, which led to an ¡°Aha!¡± moment spurring me to create a second design which, I hope, addresses a few of the comments expressed thus far.

I have reconstituted the questionnaire (still five sections but renamed), reorganized some of the original questions, and added a few new ones. In order not to lose track of the first iteration, I made a completely new page and will henceforth house all questionnaire versions under a new heading: Questionnaire Drafts/Versions (https://community.icann.org/x/ai5-Ag).

In this new Draft v2 iteration (https://community.icann.org/x/bC5-Ag), I am attempting to take into consideration the dimension raised by Mikey that was only partially accommodated in Draft v1. To evaluate any dynamic system, we could subdivide it into three basic or core components: Inputs ¡ú Processes ¡ú Outputs. In the first version, I captured many of the processes, the outputs, but only a few of the inputs, namely, team member representativeness, tools, and outside experts. I did not ask about the other critical inputs that impinge upon the success of a WG, e.g., its charter/mission (including time or other constraints) and team memberexpertise. That led to a reconceptualization of the external resources questions into three buckets: administrative, technical, and human.

This Draft v2 also shifts the rating scale to 7 points and adds ¡°Background Contributor¡± to the Role list as suggested by Avri.

I thank Mikey and Avri for their ideas/suggestions and look forward to additional feedback from the team¡­

Ken

PHONE: 651-647-6109, FAX: 866-280-2356, WEB: www.haven2.com, HANDLE: OConnorStP (ID for Twitter, Facebook, LinkedIn, etc.)

PHONE: 651-647-6109, FAX: 866-280-2356, WEB: www.haven2.com, HANDLE: OConnorStP (ID for Twitter, Facebook, LinkedIn, etc.)

PHONE: 651-647-6109, FAX: 866-280-2356, WEB: www.haven2.com, HANDLE: OConnorStP (ID for Twitter, Facebook, LinkedIn, etc.) ---------------------- For more information about Lewis and Roca LLP, please go to www.lewisandroca.com. Phoenix (602)262-5311 Reno (775)823-2900 Tucson (520)622-2090 Albuquerque (505)764-5400 Las Vegas (702)949-8200 Silicon Valley (650)391-1380 This message is intended only for the use of the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to the sender of this E-Mail by return E-Mail or by telephone. In accordance with Internal Revenue Service Circular 230, we advise you that if this email contains any tax advice, such tax advice was not intended or written to be used, and it cannot be used, by any taxpayer for the purpose of avoiding penalties that may be imposed on the taxpayer

I tend to nod - not exactly knowing whether midnight (my time) is the driver. Nevertheless I think we should start framing this work and then bring it before the council in advance to digging into details. Good night Wolf-Ulrich -----Ursprüngliche Nachricht----- From: Mike O'Connor Sent: Thursday, June 20, 2013 11:01 PM To: gnso-improvem-impl-sc@icann.org Cc: Ken Bour ; Hoggarth, Robert (ICANN) Subject: Re: [gnso-improvem-impl-sc] Draft v2-WG Self-Assessment Questionnaire let me take another run at my questions. i think by doing that, i can tidy things up a bit. starting off with our *current* charter: "The SCI will be responsible for reviewing and assessing the effective functioning of recommendations provided by the Operational Steering Committee (OSC) and Policy Process Steering Committee (PPSC) and approved by the GNSO Council. my question 1 is too broad -- we're supposed to "review and assess", not "fine tune" the way i posed the question. so what if we narrow the question to 1 aren't we supposed to review and assess the new PDP that came out of the committees that preceded us? i bet Wolf-Urich will now nod. then, my next question would become 2 wouldn't it be helpful to gather some insights about the rough edges of the new PDP that we're supposed to review and assess? again, i bet that would cheer Wolf-Urich up a bit. my third and fourth questions could then be reworked like this 3 wouldn't the pioneers who have participated as members of Working Groups operating under the revised process be a good group to ask? 4 wouldn't this questionnaire be a good place to do that? how does that go down? better? Avri, that RFP from the ATRT2 is fascinating -- and offers some collaboration and coordination opportunities with the upcoming PDP on Policy and Implementation. surely there's a way that the two efforts can leverage each other. should we drag that idea into the drafting team for that WG? thanks for sharing all that. really good stuff. mikey On Jun 20, 2013, at 3:33 PM, Avri Doria <avri@acm.org> wrote:

Hi,

In many ways, I actually agree with Mikey and beleive that especially area 2 is within the mandate we currently have. I even beleive the SCI has the ability to recommend changes to the PDP process if it encountered problems in any of the issues sent to it or found something in the periodic reviews of the various mechanisms we have never yet done.

But, be that as it may, I wanted to let you know that the ATRT2 has made a decision to put out a RFP for a Independent Consultant to do a study inot the multistakeholder effectiveness of the GNSO PDP.

A draft of the proposal* can be found at: http://mm.icann.org/pipermail/atrt2/2013/000482.html

At today's ATRT2 meeting we discussed some few changes to the wording, but I beleive we will be putting this out to bid soon.

avri

* No, I did not initiate this proposal. That was Alan Greenberg, but many of the members of the ATRT2 thought this was something that needed to be done. I did not think it was time for this yet, as the new PDP methodology is still young.

On 20 Jun 2013, at 11:00, WUKnoben wrote:

...

Let me try to recall the pioneers¡¯ thinking was or how I¡¯m interpreting it. 6¦1 aren't we supposed to be addressing fine tuning of of the new PDP that came out of the committees that preceded us? No. We don¡¯t invent new rules even less important ones.

2. wouldn't it be helpful to gather some insights about rough edges of the PDP that could use improvement?

I think this mandate should be given by the council to a special team (maybe the ¡°old¡± PDP WG)

3. wouldn't the pioneers who have gone through that process be a good group to ask?

Agreed (see 2.)

4. wouldn't this questionnaire be a good place to do that?

Yes but under a council mandate.

I could agree to all of your questions, Mikey, to move things towards improvement. But the SCI hasn¡¯t the know-how to develop PDP recommendations ¨C also some members are extremely knoledgeable.

Best regards

Wolf-Ulrich

From: Mike O'Connor Sent: Thursday, June 20, 2013 2:30 PM To: WUKnoben Cc: Ron Andruff ; 'Ken Bour' ; gnso-improvem-impl-sc@icann.org ; 'Hoggarth, Robert (ICANN)' Subject: Re: [gnso-improvem-impl-sc] Draft v2-WG Self-Assessment Questionnaire

hi all,

i don't have super-strong feelings about this. but¡­

i am still trying to figure out what our mission is. aren't we supposed to be addressing fine tuning of of the new PDP that came out of the committees that preceded us? wouldn't it be helpful to gather some insights about rough edges of the PDP that could use improvement? wouldn't the pioneers who have gone through that process be a good group to ask? wouldn't this questionnaire be a good place to do that?

confusedly yours,

mikey

On Jun 19, 2013, at 6:50 AM, WUKnoben <wolf-ulrich.knoben@t-online.de> wrote:

...

I would agree with Ron.

The new PDP is still too fresh as to start developing questions. The experience is with the WGs. If they point to questionnable rules then we should pick it up and try to find answers. I think a more general self-assessment questionnaire should be applicable.

Best regards

Wolf-Ulrich

From: Ron Andruff Sent: Tuesday, June 18, 2013 10:06 PM To: 'Ken Bour' ; 'Mike O'Connor' ; gnso-improvem-impl-sc@icann.org Cc: 'Hoggarth, Robert (ICANN)' Subject: RE: [gnso-improvem-impl-sc] Draft v2-WG Self-Assessment Questionnaire

Dear Ken,

Thank you for the excellent efforts. In following your various email exchanges with Committee members I noted (from below) the following:

3. The PDP is a complex, multi-step process in its own right and probably should be evaluated exhaustively, but separately, via survey or other method.

On the last point, we were looking for a set of questions that a WG might address collectively vs. individually. Perhaps a separate questionnaire could be developed that would delve deeply into the PDP methodology itself and could be administered to all or a random sample of appropriate WGs.

Most of us on this list will agree with you that PDPs are complex. For my part, I would like us to keep our eye on the ball vis-¨¤-vis establishing an assessment questionnaire that is more one-size-fits-all as opposed to different questionnaires for different purposes. Better to try to compare apples to apples as best we can.

PDP methodology is pushing the limits, in my view, in terms of what we are looking for now.

My two cents (as a member, rather than Chair).

Thanks,

RA

Ron Andruff RNA Partners www.rnapartners.com

From: owner-gnso-improvem-impl-sc@icann.org [mailto:owner-gnso-improvem-impl-sc@icann.org] On Behalf Of Ken Bour Sent: Tuesday, June 18, 2013 15:05 To: 'Mike O'Connor'; gnso-improvem-impl-sc@icann.org Cc: 'Hoggarth, Robert (ICANN)' Subject: RE: [gnso-improvem-impl-sc] Draft v2-WG Self-Assessment Questionnaire

Mikey:

Based upon the way I have designed the WG Self-Assessment, the PROCESS dimension is intended to refer to the WG¡¯s internal operations (norms, logistics, decision-making, etc.). In that framework, I would consider the PDP to be an INPUT to the WG (imposed methodology) and, based upon your earlier comments, I attempted to cover it generically in the 1st question of Section II. If a WG member found that the PDP (or any other requirement/constraint) was detrimental to the team¡¯s ability to accomplish its mission, he/she could address it in Section II of the questionnaire.

I would recommend that the WG Self-Assessment instrument not be a means for evaluating the PDP (per se) for three reasons:

1) The WG Guidelines and Charter Template (source documents) do not specifically integrate the PDP within the WG process scope; rather, the PDP is specified as part of the ICANN Bylaws.

2) Not all WGs deal with PDP issues, which would mean that any such questions would have to be skipped for some percentage of respondents. The WG Guidelines mention the PDP, but only as an example of the type of methodology that may be imposed upon a WG chartered to address a domain name policy issue.

3) The PDP is a complex, multi-step process in its own right and probably should be evaluated exhaustively, but separately, via survey or other method.

On the last point, we were looking for a set of questions that a WG might address collectively vs. individually. Perhaps a separate questionnaire could be developed that would delve deeply into the PDP methodology itself and could be administered to all or a random sample of appropriate WGs.

Ken

From: owner-gnso-improvem-impl-sc@icann.org [mailto:owner-gnso-improvem-impl-sc@icann.org] On Behalf Of Mike O'Connor Sent: Friday, June 14, 2013 11:12 AM To: gnso-improvem-impl-sc@icann.org Cc: Hoggarth, Robert (ICANN); Ken Bour Subject: Re: [gnso-improvem-impl-sc] Draft v2-WG Self-Assessment Questionnaire

hi Ken,

see? sometimes i'm not so lightning fast in my replies.

this is looking really good. i've got just a little bit i'd like to see us work into the "Processes" section. is there a way to get a question or two in there that gives participants a chance to talk about the PDP process? we've got decision-making methodology (consensus) in there already, maybe this question/LO is closely related to that? looking back at the projects that the SCI is working on (method to suspend/end a WG is a good example), i think some feedback on the *structure* of the PDP would be helpful.

for example, we have a lot of comment-periods (and subsequent reviews of those comments) built into the PDP right now. Marika laid all those end to end one time and came up with a minimum time to get through a PDP that's pretty long. it would be nice to start getting feedback from WG participants as to whether they felt that those were all needed and whether they were helpful to the work of the WG. i'm seeing a transition in the way that WG's review those comments. at first, the review felt like a burden that we had to get through because it was a requirement imposed on us by the PD. more recently those comment-reviews have been a really good source of discussion-points and preliminary language that we've woven into initial and final reports.

if we stick with that example, there's a process-definition dimension (have we got the right number of comment-periods?) and a process-effectiveness dimension (did the WG effectively make use of those comments in doing their analysis?). both are important. we don't want to change a good process that's being badly carried out, in that case we want to improve the effectiveness of the WG participants. we DO want to review a bad process even though the WG has implemented it well if that badly-defined process is hurting effectiveness and timeliness.

one reason i picked that particular example is because there's a lot of pressure to become "more agile" in the PDP right now. what this often turns into is a shorthand for "do it faster!" and, since the comment/review cycles are fixed/required that sometimes means that the only place to shorten the process is the deliberative portion of the WG's work. it would be nice to be able to get some data from participants that might give an early indication that something needs to change there. that analysis would also be helpful in the "let's not break/bypass the bottom-up process" and "policy vs implementation" conversations that are going on at the moment.

thanks Ken -- really like where this is headed.

mikey

On Jun 12, 2013, at 10:42 AM, Ken Bour <ken.bour@verizon.net> wrote:

SCI Members:

I have been thinking about the topics raised by Mikey and Avri in their first impressions of the Draft Self-Assessment Questionnaire. Something was definitely missing from the original formulation, which led to an ¡°Aha!¡± moment spurring me to create a second design which, I hope, addresses a few of the comments expressed thus far.

I have reconstituted the questionnaire (still five sections but renamed), reorganized some of the original questions, and added a few new ones. In order not to lose track of the first iteration, I made a completely new page and will henceforth house all questionnaire versions under a new heading: Questionnaire Drafts/Versions (https://community.icann.org/x/ai5-Ag).

In this new Draft v2 iteration (https://community.icann.org/x/bC5-Ag), I am attempting to take into consideration the dimension raised by Mikey that was only partially accommodated in Draft v1. To evaluate any dynamic system, we could subdivide it into three basic or core components: Inputs ¡ú Processes ¡ú Outputs. In the first version, I captured many of the processes, the outputs, but only a few of the inputs, namely, team member representativeness, tools, and outside experts. I did not ask about the other critical inputs that impinge upon the success of a WG, e.g., its charter/mission (including time or other constraints) and team memberexpertise. That led to a reconceptualization of the external resources questions into three buckets: administrative, technical, and human.

This Draft v2 also shifts the rating scale to 7 points and adds ¡°Background Contributor¡± to the Role list as suggested by Avri.

I thank Mikey and Avri for their ideas/suggestions and look forward to additional feedback from the team¡­

Ken

PHONE: 651-647-6109, FAX: 866-280-2356, WEB: www.haven2.com, HANDLE: OConnorStP (ID for Twitter, Facebook, LinkedIn, etc.)

PHONE: 651-647-6109, FAX: 866-280-2356, WEB: www.haven2.com, HANDLE: OConnorStP (ID for Twitter, Facebook, LinkedIn, etc.)

PHONE: 651-647-6109, FAX: 866-280-2356, WEB: www.haven2.com, HANDLE: OConnorStP (ID for Twitter, Facebook, LinkedIn, etc.)

Dear all, The next meeting is scheduled for 02 July 2013 at 19:00 UTC for 1 hour (12:00 PST , 15:00 EST, 20:00 London, 21:00 CET). It is on the wiki at: https://community.icann.org/display/gnsosci/2+July+2013. Best regards, Julie From: <Aikman-Scalese>, Anne <AAikman@lrlaw.com> Date: Wednesday, June 26, 2013 5:00 PM To: Ron Andruff <randruff@rnapartners.com>, 'Ken Bour' <ken.bour@verizon.net>, Mike O'Conner <mike@haven2.com>, "gnso-improvem-impl-sc@icann.org" <gnso-improvem-impl-sc@icann.org> Cc: Robert Hoggarth <robert.hoggarth@icann.org> Subject: RE: [gnso-improvem-impl-sc] Dear all, I am on a plane and trying to access Adobe Connect via in-air WiFI. We'll see what happens. Anne Anne E. Aikman-Scalese Of Counsel Lewis and Roca LLP € Suite 700 One South Church Avenue € Tucson, Arizona 85701-1611 Tel (520) 629-4428 € Fax (520) 879-4725 AAikman@LRLaw.com € www.LewisandRoca.com/Aikman <http://www.lewisandroca.com/Aikman> P Please consider the environment before printing this e-mail. This e-mail contains legally privileged and confidential information intended only for the individual or entity named within the message. If the reader of this message is not the intended recipient, or the agent responsible to deliver it to the intended recipient, you are hereby notified that any review, dissemination, distribution or copying of this communication is prohibited. If this communication was received in error, please notify us by reply e-mail and delete theoriginal message. For more information about Lewis and Roca LLP, please go to www.lewisandroca.com <http://www.lewisandroca.com/> . Phoenix (602)262-5311 Reno (775)823-2900 Tucson (520)622-2090 Albuquerque (505)764-5400 Las Vegas (702)949-8200 Silicon Valley (650)391-1380 This message is intended only for the use of the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to the sender of this E-Mail by return E-Mail or by telephone. In accordance with Internal Revenue Service Circular 230, we advise you that if this email contains any tax advice, such tax advice was not intended or written to be used, and it cannot be used, by any taxpayer for the purpose of avoiding penalties that may be imposed on the taxpayer.