Re: [Gnso-rpm-data] [Ext] RE: Advice Needed: Complications Regarding Saving Survey Responses
Hello Kristine and all, Please see below the responses from Analysis Group to Kristine’s question. Analysis Group has been making progress on the alternative options/workarounds, and will be able to provide another update shortly. We will keep the Sub Team posted. Please let us know if you have any further comments/questions. Best Regards, Mary, Julie, Ariel & Berry == Kristine: If we do NOT use passwords, than anyone with the link can take the survey, right? AG: That is correct. The survey will be publicly available, and anyone with the link will be able to take the survey. Kristine: So if DO use passwords and we allow users to create them, what is the risk if they don’t create “unique” passwords? Because I’m not seeing any risk to just letting them create a password. This is just a little placeholder - a dog-earing of the electronic page if you will, it’s not performing a security function. (Right??) AG: If we allow users to create passwords, the survey will remain publicly available and anyone with the link will be able to take the survey. Users who create passwords will be able to save their responses and return to them later. The concern involved with non-unique passwords is that, because the password will be the only way for our programming to identify which survey record to re-open (i.e., users will not be creating unique user IDs and we will not be collecting IP address due to GDPR issues, so saved responses will be cataloged only by their password), if any users create the same password, their survey responses will appear in the survey data as if they came from one respondent when in fact there are two distinct respondents answering the questions. This can cause consistency issues, since we will be unaware of instances where this happens. For example, if a user chooses the password "1234" and completes questions 1-10 and leaves the survey, and then a second user attempts to create a password "1234" -- the second user will unwittingly log-in to the first user's survey and pick-up the survey at question 11. We will be unaware of this happening, but may find that the data appear inconsistent and/or invalid during our analysis of the data. Kristine: Or are you saying that the password is really just a token and if someone else arbitrarily creates the same one (like “1234” or “password”) we’ll have a mess? AG: Yes, as described above, we run the risk of overwriting or combining responses in a way that would cause responses to be impossible to untangle. To continue to follow up with our progress on the passwords: One solution that we would like to suggest is that users can be allowed to create a password, and we will embed a page visit counter next to the password field. We will suggest that users use the number from the counter for their password. This will guarantee that passwords will be unique for each user, although the pattern of the password generation may be a bit transparent. We have identified a way to implement this solution but will need to be careful to make sure the directions for password creation (for first-time visitors) and log-in (for return visitors) are very clear. Unfortunately, the password creation and log-in fields appear as one field in the survey. We are unable to place them on separate pages (i.e., create a log-in page for first-time users and a separate log-in page for return users). We are also investigating issues related to cookies, and expect to have resolved that issue and any related implementation issues by the end of this weekend if not sooner. We can send you another update tomorrow as we continue to make progress. There is a trade-off to consider regarding a portion of potential respondents who choose not to complete the survey if they learn that cookies are being sent to their computer. From: "Dorrain, Kristine" <dorraink@amazon.com> Date: Thursday, August 16, 2018 at 7:55 PM To: Ariel Liang <ariel.liang@icann.org>, "gnso-rpm-data@icann.org" <gnso-rpm-data@icann.org>, "Dorrain, Kristine" <dorraink@amazon.com> Subject: [Ext] RE: [Gnso-rpm-data] Advice Needed: Complications Regarding Saving Survey Responses Thanks for this Ariel: My initial thought is this. If we do NOT use passwords, than anyone with the link can take the survey, right? So if DO use passwords and we allow users to create them, what is the risk if they don’t create “unique” passwords? Because I’m not seeing any risk to just letting them create a password. This is just a little placeholder - a dog-earing of the electronic page if you will, it’s not performing a security function. (Right??) Or are you saying that the password is really just a token and if someone else arbitrarily creates the same one (like “1234” or “password”) we’ll have a mess? (ccing myself since I’m out tomorrow and I want to see the answer!) Kristine From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Ariel Liang Sent: Thursday, August 16, 2018 9:42 AM To: gnso-rpm-data@icann.org Subject: [Gnso-rpm-data] Advice Needed: Complications Regarding Saving Survey Responses Dear Data Sub Team members, Analysis Group is in process programming the surveys; the actual/potential registrant survey has already been pushed to the beta testing phase. A complication was discovered. Analysis Group informed staff that it is not possible to allow survey takers to “save” their responses and return to the surveys later to complete, unless the survey taker is given a login credential that can be validated. To create a login credential, the simplest way is for Analysis Group to obtain a list of email addresses of the individual survey takers. This, however, cannot be achieved given the outreach method that we are using to distribute the surveys (e.g., public links accessed via mailing lists, announcements, social media). Although GDD has the email contacts of registries and registrars, ICANN Org is not allowed to share the email contacts with Analysis Group directly due to the rightful use of these email contacts and associated legal/GDPR implications. Analysis Group is exploring other options/workarounds for survey takers to “save” their responses. One option is to allow users to create their own passwords, though it is difficult for Analysis Group to enforce unique password creation. They are experimenting with methods of suggesting unique passwords to users who would like to create a password. Another option is to distribute "cookies" with the survey, which will allow respondents to return to surveys that they have started. This option requires Analysis Group to explore any legal issues involved in the use of cookies. In summary, these options may raise technical and privacy issues that need to be cleared internally by Analysis Group. Consequently, this complication may add to the time for the surveys to be launched. Staff are seeking clarification from AG on the ETA to solve these technical/privacy issues. QUESTION: The need to get the surveys out ASAP/as many responses as possible thus needs to be weighed against the benefit of enabling respondents to save their responses/impact to the response rate. What does the Sub Team advise? Thank you for input! Best Regards, Mary, Julie, Ariel, Berry
I think the proposal sounds like a reasonable way to move forward to give survey takers the ability to pause and return. Michael R. From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Ariel Liang Sent: Friday, August 17, 2018 6:42 AM To: Dorrain, Kristine <dorraink@amazon.com>; gnso-rpm-data@icann.org Subject: Re: [Gnso-rpm-data] [Ext] RE: Advice Needed: Complications Regarding Saving Survey Responses Hello Kristine and all, Please see below the responses from Analysis Group to Kristine’s question. Analysis Group has been making progress on the alternative options/workarounds, and will be able to provide another update shortly. We will keep the Sub Team posted. Please let us know if you have any further comments/questions. Best Regards, Mary, Julie, Ariel & Berry == Kristine: If we do NOT use passwords, than anyone with the link can take the survey, right? AG: That is correct. The survey will be publicly available, and anyone with the link will be able to take the survey. Kristine: So if DO use passwords and we allow users to create them, what is the risk if they don’t create “unique” passwords? Because I’m not seeing any risk to just letting them create a password. This is just a little placeholder - a dog-earing of the electronic page if you will, it’s not performing a security function. (Right??) AG: If we allow users to create passwords, the survey will remain publicly available and anyone with the link will be able to take the survey. Users who create passwords will be able to save their responses and return to them later. The concern involved with non-unique passwords is that, because the password will be the only way for our programming to identify which survey record to re-open (i.e., users will not be creating unique user IDs and we will not be collecting IP address due to GDPR issues, so saved responses will be cataloged only by their password), if any users create the same password, their survey responses will appear in the survey data as if they came from one respondent when in fact there are two distinct respondents answering the questions. This can cause consistency issues, since we will be unaware of instances where this happens. For example, if a user chooses the password "1234" and completes questions 1-10 and leaves the survey, and then a second user attempts to create a password "1234" -- the second user will unwittingly log-in to the first user's survey and pick-up the survey at question 11. We will be unaware of this happening, but may find that the data appear inconsistent and/or invalid during our analysis of the data. Kristine: Or are you saying that the password is really just a token and if someone else arbitrarily creates the same one (like “1234” or “password”) we’ll have a mess? AG: Yes, as described above, we run the risk of overwriting or combining responses in a way that would cause responses to be impossible to untangle. To continue to follow up with our progress on the passwords: One solution that we would like to suggest is that users can be allowed to create a password, and we will embed a page visit counter next to the password field. We will suggest that users use the number from the counter for their password. This will guarantee that passwords will be unique for each user, although the pattern of the password generation may be a bit transparent. We have identified a way to implement this solution but will need to be careful to make sure the directions for password creation (for first-time visitors) and log-in (for return visitors) are very clear. Unfortunately, the password creation and log-in fields appear as one field in the survey. We are unable to place them on separate pages (i.e., create a log-in page for first-time users and a separate log-in page for return users). We are also investigating issues related to cookies, and expect to have resolved that issue and any related implementation issues by the end of this weekend if not sooner. We can send you another update tomorrow as we continue to make progress. There is a trade-off to consider regarding a portion of potential respondents who choose not to complete the survey if they learn that cookies are being sent to their computer. From: "Dorrain, Kristine" <dorraink@amazon.com<mailto:dorraink@amazon.com>> Date: Thursday, August 16, 2018 at 7:55 PM To: Ariel Liang <ariel.liang@icann.org<mailto:ariel.liang@icann.org>>, "gnso-rpm-data@icann.org<mailto:gnso-rpm-data@icann.org>" <gnso-rpm-data@icann.org<mailto:gnso-rpm-data@icann.org>>, "Dorrain, Kristine" <dorraink@amazon.com<mailto:dorraink@amazon.com>> Subject: [Ext] RE: [Gnso-rpm-data] Advice Needed: Complications Regarding Saving Survey Responses Thanks for this Ariel: My initial thought is this. If we do NOT use passwords, than anyone with the link can take the survey, right? So if DO use passwords and we allow users to create them, what is the risk if they don’t create “unique” passwords? Because I’m not seeing any risk to just letting them create a password. This is just a little placeholder - a dog-earing of the electronic page if you will, it’s not performing a security function. (Right??) Or are you saying that the password is really just a token and if someone else arbitrarily creates the same one (like “1234” or “password”) we’ll have a mess? (ccing myself since I’m out tomorrow and I want to see the answer!) Kristine From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Ariel Liang Sent: Thursday, August 16, 2018 9:42 AM To: gnso-rpm-data@icann.org<mailto:gnso-rpm-data@icann.org> Subject: [Gnso-rpm-data] Advice Needed: Complications Regarding Saving Survey Responses Dear Data Sub Team members, Analysis Group is in process programming the surveys; the actual/potential registrant survey has already been pushed to the beta testing phase. A complication was discovered. Analysis Group informed staff that it is not possible to allow survey takers to “save” their responses and return to the surveys later to complete, unless the survey taker is given a login credential that can be validated. To create a login credential, the simplest way is for Analysis Group to obtain a list of email addresses of the individual survey takers. This, however, cannot be achieved given the outreach method that we are using to distribute the surveys (e.g., public links accessed via mailing lists, announcements, social media). Although GDD has the email contacts of registries and registrars, ICANN Org is not allowed to share the email contacts with Analysis Group directly due to the rightful use of these email contacts and associated legal/GDPR implications. Analysis Group is exploring other options/workarounds for survey takers to “save” their responses. One option is to allow users to create their own passwords, though it is difficult for Analysis Group to enforce unique password creation. They are experimenting with methods of suggesting unique passwords to users who would like to create a password. Another option is to distribute "cookies" with the survey, which will allow respondents to return to surveys that they have started. This option requires Analysis Group to explore any legal issues involved in the use of cookies. In summary, these options may raise technical and privacy issues that need to be cleared internally by Analysis Group. Consequently, this complication may add to the time for the surveys to be launched. Staff are seeking clarification from AG on the ETA to solve these technical/privacy issues. QUESTION: The need to get the surveys out ASAP/as many responses as possible thus needs to be weighed against the benefit of enabling respondents to save their responses/impact to the response rate. What does the Sub Team advise? Thank you for input! Best Regards, Mary, Julie, Ariel, Berry
Thank you, Michael! Hello everyone, Please see below an update from Analysis Group with regard to implementing methods for survey takers to save their progress. It seems that both options are promising but they have some shortcomings, which are hard to avoid. If either option is used, we likely need to make it clear in the messaging and/or survey intro about the ways for survey takers to save their progress/return to the survey. == AG: Regarding options for respondents who would like to save and revisit responses - We have identified a way to suggest unique passwords to users who wish to exercise this option. The secondary option of using cookies also continues to look promising. We are working to confirm our understanding of whether data will be collected from respondents through the use of cookies other than allowing respondents to revisit their survey responses. Both options continue to have the same shortcomings as identified earlier. For password creation, the method for creating a new password vs. using a pre-existing password for return visitors may be confusing or frustrating, and/or respondents may lose or forget their passwords. The consequence would be that some surveys may remain partially completed. Under the second option of using cookies, some respondents may not feel comfortable accepting cookies from the survey instrument, resulting in lower response rates. == Thank you, Mary, Julie, Ariel, and Berry From: "Michael Graham (ELCA)" <migraham@expedia.com> Date: Friday, August 17, 2018 at 1:01 PM To: Ariel Liang <ariel.liang@icann.org>, "Dorrain, Kristine" <dorraink@amazon.com>, "gnso-rpm-data@icann.org" <gnso-rpm-data@icann.org> Subject: RE: [Gnso-rpm-data] [Ext] RE: Advice Needed: Complications Regarding Saving Survey Responses I think the proposal sounds like a reasonable way to move forward to give survey takers the ability to pause and return. Michael R. From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Ariel Liang Sent: Friday, August 17, 2018 6:42 AM To: Dorrain, Kristine <dorraink@amazon.com>; gnso-rpm-data@icann.org Subject: Re: [Gnso-rpm-data] [Ext] RE: Advice Needed: Complications Regarding Saving Survey Responses Hello Kristine and all, Please see below the responses from Analysis Group to Kristine’s question. Analysis Group has been making progress on the alternative options/workarounds, and will be able to provide another update shortly. We will keep the Sub Team posted. Please let us know if you have any further comments/questions. Best Regards, Mary, Julie, Ariel & Berry == Kristine: If we do NOT use passwords, than anyone with the link can take the survey, right? AG: That is correct. The survey will be publicly available, and anyone with the link will be able to take the survey. Kristine: So if DO use passwords and we allow users to create them, what is the risk if they don’t create “unique” passwords? Because I’m not seeing any risk to just letting them create a password. This is just a little placeholder - a dog-earing of the electronic page if you will, it’s not performing a security function. (Right??) AG: If we allow users to create passwords, the survey will remain publicly available and anyone with the link will be able to take the survey. Users who create passwords will be able to save their responses and return to them later. The concern involved with non-unique passwords is that, because the password will be the only way for our programming to identify which survey record to re-open (i.e., users will not be creating unique user IDs and we will not be collecting IP address due to GDPR issues, so saved responses will be cataloged only by their password), if any users create the same password, their survey responses will appear in the survey data as if they came from one respondent when in fact there are two distinct respondents answering the questions. This can cause consistency issues, since we will be unaware of instances where this happens. For example, if a user chooses the password "1234" and completes questions 1-10 and leaves the survey, and then a second user attempts to create a password "1234" -- the second user will unwittingly log-in to the first user's survey and pick-up the survey at question 11. We will be unaware of this happening, but may find that the data appear inconsistent and/or invalid during our analysis of the data. Kristine: Or are you saying that the password is really just a token and if someone else arbitrarily creates the same one (like “1234” or “password”) we’ll have a mess? AG: Yes, as described above, we run the risk of overwriting or combining responses in a way that would cause responses to be impossible to untangle. To continue to follow up with our progress on the passwords: One solution that we would like to suggest is that users can be allowed to create a password, and we will embed a page visit counter next to the password field. We will suggest that users use the number from the counter for their password. This will guarantee that passwords will be unique for each user, although the pattern of the password generation may be a bit transparent. We have identified a way to implement this solution but will need to be careful to make sure the directions for password creation (for first-time visitors) and log-in (for return visitors) are very clear. Unfortunately, the password creation and log-in fields appear as one field in the survey. We are unable to place them on separate pages (i.e., create a log-in page for first-time users and a separate log-in page for return users). We are also investigating issues related to cookies, and expect to have resolved that issue and any related implementation issues by the end of this weekend if not sooner. We can send you another update tomorrow as we continue to make progress. There is a trade-off to consider regarding a portion of potential respondents who choose not to complete the survey if they learn that cookies are being sent to their computer. From: "Dorrain, Kristine" <dorraink@amazon.com<mailto:dorraink@amazon.com>> Date: Thursday, August 16, 2018 at 7:55 PM To: Ariel Liang <ariel.liang@icann.org<mailto:ariel.liang@icann.org>>, "gnso-rpm-data@icann.org<mailto:gnso-rpm-data@icann.org>" <gnso-rpm-data@icann.org<mailto:gnso-rpm-data@icann.org>>, "Dorrain, Kristine" <dorraink@amazon.com<mailto:dorraink@amazon.com>> Subject: [Ext] RE: [Gnso-rpm-data] Advice Needed: Complications Regarding Saving Survey Responses Thanks for this Ariel: My initial thought is this. If we do NOT use passwords, than anyone with the link can take the survey, right? So if DO use passwords and we allow users to create them, what is the risk if they don’t create “unique” passwords? Because I’m not seeing any risk to just letting them create a password. This is just a little placeholder - a dog-earing of the electronic page if you will, it’s not performing a security function. (Right??) Or are you saying that the password is really just a token and if someone else arbitrarily creates the same one (like “1234” or “password”) we’ll have a mess? (ccing myself since I’m out tomorrow and I want to see the answer!) Kristine From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Ariel Liang Sent: Thursday, August 16, 2018 9:42 AM To: gnso-rpm-data@icann.org<mailto:gnso-rpm-data@icann.org> Subject: [Gnso-rpm-data] Advice Needed: Complications Regarding Saving Survey Responses Dear Data Sub Team members, Analysis Group is in process programming the surveys; the actual/potential registrant survey has already been pushed to the beta testing phase. A complication was discovered. Analysis Group informed staff that it is not possible to allow survey takers to “save” their responses and return to the surveys later to complete, unless the survey taker is given a login credential that can be validated. To create a login credential, the simplest way is for Analysis Group to obtain a list of email addresses of the individual survey takers. This, however, cannot be achieved given the outreach method that we are using to distribute the surveys (e.g., public links accessed via mailing lists, announcements, social media). Although GDD has the email contacts of registries and registrars, ICANN Org is not allowed to share the email contacts with Analysis Group directly due to the rightful use of these email contacts and associated legal/GDPR implications. Analysis Group is exploring other options/workarounds for survey takers to “save” their responses. One option is to allow users to create their own passwords, though it is difficult for Analysis Group to enforce unique password creation. They are experimenting with methods of suggesting unique passwords to users who would like to create a password. Another option is to distribute "cookies" with the survey, which will allow respondents to return to surveys that they have started. This option requires Analysis Group to explore any legal issues involved in the use of cookies. In summary, these options may raise technical and privacy issues that need to be cleared internally by Analysis Group. Consequently, this complication may add to the time for the surveys to be launched. Staff are seeking clarification from AG on the ETA to solve these technical/privacy issues. QUESTION: The need to get the surveys out ASAP/as many responses as possible thus needs to be weighed against the benefit of enabling respondents to save their responses/impact to the response rate. What does the Sub Team advise? Thank you for input! Best Regards, Mary, Julie, Ariel, Berry
Is there a way to “split the baby”. Use cookies and then send instruction for how to delete cookie once survey is completed. Lori S. Schulman Senior Director, Internet Policy International Trademark Association (INTA) +1-202-704-0408, Skype: lsschulman From: Gnso-rpm-data <gnso-rpm-data-bounces@icann.org> On Behalf Of Ariel Liang Sent: Monday, August 20, 2018 3:35 PM To: Michael Graham (ELCA) <migraham@expedia.com>; Dorrain, Kristine <dorraink@amazon.com>; gnso-rpm-data@icann.org Subject: Re: [Gnso-rpm-data] [Ext] RE: Advice Needed: Complications Regarding Saving Survey Responses Thank you, Michael! Hello everyone, Please see below an update from Analysis Group with regard to implementing methods for survey takers to save their progress. It seems that both options are promising but they have some shortcomings, which are hard to avoid. If either option is used, we likely need to make it clear in the messaging and/or survey intro about the ways for survey takers to save their progress/return to the survey. == AG: Regarding options for respondents who would like to save and revisit responses - We have identified a way to suggest unique passwords to users who wish to exercise this option. The secondary option of using cookies also continues to look promising. We are working to confirm our understanding of whether data will be collected from respondents through the use of cookies other than allowing respondents to revisit their survey responses. Both options continue to have the same shortcomings as identified earlier. For password creation, the method for creating a new password vs. using a pre-existing password for return visitors may be confusing or frustrating, and/or respondents may lose or forget their passwords. The consequence would be that some surveys may remain partially completed. Under the second option of using cookies, some respondents may not feel comfortable accepting cookies from the survey instrument, resulting in lower response rates. == Thank you, Mary, Julie, Ariel, and Berry From: "Michael Graham (ELCA)" <migraham@expedia.com<mailto:migraham@expedia.com>> Date: Friday, August 17, 2018 at 1:01 PM To: Ariel Liang <ariel.liang@icann.org<mailto:ariel.liang@icann.org>>, "Dorrain, Kristine" <dorraink@amazon.com<mailto:dorraink@amazon.com>>, "gnso-rpm-data@icann.org<mailto:gnso-rpm-data@icann.org>" <gnso-rpm-data@icann.org<mailto:gnso-rpm-data@icann.org>> Subject: RE: [Gnso-rpm-data] [Ext] RE: Advice Needed: Complications Regarding Saving Survey Responses I think the proposal sounds like a reasonable way to move forward to give survey takers the ability to pause and return. Michael R. From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Ariel Liang Sent: Friday, August 17, 2018 6:42 AM To: Dorrain, Kristine <dorraink@amazon.com<mailto:dorraink@amazon.com>>; gnso-rpm-data@icann.org<mailto:gnso-rpm-data@icann.org> Subject: Re: [Gnso-rpm-data] [Ext] RE: Advice Needed: Complications Regarding Saving Survey Responses Hello Kristine and all, Please see below the responses from Analysis Group to Kristine’s question. Analysis Group has been making progress on the alternative options/workarounds, and will be able to provide another update shortly. We will keep the Sub Team posted. Please let us know if you have any further comments/questions. Best Regards, Mary, Julie, Ariel & Berry == Kristine: If we do NOT use passwords, than anyone with the link can take the survey, right? AG: That is correct. The survey will be publicly available, and anyone with the link will be able to take the survey. Kristine: So if DO use passwords and we allow users to create them, what is the risk if they don’t create “unique” passwords? Because I’m not seeing any risk to just letting them create a password. This is just a little placeholder - a dog-earing of the electronic page if you will, it’s not performing a security function. (Right??) AG: If we allow users to create passwords, the survey will remain publicly available and anyone with the link will be able to take the survey. Users who create passwords will be able to save their responses and return to them later. The concern involved with non-unique passwords is that, because the password will be the only way for our programming to identify which survey record to re-open (i.e., users will not be creating unique user IDs and we will not be collecting IP address due to GDPR issues, so saved responses will be cataloged only by their password), if any users create the same password, their survey responses will appear in the survey data as if they came from one respondent when in fact there are two distinct respondents answering the questions. This can cause consistency issues, since we will be unaware of instances where this happens. For example, if a user chooses the password "1234" and completes questions 1-10 and leaves the survey, and then a second user attempts to create a password "1234" -- the second user will unwittingly log-in to the first user's survey and pick-up the survey at question 11. We will be unaware of this happening, but may find that the data appear inconsistent and/or invalid during our analysis of the data. Kristine: Or are you saying that the password is really just a token and if someone else arbitrarily creates the same one (like “1234” or “password”) we’ll have a mess? AG: Yes, as described above, we run the risk of overwriting or combining responses in a way that would cause responses to be impossible to untangle. To continue to follow up with our progress on the passwords: One solution that we would like to suggest is that users can be allowed to create a password, and we will embed a page visit counter next to the password field. We will suggest that users use the number from the counter for their password. This will guarantee that passwords will be unique for each user, although the pattern of the password generation may be a bit transparent. We have identified a way to implement this solution but will need to be careful to make sure the directions for password creation (for first-time visitors) and log-in (for return visitors) are very clear. Unfortunately, the password creation and log-in fields appear as one field in the survey. We are unable to place them on separate pages (i.e., create a log-in page for first-time users and a separate log-in page for return users). We are also investigating issues related to cookies, and expect to have resolved that issue and any related implementation issues by the end of this weekend if not sooner. We can send you another update tomorrow as we continue to make progress. There is a trade-off to consider regarding a portion of potential respondents who choose not to complete the survey if they learn that cookies are being sent to their computer. From: "Dorrain, Kristine" <dorraink@amazon.com<mailto:dorraink@amazon.com>> Date: Thursday, August 16, 2018 at 7:55 PM To: Ariel Liang <ariel.liang@icann.org<mailto:ariel.liang@icann.org>>, "gnso-rpm-data@icann.org<mailto:gnso-rpm-data@icann.org>" <gnso-rpm-data@icann.org<mailto:gnso-rpm-data@icann.org>>, "Dorrain, Kristine" <dorraink@amazon.com<mailto:dorraink@amazon.com>> Subject: [Ext] RE: [Gnso-rpm-data] Advice Needed: Complications Regarding Saving Survey Responses Thanks for this Ariel: My initial thought is this. If we do NOT use passwords, than anyone with the link can take the survey, right? So if DO use passwords and we allow users to create them, what is the risk if they don’t create “unique” passwords? Because I’m not seeing any risk to just letting them create a password. This is just a little placeholder - a dog-earing of the electronic page if you will, it’s not performing a security function. (Right??) Or are you saying that the password is really just a token and if someone else arbitrarily creates the same one (like “1234” or “password”) we’ll have a mess? (ccing myself since I’m out tomorrow and I want to see the answer!) Kristine From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Ariel Liang Sent: Thursday, August 16, 2018 9:42 AM To: gnso-rpm-data@icann.org<mailto:gnso-rpm-data@icann.org> Subject: [Gnso-rpm-data] Advice Needed: Complications Regarding Saving Survey Responses Dear Data Sub Team members, Analysis Group is in process programming the surveys; the actual/potential registrant survey has already been pushed to the beta testing phase. A complication was discovered. Analysis Group informed staff that it is not possible to allow survey takers to “save” their responses and return to the surveys later to complete, unless the survey taker is given a login credential that can be validated. To create a login credential, the simplest way is for Analysis Group to obtain a list of email addresses of the individual survey takers. This, however, cannot be achieved given the outreach method that we are using to distribute the surveys (e.g., public links accessed via mailing lists, announcements, social media). Although GDD has the email contacts of registries and registrars, ICANN Org is not allowed to share the email contacts with Analysis Group directly due to the rightful use of these email contacts and associated legal/GDPR implications. Analysis Group is exploring other options/workarounds for survey takers to “save” their responses. One option is to allow users to create their own passwords, though it is difficult for Analysis Group to enforce unique password creation. They are experimenting with methods of suggesting unique passwords to users who would like to create a password. Another option is to distribute "cookies" with the survey, which will allow respondents to return to surveys that they have started. This option requires Analysis Group to explore any legal issues involved in the use of cookies. In summary, these options may raise technical and privacy issues that need to be cleared internally by Analysis Group. Consequently, this complication may add to the time for the surveys to be launched. Staff are seeking clarification from AG on the ETA to solve these technical/privacy issues. QUESTION: The need to get the surveys out ASAP/as many responses as possible thus needs to be weighed against the benefit of enabling respondents to save their responses/impact to the response rate. What does the Sub Team advise? Thank you for input! Best Regards, Mary, Julie, Ariel, Berry ________________________________
Thanks Lori. We will check with Analysis Group about the feasibility. Best Regards, Ariel Ariel Xinyue Liang GNSO Policy Support Specialist Internet Corporation for Assigned Names and Numbers (ICANN) From: Lori Schulman <lschulman@inta.org> Date: Monday, August 20, 2018 at 9:39 AM To: Ariel Liang <ariel.liang@icann.org>, "Michael Graham (ELCA)" <migraham@expedia.com>, "Dorrain, Kristine" <dorraink@amazon.com>, "gnso-rpm-data@icann.org" <gnso-rpm-data@icann.org> Subject: RE: [Gnso-rpm-data] [Ext] RE: Advice Needed: Complications Regarding Saving Survey Responses Is there a way to “split the baby”. Use cookies and then send instruction for how to delete cookie once survey is completed. Lori S. Schulman Senior Director, Internet Policy International Trademark Association (INTA) +1-202-704-0408, Skype: lsschulman From: Gnso-rpm-data <gnso-rpm-data-bounces@icann.org> On Behalf Of Ariel Liang Sent: Monday, August 20, 2018 3:35 PM To: Michael Graham (ELCA) <migraham@expedia.com>; Dorrain, Kristine <dorraink@amazon.com>; gnso-rpm-data@icann.org Subject: Re: [Gnso-rpm-data] [Ext] RE: Advice Needed: Complications Regarding Saving Survey Responses Thank you, Michael! Hello everyone, Please see below an update from Analysis Group with regard to implementing methods for survey takers to save their progress. It seems that both options are promising but they have some shortcomings, which are hard to avoid. If either option is used, we likely need to make it clear in the messaging and/or survey intro about the ways for survey takers to save their progress/return to the survey. == AG: Regarding options for respondents who would like to save and revisit responses - We have identified a way to suggest unique passwords to users who wish to exercise this option. The secondary option of using cookies also continues to look promising. We are working to confirm our understanding of whether data will be collected from respondents through the use of cookies other than allowing respondents to revisit their survey responses. Both options continue to have the same shortcomings as identified earlier. For password creation, the method for creating a new password vs. using a pre-existing password for return visitors may be confusing or frustrating, and/or respondents may lose or forget their passwords. The consequence would be that some surveys may remain partially completed. Under the second option of using cookies, some respondents may not feel comfortable accepting cookies from the survey instrument, resulting in lower response rates. == Thank you, Mary, Julie, Ariel, and Berry From: "Michael Graham (ELCA)" <migraham@expedia.com<mailto:migraham@expedia.com>> Date: Friday, August 17, 2018 at 1:01 PM To: Ariel Liang <ariel.liang@icann.org<mailto:ariel.liang@icann.org>>, "Dorrain, Kristine" <dorraink@amazon.com<mailto:dorraink@amazon.com>>, "gnso-rpm-data@icann.org<mailto:gnso-rpm-data@icann.org>" <gnso-rpm-data@icann.org<mailto:gnso-rpm-data@icann.org>> Subject: RE: [Gnso-rpm-data] [Ext] RE: Advice Needed: Complications Regarding Saving Survey Responses I think the proposal sounds like a reasonable way to move forward to give survey takers the ability to pause and return. Michael R. From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Ariel Liang Sent: Friday, August 17, 2018 6:42 AM To: Dorrain, Kristine <dorraink@amazon.com<mailto:dorraink@amazon.com>>; gnso-rpm-data@icann.org<mailto:gnso-rpm-data@icann.org> Subject: Re: [Gnso-rpm-data] [Ext] RE: Advice Needed: Complications Regarding Saving Survey Responses Hello Kristine and all, Please see below the responses from Analysis Group to Kristine’s question. Analysis Group has been making progress on the alternative options/workarounds, and will be able to provide another update shortly. We will keep the Sub Team posted. Please let us know if you have any further comments/questions. Best Regards, Mary, Julie, Ariel & Berry == Kristine: If we do NOT use passwords, than anyone with the link can take the survey, right? AG: That is correct. The survey will be publicly available, and anyone with the link will be able to take the survey. Kristine: So if DO use passwords and we allow users to create them, what is the risk if they don’t create “unique” passwords? Because I’m not seeing any risk to just letting them create a password. This is just a little placeholder - a dog-earing of the electronic page if you will, it’s not performing a security function. (Right??) AG: If we allow users to create passwords, the survey will remain publicly available and anyone with the link will be able to take the survey. Users who create passwords will be able to save their responses and return to them later. The concern involved with non-unique passwords is that, because the password will be the only way for our programming to identify which survey record to re-open (i.e., users will not be creating unique user IDs and we will not be collecting IP address due to GDPR issues, so saved responses will be cataloged only by their password), if any users create the same password, their survey responses will appear in the survey data as if they came from one respondent when in fact there are two distinct respondents answering the questions. This can cause consistency issues, since we will be unaware of instances where this happens. For example, if a user chooses the password "1234" and completes questions 1-10 and leaves the survey, and then a second user attempts to create a password "1234" -- the second user will unwittingly log-in to the first user's survey and pick-up the survey at question 11. We will be unaware of this happening, but may find that the data appear inconsistent and/or invalid during our analysis of the data. Kristine: Or are you saying that the password is really just a token and if someone else arbitrarily creates the same one (like “1234” or “password”) we’ll have a mess? AG: Yes, as described above, we run the risk of overwriting or combining responses in a way that would cause responses to be impossible to untangle. To continue to follow up with our progress on the passwords: One solution that we would like to suggest is that users can be allowed to create a password, and we will embed a page visit counter next to the password field. We will suggest that users use the number from the counter for their password. This will guarantee that passwords will be unique for each user, although the pattern of the password generation may be a bit transparent. We have identified a way to implement this solution but will need to be careful to make sure the directions for password creation (for first-time visitors) and log-in (for return visitors) are very clear. Unfortunately, the password creation and log-in fields appear as one field in the survey. We are unable to place them on separate pages (i.e., create a log-in page for first-time users and a separate log-in page for return users). We are also investigating issues related to cookies, and expect to have resolved that issue and any related implementation issues by the end of this weekend if not sooner. We can send you another update tomorrow as we continue to make progress. There is a trade-off to consider regarding a portion of potential respondents who choose not to complete the survey if they learn that cookies are being sent to their computer. From: "Dorrain, Kristine" <dorraink@amazon.com<mailto:dorraink@amazon.com>> Date: Thursday, August 16, 2018 at 7:55 PM To: Ariel Liang <ariel.liang@icann.org<mailto:ariel.liang@icann.org>>, "gnso-rpm-data@icann.org<mailto:gnso-rpm-data@icann.org>" <gnso-rpm-data@icann.org<mailto:gnso-rpm-data@icann.org>>, "Dorrain, Kristine" <dorraink@amazon.com<mailto:dorraink@amazon.com>> Subject: [Ext] RE: [Gnso-rpm-data] Advice Needed: Complications Regarding Saving Survey Responses Thanks for this Ariel: My initial thought is this. If we do NOT use passwords, than anyone with the link can take the survey, right? So if DO use passwords and we allow users to create them, what is the risk if they don’t create “unique” passwords? Because I’m not seeing any risk to just letting them create a password. This is just a little placeholder - a dog-earing of the electronic page if you will, it’s not performing a security function. (Right??) Or are you saying that the password is really just a token and if someone else arbitrarily creates the same one (like “1234” or “password”) we’ll have a mess? (ccing myself since I’m out tomorrow and I want to see the answer!) Kristine From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Ariel Liang Sent: Thursday, August 16, 2018 9:42 AM To: gnso-rpm-data@icann.org<mailto:gnso-rpm-data@icann.org> Subject: [Gnso-rpm-data] Advice Needed: Complications Regarding Saving Survey Responses Dear Data Sub Team members, Analysis Group is in process programming the surveys; the actual/potential registrant survey has already been pushed to the beta testing phase. A complication was discovered. Analysis Group informed staff that it is not possible to allow survey takers to “save” their responses and return to the surveys later to complete, unless the survey taker is given a login credential that can be validated. To create a login credential, the simplest way is for Analysis Group to obtain a list of email addresses of the individual survey takers. This, however, cannot be achieved given the outreach method that we are using to distribute the surveys (e.g., public links accessed via mailing lists, announcements, social media). Although GDD has the email contacts of registries and registrars, ICANN Org is not allowed to share the email contacts with Analysis Group directly due to the rightful use of these email contacts and associated legal/GDPR implications. Analysis Group is exploring other options/workarounds for survey takers to “save” their responses. One option is to allow users to create their own passwords, though it is difficult for Analysis Group to enforce unique password creation. They are experimenting with methods of suggesting unique passwords to users who would like to create a password. Another option is to distribute "cookies" with the survey, which will allow respondents to return to surveys that they have started. This option requires Analysis Group to explore any legal issues involved in the use of cookies. In summary, these options may raise technical and privacy issues that need to be cleared internally by Analysis Group. Consequently, this complication may add to the time for the surveys to be launched. Staff are seeking clarification from AG on the ETA to solve these technical/privacy issues. QUESTION: The need to get the surveys out ASAP/as many responses as possible thus needs to be weighed against the benefit of enabling respondents to save their responses/impact to the response rate. What does the Sub Team advise? Thank you for input! Best Regards, Mary, Julie, Ariel, Berry
participants (3)
-
Ariel Liang -
Lori Schulman -
Michael Graham (ELCA)