- vip - lists.icann.org

Re: [vip] Draft on IDN Tables in XML
by John C Klensin March 14, 2012

March 14, 2012

Ram, Agreed. Indeed, we quite deliberately made absolutely no attempt to define a machine-readable format when the idea of these tables was introduced. One reason was that we saw no chance of getting consensus on a format, partially for the reasons Ram identifies. But the more important reason was that we didn't want to encourage anyone to casually import a table without studying it and, indeed, having enough script (and language?) expertise around to be sure that they understood it. I think many of us can remember the degree of upset and concern that arose when one registry decided to allow a script that they didn't understand well because they knew that a language that used the script was used in that country. It is not clear whether importing a table that was equally poorly understood who make that situation worse, but it seems to me it would be very unlikely to make it better. Jaap's recollection is correct and a machine-readable form would be useful in some cases. However, the risk of mindless import of tables actually creates a tradeoff between "...Having a machine readable format that allows the tables to be imported and repurposed aids this greatly" and the possibility of real harm. My sense would be that, especially if the intent is to capture not just character lists but variant concepts, we would be much better off with a list of characters and explanations. I believe that the character lists and explanations that we would be likely to see fall into three categories: (1) Rather short lists, under a hundred or so characters. This would be the case for most alphabetic scripts, including Latin. It is not clear that a standardized table format helps with that case because anyone trying to import a table is really going to need considerable explanation about what is going on (e.g., the ability to represent "oe" as a variant of o-dieresis does not imply that importing that relationship would be wise). (2) Lists that involved a great deal of complexity because of concerns about look-alike characters, characters that are sometimes used interchangeably, or distinctly different uses (and rendering styles) for the same script. All three of those situations have been illustrated in the ASIWG work on Arabic (some of which is incorporated into the Arabic variant information project team), but I have every confidence that there are other scripts out there with one or more of those issues (some would argue that the Japanese - Chinese situation poses problems not much different from the [Western] Arabic - Perso-Arabic one). The complexity of those situations cannot be easily represented in a simple table, whether in XML or otherwise. (3) Chinese (Han) script. The tables there are large enough to make automatic import useful, but the CNDC table requires a specialized table because of the paired preferred variants and, as we know, neither Japanese nor Korean require variant treatment (although they may have other issues). It would make lots of sense to have a standardized machine-readable format for the Chinese use of Han script, but that involves a different set of issues than such a format for other scripts and may or may not be the correct format for a Japanese or Korean table. As a general comment, it seems to me that people on this mailing list and the technical and script communities have repeatedly told ICANN, its staff, and various "constituencies" that Han script is fundamentally different from the collection of alphabet-phonetic scripts and that trying to treat them as the same just yields one problem after another (for one group or the other). That advise has been consistently ignored in ICANN's efforts, including this one. I don't know if the reason is determined ignorance (which I certainly wouldn't not expect from Kim) or that the advice is politically inconvenient but, from my point of view, even decision made on the basis that they were really the same just makes ICANN look silly and puts the predictability and stability of the Internet at risk. Certainly it would be more convenient if they were really the same --same variant issues, same order magnitude of relevant characters, same relationships of "character" to sounds and meanings, and so on-- but the odds of everyone who thinks they would like one being awarded a pony and the wherewithal to keep it are much higher. best, john --On Wednesday, March 14, 2012 10:45 -0400 Ram Mohan <rmohan(a)afilias.info> wrote: > Kim, > > I am not certain registries would want to use an > automated/machine-readable mechanism for importing tables from > other registry IDN implementations. Implementations vary > widely from one registry to another, each IDN implementation > often requires hand-verification of allowed code points > combined with business policies. > > > > Let's try an example: > > CDNC regularly publishes and updates the set of valid > codepoints in the Han script, along with contextual and > variant generation rules/guidelines. Publishing those set of > rules in this rich format would be useful; however, registries > will still need to manually verify both the codepoints and the > rules for variant generation, and create business rules for > allocation and delegation of applied for strings. > > > > Feels like a nice to have, not a must have. > > > > -Ram > > > > -------------------------------------------------------------- > ------------------- > > Ram Mohan > > (o) +1.215.706.5700 x103 (m) +1.215.431.0958 (f) > +1.215.706.5701 > > rmohan(a)afilias.info | Skype: gliderpilot30 | Twitter @rmohan123 > > -------------------------------------------------------------- > ------------------- > > > > *From:* Kim Davies [mailto:kim.davies@icann.org] > *Sent:* Monday, March 12, 2012 9:24 AM > *To:* Jaap Akkerhuis > *Cc:* idna-update(a)alvestrand.no; Dillon, Chris; Abdulrahman I. > ALGhadir; vip(a)icann.org > *Subject:* Re: Draft on IDN Tables in XML > > > > Hi Jaap, > > > > On Mar 12, 2012, at 3:34 AM, Jaap Akkerhuis wrote: > > > As far as I know, the idea of the tables have always been to > provide a public central place where the registries could list > which characters they support and which they don't in their > registrer plolicies. It is an "for your information only" > registry and meant for human consumption. > > And Kim, do correct me if this isn't the case anymore. > > > > It is certainly correct that the notion of tables is to > publicly share registry policy as it relates to code points > that are accepted for registration. I think, however, it is a > bit beyond merely informational, as a key driver in publishing > them has been to allow sharing and re-use by other registries. > Having a machine readable format that allows the tables to be > imported and repurposed aids this greatly. > > > > One of the reasons I feel this is a good initiative is an > increasing number of tables appear to be published as PDF > files with various contextual rules described in paragraphs of > normative text. It would be nice to reverse this trend and > have a format rich enough that it can express most if not all > registry policies in a common way using a set of agreeable > primitives. > > > > kim

1 0

Session on IDN Variant TLDs Issues Report and Next Steps
by Naela Sarras March 13, 2012

March 13, 2012

Dear Colleagues, We are looking forward to Wednesday's session on IDN Variant TLDs Issues Report and Next Steps. The session will take place this week during the ICANN 43 meeting in Costa Rica: Date: Wed, 14 March 2012 - 09:00 - 10:30 local time. Room: La Paz A Session details are available at http://costarica43.icann.org/node/29647. We hope you can join remotely if you are not in Costa Rica. Remote participation details are listed at the link above. Best regards, Naela Sarras ICANN

4 3

Re: [vip] Draft on IDN Tables in XML
by JFC Morfin March 13, 2012

March 13, 2012

At 19:28 12/03/2012, Shawn Steele wrote: >I don't see a relationship between the proposed XML behavior and >JFC's ideas. Nor am I interested in JFC's ML stuff, sorry. Noted. jfc

1 0

Re: [vip] Draft on IDN Tables in XML
by JFC Morfin March 12, 2012

March 12, 2012

At 17:16 12/03/2012, Gervase Markham wrote: >On 12/03/12 03:07, J-F C. Morfin wrote: >>At 03:13 12/03/2012, Shawn Steele wrote: >>>What kinds of applications are expected to consume this data? What's >>>the target? >> >>The browsers want to use them. To validate the IDNs. This is why we need >>to get them synced. > >Shawn, representing Microsoft, asked this question, which I presume >suggests that Microsoft isn't sure what this data is for. I, >representing Mozilla, echo his question. >Which browser makers have asked for this data? Lead users. As you do not want to listen and talk to them, some grassroots people have decided the safest was to reconsider the whole user architecture, back to the IUI and ML-DNS. They probably are not on the same wave lengths as both of you? Don't ask me. My personnal focus here is only the IUI and ML-DNS architectural, documentation and safe testing framework; and to try to keep some liaison with the Internet stratum. This is juste because the "Internet+" stratum is necessary multiservice menu synergy and access, to multilinguistic support, and to my interest in the Intersem. I therefore only listen to the requests and investigate how the IUI could address them when the project matches my interest. I will know better once my I_Ds are finalized, a PLUS software architecture adopted, and some first prototyping achieved. Then IUI "slots" (smart local operating tasks) from projects will be experimented. Only by then we will know which projects have actually gone through and what they bring. You know FLOSS, it may take some time before all that preparatory process stabilizes. However, I have good hopes to see the things running in my old age. jfc

1 0

Re: [vip] Draft on IDN Tables in XML
by Kim Davies March 12, 2012

March 12, 2012

Hi Shawn, On Mar 11, 2012, at 8:13 PM, Shawn Steele wrote: So, as a 3rd party, I'm not sure how this information is very helpful to me? I mean, it'd tell me what the registry hoped to do for it's zone, but that may or may not match the DNS entries. I can discover valid domains by querying the DNS itself, I don't need to access the list. I can see how this information might be helpful for tools that help people manage their zones, however any tools that talked to themselves wouldn't necessarily need this format (though it'd help to be able to import/export the data this way, and maybe interoperate better with other tools). I guess what I'm getting at is: What kinds of applications are expected to consume this data? What's the target? The target is really registries, albeit potentially registries at multiple levels of the DNS — not just TLD registries. While other software like end-user applications consume the data, I am not sure what the benefit would be. Once it is in a DNS zone, it is a domain name, and would be expected to resolve. Tables are really beneficial as part of the provisioning process. kim

2 3

Re: [vip] Draft on IDN Tables in XML
by Jaap Akkerhuis March 12, 2012

March 12, 2012

Is it possible to add the relation type for each variant like is it exactly similar or partially? Also, what about the registry rules? Some registries impose more restriction on the IDNA protocol is it possible to add a tag the describe them? As far as I know, the idea of the tables have always been to provide a public central place where the registries could list which characters they support and which they don't in their registrer plolicies. It is an "for your information only" registry and meant for human consumption. And Kim, do correct me if this isn't the case anymore. jaap

2 1

Re: [vip] Draft on IDN Tables in XML
by JFC Morfin March 12, 2012

March 12, 2012

At 07:30 12/03/2012, Abdulrahman I. ALGhadir wrote: >Hello, >Is it possible to add the relation type for each variant like is it >exactly similar or partially? >Also, what about the registry rules? Some registries impose more >restriction on the IDNA protocol is it possible to add a tag the describe them? >AbdulRahman, There is a need for a "netlocale" file, describing the network multilinguistics (i.e. what is related to the mutual support of the linguistic diversity) as seen from a user (i.e. in the languages of the users) and operated by his/her linguistic tools. Until now the BCP47 has acknowledged three *main* parameters (langtags) which are language (ISO 639), script (ISO 15924), and "legal culture" (ISO 3166). One of the consequences of IDNA2008 has been to consolidate the powerful but not yet documented Internet Presentation Layer and to introduce the need to extend the BCP47. This also create concerns for the WG/PRECIS and the Happiana list. Vint Cerf, as the Chair of the WG/IDNABis suggested that the post-IDNA2008 work would be coordinated by ICANN. As IUsers (Intelligent Internet Independent users) we could not accept this suggestion until ICANN and IETF would have defined their position in the resulting technology stewardship. VIP is a preliminary step, as is Google+, as are the Drafts I published. All this leads us to the addition of at least three new parameters to BCP47, a new rule set and a coordination authority: - parameters are: registry (i.e. root name - the TLD for what concerns the Internet), CLASS, and presentation. TLD are used and known, but have to be extended to non-Internet needs (IRT the whole digital ecosystem administration). There are 65.536 CLASSes, only one being actually used, and ICANN (ICP-3) pushing new needs to the use of more. Presentations are not yet defined, but we know one as related to "xn--" and punycode. - new rule set: this six main parameters have to be documented through a polylingual (i.e. a 7th main parameter) format, that will have to be able to support different set of rules (by TLD manager, about variants, by local Laws, etc.) - a coordination authority. All this affects the daily operation of browsers, applications, law enforcement (for example ice.com in the USA), new intellectual protection rules (e.g. the "babel names" which have not even been alluded to in the VIP case: when all the characters of a domain names are not supported, and in some applications/browsers for security reasons, the U-label may be replaced or completed by the A-label and the A-label to display a IP protected name. With additional problem with variants, since the several A-labels may correspond to several U-labels). For obvious security and operational reasons, all this information should be at least moderately (within a TTL like delay) synced to avoid blocking situations, which may add to not considered non tested possible conflicts with IPv6 and DNSSEC. We have now to consider the figures: - languages (ISO 639-7), probably 25.000 language units. Beware: in our world, languages will be considered as high level common communications protocols between "semantic processors", natural (human brain) and mechanical, hence as "mecalanguages". There will be advanced variants, revived old languages, encrypted languages. - scripts (ISO 15924), 100+ scripts, with in addition signs and possibly accents (natural or binary). - countries and equivalent (ISO 3166 includes international bodies' code). - TLD, impossible to determine as the New gTLD Program in the ICANN/NTIA "IN" CLASS is theoretically unlimited. - CLASSes : 65.536 (reminder: a CLASS is a way of perceiving the Internet protocols and DNS) - Presentations : the number is be essence unlimited, however one may expect two kinds of presentations : externets and private. An externet is an "external network look-alike" built in using the Internet. One can also describe them as "open closed garden": an Internet subset where procedures, operational constraints and services, rates, neutrality, languages, etc. are differently supported. The common documentation should probably only concern the public externet system. Private presentations includes special languages, interface uses, encryptions, They should obey to the same format, but will most probably not be synced with all the other BCP47+ data. - DNS visions may also be added. - and then multiply all this by the number of languages again to be presented in every language and scripts (some languages being supported by several scripts). jfc

1 0

Re: [vip] Draft on IDN Tables in XML
by JFC Morfin March 11, 2012

March 11, 2012

At 17:42 08/03/2012, Shawn Steele wrote: >How do these tables stay in-sync? I mean, what happens when it >disagrees with what is actually in the DNS zone files? Dear Shawn, the need for synchronized multilinguistic (on languages) and polylinguistic (same data in different language) is a general need that can only be currently addressed in an opendata context by a registry or DDDS international system, or by an extended DDDS mutually operated architecture. It would be interesting to know what Microsoft, Google, and others would have to propose. I would be interested in working on a joint exploratory draft on this issue which is universal and not limited to the Internet. This could be part of the JTC1/SG32/WG2 effort, or a totally separate project. It should permit contributions by everyone to be kept synchronized within a registrant defined TTL and to be freely accessed for validity check of table versions and updates. Best jfc

1 0

Re: [vip] Draft on IDN Tables in XML
by Kim Davies March 10, 2012

March 10, 2012

Hi Shawn, On Mar 8, 2012, at 8:42 AM, Shawn Steele wrote: > How do these tables stay in-sync? I mean, what happens when it disagrees with what is actually in the DNS zone files? I think the simple answer is that is registry policy, and varies from zone-to-zone. It is not something where a specific approach needs to be harmonised. For the typical use case, the registry consults with the IDN table at registration time to compute the set of variants. Then, based on a specific domain's policy, the set would typically be recorded in the registry system (as delegated, reserved, blocked, etc.). I imagine if the table changed on which the registry based its policy, it would need its own procedures for dealing with that, but that would be outside the scope of the specification. Some may opt to only use the changed set on new registrations and grandfather in the old ones, depending on the reason why the maintain variants. kim

1 0

Draft on IDN Tables in XML
by Kim Davies March 8, 2012

March 8, 2012

Colleagues, I have posted a first draft regarding a format that could be used for representing IDN Tables in XML to the I-D Repository: https://datatracker.ietf.org/doc/draft-davies-idntables/ After discussion with a number of folks that felt this would be good work to undertake, I've put together a first cut which is not comprehensive, but I think goes some way toward a potential format. Unless there is interest in this being a more formal activity, my assumption is to aim to publish the final result independently as an Informational RFC. However, the mechanism of publication is secondary to coming up with something useful that would benefit TLD registries and other implementors. A list of design goals, from the document, is as follows: • MUST be in a format that can be implemented in a reasonably straightforward manner in software; • The format SHOULD be able to be checked for formatting errors, such that common mistakes can be caught; • An IDN Table MUST be able to express the set of valid code points that are allowed for registration under a specific zone administrator's policies; • MUST be able to express computed alternatives to a given domain name based on a one-to-one, or one-to-many relationship. These computed alternatives are commonly known as "IDN variants"; • IDN Variants SHOULD be able to be tagged with specific categories, such that the categories can be used to support registry policy (such as whether to list the computed variant in the zone, or to merely block it from registration); • IDN Variants MUST be able to stipulated based on contextual information. For example, specific variants may only be applicable when they follow another specific code point, or when the code point is displayed in a specific presentation form; • The data contained within the table MUST be unambiguous, such that independent implementations that utilise the contents will arrive at the same results; • IDN Tables SHOULD be suitable for comparison and re-use, such that one could easily compare the contents of two or more to see the differences, to merge them, and so on. • As many existing IDN Tables are practicable SHOULD be able to be migrated to the new format with all applicable logic retained. It is explicitly NOT the goal of this format to: • Stipulate what code points should be listed in an IDN Table by a zone administrator. What registration policies are used for a particular zone is outside the scope of this memo. • Stipulate what a consumer of an IDN Table must do when they determine a particular domain is valid or invalid; or arrive at a set of computed IDN variants. IDN Tables are only used to describe rules for computing code points, but does not prescribe how registries and other parties utilise them. I'd appreciate any feedback. cheers, kim

4 6