On 2017/04/04 06:59, Andrew Sullivan wrote:
On Mon, Apr 03, 2017 at 09:41:47PM +0000, Stuart Stuple wrote:
I believe for plain text with no fancy Unicode values, the rendering is well-defined because the period and ampersand are both neutral and assume directionality from the surrounding characters. See http://www.w3.org/International/articles/inline-bidi-markup/uba-basics.
Well, what do you mean "no fancy Unicode values"?
There are Unicode characters that aren't visible except that they affect the display order of surrounding or enclosed other characters. They can be used in running text to tweak things in situations where the Unicode bidi algorithm would lead to the wrong result. They might also be used to tweak the display of e.g. an email address in a display widget where the Unicode bidi algorithm is used, but where you want to display the email address with some tweaks. https://tools.ietf.org/html/rfc3987#section-4 contains a description of the problems, and in particular the later part of https://tools.ietf.org/html/rfc3987#section-4.1 contain a proposal of how to use these characters (on the fly for display, not as actual parts of the protocol syntax) to assure a specific display. But that part of that RFC isn't widely implemented, and there are all kinds of different opinions on how things should be displayed.
That's the problem here. You're right that that U+0040 is Bidi_Class Other_Neutral and U+002E is Common_Separator, so neither should change directionality. But that's not the problem.
It's not the only problem. If they had their own directionality, displays would vary much less, and people might just get used to what they see (the same way all of us nowadays are used to see inside-out in the domain-name part, but outside-in in the path part, of e.g. http://www.acme.com/news/latest/item1.html).
The problem is that many email addresses are of the form first-local-part.second-local-part@label.secondlabel. So let's start with the simple case:
[some ASCII characters]@arabic-string.com
now, the bidi algorithm is probably going to do something like this (at least in my reading it should):
LTR@RTL.com
Well, yes, in an LTR context (e.g. an English text). In an Arabic text (RTL context), it will be com.arabic-string@[some ASCII characters]. And in all cases, the "RTL" part will have the characters in RTL order, but that's the bit everybody is happy with (as long as there are only RTL characters in that part).
Now, what if instead it's
[some arabic characers]@arabic-string.com
? The problem is easy to see: this part
[some arabic characers]@arabic-string
should actually render
arabic-string (in rtl)@[some arabic characers] (in rtl)
And now you don't know where to put .com.
The .com will go to the right in an LTR context, and to the left (as .com) in an RTL context.
This is a genuinely hard problem for user interfaces; and again, if intuitive interfaces for native Arabic-script users have so far emerged it is news to me. (I would not be surprised that it be news -- I don't use Arabic to write anything -- but last time I checked with relevant people working on the interfaces I got a lot of wailing about this problem so I don't think I'm the only one who thinks its hard.)
Definitely you're not the only one. My understanding is that for e.g. all-Arabic (or all-Hebrew) addresses, it wouldn't be too difficult to establish a display convention for widgets, but there's the conflict between "experts" (people who also use English and are used to the order we know) and "beginners" (people who just read Arabic and may expect things to be in the same order as an address or some such). Then there's the problem that even if something works in a widget (e.g. the "To:" field in a mailer, where we know that it's an address), it won't easily work in running text (but it will surely be copied and pasted there). Then there's the problem that mono-directional stuff may work, but mixed direction (as above) may not. In particular, all-LTR ideally goes LTR, and all-RTL ideally may go RTL, so there's some place in the middle where we have a discontinuity and things suddenly switch. Then there's the problem that we may not only have e.g. domain labels with different directionality, but also mixed directionality (or a single directionality mixed with numbers, which is almost as bad) for a single label or other, similar component. Regards, Martin.
Add Unicode directionality controls and you get a wide variety of orderings
Yes, of course.
A