rfc:unicode_text_processing

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
rfc:unicode_text_processing [2022/12/16 13:54] – derickrfc:unicode_text_processing [2022/12/21 11:48] (current) – Added concat and equals methods; changed join to accept an iterator; Enhance explanation of locales and collations, and standardize on using ''$collator'' as an argument name everywhere. derick
Line 1: Line 1:
 ====== PHP RFC: Unicode Text Processing ====== ====== PHP RFC: Unicode Text Processing ======
-  * Version: 0.9 +  * Version: 0.9.2 
-  * Date: 2022-12-16 (Original date: 2022-12-15)+  * Date: 2022-12-21 (Original date: 2022-12-15)
   * Author: Derick Rethans <derick@php.net>   * Author: Derick Rethans <derick@php.net>
   * Status: Draft   * Status: Draft
Line 72: Line 72:
 If an argument to any of the methods is listed as ''string|Text'', If an argument to any of the methods is listed as ''string|Text'',
 passing in a ''string'' value will have the same semantics as replacing passing in a ''string'' value will have the same semantics as replacing
-the passed value with ''new Text($string)''. The locale from the Text +the passed value with ''new Text($string)''. The locale and default collator 
-object that this method is called on is also used for this new wrapped +from the Text object that this method is called on is also used for this new 
-value, if necessary.+wrapped value, if necessary.
  
-==== Locales and Internationalisation ====+==== Locales, Collators, and Internationalisation ====
  
-By default each string will have the "root" collator associated with it, +By default each string will have the "root" locale and "standard" collator 
-but it is possible to configure a specific collator by using the +associated with it, but it is possible to configure a specific locale and 
-''$collator'' argument in the constructor. The ''$collator'' is specified as +collator by using the ''$collation'' argument in the constructor. Collation is in 
-a string describing an ICU locale name:+addition to the locale, and affects sorting and finding operations. 
 + 
 +The ''$collation'' is specified as a string describing an ICU locale/collation 
 +name:
 https://unicode-org.github.io/icu/userguide/collation/api.html#instantiating-the-predefined-collators https://unicode-org.github.io/icu/userguide/collation/api.html#instantiating-the-predefined-collators
  
-For example, the locale (or collation) name ''en-u-ks-level1'' means +The methods on the Text object all use the ''$collation'' argument name. 
-case-insensitive sorting for the English locale. This will require + 
-extensive documentation.+For example, the locale (and collation) name ''en-u-ks-level1'' means 
 +case-insensitive sorting (''ks-level1'') for the English locale (''en-u''). 
 +The format of this locale/collation name needs extensive documentation.
  
-Numerical order collation (such as PHP's ''natsort()'') can be achieved +Numerical order collation (such as PHP's ''natsort()'') can be achieved by 
-by adding the ''kn'' flag to the locale name, such as in ''de-u-kn'' +adding the ''kn'' flag to the collator specification, such as in ''de-u-kn'' 
-(case-sensitive German, with numerics in value order).+(case-sensitive German ('''de-u''), with numerics in value order (''kn'')).
  
 Other options are described in BCP47: Other options are described in BCP47:
Line 111: Line 116:
 This section lists all the method that construct a Text object. This section lists all the method that construct a Text object.
  
-=== __construct(string $text, string $locale = 'root/standard') : \Text ===+=== __construct(string $text, string $collation = 'root/standard') : \Text ===
  
 The constructor takes a UTF-8 encoded text, and stores this in an internal The constructor takes a UTF-8 encoded text, and stores this in an internal
Line 120: Line 125:
 if present. if present.
  
-=== static Text::create(string $text, string $locale = 'root/standard') : \Text ===+ 
 +=== static Text::create(string $text, string $collation = 'root/standard') : \Text ===
  
 The Symfony String package, offers a static function to construct a String The Symfony String package, offers a static function to construct a String
Line 130: Line 136:
 For example with ''use \Text::create as t''. For example with ''use \Text::create as t''.
  
-=== static Text::join(array(string|Text) $elements, string|Text $separator, string $collator = NULL) : \Text === 
  
-Creates a new Text object by concatenating the Text element in+=== static Text::concat(string|Text ...$elements) : \Text === 
 + 
 +Creates a new Text object by concatenating all the given string/Text arguments 
 +into a new Text object.  
 + 
 +If the ''$elements'' array is empty, an empty ''Text'' object with the 
 +''root'' locale and ''standard'' collation is created. 
 + 
 + 
 +=== static Text::join(iterable<string|Text> $elements, string|Text $separator, string $collation = NULL) : \Text === 
 + 
 +Creates a new Text object by looping over all the string/Text elements in
 ''$elements'', inserting ''$separator'' in between each element. ''$elements'', inserting ''$separator'' in between each element.
  
 The semantics are like: ''implode(string $separator, array(string) $array)'' The semantics are like: ''implode(string $separator, array(string) $array)''
  
-If the ''$collator'' is not specified, it uses the collection of the first +If the ''$collation'' is not specified, it uses the collation of the first 
-element in the ''$elements'' array. This will also be then set on the created +element from the ''$elements'' iterable. This will also be then set on the 
-object.+created object.
  
-If the ''$elements'' array is empty, an empty ''Text'' object with the +If the ''$elements'' iterator has no items, an empty ''Text'' object with the 
-''root'' locale is created.+''root'' locale and ''standard'' collation is created.
  
 +If the iterator produces a non-string/Text element, then a ''\ValueException''
 +will be thrown.
  
 ==== Standard String Operations ==== ==== Standard String Operations ====
Line 180: Line 198:
 If ''$cutLongWords'' is set, no Text element will be larger than If ''$cutLongWords'' is set, no Text element will be larger than
 ''$maxWidth''. ''$maxWidth''.
- 
- 
-=== replaceText(string|Text $search, string|Text $replace, int $replaceFrom = 0, int $replaceTo = -1 ) : \Text === 
- 
-Replaces occurrences of ''$search'' with ''$replace''. 
- 
-The locale of ''$search'' is used to find sub-strings that 
-match, if it is a ''Text'' object, otherwise the locale embedded in the object 
-that the method is called on. 
- 
-The ''$replaceFrom'' and ''$replaceTo'' arguments control which found 
-items are being replaced. The ''$replaceFrom'' argument is the first 
-argument that is being replaced (0-indexed), and ''$replaceTo'' is the 
-last item. Positive numbers are counted from the first occurrence of 
-''$search'' in the Text, and negative numbers from the last found 
-occurrence. 
- 
-In order to find sub-strings case-insensitively, you can use the ''$collator'' 
-argument to ''Text::__construct'' of the ''$search'' argument. 
  
 === reverse() : \Text === === reverse() : \Text ===
Line 209: Line 208:
 Methods to find text in other text. Methods to find text in other text.
  
-In all these methods, the locale of ''$search'' is used to find sub-strings that  +In all these methods, the locale and collator of ''$search'' are used to find 
-match, if it is a ''Text'' object, otherwise the locale embedded in the object +sub-strings that match, if it is a ''Text'' object, otherwise the locale and 
-that the method is called on.+collator that are embedded in the object that the method is called on is used.
  
  
Line 241: Line 240:
 (https://www.php.net/manual/en/function.grapheme-strstr.php) (https://www.php.net/manual/en/function.grapheme-strstr.php)
  
-Alternative suggested names: ''startingWith''+Alternative suggested names: ''startingWith'', ''startingAt''
  
  
Line 247: Line 246:
  
 Like ''returnFromFirstOccurence'' but then from the end of the text. Like ''returnFromFirstOccurence'' but then from the end of the text.
 +
 +Alternative suggested names: ''startingWithLast'', ''startingAtLast''
 +
  
 === contains(string|Text $search) === === contains(string|Text $search) ===
Line 253: Line 255:
  
 Like ''str_contains''. Like ''str_contains''.
- 
-Alternative suggested names: ''startingWithLast'' 
  
  
Line 262: Line 262:
  
 Case-insensitive comparison can be achieved by setting the right Case-insensitive comparison can be achieved by setting the right
-''$collator'' on ''$search''.+''$collation'' on ''$search''.
  
 Could be constructed from ''getPositionOflastOccurrence()'' and Could be constructed from ''getPositionOflastOccurrence()'' and
Line 274: Line 274:
  
 Case-insensitive comparison can be achieved by setting the right Case-insensitive comparison can be achieved by setting the right
-''$collator'' on ''$search''.+''$collation'' on ''$search''.
  
 Could be constructed from ''getPositionOfFirstOccurrence()'', Could be constructed from ''getPositionOfFirstOccurrence()'',
 but it's an often required method, and standard PHP has it but it's an often required method, and standard PHP has it
 too. too.
 +
 +=== replaceText(string|Text $search, string|Text $replace, int $replaceFrom = 0, int $replaceTo = -1 ) : \Text ===
 +
 +Replaces occurrences of ''$search'' with ''$replace''.
 +
 +The ''$replaceFrom'' and ''$replaceTo'' arguments control which found
 +items are being replaced. The ''$replaceFrom'' argument is the first
 +argument that is being replaced (0-indexed), and ''$replaceTo'' is the
 +last item. Positive numbers are counted from the first occurrence of
 +''$search'' in the Text, and negative numbers from the last found
 +occurrence.
 +
 +In order to find sub-strings case-insensitively, you can use the ''$collation''
 +argument to ''Text::__construct'' of the ''$search'' argument.
  
  
 ==== Comparing Text Objects ==== ==== Comparing Text Objects ====
  
-=== compareWith(Text $other, string $collator = NULL) : int ===+=== compareWith(Text $other, string $collation = NULL) : int ===
  
-Uses the configured ''$collator'' of ''$this'' to compare it against +Uses the configured ''$collation'' of ''$this'' to compare it against 
-''$other'', unless the ''$collator'' argument is specified as an override.+''$other'', unless the ''$collation'' argument is specified as an override.
  
 This same method is also used for comparing two Text objects as "compare This same method is also used for comparing two Text objects as "compare
-handler". Here only the locale on ''$this'' is taken into account.+handler" (an overloaded ''=='' operator). Here only the locale on ''$this'' is 
 +taken into account. 
 + 
 +=== equals(Text $other, string $collation = NULL) : boolean === 
 + 
 +Alias for ''compareWith($other, $collation) === 0''.
  
  
Line 383: Line 402:
 These functions return an iterator that can be used to iterator over the text. These functions return an iterator that can be used to iterator over the text.
 The return of the iterators are effected by the text's locale. The return of the iterators are effected by the text's locale.
-i+
 These are inspired by ICU4J's BreakIterators These are inspired by ICU4J's BreakIterators
 (https://unicode-org.github.io/icu-docs/apidoc/released/icu4j/com/ibm/icu/text/BreakIterator.html) (https://unicode-org.github.io/icu-docs/apidoc/released/icu4j/com/ibm/icu/text/BreakIterator.html)
Line 477: Line 496:
  
   - Add a method a like mb_strcut, to extract a string of a maximum amount of bytes from a position, as encoded through UTF-8.   - Add a method a like mb_strcut, to extract a string of a maximum amount of bytes from a position, as encoded through UTF-8.
-  - Tidy up language related to locale/collator. As Tim Starling says: "If the input is an ICU locale string, then I think you should just call it locale. Then the user will be armed with the correct terminology when they go looking for more information in the ICU manual. In ICU, case conversion and BreakIterator need a locale, not a collator. 
  
 ===== Questions and Answers ===== ===== Questions and Answers =====
Line 522: Line 540:
  
 ===== Changes ===== ===== Changes =====
 +
 +0.9.2 — 2022-12-21
 +
 +  * Tim Düsterhus: Added concat and equals methods; changed join to accept an iterator.
 +  * Enhance explanation of locales and collations, and standardize on using ''$collator'' as an argument name everywhere.
  
 0.9.1 — 2022-12-16 0.9.1 — 2022-12-16
rfc/unicode_text_processing.1671198870.txt.gz · Last modified: 2022/12/16 13:54 by derick

Page Tools