rfc:token_as_object
Differences
This shows you the differences between two versions of the page.
Next revision | Previous revisionNext revisionBoth sides next revision | ||
rfc:token_as_object [2020/02/13 09:37] – created nikic | rfc:token_as_object [2020/02/14 16:22] – nikic | ||
---|---|---|---|
Line 1: | Line 1: | ||
- | ====== PHP RFC: token_get_all() | + | ====== PHP RFC: Object-based |
* Date: 2020-02-13 | * Date: 2020-02-13 | ||
* Author: Nikita Popov < | * Author: Nikita Popov < | ||
Line 8: | Line 8: | ||
===== Introduction ===== | ===== Introduction ===== | ||
- | The '' | + | The '' |
===== Proposal ===== | ===== Proposal ===== | ||
- | '' | + | A new '' |
<PHP> | <PHP> | ||
Line 24: | Line 24: | ||
/** The starting position (0-based) in the tokenized string. */ | /** The starting position (0-based) in the tokenized string. */ | ||
public int $pos; | public int $pos; | ||
+ | | ||
+ | /** | ||
+ | * Same as token_get_all(), | ||
+ | * @return PhpToken[] | ||
+ | */ | ||
+ | public static function getAll(string $code, int $flags = 0): array; | ||
+ | | ||
+ | public function __construct(int $id, string $text, int $line = -1, int $pos = -1); | ||
} | } | ||
</ | </ | ||
+ | |||
+ | The '' | ||
It should be emphasized that **all** tokens are returned as objects, including single-char tokens. While this uses more memory than returning them as strings, experience has shown that the current string/ | It should be emphasized that **all** tokens are returned as objects, including single-char tokens. While this uses more memory than returning them as strings, experience has shown that the current string/ | ||
Line 35: | Line 45: | ||
* The token stores the position in the file, so that consumers don't have to compute and store it separately. | * The token stores the position in the file, so that consumers don't have to compute and store it separately. | ||
- | Finally (and this is the real motivation here), the tokens take up signficiantly | + | Finally (and this is the real motivation here), the tokens take up significantly |
< | < | ||
Line 46: | Line 56: | ||
</ | </ | ||
- | ==== Additional methods | + | ===== Open Questions ===== |
- | There are a few useful helper | + | ==== Additional |
- | Three suggestions are given as PHP code below. The '' | + | There are a few useful helper methods that could be added to the '' |
<PHP> | <PHP> | ||
Line 88: | Line 98: | ||
T_DOC_COMMENT, | T_DOC_COMMENT, | ||
T_OPEN_TAG, | T_OPEN_TAG, | ||
- | T_BAD_CHARACTER, | ||
]); | ]); | ||
} | } | ||
Line 102: | Line 111: | ||
} | } | ||
</ | </ | ||
+ | |||
+ | ==== Allowing extension of the class ==== | ||
+ | |||
+ | If the class is extended, should '' | ||
+ | |||
+ | ===== Rejected Features ===== | ||
+ | |||
+ | ==== Lazy token stream ==== | ||
+ | |||
+ | '' | ||
+ | |||
+ | This is not supported by the current proposal, because the current PHP lexer doesn' | ||
===== Backward Incompatible Changes ===== | ===== Backward Incompatible Changes ===== | ||
- | There are no backwards compatibility breaks, apart from the new constant name and the new class name. | + | There are no backwards compatibility breaks, apart from the new class name. |
===== Vote ===== | ===== Vote ===== |
rfc/token_as_object.txt · Last modified: 2020/11/12 13:33 by nikic