Token<TKind> Struct ​
Definition ​
A token produced by lexing a TextFile.
c#
public sealed struct Token<TKind> : IEquatable<Token<TKind>>
where TKind : struct, EnumType Parameters ​
| Name | Description |
|---|---|
TKind | The type of the enum that identifies the different kinds of token. |
Constructors ​
| Name | Description |
|---|---|
Token(TKind, TextFile, Int32, Int32, Int32, Int32) | Initialises a new instance of the Token<TKind> struct. No validation is performed for performance; use CreateToken<TKind>(TKind, SourceMark) to create tokens with guaranteed consistent values. |
Properties ​
| Name | Description |
|---|---|
File | The TextFile the token comes from. |
Kind | The kind of the token. |
Length | The length of the token. |
Position | The TextFilePosition of the token in File. The start index, line and column are clamped to valid positions in the file for tokens at positions a TextFilePosition cannot represent, that is, a token starting on a line's terminator or a zero length token at the end of the file. |
StartColumnIndex | Zero based index of the start column of the token in File. |
StartIndex | The index of the start of the token in File. |
StartLineIndex | Zero based index of the start line of the token in File. |
Text | The text of the token. |
Methods ​
| Name | Description |
|---|---|
IsDirectlyBefore(Token<TKind>) | Returns true if this token ends exactly where other starts, that is, the tokens are directly adjacent in the file with nothing between them. Both tokens are assumed to come from the same file. |
ToString() | Returns a string representation of this token, that is, the Kind followed by the Text if the token has any. |