Token<TKind> Struct

Definition

A token produced by lexing a TextFile.

public sealed struct Token<TKind> : IEquatable<Token<TKind>>
   where TKind : struct, Enum

Name	Description
`TKind`	The type of the enum that identifies the different kinds of token.

Name	Description
`Token(TKind, TextFile, Int32, Int32, Int32, Int32)`	Initialises a new instance of the `Token<TKind>` struct. No validation is performed for performance; use `CreateToken<TKind>(TKind, SourceMark)` to create tokens with guaranteed consistent values.

Name	Description
`File`	The `TextFile` the token comes from.
`Kind`	The kind of the token.
`Length`	The length of the token.
`Position`	The `TextFilePosition` of the token in `File`. The start index, line and column are clamped to valid positions in the file for tokens at positions a `TextFilePosition` cannot represent, i.e. a token starting on a line's terminator or a zero length token at the end of the file.
`StartColumnIndex`	Zero based index of the start column of the token in `File`.
`StartIndex`	The index of the start of the token in `File`.
`StartLineIndex`	Zero based index of the start line of the token in `File`.
`Text`	The text of the token.

Name	Description
`IsDirectlyBefore(Token<TKind>)`	Returns `true` if this token ends exactly where `other` starts, i.e. the tokens are directly adjacent in the file with nothing between them. Both tokens are assumed to come from the same file.
`ToString()`	Returns a string representation of this token, i.e. the `Kind` followed by the `Text` if the token has any.