Appendix:Control characters
Besides alphabetic characters and symbols, Unicode also includes a variety of control characters with no graphical representation. While some of these are actually used to modify certain characters, others are largely disused remnants from older computing systems.
C0 (ASCII and derivatives)
[edit]C0 control codes are in the Unicode range U+0000-U+001F, and were inherited from the ASCII standard. Often, these cannot be input directly because they fulfill specific low-level functions in the operating system.
Seq | Dec | Hex | Acro | Name | C | Description | |||
---|---|---|---|---|---|---|---|---|---|
^@ |
00 | 00 | NUL | Null | \0
|
Originally used to allow gaps to be left on paper tape for edits. Later used for padding after a code that might take a terminal some time to process (e.g. a carriage return or line feed on a printing terminal). Now often used as a string terminator, especially in the C programming language. | |||
^A |
01 | 01 | SOH | Start of Heading | First character of a message header. | ||||
^B |
02 | 02 | STX | Start of text | First character of message text, and may be used to terminate the message heading. | ||||
^C |
03 | 03 | ETX | End of Text | Marks the end of the message text, although other data may still follow. Often used as a "break" character (Ctrl-C) to interrupt or terminate a program or process. | ||||
^D |
04 | 04 | EOT | End of Transmission | Marks the end of a transmission. Used on Unix to signal end-of-file condition on, or to logout from a terminal. | ||||
^E |
05 | 05 | ENQ | Enquiry | Signal intended to trigger a response at the receiving end, to see if it is still present. | ||||
^F |
06 | 06 | ACK | Acknowledge | Response to an ENQ, or an indication of successful receipt of a message. | ||||
^G |
07 | 07 | BEL | Bell | \a
|
Originally used to sound a bell on the terminal. Later used for a beep on systems that didn't have a physical bell. May also quickly turn on and off inverse video (a visual bell). | |||
^H |
08 | 08 | BS | Backspace | \b
|
Move the cursor one position leftwards. On input, this may delete the character to the left of the cursor. On output, where in early computer technology a character once printed could not be erased, the backspace was sometimes used to generate accented characters in ASCII. For example, à could be produced using the three character sequence a BS ` (0x61 0x08 0x60 ). This usage is now deprecated and generally not supported. To provide disambiguation between the two potential uses of backspace, the cancel character control code was made part of the standard C1 control set.
| |||
^I |
09 | 09 | HT | Character Tabulation, Horizontal Tabulation | \t
|
Position to the next character tab stop. | |||
^J |
10 | 0A | LF | Line Feed | \n
|
On typewriters, printers, and some terminal emulators, moves the cursor down one row without affecting its column position. On Unix, used to mark end-of-line. In MS-DOS, Windows, and various network standards, LF is used following CR as part of the end-of-line mark. | |||
^K |
11 | 0B | VT | Line Tabulation, Vertical Tabulation | \v
|
Position the form at the next line tab stop. | |||
^L |
12 | 0C | FF | Form Feed | \f
|
On printers, load the next page. Treated as whitespace in many programming languages, and may be used to separate logical divisions in code. In some terminal emulators, it clears the screen. | |||
^M |
13 | 0D | CR | Carriage Return | \r
|
Originally used to move the cursor to column zero while staying on the same line. On Mac OS (pre-Mac OS X), as well as in earlier systems such as the Apple II and Commodore 64, used to mark end-of-line. In MS-DOS, Windows, and various network standards, it is used preceding LF as part of the end-of-line mark. The Enter or Return key on a keyboard will send this character, but it may be converted to a different end-of-line sequence by a terminal program. | |||
^N |
14 | 0E | SO | Shift Out | Switch to an alternate character set. | ||||
^O |
15 | 0F | SI | Shift In | Return to regular character set after Shift Out. | ||||
^P |
16 | 10 | DLE | Data Link Escape | Cause the following octets to be interpreted as raw data, not as control codes or graphic characters. Returning to normal usage would be implementation-dependent. | ||||
^Q |
17 | 11 | DC1 | Device Control One (XON) | These four control codes are reserved for device control, with the interpretation dependent upon the device they were connected. DC1 and DC2 were intended primarily to indicate activating a device while DC3 and DC4 were intended primarily to indicate pausing or turning off a device. In actual practice DC1 and DC3 (known also as XON and XOFF respectively in this usage) quickly became the de facto standard for software flow control. | ||||
^R |
18 | 12 | DC2 | Device Control Two | |||||
^S |
19 | 13 | DC3 | Device Control Three (XOFF) | |||||
^T |
20 | 14 | DC4 | Device Control Four | |||||
^U |
21 | 15 | NAK | Negative Acknowledge | Sent by a station as a negative response to the station with which the connection has been set up. In binary synchronous communication protocol, the NAK is used to indicate that an error was detected in the previously received block and that the receiver is ready to accept retransmission of that block. In multipoint systems, the NAK is used as the not-ready reply to a poll. | ||||
^V |
22 | 16 | SYN | Synchronous Idle | Used in synchronous transmission systems to provide a signal from which synchronous correction may be achieved between data terminal equipment, particularly when no other character is being transmitted. | ||||
^W |
23 | 17 | ETB | End of Transmission Block | Indicates the end of a transmission block of data when data are divided into such blocks for transmission purposes. | ||||
^X |
24 | 18 | CAN | Cancel | Indicates that the data preceding it are in error or are to be disregarded. | ||||
^Y |
25 | 19 | EM | End of medium | Intended as means of indicating on paper or magnetic tapes that the end of the usable portion of the tape had been reached. | ||||
^Z |
26 | 1A | SUB | Substitute | Originally intended for use as a transmission control character to indicate that garbled or invalid characters had been received. It has often been put to use for other purposes when the in-band signaling of errors it provides is unneeded, especially where robust methods of error detection and correction are used, or where errors are expected to be rare enough to make using the character for other purposes advisable. | ||||
^[ |
27 | 1B | ESC | Escape | The Esc key on the keyboard will cause this character to be sent on most systems. It can be used in software user interfaces to exit from a screen, menu, or mode, or in device-control protocols (e.g., printers and terminals) to signal that what follows is a special command sequence rather than normal text. In systems based on ISO/IEC 2022, even if another set of C0 control codes are used, this octet is required to always represent the escape character. | ||||
^\ |
28 | 1C | FS | File Separator | Can be used as delimiters to mark fields of data structures. If used for hierarchical levels, US is the lowest level (dividing plain-text data items), while RS, GS, and FS are of increasing level to divide groups made up of items of the level beneath it. | ||||
^] |
29 | 1D | GS | Group separator | |||||
^^ |
30 | 1E | RS | Record Separator | |||||
^_ |
31 | 1F | US | Unit separator | |||||
While not technically part of the C0 control character range, the following two characters are defined in ISO/IEC 2022 as always being available regardless of which sets of control characters and graphics characters have been registered. They can be thought of as having some characteristics of control characters. | |||||||||
32 | 20 | SP | Space | Space is a graphic character. Its visual representation consists of the absence of a graphic symbol. Analogous to old typesetter technology, where pressing spacebar causes the active position to be advanced by one character position. In some electronics, still used as a control character in this sense. In some applications, Space can be considered a lowest-level "word separator" to be used with the adjacent separator characters. Contrast with backspace. | |||||
^? |
127 | 7F | DEL | Delete | Not technically part of the C0 control character range, this was originally used to mark deleted characters on paper tape, since any character could be changed to all ones by punching holes everywhere. On VT100 compatible terminals, this is the character generated by the key labelled ⌫, usually called backspace on modern machines, and does not correspond to the PC delete key. |
C1 set
[edit]C1 control codes are in the Unicode range U+0080-U+009F, and were inherited from the ISO 8859 series of standards. Nowadays, these control codes are rarely if ever used for their intended purpose, and often their presence in the text indicates an ill-formatted alternate character set, such as Windows-1252.
Esc+ | Dec | Hex | Acro | Name | Description |
---|---|---|---|---|---|
@ | 128 | 80 | PAD | Padding Character | Listed as XXX in Unicode. Not part of ISO/IEC 6429 (ECMA-48). For use as a padding byte in multibyte encodings. |
A | 129 | 81 | HOP | High Octet Preset | Listed as XXX in Unicode. Not part of ISO/IEC 6429 (ECMA-48). Originally intended for defining the value of the high octet, i.e. high byte, in multibyte encodings (especially ISO/IEC 2022) to reduce redundancy by only transmitting the lower byte of each character. |
B | 130 | 82 | BPH | Break Permitted Here | Follows a graphic character where a line break is permitted. Roughly equivalent to a soft hyphen except that the means for indicating a line break is not necessarily a hyphen. Not part of the first edition of ISO/IEC 6429. |
C | 131 | 83 | NBH | No Break Here | Follows the graphic character that is not to be broken. Not part of the first edition of ISO/IEC 6429. |
D | 132 | 84 | IND | Index | Move the active position one line down, to eliminate ambiguity about the meaning of LF. Deprecated in 1988 and withdrawn in 1992 from ISO/IEC 6429 (1986 and 1991 respectively for ECMA-48). |
E | 133 | 85 | NEL | Next Line | Equivalent to CR+LF. Used to mark end-of-line on some IBM mainframes. |
F | 134 | 86 | SSA | Start of Selected Area | Used by block-oriented terminals. Selected areas are eligible for transmission. |
G | 135 | 87 | ESA | End of Selected Area | |
H | 136 | 88 | HTS | Character Tabulation Set Horizontal Tabulation Set |
Causes a character tabulation stop to be set at the active position. |
I | 137 | 89 | HTJ | Character Tabulation With Justification Horizontal Tabulation With Justification |
Similar to Character Tabulation, except that instead of spaces or lines being placed after the preceding characters until the next tab stop is reached, the spaces or lines are placed preceding the active field so that preceding graphic character is placed just before the next tab stop. |
J | 138 | 8A | VTS | Line Tabulation Set Vertical Tabulation Set |
Causes a line tabulation stop to be set at the active position. |
K | 139 | 8B | PLD | Partial Line Forward Partial Line Down |
Used to produce subscripts and superscripts in ISO/IEC 6429, e.g., in a printer. Subscripts use PLD text PLU while superscripts use PLU text PLD. .
|
L | 140 | 8C | PLU | Partial Line Backward Partial Line Up | |
M | 141 | 8D | RI | Reverse Line Feed Reverse Index |
The opposite of Index (0x84, IND). Moves the cursor one line up. |
N | 142 | 8E | SS2 | Single-Shift 2 | Next character invokes a graphic character from the G2 or G3 graphic sets respectively. In systems that conform to ISO/IEC 4873 (ECMA-43), even if a C1 set other than the default is used, these two octets may only be used for this purpose. |
O | 143 | 8F | SS3 | Single-Shift 3 | |
P | 144 | 90 | DCS | Device Control String | Followed by a string of printable characters (0x20 through 0x7E) and format effectors (0x08 through 0x0D), terminated by ST (0x9C). |
Q | 145 | 91 | PU1 | Private Use 1 | Reserved for a function without standardized meaning for private use as required, subject to the prior agreement of the sender and the recipient of the data. |
R | 146 | 92 | PU2 | Private Use 2 | |
S | 147 | 93 | STS | Set Transmit State | A command to the receiving device to prepare to transmit data. |
T | 148 | 94 | CCH | Cancel Character | Destructive backspace, intended to eliminate ambiguity about meaning of BS. |
U | 149 | 95 | MW | Message Waiting | Lights the message-waiting indicator on the receiving device, |
V | 150 | 96 | SPA | Start of Protected Area | Used by block-oriented terminals. Protected areas are guarded from transmission and editing. |
W | 151 | 97 | EPA | End of Protected Area | |
X | 152 | 98 | SOS | Start of String | Followed by a control string terminated by ST (0x9C) that may contain any character except SOS or ST. Not part of the first edition of ISO/IEC 6429. |
Y | 153 | 99 | SGC SGCI |
Single Graphic Character Introducer | Listed as XXX in Unicode. Not part of ISO/IEC 6429. Intended for use as an escape code to encode any available character regardless of the currently selected character set or high octet preset. |
Z | 154 | 9A | SCI | Single Character Introducer | To be followed by a single printable character (0x20 through 0x7E) or format effector (0x08 through 0x0D). The intent was to provide a means by which a control function or a graphic character that would be available regardless of which graphic or control sets were in use could be defined. Definitions of what the following byte would invoke was never implemented in an international standard. Not part of the first edition of ISO/IEC 6429. |
[ | 155 | 9B | CSI | Control Sequence Introducer | Used to introduce control sequences that take parameters. |
\ | 156 | 9C | ST | String Terminator | Terminates the command string for DCS (0x90), SOS (0x98), OSC (0x9D), PM (0x9E) and APC (0x9F). |
] | 157 | 9D | OSC | Operating System Command | Followed by a string of printable characters (0x20 through 0x7E) and format effectors (0x08 through 0x0D), terminated by ST (0x9C). These three control codes were intended for use to allow in-band signaling of protocol information, but are rarely used for that purpose. |
^ | 158 | 9E | PM | Privacy Message | |
_ | 159 | 9F | APC | Application Program Command |
Unicode control characters
[edit]Unicode control characters are control characters which do not occur in the C0 or C1 sets. While the C0 and C1 control characters are of the type Cc
(Other, Control), control characters exclusive to Unicode are in the category Cf
(Other, Format).
Dec | Hex | Acro | Name | Description |
---|---|---|---|---|
173 | AD | SHY | Soft Hyphen | This character allows line breaking of words at specific points, and if a word is broken up like this this character will show up as a hyphen. |
847 | 34F | CGJ | Combining Grapheme Joiner | Separates diacritical marks. Can sometimes act as a variation selector (for example, to differentiate between the dieresis and umlaut usages of ¨). |
1807 | 70F | SAM | Syriac Abbreviation Mark | In Syriac script, this marks the beginning of an abbreviation. This control character is rendered as a line at the top with dots at each end as well as in the middle which continues until the end of the word. It is used to mark contractions as well as Syriac numerals. |
6158 | 180E | MVS | Mongolian Vowel Separator | Used orthographically in Mongolian script. |
8203 | 200B | ZWSP | Zero-Width Space | Used to allow line breaking for scripts which traditionally do not use spaces, like Thai. This character is always invisible. |
8204 | 200C | ZWNJ | Zero-Width Non-Joiner | Specifically prevents joining of characters, especially in cursive scripts such as Arabic. This character is needed for correct notation of some languages, most notably Persian. When used in Indic scripts, it specifically enforces the halant form. |
8205 | 200D | ZWJ | Zero-Width Joiner | Specifically enforces joining of characters, especially in cursive scripts such as Arabic. This is used, for example, to display Arabic characters standalone in various positions. When used in Indic scripts, it specifically enforces the half form. |
8206 | 200E | LRM | Left-to-Right Mark | Specifies the writing direction in bidirectional text. |
8207 | 200F | RLM | Right-to-Left Mark | |
8234 | 202A | LRE | Left-to-Right Embedding | Used to embed a text in a specific writing direction. Used specifically where the writing direction of the surrounding environment is not known. |
8235 | 202B | RLE | Right-to-Left Embedding | |
8236 | 202C | Pop Directional Formatting | Acts as a closing tag for LRE, RLE, LRO and RLO, restoring the writing direction to its previous state. | |
8237 | 202D | LRO | Left-to-Right Override | Forces a text to be written in a specific writing direction, even if the writing system would normally not be written in this direction. |
8238 | 202E | RLO | Right-to-Left Override | |
8288 | 2060 | WJ | Word Joiner | Used specifically as a zero-width non-breaking space, to disambiguate from the use of ZWNBSP as a byte order mark. |
8289 | 2061 | FA | Function Application | In mathematical formulae, indicates an implied application of a function, for use by automated programs. |
8290 | 2062 | IT | Invisible Times | In mathematical formulae, indicates an implied multiplication operator, for use by automated programs. |
8291 | 2063 | IS | Invisible Separator | In mathematical formulae, indicates an implied separator of items in a list (of e.g. indices), for use by automated programs. |
8292 | 2064 | IP | Invisible Plus | In mathematical formulae, indicates an implied addition operator, for use by automated programs. |
8294 | 2066 | LRI | Left-to-Right Isolate | Used to set the writing direction of a section of text independently from that of the surrounding text. |
8295 | 2067 | RLI | Right-to-Left Isolate | |
8296 | 2068 | FSI | First Strong Isolate | Used to set the writing direction of a section of text to match the default writing direction of the first strongly-directional character in the section, independently of the writing direction of the surrounding text. |
8297 | 2069 | PDI | Pop Directional Isolate | Acts as a closing tag for LRI, RLI, and FSI, restoring the writing direction to its previous state. |
8298 | 206A | ISS | Inhibit Symmetric Swapping | Enables or disables the mirroring of specific glyphs, such as the parentheses, in a bidirectional context. Deprecated. |
8299 | 206B | ASS | Activate Symmetric Swapping | |
8300 | 206C | IAFS | Inhibit Arabic Form Shaping | Enables or disables the shaping of Arabic Presentation Forms. Deprecated. |
8301 | 206D | AAFS | Activate Arabic Form Shaping | |
8302 | 206E | NADS | National Digit Shapes | Replaces the Arabic digits (123) with Arabic-Indic digits (١٢٣). Deprecated. |
8303 | 206F | NODS | Nominal Digit Shapes | |
65279 | FEFF | ZWNBSP | Zero-Width Non-Breaking Space | This character is nowadays used as a Byte Order Mark (BOM), to allow programs to detect whether a given UTF-16 or UTF-32 text is formatted in big-endian or little-endian. Its use as a zero-width non-breaking space is now deprecated - the WJ control character should be used for this purpose. |
65529 | FFF9 | IAA | Interlinear Annotation Anchor | These characters are intended to be used to add annotations to words, as in, for example, Japanese furigana. The first character, IAA, is to be added at the beginning of the word to be annotated. The IAS character marks the start of the annotation, and the IAT character ends the annotation. Note however that most implementations do not support these control characters. |
65530 | FFFA | IAS | Interlinear Annotation Separator | |
65531 | FFFB | IAT | Interlinear Annotation Terminator | |
78896 | 13430 | VERT | Egyptian Hieroglyph Vertical Joiner | Stacks the preceding hieroglyph group on top of the following hieroglyph group. |
78897 | 13431 | HOR | Egyptian Hieroglyph Horizontal Joiner | Stacks the preceding hieroglyph group before the following hieroglyph group (to the left of it if left-to-right, else to the right). |
78898 | 13432 | ST | Egyptian Hieroglyph Insert at Top Start | Stacks the following hieroglyph group in the top initial corner (left corner if left-to-right, else right corner) of the preceding hieroglyph group. |
78899 | 13433 | SB | Egyptian Hieroglyph Insert at Bottom Start | Stacks the following hieroglyph group in the bottom initial corner (left corner if left-to-right, else right corner) of the preceding hieroglyph group. |
78900 | 13434 | ET | Egyptian Hieroglyph Insert at Top End | Stacks the following hieroglyph group in the top final corner (right corner if left-to-right, else left corner) of the preceding hieroglyph group. |
78901 | 13435 | EB | Egyptian Hieroglyph Insert at Bottom End | Stacks the following hieroglyph group in the bottom final corner (right corner if left-to-right, else left corner) of the preceding hieroglyph group. |
78902 | 13436 | OVERLAY | Egyptian Hieroglyph Overlay Middle | Stacks the following hieroglyph group in the middle of the preceding hieroglyph group. |
78903 | 13437 | BEGIN | Egyptian Hieroglyph Begin Segment | Defines the start of a hieroglyph group. |
78904 | 13438 | END | Egyptian Hieroglyph End Segment | Defines the end of a hieroglyph group. |
Language tags
[edit]Language tags are encoded in the Unicode range U+E0000-U+E007F. These were added as a way to represent the language of a text without having to resort to higher-level mechanisms. They were intended specifically for Chinese, to allow a text to unambiguously represent the simplified or traditional form of a Chinese character. However, these characters haven't caught on, and their use is now deprecated by Unicode.
Example:
Literal input | Unicode character codes | Displayed output |
---|---|---|
次次次 | 6B21 E0001 E007A E0068 E002D E0048 E0061 E006E E0074 6B21 E007F 6B21
|
次次次 |
Variation selectors
[edit]Variation selectors are encoded in three Unicode ranges scattered across the planes. Their purpose is to mark variations of a character in certain cases where it may be appropriate to specify one form of a character. The variations are not arbitrary, but specifically listed in the Unicode standard. There are currently three main uses of variation selectors:
- Variation selectors 1-16 are encoded in the U+FE00-U+FE0F block. Only the first three and the last two are ever actually used in Unicode. The first (VARIATION SELECTOR-1, U+FE00) is used for many different characters from several scripts, while the second (VARIATION SELECTOR-2, U+FE01) and third (VARIATION SELECTOR-3, U+FE02) are used only for variant forms of Han ideographs. The last two are used to specify text (monochrome) or emoji forms of characters; VARIATION SELECTOR-15 (U+FE0E) selects the text form, while VARIATION SELECTOR-16 (U+FE0F) selects the emoji form.
- Variation selectors 17-256 are encoded in the U+E0100-U+E01EF block. This set is intended to define special variants of Han ideographs.
- The Mongolian block has its own set of variation selectors which is used orthographically in this script.