Chinese Word Segmentation and Counting
人工智慧 contains four Chinese characters, but a segmentation convention may treat it as one word. A submission limit, a JavaScript string length and a linguistic word count measure different things. There is no fixed conversion ratio between them. Start with the receiving platform’s definition.
Code units, code points and visible characters
- UTF-16 code unit: JavaScript
text.lengthcounts these units. Common Chinese characters usually take one; some rare characters and emoji take two or more. - Unicode code point:
[...text].lengthkeeps a surrogate pair together. Combining marks and emoji sequences can still contain several code points. - Grapheme cluster:
Intl.Segmenterwithgranularity: 'grapheme'groups text closer to a visible character. This is still different from a Chinese word.
These are complete, constructed examples, not a corpus benchmark. In the fourth row, é is e plus U+0301, rather than the precomposed é.
| Complete text | UTF-16 units | Code points | Grapheme clusters |
|---|---|---|---|
| 中文 | 2 | 2 | 2 |
| 𠮷 | 2 | 1 | 1 |
| 😀 | 2 | 1 | 1 |
| é | 2 | 2 | 1 |
| 👨👩👧👦 | 11 | 7 | 1 |
To reproduce the table in a browser supporting Intl.Segmenter, set text to a complete example and compare text.length, [...text].length and [...new Intl.Segmenter('zh-TW', {granularity: 'grapheme'}).segment(text)].length.
Counting characters is not Chinese word segmentation
我們使用人工智慧 has eight common Chinese characters. Under the explicit manual annotation 我們／使用／人工智慧, it has three words. That annotation illustrates a convention; it is not a claim that every segmenter must return the same boundaries.
Names, new terms, specialist vocabulary and compound-word conventions change the result. A product name may be kept whole for one task and split for search. To compare segmenters, fix the input corpus, software version, dictionary and reference annotations first. An accuracy or F1 figure without those conditions is not a general guarantee.
What this counter actually counts
The tool uses a hybrid count, not dictionary-based Chinese segmentation. Its character total is UTF-16 length. Its word total counts covered BMP Han, kana and Hangul characters individually, then adds the remaining browser Intl.Segmenter segments marked isWordLike.
| Complete input | Characters including spaces | Hybrid word count |
|---|---|---|
| 中文 ABC 123 | 10 | 4 |
The input contains two Han characters, two spaces, three Latin letters and three digits: ten UTF-16 units. The hybrid count is 中, 文, ABC and 123: four units. Punctuation and spaces are not separate words. Rare Han characters, combining sequences and other languages can behave differently because of character coverage and browser segmentation data.
Estimated reading time is covered CJK units divided by 500 plus other word units divided by 200, in minutes. These fixed speeds are assumptions, not measurements of an individual’s speed or comprehension.
When you need linguistic segmentation
- The official Jieba project documents dictionary and word-frequency processing, an HMM, custom dictionaries and different cutting modes.
- The official CKIP Transformers project provides Traditional Chinese models and segmentation tools. Evaluate them on reference examples from your own domain.
This counter does not call either model. For indexing, research or terminology analysis, inspect complete examples containing names and mixed languages instead of choosing by popularity alone.
A submission checklist
- Check whether the limit includes punctuation, spaces, headings and footnotes, and whether it measures characters, words or bytes.
- Compare this counter with the receiving platform using one complete sample containing Chinese, English, digits and emoji.
- Follow the receiving platform’s rules if results differ. Keep the input, counting convention and software version so another person can reproduce the count.