Multibyte Strings

Why Multibyte Strings Need Special Handling

PHP strings are sequences of bytes. Most core string functions — strlen(), substr(), strtoupper(), strpos(), str_split() — assume that one byte is one character. That assumption is true for plain English text (ASCII), but not for most of the world's writing.

On the web, text is encoded in UTF-8, where a single character can take 1 to 4 bytes:

Character	Description	Bytes in UTF-8
A	Latin letter	1
é	Latin letter with accent	2
₹	Rupee sign	3
த	Tamil letter	3
日	Chinese/Japanese character	3
😀	Emoji	4

When byte-based functions process such text, they count wrong, cut characters in half, and fail to change case. The mbstring extension ("multibyte string") provides functions that understand character encodings and work on characters instead of bytes.

---

Key Terms

Understanding three levels of "character" explains most multibyte behavior.

Term	Simple Meaning	Example
Byte	The smallest unit of storage	é is 2 bytes: C3 A9
Code point	One Unicode character number, written U+XXXX	é is U+00E9
Grapheme	What a reader sees as one character	👍🏽 (thumbs up + skin tone) is 1 grapheme made of 2 code points

• Byte functions (strlen(), substr()) count bytes.
• mb_* functions count code points.
• grapheme_* functions (from the intl extension) count graphemes.

Example

<?php
$samples = ["Hello", "Café", "₹500", "தமிழ்", "日本", "👍🏽"];

printf("%-8s %6s %8s %10s\n", "Text", "Bytes", "mb_chars", "graphemes");

foreach ($samples as $text) {
    printf(
        "%-8s %6d %8d %10d\n",
        $text,
        strlen($text),
        mb_strlen($text),
        grapheme_strlen($text)
    );
}

Output

Text      Bytes mb_chars  graphemes
Hello         5        5          5
Café         5        4          4
₹500        6        4          4
தமிழ்     15        5          3
日本        6        2          2
👍🏽      8        2          1

What the Results Show

• English text gives the same count at every level.
• Café: é is 2 bytes but 1 code point.
• தமிழ் (Tamil): 5 code points, but readers see 3 letters (த, மி, ழ்), because vowel signs and the virama combine with the previous consonant.
• 👍🏽: an emoji with a skin-tone modifier is 2 code points but displays as 1 symbol.
• The columns are misaligned. printf() padding (%-8s) counts bytes, so multibyte words receive too little padding. This is the problem this lesson is about, appearing in the example itself. mb_str_pad() (PHP 8.3+) pads by characters.

For most validation and display tasks, code-point counting with mb_* functions is the practical standard. Use grapheme_* functions when the exact visible character count matters, such as cursor positions, character-limited fields in scripts like Tamil and Hindi, or emoji handling.

---

The mbstring Extension

Is It Installed?

php -m | grep mbstring

On Windows, use findstr instead of grep. If it is missing, install php-mbstring (Linux) or enable extension=mbstring in php.ini (Windows). Almost every modern framework requires it.

Default Encoding

<?php
echo mb_internal_encoding();

Output

UTF-8

mbstring functions use the internal encoding when no encoding argument is given. It follows the default_charset setting in php.ini, which is UTF-8 by default. Keep it that way; all examples in this tutorial assume UTF-8.

---

Essential Multibyte Functions

Byte Function	Multibyte Function	Purpose
strlen()	mb_strlen()	Length in characters
substr()	mb_substr()	Extract part of a string
strpos() / stripos()	mb_strpos() / mb_stripos()	Find a position
strrpos()	mb_strrpos()	Find the last position
strtolower() / strtoupper()	mb_strtolower() / mb_strtoupper()	Change case
ucwords()	mb_convert_case($s, MB_CASE_TITLE)	Title case
ucfirst() / lcfirst()	mb_ucfirst() / mb_lcfirst()	First letter case (PHP 8.4+)
str_split()	mb_str_split()	Split into characters (PHP 7.4+)
str_pad()	mb_str_pad()	Pad to a length (PHP 8.3+)
trim()	mb_trim(), mb_ltrim(), mb_rtrim()	Trim, including Unicode whitespace (PHP 8.4+)
substr_count()	mb_substr_count()	Count occurrences

Length and Substring

<?php
$name = "Sébastien";

echo strlen($name) . "\n";
echo mb_strlen($name) . "\n";
echo substr($name, 0, 2) . "\n";
echo mb_substr($name, 0, 2) . "\n";

Output

10
9
S�
Sé

substr($name, 0, 2) takes S and only the first byte of é, producing an invalid character (�). mb_substr() takes two characters.

Changing Case

<?php
$texts = ["café", "straße", "ПРИВЕТ"];

foreach ($texts as $text) {
    echo strtoupper($text) . " | " . mb_strtoupper($text) . " | " . mb_strtolower($text) . "\n";
}

echo mb_convert_case("hello wORLD from chennai", MB_CASE_TITLE) . "\n";

Output

CAFé | CAFÉ | café
STRAßE | STRASSE | straße
ПРИВЕТ | ПРИВЕТ | привет
Hello World From Chennai

• Since PHP 8.2, strtoupper() changes only ASCII letters, so é and ß stay lowercase.
• mb_strtoupper() applies full Unicode case rules, including special cases such as German ß becoming SS.
• mb_strtolower() correctly lowercases Cyrillic text.
• MB_CASE_TITLE capitalizes the first letter of each word and lowercases the rest.

Splitting Into Characters

<?php
var_dump(count(str_split("₹50")));
print_r(mb_str_split("₹50"));

Output

int(5)
Array
(
    [0] => ₹
    [1] => 5
    [2] => 0
)

str_split() splits the three bytes of ₹ into three separate, invalid pieces, so it returns 5 elements. mb_str_split() returns 3 whole characters.

Padding (PHP 8.3+)

<?php
echo str_pad("₹50", 6, "*", STR_PAD_LEFT) . "\n";
echo mb_str_pad("₹50", 6, "*", STR_PAD_LEFT) . "\n";

Output

*₹50
***₹50

str_pad() sees ₹50 as 5 bytes and adds only one *. mb_str_pad() sees 3 characters and adds three.

Display Width: mb_strwidth() and mb_strimwidth()

East Asian full-width characters occupy two columns in fixed-width displays such as terminals:

<?php
echo mb_strwidth("abc") . "\n";
echo mb_strwidth("日本語") . "\n";
echo mb_strimwidth("日本語のテキスト", 0, 9, "…") . "\n";

Output

3
6
日本語の…

mb_strimwidth() trims to a display width (here 9 columns, including the 1-column …). Each Japanese character takes 2 columns, so four characters (8 columns) plus … fit.

---

Grapheme Functions (intl Extension)

When users see one character but it consists of several code points, mb_* functions can still split it. The intl extension provides grapheme-aware functions.

<?php
$review = "👍🏽 Great";

echo mb_substr($review, 0, 1) . "\n";
echo grapheme_substr($review, 0, 1) . "\n";
echo grapheme_strlen("தமிழ்") . "\n";

Output

👍
👍🏽
3

• mb_substr() takes only the first code point (the thumbs-up) and drops the skin-tone modifier.
• grapheme_substr() keeps the complete visible symbol.
• grapheme_strlen() counts the three visible Tamil letters.

Useful grapheme functions include grapheme_strlen(), grapheme_substr(), grapheme_strpos(), and grapheme_str_split() (PHP 8.4+). They require the intl extension (php-intl on Linux).

---

Validating and Converting Encodings

Checking for Valid UTF-8

Data from files, old databases, CSV exports, or external systems is sometimes not valid UTF-8.

<?php
$valid = "Café";
$latin1 = "Caf\xE9"; // "Café" encoded as ISO-8859-1

var_dump(mb_check_encoding($valid, 'UTF-8'));
var_dump(mb_check_encoding($latin1, 'UTF-8'));

Output

bool(true)
bool(false)

In ISO-8859-1 (Latin-1), é is the single byte E9. In UTF-8, that byte alone is invalid, so the check fails.

Converting to UTF-8

<?php
$latin1 = "Caf\xE9";
$utf8 = mb_convert_encoding($latin1, 'UTF-8', 'ISO-8859-1');

echo $utf8 . "\n";
var_dump(mb_check_encoding($utf8, 'UTF-8'));

Output

Café
bool(true)

mb_convert_encoding($string, $to, $from) re-encodes text. You must know the source encoding: common ones are ISO-8859-1, Windows-1252 (files from older Windows software), and UTF-16.

Note: mb_detect_encoding() can guess an encoding, but it is unreliable for short text because the same bytes can be valid in several encodings. Prefer knowing the source encoding. The old utf8_encode() and utf8_decode() functions are deprecated since PHP 8.2; use mb_convert_encoding() instead.

Why Invalid UTF-8 Matters

Invalid bytes cause failures in places that require valid UTF-8:

<?php
$broken = "Caf\xE9";

var_dump(json_encode($broken));
echo json_last_error_msg() . "\n";
echo json_encode($broken, JSON_INVALID_UTF8_SUBSTITUTE) . "\n";

Output

bool(false)
Malformed UTF-8 characters, possibly incorrectly encoded
"Caf\ufffd"

• json_encode() returns false for invalid UTF-8, which can silently produce an empty API response.
• JSON_INVALID_UTF8_SUBSTITUTE replaces invalid bytes with the replacement character U+FFFD instead of failing. Converting the data correctly at the source is still the better solution.
• htmlspecialchars() replaces invalid sequences too, because its default flags include ENT_SUBSTITUTE (since PHP 8.1).

---

Unicode Normalization

The same visible text can be stored with different code points. é can be:

• One code point: U+00E9 (precomposed, "NFC" form)
• Two code points: e + U+0301 combining acute accent (decomposed, "NFD" form)

<?php
$composed = "\u{00E9}";
$decomposed = "e\u{0301}";

var_dump($composed === $decomposed);
var_dump(mb_strlen($composed), mb_strlen($decomposed));

$normalized = Normalizer::normalize($decomposed, Normalizer::FORM_C);
var_dump($normalized === $composed);

Output

bool(false)
int(1)
int(2)
bool(true)

Both strings look identical on screen, yet they are not equal and have different lengths. Text pasted from macOS applications or certain keyboards often arrives in decomposed form. Normalizer::normalize() (from the intl extension) converts text to a standard form. Normalize to NFC before comparing, storing usernames, or checking for duplicates.

---

Regular Expressions and UTF-8

PCRE functions also work on bytes unless you add the u modifier:

<?php
var_dump(preg_match('/^.{4}$/', 'Café'));
var_dump(preg_match('/^.{4}$/u', 'Café'));
var_dump(preg_split('//u', '₹50', -1, PREG_SPLIT_NO_EMPTY));

Output

int(0)
int(1)
array(3) {
  [0]=>
  string(3) "₹"
  [1]=>
  string(1) "5"
  [2]=>
  string(1) "0"
}

• Without u, . matches a single byte, so the 5-byte Café does not match exactly four ..
• With u, the pattern and subject are treated as UTF-8, so . matches one character.
• preg_split('//u', ...) splits into characters, another way to get the same result as mb_str_split().

With u, invalid UTF-8 in the subject makes the match fail (returning false), which also acts as a validity check. Always use u for patterns that handle user-facing text.

---

UTF-8 Across the Whole Application

Multibyte functions only help if text stays UTF-8 from input to output. Configure every layer:

Layer	Setting
PHP files	Save as UTF-8 without BOM
PHP configuration	default_charset = "UTF-8" (the default)
HTML	<meta charset="UTF-8"> in <head>
HTTP header	Content-Type: text/html; charset=UTF-8 (PHP sends this by default when default_charset is UTF-8)
MySQL / MariaDB	Use utf8mb4 for databases, tables, and connections
PDO connection	mysql:host=localhost;dbname=app;charset=utf8mb4
JSON APIs	Content-Type: application/json (JSON is UTF-8 by definition)

Warning: In MySQL, the utf8 character set (an alias of utf8mb3) supports only up to 3 bytes per character and cannot store emoji or some rare characters. Inserts fail or data is truncated. Always use utf8mb4.

---

Practical Example: Validating a Display Name

<?php
function validateDisplayName(string $input): array
{
    $errors = [];

    if (!mb_check_encoding($input, 'UTF-8')) {
        return ['Name contains invalid characters.'];
    }

    $name = preg_replace('/\s+/u', ' ', trim($input));

    if (class_exists('Normalizer')) {
        $name = Normalizer::normalize($name, Normalizer::FORM_C);
    }

    $length = mb_strlen($name);

    if ($length < 2) {
        $errors[] = 'Name must have at least 2 characters.';
    }

    if ($length > 30) {
        $errors[] = 'Name must have at most 30 characters.';
    }

    if (!preg_match('/^[\p{L}\p{M}][\p{L}\p{M}\s.\'-]*$/u', $name)) {
        $errors[] = 'Name may contain only letters, spaces, dots, apostrophes, and hyphens.';
    }

    return $errors ?: ['OK: ' . $name];
}

$names = ["  Sébastien   Roux ", "தமிழ் செல்வி", "A", "Robot#42"];

foreach ($names as $input) {
    echo implode(' ', validateDisplayName($input)) . "\n";
}

Output

OK: Sébastien Roux
OK: தமிழ் செல்வி
Name must have at least 2 characters.
Name may contain only letters, spaces, dots, apostrophes, and hyphens.

Step-by-Step Explanation

1. mb_check_encoding() rejects invalid byte sequences before any processing.
2. trim() removes outer whitespace, and preg_replace('/\s+/u', ' ', ...) collapses repeated spaces.
3. Normalizer::normalize() (when the intl extension is available) converts the name to NFC so that visually identical names are stored identically.
4. mb_strlen() checks the length in characters, so non-English names are not unfairly rejected.
5. The regular expression uses Unicode properties:
   • \p{L} matches any letter in any language.
   • \p{M} matches marks, such as Tamil vowel signs and combining accents, which are needed for many scripts.
   • The name must start with a letter, followed by letters, marks, spaces, dots, apostrophes, or hyphens.
6. The u modifier makes \p{...} and \s work with UTF-8.
7. $errors ?: [...] returns the errors if there are any; otherwise it returns the success message.

An ASCII-only check such as ctype_alpha() or /^[a-zA-Z ]+$/ would reject both valid names, which is a common accessibility and localization bug.

---

Common Mistakes

Using strlen() for Character Limits

Effect: Users writing in Tamil, Hindi, Chinese, or with accents hit limits much earlier (3 bytes per character).

Fix: mb_strlen().

Cutting Text With substr()

Effect: Broken characters (�) and invalid UTF-8 that can break JSON, databases, and HTML output.

Fix: mb_substr() or mb_strimwidth().

Expecting strtoupper() to Handle Accents

Effect: café becomes CAFé.

Fix: mb_strtoupper().

Using MySQL utf8 Instead of utf8mb4

Effect: Emoji and some characters cannot be stored.

Fix: Use utf8mb4 for tables and the connection charset.

Forgetting the u Modifier in Regular Expressions

Effect: Patterns match bytes; character classes and quantifiers behave incorrectly.

Fix: Add u to patterns that process text.

Comparing Unnormalized Strings

Effect: Two identical-looking usernames are treated as different, or duplicates slip through validation.

Fix: Normalize to NFC with Normalizer::normalize().

---

Best Practices

• Use UTF-8 everywhere: files, PHP configuration, HTML, HTTP headers, and the database (utf8mb4).
• Use mb_* functions for any text a human writes or reads; keep byte functions for binary data and ASCII-only formats.
• Use grapheme_* functions when visible character counts must be exact.
• Validate external input with mb_check_encoding() and convert legacy data with mb_convert_encoding().
• Always use the u modifier in regular expressions for text, and Unicode properties such as \p{L} for letters.
• Normalize text to NFC before comparing or storing identifiers.

Official reference: https://www.php.net/manual/en/book.mbstring.php