yshrsmz/unorm-dart

unorm_dart

Unit Test

Unicode 17.0 Normalization - NFC, NFD, NFKC, NFKD

Dart2 port of unorm

Functions

This module exports four functions: nfc, nfd, nfkc, and nfkd; one for each Unicode normalization. In the browser the functions are exported in the unorm global. In CommonJS environments you just require the module. Functions:

• unorm.nfd(str) – Canonical Decomposition
• unorm.nfc(str) – Canonical Decomposition, followed by Canonical Composition
• unorm.nfkd(str) – Compatibility Decomposition
• unorm.nfkc(str) – Compatibility Decomposition, followed by Canonical Composition

Usage

A simple usage example:

import "package:unorm_dart/unorm_dart.dart" as unorm;

void main() {
  var text = "The \u212B symbol invented by A. J. \u00C5ngstr\u00F6m " +
      "(1814, L\u00F6gd\u00F6, \u2013 1874) denotes the length " +
      "10\u207B\u00B9\u2070 m.";

  var combining = RegExp(r"[\u0300-\u036F]");

  print("Regular:  ${text}");
  print("NFC:      ${unorm.nfc(text)}");
  print("NFKC:     ${unorm.nfkc(text)}");
  print("NFKD: *   ${unorm.nfkd(text).replaceAll(combining, "")}");
  print(" * = Combining characters removed from decomposed form.");
}

Generating Unicode data

• Unicode normalization forms report
• Unicode data can be found from https://www.unicode.org/Public/UCD/latest/ucd

Manual Generation

To manually update the Unicode data and generate normalization files:

# Download latest Unicode data files
dart run tools/unicode_data_updater.dart

# Update lib/src/unormdata.dart
dart run tools/normalizer_gen.dart

Features and bugs

Please file feature requests and bugs at the issue tracker.