The DNS carries only ASCII, so a name such as münchen.example.com has to be turned into xn--mnchen-3ya.example.com before it is looked up, and back again before it is shown to a user. Punycode is the encoding that does this for one label, and IDNA is the protocol around it that normalises the name, splits it into labels and adds the xn-- prefix. The punycode library implements the Bootstring algorithm of RFC 3492, including the optional mixed-case annotation that preserves letter case through a round trip. The punycode.idna library implements the ToASCII and ToUnicode operations of RFC 5891, with NFC normalisation through uunf and conversions for Domain_name.t values from the domain-name library. Punycode failures raise an exception that carries the byte offset and character index of the offending input.

Usage #

Basic Punycode Encoding/Decoding #

(* Encode a UTF-8 string to Punycode *)
let encoded = Punycode.encode_utf8 "münchen"
(* = "mnchen-3ya" *)

(* Decode Punycode back to UTF-8 *)
let decoded = Punycode.decode_utf8 "mnchen-3ya"
(* = "münchen" *)
(* Raises Punycode.Error on invalid input *)

Domain Label Operations #

(* Encode a domain label with ACE prefix *)
let label = Punycode.encode_label "münchen"
(* = "xn--mnchen-3ya" *)

(* Decode an ACE-prefixed label *)
let original = Punycode.decode_label "xn--mnchen-3ya"
(* = "münchen" *)

IDNA Domain Name Conversion #

(* Convert internationalized domain to ASCII for DNS lookup *)
let ascii_domain = Punycode_idna.to_ascii "münchen.example.com"
(* = "xn--mnchen-3ya.example.com" *)

(* Convert ASCII domain back to Unicode for display *)
let unicode_domain = Punycode_idna.to_unicode "xn--mnchen-3ya.example.com"
(* = "münchen.example.com" *)

Working with Unicode Code Points #

(* Encode an array of Unicode code points *)
let codepoints = [| Uchar.of_int 0x4ED6; Uchar.of_int 0x4EEC |]
let encoded = Punycode.encode codepoints
(* Result is Punycode string *)

(* Decode to code points *)
let decoded = Punycode.decode "ihqwcrb4cv8a8dqg056pqjye"
(* Result is Uchar.t array *)

Integration with domain-name Library #

(* Convert a Domain_name.t to ASCII *)
let domain = Domain_name.of_string_exn "münchen.example.com"
let ascii = Punycode_idna.ascii_of_domain domain
(* = Domain_name for "xn--mnchen-3ya.example.com" *)

(* Convert back to Unicode *)
let unicode = Punycode_idna.unicode_of_domain ascii
(* = original domain *)

Installation #

Install with opam:

$ opam install punycode

If opam cannot find the package, it may not yet be released in the public opam-repository. Add the overlay repository, then install it:

$ opam repo add samoht https://tangled.org/gazagnaire.org/opam-overlay.git
$ opam update
$ opam install punycode

Limitations #

Two IDNA 2008 checks are not yet implemented: the bidirectional text rules of RFC 5893 for right-to-left scripts, and the contextual rules of RFC 5892 Appendix A.1 for the zero-width joiner and non-joiner. Names in European and CJK scripts do not depend on them.

References #

RFC 3492 - Punycode: A Bootstring encoding of Unicode for IDNA
RFC 5891 - Internationalized Domain Names in Applications (IDNA): Protocol
RFC 5892 - Unicode Code Points and IDNA
RFC 5893 - Right-to-Left Scripts for IDNA
RFC 1035 - Domain Names Implementation and Specification

Licence #

ISC. See LICENSE.md.