<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Ubase</title><link rel="stylesheet" href="odoc.css"/><meta charset="utf-8"/><meta name="generator" content="odoc 2.2.0"/><meta name="viewport" content="width=device-width,initial-scale=1.0"/><script src="highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script></head><body class="odoc"><nav class="odoc-nav"><a href="../index.html">Up</a> – <a href="../index.html">ubase</a> &#x00BB; Ubase</nav><header class="odoc-preamble"><h1>Module <code><span>Ubase</span></code></h1><p>Conversion from UTF8 latin letters to their base character.</p><p>Use this library to remove diacritics (accents, etc.) from Latin letters in UTF8 string.</p><p>It should work for all utf8 strings, regardless of normalization NFC, NFD, NFKD, NFKC.</p><p>PLEASE don't use this library to store your strings without accents! On the contrary, store them in full UTF8 encoding, and use this library to simplify searching and comparison.</p><p><a href="https://github.com/sanette/ubase">Source on github</a></p><ul class="at-tags"><li class="author"><span class="at-tag">author</span> San Vu Ngoc, 2019-2023</li></ul></header><nav class="odoc-toc"><ul><li><a href="#example">Example</a></li><li><a href="#removing-accents">Removing accents</a></li><li><a href="#utilities">Utilities</a></li></ul></nav><div class="odoc-content"><h2 id="example"><a href="#example" class="anchor"></a>Example</h2><pre class="language-ocaml"><code>let nfc = &quot;V\197\169 Ng\225\187\141c Phan&quot;;;
let nfd = &quot;Vu\204\131 Ngo\204\163c Phan&quot;;;

print_endline nfc;;
Vũ Ngọc Phan

print_endline nfd;;
Vũ Ngọc Phan

from_utf8 nfc;;
 - : string = &quot;Vu Ngoc Phan&quot;

from_utf8 nfd;;
- : string = &quot;Vu Ngoc Phan&quot;</code></pre><h2 id="removing-accents"><a href="#removing-accents" class="anchor"></a>Removing accents</h2><div class="odoc-spec"><div class="spec value anchored" id="val-from_utf8"><a href="#val-from_utf8" class="anchor"></a><code><span><span class="keyword">val</span> from_utf8 : <span>?malformed:string <span class="arrow">&#45;&gt;</span></span> <span>?strip:string <span class="arrow">&#45;&gt;</span></span> <span>string <span class="arrow">&#45;&gt;</span></span> string</span></code></div><div class="spec-doc"><p>Remove all diacritics on Latin letters from a standard string containing UTF8 text. Any malformed UTF8 will be replaced by the <code>malformed</code> parameter (by default &quot;?&quot;). If the optional parameter <code>strip</code> is present, all non-ASCII, non-Latin unicode characters will be replaced by the <code>strip</code> string (which can be empty). If both <code>malformed</code> and <code>strip</code> contain only ASCII characters, then the result of <code>from_utf8</code> is guaranteed to contain only ASCII characters.</p></div></div><div class="odoc-spec"><div class="spec value anchored" id="val-from_utf8_string"><a href="#val-from_utf8_string" class="anchor"></a><code><span><span class="keyword">val</span> from_utf8_string : <span>?malformed:string <span class="arrow">&#45;&gt;</span></span> <span>?strip:string <span class="arrow">&#45;&gt;</span></span> <span>string <span class="arrow">&#45;&gt;</span></span> string</span></code></div><div class="spec-doc"><p>Deprecated. Same as <a href="#val-from_utf8"><code>from_utf8</code></a>.</p></div></div><div class="odoc-spec"><div class="spec value anchored" id="val-uchar_replacement"><a href="#val-uchar_replacement" class="anchor"></a><code><span><span class="keyword">val</span> uchar_replacement : <span><span class="xref-unresolved">Stdlib</span>.Uchar.t <span class="arrow">&#45;&gt;</span></span> <span>string option</span></span></code></div><div class="spec-doc"><p><code>uchar_replacement u</code> returns the replacement string for <code>u</code>, representing its base letter without accent or diacritics, but only if <code>u</code> admits such a replacement.</p><p>A number of other conversions are performed, which are not about finding the base letter, but an ascii equivalent of some non-ascii utf8 characters: for instance, single quotation marks/apostrophes (U+2018, U+2019) and double quotation marks (U+201c, U+201d).</p><p>The returned string is guaranteed to be represented by a char that is different from <code>u</code>.</p></div></div><div class="odoc-spec"><div class="spec value anchored" id="val-uchar_to_char"><a href="#val-uchar_to_char" class="anchor"></a><code><span><span class="keyword">val</span> uchar_to_char : <span>?unknown:char <span class="arrow">&#45;&gt;</span></span> <span><span class="xref-unresolved">Stdlib</span>.Uchar.t <span class="arrow">&#45;&gt;</span></span> char</span></code></div><div class="spec-doc"><p>Convert a Uchar to a single char representing its base letter, without diacritics. For this, some simplifications have to be made. Unusual letters will be replaced by <code>unknown</code> (which defaults to '?').</p></div></div><h2 id="utilities"><a href="#utilities" class="anchor"></a>Utilities</h2><div class="odoc-spec"><div class="spec value anchored" id="val-isolatin_to_utf8"><a href="#val-isolatin_to_utf8" class="anchor"></a><code><span><span class="keyword">val</span> isolatin_to_utf8 : <span>?control:int <span class="arrow">&#45;&gt;</span></span> <span>string <span class="arrow">&#45;&gt;</span></span> string</span></code></div><div class="spec-doc"><p>Convert an ISO_8859_1 string to a UTF8 string. ISO_8859_1 chars in the range ''0x80..0x9F' are supposed to encode 'controls', and will be rendered by the UTF8 symbol whose integer code is given by the <code>control</code> parameter (defaults to <code>0x0080</code>)</p></div></div><div class="odoc-spec"><div class="spec value anchored" id="val-cp1252_to_utf8"><a href="#val-cp1252_to_utf8" class="anchor"></a><code><span><span class="keyword">val</span> cp1252_to_utf8 : <span>?undefined:int <span class="arrow">&#45;&gt;</span></span> <span>string <span class="arrow">&#45;&gt;</span></span> string</span></code></div><div class="spec-doc"><p>Convert a cp1252 string (also known as &quot;windows-1252&quot;) to a UTF8 string. The 5 unassigned codes will be rendered by the UTF8 symbol whose integer code is given by the <code>undefined</code> parameter (defaults to <code>0xFFFD</code>).</p></div></div><div class="odoc-spec"><div class="spec value anchored" id="val-is_space"><a href="#val-is_space" class="anchor"></a><code><span><span class="keyword">val</span> is_space : <span><span class="xref-unresolved">Stdlib</span>.Uchar.t <span class="arrow">&#45;&gt;</span></span> bool</span></code></div><div class="spec-doc"><p>Return true if the character is considered as a white space (this includes tab and newline).</p></div></div></div></body></html>