2017 lines
88 KiB
HTML
2017 lines
88 KiB
HTML
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
|
|
"http://www.w3.org/TR/html4/loose.dtd">
|
|
<!--
|
|
- Do a paragraph check <p>
|
|
- The Unicode char tables are messed up, but it can't be fixed w/o CSS2
|
|
support, which I do not currently find in web browsers.
|
|
- Can I have bangs, plusses, or slashes in #tags? Spaces?
|
|
Yes: plus, bang, star No: space Yes: slash, question, ampersand
|
|
You can't put sharp in a path, so anything goes, really.
|
|
Nonetheless, some of these confuse Netscape, so I'll avoid them.
|
|
-->
|
|
|
|
<!--========================================================================-->
|
|
<html lang=en-US>
|
|
<head>
|
|
<meta name="keywords" content="Scheme, programming language, list processing, SRFI, underage lesbian sluts">
|
|
<link rev=made href="mailto:shivers@ai.mit.edu">
|
|
<title>SRFI 14: Character-set Library</title>
|
|
|
|
<!-- Should have a media=all to get, for example, printing to work.
|
|
== But my Netscape will completely ignore the tag if I do that.
|
|
-->
|
|
<style type="text/css">
|
|
/* A little general layout hackery for headers & the title. */
|
|
body { margin-left: +7%;
|
|
font-family: "Helvetica", sans-serif;
|
|
}
|
|
/* Netscape workaround: */
|
|
td, th { font-family: "Helvetica", sans-serif; }
|
|
|
|
code, pre { font-family: "courier new", "courier"; }
|
|
|
|
div.inset { margin-left: +5%; }
|
|
|
|
h1 { margin-left: -5%; }
|
|
h1, h2 { clear: both; }
|
|
h1, h2, h3, h4, h5, h6 { color: blue }
|
|
div.title-text { font-size: large; font-weight: bold; }
|
|
h3 { margin-top: 2em; margin-bottom: 0em }
|
|
|
|
div.indent { margin-left: 2em; } /* General indentation */
|
|
pre.code-example { margin-left: 2em; } /* Indent code examples. */
|
|
|
|
/* "Continue" class marks text that isn't really the start
|
|
** of a new paragraph -- e.g., continuing a para after a
|
|
** code sample.
|
|
*/
|
|
p.continue { text-indent: 0em; margin-top: 0em}
|
|
|
|
/* This stuff is for definition lists of defined procedures.
|
|
** A proc-def1 is used when you want a stack of procs to go
|
|
** with one dd body. In this case, make the first
|
|
** proc a proc-def1, following ones proc-defi's, and the last one
|
|
** a proc-defn.
|
|
**
|
|
** Unfortunately, Netscape has huge bugs with respect to style
|
|
** sheets and dl list rendering. We have to set truly random
|
|
** values here to get the rendering to come out. The proper values
|
|
** are in the following style sheet, for Internet Explorer.
|
|
** In the following settings, the *comments* say what the
|
|
** setting *really* causes Netscape to do.
|
|
**
|
|
** Ugh. Professional coders sacrifice their self-respect,
|
|
** that others may live.
|
|
*/
|
|
/* m-t ignored; m-b sets top margin space. */
|
|
dt.proc-def1 { margin-top: 0ex; margin-bottom: 3ex; }
|
|
dt.proc-defi { margin-top: 0ex; margin-bottom: 0ex; }
|
|
dt.proc-defn { margin-top: 0ex; margin-bottom: 0ex; }
|
|
|
|
/* m-t works weird depending on whether or not the last line
|
|
** of the previous entry was a pre. Set to zero.
|
|
*/
|
|
dt.proc-def { margin-top: 0ex; margin-bottom: 3ex; }
|
|
|
|
/* m-b sets space between dd & dt; m-t ignored. */
|
|
dd.proc-def { margin-bottom: 0.5ex; margin-top: 0ex; }
|
|
|
|
|
|
/* Boldface the name of a procedure when it's being defined. */
|
|
code.proc-def { font-weight: bold; font-size: 110%}
|
|
|
|
/* For the index of procedures.
|
|
** Same hackery as for dt.proc-def, above.
|
|
*/
|
|
/* m-b sets space between dd & dt; m-t ignored. */
|
|
dd.proc-index { margin-bottom: 0ex; margin-top: 0ex; }
|
|
/* What the fuck? */
|
|
pre.proc-index { margin-top: -2ex; }
|
|
|
|
/* Pull the table of contents back flush with the margin.
|
|
** Both NS & IE screw this up in different ways.
|
|
*/
|
|
#toc-table { margin-top: -2ex; margin-left: -5%; }
|
|
|
|
/* R5RS proc names are in italic; extended R5RS names
|
|
** in italic boldface.
|
|
*/
|
|
span.r5rs-proc { font-weight: bold; }
|
|
span.r5rs-procx { font-style: italic; font-weight: bold; }
|
|
|
|
/* Spread out bibliographic lists. */
|
|
/* More Netscape-specific lossage; see the following stylesheet
|
|
** for the proper values (used by IE).
|
|
*/
|
|
dt.biblio { margin-bottom: 3ex; }
|
|
|
|
/* Links to draft copies (e.g., not at the official SRFI site)
|
|
** are colored in red, so people will use them during the
|
|
** development process and kill them when the document's done.
|
|
*/
|
|
a.draft { color: red; }
|
|
|
|
</style>
|
|
|
|
<style type="text/css" media=all>
|
|
/* Nastiness: Here, I'm using a bug to work around a bug.
|
|
** Netscape rendering bugs mean you need bogus <dt> and <dd>
|
|
** margin settings -- settings which screw up IE's proper rendering.
|
|
** Fortunately, Netscape has *another* bug: it will ignore this
|
|
** media=all style sheet. So I am placing the (proper) IE values
|
|
** here. Perhaps, one day, when these rendering bugs are fixed,
|
|
** this gross hackery can be removed.
|
|
*/
|
|
dt.proc-def1 { margin-top: 3ex; margin-bottom: 0ex; }
|
|
dt.proc-defi { margin-top: 0ex; margin-bottom: 0ex; }
|
|
dt.proc-defn { margin-top: 0ex; margin-bottom: 0.5ex; }
|
|
dt.proc-def { margin-top: 3ex; margin-bottom: 0.5ex; }
|
|
|
|
pre { margin-top: 1ex; }
|
|
|
|
dd.proc-def { margin-bottom: 2ex; margin-top: 0.5ex; }
|
|
|
|
/* For the index of procedures.
|
|
** Same hackery as for dt.proc-def, above.
|
|
*/
|
|
dd.proc-index { margin-top: 0ex; }
|
|
pre.proc-index { margin-top: 0ex; }
|
|
|
|
/* Spread out bibliographic lists. */
|
|
dt.biblio { margin-top: 3ex; margin-bottom: 0ex; }
|
|
dd.biblio { margin-bottom: 1ex; }
|
|
</style>
|
|
</head>
|
|
|
|
<body>
|
|
|
|
<!--========================================================================-->
|
|
<h1>Title</h1>
|
|
<div class=title-text>
|
|
Character-set Library
|
|
</div>
|
|
|
|
<!--========================================================================-->
|
|
<h1>Author</H1>
|
|
<address>
|
|
<a href="http://www.ai.mit.edu/~shivers/">Olin Shivers</A> /
|
|
<a href="mailto:shivers@ai.mit.edu">shivers@ai.mit.edu</A>
|
|
</address>
|
|
|
|
<!--========================================================================-->
|
|
<h1>Table of contents</H1>
|
|
|
|
<!-- A bug in netscape (?) keeps the first link in this UL from being active.
|
|
==== So the Abstract link be dead. 99/8/22 -Olin
|
|
-->
|
|
<ul id=toc-table>
|
|
<li><a href="#Abstract">Abstract</a>
|
|
<li><a href="#VariableIndex">Variable index</a>
|
|
<li><a href="#Rationale">Rationale</a>
|
|
<ul>
|
|
<li><a href="#LinearUpdateOperations">"Linear-update" operations</a>
|
|
<li><a href="#ExtraSRFI">Extra-SRFI recommendations</a>
|
|
</ul>
|
|
|
|
<li><a href="#Specification">Specification</a>
|
|
<ul>
|
|
<li><a href="#GeneralProcs">General procedures</a>
|
|
<li><a href="#Iterating">Iterating over character sets</a>
|
|
<li><a href="#Creating">Creating character sets</a>
|
|
<li><a href="#Querying">Querying character sets</a>
|
|
<li><a href="#Algebra">Character set algebra</a>
|
|
<li><a href="#StandardCharsets">Standard character sets</a>
|
|
</ul>
|
|
|
|
<li><a href="#StandardCharsetDefs">Unicode, Latin-1 and ASCII definitions of the standard character sets</a>
|
|
<li><a href="#ReferenceImp">Reference implementation</a>
|
|
<li><a href="#Acknowledgements">Acknowledgements</a>
|
|
<li><a href="#Links">References & Links</a>
|
|
<li><a href="#Copyright">Copyright</a>
|
|
</ul>
|
|
|
|
<!--========================================================================-->
|
|
<h1><a name="Abstract">Abstract</a></H1>
|
|
<p>
|
|
|
|
The ability to efficiently represent and manipulate sets of characters is an
|
|
unglamorous but very useful capability for text-processing code -- one that
|
|
tends to pop up in the definitions of other libraries. Hence it is useful to
|
|
specify a general substrate for this functionality early. This SRFI defines a
|
|
general library that provides this functionality.
|
|
|
|
It is accompanied by a reference implementation for the spec. The reference
|
|
implementation is fairly efficient, straightforwardly portable, and has a
|
|
"free software" copyright. The implementation is tuned for "small" 7 or 8
|
|
bit character types, such as ASCII or Latin-1; the data structures and
|
|
algorithms would have to be altered for larger 16 or 32 bit character types
|
|
such as Unicode -- however, the specs have been carefully designed with these
|
|
larger character types in mind.
|
|
|
|
Several forthcoming SRFIs can be defined in terms of this one:
|
|
<ul>
|
|
<li> string library
|
|
<li> delimited input procedures (<em>e.g.</em>, <code>read-line</code>)
|
|
<li> regular expressions
|
|
</ul>
|
|
|
|
|
|
<!--========================================================================-->
|
|
<h1><a name="VariableIndex">Variable Index</a></h1>
|
|
<p>
|
|
Here is the complete set of bindings -- procedural and otherwise --
|
|
exported by this library. In a Scheme system that has a module or package
|
|
system, these procedures should be contained in a module named "char-set-lib".
|
|
|
|
<div class=indent>
|
|
<dl>
|
|
<dt class=proc-index> Predicates & comparison
|
|
<dd class=proc-index>
|
|
<pre class=proc-index>
|
|
<a href="#char-set-p">char-set?</a> <a href="#char-set=">char-set=</a> <a href="#char-set<=">char-set<=</a> <a href="#char-set-hash">char-set-hash</a>
|
|
</pre>
|
|
|
|
<dt class=proc-index> Iterating over character sets
|
|
<dd class=proc-index>
|
|
<pre class=proc-index>
|
|
<a href="#char-set-cursor">char-set-cursor</a> <a href="#char-set-ref">char-set-ref</a> <a href="#char-set-cursor-next">char-set-cursor-next</a> <a href="#end-of-char-set-p">end-of-char-set?</a>
|
|
<a href="#char-set-fold">char-set-fold</a> <a href="#char-set-unfold">char-set-unfold</a> <a href="#char-set-unfold!">char-set-unfold!</a>
|
|
<a href="#char-set-for-each">char-set-for-each</a> <a href="#char-set-map">char-set-map</a>
|
|
</pre>
|
|
|
|
<dt class=proc-index> Creating character sets
|
|
<dd class=proc-index>
|
|
<pre class=proc-index>
|
|
<a href="#char-set-copy">char-set-copy</a> <a href="#char-set">char-set</a>
|
|
|
|
<a href="#list->char-set">list->char-set</a> <a href="#string->char-set">string->char-set</a>
|
|
<a href="#list->char-set!">list->char-set!</a> <a href="#string->char-set!">string->char-set!</a>
|
|
|
|
<a href="#char-set-filter">char-set-filter</a> <a href="#ucs-range->char-set">ucs-range->char-set</a> <a href="#
|
|
char-set-filter!">
|
|
char-set-filter!</a> <a href="#ucs-range->char-set!">ucs-range->char-set!</a>
|
|
|
|
<a href="#->char-set">->char-set</a>
|
|
</pre>
|
|
|
|
<dt class=proc-index> Querying character sets
|
|
<dd class=proc-index>
|
|
<pre class=proc-index>
|
|
<a href="#char-set->list">char-set->list</a> <a href="#char-set->string">char-set->string</a>
|
|
<a href="#char-set-size">char-set-size</a> <a href="#char-set-count">char-set-count</a> <a href="#char-set-contains-p">char-set-contains?</a>
|
|
<a href="#char-set-every">char-set-every</a> <a href="#char-set-any">char-set-any</a>
|
|
</pre>
|
|
|
|
<dt class=proc-index> Character-set algebra
|
|
<dd class=proc-index>
|
|
<pre class=proc-index>
|
|
<a href="#char-set-adjoin">char-set-adjoin</a> <a href="#char-set-delete">char-set-delete</a>
|
|
<a href="#char-set-adjoin!">char-set-adjoin!</a> <a href="#char-set-delete!">char-set-delete!</a>
|
|
|
|
<a href="#char-set-complement">char-set-complement</a> <a href="#char-set-union">char-set-union</a> <a href="#char-set-intersection">char-set-intersection</a>
|
|
<a href="#char-set-complement!">char-set-complement!</a> <a href="#char-set-union!">char-set-union!</a> <a href="#char-set-intersection!">char-set-intersection!</a>
|
|
|
|
<a href="#char-set-difference">char-set-difference</a> <a href="#char-set-xor">char-set-xor</a> <a href="#char-set-diff+intersection">char-set-diff+intersection</a>
|
|
<a href="#char-set-difference!">char-set-difference!</a> <a href="#char-set-xor!">char-set-xor!</a> <a href="#char-set-diff+intersection!">char-set-diff+intersection!</a>
|
|
</pre>
|
|
|
|
<dt class=proc-index> Standard character sets
|
|
<dd class=proc-index>
|
|
<pre class=proc-index>
|
|
<a href="#char-set:lower-case">char-set:lower-case</a> <a href="#char-set:upper-case">char-set:upper-case</a> <a href="#char-set:title-case">char-set:title-case</a>
|
|
<a href="#char-set:letter">char-set:letter</a> <a href="#char-set:digit">char-set:digit</a> <a href="#char-set:letter+digit">char-set:letter+digit</a>
|
|
<a href="#char-set:graphic">char-set:graphic</a> <a href="#char-set:printing">char-set:printing</a> <a href="#char-set:whitespace">char-set:whitespace</a>
|
|
<a href="#char-set:iso-control">char-set:iso-control</a> <a href="#char-set:punctuation">char-set:punctuation</a> <a href="#char-set:symbol">char-set:symbol</a>
|
|
<a href="#char-set:hex-digit">char-set:hex-digit</a> <a href="#char-set:blank">char-set:blank</a> <a href="#char-set:ascii">char-set:ascii</a>
|
|
<a href="#char-set:empty">char-set:empty</a> <a href="#char-set:full">char-set:full</a>
|
|
</pre>
|
|
|
|
</dl>
|
|
</div>
|
|
|
|
<!--========================================================================-->
|
|
<h1><a name="Rationale">Rationale</a></h1>
|
|
|
|
<p>
|
|
The ability to efficiently manipulate sets of characters is quite
|
|
useful for text-processing code. Encapsulating this functionality in
|
|
a general, efficiently implemented library can assist all such code.
|
|
This library defines a new data structure to represent these sets, called
|
|
a "char-set." The char-set type is distinct from all other types.
|
|
|
|
<p>
|
|
This library is designed to be portable across implementations that use
|
|
different character types and representations, especially ASCII, Latin-1
|
|
and Unicode. Some effort has been made to preserve compatibility with Java
|
|
in the Unicode case (see the definition of <code>char-set:whitespace</code> for the
|
|
single real deviation).
|
|
|
|
<!--========================================================================-->
|
|
<h2><a name="LinearUpdateOperations">Linear-update operations</a></h2>
|
|
|
|
<p>
|
|
The procedures of this SRFI, by default, are "pure functional" -- they do not
|
|
alter their parameters. However, this SRFI defines a set of "linear-update"
|
|
procedures which have a hybrid pure-functional/side-effecting semantics: they
|
|
are allowed, but not required, to side-effect one of their parameters in order
|
|
to construct their result. An implementation may legally implement these
|
|
procedures as pure, side-effect-free functions, or it may implement them using
|
|
side effects, depending upon the details of what is the most efficient or
|
|
simple to implement in terms of the underlying representation.
|
|
|
|
<p>
|
|
The linear-update routines all have names ending with "!".
|
|
|
|
<p>
|
|
Clients of these procedures <em>may not</em> rely upon these procedures working by
|
|
side effect. For example, this is not guaranteed to work:
|
|
<pre class=code-example>
|
|
(let* ((cs1 (char-set #\a #\b #\c)) ; cs1 = {a,b,c}.
|
|
(cs2 (char-set-adjoin! cs1 #\d))) ; Add d to {a,b,c}.
|
|
cs1) ; Could be either {a,b,c} or {a,b,c,d}.
|
|
</pre>
|
|
<p class=continue>
|
|
However, this is well-defined:
|
|
<pre class=code-example>
|
|
(let ((cs (char-set #\a #\b #\c)))
|
|
(char-set-adjoin! cs #\d)) ; Add d to {a,b,c}.
|
|
</pre>
|
|
|
|
<p>
|
|
So clients of these procedures write in a functional style, but must
|
|
additionally be sure that, when the procedure is called, there are no other
|
|
live pointers to the potentially-modified character set (hence the term
|
|
"linear update").
|
|
|
|
<p>
|
|
There are two benefits to this convention:
|
|
<ul>
|
|
<li> Implementations are free to provide the most efficient possible
|
|
implementation, either functional or side-effecting.
|
|
<li> Programmers may nonetheless continue to assume that character sets
|
|
are purely functional data structures: they may be reliably shared
|
|
without needing to be copied, uniquified, and so forth.
|
|
</ul>
|
|
|
|
<p>
|
|
Note that pure functional representations are the right thing for
|
|
ASCII- or Latin-1-based Scheme implementations, since a char-set can
|
|
be represented in an ASCII Scheme with 4 32-bit words. Pure set-algebra
|
|
operations on such a representation are very fast and efficient. Programmers
|
|
who code using linear-update operations are guaranteed the system will
|
|
provide the best implementation across multiple platforms.
|
|
|
|
<p>
|
|
In practice, these procedures are most useful for efficiently constructing
|
|
character sets in a side-effecting manner, in some limited local context,
|
|
before passing the character set outside the local construction scope to be
|
|
used in a functional manner.
|
|
|
|
<p>
|
|
Scheme provides no assistance in checking the linearity of the potentially
|
|
side-effected parameters passed to these functions --- there's no linear
|
|
type checker or run-time mechanism for detecting violations. (But
|
|
sophisticated programming environments, such as DrScheme, might help.)
|
|
|
|
<!--========================================================================-->
|
|
<h2><a name="ExtraSRFI">Extra-SRFI recommendations</a></h2>
|
|
<p>
|
|
Users are cautioned that the R5RS predicates
|
|
<div class=inset><code>
|
|
char-alphabetic? <br>
|
|
char-numeric? <br>
|
|
char-whitespace? <br>
|
|
char-upper-case? <br>
|
|
char-lower-case? <br>
|
|
</code>
|
|
</div>
|
|
<p class=continue>
|
|
may or may not be in agreement with the SRFI 14 base character sets
|
|
<div class=inset>
|
|
<code>
|
|
char-set:letter<br>
|
|
char-set:digit<br>
|
|
char-set:whitespace<br>
|
|
char-set:upper-case<br>
|
|
char-set:lower-case<br>
|
|
</code>
|
|
</div>
|
|
<p class=continue>
|
|
Implementors are strongly encouraged to bring these predicates into
|
|
agreement with the base character sets of this SRFI; not to do so risks
|
|
major confusion.
|
|
|
|
|
|
<!--========================================================================-->
|
|
<h1><a name="Specification">Specification</a></h1>
|
|
<p>
|
|
In the following procedure specifications:
|
|
<ul>
|
|
<li> A <var>cs</var> parameter is a character set.
|
|
|
|
<li> An <var>s</var> parameter is a string.
|
|
|
|
<li> A <var>char</var> parameter is a character.
|
|
|
|
<li> A <var>char-list</var> parameter is a list of characters.
|
|
|
|
<li> A <var>pred</var> parameter is a unary character predicate procedure, returning
|
|
a true/false value when applied to a character.
|
|
|
|
<li> An <var>obj</var> parameter may be any value at all.
|
|
</ul>
|
|
|
|
<p>
|
|
Passing values to procedures with these parameters that do not satisfy these
|
|
types is an error.
|
|
|
|
<p>
|
|
Unless otherwise noted in the specification of a procedure, procedures
|
|
always return character sets that are distinct (from the point of view
|
|
of the linear-update operations) from the parameter character sets. For
|
|
example, <code>char-set-adjoin</code> is guaranteed to provide a fresh character set,
|
|
even if it is not given any character parameters.
|
|
|
|
<p>
|
|
Parameters given in square brackets are optional. Unless otherwise noted in the
|
|
text describing the procedure, any prefix of these optional parameters may
|
|
be supplied, from zero arguments to the full list. When a procedure returns
|
|
multiple values, this is shown by listing the return values in square
|
|
brackets, as well. So, for example, the procedure with signature
|
|
<pre class=code-example>
|
|
halts? <var>f [x init-store]</var> -> <var>[boolean integer]</var>
|
|
</pre>
|
|
would take one (<var>f</var>), two (<var>f</var>, <var>x</var>)
|
|
or three (<var>f</var>, <var>x</var>, <var>init-store</var>) input parameters,
|
|
and return two values, a boolean and an integer.
|
|
|
|
<p>
|
|
A parameter followed by "<code>...</code>" means zero-or-more elements.
|
|
So the procedure with the signature
|
|
<pre class=code-example>
|
|
sum-squares <var>x ... </var> -> <var>number</var>
|
|
</pre>
|
|
takes zero or more arguments (<var>x ...</var>),
|
|
while the procedure with signature
|
|
<pre class=code-example>
|
|
spell-check <var>doc dict<sub>1</sub> dict<sub>2</sub> ...</var> -> <var>string-list</var>
|
|
</pre>
|
|
takes two required parameters
|
|
(<var>doc</var> and <var>dict<sub>1</sub></var>)
|
|
and zero or more optional parameters (<var>dict<sub>2</sub> ...</var>).
|
|
|
|
|
|
<!--========================================================================-->
|
|
<h2><a name="GeneralProcs">General procedures</a></h2>
|
|
<dl>
|
|
|
|
<!--
|
|
==== char-set?
|
|
============================================================================-->
|
|
<dt class=proc-def>
|
|
<a name="char-set-p"></a>
|
|
<code class=proc-def>char-set?</code><var> obj -> boolean</var>
|
|
<dd class=proc-def>
|
|
|
|
Is the object <var>obj</var> a character set?
|
|
|
|
<!--
|
|
==== char-set=
|
|
============================================================================-->
|
|
<dt class=proc-def>
|
|
<a name="char-set="></a>
|
|
<code class=proc-def>char-set=</code><var> cs<sub>1</sub> ... -> boolean</var>
|
|
<dd class=proc-def>
|
|
Are the character sets equal?
|
|
<p>
|
|
Boundary cases:
|
|
<pre class=code-example>
|
|
(char-set=) => <var>true</var>
|
|
(char-set= cs) => <var>true</var>
|
|
</pre>
|
|
|
|
<p>
|
|
Rationale: transitive binary relations are generally extended to n-ary
|
|
relations in Scheme, which enables clearer, more concise code to be
|
|
written. While the zero-argument and one-argument cases will almost
|
|
certainly not arise in first-order uses of such relations, they may well
|
|
arise in higher-order cases or macro-generated code.
|
|
<em>E.g.,</em> consider
|
|
<pre class=code-example>
|
|
(apply char-set= cset-list)
|
|
</pre>
|
|
<p class=continue>
|
|
This is well-defined if the list is empty or a singleton list. Hence
|
|
we extend these relations to any number of arguments. Implementors
|
|
have reported actual uses of n-ary relations in higher-order cases
|
|
allowing for fewer than two arguments. The way of Scheme is to handle the
|
|
general case; we provide the fully general extension.
|
|
<p>
|
|
A counter-argument to this extension is that
|
|
<abbr title="Revised^5 Report on Scheme"><a href="#R5RS">R5RS</a></abbr>'s
|
|
transitive binary arithmetic relations
|
|
(<code>=</code>, <code><</code>, <em>etc.</em>)
|
|
require at least two arguments, hence
|
|
this decision is a break with the prior convention -- although it is
|
|
at least one that is backwards-compatible.
|
|
|
|
<!--
|
|
==== char-set<=
|
|
============================================================================-->
|
|
<dt class=proc-def>
|
|
<a name="char-set<="></a>
|
|
<code class=proc-def>char-set<=</code><var> cs<sub>1</sub> ... -> boolean</var>
|
|
<dd class=proc-def>
|
|
Returns true if every character set <var>cs<sub>i</sub></var> is
|
|
a subset of character set <var>cs<sub>i+1</sub></var>.
|
|
|
|
<p>
|
|
Boundary cases:
|
|
<pre class=code-example>
|
|
(char-set<=) => <var>true</var>
|
|
(char-set<= cs) => <var>true</var>
|
|
</pre>
|
|
<p>
|
|
Rationale: See <code>char-set=</code> for discussion of zero- and one-argument
|
|
applications. Consider testing a list of char-sets for monotonicity
|
|
with
|
|
<pre class=code-example>
|
|
(apply char-set<= cset-list)
|
|
</pre>
|
|
|
|
<!--
|
|
==== char-set-hash
|
|
============================================================================-->
|
|
<dt class=proc-def>
|
|
<a name="char-set-hash"></a>
|
|
<code class=proc-def>char-set-hash</code><var> cs [bound] -> integer</var>
|
|
<dd class=proc-def>
|
|
Compute a hash value for the character set <var>cs</var>.
|
|
<var>Bound</var> is a non-negative
|
|
exact integer specifying the range of the hash function. A positive
|
|
value restricts the return value to the range [0,<var>bound</var>).
|
|
|
|
<p>
|
|
If <var>bound</var> is either zero or not given, the implementation may use
|
|
an implementation-specific default value, chosen to be as large as
|
|
is efficiently practical. For instance, the default range might be chosen
|
|
for a given implementation to map all strings into the range of
|
|
integers that can be represented with a single machine word.
|
|
|
|
|
|
<p>
|
|
Invariant:
|
|
<pre class=code-example>
|
|
(char-set= cs<sub>1</sub> cs<sub>2</sub>) => (= (char-set-hash cs<sub>1</sub> b) (char-set-hash cs<sub>2</sub> b))
|
|
</pre>
|
|
|
|
<p>
|
|
A legal but nonetheless discouraged implementation:
|
|
<pre class=code-example>
|
|
(define (char-set-hash cs . maybe-bound) 1)
|
|
</pre>
|
|
|
|
<p>
|
|
Rationale: allowing the user to specify an explicit bound simplifies user
|
|
code by removing the mod operation that typically accompanies every hash
|
|
computation, and also may allow the implementation of the hash function to
|
|
exploit a reduced range to efficiently compute the hash value.
|
|
<em>E.g.</em>, for
|
|
small bounds, the hash function may be computed in a fashion such that
|
|
intermediate values never overflow into bignum integers, allowing the
|
|
implementor to provide a fixnum-specific "fast path" for computing the
|
|
common cases very rapidly.
|
|
|
|
</dl>
|
|
|
|
<!--========================================================================-->
|
|
<h2><a name="Iterating">Iterating over character sets</a></h2>
|
|
|
|
<dl>
|
|
<!--
|
|
==== char-set-cursor char-set-ref char-set-cursor-next end-of-char-set?
|
|
============================================================================-->
|
|
<dt class=proc-def1>
|
|
<a name="char-set-cursor"></a>
|
|
<a name="char-set-ref"></a>
|
|
<a name="char-set-cursor-next"></a>
|
|
<a name="end-of-char-set-p"></a>
|
|
<code class=proc-def>char-set-cursor</code><var> cset -> cursor</var>
|
|
<dt class=proc-defi>
|
|
<code class=proc-def>char-set-ref</code><var> cset cursor -> char</var>
|
|
<dt class=proc-defi>
|
|
<code class=proc-def>char-set-cursor-next</code><var> cset cursor -> cursor</var>
|
|
<dt class=proc-defn>
|
|
<code class=proc-def>end-of-char-set?</code><var> cursor -> boolean</var>
|
|
<dd class=proc-def>
|
|
Cursors are a low-level facility for iterating over the characters in a
|
|
set. A cursor is a value that indexes a character in a char set.
|
|
<code>char-set-cursor</code> produces a new cursor for a given char set.
|
|
The set element indexed by the cursor is fetched with
|
|
<code>char-set-ref</code>.
|
|
A cursor index is incremented with <code>char-set-cursor-next</code>;
|
|
in this way, code can step through every character in a char set.
|
|
Stepping a cursor "past the end" of a char set produces a cursor that
|
|
answers true to <code>end-of-char-set?</code>.
|
|
It is an error to pass such a cursor to <code>char-set-ref</code> or to
|
|
<code>char-set-cursor-next</code>.
|
|
|
|
<p>
|
|
A cursor value may not be used in conjunction with a different character
|
|
set; if it is passed to <code>char-set-ref</code> or
|
|
<code>char-set-cursor-next</code> with
|
|
a character set other than the one used to create it, the results and
|
|
effects are undefined.
|
|
|
|
<p>
|
|
Cursor values are <em>not</em> necessarily distinct from other types.
|
|
They may be
|
|
integers, linked lists, records, procedures or other values. This license
|
|
is granted to allow cursors to be very "lightweight" values suitable for
|
|
tight iteration, even in fairly simple implementations.
|
|
|
|
<p>
|
|
Note that these primitives are necessary to export an iteration facility
|
|
for char sets to loop macros.
|
|
|
|
<p>
|
|
Example:
|
|
<pre class=code-example>
|
|
(define cs (char-set #\G #\a #\T #\e #\c #\h))
|
|
|
|
;; Collect elts of CS into a list.
|
|
(let lp ((cur (char-set-cursor cs)) (ans '()))
|
|
(if (end-of-char-set? cur) ans
|
|
(lp (char-set-cursor-next cs cur)
|
|
(cons (char-set-ref cs cur) ans))))
|
|
=> (#\G #\T #\a #\c #\e #\h)
|
|
|
|
;; Equivalently, using a list unfold (from SRFI 1):
|
|
(unfold-right end-of-char-set?
|
|
(curry char-set-ref cs)
|
|
(curry char-set-cursor-next cs)
|
|
(char-set-cursor cs))
|
|
=> (#\G #\T #\a #\c #\e #\h)
|
|
</pre>
|
|
|
|
<p>
|
|
Rationale: Note that the cursor API's four functions "fit" the functional
|
|
protocol used by the unfolders provided by the list, string and char-set
|
|
SRFIs (see the example above). By way of contrast, here is a simpler,
|
|
two-function API that was rejected for failing this criterion. Besides
|
|
<code>char-set-cursor</code>, it provided a single
|
|
function that mapped a cursor and a character set to two values, the
|
|
indexed character and the next cursor. If the cursor had exhausted the
|
|
character set, then this function returned false instead of the character
|
|
value, and another end-of-char-set cursor. In this way, the other three
|
|
functions of the current API were combined together.
|
|
|
|
<!--
|
|
==== char-set-fold
|
|
============================================================================-->
|
|
<dt class=proc-def>
|
|
<a name="char-set-fold"></a>
|
|
<code class=proc-def>char-set-fold</code><var> kons knil cs -> object</var>
|
|
<dd class=proc-def>
|
|
This is the fundamental iterator for character sets. Applies the function
|
|
<var>kons</var> across the character set <var>cs</var> using initial state value <var>knil</var>. That is,
|
|
if <var>cs</var> is the empty set, the procedure returns <var>knil</var>. Otherwise, some
|
|
element <var>c</var> of <var>cs</var> is chosen;
|
|
let <var>cs'</var> be the remaining, unchosen characters.
|
|
The procedure returns
|
|
<pre class=code-example>
|
|
(char-set-fold <var>kons</var> (<var>kons</var> <var>c</var> <var>knil</var>) <var>cs'</var>)
|
|
</pre>
|
|
<p>
|
|
Examples:
|
|
<pre class=code-example>
|
|
;; CHAR-SET-MEMBERS
|
|
(lambda (cs) (char-set-fold cons '() cs))
|
|
|
|
;; CHAR-SET-SIZE
|
|
(lambda (cs) (char-set-fold (lambda (c i) (+ i 1)) 0 cs))
|
|
|
|
;; How many vowels in the char set?
|
|
(lambda (cs)
|
|
(char-set-fold (lambda (c i) (if (vowel? c) (+ i 1) i))
|
|
0 cs))
|
|
</pre>
|
|
|
|
<!--
|
|
==== char-set-unfold char-set-unfold!
|
|
============================================================================-->
|
|
<dt class=proc-def1>
|
|
<a name="char-set-unfold"></a>
|
|
<a name="char-set-unfold!"></a>
|
|
<code class=proc-def>char-set-unfold </code><var> f p g seed [base-cs] -> char-set</var>
|
|
<dt class=proc-defn><code class=proc-def>char-set-unfold!</code><var> f p g seed base-cs -> char-set</var>
|
|
<dd class=proc-def>
|
|
This is a fundamental constructor for char-sets.
|
|
<ul>
|
|
<li> <var>G</var> is used to generate a series of "seed" values from the initial seed:
|
|
<var>seed</var>, (<var>g</var> <var>seed</var>), (<var>g<sup>2</sup></var> <var>seed</var>), (<var>g<sup>3</sup></var> <var>seed</var>), ...
|
|
<li> <var>P</var> tells us when to stop -- when it returns true when applied to one
|
|
of these seed values.
|
|
<li> <var>F</var> maps each seed value to a character. These characters are added
|
|
to the base character set <var>base-cs</var> to form the result; <var>base-cs</var> defaults to
|
|
the empty set. <code>char-set-unfold!</code> adds the characters to <var>base-cs</var> in a
|
|
linear-update -- it is allowed, but not required, to side-effect
|
|
and use <var>base-cs</var>'s storage to construct the result.
|
|
</ul>
|
|
|
|
<p>
|
|
More precisely, the following definitions hold, ignoring the
|
|
optional-argument issues:
|
|
|
|
<pre class=code-example>
|
|
(define (char-set-unfold p f g seed base-cs)
|
|
(char-set-unfold! p f g seed (char-set-copy base-cs)))
|
|
|
|
(define (char-set-unfold! p f g seed base-cs)
|
|
(let lp ((seed seed) (cs base-cs))
|
|
(if (p seed) cs ; P says we are done.
|
|
(lp (g seed) ; Loop on (G SEED).
|
|
(char-set-adjoin! cs (f seed)))))) ; Add (F SEED) to set.
|
|
</pre>
|
|
|
|
(Note that the actual implementation may be more efficient.)
|
|
|
|
<p>
|
|
Examples:
|
|
<pre class=code-example>
|
|
(port->char-set p) = (char-set-unfold eof-object? values
|
|
(lambda (x) (read-char p))
|
|
(read-char p))
|
|
|
|
(list->char-set lis) = (char-set-unfold null? car cdr lis)
|
|
</pre>
|
|
<!--
|
|
==== char-set-for-each
|
|
============================================================================-->
|
|
<dt class=proc-def>
|
|
<a name="char-set-for-each"></a>
|
|
<code class=proc-def>char-set-for-each</code><var> proc cs -> unspecified</var>
|
|
<dd class=proc-def>
|
|
Apply procedure <var>proc</var> to each character in the character set <var>cs</var>.
|
|
Note that the order in which <var>proc</var> is applied to the characters in the
|
|
set is not specified, and may even change from one procedure application
|
|
to another.
|
|
|
|
<p>
|
|
Nothing at all is specified about the value returned by this procedure; it
|
|
is not even required to be consistent from call to call. It is simply
|
|
required to be a value (or values) that may be passed to a command
|
|
continuation, <em>e.g.</em> as the value of an expression appearing as a
|
|
non-terminal subform of a <code>begin</code> expression.
|
|
Note that in
|
|
<abbr title="Revised^5 Report on Scheme"><a href="#R5RS">R5RS</a></abbr>,
|
|
this restricts the procedure to returning a single value;
|
|
non-R5RS systems may not even provide this restriction.
|
|
|
|
<!--
|
|
==== char-set-map
|
|
============================================================================-->
|
|
<dt class=proc-def>
|
|
<a name="char-set-map"></a>
|
|
<code class=proc-def>char-set-map</code><var> proc cs -> char-set</var>
|
|
<dd class=proc-def>
|
|
<var>proc</var> is a char->char procedure. Apply it to all the characters in
|
|
the char-set <var>cs</var>, and collect the results into a new character set.
|
|
|
|
<p>
|
|
Essentially lifts <var>proc</var> from a char->char procedure to a char-set ->
|
|
char-set procedure.
|
|
|
|
<p>
|
|
Example:
|
|
<pre class=code-example>
|
|
(char-set-map char-downcase cset)
|
|
</pre>
|
|
</dl>
|
|
|
|
|
|
<!--========================================================================-->
|
|
<h2><a name="Creating">Creating character sets</a></h2>
|
|
<dl>
|
|
|
|
<!--
|
|
==== char-set-copy
|
|
============================================================================-->
|
|
<dt class=proc-def>
|
|
<a name="char-set-copy"></a>
|
|
<code class=proc-def>char-set-copy</code><var> cs -> char-set</var>
|
|
<dd class=proc-def>
|
|
Returns a copy of the character set <var>cs</var>. "Copy" means that if either the
|
|
input parameter or the result value of this procedure is passed to one of
|
|
the linear-update procedures described below, the other character set is
|
|
guaranteed not to be altered.
|
|
|
|
<p>
|
|
A system that provides pure-functional implementations of the
|
|
linear-operator suite could implement this procedure as the identity
|
|
function -- so copies are <em>not</em> guaranteed to be distinct by <code>eq?</code>.
|
|
|
|
<!--
|
|
==== char-set
|
|
============================================================================-->
|
|
<dt class=proc-def>
|
|
<a name="char-set"></a>
|
|
<code class=proc-def>char-set</code><var> char<sub>1</sub> ... -> char-set</var>
|
|
<dd class=proc-def>
|
|
Return a character set containing the given characters.
|
|
|
|
<!--
|
|
==== list->char-set list->char-set
|
|
============================================================================-->
|
|
<dt class=proc-def1>
|
|
<a name="list->char-set"></a>
|
|
<a name="list->char-set!"></a>
|
|
<code class=proc-def>list->char-set </code><var> char-list [base-cs] -> char-set</var>
|
|
<dt class=proc-defn><code class=proc-def>list->char-set!</code><var> char-list base-cs -> char-set</var>
|
|
<dd class=proc-def>
|
|
Return a character set containing the characters in the list of
|
|
characters <var>char-list</var>.
|
|
|
|
<p>
|
|
If character set <var>base-cs</var> is provided, the characters from <var>char-list</var>
|
|
are added to it. <code>list->char-set!</code> is allowed, but not required,
|
|
to side-effect and reuse the storage in <var>base-cs</var>;
|
|
<code>list->char-set</code> produces a fresh character set.
|
|
|
|
<!--
|
|
==== string->char-set string->char-set!
|
|
============================================================================-->
|
|
<dt class=proc-def1>
|
|
<a name="string->char-set"></a>
|
|
<a name="string->char-set!"></a>
|
|
<code class=proc-def>string->char-set </code><var> s [base-cs] -> char-set</var>
|
|
<dt class=proc-defn><code class=proc-def>string->char-set!</code><var> s base-cs -> char-set</var>
|
|
<dd class=proc-def>
|
|
|
|
Return a character set containing the characters in the string <var>s</var>.
|
|
|
|
<p>
|
|
If character set <var>base-cs</var> is provided, the characters from <var>s</var> are added to
|
|
it. <code>string->char-set!</code> is allowed, but not required, to side-effect and
|
|
reuse the storage in <var>base-cs</var>; <code>string->char-set</code> produces a fresh character
|
|
set.
|
|
|
|
<!--
|
|
==== char-set-filter char-set-filter!
|
|
============================================================================-->
|
|
<dt class=proc-def1>
|
|
<a name="char-set-filter"></a>
|
|
<a name="char-set-filter!"></a>
|
|
<code class=proc-def>char-set-filter </code><var> pred cs [base-cs] -> char-set</var>
|
|
<dt class=proc-defn><code class=proc-def>char-set-filter!</code><var> pred cs base-cs -> char-set</var>
|
|
<dd class=proc-def>
|
|
|
|
Returns a character set containing every character <var>c</var>
|
|
in <var>cs</var> such that <code>(<var>pred</var> <var>c</var>)</code>
|
|
returns true.
|
|
|
|
<p>
|
|
If character set <var>base-cs</var> is provided, the characters specified
|
|
by <var>pred</var> are added to it.
|
|
<code>char-set-filter!</code> is allowed, but not required,
|
|
to side-effect and reuse the storage in <var>base-cs</var>;
|
|
<code>char-set-filter</code> produces a fresh character set.
|
|
|
|
<p>
|
|
An implementation may not save away a reference to <var>pred</var> and
|
|
invoke it after <code>char-set-filter</code> or
|
|
<code>char-set-filter!</code> returns -- that is, "lazy,"
|
|
on-demand implementations are not allowed, as <var>pred</var> may have
|
|
external dependencies on mutable data or have other side-effects.
|
|
|
|
<p>
|
|
Rationale: This procedure provides a means of converting a character
|
|
predicate into its equivalent character set; the <var>cs</var> parameter
|
|
allows the programmer to bound the predicate's domain. Programmers should
|
|
be aware that filtering a character set such as <code>char-set:full</code>
|
|
could be a very expensive operation in an implementation that provided an
|
|
extremely large character type, such as 32-bit Unicode. An earlier draft
|
|
of this library provided a simple <code>predicate->char-set</code>
|
|
procedure, which was rejected in favor of <code>char-set-filter</code> for
|
|
this reason.
|
|
|
|
|
|
<!--
|
|
==== ucs-range->char-set ucs-range->char-set!
|
|
============================================================================-->
|
|
<dt class=proc-def1>
|
|
<a name="ucs-range->char-set"></a>
|
|
<a name="ucs-range->char-set!"></a>
|
|
<code class=proc-def>ucs-range->char-set </code><var> lower upper [error? base-cs] -> char-set</var>
|
|
<dt class=proc-defn><code class=proc-def>ucs-range->char-set!</code><var> lower upper error? base-cs -> char-set</var>
|
|
<dd class=proc-def>
|
|
<var>Lower</var> and <var>upper</var> are exact non-negative integers;
|
|
<var>lower</var> <= <var>upper</var>.
|
|
|
|
<p>
|
|
Returns a character set containing every character whose ISO/IEC 10646
|
|
UCS-4 code lies in the half-open range [<var>lower</var>,<var>upper</var>).
|
|
|
|
<ul>
|
|
<li> If the requested range includes unassigned UCS values, these are
|
|
silently ignored (the current UCS specification has "holes" in the
|
|
space of assigned codes).
|
|
|
|
<li> If the requested range includes "private" or "user space" codes, these
|
|
are handled in an implementation-specific manner; however, a UCS- or
|
|
Unicode-based Scheme implementation should pass them through
|
|
transparently.
|
|
|
|
<li> If any code from the requested range specifies a valid, assigned
|
|
UCS character that has no corresponding representative in the
|
|
implementation's character type, then (1) an error is raised if <var>error?</var>
|
|
is true, and (2) the code is ignored if <var>error?</var> is false (the default).
|
|
This might happen, for example, if the implementation uses ASCII
|
|
characters, and the requested range includes non-ASCII characters.
|
|
</ul>
|
|
|
|
<p>
|
|
If character set <var>base-cs</var> is provided, the characters specified by the
|
|
range are added to it. <code>ucs-range->char-set!</code> is allowed, but not required,
|
|
to side-effect and reuse the storage in <var>base-cs</var>;
|
|
<code>ucs-range->char-set</code> produces a fresh character set.
|
|
|
|
<p>
|
|
Note that ASCII codes are a subset of the Latin-1 codes, which are in turn
|
|
a subset of the 16-bit Unicode codes, which are themselves a subset of the
|
|
32-bit UCS-4 codes. We commit to a specific encoding in this routine,
|
|
regardless of the underlying representation of characters, so that client
|
|
code using this library will be portable. <em>I.e.</em>, a conformant Scheme
|
|
implementation may use EBCDIC or SHIFT-JIS to encode characters; it must
|
|
simply map the UCS characters from the given range into the native
|
|
representation when possible, and report errors when not possible.
|
|
|
|
<!--
|
|
==== ->char-set
|
|
============================================================================-->
|
|
<dt class=proc-def>
|
|
<a name="->char-set"></a>
|
|
<code class=proc-def>->char-set</code><var> x -> char-set</var>
|
|
<dd class=proc-def>
|
|
Coerces <var>x</var> into a char-set.
|
|
<var>X</var> may be a string, character or
|
|
char-set. A string is converted to the set of its constituent characters;
|
|
a character is converted to a singleton set; a char-set is returned
|
|
as-is.
|
|
This procedure is intended for use by other procedures that want to
|
|
provide "user-friendly," wide-spectrum interfaces to their clients.
|
|
|
|
</dl>
|
|
|
|
<!--========================================================================-->
|
|
<h2><a name="Querying">Querying character sets</a></h2>
|
|
<dl>
|
|
|
|
<!--
|
|
==== char-set-size
|
|
============================================================================-->
|
|
<dt class=proc-def>
|
|
<a name="char-set-size"></a>
|
|
<code class=proc-def>char-set-size</code><var> cs -> integer</var>
|
|
<dd class=proc-def>
|
|
Returns the number of elements in character set <var>cs</var>.
|
|
|
|
<!--
|
|
==== char-set-count
|
|
============================================================================-->
|
|
<dt class=proc-def>
|
|
<a name="char-set-count"></a>
|
|
<code class=proc-def>char-set-count</code><var> pred cs -> integer</var>
|
|
<dd class=proc-def>
|
|
Apply <var>pred</var> to the chars of character set <var>cs</var>, and return the number
|
|
of chars that caused the predicate to return true.
|
|
|
|
<!--
|
|
==== char-set->list
|
|
============================================================================-->
|
|
<dt class=proc-def>
|
|
<a name="char-set->list"></a>
|
|
<code class=proc-def>char-set->list</code><var> cs -> character-list</var>
|
|
<dd class=proc-def>
|
|
This procedure returns a list of the members of character set <var>cs</var>.
|
|
The order in which <var>cs</var>'s characters appear in the list is not defined,
|
|
and may be different from one call to another.
|
|
|
|
<!--
|
|
==== char-set->string
|
|
============================================================================-->
|
|
<dt class=proc-def>
|
|
<a name="char-set->string"></a>
|
|
<code class=proc-def>char-set->string</code><var> cs -> string</var>
|
|
<dd class=proc-def>
|
|
This procedure returns a string containing the members of character set <var>cs</var>.
|
|
The order in which <var>cs</var>'s characters appear in the string is not defined,
|
|
and may be different from one call to another.
|
|
|
|
<!--
|
|
==== char-set-contains?
|
|
============================================================================-->
|
|
<dt class=proc-def>
|
|
<a name="char-set-contains-p"></a>
|
|
<code class=proc-def>char-set-contains?</code><var> cs char -> boolean</var>
|
|
<dd class=proc-def>
|
|
This procedure tests <var>char</var> for membership in character set <var>cs</var>.
|
|
|
|
<p>
|
|
The MIT Scheme character-set package called this procedure
|
|
<var>char-set-member?</var>, but the argument order isn't consistent with the name.
|
|
|
|
<!--
|
|
==== char-set-every char-set-any
|
|
============================================================================-->
|
|
<dt class=proc-def1>
|
|
<a name="char-set-every"></a>
|
|
<a name="char-set-any"></a>
|
|
<code class=proc-def>char-set-every</code><var> pred cs -> boolean</var>
|
|
<dt class=proc-defn><code class=proc-def>char-set-any </code><var> pred cs -> boolean</var>
|
|
<dd class=proc-def>
|
|
The <code>char-set-every</code> procedure returns true if predicate <var>pred</var>
|
|
returns true of every character in the character set <var>cs</var>.
|
|
Likewise, <code>char-set-any</code> applies <var>pred</var> to every character in
|
|
character set <var>cs</var>, and returns the first true value it finds.
|
|
If no character produces a true value, it returns false.
|
|
The order in which these procedures sequence through the elements of
|
|
<var>cs</var> is not specified.
|
|
|
|
<p>
|
|
Note that if you need to determine the actual character on which a
|
|
predicate returns true, use <code>char-set-any</code> and arrange for the predicate
|
|
to return the character parameter as its true value, <em>e.g.</em>
|
|
<pre class=code-example>
|
|
(char-set-any (lambda (c) (and (char-upper-case? c) c))
|
|
cs)
|
|
</pre>
|
|
</dl>
|
|
|
|
<!--========================================================================-->
|
|
<h2><a name="Algebra">Character-set algebra</a></h2>
|
|
<dl>
|
|
|
|
<!--
|
|
==== char-set-adjoin char-set-delete
|
|
============================================================================-->
|
|
<dt class=proc-def1>
|
|
<a name="char-set-adjoin"></a>
|
|
<a name="char-set-delete"></a>
|
|
<code class=proc-def>char-set-adjoin</code><var> cs char<sub>1</sub> ... -> char-set</var>
|
|
<dt class=proc-defn><code class=proc-def>char-set-delete</code><var> cs char<sub>1</sub> ... -> char-set</var>
|
|
<dd class=proc-def>
|
|
Add/delete the <var>char<sub>i</sub></var> characters to/from character set <var>cs</var>.
|
|
|
|
<!--
|
|
==== char-set-adjoin! char-set-delete!
|
|
============================================================================-->
|
|
<dt class=proc-def1>
|
|
<a name="char-set-adjoin!"></a>
|
|
<a name="char-set-delete!"></a>
|
|
<code class=proc-def>char-set-adjoin!</code><var> cs char<sub>1</sub> ... -> char-set</var>
|
|
<dt class=proc-defn><code class=proc-def>char-set-delete!</code><var> cs char<sub>1</sub> ... -> char-set</var>
|
|
<dd class=proc-def>
|
|
|
|
Linear-update variants. These procedures are allowed, but not
|
|
required, to side-effect their first parameter.
|
|
|
|
<!--
|
|
==== char-set-complement char-set-union char-set-intersection
|
|
==== char-set-difference char-set-xor char-set-diff+intersection
|
|
============================================================================-->
|
|
<dt class=proc-def1>
|
|
<a name="char-set-complement"></a>
|
|
<a name="char-set-union"></a>
|
|
<a name="char-set-intersection"></a>
|
|
<a name="char-set-difference"></a>
|
|
<a name="char-set-xor"></a>
|
|
<a name="char-set-diff+intersection"></a>
|
|
<code class=proc-def>char-set-complement</code><var> cs -> char-set</var>
|
|
<dt class=proc-defi><code class=proc-def>char-set-union</code><var> cs<sub>1</sub> ... -> char-set</var>
|
|
<dt class=proc-defi><code class=proc-def>char-set-intersection</code><var> cs<sub>1</sub> ... -> char-set</var>
|
|
<dt class=proc-defi><code class=proc-def>char-set-difference</code><var> cs<sub>1</sub> cs<sub>2</sub> ... -> char-set</var>
|
|
<dt class=proc-defi><code class=proc-def>char-set-xor</code><var> cs<sub>1</sub> ... -> char-set</var>
|
|
<dt class=proc-defn><code class=proc-def>char-set-diff+intersection</code><var> cs<sub>1</sub> cs<sub>2</sub> ... -> [char-set char-set]</var>
|
|
<dd class=proc-def>
|
|
These procedures implement set complement, union, intersection,
|
|
difference, and exclusive-or for character sets. The union, intersection
|
|
and xor operations are n-ary. The difference function is also n-ary,
|
|
associates to the left (that is, it computes the difference between
|
|
its first argument and the union of all the other arguments),
|
|
and requires at least one argument.
|
|
|
|
<p>
|
|
Boundary cases:
|
|
<pre class=code-example>
|
|
(char-set-union) => char-set:empty
|
|
(char-set-intersection) => char-set:full
|
|
(char-set-xor) => char-set:empty
|
|
(char-set-difference <var>cs</var>) => <var>cs</var>
|
|
</pre>
|
|
|
|
<p>
|
|
<code>char-set-diff+intersection</code> returns both the difference and the
|
|
intersection of the arguments -- it partitions its first parameter.
|
|
It is equivalent to
|
|
<pre class=code-example>
|
|
(values (char-set-difference <var>cs<sub>1</sub></var> <var>cs<sub>2</sub></var> ...)
|
|
(char-set-intersection <var>cs<sub>1</sub></var> (char-set-union <var>cs<sub>2</sub></var> ...)))
|
|
</pre>
|
|
but can be implemented more efficiently.
|
|
|
|
<p>
|
|
Programmers should be aware that <code>char-set-complement</code> could potentially
|
|
be a very expensive operation in Scheme implementations that provide
|
|
a very large character type, such as 32-bit Unicode. If this is a
|
|
possibility, sets can be complimented with respect to a smaller
|
|
universe using <code>char-set-difference</code>.
|
|
|
|
|
|
<!--
|
|
==== char-set-complement! char-set-union! char-set-intersection!
|
|
==== char-set-difference! char-set-xor! char-set-diff+intersection!
|
|
============================================================================-->
|
|
<dt class=proc-def1>
|
|
<a name="char-set-complement!"></a>
|
|
<a name="char-set-union!"></a>
|
|
<a name="char-set-intersection!"></a>
|
|
<a name="char-set-difference!"></a>
|
|
<a name="char-set-xor!"></a>
|
|
<a name="char-set-diff+intersection!"></a>
|
|
<code class=proc-def>char-set-complement!</code><var> cs -> char-set</var>
|
|
<dt class=proc-defi><code class=proc-def>char-set-union!</code><var> cs<sub>1</sub> cs<sub>2</sub> ... -> char-set</var>
|
|
<dt class=proc-defi><code class=proc-def>char-set-intersection!</code><var> cs<sub>1</sub> cs<sub>2</sub> ... -> char-set</var>
|
|
<dt class=proc-defi><code class=proc-def>char-set-difference!</code><var> cs<sub>1</sub> cs<sub>2</sub> ... -> char-set</var>
|
|
<dt class=proc-defi><code class=proc-def>char-set-xor!</code><var> cs<sub>1</sub> cs<sub>2</sub> ... -> char-set</var>
|
|
<dt class=proc-defn><code class=proc-def>char-set-diff+intersection!</code><var> cs<sub>1</sub> cs<sub>2</sub> cs<sub>3</sub> ... -> [char-set char-set]</var>
|
|
<dd class=proc-def>
|
|
These are linear-update variants of the set-algebra functions.
|
|
They are allowed, but not required, to side-effect their first (required)
|
|
parameter.
|
|
|
|
<p>
|
|
<code>char-set-diff+intersection!</code> is allowed to side-effect both
|
|
of its two required parameters, <var>cs<sub>1</sub></var>
|
|
and <var>cs<sub>2</sub></var>.
|
|
</dl>
|
|
|
|
<!--========================================================================-->
|
|
<h2><a name="StandardCharsets">Standard character sets</a></h2>
|
|
<p>
|
|
Several character sets are predefined for convenience:
|
|
<a name="char-set:lower-case"></a>
|
|
<a name="char-set:lower-case"></a>
|
|
<a name="char-set:upper-case"></a>
|
|
<a name="char-set:title-case"></a>
|
|
<a name="char-set:letter"></a>
|
|
<a name="char-set:digit"></a>
|
|
<a name="char-set:letter+digit"></a>
|
|
<a name="char-set:graphic"></a>
|
|
<a name="char-set:printing"></a>
|
|
<a name="char-set:whitespace"></a>
|
|
<a name="char-set:iso-control"></a>
|
|
<a name="char-set:punctuation"></a>
|
|
<a name="char-set:symbol"></a>
|
|
<a name="char-set:hex-digit"></a>
|
|
<a name="char-set:blank"></a>
|
|
<a name="char-set:ascii"></a>
|
|
<a name="char-set:empty"></a>
|
|
<a name="char-set:full"></a>
|
|
<div class=inset>
|
|
<table cellpadding=0 cellspacing=0>
|
|
<tr><td><code>char-set:lower-case</code> </td><td>Lower-case letters</td></tr>
|
|
<tr><td><code>char-set:upper-case</code> </td><td>Upper-case letters</td></tr>
|
|
<tr><td><code>char-set:title-case</code> </td><td>Title-case letters</td></tr>
|
|
<tr><td><code>char-set:letter</code> </td><td>Letters</td></tr>
|
|
<tr><td><code>char-set:digit</code> </td><td>Digits</td></tr>
|
|
<tr><td><code>char-set:letter+digit</code> </td><td>Letters and digits</td></tr>
|
|
<tr><td><code>char-set:graphic</code> </td><td>Printing characters except spaces</td></tr>
|
|
<tr><td><code>char-set:printing</code> </td><td>Printing characters including spaces</td></tr>
|
|
<tr><td><code>char-set:whitespace</code> </td><td>Whitespace characters </td></tr>
|
|
<tr><td><code>char-set:iso-control</code> </td><td>The ISO control characters </td></tr>
|
|
<tr><td><code>char-set:punctuation</code> </td><td>Punctuation characters</td></tr>
|
|
<tr><td><code>char-set:symbol</code> </td><td>Symbol characters</td></tr>
|
|
<tr><td><code>char-set:hex-digit</code> </td><td>A hexadecimal digit: 0-9, A-F, a-f </td></tr>
|
|
<tr><td><code>char-set:blank</code> </td><td>Blank characters -- horizontal whitespace</td></tr>
|
|
<tr><td><code>char-set:ascii</code> </td><td>All characters in the ASCII set. </td></tr>
|
|
<tr><td><code>char-set:empty</code> </td><td>Empty set </td></tr>
|
|
<tr><td><code>char-set:full</code> </td><td>All characters </td></tr>
|
|
</table>
|
|
</div>
|
|
|
|
<p>
|
|
Note that there may be characters in <code>char-set:letter</code> that are neither upper or
|
|
lower case---this might occur in implementations that use a character type
|
|
richer than ASCII, such as Unicode. A "graphic character" is one that would
|
|
put ink on your page. While the exact composition of these sets may vary
|
|
depending upon the character type provided by the underlying Scheme system,
|
|
here are the definitions for some of the sets in an ASCII implementation:
|
|
<div class=inset>
|
|
<table cellpadding=0 cellspacing=0>
|
|
<tr><td><code>char-set:lower-case</code> </td><td>a-z </td></tr>
|
|
<tr><td><code>char-set:upper-case</code> </td><td>A-Z </td></tr>
|
|
<tr><td><code>char-set:letter</code> </td><td>A-Z and a-z </td></tr>
|
|
<tr><td><code>char-set:digit</code> </td><td>0123456789</td></tr>
|
|
<tr><td><code>char-set:punctuation</code> </td><td><code>!"#%&'()*,-./:;?@[\]_{}</code></td></tr>
|
|
<tr><td><code>char-set:symbol</code> </td><td><code>$+<=>^`|~</code></td></tr>
|
|
<tr><td><code>char-set:whitespace</code> </td><td>Space, newline, tab, form feed, </td></tr>
|
|
<tr><td></td><td> vertical tab, carriage return </td></tr>
|
|
<tr><td><code>char-set:blank</code> </td><td>Space and tab </td></tr>
|
|
<tr><td><code>char-set:graphic</code> </td><td>letter + digit + punctuation + symbol</td></tr>
|
|
<tr><td><code>char-set:printing</code> </td><td>graphic + whitespace</td></tr>
|
|
<tr><td><code>char-set:iso-control</code> </td><td>ASCII 0-31 and 127 </td></tr>
|
|
</table>
|
|
</div>
|
|
|
|
<p>
|
|
Note that the existence of the <code>char-set:ascii</code> set implies that the underlying
|
|
character set is required to be at least as rich as ASCII (including
|
|
ASCII's control characters).
|
|
|
|
<p>
|
|
Rationale: The name choices reflect a shift from the older "alphabetic/numeric"
|
|
terms found in
|
|
<abbr title="Revised^5 Report on Scheme"><a href="#R5RS">R5RS</a></abbr>
|
|
and Posix to newer, Unicode-influenced "letter/digit" lexemes.
|
|
|
|
<!--========================================================================-->
|
|
<h1><a name="StandardCharsetDefs">
|
|
Unicode, Latin-1 and ASCII definitions of the standard character sets</a>
|
|
</h1>
|
|
<p>
|
|
In Unicode Scheme implementations, the base character sets are compatible with
|
|
Java's Unicode specifications. For ASCII or Latin-1, we simply restrict the
|
|
Unicode set specifications to their first 128 or 256 codes, respectively.
|
|
Scheme implementations that are not based on ASCII, Latin-1 or Unicode should
|
|
attempt to preserve the sense or spirit of these definitions.
|
|
|
|
<p>
|
|
The following descriptions frequently make reference to the "Unicode character
|
|
database." This is a file, available at URL
|
|
<div class=inset>
|
|
<a href="ftp://ftp.unicode.org/Public/UNIDATA/UnicodeData.txt">
|
|
ftp://ftp.unicode.org/Public/UNIDATA/UnicodeData.txt</a>
|
|
</div>
|
|
<p class=continue>
|
|
Each line contains a description of a Unicode character. The first
|
|
semicolon-delimited field of the line gives the hex value of the character's
|
|
code; the second field gives the name of the character, and the third field
|
|
gives a two-letter category. Other fields give simple 1-1 case-mappings for
|
|
the character and other information; see
|
|
<div class=inset>
|
|
<a href="ftp://ftp.unicode.org/Public/UNIDATA/UnicodeData.html">
|
|
ftp://ftp.unicode.org/Public/UNIDATA/UnicodeData.html</a>
|
|
</div>
|
|
<p class=continue>
|
|
for further description of the file's format. Note in particular the
|
|
two-letter category specified in the the third field, which is referenced
|
|
frequently in the descriptions below.
|
|
|
|
<!--========================================================================-->
|
|
<h2><a name="lower-case-def">char-set:lower-case</a></h2>
|
|
<p>
|
|
For Unicode, we follow Java's specification: a character is lowercase if
|
|
<ul>
|
|
<li> it is not in the range [U+2000,U+2FFF], and
|
|
<li> the Unicode attribute table does not give a lowercase mapping for it, and
|
|
<li> at least one of the following is true:
|
|
<ul>
|
|
<li> the Unicode attribute table gives a mapping to uppercase
|
|
for the character, or
|
|
<li> the name for the character in the Unicode attribute table contains
|
|
the words "SMALL LETTER" or "SMALL LIGATURE".
|
|
</ul>
|
|
</ul>
|
|
|
|
<p>
|
|
The lower-case ASCII characters are
|
|
<div class=inset>
|
|
abcdefghijklmnopqrstuvwxyz
|
|
</div>
|
|
<p class=continue>
|
|
Latin-1 adds another 33 lower-case characters to the ASCII set:
|
|
<div class=inset>
|
|
<table cellpadding=0 cellspacing=0>
|
|
<tr><td>00B5</td> <td>MICRO SIGN</td></tr>
|
|
<tr><td>00DF</td> <td>LATIN SMALL LETTER SHARP S</td></tr>
|
|
<tr><td>00E0</td> <td>LATIN SMALL LETTER A WITH GRAVE</td></tr>
|
|
<tr><td>00E1</td> <td>LATIN SMALL LETTER A WITH ACUTE</td></tr>
|
|
<tr><td>00E2</td> <td>LATIN SMALL LETTER A WITH CIRCUMFLEX</td></tr>
|
|
<tr><td>00E3</td> <td>LATIN SMALL LETTER A WITH TILDE</td></tr>
|
|
<tr><td>00E4</td> <td>LATIN SMALL LETTER A WITH DIAERESIS</td></tr>
|
|
<tr><td>00E5</td> <td>LATIN SMALL LETTER A WITH RING ABOVE</td></tr>
|
|
<tr><td>00E6</td> <td>LATIN SMALL LETTER AE</td></tr>
|
|
<tr><td>00E7</td> <td>LATIN SMALL LETTER C WITH CEDILLA</td></tr>
|
|
<tr><td>00E8</td> <td>LATIN SMALL LETTER E WITH GRAVE</td></tr>
|
|
<tr><td>00E9</td> <td>LATIN SMALL LETTER E WITH ACUTE</td></tr>
|
|
<tr><td>00EA</td> <td>LATIN SMALL LETTER E WITH CIRCUMFLEX</td></tr>
|
|
<tr><td>00EB</td> <td>LATIN SMALL LETTER E WITH DIAERESIS</td></tr>
|
|
<tr><td>00EC</td> <td>LATIN SMALL LETTER I WITH GRAVE</td></tr>
|
|
<tr><td>00ED</td> <td>LATIN SMALL LETTER I WITH ACUTE</td></tr>
|
|
<tr><td>00EE</td> <td>LATIN SMALL LETTER I WITH CIRCUMFLEX</td></tr>
|
|
<tr><td>00EF</td> <td>LATIN SMALL LETTER I WITH DIAERESIS</td></tr>
|
|
<tr><td>00F0</td> <td>LATIN SMALL LETTER ETH</td></tr>
|
|
<tr><td>00F1</td> <td>LATIN SMALL LETTER N WITH TILDE</td></tr>
|
|
<tr><td>00F2</td> <td>LATIN SMALL LETTER O WITH GRAVE</td></tr>
|
|
<tr><td>00F3</td> <td>LATIN SMALL LETTER O WITH ACUTE</td></tr>
|
|
<tr><td>00F4</td> <td>LATIN SMALL LETTER O WITH CIRCUMFLEX</td></tr>
|
|
<tr><td>00F5</td> <td>LATIN SMALL LETTER O WITH TILDE</td></tr>
|
|
<tr><td>00F6</td> <td>LATIN SMALL LETTER O WITH DIAERESIS</td></tr>
|
|
<tr><td>00F8</td> <td>LATIN SMALL LETTER O WITH STROKE</td></tr>
|
|
<tr><td>00F9</td> <td>LATIN SMALL LETTER U WITH GRAVE</td></tr>
|
|
<tr><td>00FA</td> <td>LATIN SMALL LETTER U WITH ACUTE</td></tr>
|
|
<tr><td>00FB</td> <td>LATIN SMALL LETTER U WITH CIRCUMFLEX</td></tr>
|
|
<tr><td>00FC</td> <td>LATIN SMALL LETTER U WITH DIAERESIS</td></tr>
|
|
<tr><td>00FD</td> <td>LATIN SMALL LETTER Y WITH ACUTE</td></tr>
|
|
<tr><td>00FE</td> <td>LATIN SMALL LETTER THORN</td></tr>
|
|
<tr><td>00FF</td> <td>LATIN SMALL LETTER Y WITH DIAERESIS</td></tr>
|
|
</table>
|
|
</div>
|
|
<p class=continue>
|
|
Note that three of these have no corresponding Latin-1 upper-case character:
|
|
<div class=inset>
|
|
<table cellpadding=0 cellspacing=0>
|
|
<tr><td>00B5</td> <td>MICRO SIGN</td></tr>
|
|
<tr><td>00DF</td> <td>LATIN SMALL LETTER SHARP S</td></tr>
|
|
<tr><td>00FF</td> <td>LATIN SMALL LETTER Y WITH DIAERESIS</td></tr>
|
|
</table>
|
|
</div>
|
|
<p class=continue>
|
|
(The compatibility micro character uppercases to the non-Latin-1 Greek capital
|
|
mu; the German sharp s character uppercases to the pair of characters "SS,"
|
|
and the capital y-with-diaeresis is non-Latin-1.)
|
|
|
|
<p>
|
|
(Note that the Java spec for lowercase characters given at
|
|
<div class=inset>
|
|
<a href="http://java.sun.com/docs/books/jls/html/javalang.doc4.html#14345">
|
|
http://java.sun.com/docs/books/jls/html/javalang.doc4.html#14345</a>
|
|
</div>
|
|
<p class=continue>
|
|
is inconsistent. U+00B5 MICRO SIGN fulfills the requirements for a lower-case
|
|
character (as of Unicode 3.0), but is not given in the numeric list of
|
|
lower-case character codes.)
|
|
|
|
<p>
|
|
(Note that the Java spec for <code>isLowerCase()</code> given at
|
|
<div class=inset>
|
|
<a href="http://java.sun.com/products/jdk/1.2/docs/api/java/lang/Character.html#isLowerCase(char)">
|
|
http://java.sun.com/products/jdk/1.2/docs/api/java/lang/Character.html#isLowerCase(char)</a>
|
|
</div>
|
|
<p class=continue>
|
|
gives three mutually inconsistent definitions of "lower case." The first is
|
|
the definition used in this SRFI. Following text says "A character is
|
|
considered to be lowercase if and only if it is specified to be lowercase by
|
|
the Unicode 2.0 standard (category Ll in the Unicode specification data
|
|
file)." The former spec excludes U+00AA FEMININE ORDINAL INDICATOR and
|
|
U+00BA MASCULINE ORDINAL INDICATOR; the later spec includes them. Finally,
|
|
the spec enumerates a list of characters in the Latin-1 subset; this list
|
|
excludes U+00B5 MICRO SIGN, which is included in both of the previous specs.)
|
|
|
|
<!--========================================================================-->
|
|
<h2><a name="upper-case-def">char-set:upper-case</a></h2>
|
|
<p>
|
|
For Unicode, we follow Java's specification: a character is uppercase if
|
|
<ul>
|
|
<li> it is not in the range [U+2000,U+2FFF], and
|
|
<li> the Unicode attribute table does not give an uppercase mapping for it
|
|
(this excludes titlecase characters), and
|
|
<li> at least one of the following is true:
|
|
<ul>
|
|
<li> the Unicode attribute table gives a mapping to lowercase
|
|
for the character, or
|
|
<li> the name for the character in the Unicode attribute table contains
|
|
the words "CAPITAL LETTER" or "CAPITAL LIGATURE".
|
|
</ul>
|
|
</ul>
|
|
|
|
<p>
|
|
The upper-case ASCII characters are
|
|
<div class=inset>
|
|
ABCDEFGHIJKLMNOPQRSTUVWXYZ
|
|
</div>
|
|
<p class=continue>
|
|
Latin-1 adds another 30 upper-case characters to the ASCII set:
|
|
<div class=inset>
|
|
<table cellspacing=0 cellpadding=0>
|
|
<tr><td>00C0</td> <td>LATIN CAPITAL LETTER A WITH GRAVE</td></tr>
|
|
<tr><td>00C1</td> <td>LATIN CAPITAL LETTER A WITH ACUTE</td></tr>
|
|
<tr><td>00C2</td> <td>LATIN CAPITAL LETTER A WITH CIRCUMFLEX</td></tr>
|
|
<tr><td>00C3</td> <td>LATIN CAPITAL LETTER A WITH TILDE</td></tr>
|
|
<tr><td>00C4</td> <td>LATIN CAPITAL LETTER A WITH DIAERESIS</td></tr>
|
|
<tr><td>00C5</td> <td>LATIN CAPITAL LETTER A WITH RING ABOVE</td></tr>
|
|
<tr><td>00C6</td> <td>LATIN CAPITAL LETTER AE</td></tr>
|
|
<tr><td>00C7</td> <td>LATIN CAPITAL LETTER C WITH CEDILLA</td></tr>
|
|
<tr><td>00C8</td> <td>LATIN CAPITAL LETTER E WITH GRAVE</td></tr>
|
|
<tr><td>00C9</td> <td>LATIN CAPITAL LETTER E WITH ACUTE</td></tr>
|
|
<tr><td>00CA</td> <td>LATIN CAPITAL LETTER E WITH CIRCUMFLEX</td></tr>
|
|
<tr><td>00CB</td> <td>LATIN CAPITAL LETTER E WITH DIAERESIS</td></tr>
|
|
<tr><td>00CC</td> <td>LATIN CAPITAL LETTER I WITH GRAVE</td></tr>
|
|
<tr><td>00CD</td> <td>LATIN CAPITAL LETTER I WITH ACUTE</td></tr>
|
|
<tr><td>00CE</td> <td>LATIN CAPITAL LETTER I WITH CIRCUMFLEX</td></tr>
|
|
<tr><td>00CF</td> <td>LATIN CAPITAL LETTER I WITH DIAERESIS</td></tr>
|
|
<tr><td>00D0</td> <td>LATIN CAPITAL LETTER ETH</td></tr>
|
|
<tr><td>00D1</td> <td>LATIN CAPITAL LETTER N WITH TILDE</td></tr>
|
|
<tr><td>00D2</td> <td>LATIN CAPITAL LETTER O WITH GRAVE</td></tr>
|
|
<tr><td>00D3</td> <td>LATIN CAPITAL LETTER O WITH ACUTE</td></tr>
|
|
<tr><td>00D4</td> <td>LATIN CAPITAL LETTER O WITH CIRCUMFLEX</td></tr>
|
|
<tr><td>00D5</td> <td>LATIN CAPITAL LETTER O WITH TILDE</td></tr>
|
|
<tr><td>00D6</td> <td>LATIN CAPITAL LETTER O WITH DIAERESIS</td></tr>
|
|
<tr><td>00D8</td> <td>LATIN CAPITAL LETTER O WITH STROKE</td></tr>
|
|
<tr><td>00D9</td> <td>LATIN CAPITAL LETTER U WITH GRAVE</td></tr>
|
|
<tr><td>00DA</td> <td>LATIN CAPITAL LETTER U WITH ACUTE</td></tr>
|
|
<tr><td>00DB</td> <td>LATIN CAPITAL LETTER U WITH CIRCUMFLEX</td></tr>
|
|
<tr><td>00DC</td> <td>LATIN CAPITAL LETTER U WITH DIAERESIS</td></tr>
|
|
<tr><td>00DD</td> <td>LATIN CAPITAL LETTER Y WITH ACUTE</td></tr>
|
|
<tr><td>00DE</td> <td>LATIN CAPITAL LETTER THORN</td></tr>
|
|
</table>
|
|
</div>
|
|
<!--========================================================================-->
|
|
<h2><a name="title-case-def">char-set:title-case</a></h2>
|
|
<p>
|
|
In Unicode, a character is titlecase if it has the category Lt in
|
|
the character attribute database. There are very few of these characters;
|
|
here is the entire 31-character list as of Unicode 3.0:
|
|
<div class=inset>
|
|
<table cellspacing=0 cellpadding=0>
|
|
<tr><td>01C5 </td><td nowrap> LATIN CAPITAL LETTER D WITH SMALL LETTER Z WITH CARON
|
|
</td></tr>
|
|
<tr><td>01C8 </td><td nowrap> LATIN CAPITAL LETTER L WITH SMALL LETTER J
|
|
</td></tr>
|
|
<tr><td>01CB </td><td nowrap> LATIN CAPITAL LETTER N WITH SMALL LETTER J
|
|
</td></tr>
|
|
<tr><td>01F2 </td><td nowrap> LATIN CAPITAL LETTER D WITH SMALL LETTER Z
|
|
</td></tr>
|
|
<tr><td>1F88 </td><td nowrap> GREEK CAPITAL LETTER ALPHA WITH PSILI AND PROSGEGRAMMENI
|
|
</td></tr>
|
|
<tr><td>1F89 </td><td nowrap> GREEK CAPITAL LETTER ALPHA WITH DASIA AND PROSGEGRAMMENI
|
|
</td></tr>
|
|
<tr><td>1F8A </td><td nowrap>GREEK CAPITAL LETTER ALPHA WITH PSILI AND VARIA AND PROSGEGRAMMENI
|
|
</td></tr>
|
|
<tr><td>1F8B </td><td nowrap> GREEK CAPITAL LETTER ALPHA WITH DASIA AND VARIA AND PROSGEGRAMMENI
|
|
</td></tr>
|
|
<tr><td>1F8C </td><td nowrap> GREEK CAPITAL LETTER ALPHA WITH PSILI AND OXIA AND PROSGEGRAMMENI
|
|
</td></tr>
|
|
<tr><td>1F8D </td><td nowrap> GREEK CAPITAL LETTER ALPHA WITH DASIA AND OXIA AND PROSGEGRAMMENI
|
|
</td></tr>
|
|
<tr><td>1F8E </td><td nowrap> GREEK CAPITAL LETTER ALPHA WITH PSILI AND PERISPOMENI AND PROSGEGRAMMENI
|
|
</td></tr>
|
|
<tr><td>1F8F </td><td nowrap> GREEK CAPITAL LETTER ALPHA WITH DASIA AND PERISPOMENI AND PROSGEGRAMMENI
|
|
</td></tr>
|
|
<tr><td>1F98 </td><td nowrap> GREEK CAPITAL LETTER ETA WITH PSILI AND PROSGEGRAMMENI
|
|
</td></tr>
|
|
<tr><td>1F99 </td><td nowrap> GREEK CAPITAL LETTER ETA WITH DASIA AND PROSGEGRAMMENI
|
|
</td></tr>
|
|
<tr><td>1F9A </td><td nowrap> GREEK CAPITAL LETTER ETA WITH PSILI AND VARIA AND PROSGEGRAMMENI
|
|
</td></tr>
|
|
<tr><td>1F9B </td><td nowrap> GREEK CAPITAL LETTER ETA WITH DASIA AND VARIA AND PROSGEGRAMMENI
|
|
</td></tr>
|
|
<tr><td>1F9C </td><td nowrap> GREEK CAPITAL LETTER ETA WITH PSILI AND OXIA AND PROSGEGRAMMENI
|
|
</td></tr>
|
|
<tr><td>1F9D </td><td nowrap> GREEK CAPITAL LETTER ETA WITH DASIA AND OXIA AND PROSGEGRAMMENI
|
|
</td></tr>
|
|
<tr><td>1F9E </td><td nowrap> GREEK CAPITAL LETTER ETA WITH PSILI AND PERISPOMENI AND PROSGEGRAMMENI
|
|
</td></tr>
|
|
<tr><td>1F9F </td><td nowrap> GREEK CAPITAL LETTER ETA WITH DASIA AND PERISPOMENI AND PROSGEGRAMMENI
|
|
</td></tr>
|
|
<tr><td>1FA8 </td><td nowrap> GREEK CAPITAL LETTER OMEGA WITH PSILI AND PROSGEGRAMMENI
|
|
</td></tr>
|
|
<tr><td>1FA9 </td><td nowrap> GREEK CAPITAL LETTER OMEGA WITH DASIA AND PROSGEGRAMMENI
|
|
</td></tr>
|
|
<tr><td>1FAA </td><td nowrap> GREEK CAPITAL LETTER OMEGA WITH PSILI AND VARIA AND PROSGEGRAMMENI
|
|
</td></tr>
|
|
<tr><td>1FAB </td><td nowrap> GREEK CAPITAL LETTER OMEGA WITH DASIA AND VARIA AND PROSGEGRAMMENI
|
|
</td></tr>
|
|
<tr><td>1FAC </td><td nowrap> GREEK CAPITAL LETTER OMEGA WITH PSILI AND OXIA AND PROSGEGRAMMENI
|
|
</td></tr>
|
|
<tr><td>1FAD </td><td nowrap> GREEK CAPITAL LETTER OMEGA WITH DASIA AND OXIA AND PROSGEGRAMMENI
|
|
</td></tr>
|
|
<tr><td>1FAE </td><td nowrap> GREEK CAPITAL LETTER OMEGA WITH PSILI AND PERISPOMENI AND PROSGEGRAMMENI
|
|
</td></tr>
|
|
<tr><td>1FAF </td><td nowrap> GREEK CAPITAL LETTER OMEGA WITH DASIA AND PERISPOMENI AND PROSGEGRAMMENI
|
|
</td></tr>
|
|
<tr><td>1FBC </td><td nowrap> GREEK CAPITAL LETTER ALPHA WITH PROSGEGRAMMENI
|
|
</td></tr>
|
|
<tr><td>1FCC </td><td nowrap> GREEK CAPITAL LETTER ETA WITH PROSGEGRAMMENI
|
|
</td></tr>
|
|
<tr><td>1FFC </td><td nowrap> GREEK CAPITAL LETTER OMEGA WITH PROSGEGRAMMENI
|
|
</td></tr>
|
|
</table>
|
|
</div>
|
|
<p>
|
|
There are no ASCII or Latin-1 titlecase characters.
|
|
|
|
|
|
<!--========================================================================-->
|
|
<h2><a name="letter-def">char-set:letter</a></h2>
|
|
<p>
|
|
In Unicode, a letter is any character with one of the letter categories
|
|
(Lu, Ll, Lt, Lm, Lo) in the Unicode character database.
|
|
|
|
<p>
|
|
There are 52 ASCII letters
|
|
<div class=indent>
|
|
abcdefghijklmnopqrstuvwxyz <br>
|
|
ABCDEFGHIJKLMNOPQRSTUVWXYZ <br>
|
|
</div>
|
|
<p>
|
|
There are 117 Latin-1 letters. These are the 115 characters that are
|
|
members of the Latin-1 <code>char-set:lower-case</code> and <code>char-set:upper-case</code> sets,
|
|
plus
|
|
<div class=inset>
|
|
<table cellspacing=0 cellpadding=0>
|
|
<tr><td>00AA</td> <td>FEMININE ORDINAL INDICATOR</td></tr>
|
|
<tr><td>00BA</td> <td>MASCULINE ORDINAL INDICATOR</td></tr>
|
|
</table>
|
|
</div>
|
|
<p class=continue>
|
|
(These two letters are considered lower-case by Unicode, but not by
|
|
Java or SRFI 14.)
|
|
|
|
<!--========================================================================-->
|
|
<h2><a name="digit-def">char-set:digit</a></h2>
|
|
|
|
<p>
|
|
In Unicode, a character is a digit if it has the category Nd in
|
|
the character attribute database. In Latin-1 and ASCII, the only
|
|
such characters are 0123456789. In Unicode, there are other digit
|
|
characters in other code blocks, such as Gujarati digits and Tibetan
|
|
digits.
|
|
|
|
|
|
<!--========================================================================-->
|
|
<h2><a name="hex-digit-def">char-set:hex-digit</a></h2>
|
|
<p>
|
|
The only hex digits are 0123456789abcdefABCDEF.
|
|
|
|
|
|
<!--========================================================================-->
|
|
<h2><a name="letter+digit-def">char-set:letter+digit</a></h2>
|
|
<p>
|
|
The union of <code>char-set:letter</code> and <code>char-set:digit.</code>
|
|
|
|
<!--========================================================================-->
|
|
<h2><a name="graphic-def">char-set:graphic</a></h2>
|
|
<p>
|
|
A graphic character is one that would put ink on paper. The ASCII and Latin-1
|
|
graphic characters are the members of
|
|
<div class=inset>
|
|
<table cellspacing=0 cellpadding=0>
|
|
<tr><td><code>char-set:letter</code></td></tr>
|
|
<tr><td><code>char-set:digit</code></td></tr>
|
|
<tr><td><code>char-set:punctuation</code></td></tr>
|
|
<tr><td><code>char-set:symbol</code></td></tr>
|
|
</table>
|
|
</div>
|
|
|
|
<!--========================================================================-->
|
|
<h2><a name="printing-def">char-set:printing</a></h2>
|
|
<p>
|
|
A printing character is one that would occupy space when printed, <em>i.e.</em>,
|
|
a graphic character or a space character. <code>char-set:printing</code> is the union
|
|
of <code>char-set:whitespace</code> and <code>char-set:graphic.</code>
|
|
|
|
<!--========================================================================-->
|
|
<h2><a name="whitespace-def">char-set:whitespace</a></h2>
|
|
<p>
|
|
In Unicode, a whitespace character is either
|
|
<ul>
|
|
<li> a character with one of the space, line, or paragraph separator categories
|
|
(Zs, Zl or Zp) of the Unicode character database.
|
|
<li> U+0009 Horizontal tabulation (\t control-I)
|
|
<li> U+000A Line feed (\n control-J)
|
|
<li> U+000B Vertical tabulation (\v control-K)
|
|
<li> U+000C Form feed (\f control-L)
|
|
<li> U+000D Carriage return (\r control-M)
|
|
</ul>
|
|
|
|
<p>
|
|
There are 24 whitespace characters in Unicode 3.0:
|
|
<div class=inset>
|
|
<table cellspacing=0 cellpadding=0>
|
|
<tr><td>0009</td> <td>HORIZONTAL TABULATION </td> <td> \t control-I</td></tr>
|
|
<tr><td>000A</td> <td>LINE FEED </td> <td> \n control-J</td></tr>
|
|
<tr><td>000B</td> <td>VERTICAL TABULATION </td> <td> \v control-K</td></tr>
|
|
<tr><td>000C</td> <td>FORM FEED </td> <td> \f control-L</td></tr>
|
|
<tr><td>000D</td> <td>CARRIAGE RETURN </td> <td> \r control-M</td></tr>
|
|
<tr><td>0020</td> <td>SPACE </td> <td> Zs</td></tr>
|
|
<tr><td>00A0</td> <td>NO-BREAK SPACE </td> <td> Zs</td></tr>
|
|
<tr><td>1680</td> <td>OGHAM SPACE MARK </td> <td> Zs</td></tr>
|
|
<tr><td>2000</td> <td>EN QUAD </td> <td> Zs</td></tr>
|
|
<tr><td>2001</td> <td>EM QUAD </td> <td> Zs</td></tr>
|
|
<tr><td>2002</td> <td>EN SPACE </td> <td> Zs</td></tr>
|
|
<tr><td>2003</td> <td>EM SPACE </td> <td> Zs</td></tr>
|
|
<tr><td>2004</td> <td>THREE-PER-EM SPACE </td> <td> Zs</td></tr>
|
|
<tr><td>2005</td> <td>FOUR-PER-EM SPACE </td> <td> Zs</td></tr>
|
|
<tr><td>2006</td> <td>SIX-PER-EM SPACE </td> <td> Zs</td></tr>
|
|
<tr><td>2007</td> <td>FIGURE SPACE </td> <td> Zs</td></tr>
|
|
<tr><td>2008</td> <td>PUNCTUATION SPACE </td> <td> Zs</td></tr>
|
|
<tr><td>2009</td> <td>THIN SPACE </td> <td> Zs</td></tr>
|
|
<tr><td>200A</td> <td>HAIR SPACE </td> <td> Zs</td></tr>
|
|
<tr><td>200B</td> <td>ZERO WIDTH SPACE </td> <td> Zs</td></tr>
|
|
<tr><td>2028</td> <td>LINE SEPARATOR </td> <td> Zl</td></tr>
|
|
<tr><td>2029</td> <td>PARAGRAPH SEPARATOR </td> <td> Zp</td></tr>
|
|
<tr><td>202F</td> <td>NARROW NO-BREAK SPACE </td> <td> Zs</td></tr>
|
|
<tr><td>3000</td> <td>IDEOGRAPHIC SPACE </td> <td> Zs</td></tr>
|
|
</table>
|
|
</div>
|
|
<p>
|
|
The ASCII whitespace characters are the first six characters in the above list
|
|
-- line feed, horizontal tabulation, vertical tabulation, form feed, carriage
|
|
return, and space. These are also exactly the characters recognised by the
|
|
Posix <code>isspace()</code> procedure. Latin-1 adds the no-break space.
|
|
|
|
<p>
|
|
Note: Java's <code>isWhitespace()</code> method is incompatible, including
|
|
<div class=inset>
|
|
<table cellspacing=0 cellpadding=0>
|
|
<tr><td>0009</td> <td>HORIZONTAL TABULATION </td> <td> (\t control-I)</td></tr>
|
|
<tr><td>001C</td> <td>FILE SEPARATOR </td> <td> (control-\)</td></tr>
|
|
<tr><td>001D</td> <td>GROUP SEPARATOR </td> <td>(control-])</td></tr>
|
|
<tr><td>001E</td> <td>RECORD SEPARATOR </td> <td>(control-^)</td></tr>
|
|
<tr><td>001F</td> <td>UNIT SEPARATOR </td> <td>(control-_)</td></tr>
|
|
</table>
|
|
</div>
|
|
<p class=continue>
|
|
and excluding
|
|
<div class=inset>
|
|
<table cellspacing=0 cellpadding=0>
|
|
<tr><td>00A0</td> <td>NO-BREAK SPACE</td></tr>
|
|
</table>
|
|
</div>
|
|
<p>
|
|
Java's excluding the no-break space means that tokenizers can simply break
|
|
character streams at "whitespace" boundaries. However, the exclusion introduces
|
|
exceptions in other places, <em>e.g.</em> <code>char-set:printing</code> is no longer simply the
|
|
union of <code>char-set:graphic</code> and <code>char-set:whitespace.</code>
|
|
|
|
|
|
<!--========================================================================-->
|
|
<h2><a name="iso-control-def">char-set:iso-control</a></h2>
|
|
<p>
|
|
The ISO control characters are the Unicode/Latin-1 characters in the ranges
|
|
[U+0000,U+001F] and [U+007F,U+009F].
|
|
|
|
<p>
|
|
ASCII restricts this set to the characters in the range [U+0000,U+001F]
|
|
plus the character U+007F.
|
|
|
|
<p>
|
|
Note that Unicode defines other control characters which do not belong to this
|
|
set (hence the qualifying prefix "iso-" in the name). This restriction is
|
|
compatible with the Java <code>IsISOControl()</code> method.
|
|
|
|
|
|
<!--========================================================================-->
|
|
<h2><a name="punctuation-def">char-set:punctuation</a></h2>
|
|
<p>
|
|
In Unicode, a punctuation character is any character that has one of the
|
|
punctuation categories in the Unicode character database (Pc, Pd, Ps,
|
|
Pe, Pi, Pf, or Po.)
|
|
|
|
<p>
|
|
ASCII has 23 punctuation characters:
|
|
<pre class=code-example>
|
|
!"#%&'()*,-./:;?@[\]_{}
|
|
</pre>
|
|
<p>
|
|
Latin-1 adds six more:
|
|
<div class=inset>
|
|
<table cellspacing=0 cellpadding=0>
|
|
<tr><td>00A1 </td> <td> INVERTED EXCLAMATION MARK
|
|
<tr><td>00AB </td> <td> LEFT-POINTING DOUBLE ANGLE QUOTATION MARK
|
|
<tr><td>00AD </td> <td> SOFT HYPHEN
|
|
<tr><td>00B7 </td> <td> MIDDLE DOT
|
|
<tr><td>00BB </td> <td> RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK
|
|
<tr><td>00BF </td> <td> INVERTED QUESTION MARK
|
|
</table>
|
|
</div>
|
|
|
|
<p>
|
|
Note that the nine ASCII characters <code>$+<=>^`|~</code> are <em>not</em>
|
|
punctuation. They are "symbols."
|
|
|
|
|
|
<!--========================================================================-->
|
|
<h2><a name="symbol-def">char-set:symbol</a></h2>
|
|
<p>
|
|
In Unicode, a symbol is any character that has one of the symbol categories
|
|
in the Unicode character database (Sm, Sc, Sk, or So). There
|
|
are nine ASCII symbol characters:
|
|
<pre class=code-example>
|
|
$+<=>^`|~
|
|
</pre>
|
|
<p>
|
|
Latin-1 adds 18 more:
|
|
<div class=inset>
|
|
<table cellspacing=0 cellpadding=0>
|
|
<tr><td>00A2 </td> <td> CENT SIGN </td></tr>
|
|
<tr><td>00A3 </td> <td> POUND SIGN </td></tr>
|
|
<tr><td>00A4 </td> <td> CURRENCY SIGN </td></tr>
|
|
<tr><td>00A5 </td> <td> YEN SIGN </td></tr>
|
|
<tr><td>00A6 </td> <td> BROKEN BAR </td></tr>
|
|
<tr><td>00A7 </td> <td> SECTION SIGN </td></tr>
|
|
<tr><td>00A8 </td> <td> DIAERESIS </td></tr>
|
|
<tr><td>00A9 </td> <td> COPYRIGHT SIGN </td></tr>
|
|
<tr><td>00AC </td> <td> NOT SIGN </td></tr>
|
|
<tr><td>00AE </td> <td> REGISTERED SIGN </td></tr>
|
|
<tr><td>00AF </td> <td> MACRON </td></tr>
|
|
<tr><td>00B0 </td> <td> DEGREE SIGN </td></tr>
|
|
<tr><td>00B1 </td> <td> PLUS-MINUS SIGN </td></tr>
|
|
<tr><td>00B4 </td> <td> ACUTE ACCENT </td></tr>
|
|
<tr><td>00B6 </td> <td> PILCROW SIGN </td></tr>
|
|
<tr><td>00B8 </td> <td> CEDILLA </td></tr>
|
|
<tr><td>00D7 </td> <td> MULTIPLICATION SIGN </td></tr>
|
|
<tr><td>00F7 </td> <td> DIVISION SIGN </td></tr>
|
|
</table>
|
|
</div>
|
|
|
|
<!--========================================================================-->
|
|
<h2><a name="blank-def">char-set:blank</a></h2>
|
|
|
|
<p>
|
|
Blank chars are horizontal whitespace. In Unicode, a blank character is either
|
|
<ul>
|
|
<li> a character with the space separator category (Zs) in the Unicode
|
|
character database.
|
|
<li> U+0009 Horizontal tabulation (\t control-I)
|
|
</ul>
|
|
|
|
<p>
|
|
There are eighteen blank characters in Unicode 3.0:
|
|
<div class=inset>
|
|
<table cellspacing=0 cellpadding=0>
|
|
<tr><td>0009 </td> <td> HORIZONTAL TABULATION </td> <td> \t control-I </td></tr>
|
|
<tr><td>0020 </td> <td> SPACE </td> <td> Zs </td></tr>
|
|
<tr><td>00A0 </td> <td> NO-BREAK SPACE </td> <td> Zs </td></tr>
|
|
<tr><td>1680 </td> <td> OGHAM SPACE MARK </td> <td> Zs </td></tr>
|
|
<tr><td>2000 </td> <td> EN QUAD </td> <td> Zs </td></tr>
|
|
<tr><td>2001 </td> <td> EM QUAD </td> <td> Zs </td></tr>
|
|
<tr><td>2002 </td> <td> EN SPACE </td> <td> Zs </td></tr>
|
|
<tr><td>2003 </td> <td> EM SPACE </td> <td> Zs </td></tr>
|
|
<tr><td>2004 </td> <td> THREE-PER-EM SPACE </td> <td> Zs </td></tr>
|
|
<tr><td>2005 </td> <td> FOUR-PER-EM SPACE </td> <td> Zs </td></tr>
|
|
<tr><td>2006 </td> <td> SIX-PER-EM SPACE </td> <td> Zs </td></tr>
|
|
<tr><td>2007 </td> <td> FIGURE SPACE </td> <td> Zs </td></tr>
|
|
<tr><td>2008 </td> <td> PUNCTUATION SPACE </td> <td> Zs </td></tr>
|
|
<tr><td>2009 </td> <td> THIN SPACE </td> <td> Zs </td></tr>
|
|
<tr><td>200A </td> <td> HAIR SPACE </td> <td> Zs </td></tr>
|
|
<tr><td>200B </td> <td> ZERO WIDTH SPACE </td> <td> Zs </td></tr>
|
|
<tr><td>202F </td> <td> NARROW NO-BREAK SPACE </td> <td> Zs </td></tr>
|
|
<tr><td>3000 </td> <td> IDEOGRAPHIC SPACE </td> <td> Zs </td></tr>
|
|
</table>
|
|
</div>
|
|
<p>
|
|
The ASCII blank characters are the first two characters above --
|
|
horizontal tab and space. Latin-1 adds the no-break space.
|
|
|
|
<p>
|
|
Java doesn't have the concept of "blank" characters, so there are no
|
|
compatibility issues.
|
|
|
|
|
|
<!--========================================================================-->
|
|
<h1><a name="ReferenceImp">Reference implementation</a></h1>
|
|
<p>
|
|
This SRFI comes with a reference implementation. It resides at:
|
|
<div class=inset>
|
|
<a href="http://srfi.schemers.org/srfi-14/srfi-14.scm">
|
|
http://srfi.schemers.org/srfi-14/srfi-14.scm</a>
|
|
</div>
|
|
<p class=continue>
|
|
I have placed this source on the Net with an unencumbered, "open" copyright.
|
|
Some of the code in the reference implementation bears a distant family
|
|
relation to the MIT Scheme implementation, and being derived from that code,
|
|
is covered by the MIT Scheme copyright (which is a generic BSD-style
|
|
open-source copyright -- see the source file for details). The remainder of
|
|
the code was written by myself for scsh or for this SRFI; I have placed this
|
|
code under the scsh copyright, which is also a generic BSD-style open-source
|
|
copyright.
|
|
|
|
<p>
|
|
The code is written for portability and should be simple to port to
|
|
any Scheme. It has only the following deviations from R4RS, clearly
|
|
discussed in the comments:
|
|
<ul>
|
|
<li> an <code>error</code> procedure;
|
|
<li> the R5RS <code>values</code> procedure for producing multiple return values;
|
|
<li> a simple <code>check-arg</code> procedure for argument checking;
|
|
<li> <code>let-optionals*</code> and <code>:optional</code> macros for for parsing, checking and defaulting
|
|
optional arguments from rest lists;
|
|
<li> The SRFI-19 <code>define-record-type</code> form;
|
|
<li> <code>bitwise-and</code> for the hash function;
|
|
<li> <code>%latin1->char</code> and <code>%char->latin1</code>.
|
|
</ul>
|
|
|
|
<p>
|
|
The library is written for clarity and well-commented; the current source is
|
|
about 375 lines of source code and 375 lines of comments and white space.
|
|
It is also written for efficiency. Fast paths are provided for common cases.
|
|
|
|
<p>
|
|
This is not to say that the implementation can't be tuned up for
|
|
a specific Scheme implementation. There are notes in comments addressing
|
|
ways implementors can tune the reference implementation for performance.
|
|
|
|
<p>
|
|
In short, I've written the reference implementation to make it as painless
|
|
as possible for an implementor -- or a regular programmer -- to adopt this
|
|
library and get good results with it.
|
|
|
|
<p>
|
|
The code uses a rather simple-minded, inefficient representation for
|
|
ASCII/Latin-1 char-sets -- a 256-character string. The character whose code is
|
|
<var>i</var> is in the set if <var>s[i]</var> = ASCII 1 (soh, or ^a);
|
|
not in the set if <var>s[i]</var> = ASCII 0 (nul).
|
|
A much faster and denser representation would be 16 or 32 bytes worth
|
|
of bit string. A portable implementation using bit sets awaits standards for
|
|
bitwise logical-ops and byte vectors.
|
|
|
|
<p>
|
|
"Large" character types, such as Unicode, should use a sparse representation,
|
|
taking care that the Latin-1 subset continues to be represented with a
|
|
dense 32-byte bit set.
|
|
|
|
|
|
<!--========================================================================-->
|
|
<h1><a name="Acknowledgements">Acknowledgements</a></h1>
|
|
<p>
|
|
The design of this library benefited greatly from the feedback provided during
|
|
the SRFI discussion phase. Among those contributing thoughtful commentary and
|
|
suggestions, both on the mailing list and by private discussion, were Paolo
|
|
Amoroso, Lars Arvestad, Alan Bawden, Jim Bender, Dan Bornstein, Per Bothner,
|
|
Will Clinger, Brian Denheyer, Kent Dybvig, Sergei Egorov, Marc Feeley,
|
|
Matthias Felleisen, Will Fitzgerald, Matthew Flatt, Arthur A. Gleckler, Ben
|
|
Goetter, Sven Hartrumpf, Erik Hilsdale, Shiro Kawai, Richard Kelsey, Oleg
|
|
Kiselyov, Bengt Kleberg, Donovan Kolbly, Bruce Korb, Shriram Krishnamurthi,
|
|
Bruce Lewis, Tom Lord, Brad Lucier, Dave Mason, David Rush, Klaus Schilling,
|
|
Jonathan Sobel, Mike Sperber, Mikael Staldal, Vladimir Tsyshevsky, Donald
|
|
Welsh, and Mike Wilson. I am grateful to them for their assistance.
|
|
|
|
<p>
|
|
I am also grateful the authors, implementors and documentors of all the
|
|
systems mentioned in the introduction. Aubrey Jaffer should be noted for his
|
|
work in producing Web-accessible versions of the R5RS spec, which was a
|
|
tremendous aid.
|
|
|
|
<p>
|
|
This is not to imply that these individuals necessarily endorse the final
|
|
results, of course.
|
|
|
|
<p>
|
|
During this document's long development period, great patience was exhibited
|
|
by Mike Sperber, who is the editor for the SRFI, and by Hillary Sullivan,
|
|
who is not.
|
|
|
|
<!--========================================================================-->
|
|
<h1><a name="Links">References & links</a></h1>
|
|
|
|
<dl>
|
|
<dt class=biblio><strong><a name="Java">[Java]</a></strong>
|
|
<dd>
|
|
The following URLs provide documentation on relevant Java classes. <br>
|
|
|
|
<a href="http://java.sun.com/products/jdk/1.2/docs/api/java/lang/Character.html">http://java.sun.com/products/jdk/1.2/docs/api/java/lang/Character.html</a>
|
|
<br>
|
|
<a href="http://java.sun.com/products/jdk/1.2/docs/api/java/lang/String.html">http://java.sun.com/products/jdk/1.2/docs/api/java/lang/String.html</a>
|
|
<br>
|
|
<a href="http://java.sun.com/products/jdk/1.2/docs/api/java/lang/StringBuffer.html">http://java.sun.com/products/jdk/1.2/docs/api/java/lang/StringBuffer.html</a>
|
|
<br>
|
|
<a href="http://java.sun.com/products/jdk/1.2/docs/api/java/text/Collator.html">http://java.sun.com/products/jdk/1.2/docs/api/java/text/Collator.html</a>
|
|
<br>
|
|
<a href="http://java.sun.com/products/jdk/1.2/docs/api/java/text/package-summary.html">http://java.sun.com/products/jdk/1.2/docs/api/java/text/package-summary.html</a>
|
|
|
|
<dt class=biblio><strong><a name="MIT-Scheme">[MIT-Scheme]</a></strong>
|
|
<dd>
|
|
<a href="http://www.swiss.ai.mit.edu/projects/scheme/">http://www.swiss.ai.mit.edu/projects/scheme/</a>
|
|
|
|
<dt class=biblio><strong><a name="R5RS">[R5RS]</a></strong></dt>
|
|
<dd>Revised<sup>5</sup> report on the algorithmic language Scheme.<br>
|
|
R. Kelsey, W. Clinger, J. Rees (editors). <br>
|
|
Higher-Order and Symbolic Computation, Vol. 11, No. 1, September, 1998. <br>
|
|
and ACM SIGPLAN Notices, Vol. 33, No. 9, October, 1998. <br>
|
|
Available at <a href="http://www.schemers.org/Documents/Standards/">
|
|
http://www.schemers.org/Documents/Standards/</a>.
|
|
|
|
<dt class=biblio><strong>[SRFI]</strong></dt>
|
|
<dd>
|
|
The SRFI web site. <br>
|
|
<a href="http://srfi.schemers.org/">http://srfi.schemers.org/</a>
|
|
|
|
<dt class=biblio><strong>[SRFI-14]</strong></dt>
|
|
<dd>
|
|
SRFI-14: String libraries. <br>
|
|
<a href="http://srfi.schemers.org/srfi-14/">http://srfi.schemers.org/srfi-14/</a>
|
|
|
|
<dl>
|
|
<dt>
|
|
This document, in HTML:
|
|
<dd><a href="http://srfi.schemers.org/srfi-14/srfi-14.html">
|
|
http://srfi.schemers.org/srfi-14/srfi-14.html</a>
|
|
|
|
<dt>
|
|
This document, in plain text format:
|
|
<dd><a href="http://srfi.schemers.org/srfi-14/srfi-14.txt">
|
|
http://srfi.schemers.org/srfi-14/srfi-14.txt</a>
|
|
|
|
<dt> Source code for the reference implementation:
|
|
<dd>
|
|
<a href="http://srfi.schemers.org/srfi-14/srfi-14.scm">
|
|
http://srfi.schemers.org/srfi-14/srfi-14.scm</a>
|
|
|
|
<dt> Scheme 48 module specification, with typings:
|
|
<dd>
|
|
<a href="http://srfi.schemers.org/srfi-14/srfi-14-s48-module.scm">
|
|
http://srfi.schemers.org/srfi-14/srfi-14-s48-module.scm</a>
|
|
|
|
<dt> Regression-test suite:
|
|
<dd> <a href="http://srfi.schemers.org/srfi-14/srfi-14-tests.scm">
|
|
http://srfi.schemers.org/srfi-14/srfi-14-tests.scm</a>
|
|
|
|
</dl>
|
|
</dd>
|
|
|
|
<dt class=biblio><strong><a name="Unicode">[Unicode]</a></strong>
|
|
<dd>
|
|
<a href="http://www.unicode.org/">http://www.unicode.org/</a>
|
|
|
|
<dt class=biblio><strong><a name="UnicodeData">[UnicodeData]</a></strong>
|
|
<dd>
|
|
The Unicode character database. <br>
|
|
<a href="ftp://ftp.unicode.org/Public/UNIDATA/UnicodeData.txt">ftp://ftp.unicode.org/Public/UNIDATA/UnicodeData.txt</a>
|
|
<br>
|
|
<a href="ftp://ftp.unicode.org/Public/UNIDATA/UnicodeData.html">ftp://ftp.unicode.org/Public/UNIDATA/UnicodeData.html</a>
|
|
</dl>
|
|
|
|
<!--========================================================================-->
|
|
<h1><a name="Copyright">Copyright</a></h1>
|
|
|
|
<p>
|
|
Certain portions of this document -- the specific, marked segments of text
|
|
describing the <abbr title="Revised^5 Report on Scheme"><a href="#R5RS">R5RS</a></abbr> procedures -- were adapted with permission from the R5RS
|
|
report.
|
|
|
|
<p>
|
|
All other text is copyright (C) Olin Shivers (1998, 1999, 2000).
|
|
All Rights Reserved.
|
|
|
|
<p>
|
|
This document and translations of it may be copied and furnished to others,
|
|
and derivative works that comment on or otherwise explain it or assist in its
|
|
implementation may be prepared, copied, published and distributed, in whole or
|
|
in part, without restriction of any kind, provided that the above copyright
|
|
notice and this paragraph are included on all such copies and derivative
|
|
works. However, this document itself may not be modified in any way, such as
|
|
by removing the copyright notice or references to the Scheme Request For
|
|
Implementation process or editors, except as needed for the purpose of
|
|
developing SRFIs in which case the procedures for copyrights defined in the
|
|
SRFI process must be followed, or as required to translate it into languages
|
|
other than English.
|
|
|
|
<p>
|
|
The limited permissions granted above are perpetual and will not be revoked by
|
|
the authors or their successors or assigns.
|
|
|
|
<p>
|
|
This document and the information contained herein is provided on an
|
|
"<strong>as is</strong>" basis and <strong>the authors and the SRFI editors
|
|
disclaim all warranties, express or implied, including but not limited to any
|
|
warranty that the use of the information herein will not infringe any rights
|
|
or any implied warranties of merchantability or fitness for a particular
|
|
purpose.</strong>
|
|
|
|
</body>
|
|
</html>
|
|
<!--
|
|
LocalWords: SRFI refs HTML css hackery sans Netscape td pre div para
|
|
LocalWords: proc def procs defi's defn dl dt defi dd NS RS rs procx
|
|
LocalWords: stylesheet IE biblio IE's Internationalisation ascii doc
|
|
LocalWords: normalisation lib ref ci ok titlecase upcase downcase
|
|
LocalWords: xsubstring xcopy tokenize kmp slib RScheme MzScheme init
|
|
LocalWords: Bigloo Chez APL SML Unicode API eszet SS dz downcases
|
|
LocalWords: titlecasing normalised normalise underbar ss eq vs dict
|
|
LocalWords: backquote parameterised denmark taiwan UnicodeData txt
|
|
LocalWords: pred nchars obj len cBa epilog foo baz wrt subst tstart
|
|
LocalWords: Szilagyi zilagyi cs abcdefgh ca cd cond eek ee tHIS com
|
|
LocalWords: elba elbA ary consed XXXX ac bc kons knil ans plusses
|
|
LocalWords: catamorphism lp eof lis cdr knull kar kdr anamorphism
|
|
LocalWords: abcdefg sfrom sto TCL perl slen rv exp initialisation
|
|
LocalWords: plen SJ PJ si sj pj IPORT iport patlen DF buf Bevan
|
|
LocalWords: Denheyer scsh Paolo Amoroso Arvestad Bawden Dybvig
|
|
LocalWords: Bornstein Bothner Egorov Feeley Matthias Felleisen
|
|
LocalWords: Flatt ucs Gleckler Goetter Sven Hartrumpf Hilsdale
|
|
LocalWords: Kiselyov Bengt Korb Kleberg Kolbly Shriram bignum
|
|
LocalWords: Krishnamurthi Lucier Schilling Sobel Mikael Staldal
|
|
LocalWords: Tsyshevsky documentors Jaffer Sperber cltl AE fixnum
|
|
LocalWords: CommonLisp HyperSpec Clinger Rees SIGPLAN uniquified
|
|
LocalWords: cset EA DrScheme IEC conformant JIS xor diff Posix URL
|
|
LocalWords: FFF DIAERESIS abcdefghijklmnopqrstuvwxyz EB EC EF ETH
|
|
LocalWords: FA FB FC FD FF Ll AA diaeresis isLowerCase BA CB CC CE
|
|
LocalWords: CF DA DC Lt CARON PSILI Lu PROSGEGRAMMENI DASIA VARIA
|
|
LocalWords: OXIA PERISPOMENI FAA FAB FAC FAE FAF FBC FFC Lm Lo
|
|
LocalWords: abcdefABCDEF Zs Zl Zp OGHAM IDEOGRAPHIC Pc recognised
|
|
LocalWords: tokenizers iso Pd Ps Pe Pf AB BB BF Sm Sc Sk AF MACRON
|
|
LocalWords: PILCROW soh nul ops Shiro Kawai subform
|
|
-->
|