{"size": 2581, "ext": "scrbl", "max_stars_repo_path": "scribblings/generator.scrbl", "max_stars_repo_name": "johnstonskj/racket-codepoint", "max_stars_repo_stars_event_min_datetime": null, "max_stars_repo_stars_event_max_datetime": null, "max_issues_repo_path": "scribblings/generator.scrbl", "max_issues_repo_name": "johnstonskj/racket-codepoint", "max_issues_repo_issues_event_min_datetime": null, "max_issues_repo_issues_event_max_datetime": null, "max_forks_repo_path": "scribblings/generator.scrbl", "max_forks_repo_name": "johnstonskj/racket-codepoint", "max_forks_repo_forks_event_min_datetime": null, "max_forks_repo_forks_event_max_datetime": null}
{"max_stars_count": null, "max_issues_count": null, "max_forks_count": null, "avg_line_length": 49.6346153846, "max_line_length": 252, "alphanum_fraction": 0.7667570709}
#lang scribble/manual

@(require racket/file scribble/core)

@title[#:tag "data-generator"]{Data Generator}

These data files themselves are generated by tooling from the @hyperlink["https://unicode.org/ucd/"]{Unicode Character Database} primarily @code{UCD.zip} and @code{Unihan.zip}. These zip files, along with other reference material are downloaded into a 
temporary location and used to generate the Racket data files used by @racket[codepoint/properties]. The fetch and generate
processes @italic{may} be used separately but are intended to be run together by the executable @code{ucd-generator}.

@section[]{Running the generator}

As a part of the build executed by @code{raco setup}, an executable named @code{ucd-generator} is generated. This tool is
used to create a set of data files according to the following steps.

@itemlist[#:style 'ordered
  @item{If the temporary directory, @code{data}, is not present:
	@itemlist[#:style 'ordered
	  @item{create the temporary directory,}
	  @item{use the curl command to download the required files, and}
	  @item{unzip the @code{UCD.zip} file.}
    ]}
  @item{If the output directory, @code{generated}, is not present then create it.}
  @item{For each of the relevant source files generate each target data file.}
]

Note that this tool @bold{must} be run in the root directory of the package.

@section[]{Internal module codepoint/generator}
@defmodule[codepoint/generator]

This module has both a provided function interface as well as a @code{main} module. The @code{main} module is actually
the entry point for the generated @code{ucd-generator} executable, but can also be invoked directly as @code{racket ./private/generator.rkt}. 

@defproc[(generate-modules [root-dir path-string?]) boolean?]{
    Perform the steps outlined above to download UCD data files and generate the data modules. Generated modules are placed
    in the @code{generated} directory and have a file extension of @code{"rkt-src"} as these are not actually modules but the
    serialized data structures used by the functions in @racket[codepoint/properties].
}


@section[]{Internal module codepoint/ucd}
@defmodule[codepoint/ucd]

This module also has both a provided function interface as well as a @code{main} module. While this can be invoked on it's own
to download data files it is typically invoked by the @racket[generate-modules] function, or @code{ucd-generator} executable.

@defproc[(fetch-unicode-character-data [root-dir path-string?]) boolean?]{
    Perform the steps outlined above to download and unpack the UCD data files.
}

