The page for "string.ss" starts with load instructions.
NORVIG SPELLING CORRECTOR HOW TO
Here is how to figure this out: Search for regexp-split in the HelpDesk. Hi Phil, ad 1) Use (require (lib "string.ss")) to load regexp-split. How could we get that kind of improvement in Scheme? 15:05 Jens Axel Søgaard said. I assume the difference is that sets are built in to Python. (Note: I had to add spaces around the word "set" in the error message to get around the html verifier in the comment box, which thought it was seeing an html tag named "set".) What's wrong? 3) Norvig achieves a speed of 0.1 seconds per correction with Python, but the Scheme equivalent is much slower: on my machine, (time (correct "korrecter")) takes 8578 milliseconds, a factor of about a hundred times slower.
Calling (correct "addresable") causes an error: top-level broke the contract (-> set boolean) on empty? expected, given: ("addresable"). Is there a way in DrScheme to force the language version? 2) Addresable is one of the words in Norvig's test corpus. But regex-split requires the Pretty Big language. The sticky point was the string manipulation. Hi Kyle, It was fun to see, whether I could make it almost as short as the Python version. The action is in raw-red-black-tree-set.scm and the finishing touches is in: red-black-tree-set.scm" Since Gauche has a port of Wright's pattern matcher it ought to straight forward to port.
NORVIG SPELLING CORRECTOR CODE
Hi Shiro, Feel free to steel any code you like from Galore. And I certainly wouldn't have done as well at explaining it as you've done. Besides, I've got too many open projects as it is. I had read Norvig's piece on the probabilistic spelling checker in Python, and was going to implement it myself, but I couldn't think of a source for words the corpus as you put it. Nice article! I love how you analyze the problem and nail down where to attack (namely, concat) I've just gone blindly instead. ( maximum ( elements candidates) word-count))) ( define ( falsify s) ( if ( empty? s) #f s)) ( set-ec string-compare ( : i n) ( : c alphabet) ( concat word 0 i c word i _)))) ( set-ec string-compare ( : i n) ( : c alphabet) ( concat word 0 i c word ( + i 1) _)) ( set-ec string-compare ( : i ( - n 1)) ( concat word 0 i word ( + i 1) word i word ( + i 2) _)) ( set-ec string-compare ( : i n) ( concat word 0 i word ( + i 1) _)) ( define alphabet ( string-ec ( : c #\a #\z) c)) ( foldl union ( empty string-compare) sets)) ( cons ( symbol->string s) ( subs spec))] ( cons ( string ( string-ref s n)) ( subs spec))] ( cons ( substring s 0 n2) ( subs spec))] ( cons ( substring s n1 ( string-length s)) ( subs spec))] ( cons ( substring s n1 n2) ( subs spec))] ( define NWORDS ( train ( words ( open-input-file file)))) ( hash-table-put! model word ( add1 ( hash-table-get model word 1)))) train : list-of-strings -> hash-table-from-strings-to-numbers
( string-downcase ( bytes->string/latin-1 word)))) ( list-ec ( : word ( regexp-split #rx "" text)) The concatenation operator + converts automatically the character c to a string before the strings are concatenated.Ī literal translation to Scheme reads as follows: The c is a character, and word is the substring from index i to the end. + c + word Here word is the substring from index 0 (inclusive) to index i (exclusive). Īs an example consider this expression from Norvig's spelling corrector: Compared to other languages code for string manipulations tend to become verbose, unless one "cheats" and uses some sort of utility such as format. Scheme offers the usual operations on strings: string concatenation (string-append), referencing a character in a string (string-ref), extracting a substring (substring) and converting characters to strings (string). But first let's look at string manipulations. In the following I'll present a solution in PLT Scheme. Since Norvig used Python, Shiro decided to write a version in Gauche Scheme. Peter Norvig recently wrote a great piece on How to Write a Spelling Corrector.