This text is also available as comment of the class AGenieIntroduction

Genie version: 2.01
NS 8/8/2001 15:18


*** Genie: An introduction ***

Genie is a character and gesture recognition system inside Squeak. It is completely configurable and allows you to control everything in Squeak by just using a pen.

Contents of this introduction

1) Getting started
1.1) Filing it in
1.2) Genie specific items in the World's help menu
1.3) How to enter gestures with Genie?
1.3.1) Genie's behavior is Morph specific
1.3.2) Focus Genie on a certain Morph
1.3.3) Escaping the recognizer
1.4) Genie and events: How does it work?

2) Gesture dictionaries
2.1) Specifiying and changing the gesture dictionary of a Morph
2.1.1) Name and exported name of a gesture dictionary
2.1.2) Default gesture dictionary of a Morph subclass
2.1.3) Gesture dictionary of a Morph instance
2.2) Inspecting, editing and browsing dictionaries
2.2.1) Browsing a dictionary
2.2.2) Adding new gestures for a character
2.2.3) Adding new characters or editing characters
2.2.4) Dictionary inheritance
2.3) The different character types

3) Display properties

4) Integration into Morphic
4.1) Genie and events: The details
4.2) How a Morph can influence gesture handling
4.3) Information available in a CRGesture

5) Examples and tips
5.1) Available exmaple dictionaries
5.2) Tips on building your own dictionaries

6) Version history

7) The Genie plugin (testing and compiling)
7.1) Testing the plugin
7.2) Compiling and installing the plugin

8) Acknowledgement



1) Getting started


1.1) Filing it in

The genie system consists of the following 3 change-sets:

a) Genie engine: Contains all the classes involved in recognizing a gesture and looking it up
in a dictionary. [Category: Genie-Engine].
b) Genie UI: The UI to configure the gesture recognizer.
(Inspect/edit/load/save/copy/delete gesture dictionaries, display properties, etc.)
[Category: Genie-UI]
c) Genie Integration: This code integrates Genie into Morphic.
It consists of system integration code in classes like HandMorph, Morph, TextMorph,
MorphicEvent, etc.

To get a ready-to-use Genie environment, you have to file in the change-sets in the order a) to c).
(It would also be possible to create a Genie environment that has no UI or only reduced integration into the system.)



1.2) Genie specific items in the World's help menu

There are 3 items in a World's help menu that are genie specific:
- Enable/disable Genie: Enables/disables Genie for the current hand.
- Genie character dictionaries: Inspect and edit the named genie character dictionaries.
- Choose Genie text dictionary: Select the default dictionary for text input.
- Genie display properties: Inspect and edit the named genie display properties.


1.3) How to enter gestures with Genie?

This is a little description of how to enter gestures with Genie. Note that Genie is controlled by the red (first) mouse button. All the other mouse buttons have no effect on Genie. Of course, Genie can also be controlled by a pen. (Squeak treats pen operations in the same way as red mouse button operations).


1.3.1) Genie's behavior is Morph specific

The behavior of Genie is Morph specific. Every Morph that handles Genie gestures has an associated Gesture dictionary. This dictionary specifies the parameters to capture and lookup gestures. You can start entering a gesture for such a Morph by pressing down the red mouse button somewhere on that Morph.
When you release the button, the Morph dispatches the gesture using the associated dictionary.
Morphs that don't handle Gesture dictionaries react with the default behavior to mouse down events. You can change the gesture dictionary of any Morph in several ways. See section 2 for more details.


1.3.2) Focus Genie on a certain Morph

Some Morphs are very small (small text panes, etc.) and it may not be very handy that every gesture for such a Morph has to start somewhere on this Morph. Therefore, Genie provides the option to focus itself on a certain Morph. This means that all the red button mouse actions are treated as if they would start on that Morph.
To focus Genie on a Morph you have to enter the gesture that is associated to the command #switchFocus on that Morph. You can use the same gesture to diasble the focus later. While Genie is focused on a certain Morph, the cursor shape is a dot instead of an arrow.


1.3.3) Escaping the recognizer

If a Morph handles Genie gestures a red button mouse down is interpreterd as the start event for capturing a gesture. However, sometimes a user wants to do a traditional red mouse button click or he wants to drag/select with the red mouse button (select text, etc.). This can be done by pressing the red mouse button and hold it down for the 'escape time' without moving the mouse. The 'escape time' is a parameter of the gesture dictionary and can be changed in the dictionary tool (See section 2.2 for more information). Using this feature it is for example possible to move the cursor or select text in a pane that has an associated gesture dictionary.
Note: Genie offers commands to do mouse actions with any mouse button. You can find more about that in the following sections. See also section 5.2.


1.4) Genie and events: How does it work?

If Genie is enabled for a certain hand, handling of mouse events generated by this hand is slightly modified. Here is a short description of how it works. (For more details see section 4.1):

On every mouse down event, the hand determines the target morph (handler) and the event is sent to this Morph. (This part happens in exectly the same way as it does without Genie). If it is a red button event the target morph decides wheter

a) it doesn't want to handle gestures
This means that the mouse down event gets processed in the usual way and Genie doesn't get involved at all.

b) it wants to handle gestures
In this case the Morph tells Genie to capture a gesture in according to the parameters of the gesture dictionary that is associated with the Morph. As a consequence, Genie captures all the mouse move events until the next mouse up event occurs. Then it informs the target Morph about the captured gesture and the Morph can take the appropriate action.

NOTE: Every gesture dictionary can define an 'escape time'. If the cursor is not moved away from the start position during this time, the recognizer escapes and simulates default mouse events. Like this it is possible to use default mouse events even with a Morph that handles gestures. (There are also other special commands that escape the recognizer and allow simulation of mouse events with any possible mouse button).



2) Gesture dictionaries

The main reason for Genie's flexibility are gesture dictionaries. Every Morph that handles gestures has an associated gesture dictionary. This dictionary is used to lookup the characters or actions associated to a certain gesture. In addition, it specifies lots of parameters about the capturing and lookup process.


2.1) Specifiying and changing the gesture dictionary of a Morph

There are a lot of different ways how to specify and change the gesture dictionary of a Morph. You can change the dictionary for Morph classes or individual Morph instances. This section tells you how.


2.1.1) Name and exported name of a gesture dictionary

Every gesture dictionary has two kinds of names: An (internal) name for descriptive reasons and an exported name in order to access the dictionary from within a Morph.

- (Internal) name: This name is just a descriptive name of the dictionary. It has no effect on how and wheter the dictionary is used by certain Morphs. As an example, my personal dictionary with all the 26 letters could be named 'Letters Nathanael'. My dictionary with numbers could be called 'Numbers Nathanael' and my dictionary with letters, numbers, brackets and other special characters for programming could be called 'Programming Nathanael'.

- Exported name: This name is the key to associate dictionaries and Morphs. If a dictionary has an exported name XYZ it means that the dictionary can be accessed from any Morph by using this name. Thus, whenever you want a Morph class or instance to use a certain gesture dictionary, you can refer to it by using it's exported name. As an example, you can specify that all the text related Morphs (TextMorph, PluggableTextMorph, etc.) should use the dictionary that is exported under the name 'Text'.

This principle of (internal) name and exported name may sound a bit complicated, but it's very useful because it makes it possible to refer to dictionaries in an abstract (indirect) way. Let's for example assume that Peter and Nathanael are sometimes working with the same image and want to use different gesture dictionaries for editing text. (Let's further assume that all the text related Morphs refer to the dictionry that is exported with the name 'Text'). Now, the only thing to do in order to switch from Nathanael to Peter is to open the dictionary browser and to export the dictionary 'Text Peter' (instead of maybe 'Programming Nathanael') under the name 'Text'. In a similar way it's possible to use 'Letters Nathanael' instead of 'Programming Nathanael' if Nathanael only wants to use only letters for the moment.


2.1.2) Default gesture dictionary of a Morph subclass

A very easy way to change the gesture dictionary of a Morph subclass is to override the method gestureDictionaryOrName and return the *exported* name of the gesture dictionary that should be used (return the name as a Symbol!).
If this method is not overridden, the following happens per default: Every Morph subclass tries to use a dictionary with an exported name that is equal to the class name. If there is no such dictionary available, it tries to find a dictionary with an exported name that is equal to a superclass name, and uses this one. If there is also no such dictionary available, the Morph subclass doesn't handle gestures. An exception to this rule about the default gesture dictionary of Morphs are the classes that support text editing (TextMorph, PluggableTextMorph, etc): All these classes use per default the gesture dictionary named 'Text'.
Example: To use a dictionary for all the instances of the class EllipseMorph, set the exported name of this dictionary to 'EllipseMorph'. Or, if every Morph subclass (that doesn't define a special gesture dictionary) should use a certain dictionary, just export it with the name 'Morph'.


2.1.3) Gesture dictionary of a Morph instance

It's also possible to change the gesture dictionary of an individual instance of a Morph subclass. In the red halo menu of every morph, there is an item called 'change gesture dictionary'. It can be used to assign an exported gesture dictionary to the individual Morph instance or to create a new dictionary for this instance. Every Morph instance that has an associated gesture dictionary provides also other options in the red halo menu:

- Inspect gesture dictionary: Inspect/edit the associated gesture dictionary
- Make own copy of gesture dictionary: Makes a copy of the gesture dictionary that is then
only assigned to this individual Morph. (This dictionary doesn't even have an exported
name and it is referenced directly by the Morph instance.) Like this, it is possible to
modify the gesture dictionary of the individual instance without affecting other Morphs.
- Make own sub-dictionary: Similar to the previous item, but instead of copying the associated
dictionary, an empty dictionary that inherits from the previously assigned dictionary
is created and assigned to this Morph instance.

NOTE: As for all the other events, EventHandler can be used to add special behavior to certain Morph instances. (Whenever you assign a gesture dictionary to a Morph instance by using the red halo menu, an EventHandler gets automatically installed.) As soon as there is an EventHandler assigned to a Morph instance, the EventHandler determines the Morph's behavior as far as gesture handling is concerned. This is the reason why Morphs with an associated EventHandler often don't respond to gesture events per default. (They associated event handler just doesn't specify a gesture dictionary.)

For more information see section 4.


2.2) Inspecting, editing and browsing dictionaries

There are several possibilities to open the tool to inspect/edit dictionaries. You can use the help menu of the World and select the item 'genie gesture dictionaries'. This opens a tool with all the named dictionaries in the system. Every dictionary shows its contents using 5 tabs. Use the 'Basic' tab to modify basic properties of the dictionary. (Have a look at the balloon help text to get more information about the available preferences and options.)
The menu of the dictionaries browser (click button labeled 'M') allows you to save or load dictionaries, to browse parent dictionaries, etc. (In addition, dictionaries and display properties can also be loaded from within the Squeak file list. Just select the file name and choos 'load' from the context menu.)
Notes:
- Every dictionary has its own properties (controlling lookup, etc.) and it's possible to inherit from dictionaries that have completely different properties. Further, the inheritance mechanism of Genie automatically resolves cycles in the parent hierarchy.
- All the tools to inspect/edit/browse are built in according to MVC philosophy. So, it's no problem to open several browsers/inspectors on the same dictionary.


2.2.1) Browsing a dictionary

To browse the contents of the dictionary, choose either 'Browse' from the menu (button labeled 'M') or select the 'Browse' tab.
A browser consists of two nested panes. The outer pane can be used to browse through all the characters definied in the dictionary. The current character is shown in the top left corner of the pane. For every character, the inner pane can be used to browse all the associated gestures. (There is often only one gesture associated to one character.) The leftmost gesture in the inner pane is the gesture that is associated to the character, while the gestures to the right are the most similar gestures in the dictionary. The text above the graphics says what characters are most similar and what's the distance to this characters (the distance is normalized so that 100 is the maximum). Use the menu button (labeled 'M') of the outer pane to change properties (appearance, etc.) of the browser.


2.2.2) Adding new gestures for a character

To add a new gesture for an already existing character, click 'New' in the inner pane. This replaces the inner pane by a pane with a purple background. Enter a new gesture by drawing a stroke starting at any point in the purple area. When finished, the new stroke is displayed in the left part of the purple pane. The most similar gestures of the dictionary are shown at the right side and the text at the top informs about the distances to these gestures. (Note: The maximum distance is 100.)
Basically it's enough to enter each gesture only once. Thus, you can now hit 'New' to assign the new gesture to the character. However, if a stroke is entered only once, there is a certain probability that it is accidentally not completely entered as it should be. Therefore, Genie allows the user to enter the stroke for a gesture as many times as he wants to. Just start a new stroke in the purple area to enter a new variant. Note the number in the top left corner that indicates how many strokes were entered for the gesture. If there is more than one stroke entered for a new gesture, the avarage of all these strokes is finally associated to the character. The menu (button labeled 'M' in the purple pane) can be used to remove the last entered stroke, or to switch between showing the last entered stroke or the avarage stroke. Further, the menu can be used to assign a hotspot to the stroke. Hit 'Reset' to remove all the previously entered strokes for the new gesture.

Important: Don't mix up the possibility to enter the stroke for a single gesture more than once with the possibility to assign more than one gesture to a single character! In the first case, the user enters the stroke for one and the same gesture more than once to reduce noise that happens per accident. (Once the 'New' button is hit, only the avarage gesture is assigned to the character). In the second case the user assigns more than one (possibly completely different) gestures to one single character).
IMPORTANT: If the system fails to recognize a certain character several times, it often helps to enter the gesture again. (Just enter the gesture for the character again. You don't have to delete the gesture(s) that are already assigned to the character). Even if two gestures that are assigned to a character look nearly the same, there can be relevant differences in the way they have been captured.


2.2.3) Adding new characters or editing characters

Use the 'New' button in the browser's outer pane to add a new character to the dictionary. You can now enter the new character in the text pane. Use the context menu (yellow mouse button) to choose from a list of characters and predefined commands (This is especially useful if you don't have a keyboard). Note that most of the commands have a descriptive balloon help associated).
After you entered the character, the purple pane to enter a new gesture gets automatically opened.

Click on the character representation shown in the top left corner of the outer pane to inspect or edit a character. Similar to the add character text pane, there is a context menu available that presents some predefined characters and their meaning. Note that most of the commands also have an associated balloon help.


2.2.4) Dictionary inheritance

Gesture dictionaries can inherit from one or more parent dictionaries. Whenever a gesture is looked up in a dictionary with parents, the gesture is compared to all the gestures in all the dictionaries that are accessible using the inheritance hierarchy. (Cycles in the parent hierarchies are automatically resolved).
Using the concept of dictionary inheritance a user should never have to define a gesture for a certain character in more than one dictionary. For example, a user can define all the gestures for basic mouse and command key support in a dictionary 'Mouse'. Then he can inherit from this dictionary for all the other dictionaries that should have basic mouse and keyboard support. Further he can define a dictionary 'BasicText' containing all the basic characters. In another dictionary he can enter special programming macros and when he makes this dictionary inheriting from 'BasicText', there is a dictionary containing both text characters and programming macros. Note that dictionaries in an inheritance hierarchy can still have different parameters (like speed, size relevance, etc).
Dictionary inheritance is really nice but there is one disadvantage: It takes more time to look up gestures in nested dictionaries than otherwise. So, if performance is an important issue (slow systems, etc), too much inheritance should be avoided.


2.3) The different character types

Originally, characters were real characters (one single ascii character), but now, a character is much more general.
Basically, there are 3 different types of characters:

- Sequence of characters (keystrokes):
This category consists of any character sequence not starting with
a #. Sequences starting with a # can be entered by enclosing them into ''.
(e.g. '#keystrokes' instead of #keystrokes)

- Command:
A command is basically a symbol.
Commands are used to tell the target Morph to do certain actions.
There are a lot of predefinied commands available (#inspectLastGesture, #switchCase,
#inspectDictionaries, etc.). New commands can be added.
Whenever you have to enter/edit a character, there is a context menu available that
shows the predfinied commands and more... (click the yellow (2) mouse button to open the
context menu)

- Code:
A Genie character can contain any kind of Squeak code.
The format to enter a code character is the following: #Header#Code, whereas
Code is general Squeak code and Header is a descriptive string that can be omitted.
(Example: '#beep#Smalltalk beep' or just: '##Smalltalk beep').
When the gesture assigned to the code character is recognized, an instance of CRGesture
is sent to the target Morph. Then the entered code gets executed with the
CRGesture as the receiver. Every CRGesture provides a lot of informations that can be
used to take specifi actions. E.g., you can use the message #target to get the target Morph
or you can use #coordinates to get the coordinates of the captured gestures, etc.
Just have a look at the class CRGesture and the methods in the protocol 'genie-dispatching'
of the class Morph and TextMorph.



3) Display properties

One of the imortant features of Genie is the platform independence of the gesture dictionaries. This means that all the gsture dictionaries can be created, changed and used on platforms with completely different properties (PC, Laptop, PDA, ...).
If a user wants to use Genie on a specific platform, he has to tell the Genie system about the display properties of the platform. There can be many different display properties in a Squeak image, but only one of them is active at a time. To create, delete, inspect, edit or activate/deactivate display properties, select the item 'Genie display properties' from the World's help menu. See the balloon help assigned with the different options for more information.



4) Integration into Morphic

This section covers some important issues about how Genie is integrated into Morphic. This may not be very useful for a simple user, but it provides some informations a programmer can use to easily add specific gesture handling behavior to certain Morphs.


4.1) Genie and events: The details

Here is a detailed description what happens if Genie is enabled for a certain hand and a red button mouse down event occurs:

1) The first step happens exactly in the same way as if Genie would be disabled. First, the method HandMorph>>handleEvent: gets called with the event as an argument. Here, the listeners get notified and then, the target morph is determined. (This process depends on the state of the hand, the involved Morphs and the used MorphicEventDispatcher. Several methods in HandMorph, Morph and MorphicEventDispatcher are involved here). Once the target Morph is determined, the message #handleMouseDown: with the event as an argument is sent to this Morph.

2) In the method handleMouseDown, the target Morph decides wheter it wants to handle the mouse down event traditionally or wheter it wants to start capturing a gesture.
The basic implementation Morph>>handleMouseDown: does that by calling the method Morph>>allowsGestureStart:. This method checks a) wheter the Morph itself wants to handle gestures (method Morph>>handlesGestureStart:) and b) wheter the current state of Genie and the hand allows the start of a gesture (HandMorph>>allowsGestureStart:). If a) and b) are true, the method Morph>>gestureStart: gets called instead of executing the traditional mouse handling code. (The mouse down event gets passed as an argument to all the mentioned methods).

3) The method Morph>>gestureStart: simply invokes the method HandMorph>>gestureStart: and this just forwards the call to the instance variable genieGestureProcessor. (Note: All the Genie related methods in HandMorph (see protocol 'genie') are simply forwarding the calls to the instance variable genieGestureProcessor.)
In CRGestureProcessor>>gestureStart: the gesture processor gets prepared to capture a new gesture (create a CRRecognizer instance, etc.). Besides others, the mouseFocus of the current hand is set to the CRGestureProcessor instance in order to get all the mouse events.

4) The CRGestureProcessor catches all the mouse move events and adds their positions to the CRRecognizer instance that is in charge of recognizing the new feature. This is done until the next mouse up event occurs. Then the CRGestureProcessor requests the newly captured feature from the CRRecognizer and then it builds an instance of CRGesture that contains all the information related to the gesture (feature, coordinates of the feature, gesture dictionary, lookup result, target morph, etc.).

5) Now, the CRGestureProcessor askes the target morph wheter it allows preprocessing of gestures (Morph>>allowsGesturePreprocessing). If yes, it performs some preprocessing like updating of the caps lock state if necessary. Then it sends the message #gesture: to the target Morph and passes the CRGesture instance as an argument.

6) From now on it's up to the Morph how to dispatch the gesture. The default behavior implemented in Morph works basically as follows:
If the Morph has an associated gesture handler, it lets it handle the gesture. Otherwise it calls the method Morph>>handleGesture: with the CRGesture as an argument. (See next section)


4.2) How a Morph can influence gesture handling

Every Morph has several possibilities to influence gesture handling. Note that the involved methods are basically organized in according to the template method design pattern. So, often it's enough to only ovverride a single hook method.

a) What events are handled?
Every Morph can decide whether it wants to handle gesture or whether it just wants to get traditional mouse events.
Related Morph methods: handlesGestureStart:, disableGestures

b) What is the CRGestureProcessor allowed to do?
The CRGestureProcessor stored in the instance variable genieGestureProcessor of HandMorph is in charge of capturing the gestures for a Morph. The Morph can define what's the CRGestureProcessor allowed to do.
Related Morph methods: allowsGestureEscape, allowsGesturePreprocessing

c) What gesture dictionary is used?
Every Morph can determine which gesture dictionary gets used.
Related Morph methods: defaultGestureDictionaryOrName, gestureDictionaryOrName, gestureDictionaryOrName:, gestureDictionary

d) How are gesture events dispatched?
After a gesture is captured, the HandMorph creates a CRGesture instance containing all the related information and sends it as an argument to the method gesture: of the target morph.
Now, the morph can decide how to dispatch the event.
Related Morph methods: gesture:, handleGesture: and all the other methods in the protocol 'genie-dispatching'

As all the other events, gesture events can be handled by an EventHandler that is associated to a certain Morph instance. See Morph>>onGestureUse:send:to: and the gesture related methods in EventHandler.


4.3) Information available in a CRGesture

Instances of CRGesture represent a gesture that was captured and looked up by a certain dictionary. Once a gesture is entirely captured, the CRGestureProcessor builds such an instance and sends it to the method gesture: of the target Morph. The target morph can use all the information provided by this class to take the appropriate action. Besides the basic information about the gesture, CRGesture also contains the coordinates and the hotspot of the captured gesture.
The methods in the 'genie-dispatching' protocol of the classes Morph and TextMorph are a good examples of how to use the information provided by CRGesture.



5) Examples and tips


5.1) Available exmaple dictionaries

The following example dictionaries are available at http://www.iam.unibe.ch/~schaerli/genie2/.
Please note that the gestures in these dictionaries are quite specific to my handwriting and that they may not be very useful for other users. However, they should be a good example to show how you can use and configure Genie.

- Basic [No parents]
Includes the basic mouse, halo and command/modifier key operations. Further it
includes Genie specific operations (inspect last gesture, inspect dictionaries, etc.).
Note: This operations definied here are very useful. So, consider building a similar
dictionary as a base for your own gesture dictionaries. In particular, a basic gesture
dictionary should associate the 'point gesture' (just a click without moving the mouse)
to the command 'redClick' which simulates a red mouse click.

- BasicText [Parents: 'Basic']
Includes letters, numbers and some special keys (return, backspace, tab, ...)
Further there are some special gestures. (E.g., you can change the alignment of text
to {left, center, right} by drawing a long vertical line in the {left, center, right} of the pane).

- SpecialCharacters [Parents: 'Basic']
Special characters and special keys available on a general keyboard.

- ExtendedText [Parents: 'SpecialCharacters' 'BasicText']
Includes all letters, numbers, special characters and special keys available on a general
keyboard.

- BasicProgramming [Parents: 'ExtendedText']
Some programming gestures (ifTrue: [], ifFalse: [], etc).

- MorphColors [No parents]
Allows to change the color of the target morph.

- BasicWorld [Parents: 'Basic' 'MorphColors']
Includes some gestures to open new Morphs in the World.
(Workspace, Browser, EllispeMorph, StarMorph, TextMorph, etc). It also has a gesture to quit
Squeak.
Note: The gesturs in this dictionary should be size and shape independent. Therefore,
some parameters in the 'Basic' resp. 'Advanced' section have been changed!

- BasicMorph [Parents: 'Basic' 'MorphColors']
Includes gestures to delete a Morph.

- GraffitiLetters [Parents: 'Basic'].
Letters of the Palm Graffiti style.


To get a nice demo environment you can do the following"

- Export 'BasicText' or ('ExtendedText') as 'Text'.
- Export 'BasicWorld' as 'PasteUpMorph'.
- Export 'BasicMorph' as 'BasicMorph'.
Change the method defaultGestureDictionaryOrName of the classes EllipseMorph and
StarMorph to:
defaultGestureDictionaryOrName
^ #BasicMorph.

Now you can open new Workspaces, Browsers, etc. by drawing directly into the World.
Also StarMorph and EllipseMorph can be opened like this. Further StarMorph and EllipseMorph support gesture handling. For example, you can delete them using a gesture.
If you want to enable this basic gesture handling for all the Morphs that have no special
behavior definied (for example in an Eventhandler), just export 'BasicMorph' as 'Morph'.

These examples give a little taste of Genie's possibilities. However, these are very basic examples and there is much more possible. Just adjust the dictionaries and write integration code for the individual Morphs...


5.2) Tips on building your own dictionaries

- Use dictionary inheritance to achieve modular dictionaries. Like that, you don't have to define gestures in more than one dictionary. But note that the lookup in dictionary hierarchies takes a bit more time. So, if speed is an important issue you should avoid too big inheritance hierarchies.

- If the system fails to recognize a certain character several times, it often helps to enter the gesture for this character again. (Just enter the gesture for the character again. You don't have to delete the gesture(s) that are already assigned to the character). Even if two gestures that are assigned to a character look nearly the same, there can be relevant differences in the way they have been captured.

- It's usually a good thing to associate the 'point gesture' (just a click without moving the mouse) to the command 'redClick' which simulates a red mouse click. Like that, you can use mouse clicks as if Genie was disabled. Have a look at the dictionary 'Basic' for other important operations (mouse, halos, etc.).

- There are many predefined commands related to mouse, keyboard, halos and often used Genie operations. So, you can define gestures that simluate mouse actions with any button, toggle special keys (caps lock, shift, command key, control key, etc.), bring up halos, inspect the last gesture, open the active gesture dictionary, etc. The context menu of the 'enter a new character' pane shows you the predefined commands. In addition, you can have a look at the example dictionary 'Basic' to see how these commands can be used.

- Don't forget that you can associate any Squeak code to a Genie gesture. Note that this code is always executed inside the CRGesture object that is created when you enter the gesture. Thus, you can access all the available CRGesture methods by just using 'self ...'. (E.g. 'self target' refers to the target morph, 'self coordinates' to the global coordinates of the gesture, 'self position' to the hotspot of the gesture', etc.). Have a look at the dictionary 'WorldExample' for some concrete examples.

- Note that Genie can be focused on a specific Morph. This means that all the Genie input is redirected to this Morph. This mode is indicated by a mouse cursor with the shape of a dot (instead of an arrow). See section 1.3.2 for details.

- The available example dictionaries may not be very general as far as my handwriting is concerned ;). But even if you define your own dictionaries they may give you some ideas of how to use Genie's features.



6) Version history

02/25/2001: Version 1.0

08/08/2001: Version 2.01
- More than 10 times faster lookup
- New VM plugin and optimized image level code
- Execute 'CRRecognizer checkPluginVersion' and watch the Transcript output in order
to check availability/compatibility of the VM plugin.
- Faster graphical feedback (especially imporant for PDAs)
- New dictionary format
- Old dictionaries are automatically converted to the new format. This convertion includes
modification of the parameters 'speed' and 'curvature relevance'. This is necessary to retain
the behavior of the dictionaries with the optimized algorithms.
- The new format is backwards compatible. This means that new dictionaries can still be used
with Genie v1.0. However, the parameters 'speed' and 'curveture relevance' are interpreted
slightly different, which may lead to slower recognition or inaccuracies in Genie v1.0.
- Improved UI (Improved morphs to inspect/modify gesture dictionaries resp. display properties)



7) The Genie plugin

The Genie VM plugin (written by Stephan Rudlof) makes Genie about 10 to 20 times faster. Especially on slower machines (PDA, etc.), it is really worth using it and it makes Genie much more fun!!


7.1) Testing the plugin

To see whether the plugin is available and compatible to the image level code execute 'CRRecognizer checkPluginVersion' and watch the Transcript output.

To compare the performance of the primitive to the original performance do the following:
1) Load the gesture dictionary 'http://www.iam.unibe.ch/~schaerli/genie2/GenieTest.ggd' in your image.
2) Execute 'CRStrokeFeature testPrimitivePerformance' and watch the transcript output.
3) Execute 'CRStrokeFeature testOriginalPerformance' and watch the transcript output.
4) Compare the output of steps 2) and 3). The primitive should be about 50 times faster.

If you are not sure whether the primitive works correctly, do the following:
1) Load the gesture dictionary 'http://www.iam.unibe.ch/~schaerli/genie2/GenieTest.ggd' in your image.
2) Execute 'CRStrokeFeature testPrimitive' and watch the transcript output.


7.2) Compiling and installing the plugin

The Genie plugin may already be part of your VM (execute 'CRRecognizer checkPluginVersion' to check). If not, you can compile and install it yourself, by doing the following:

1) Generate the C-file by executing 'GeniePlugin translate'. This writes the file 'GeniePlugin.c' into the subdiretory 'GeniePlugin'.

2) Compile the plugin and make the resulting dynamic library available for the VM. (This step is platform dependent. On Windows, you have to get the VM sources, build the plugin (enter 'build GeniePlugin.dll') and copy the resulting file GeniePlugin.dll into the Squeak VM folder).



8) Acknowledgement

Many thanks to Stephan Rudlof for writing the Genie Plugin. Thanks also to Kevin Fisher for his valuable feedback about using Genie on the iPaq. Finally, thanks to SqC for creating Squeak and giving me the opportunity to work on Genie.



I hope you enjoy!

Nathanael