Dynabook, user interface point of view

In our previous writings, we took the point of view of said innovative teaching approaches. It differs from the children's point of view historically taken by A. Kay. This provided us a set of general features, curriculum neutral, we may want for a teacher centered Dynabook; we summarize it in our "Dynabook and learning models, final words" writing.

Three features came out in several learning models: text processor, question tool and tools to collect and represent data. The first one is linked to the human concept of writing and reading, the last one relates to how to represent information: statically or dynamically. They are related to the representation of knowledge and its transport both in space and time. Appearing as key features in a learning system should not be a surprise, though it may seem to be obvious, it is important not to be overlooked.

To enrich our thinking, we want to change our point of view and look from the curriculum and user interface perspectives. Based on our previous thinking, this will be examined with text and representation of knowledge as the two backbones. Later, we will discuss the third feature, the question tool.

In the following paragraphs, we discuss the user interface point of view.

From the user interface point of view, we want to think about how text and knowledge representation should be linked together. How a teacher will have access to tools to represent data and information: statically or dynamically? How such tools could be malleable? How a teacher could discover, learn and adapt from a corpus of such example tools? And how text will be part of it?

From the curriculum perspective, we want to think about the type of visualization a teacher teaching one curriculum may want to integrate this into her dynamic media. How such visualization will be malleable enough to fit the specific teacher need? And how such visualization could be shared with another teacher?

User interface perspective

Text is an important aspect in human culture. It is a way to transport information and knowledge in space and time. It is a common cultural ground to humanity, part of the human heritage since the beginning of writing. It can be enriched and illustrated with visualization aids, since visual expression likely preceded writing.

Leonardo Da Vinci excelled in his anatomy illustrations[1]: the preparation, the precision and the accuracy of his productions make him the inventor of modern anatomy. In his production, text was liberally flowing around the illustration, almost turning the text as the illustration of his beautiful anatomy drawings.

Dynamic media combines both textual and non-textual information, the textual information flows around the non-textual information. The non-textual information can be anything such as a static image, chart or any arbitrary interactive content.

The former is produced directly from the keyboard or imported from another source as a file or pasted from a clipboard. The latter is more diverse in its form: a beautiful anatomy sketch could be imported from a static or an animated image file, a dynamic geometry sketch imported from Dr. Geo, etc.

Could such a non-textual document be produced directly within a dynamic media content editor? After all, the dynamic media textual information is easily produced in-situ with the keyboard. Obviously producing Leonardo's beautiful sketches requires a dedicated set up, animating several sketches require another tool. Constructing a Euclidean geometry sketch also requires a dedicated tool such as Dr. Geo.

To produce such non-textual documents, the user can operate with two categories of dedicated tools: a tool with a graphic user interface or a tool with a programming interface by the mean of a domain specific language (DSL). Both tools are specific to the targeted domain (drawing, geometry sketch, timeline tool, etc.).

As an example, Dr. Geo offers both: an iconic user interface and a source code textual representation to produce interactive sketches.

The iconic user interface comes with menus and buttons to activate commands to construct step by step the visual content. It is user manipulated by means of the mouse cursor. It usually takes a good amount of space in the dynamic media view.

In the programming user interface, the dynamic content is described by the means of source code instructions to be executed by the underlying Smalltalk system; then the executed code returns interactive visual content. As so, it is not only textual information but also programming instructions using both the Smalltalk programming language and a dedicated domain specific language to describe the interactive content to produce.

The text below uses both the Smalltalk and the Dr. Geo Euclidean geometry DSL instructions to produce a circumscribed circle to a triangle:

| sketch center |
sketch := DrGeoCanvas new.
sketch polygon: {-2@0  .  2@0 . 1@3}.
center := sketch 
  intersectionOf: (sketch perpendicularBisector: -2@0 to: 2@0)
  and: (sketch perpendicularBisector: 2@0 to: 1@3).
sketch circleCenter: center to: 2@0

The resulting interactive content can be mouse manipulated by the user, i.e. drag the triangle or the lines. Bellow a view of the interactive content exported as a static image:

The graphic user interface tool is required to learn the iconic interface while the programming interface tool is required to learn both the Smalltalk and DSL programming instructions.

The graphic user interface learning effort is not transferable from one tool to another while in the programming interface the Smalltalk instructions learning effort is 100% transferable from one tool to another one. The learning effort in the domain specific language (DSL) part is partially transferable, several DSLs may semantically overlap in the way their specific domain is expressed.

The graphic user interface is user discoverable: menus and buttons tool tips give textual information on how to use the graphic commands. The programming interface is not discoverable, it requires a dedicated corpus of examples, how-to and documentation to discover what it can offer. An assistant agent may provide valuable guidance, though.

With its 100% textual representation, the programming interface blends nicely with the other text content. Moreover, it makes the user interface for the whole dynamic media simpler but requires more user assistance to elaborate the expected visual content.

Each interface offers pros and cons: graphical user interface seems easier to start with while the programming interface may offer more rewards in the long term. It is rooted to more fundamental computer science knowledge such as programming and data representation.

The learning code trend

For 10 years, there has been an important trend on children learning code. Dedicated tools for visual programming were developed for kids. Etoys development started around 1996, it was developed as a tile based programming interface to the Squeak Smalltalk programming language. Etoys aim is to assist children on manipulating ideas and concepts with multimedia contents. The first version of Scratch, the popular visual programming environment, was released in 2007 and developed with Squeak Smalltalk. When comparing to Etoys, Scratch is focused on multimedia creation, its narrow goal likely helps for better success.

Then what about the programming for teachers? Looking again at the dynamic media content, a programming interface, to design interactive contents, makes perfect sense: teachers comfortable with programming techniques are more capable of helping students on programming. It requires the teacher to learn basic programming, though. Nevertheless, it is a kind of alphabetization effort, with life-long benefits to acquire the deep knowledge on computing which is transferable to related domains. The reward of this effort is to be able to produce dynamic media content.

Of course, dedicated, well designed dynamic media tools and a large corpus of DSL implementations, documentation and assistants are necessary to make the learning effort accessible to teachers.

We have discussed the importance of programming from a teacher point of view, so what kind of environment should be proposed to a teacher?

With this environment a teacher designs her pedagogical documents as dynamic media. Such documents are used by the teacher in the classroom or/and shared on the students own Dynabook.

It is composed of both textual and dynamic contents, designed with a programming interface. In such a document, text flows around dynamic content inserted in the text from templates. These templates are accessible from an indexed corpus. The teacher browses the corpus templates through its categories or she searches it with keyword tags.

Once the teacher finds the desired template, she drags it into her text. The template is instantiated as a frame view in the text. This view is the result of the execution of the template code. From the template frame box, the teacher can also edit the source code and recompile it to get the precise intended interactive content. The edit and recompile cycle can be repeated as many times as necessary. Optionally, she saves her edited interactive content as a new template for later reuse.

The switch between the source code and its resulting interactive content is sketched as bellow:

More precisely, the templates come in the form of pure Smalltalk code, with a preamble declaring, among other things, the Smalltalk DSL package (a collection of classes) it depends on.

To summarize, in term of tools, the dynamic media editor comes with:

1. A text editor and viewer tool, optionally with rich text format capabilities. In this editor, templates are dropped to produce a frame view where the template logic is rendered.
2. Tools to browse the templates by categories or to search with keywords.
3. A tool to edit, to compile and to execute the template code. The Smalltalk developer tools such as the debugger, the inspector and the class browser will be naturally accessible from there.
4. Template documentation: each template will come with dedicated documentation explaining its specific DSL.
5. Smalltalk documentation: programmers starting guide, technical class memo searchable by indexes or keyword.
6. Assistant agent to help the teacher designing its interactive content with template.

 

Any opinion on the topic? If so leave a comment for further reflection.

Thanks to Michael Davis for his editing.

[1] L. Da Vinci, Anatomy sketches, 1487-1516