June 10 2005

Reboot7 conference – 300 people estimated – in Copenhagen
Video of the Q&A is available here requires Quicktime 7 or the latest Videolan

Ross Mayfield is CEO of SocialText – integrating wikis and blogs

Thomas Madsen-Mygdal

An experience that changed Alan Kay’s life.

Hypertext, video conferencing, mouse – 40 years later

Context of demo?

DE – all functions were already part of NLS system – our objective was to communicate this to people by demonstration – so it’s the first time we ever tried that, the details to put together for this took a lot of work – bill English – hardware, Jeff Rulifson – software.
Stomach tightening – so many opps for things crashing – huge sense of relief when it finished. Important things we wanted to show – interesting social cultural phenomenon – it seemed to have no impact over following months and years. It was one of the lessons about paradigms what people really think – how paradigms persist. It was live – system evolved over following years, I still use it every day. It was all real, not just for the purpose of the demo. Except for my face up on the screen.

See the Video (concatenated version on Google Video)

(photo by Ton Zijlstra)

If computer was live for you all day and instantly responsive, how much value could you derive from that. This is what we’ve been working on.

We are trying to describe and present the nature of this program, but fortunately the products lend themselves to an interesting way to present it, we’re going to show you not tell you.,

I’m not going to tell you, but just run thru it.

In my office we have a terminal, and there are 12 others. This is how I look at a completely blank piece of paper – this is the way I start a lot of projects – so I’m putting in an entity called a statement, if I make mistakes, I back up.

He put sound effects to help people hear what was going on – microphone at the registers.

Can point and copy – get material and put it on blank piece of paper. Now output to a file – need a name, say sample file, and please out put it. Then it comes back automatically with origin statement – with date, time – and I can ask it for status – and it will tell me. In those days, no one knew what I was doing when they saw my fingers. I can copy groups of statements – I can look at that and collapse that – first line of each statement – we’ll use it over and over – View control – no matter where file is looking, we can use any one of a large number of parameters to view.

To make them look prettier – I can neaten them up. Lined up – I can see an invisible string and if I want to replaced that string with one character, it will do it

So I have files 1,2,3,4,5,6 and named them that way. I can ask them to number themselves – and give a list numbering, I can output them. So we’ve seen ways to work fast with entities in a statement, create a file, I can delete that file or mess it up considerably, - replace the whole thing – so I’ll load that file and it’ll come back in the way I last saved it.

My wife called and said do a little shopping on the way home. I got my system organized and made a shopping list – I can see that yes those are numbered statements – I can point to 10 or 23. so she said call me back later. So I’ll re-organize and put the fruits together so aspirin and paper towels together.

So now I’ll organize it by saying Produce and I’ll categorize things – let me move a statement. Move carrots and bananas and can move a whole group, oranges and apples. I began categorizing and these items fit under as a subset – I can introduce a new thing – a skinless banana and part of view control, is also whether we can show different levels- 2 levels or 1 level deep. Let’s put Produce at the top and open one level below it and only show it whats below it;

Moving around with jump to identities – this is one of the basic features – then I call my wife to get the rest of the list – I end up with a whole structure – shoe store, market – cereals – bread, noodles. If I want to open up everythinmg, there are many ways in which I can look at that.

One interesting thing is that when the numbers are on instead of saying jump to identity, I can say jump to a name and go to 2A 4 and it will do it. I can jump to a location number – or if I wish, I can add it, and point to it and it will go to it. Carrots – I give it the name 2A4 and say go to 2A4 – its very easy to make cross reference – where I give it a label or name – call it alpha – like we do in programming. I can say jump to Name. I would now like to re-conform my whole picture. So I can give names and say in here a reference like that, or say jump to name, point to that, and go to this thing I named alpha.

So what else can we do? There’s this file, its structured, I can walk down the hierarchy levels, and I can also do more – I drew a picture of my route to work. And its my plan for going home – I can go to library – point to that and see what I’m supposed to pick up at the drug store. Market – all that produce.

So structuring material hierachically, being able to move very well, when we get a hierarchy – I can interchange produce and canned materials – and it all gets renumbered. Go to specified points and modifying structure – I can see my repertoire of different entitites that I can say I want to do operations on, and basic structure I can study and do things with.

Tool we have – a lot of details I’ve left out.

We use this tool to do our daily work, and our system has been working for 6 months, and this is our 4th computer, so we’vce learned about what we want, and how we use it and put it to work, how to organize ourselves, what kinds of things you do.

(photo by Ton Zijlstra)

This is just the outline – file, statement, introduction, overall about program, and a link is something that will go between files. Where in that file is what you want. Go to the other file and restore – brings in things from different files with different format structures and numbering. For instance, I’d like to restore the viewing of file 7 in a simple large view – usually we work with a view which is enlarged, we see less, but larger and clearer.

I’d like to now jump and learn about the program – it looks different – this other file, I’ve prepared a chain of views and using some of the tools, I go from one scene to another.
The chains represent the way we work. I’m going to freeze stmt named A and create a link to jump to statement A and after that abbreviated way to show statement A.

This statement I’ve asked to be frozen – as the viewing part of my screen below will jump. A hidden link will take me to the next one. And jump on another hidden link – set viewing parameters so wouldn’t show link, and computer obeys link when it comes to it.

This program took 17 people sponsored by NASA, ARPA RADC – airforce…

Its been a goal oriented pursuit and from 1950 which has had specialized history. During these years, there’s been only 1 of us….

Jump back to file, now link to another to say, HAIRC is pursuing these goals….

What does this involve? Better use of human capabilities. Besides improving effectiveness – the approach should result in a system oriented discipline.

Hidden link goes back to the view with slightly different view parameters – to give a different view, now another hidden link – to approach it evolutionary wise because it is a whole system problem. Need to have a person looking at his effectiveness, not only his computer tools. At any one point, we can’t decide what best is the next step. We need to work out where best to invest resources – we need a research subject group, study and improve them. These tools should improve our ability to produce these tools and create this system oriented discipline.

This tool and pursuing those goals, is our principal tool.

NLS – online system will evolve over many years, an instrument and vehicle to help humans operate within domain of complex information structures.

What do we mean by operate?

Many analytic things we can do, but we want to study and modify – complex structure – although content represents contents – structure represents relationship between concepts of human thought. Inside computer we can represent that quite well – far too complex to study directly. NLS serves as a tool to navigate thru a complex structure, see what you want to see at any given point. These are very important concepts to us, these, togegher with bootstrapping tells us where to start, to organize ourselves to write plans, programs, user guides, documentations, reports and proposals. Finished that version and keep track of where we are.

Control dialogue and meta language

Control devices, we’ll use overhead camera shot – 3 devices – pointing device , the mouse, std keybd and special keyset.

Don Andrews in Menlo park – can see screen and the mouse- tracking spot moves.

Chordkeyset – tracking spot changes into an arrow or a spot.

We had an echo register that would give the last characters you just struck.

Some characters will show different view specs – I can add single strokes that change those view parameters. Each means something to me. I can add very quick abbreviation for changing the view. I can go to Produce and one branch only, and see 1 line only.

Ways I move around, use both hands to coordinate – all carefully designed to go together to make the repertoire – dialogue and feedback – each commands and control dialogue
Come together to do design, must be familiar with the repertoire – we’ve developed a special language to talk with extreme precision to talk about control functions – we have languages to program at a high level the control functions

Jeff Rulifson – user features, repertoire of commands and control dialogue – programmers are the ones who find most intense use online – so make the most use of a system like this.

A system programmer put together a file to get around NLS – here’s a picture of the overlap system – each label and its picture names a code file and each is 3-20 text pages long. How to use a file like this, and how code is put together, each label is the name of an overlay which is also the name of the file and the name of a statement in that file.

Picture of statements – a little section of file, with comments on code. Can move to file, and use control meta language – many special languages – find move commands – e.g. MOVE word – the routine has its own name – can shift between overlay Text Edit and the code. Text Edit overlay illustrates a lot – 2 major kinds of code – first is special language we built up – QMW subroutine calls a second subroutine which is a Word Delimiter. This is written in our content analysis language – it looks for the definition of the word, pointers that describe that word. Next subroutine is in another special language – takes pointers left over from edit routine and reconstructs a statement after the edit has been done.

Other portion of file is written in MOL – machine oriented language – a high level language with phrase structure and if statement, also close to machine language – registers, indirect addressing. This has helped us write the fast tight code we need to operate this, and given us a lot of flexibility. MOL phrase structure has meshed with statements in NLS – so by using MOL like this, can move around quickly, zoom in and out – fits with our linking structure – so takes advantage of power of NLS.

We managed to implement these languages to design a compiler compiler –metacompiler written in 3 Meta – syntax of command language for NLS and meanings of commands are in a high level language – program structure – built up by systems programmers; second system is retrieval, 3rd system – bugs reporting system. NLS advantage keeps track of who you are and what you are doing – on each statement, who did last edit, when. Can set up search patterns.

What it looks like and how its put together – Jeff has finished.

Control meta language – dialog protocol someone would use – jump to a file where there’s real code – open it one level down, all the statements names provide the command name and the code next to the name says what the system needs to do in response, when that name is typed in the command line.

Can work at the level user wants to work. We don’t have to look at someone’s English text translation of what the system is supposed to do.

Control techniques and implementation – some of the ways it can be used. Programming itself is one, database, information getting around it, we used hierarchy and using names of places in there – compiler treats them as labels for subroutines –

Application examples – user documentation is straightforward use. So here’s a users guide in a file. Can point to successive terms and click there to go to the glossary of the terms pointed to. Can even have a sketch in the glossary.

So user can find your own way around to find definitions, procedures for commands etc.

Next thing, study and modify papers – its in the file. Top level – see table of contents, can point to chapter and open that. Can have overall commands for the file about how to print it and how to interpret the directives for printing out.

Calendar for planning a project – kept all messages between people working on the project – can leave a message for someone in specific context. Can have a marking which indicates a message for Jeff from Doug. Can have another marking from Doug to the project team. So when you are there, you can see that there’s something about this that is new from Mr XXX .

You can also look for all messages from Doug – that passes the test that the message must have a string. Now turn off that filter. Set up another one, message needs to be for Jeff, and also include a # sign. So what passes the test? You can find exactly what you need.

Content analyzer – uses categories – e.g. roles etc – coordinator, skills – knows NLS hw, scheduling – who’s available when

Duty roster – code terms – Events – need categories of people and use content analyzer to set up find patterns of people who meet the needs for each duty.

Can look at a view which only includes the categories I’m focused on.

There’s markers – place on character and give it a 3 letter name – can execute them –

Pattern matches, markers and freezing things give a great deal of capability that we don’t have today. (2005)

Design documentation – we used same structured stuff for our code and hardware design. Logic equations, can search for patterns, modify, recompile, move back.

Most of the way, this is powerful for an individual for studying, planning, debugging, coding

But also people can collaborate over time by working on joint files, and leave a message and get a response.

But there’s another degree of collaboration – which is very important.

Establish a collaborative mode with a terminal in SRI – Bill Paxton. Go into executive mode and continue with NLS

Link to Bill, need to know his terminal. Bill says ‘13” So Doug wants to show Bill his text.

Now Bill is looking at my text, and can put in a marker which Bill controls. But Doug’s pointer is more powerful. Can have a bug fight.

Audio coupling and set up same display. Later I’ll hand you the chalk.

I’d like to see you while I’m working on it. So I set up my display in a way so that there’s a corner which a camera on his console shows him. Connected audio, video and now lets do some collaboration –

Lets talk about information retrieval – relate back to NLS as an instrument on complex data structures –

Now I want to point to this text – now give the pointer to Bill and he will move the pointer – everytime you change a view, you are doing retrieval.

2 types

Known destination and unknown destination.

2 different problems with different commands to look at these two different things.

Direct – expicit direction

Indirect – implicit direction

First point at your information and move it into view

With jump to identity command, can point to it and explicitly specifying the name of the destination.

With implicit specification – using information stored by system – case 1, info stored in memory of what you’v ebeen doing recently. Can say jump to the view I just had. Or can say jump ahead and move forward.

Jumps referring to structure of file – can say, take me to successor to the statement I’ve at. Or to the head of this branch.

Other retrieval modes –

Direct –
Indirect –

Unknown destination– can describe information sufficiently to know enough to find it in a hierarchy or describe it with keywords.

Suppose we want to make hardcopy – specify how to print. Look at page formatting, page numbering specifications – we’ve worked down the hierarchy to quickly find what we wanted.

By keywords and associateive re-ordering, using terms from a specified vocabulary to specify what you are after – keywords can be selected from a list, can select from the list – named with a naming system – keyword can point to various items in the catalog.

Could have both keywords point to same item in catalog.

When I point to Keyword, can then refer to particular items in catalog

Going back to systems guide file – documentation – usually someone wants to find something. Broken into 2 parts – catalog and list of keywords

Catalog contains links to various items, entries for each routine in the system

List of keywords – also categorized – eg file handling category contains list of keywords from the file handling category.

First name, then descriptive phrase, then a list of items in the catalog that are referred to in the keyword

So now, can select keywords to use to describe what I’m interested in.

Can accept the selections.

Now can give particular keywords greater weight,e.g. 2 where normal weight is 1. Means items referred to with heavier weight will appear earlier.

Now can use these to jump to various places in the file. Go to the name of a utility and jump to that utility, go back to previous view, can now jump to link, and documentation on that utility – and now can go back again. And continue looking at next item and going back to explore, or re-specify the search parameters and look at new keywords.

Keyword is very powerful and warrants a whole paper itself – gives power of keyword descriptor with ability to give weights – can do it using Delicious.com

Combination of techniques, organization, and special languages is where the power lies.

End of demo.

Q&A with Copenhagen

Mathias – you decided to become rich or do something useful?

DE – I was very green country kid grown up in the depression and father died when I was nine and times were tough. I lived in a nice place with a creek, my mother was a very caring soul, I went into engineering because it was interesting. I had training in the Navy as a technician and went to NASA with windtunnels – suddenly I realized after I got engaged, that a day or so later, I should get attention to my job – the strange thing was a featureless long well lit hall going to eternity. So I said I should get a goal for my career, so then in the next hour I decided to maximize the benefit my career could have on mankind. Going after big problems is complex, need a lot of people to do it collectively. But we are not getting collectively smarter – spells disaster – maybe I can contribute to that – to contribute to our collective ability to understand and cope with complex problems. So went to graduate school and got PhD, teaching EE at UC Berkeley, but didn’t find any interest in my purpose for what we should do for the world. So I left university but my ideas were just as crazy to industry. SRI took me in because my friend told me not to tell SRI about your purpose, but tell only about your PhD patents in gas.
It took years. 1951 goal, 1957 – got in SRI – then 1960 before report on Augmenting human intellect

1968
1974 – Arpanet started (69-70)
By 74 had timesharing system working with minimal client-server, to support customers over Arpanet. Had to give up graphics because awn’t bandwidth for that.
Had users to – evolutionary process

Late 80’s – couldn’t keep it alive any more
Still use it every day
Could give demo fo how Augmnet works now, at a distance.

Q: Hans Christen – did you have any idea of how the stuff you were doing would affect society? And have you been impressed or disappointed?

DE – I was thinking of how much capability we could improve – whole orientation for me was how to push on developing capability – intellectual, group, knowledge work etc – then see how to make it easier and easier to exercise that capability. The way we approached it – where new people came in, to use OO – like language with a simple vocabulary with it getting richer for each person. Difficulty in learning – it surprised me in the 1970’s who said we were too complex and hard to learn, so we lost all of our support in 1977. So paradigm resistance – affects how rapidly our culture will adjust to learn. We can do 4-5 times better than what we have now.

Everyone is still wedded to what the printed page does for you. Computer can provide so many conceptual views that we can learn to grok qauickly. They became very valuable to help people get organized. It took me years to not take it personally so badly – I said – there will come a time when the world is ready to develop thihs new capability – evolution-wise – accept the fact that it is a very large scale evolutionary challenge – to affect the way universities and schools work – it is very encouraging to see increasing interest.

Q – what happened immediately after 1968 demo – prevailing paradigms –

DE – audience was standing and applauding madly. We were very excited, and people wanted the movie. I thought that would be a lot of people doing things. But there was almost no change in the world. They were interest first in the cut and paste – to me that was grunting, pidgin English – we had a fluid natural language. The paradigm has yet to shift as far as I can see. I am confident and wishful that it will happen and people will shift and talk about capabilities.

Q – When I think how we were programming in 1968, people here don’t realize how revolutionary what you showed was.

DE – thank you.

Q – considering your research back then, most of it is in the hands of the general public. Are there aspects that have not come to the general public? What’s still missing?

DE – if hypertext was a link to any object to any document, not just a link to a file. We had syntax in the URL for a view specification – so a reference link – you could click and movce your cursor to another window, and see it with the particular view that the author of the link wanted you to see. There is no comparison – that gave you so much more power and flexibility. We took it for granted that you could point to any passage in an email. Command language tho looked complicated, untutored people could say “execute that sequence”. Viewing and Linnking and flexible command language have not been taken up.

Q – What was the sound during the demo?

DE – those sounds beeping were generated experimentally, there was a probe on computer register, so computer action would be echoed in sound, soon, you could recognize those sounds, so you could recognize what was happening inside the computer.

Q – what is your point of view on the Open Source movement? HTML, wikis, blogs?

DE – NLS was renamed when we went commercial – its is called Augment and still runs.
Quite a few of the features are valuable to you. Open to not just open source technology development, but also how culture, practice, social and business activity evolves. Don’t be curtailed on will it sell, but ask what kind of capability will I get? Will my team get? Let’s get the capability first, then we can make it easier to learn. Most of the things you do as a professional is very hard to learn – why are we restricting our evolution of computing?

Society needs to give serious consideration to how it evolves its paradigms

Q – explain your concept of augmentation and bootstrapping

De – it will only take me an hour or so. Augmentation evolved in the late 50’s and early 60’s – impetus came because so many were talking about computers automating what people did. Artifacts that automate – you need to learn how to include them in your daily life – so it’s a co-evolution of the artifacts and the way a person lived and worked and operated. Way you organize your cities and town – more and more you have to learn before you are qualified to learn. These things I really had trouble with – Augmentation goes down to how inventing a spoken language augments your ability to communicate. So people then said, lets find symbols to represent words, then written language, then alphabet, then you can look at a word and don’t have to piece together the spelling – perceptual machinery in your head – its built in that we think we have to learn in codified symbols. A computer can give you many more flexible ways to learn new symbols. Augmentation comes down to language and symbols we use and skills we have in handling artifacts. I have failed in understanding enough about how people’s minds and attitudes are locked in.

When are we going to start evolving? I lose friends by saying how long will it be before books are obsolete. What if we have other ways to portray that gives even more flexibility and capability? A lot of it would be remedied if we consider what kind of capability we want to improve. Don’t restrict it by how hard it is to learn.

Q – I’m intrigued by the idea that the human and tool systems influence each other so the entire system gets more capable over time. Does it mean that new users to the Augment system need time to get used to it?

DE – you can give a new user a simple subset of the verbs and nouns – within an hour a secretary can be doing word processing. Nobody complained once they got started. When you watch someone who is an expert, then people say that’s too hard. None of the people who came to use it complained about it.

Q – is there something we can do to contribute to the Augment system?

De – I would like to engage with you. Its critical that we get different communities of people to engage with us. I have a series of technology steps which will make the transition to give people capabilities in an easy way. I’m extremely suspicious if someone says I have the solution for optimal use of computers. If you are interested, contact us.

Thank you from Copenhagen.

DE – I’ve very pleased to have the opportunity and that we can communicate. Thank you.

(Original Transcript by MeiLin)