Programmer Documentation for the SocialCalc Engine Javascript

Revision 6, 17 July 2007
Originally authored by Dan Bricklin


This document explains the Javascript code that is part of the SocialCalc Engine. It starts with an overview of the routines and data structures, and then demonstrates their use with some sample code. Finally it describes some of the major routines in more detail.

The SocialCalc Engine Javascript consists of two files in the jslib/SocialCalc directory: Common.js and Edit.js. The Common.js file contains routines that are used for the Edit tab, the Format tab, and the Backup command. The Edit.js file has routines specific to the Edit tab.


Overview of Functionality and Variables

Javascript is used by SocialCalc for a variety of purposes. It is used to allow user control of the system with the mouse and keyboard. It is used to modify the display of various blocks of text and controls in response to mouse clicks and key presses, such as displaying help text and implementing the "/More" command tree and the ":Range" commands. It is used to scroll the spreadsheet display. It is used to communicate via the XMLHttpRequest object with the server when making value changes to cells ("Ajax style" interaction). It is used to communicate with the server to get lists of other pages and text for the help display.

The Javascript code grew as the original wikiCalc code went through successive alpha and then beta versions. The initial design made little use of Javascript, relying on frequent trips to the server for complete screen refreshes. The code was written to run on a wide range of browsers, old and new. As the project progressed, it became apparent that a more interactive system was needed and more and more use of Ajax techniques was made, added on top of the old code. The new functionality was coded directly without use of a separate off-the-shelf library. The code reflects this evolution. The first SocialCalc release (1.1.0) introduced the use of an off-the-shelf library, Mootools, but that was only used to add keyboard support for shift-Tab. In future releases further use of this library is planned.


Some Common Variables

The Common.js file includes the declarations of the main common variables. These include the major state variables.

The jsedit variable is true while editing a cell directly. That is, it is true when the characters being typed are to be reflected directly in the cell as well as in the formula bar text box (that text box is in the "valueedit" input tag). This is a "smart" type of input such that arrow keys or mouse clicks on cells do the "right" thing depending upon whether the last character is an operator (for example if the last character is "+" the pointed to cell's coordinates would be automatically entered into the text being typed in) or not (the entry would be considered complete and the spreadsheet updated).

The veedit variable is true while editing directly in the formula bar text box. This state allows normal keyboard control of this editing. For example, the arrow keys move the editing cursor in the text box and editing operations such as Ctrl-C/Ctrl-V/Copy/Paste work, depending upon normal browser operation.

The vetedit variable is true while editing in the multi-line input area (the "valueedittext" textarea tag).

The cmdedit variable keeps track of which command characters have been typed (e.g., "/P") as well as the accumulating veedit input.

The editconfig variable indicates which configuration of hidden and displayed text areas is being displayed.

Other variables keep track of the cell being edited, the state of the help display, etc.

The editing environment implements a scrolling grid. This is accomplished by using Javascript to remove and add rows to the original rendered spreadsheet table. To recreate rows that have been scrolled off the top, and to provide the extra information needed for editing cell contents and formats, extra information is stored about each cell. This information goes in the "sheet" variables. They are objects, addressed by cell coordinates (e.g., sheetvals["C3"]), containing the cell types, display values, editing values, alignment, etc., for each defined cell.


The parse_sheet Format

The "sheet" variables are loaded with data created by the server. The function that takes the server-created data and puts it into these variables is usually called parse_sheet. There are parse_sheet functions in both the Edit.js and jslib/App/SocialCalc/Format.js files. (The Format.js version includes extra code to parse additional global information needed on the Format tab.)

The parse_sheet function takes as an argument a string in a multi-line, human-readable format. Each line consists of sections separated by ":" characters. The first section is the line type. Normally the line type is the coordinate of the cell for which the line defines values. The sections of this form of line are:

   coord:type:display-value:edit-value:alignment:colspan:rowspan:skip:csss-val

Where: coord is the cell coordinates (e.g., "C3"), display-value is the text to be displayed in the cell rendered as HTML (e.g., "$4,321.00"), edit-value is the value to show on the edit line (e.g., "4321"), alignment is the CSS alignment setting (e.g., "right"), colspan and rowspan are the values for the <td> tag which allow for merged cells (e.g., "1" for both), skip is the coordinate of the cell to move the cursor to if the cursor lands on this cell or "" if the cell is not one hidden by a merged cell, and csss-val is the added explicit CSS style value for the cell (csss, set on the Format Misc tab).

Other forms of lines for the parse_sheet function are "error:Error message" (to set the error message display above the formula bar), "footer:time-string" (to update the time display in the page footer during a long editing session), and "needsrecalc" to set the "May need recalculation" indication when recalc is set to manual. The Format.js version of parse_sheet includes additional line forms to initialize lists of available formats, fonts, etc., for use on drop downs on the Format tab.

The parse_sheet routine is used in two main situations. It is used when the page is first displayed to initialize the "sheet" variables. This lets the server compress all of the settings into one string, rather than needing to download multiple variable setting statements. You will see in such situations a Javascript statement in the page source starting with "isheet=...", followed by a "parse_sheet(isheet);" statement.

The other situation is during the Ajax interaction when new values are set for cells. The browser makes an XMLHttpRequest to the server, specifying the cell coordinates and the new value. The server responds with text in the parse_sheet format. This string is processed with the parse_sheet function, which keeps track of the cells given updated values. Those cells are then updated in the DOM by the update_screen function and reflected on the screen.


Sample Code Showing Use of Common.js and Edit.js

Here is a Perl program, simpleedit1.pl, that can act as a testbed and as an illustration of the interaction between a server and the SocialCalc Engine Javascript routines. It is meant to be used in response to a CGI request with a web server. It should be installed in a directory with a sub-directory of "SocialCalc" containing Sheet.pm, SheetFunctions.pm, and Strings.pm from the /lib/SocialCalc directory in the distribution, and a sub-directory of jslib with a subdirectory of SocialCalc containing the Common.js and Edit.js files. It assumes the Javascript files are the same as those in the SocialCalc 1.1.0 release.

It is assumed that you are familiar with the documentation of the SocialCalc Engine and the simplesheet1.pl program presented there.

The simpleedit1.pl program starts out being similar to simplesheet1.pl. When first invoked, it displays a rendered empty spreadsheet and the save format text that can create it. It allows you to edit that saved text and then re-invoke the program to re-render the sheet from the updated saved data. (It does not have the command interface to issue Sheet.pm commands present in the simplesheet1.pl program since that would just make it longer without teaching anything new.)

In addition to doing what simplesheet1.pl does, the simpleedit1.pl program implements a simple system that:

Displays the HTML and Javascript for a page that allows simple editing of the spreadsheet.
Gives access to many of the facilities of SocialCalc's editing so you can move the cursor around, scroll with Page Up and Page Down keys, and view the contents of selected cells in a formula bar.
Allows you to type new values into cells.
Uses the XMLHttpRequest method for the browser to tell the server of changes to a cell.
Receives the server's response and updates the displayed sheet accordingly.
Displays the text of the Ajax request and response as a view into the internal operation.

Since this is a minimal implementation using the full Javascript, some of the operations you can try to perform when it is running (such as invoking the "/More" commands) will try to access items in the HTML that are not present and will result in Javascript errors. It does, though, perform a substantial amount of the functionality. Simple editing of the Javascript files can remove those references and make this program more robust. The object, though, of this exercise is to work from the standard released code and be able to be enhanced to test out other functionality.

Here is the complete source code (with an explanation following it):

--- Start source of simpleedit1.pl ---
#!/usr/bin/perl

   use strict;
   use SocialCalc::Sheet;
   use SocialCalc::SheetFunctions;
   use SocialCalc::Strings;
   use CGI qw(:standard);

   my %colorlookup;
   initialize_colorlookup();
   fill_in_colors(\$SocialCalc::Sheet::sheetstyleheader);

   my $q = new CGI;

   my @lines = split(/\n/,$q->param("savestr")); # sheet data in textarea
   my %sheetdata;
   parse_sheet_save(\@lines, \%sheetdata);

   if ($q->param("edit")) { # when "edit" button is pressed
      start_edit(\%sheetdata);
      }

   elsif ($q->param("ajaxsetcell")) { # Ajax call to change cell value
      ajax_call(\%sheetdata, $q->param("ajaxsetcell"));
      }

   else { # default - on initial use or when "render" button is pressed
      recalc_sheet(\%sheetdata);

      my ($stylestr, $outstr) =
         render_sheet(\%sheetdata, "", "", "s", "a", "inline", "", "", "");

      my $savestr = create_sheet_save(\%sheetdata);

      print <<"EOF";
Content-type: text/html

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
 "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Simple Sheet With Editing</title>
<style>
body, td, input, texarea
 {font-family:verdana,helvetica,sans-serif;font-size:small;}
</style>
</head>
<body>
<h2>SIMPLE SPREADSHEET SYSTEM WITH EDITING</h2>
<h3>Rendered Sheet:</h3><hr>
$outstr
<hr>
<form action="" method="POST">
<h3>Saved Data:</h3>
<textarea name="savestr" rows="7" cols="60">
$savestr
</textarea>
<br><br>
<input type="submit" name="render" value="Render">
<input type="submit" name="edit" value="Edit">
</form>
</body>
</html>
EOF
      }

#
# start_edit(\%sheetdata) - render initial editing display
#

sub start_edit {

   my $sheetdata = shift @_;

   my $savestr = create_sheet_save($sheetdata);

   # Load scripts from a file

   my $inlinescripts .= $WKCStrings{"jsdefinestrings"};
   open JSFILE, "jslib/SocialCalc/Common.js";
   while (my $line = <JSFILE>) {
      $inlinescripts .= $line;
      }
   close JSFILE;

   $inlinescripts .= $WKCStrings{"editjsdefinestrings"};
   open JSFILE, "jslib/SocialCalc/Edit.js";
   while (my $line = <JSFILE>) {
      $inlinescripts .= $line;
      }
   close JSFILE;
 
   fill_in_colors(\$inlinescripts);

   # patch in passing savestr back and forth

   $inlinescripts =~ # add to ajaxsetcell_request function
      s/
       (function\ ajaxsetcell_request\(.*?)\ makeRequest
       /
        $1
        reqcontents = reqcontents + 
        "&savestr="+encodeURIComponent(document.f0.savestr.value); 
        document.f0.ajaxreq.value=reqcontents;
        makeRequest/xs;

   $inlinescripts =~ # add to parse_sheet function
      s/
        (function\ parse_sheet\(.*?)\ var\ col
       /
        $1
        else if (cr=="sheettext") {
         document.f0.savestr.value=decode_field(parts[1]);
         document.f0.ajaxres.value=str;continue;
         }
        var col/xs;

   my ($sheetstyle, $sheetstr) =
      render_sheet($sheetdata, 'id="sheet0" class="wkcsheet"',
         "", "s", "a", "ajax", "A1", q! onclick="rc0('$coord');"!, "");

   my %celldata; # get initial cell settings
   my ($lcol, $lrow) = render_values_only($sheetdata, \%celldata, "");
   my $jsdata = qq!var isheet="";\nisheet="!; # cell information

   foreach my $cr (sort keys %celldata) { # construct output
      my $str = prepare_celldata(\%celldata, $sheetdata, $cr);
      $str =~ s/\\/\\\\/g;
      $str =~ s/"/\\x22/g;
      $str =~ s/</\\x3C/g;
      $jsdata .= "$str\\n";
      }

   $jsdata .= qq!"\n!; # end isheet definition

   $inlinescripts .= <<"EOF"; # put together initializing JS code
<script>
$jsdata
sheetlastcol=$lcol;
sheetlastrow=$lrow;
parse_sheet(isheet);
ecell="A1";
needsrecalc="";
cliprange="A1"; //dummy to avoid msg
</script>
EOF

   print <<"EOF"; # output page with edit JS code
Content-type: text/html

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
 "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Simple Sheet With Editing</title>
<style>
body, td, input, texarea
 {font-family:verdana,helvetica,sans-serif;font-size:small;}
.smaller {font-size:smaller;}
.cellnormal {}
.cellcursor {color:#FFFFFF;background-color:#394F87}
$sheetstyle
.selectedcolname 
 {text-align: center;color: #FFFFFF;background-color: #394F87;
  border-left:0px none;border-right:0px none;}
.selectedrowname
 {text-align: right;color: #FFFFFF;background-color: #394F87;
  padding-left:1em;border-top:0px none;border-bottom:0px none;}
</style>
</head>
<body onKeydown="return ev1(event);"
      onKeypress="return ev2(event);"
      onLoad="save_initial_sheet_data();move_cursor(ecell,ecell,true);">
<h2>SIMPLE SPREADSHEET SYSTEM WITH EDITING</h2>
$inlinescripts
<h3>Editing Sheet:</h3><hr>
<form name="f0" action="" method="POST">
<table cellpadding="0" cellspacing="0">
<tr>
 <td class="smaller">&nbsp;</td>
 <td valign="bottom">
  <div id="mode1"><span class="smaller" id="valuetype">Loading...
   </span><span id="warning" class="warning"></span>
   <span id="recalcmsg" class="smaller" style="display:none"></span>
   &nbsp;</div>

  <div id="mode4" style="display:none;"></div>
  <div id="mode5" style="display:none;"></div>
  <div id="mode6" style="display:none;"></div>
  <div id="mode7" style="display:none;"></div>
 </td>
</tr>
<tr>
 <td valign="top" width="1">
  <span id="coordtext">&nbsp;</span>&nbsp;
 </td>
 <td valign="bottom" width="100%" nowrap><span id="config1a">
   <input class="smaller" type="text" size="80" name="valueedit"
    value="" autocomplete="off" onFocus="ve_focus()">
    <input class="smaller" type="submit" name="okedit" id="okeditve" 
     value="OK" onClick="this.blur();return process_OK();">

   </span><span id="config2a" style="display:none;">
   <input class="smaller" style="font-style:italic;" type="text"
    size="80" value="Multi-line edit required" disabled>
   </span><span id="config45a" style="display:none;"><span
    id="rangeend"></span>
    <span id="kbdprompt" class="smaller"></span></span><span id="config3a"
     style="display:none;">
    <textarea cols="80" rows="10" name="valueedittext"
     onFocus="vet_focus()"></textarea><br>
    <input class="smaller" type="submit" name="okeditvet" value="OK"
     onClick="this.blur();return process_OK();">
    <input class="smaller" type="submit" name="canceleditvet" value="Cancel"
     onClick="this.blur();process_typed_char('[esc]');return false;">
   </span>
 </td>
</tr>
</table>
<div id="draghandle" style="display:none;"></div>
<div style="height:6px;"></div>
$sheetstr
<hr>
<h3>Saved Data:</h3>
<textarea name="savestr" rows="7" cols="60">
$savestr
</textarea>
<br>
<h3>Last Ajax Request:</h3>
<textarea name="ajaxreq" rows="2" cols="60">
</textarea>
<br>
<h3>Last Ajax Response:</h3>
<textarea name="ajaxres" rows="4" cols="60">
</textarea>
<br><br>

<input type="submit" name="render" value="Render">
<input type="hidden" name="scrollrow" value="1">
<input type="hidden" name="editcoords" value="A1">
<input type="hidden" name="sitename" value="">
<input type="hidden" name="datafilename" value="">
<input type="hidden" name="loggedinusername" value="">
<input type="hidden" name="loggedinuserpassword" value="">
</form>
<form name="ftabs">
<input type="hidden" name="scrollrow" value="1">
<input type="hidden" name="editcoords" value="A1">
</form>
<form name="fsl">
<input type="hidden" name="scrollrow" value="1">
<input type="hidden" name="editcoords" value="A1">
</form>
</body>
</html>
EOF
   }


#
# prepare_celldata(\%celldata, \%sheetdata, $cr) - render definition
#    line for $cr in isheet and ajax response data format
#

sub prepare_celldata {

   my ($celldata, $sheetdata, $cr) = @_;

   my $cellspecifics = $celldata->{$cr};
   my $displayvalue = encode_for_save($cellspecifics->{display});
   $displayvalue = "" if $displayvalue eq "&nbsp;"; # this is the default
   my $csssvalue = encode_for_save($sheetdata->{cellattribs}->{$cr}->{csss});
   my $editvalue;
   if ($sheetdata->{datatypes}->{$cr} eq 'f' ||
         $sheetdata->{datatypes}->{$cr} eq 'c') { # formula or constant
      $editvalue = encode_for_save($sheetdata->{formulas}->{$cr});
      }
   else {
      $editvalue = encode_for_save($sheetdata->{datavalues}->{$cr});
      }
   my $str = "$cr:$cellspecifics->{type}:$displayvalue:$editvalue:"
      . "$cellspecifics->{align}:$cellspecifics->{colspan}:"
      . "$cellspecifics->{rowspan}:$cellspecifics->{skip}:$csssvalue";

   return $str;
   }


#
# ajax_call(\%sheetdata, $command) - respond to ajaxsetcell request
#

sub ajax_call {

   my ($sheetdata, $command) = @_;

   my ($psite, $pname, $coord, $value) = split(/:/, $command, 4);
   $value = decode_from_ajax($value);

   my %celldatabefore;
   render_values_only($sheetdata, \%celldatabefore, "");

   # Determine value type and do appropriate command to set it

   my $type;
   my $fch = substr($value, 0, 1);
   if ($fch eq "=" && $value !~ m/\n/) {
      $type = "formula";
      $value = substr($value, 1);
      }
   elsif ($fch eq "'") {
      $type = "text t";
      $value = substr($value, 1);
      }
   elsif (length $value == 0) {
      $type = "empty";
      }
   else {
      my $v1 = determine_value_type($value, \$type);
      if ($type eq 'n' && $v1 == $value) { # check we don't need "constant"
         $type = "value n";
         }
      elsif ($type eq 't') {
         $type = "text t";
         }
      else { # handle all the special types
         $type = "constant $type $v1";
         }
      }

   my $cmdline = "set $coord $type $value"; # create command
   my $ok = execute_sheet_command($sheetdata, $cmdline); # set cell value
   my $aerrtext = recalc_sheet($sheetdata);

   my %celldataafter;
   render_values_only($sheetdata, \%celldataafter, "");

   my $response;
   foreach my $cr (sort keys %celldataafter) { # construct output
      my $cdbefore = $celldatabefore{$cr};
      my $cdafter = $celldataafter{$cr};
      next
       if $cdbefore->{type} eq $cdafter->{type}
       && $cdbefore->{display} eq $cdafter->{display}
       && $cdbefore->{align} eq $cdafter->{align}
       && $cr ne $coord; # only stuff that has changed unknown to client
                         # (but send at least one -- the one with "loading")

      my $str = prepare_celldata(\%celldataafter, $sheetdata, $cr);
      $str =~ s/\\r//g;
      $str =~ s/]]>/\\e/g;
      $response .= "$str\n";
      }
   my $savestr = create_sheet_save(\%sheetdata);
   $response .= "sheettext:" . encode_for_ajax($savestr) . "\n";

   print <<"EOF";
Content-type: text/xml

<?xml version="1.0" ?>
<root><![CDATA[
$response
]]></root>
EOF
   }


#
# fill_in_colors(\$str) - Replaces {[colorname]} in $$str with color value
#

sub fill_in_colors {

   my $str = shift @_;

   $$str =~ s/\{\[(\w+?)\]\}/$colorlookup{$1}/ge;

}


#
# initialize_colorlookup() - Fills in %colorlookup
#

sub initialize_colorlookup {

   my %colordata = ( # normally this comes from $WKCStrings
"colornames" => "error cursor rangebody rangeend
                 gridheaderbackground gridheadertext 
                 gridheaderbackgroundselected gridheadertextselected 
                 skippedcellbackground
                 draghandleup draghandledown draghandleoff 
                 scrolloff scrollon
                ",
"color-error" => "#FF0000",
"color-cursor" => "#394F87",
"color-rangebody" => "#EEEEEE",
"color-rangeend" => "#80A9F3", #99CC99
"color-gridheaderbackground" => "#80A9F3",
"color-gridheadertext" => "#FFFFFF", #FFFFFF
"color-gridheaderbackgroundselected" => "#394F87",
"color-gridheadertextselected" => "#FFFFFF",
"color-skippedcellbackground" => "#CCCCCC",
"color-draghandleup" => "#CCCCCC",
"color-draghandledown" => "#999999",
"color-draghandleoff" => "#80A9F3",
"color-scrolloff" => "#FFFFFF",
"color-scrollon" => "#999999",
);

   foreach my $c (split(/\s/, $colordata{colornames})) {
      $colorlookup{$c} = $colordata{"color-$c"};
      }
}


#
# decode_from_ajax($string) - Returns a string with 
#       \n, \b, and \c escaped to \n, \, and :
#

sub decode_from_ajax {
   my $string = shift @_;

   $string =~ s/\\n/\n/g;
   $string =~ s/\\c/:/g;
   $string =~ s/\\b/\\/g;

   return $string;
}


#
# encode_for_ajax($string) - Returns a string with 
#       \n, \, :, and ]]> escaped to \n, \b, \c, and \e
#

sub encode_for_ajax {
   my $string = shift @_;

   $string =~ s/\\/\\b/g;
   $string =~ s/\n/\\n/g;
   $string =~ s/\r//g;
   $string =~ s/:/\\c/g;
   $string =~ s/]]>/\\e/g;

   return $string;
}
--- End source of simpleedit1.pl ---

When invoked using a browser, this script should display an empty rendered sheet, a text area labeled "Saved Data" displaying the save format data for the empty sheet, and buttons labeled "Render" and "Edit".

To load a more interesting spreadsheet, namely our example from the other documentation, copy the following into the Saved Data text area:

version:1.3
cell:A1:t:Value\c:f:1
cell:B1:v:9
cell:A2:t:Square root\c:f:1
cell:B2:vtf:n:3:SQRT(B1)
col:A:w:120
sheet:r:2:c:2
font:1:normal bold * *

If you then press the "Render" button the page should repaint with the appropriate rendered sheet.

You can also copy saved data from SocialCalc save files, such as the wkcdata/sites/demosite/demopage1.edit.demoauthor.txt file created when using the demo setup. The text you should copy is from the "version" line in the last MIME component (e.g., "version:1.2") to the end right before the "--wkc-boundary--" line (e.g., "valueformat:2:#,##0.0"). That particular file (demopage1...) is interesting because it has merged cells and a cell with multi-line text as well as many formulas with cascading values after a recalculation.

When you press the "Edit" button you will see the screen replaced by the spreadsheet in a grid with 1-2-3/A-B-C row/column labels. Above the grid is a simple formula bar displaying the contents of the cell with the cursor. Below it is the same Saved Data textarea. Below that are two textareas to show text of the last Ajax request to the server and the server's response.

You can use the arrow keys or the mouse to move the cursor to other cells. Their values will be shown in the formula bar. If you type a new value into a cell (either text, a number, or a "=formula") it will echo into the cell and on the formula bar. If you type something like "=1+" and then press the arrow key, the cell coordinates will be added to the formula appropriately. You can also click on the formula bar text area and edit there.

If you encounter a cell with a multi-line value and click a second time on the cell, or start typing in a cell with the quote (") character, you will be able to use a textarea for editing that replaces the single-line formula bar on the screen.

Pressing Enter or clicking the "OK" button sends the new cell value along with the sheet data to the server using an XMLHttpRequest object. The server responds with parse_sheet-format data to update the sheet as well as updated sheet data to be displayed. The text of the request and the response is shown in the textareas below the sheet.

Pressing the "Render" button returns to the normal rendered-spreadsheet display.

This program implements a "stateless" spreadsheet editing system. The program running on the server does not remember anything about the sheet between invocations. All of the information comes from forms sent with the request and then populated with updated information in the response. It operates differently than the full SocialCalc program in that it does not require user authentication and all of the intermediate editing files. It can be the seed from which you can figure out how to add simple spreadsheeting to other online content creation systems. The fact that you can implement this with just the SocialCalc Engine components shows why the split between Engine and App was done the way that it was.

Here is an explanation of the simpleedit1.pl code itself:

The first part of the code initializes the %colorlookup hash. This hash is used to help dynamically fill in the color values for the CSS controlling the look of the sheet when editing as well as any colors used by the Javascript code. Those colors are indicated by [{colorname}] in the text. The fill_in_colors routine (part of simpleedit1.pl along with the initialize_colorlookup routine) does the actual replacements. (In the SocialCalc app, these routines are in Sheet.pm.)

The program then retrieves the saved sheet data from the HTTP request and parses it, resulting in a %sheetdata structure. Then, depending upon the type of invocation, either the start_edit routine (for presses of the "Edit" button), the ajax_call routine (for XMLHttpRequest posts that have the "ajaxsetcell" parameter), or plain rendering is performed.

The plain rendering is similar to simplesheet1.pl, but it includes the extra button and no command line.

The start_edit routine puts together the HTML and Javascript needed for editing and then returns that to the browser.

It starts by making a copy of the save data for including in the output. It then reads in the contents of Common.js and Edit.js for inclusion. It replaces the color placeholders with the appropriate color values.

In order to have the browser include a copy of the saved data as part of the Ajax interaction (not something the normal use of SocialCalc does -- that information is available to the server from files) the standard text of Edit.js is "patched" before downloading. This lets this sample program run from the standard release of the SocialCalc Engine without including a special version of Edit.js in this documentation. There are two places where a bit of code is added to include sheet data as part of both the request and response and to display the request and response strings. The first place is in the ajaxsetcell_request function where a "savestr" parameter is added to the request and the request textarea on the screen is updated. The second place is in the parse_sheet function where code to handle a "sheet" line form is added that updates the saved data and Ajax Response textareas.

The start_edit routine then renders the sheet using render_sheet, setting the $editmode to "ajax" to have it include a display of the grid. The $extratableattributes argument is set to 'id="sheet0" class="wkcsheet"'. This is needed for the Javascript code to have easy access to that table in the DOM and to help format the table. The $onclickstr is set to q! onclick="rc0('$coord');"! so that clicks on any cell will result in a call to the Javascript rc0 function with an argument containing the cell coordinates. (The render_sheet routine replaces the characters "$coord" in the string with the cell coordinates.) This rendering results in a string with both the CSS style information needed by the sheet ($sheetstyle) and the sheet HTML itself ($sheetstr).

The render_values_only routine in Sheet.pm is then used to just render the sheet into a structure with display values and attributes. This structure is then used by the prepare_celldata function in simpleedit1.pl to create the isheet definition for inclusion as part of the Javascript.

The Javascript of the libraries, along with the isheet data, is put together with the setting of some Javascript variables and a call to the Edit.js function parse_sheet. This string is then returned to the browser as part of the HTML, along with the $sheetstyle and $sheetstr at appropriate places.

The HTML template includes extra CSS needed by the rendered sheet with a grid. It includes attributes in the <body> tag to initialize information needed for re-rendering the sheet and to catch the keyboard events.

The HTML for the formula bar includes many <div> sections that have IDs but are not displayed. These are referenced by some of the Javascript code and need to be there to avoid errors even though some are not used here. The valueedit <input> field and valueedittext <textarea> are used for editing. The "config1a", "config2a", and similar <div> sections are used to turn on and off various screen components. The hidden <input> fields at the bottom are used by the Javascript code and must be here to avoid errors even though this simple system is not using them.

At this point, once the string is "printed" and returned by the server, the browser is displaying the sheet and is ready for editing.

The prepare_celldata function, used by both the start_edit and ajax_call routines, encodes the information about a cell for parsing by the parse_sheet function in the Javascript code. The results need additional encoding to handle the peculiarities of being part of a Javascript string constant (in start_edit) or XML (in ajax_call).

When an XMLHttpRequest is made, the HTTP POST request comes into the simpleedit1.pl program and is handled by the ajax_call routine.

The ajax_call routine starts by extracting the coordinate of the cell being changed and the new value from the parameter string. It then calls the render_values_only routine to set the %celldatabefore structure with copies of the values and attributes of all cells before the modification.

The routine then examines the value typed by the user to determine its type. This can be determined by the first character (the "'" and "=" used for explicit text data and formulas) and using the determine_value_and_type routine in Sheet.pm. Given that result, the routine then uses the execute_sheet_command routine to make the change and the recalc_sheet routine to recalculate the sheet values to reflect that change.

The render_values_only routine is used again, this time to set the %celldataafter structure. The program loops through all these values and looks for changes from the values or attributes in the %celldatabefore structure. When it finds a change (and for the "modified" cell, even if the value is the same) it creates a new line of text in the parse_sheet format using the prepare_celldata function. It adds a parse_sheet command, "sheet", to set the saved data. In a simple case there will be only one line of changed value and one long line for the saved data. In a case with a cell whose value is used in calculations by other cells, many lines of data may be created.

The resulting lines of text are encoded in XML and returned to the browser which processes it. The Last Ajax Request and Last Ajax Response textareas may be used to examine the inputs and outputs of the operation.


The Details of Common.js

The first section of Common.js defines many of the main variables. It has already been described above.

The next section of Common.js are the "rc" routines. These routines convert between the "E8" string-valued style of coordinates and numeric-valued row/columns (such as 5 and 8 in this case).

The next section contains the ev1 and ev2 routines. These routines are called as event handlers for the Keydown and Keypress events, respectively. They are coded based on experimentation with a variety of browsers (old Netscape, IE 6 and 7, Firefox 1 and 2, Opera, Safari, etc.). They result in either passing the event on to the next handler, for example when editing in an input field or scrolling through a dropdown and we want to let the browser code handle the event, or passing the appropriate key string to the process_typed_char function (which is defined in Edit.js and Format.js). This string is either the typed character or a readable representation of special characters (such as "[aright]" for right-arrow and "[esc]" for the Esc key).

The next section are three routines to perform the generalized XMLHttpRequest (XHR) interaction (makeRequest, alertContents, and ajaxtimeout) and two routines to use with those routines to request named text strings from the server and then set the innerHTML of a specified DOM node with the text returned by the server.

The comments in the code list the sources for information used when writing the generalized XHR routines. The request to the server is made using an HTTP "POST" request. The response is assumed to be in XML with a <root> element at top level. The contents of that element are assumed to be a string of characters, usually as a CDATA. Errors are indicated with alert boxes using strings from the Strings.pm module. The loading flag is set to true so that other parts of the system can avoid race conditions while the interaction with the server goes on in the background.

The ajaxgetnamedtext_request routine takes a named text name (stored on the server in the share/app/socialcalc/textdata.txt file) and the ID of an element in the DOM. It makes a "ajaxgetnamedtext" request of the server. The setnamedtext routine is invoked upon return from the server. It sets the innerHTML of the requested node to the returned string. This is used for Help display.

After the XHR-related routines come the move_cursor and hide_cursor routines. The cursor is represented on screen by a <div> within the cell's <td>. By changing the class of that node in the DOM you can change the look of a cell. The move_cursor routine turns the current cursor position into a "normal" looking cell and then sets the new position to look like a cursor. It ensures that new cell is visible and not scrolled off the top of the scrolling area (i.e., it calls scroll_to_row to make sure the row with the cursor exists). It also uses the sheetskip object's value (filled with data from the server using parse_sheet). This value tells move_cursor where the cursor should actually end up when it is moved onto a cell hidden as part of a group of merged cells.

Next in Common.js start a section of utility routines used to set attributes of various screen elements. The set_display and set_styles routines are used to hide and display elements. For example, set_display is used by various "set_editconfig" routines to show and hide the different formula bars, status lines, edit controls, etc., above the grid. The set_editcoords routine is used to set the hidden "editcoords" <input> fields in the various <forms> on the screen so the current value is always passed back to the server. The set_text, set_properties_name, set_styles_id, set_option_selected, set_option_selected_indirect, and set_radio_checked routines are used for other attribute setting functions. They reflect the different types of objects and the different coding techniques used at various points in development.

The encode_field and decode_field routines are used for Ajax-type communication. The check_error routine updates the warning display (a <span> at the top of the status and formula bar area above the grid) appropriately. The toggle_help shows and hides the visible help components and calls ajaxgetnamedtext_request to set the main help text.

The next section of routines handles the scrolling of the grid. First there is the save_initial_sheet_data routine. This is called after the page is loaded by the "onLoad" event setting on the <body> tag. The routine walks the tree of cells in the table with the ID "sheet0" and fills in the idlist object to remember references to all of the row and cell elements in the DOM. (The rows have an ID in the form of "r_rownumber", the row number cells have an ID in the form of "rn_rownumber", the column headers have an ID in the form "cn_columnLetter", and the cells have an ID of the coordinate in "E8" form.) It also fills in the sheetclass object with the CSS class name of each cell to be used when recreating cells during scrolling. After that it uses scroll_to_row to make sure that the top row specified by the hidden <input> field scrollrow is scrolled down to.

One of the most important routines is the scroll_to_row routine. This does the actual work of deleting or recreating rows to give the impression of scrolling. One of the tricky things about this is the support for a vertical group of merged cells. It is possible to scroll the sheet such that only part of a vertically merged cell is visible. Only the visible portions are recreated, but they are shown as empty gray cells to indicate that they are only partially visible.

The scroll_to_row routine determines the direction of the scroll. If the new top row is below the current one, then the appropriate number of rows are deleted. If the new top row is above the current one, the rows are recreated using the make_row routine.

After the new row configuration is completed, existing rows must be checked for gray partial-rowspan cells that must be added or gray partial-rowspan cells that need to be removed (to allow for rowspan's to cover them). This is done by starting at the first row where this might be a problem and continuing down until the make_row routine indicates that vertical-cell skipping isn't present. Each of those rows is recreated, taking into account the new set of visible rows.

The next routines, scroll_relative and check_draglook, are helper routines. They do the checks to keep scrolling within bounds and to turn off scrolling during edit. (Scrolling while editing could change the cursor position which is forced to always be in the visible sheet.)

The make_row routine creates new <tr> and <td> elements. It sets the attributes and sub-elements according to the values in the "sheet" variables. It uses the sheetskip values to determine if this cell is covered by a vertical cell merge above. If so, it indicates this fact to the calling scroll_to_row by setting processedskip to true and then sets the cell CSS class to "skippedcell".

The next routines (begindrag, dragmove, dragend, set_slider, slider_page, do_pageupdn, scroll_to_home, begin_scroll, and scroll_timeout) deal with the SocialCalc scrollbar. The Javascript code assumes that there is a pseudo-scrollbar with a slider to drag. There is also code to handle PageUp and PageDown key presses, clicks within the scrollbar, and repeated scrolling from click-and-holding on the top or bottom ends of the scrollbar. This code should be relatively straightforward. There are some limits put in to keep the scrolling within a range even though the code does not know the actual size of the browser window.

The final routines in Common.js are for highlighting a range of cells: highlight_block, start_highlight, and end_highlight. In addition to controlling the look of the cells in the range, there is also code to update the rangeend <span> that displays the bottom-right cell being pointed to as well as the size of the range in rows and columns.


The Details of Edit.js

The Edit.js file contains code used on the Edit tab of SocialCalc as an app. This includes code to handle XHR requests for setting cell values getting a list of names for the /Range Names command, handling the keystrokes during editing and in the menu tree, handling some of the buttons displayed, and maintaining the look of the formula bar area.

It starts with the ajaxsetcell_request and ajaxrangenames_request routines that set up the arguments for XHR requests to the server. The ajaxsetcell_request routine handles changes to cell values, with the responses handled by the setcell routine. The ajaxrangenames_request routine handles a request to get range names, with the response handled by the gotrangenames routine.

The next two routines, ve_focus and vet_focus, handle the cases of when the user clicks in the input areas in the formula bar. This involves setting various flags and removing newly echoed text that may be showing in the selected cell. Some of the steps done are taken to leave the edit field in a state that seems natural for the browser.

The process_OK routine handles the two OK buttons for editing.

The process_typed_char routine is called by the routines in Common.js that catch the keyboard events. It is for keys pressed during normal editing; that is, not when the keys are to be processed by a normal browser input field. To understand its operation, you should be well acquainted with the way that SocialCalc behaves during editing. See the help text for Cell Editing and watch the operation while you typed. Like many keyboard-aware spreadsheets from VisiCalc to Excel, SocialCalc has many states while you are typing. It can go into different input modes (text, formula, numeric, command tree) and it displays various status indicators that are constantly updated. If process_typed_char returns a value of true, the key event is passed on for normal processing by the browser; if it returns a value of false, then the browser is supposed to ignore the key, having let the Javascript code here handle it.

The reset_pointing routine is used to clean up when the operation that lets you fill in a cell address in formula just by pointing to the cell with the arrow keys completes.

The save_value routine saves the value being sent to the server to the "sheet" values and then actually invokes the ajaxsetcell_request routine to send the value to the server. It leaves a "Loading..." message in the cell that will be replaced by the formatted value when the server responds.

The process_typed_char_ve routine processes keystrokes that go to the formula bar input field to enforce no-editing while waiting for an ajaxsetcell request to be processed, and to perform the Enter and Esc functions. In other cases, it just returns no value to let the event propagate to the browser's handler.

The update_valueedits routine takes care of setting various status information when a cell is displayed. This includes updating the cell type listed above the formula bar input field, checking for multi-line data that cannot be displayed in a single line, and enabling or disabling various command buttons in the menu system.

The set_editconfig routine is the main way that elements of the formula bar and menu area display are turned on and off for display. Each configuration (there are 7) display a different set. The configurations are as follows: 1 is the normal editing mode, 2 is the display that multi-line editing is required, 3 is multi-line editing mode, 4 is the range commands, 5 is the "More..." commands, 6 is the table/sort commands, and 7 is the range names command display. In addition to hiding and displaying the different <div> areas and such, the Help display is kept updated appropriately if it is enabled.

The set_kbdprompt routine processes command keys (the "/" commands). It takes care of handling the side-effects, setting the config numbers, and updating the prompt display. Even though it is usually run on a GUI system, a spreadsheet can be a very keyboard-centric program because most of the data is typed in, and therefore it is helpful to have commands that can be executed without moving your hands from the keyboard. Of course, there are clickable equivalents on the screen, but for many users the keyboard is important, too. In any case, the number of potential commands is large, so to minimize wasted screen space and keep a clean look, the screen needs to respond as the command tree is navigated.

The next routine is parse_sheet, which has been covered pretty well above.

The update_screen routine updates the display on the screen of cells that have been updated by the server as part of an ajaxsetcell operation.

The rc0 routine is the one called when the user clicks on a cell. It performs some of the same type of operations as the process_typed_char routine to move the cursor and handle editing appropriately.

The more_button, cancel_more, morecmd, range_button, cancel_range, set_range, table_button, and cancel_table routines handle the command buttons. The set_more routine sets the buttons that determine what is copied from the selected range of cells.

The update_sort_select routine fills in the dropdown boxes used in the Sort dialog. You have the option of choosing which column is the major, minor, and last sort, and the dropdown only lists the columns in the selected range.

The editrangecmd routine actually submits the request to the server for range commands (e.g., copy, cut, paste, etc.). These are handled by a normal server request, not an Ajax request, because they can change the whole layout of the sheet.

The update_page_list and set_select routines are used to make an XHR request for a list of pages to link to in the multi-line edit mode. The choose_pagelink routine is used to choose one of them and insert the appropriate wiki-text in the cell.

The selectrangename routine is used to fill in the name, description, and value when an existing range name is selected in the Range Name dialog.

[END]