Input files for WHS are most conveniently made using the computer algebra system Maple 9.5 or the Classic interface of versions 10 or 11. This means that the author can use the computational power of Maple to check results and draw plots for the assignment as well as the high level worksheet interface to format mathematical formulas. One intersperses these supporting computations with the input to be read by WHS. The Maple .mws file is then written out as HTML and .gif image files; these are made into a .zip file which is uploaded to WHS and converted into the .xml file which is actually read by WHS. The upload process also renames .doc, .docx, .tex, and .mws files at top level in the unzip file as well as the html file from which the .xml file is built so that they are not accessible via URL (since these files typically contain answers to the the assignment questions). It is expected that the author will not include other files which might be accessible via URL and hence be used to reveal the answers.

In order for the worksheet to be usable by WHS, it must have the format which we are about to describe. Note that the current implementation of WHS should accept any worksheet input in the specified format. It does, in fact, accept certain no longer supported formats in the interest of backwards compatibility.

The lines of text in the spreadsheet can be divided into two classes:

1. Control lines are instructions to WHS. They specify such things as the beginning of a question, the location of an input widget such as a text box, and the beginning of a part of the worksheet that should be ignored by WHS. Control lines have a control entity, strings of a particular form which always contains an underscore character. The precise format of these entities are described below. But, it is recommended that worksheet authors should simply avoid use of the underscore character entirely except in control lines.
2. Other lines are simply referred to as normal text lines.

Input Ignored by WHS

Each control line together with all the normal text lines immediately following it and appearing before the next control line make up a section of the input file. The normal text lines appearing before the first control line are also said to form a section. The following sections are ignored by WHS:

1. All sections prior to the first control line containing H_, T_ or Q_ are ignored.
2. All sections with a control line containing SKIP_ and all immediately following sections with control lines not containing T_ or Q_ are ignored by WHS.

The all backquote (grave) characters (`) will be stripped from the uploaded file. Also, the input file should not have any occurrences of two contiguous dollar signs ($$) unless it is preceded by a backslash (\$$).

General Question Format

If the first section not ignored by WHS has a control line containing H_[x] where x is a positive integer or H_, then the text of the section is the global header. It is output at the beginning of the Homework assignment. Typically, it is used for a title. If the [x] is specified, then x is the multiplicity, which defaults to the value 1. If the multiplicity is greater than 1, then out of each group of x questions, only one randomly selected one is included in the displayed assignment. Each time the assignment is displayed, different selections of problems are obtained.

In more detail, when an assignment is requested, one can specify a numerical version number. There are several possibilities:

1. If the version is -1, one gets the Common version. If the multiplicity is m, then this consists of questions whose numbers are 1 mod m.
2. If one requests the Personal version, then one gets the version whose version number is equal to the account id of the user.
3. One can specify a Specific version by giving the version number.
4. One can specify a Specific version 0 or simply leave the version number field blank. This causes a random version to be selected.
5. Versions other than the Common version are made up by pseudo random selection of one problem from the last m - 1 of the problems in each group of m contiguous problems.

Character Entities One can specify some special characters by their character entities, as well as by the symbols themselves. For example, the less than symbol is either < or can be specified as the character entity &lt; Because WHS uses semicolons as separators, and because character entities end in a semicolon, some care is needed in their use in the parameter portions of question and answer tags.

1. In general, you can use either the character itself or the character entity.
2. The ampersand character & cannot be used except in character entities. In particular, if you want the ampersand character, you must use its character entity: &amp; and not the character itself.
3. Not all character entities are present in the fonts used by WHS.

Parameter Details for the H_ tag The parameters between the square brackets are semicolon separated. The first is the multiplicity as defined above. The second is the filename of the .dll file where the Checker class is defined. This is typically code.dll; it is located at top level in the directory where the homework assignment is stored, and so should be at top level in the .zip file which is uploaded to the server. If there is no code file, the parameter should be left empty, e.g. H_[4;;1].

By putting a third parameter called base, you can cause the randomization to start after question number base, and for the Question XXX not to be displayed. This was a hack for KEMTP. Experimental feature: if you set this parameter to -1 and if the multiplicity is greater than 1, then the problems selected for a version will appear in the same order as in the Maple or Word document; if you need to have the original functionality of base, then replace it with -1 - base, e.g. base = 1 gives a new value base of -2 if you want no scrambling. Make sure that base ( or -1 - base if base < 0) is no larger than the number of questions in the assignment or else anomalous behavior may result.

By putting a fourth parameter called header, one can vary the logo image at the top of the page. The value of the parameter is a filename; the file itself must be located in the html subdirectory of the main wqs directory of the server.

By putting a fifth parameter called hwFlags, one can affect how the homework xml is generated. If the low order bit (bit 0) is set, then video links on the client machine will be to files in the root directory. If bit 1 is set, then the problems codes selected for display in any version will all have the same residue modulo the multiplicity. (If you set bit 1 of hwFlags and use a negative base value, then one can make up threads of problems which span multiple problems and know that each variant will display a consistent thread.)

Finally, by putting a sixth parameter called files, one can add a colon separated list of comma separated pairs consisting of a filename and display name for the file. The files typically contain supplementary materials and should be in html format appropriate for being the content of a div tag. Alternatively, one can specify the filename in the form poIdxxx where xxx is a posting identifier number; in this case, the poText field of the posting will be used as the file content. Note that this will work correctly only if the student accessing the homework assignment has permission to read the posting.

If you need to include material pertaining to all the problems, it is possible to include it in the data portion of the H_ section. In particular, one can include a script section to include a javascript file, whose path is relative to the top level of the zip input file tree. A function initHw() will be called if it exists in the homework. (As there is the obvious possibility of name collisions with the existing JavaScript code and because one will typically need additional implementation details (which may not be guaranteed to be supported in the future), you are asked to check with the maintainer before using this capability.)

The remaining sections not ignored by WHS can be grouped into questions. Each question consists of

1. An optional leader section. The control line contains T_. The normal text is used as a lead-in to the question. In extreme cases, it can be an entire lecture of material before the first question.
2. A mandatory question section whose control line contains QM_ or QN_ (or Qx_ where x refers to any of a certain number of upper case letters, specifying particular deprecated question formats.)
3. A series of answer sections. There should be at least one in a QM_ question. The control line is identified by the string Ax_ where x is one of the upper case letters C, F, H, L, R, and S. The letter specifies particular answer formats. In addition, for QN_ questions, one can use X.

Question Control Line Formats

Questions whose answers are not to be graded by WHS are QN_ questions; those whose answers are to be graded by WHS are QM_ questions. Students are given the opportunity to submit comments about QM_ questions only if they submit a proposed answer; these comments are sent via e-mail to their instructors.

If the character immediately following the underscore is an upper case N, then the correct answer is not displayed when the student submits an incorrect answer to the question. Otherwise, the correct answer is displayed when an incorrect answer is submitted. The correct answer is always displayed when a correct answer is submitted.

Addition for homework deadlines: A homework assignment in a homework list can be assigned a due date. This due date is per homework list. If a homework assignment has been given a due date and a submission is made before the due date in a class using the homework list where the due date is set, then the correct answer is not displayed if the answer is incorrect and the version is not the Common version or the multiplicity is equal to 1.

Correct answer specification occurs immediately after the underscore character (or the letter N following the underscore character. It should occur for both QM_ and QN_ questions, even though the answers are ignored in the case of QN_ questions. The specification is enclosed in square brackets and consists of a sequence of at least two quantities. The quantities are semicolon separated. The first quantity is the error bound; its use will be described momentarily. The remaining quantities are the correct answers for each of the input fields in the answer section. The format of these will also be described below.

When answers are verified, one uses the error bound, the correct answer, and the submitted answer. Each of these may be expressions whose value is first calculated. If the error bound is non-negative, one verifies that the absolute error is at most equal to the error bound. If the error bound is negative and the correct answer is non-zero, then one verifies that the relative error is at most equal to the absolute value of the error bound. If the error bound is negative and the correct answer is exactly zero, then one verifies that the absolute error is at most equal to the absolute value of the error bound. When one is verifying equality of functions, the same tests are carried out for a specified number of functional values.

The correct answer specification of QM_ questions can be followed by an additional square bracket enclosed semicolon separated list of non-empty description strings. These strings should be case insensitive and contain neither square bracket characters nor single or double quote characters or semicolons. The intention is that these strings will be the names of standard skills being tested by the question. The list of the strings together with the question numbers where they are listed is stored in the homework descriptor. The homework results pages show numbers of problems correctly answered for each description string.

Author Checked Questions This format is only available in the new interface. They are specified as QA_ questions and where the additional parameters are specified between square brackets. The additional parameters are an error bound, the name of the check routine, and a string parameter passed to the check routine;the parameters are semicolon separated. In addition, one must add n - 2 additional semicolons where n is the number of answer tags in the question. (The format requires at least two answer tags.)

The answer formats are the same as for the answers for QM_ questions except that the additional checking parameters for functions are not used. In order for the checking to occur, the author's account must have an additional privilege and there must be a C# file wich with a class which derives from WhsCheck.Checker and has nine public methods which override abstract methods in WhsCheck.Checker. These methods are named Check, Check2, ..., Check9, return string values and take three string parameters attempt, answer, and errLevel. Typically, most of these will be dummy routines. There should be nothing other than the definition of this class in the file -- the name of the file could be code.dll; it must be specified in the second parameter to the global H_ header and the source file should be at top level in the zip archive which is uploaded to the server in order to install the homework assignment. (Code to compile the C# file at upload time will be put in place.) The class should use at least one Check routine (though there are 9 names Check, Check2, etc. which can be used. These routines take three parameters:

1. The first parameter is a comma separated string made up of the answers submitted by the student.
2. The second parameter is the string parameter passed as the third parameter between the square brackets in QA[...]. The typical format of this is also a comma separated list of strings
3. The third parameter is the error Bound, the first parameter between the square brackets in QA[...].

Recall that the 2nd parameter between the square brackets in QA[...] was the name of a function -- it is either one of the Check methods mentioned earlier or the word True. If it is the word True, the homework grader will always return "true". WHS instantiates an instance of checker called oCheck. It then calls Check(xVariable, yVariable, errLevel). The return value can be:

1. "true" -- in which case the answer is graded correct.
2. "false" -- in which case the answer is graded incorrect.
3. A string -- this string is returned to the user and the question is graded as not correct. Typically, the string is something like "Syntax Error".

Clearly, QA_ problems pose a security risk, which is why special permissions are needed to author them. In addition, authors will need additional materials such as a copy of the WhsChecker.dll for use in testing prior to upload.

Details of how to use the plot interface are available at https://www.mathclass.org/wqs/docs/plotdoc.htm.

Answer Control Line Formats

The various answer types correspond to different ways to input or to check answers. In order for a question to be marked correct, all the input answers must be correct.

1. Header Answers: These have AH_ in the control line. A single parameter with value 0, 1, 2, or 3 can be specified by adding [x] after the AH_. The normal text lines are output and, if the parameter is specified and has value 1 or 3, then it is followed by a line break, i.e. <br/>. If the value is 2 or 3, then it is preceded by a line break. There is no (correct) answer for a header answer section.

   Header answers can also be used to put in a video link. In this case the first parameter is as before, the second parameter is the name of the video file, and the third parameter is a label. The body of the header will be displayed followed on the right with a pull down menu and a button with the label specified as the third parameter. The pulldown menu allows the user to choose a disk drive or more typically a cd rom drive; in this case the video file is assumed to be located in a subdirectory of the root directory; the name of this subdirectory should be the homework identifier. The default value of the drive letter is simply a blank; this causes the file to be retrieved from the video server. So, a typical example would be

   AH_[0;myvideo.wmv;Show Video]
   For a video about this this or a similar problem:
2. Clue Answers: This is a variant on header answers and are, like headers, not for responding at all. The control line is of the form AK_[x;label] where x is a numeric parameter with values 0, 1, 2, or 3 and label is a string which cannot contain either a semicolon or a right bracket. The meaning of the numeric parameter is the same as with AH_[x] answers. The text which follows the tag is put into an html div tag which originally has its display style set to 'none'. The label parameter is the value of a button which is displayed at this location in the response. Clicking the button toggles the display value of the style of the div making it switch between hidden and visible.
3. Constant Answers: These have AC_[x] in their control line where x is positive integer indicating the length in characters of the input text box. The input text box is followed by the normal text lines of the section. The student should input a single number expressed in 'calculator syntax'. This means an expression with explicit asterisks for multiplication and can include pi for 3.14159..., e for 2.718..., and elementary function calls such as ln(...), exp(...), sin(...), etc. The correct answer should have the same format. The submitted answer is graded correct if it agrees with the correct answer to within the limits set by the error bound -- this is either absolute error or relative error as described above. An alternative form is AN_[x] in which the answer is required to be numerical, rather than an expression which evaluates to a numerical answer. The parameter is the same as in AC_[x]. Note that the two answer tags display in the webpage identically and so it is up to the problem author to indicate when one needs to enter a numerical answer versus an numerical expression; also, if the "right answer" is specified as an expression, it will be displayed to the user in the same way, even though entering the expression will be graded as a syntax error.

   Extended constant answers Answers to AC_[x] and AN_[x] can now be more complex. A right answer string is now a sharp (#) separated list of comma separated lists of expressions. The user input should be a comma separated list of expressions. In order for the input to be considered correct, there must be at least one of the # separated strings such that (i) the number of comma separated entries in the string is the same as the number of comma separated expressions in the user's input and (ii) the set of right answer expressions in the comma separated list of right answers must evaluate to the same set of values of the expressions in the user's input to within the error bound (the test can be either for absolute or relative error as described above). Note that the order of of the user's answers do NOT have to be the same as the order of the right answers. On the other hand, the author must have chosen the error bound so that it is less than half the distance between the closest distinct values of the right answer expressions -- so that there is no ambiguity as to where the matches should occur. (This allows for answer formats such as -1,-1#-1.)

4. Word Answers: These have the same format as Constant Answers except that you use AW_[x] instead of AC_[x]. The answer is a sharp character separated list of strings of characters. The submitted answer is said to be correct if at least one of these strings is the same string as the answer. Both answer and submitted string are trimmed of leading and trailing whitespace. The comparison is case sensitive.
5. Table Answers: Constant answers can also be input as entries in tabular arrays. To do so, use AT_[....] control lines where the semicolon separated parameters are:
   1. The size S of the textboxes, in characters.
   2. The number R of rows in the table
   3. The number C of columns in the table
   4. The actual entries in the table -- there should be R times C of these entries. Any entry which is empty is replaced by a textbox of length S.

   The correct answers are specified as several entries in the list of correct answers in the QM_[...] control line for the question. The answers occur in row major order.

   An alternative to this is to use a Maple Spreadsheet. You just put AC_[n] (where n is the size of the textbox) as the entry to the spreadsheet. You include answers as usual in row major order.

6. Function Answers: These have AF_[x;t;n;a;b] in their control line. The parameters are semicolon separated. The integer parameter x is the size in characters of the input text box. The t parameter is the variable of the function used in both the correct answer and in the input answer. When the answer is checked, n random values of this variable are chosen from the half closed interval [a,b); the functional values of the correct and submitted answers are calculated, and the absolute values of the difference of these values must all be no larger than the error bound in order for the answer to be considered correct. (The sign of the error bound is used to determine whether this is a check for relative or absolute error as described above.) The domain of the correct solution must contain the interval from a to b.
7. Extended Function Answers: These are like function answers but allow for multiple variables. These answers have AE_[x;m;x1;...;xm;n;lo1;hi1;...;lom;him] in their control line. The parameters are semicolon separated. The integer parameter x is the size in characters of the input text box. The parameter m is the number of variables, and the variable names expected in the submitted answer are x1, x2, ..., xm. When the answer is checked, n random m-tuples will be generated where the jth coordinate lies in the range of [loj, hij) and is used for the variaable xj. The functional values of the correct and submitted answers are calculated, and the absolute values of the difference of these values must all be no larger than the error bound in order for the answer to be considered correct. (This is a test for relative or absolute error depending on the sign of the error bound.) The domain of the correct answer must contain the m dimensional rectangle [lo1, hi1] x ... x [lom, him].
8. Integral Answers: These have exactly the same format as Function answers except that they use AI_ in place of AF_. When integral answers are checked, the functions are checked for equality up to a constant summand. The test is done with n - 1 randomly selected points in (a,b).
9. Selection Answers: These have AS_[val1;val2;...;valk] in their control line. The input is a pull down menu with val1, val2, ..., valk as the menu choices plus the possibility of a blank answer. The possible answers are semicolon separated strings. The blank answer is originally displayed; an answer is selected if the display is any of the other values. The correct answer must be one of the strings val1, val2, ..., valk. The normal text lines are displayed after the pull down menu. The strings val1, val2, ..., valk should not contain either semicolons or sharp characters.
10. Horizontal Line of Radio Button Answers: These have AL_[val1;val2;...;valk] in their control line. The possible answers are the semicolon separated strings val1, val2, ..., valk. The correct answer must be one of the strings val1, val2, ..., valk. The radio buttons are displayed in a centered single row table with the corresponding answer string to the right of the radio button. The strings val1, val2, etc. should not contain either semicolons or sharp characters.
11. Horizontal Line of Checkbox Answers: these have AB_[val1;val2;...;valk] in their control line. The interface generates a one line centered table of checkboxes labeled by val1, val2, ..., valk. The correct answer string should have k semicolon separated answers which are either vali or the empty string depending on whether or not the i-th checkbox should be checked in order for the result to be counted right. Because checkboxes are two-state, one does not distinguish between the answer not being given and not having any of the boxes checked. So, the author should always word the question in such a way that the student should have at least one box checked in order for the question to be answered -- e.g. one of the boxes could have the value 'none' and the others could have numbers for various possibilities which might be true. One does not allow the empty string as a possible value of any vali.
12. Vertical Row of Radio Button Answers: This is used when the value of the button needs a longer description. Each radio button is described by its own answer section, whose control line is of the form AR_[x] where x is a small integer used to identify the radio button group. A group consists of contiguous AR sections with the same group identifier -- they act in unison, in that only one of the group may be selected at once. The entire group of radio buttons is indented as a <dir> group. After each radio button, the section's normal text lines are inserted. The correct answer is an integer corresponding to the number of the radio button in its group (counting from 1). When the submitted answer is displayed on the feedback page, it is specified in the same way. Because some authors prefer to list the correct answer first in the list, the first answer is re-inserted in a random location in the group unless the parameter x is negative. The AR_ tag takes an optional second integer parameter y, as in AR_[x;y]. If y = 1, then a break is not inserted after the html content that follows the AR_ tag; if the parameter y is 0 or is not provided, then a break is inserted.
13. Vertical Row of Checkbox Answers: This is analogous to the previous tags except that it uses checkboxes instead of radio buttons in a group. The tags are of the form AQ_[x] where x is a small integer used to identify the checkbox group. The correct answer to a group of radio buttons is a sequence of either numbers (starting from 1) or empty strings separated by semicolons; for example, 1;;;4 is an answer to a group of four checkboxes in which you must have the first and fourth checked and the 2nd and 3rd clear in order for the group to be correct. There is no randomizing of the order of the checkboxes in a group regardless of the sign of the group parameter x. The AQ_ tag can also have a second parameter y which behaves the same as with the AR_ tag.
14. Multiline Textual Answers: These are used only with QN_ and QA_ type questions. The AX_[x;y] where x is a small positive integer causes a textarea element to appear; it has x rows and y columns. If one specifies AX_[x], then the textarea will have x rows and 80 columns. Typically, you will want to proceed it with a line break, which can be accomplished with an AH_[3] section before it. Since they only occur in QN_ questions, there is no grading of AX_ answers.
15. Hidden Answers: These are used only with QN_ and QA_ type questions. The AZ_[x] where is a string without semicolons or double quote characters causes two input type="hidden" html elements to be entered into the problem. The first provides a location where the respondent's answer is written and the second has its value parameter set to x and so provides the means to pass a parameter to the scripts which might run to support the problem. If this is used in a QM_ problem, then the answer should be specified in the form e:Check:str where e is an expression for the error bound, Check is the name of the author provided checking routine (which must be one of the strings Check, Check2, ..., Check9 or the word True), str is a string parameter to be passed to the checking routine; except for the colon separators, there should be no other colons in the answer string.

Video Links

Video links can be added to problems. To do so, just make the link in the form:

<a href="" target="_blank" onclick="makeVidWin('http://XXX_VIDEO/myvideo.asf')">Click here for a video solution to this or a similar problem</a>

QN_ questions can be hand-graded. To do so, the instructor goes to Access Records for the class and then clicks on Assignment submission. For the particular assignment, one can click on Display and then click for each student to display submission of the student. There is a Right/Wrong box for each problem to be hand graded.

Anomalous Behavior

There is a problem with expression evaluation caused by peculiarities in the properties of expression evaluation in VBScript, the language on which active server pages is based. The problem is that typically, the exponentiation operation (denoted by a caret) binds more strongly than unary minus. This causes -2^2 to evaluate as 4 rather than -4 as one would normally expect. This is documented behavior. The only workaround is to avoid problems in which it is likely that students might submit answers which trigger this behavior.

Standards

It is possible specify associate standards with problems. To do so, add a square bracket enclosed semicolon separated list o standards to the end of the question header to make a header of the form QM_[...][std1;std2;std3] where the standard entries std1, std2, etc can contain neither semicolons nor square bracket characters.

The form of a standard entry is it is a short name followed by a description. The name part is obtained by taking the part of the standard which occurs before the first colon (or end of the string if there is none). Then one takes the part of this string which occurs before the first space character. (Sorry for the complicated syntax; it is in order to be compatible with all the previous types of standards.

The short name in turn is then split up pieces using the underscore character (_) as the separator. The form is
stdSet_stdStd_level
where stdSet is the name of the standard set, stdStd is the name of the standard within the standard, and level is an integer in the range between 1 and 5 (inclusive). For example,
KY_MAH-1.5.1_2
might be Kentucky Core Standard set (KY), with standard MAH-1.5.1, at level 2 (Procedural level). It is expected that one will add standardized standard set names as they are needed, e.g. the new beta forms of the KY Core standards might need a different name to distinguish them from the final form of the revised Core standard set.

One can use multiple standards sets in the same assignment. For example, I would expect one to use the ADP and the KEMTP standard sets for the KEMTP Jr assignment. It is expected that in any one assignment, either all occurrences of standards from a particular set will use levels or none of them will. Homework results will be summarized in terms of the number of correctly answered questions which refer to particular standards. Each standard set will have its own summarization table.