Back to [ Homepage (Experimental) | Homepage (Official) | Hypertext Refs ]
(May 1999): My PhD thesis is done! The abstract is in my publications list, and I have a thesis in brief webpage too.
Electronic versions of journal are already appearing. I believe that scientific articles with automatically generated hypertext links will be more useful to readers than versions without links. I plan to test that hypothesis as part of my PhD thesis. My overall objective is to review, develop and evaluate ways of automatically incorporating hypertext links into pre-existing long scientific articles. I have chosen to work with lengthy articles because preliminary experiments with short articles showed that hypertext links did little to improve them. My proposal document presents the necessary background to explain (1) why I think an automatic hypertext generator will be useful, and (2) the various decisions I have made in developing a plan to make and test one.
[forward to `Evaluation']
I will generate HTML versions of scientific journal articles. The articles will contain three types of links:
Each link type will be created by an application of a rule. The quality of the links will depend on how good the rules used to create them are and how well those rules are applied.
Structural and definition links will be created mainly through the interpretation of mark up in the text, e.g. defined terms are printed in italics. Semantic links will be generated two ways, each using a different information retrieval system:
[back to `Link creation']
Once I have made the links, I will perform tests to evaluate how useful such links are to readers. If the links are useful then I'll be able to conclude that the rules used to make them could be successfully applied to other documents. If the links are not useful then I'll need to know why they were not. In either case I expect to be able to tell by debriefing the people who use the hypertexts, i.e. the readers.
I will evaluate the performance of the system for reader comprehension (to ensure that the system is really helpful) and a measure of readers' satisfaction with the rules used to build links. I will collect some additional information about about the readers in case such information is needed for future work.
Each reader will read three HTML documents: one with links created using SMART, one with links created using LSI, and one with only structural links. The third version will serve as a control for the other two. I will use a total of six documents. No reader will see two versions of the same document and all orders of presentation will be used.
While they read each of the documents, readers will perform a comprehension task such as briefly summarising the document. These summaries will later be scored by independent experts. After they have read each of the documents, the readers will rate the usefulness and appropriateness of each link that they followed.
I want to determine how well the rules worked and how well the two programs are at detecting when to apply the rules. To test the hypothesis that with document structure, reading articles with hypertext links will improve the reader's satisfaction rating and comprehension and reduce reading time I will use a two-factor within subjects design. The two factors will be document structure and article group. The hypothesis that semantic, definitional and structural links will have differential user quality ratings, will be tested with a one-way unequal n ANOVA design for analysis of user preferences for link-making rules. A fractional repeated measures design will be used.
The much more detailed proposal was accepted on 7 November 1996 following a public lecture and in camera exam.
I defended the completed thesis on 12 April 1999 and will graduate in June 1999. A copy of the completed thesis, now titled Hypertext Versions of Journal Articles: Computer-aided linking and realistic human-based evaluation, is available for download from the UWO Computer Science Department's FTP site. The file is in Postscript form and compressed with the gzip program.
I discussed my proposal in my paper and presentation entitled A design for the construction and evaluation of an automatic hypertext generator at the 1997 conference of the Canadian Association for Information Science/Association canadienne des sciences de l'information (CAIS/ACSI) conference on 10 June 1997. (I noted at the beginning of my talk that I preferred to think of it as `Hypertext versions of journal articles'. I use the longer form because I believe it is more descriptive and therefore easier to find.) The conference was held in conjunction with Congress of Learned Societies/Congrès des sociétés savantes in St. John's, Newfoundland. More details about the conference might be available from the Congress web page.
I discussed the results of some pilot tests and refinements of my method in a paper entitled Evaluating Automatically Generated Hypertext Versions of Scholarly Articles at a workshop at the ACM Computer-Human Interaction conference on 19 April 1998. The workshop was entitled Hyped-Media to Hyper-Media: Toward Theoretical Foundations of Design, Use and Evaluation.
This document is copyright by its author, J. Blustein.