End-user computing is the term used when users outside the information systems (IS) department develop software applications for themselves and/or others. The ability of end-users to develop their own applications benefits them in many ways. It gives them greater control over their information needs, more timely results, greater flexibility, and fewer problems communicating with the IS department. There are significant risks associated with end-user computing, however. End-users who develop their own applications usually do not follow formal development procedures. Studies have shown that applications developed by end-users are typically undocumented and frequently contain substantive errors. The possibility of erroneous decisions based on incorrect information has important implications for the end-users and the organizations in which they work. Many IS managers and researchers believe that training end-users in formal systems development methods will improve the quality of their applications.

The purpose of this study was to determine whether training end-users in systems analysis and design techniques will result in higher quality applications. The research question was:

Do end-users develop higher quality applications when they have received systems analysis and design training?

The experiment was conducted through a World Wide Web site. Some MBA graduates from the University of Arkansas and some members of Internet discussion lists related to spreadsheets were contacted by letter or e-mail and asked to participate. Participants provided information through HTML forms throughout the four parts of the experiment. They received their training by downloading a computer-based training module which could be run on their personal computers. A quiz was programmed into the end of the training module as a control check for completion of the training. The participants had to successfully complete the quiz before the training module displayed the information required for the participants to proceed with the experiment.

Eight-six (86) end-users completed the experiment and 73 of these were used in the statistical analyses--36 were in the control group and 37 were in the treatment group.

The following two tables show the average ratings for completeness, accuracy, and design for the pre-test and post-test spreadsheets the study participants developed. The pre-test and post-test were different problems but similar.  For accuracy and completeness the maximum evaluation score possible was 3 and for design it was 6.
 

Assessment of Pre-test Spreadsheet Quality
 

Subjects	Completeness  (3 points were possible)	Accuracy  (3 points were possible)	Design  (6 points were possible)	Composite   Quality Score*
Overall	2.42 (81%)	2.92 (97%)	0.99 (17%)	8.75 (73%)
Treatment	2.43 (81%)	2.95 (98%)	1.11 (18%)	8.88 (74%)
Control	2.42 (81%)	2.89 (96%)	0.87 (15%)	8.63 (72%)

Note. The average score is shown with the percentage of the total score possible in parentheses.

• No significant difference was found between the groups' scores.
• There were 24 non-label cells required for this spreadsheet.
• The composite score weighted the quality measures as follows:  accuracy (50%), completeness (25%), and design (25%).

Average Scores for Quality Components for the Post-test Spreadsheet and Change from Pre-test Scores
 

Subjects	Completeness	Accuracy	Design*	Composite   Quality Score*
Overall  Change	2.63 (88%)  0.21	2.89 (96%)  - 0.03	1.97 (33%)  0.98	9.40 (78%)  0.65
Treatment  Change	2.59 (86%)  0.16	2.83 (94%)  - 0.12	2.97 (50%)  1.86	9.76 (81%)  0.88
Control  Change	2.67 (89%)  0.25	2.94 (98%)  0.05	0.94 (16%)  0.07	9.02 (75%)  0.39

Note. The percentage of maximum possible score is in parentheses.

* Significant difference between the groups at the .01 level.

• There were 25 non-label cells in this spreadsheet.

It is important to note that accuracy was evaluated for original errors in non-label cells.  It is well known that one error in a spreadsheet can have a cascading effect, resulting in erroneous calculations even though the formulas are correct.  In this study propagated errors--where an error is referenced in other formulas--were not counted in this rating.  In other words, if a formula was correct, but referenced another cell with an error, the formula itself was counted as correct.  It should also be noted that one of the design elements graded in these spreadsheets, the use of literal values in formulas instead of cell references, has the potential for creating future errors when a spreadsheet is reused.  Forty-two percent of the spreadsheets created for the pre-test contained literal values for the weights in the weighted rating formulas.  The literal values were keyed in even though many of these spreadsheets also had the weights for the evaluation factors entered in spreadsheet cells.

The statistical analyses found one measure of application quality was significantly related to the type of training end-users received. After the training treatment subjects were more likely to incorporate proper design features, such as documentation, in their applications. The other quality measures--completeness, accuracy, and end-user satisfaction--were not significantly related to the training.  

• Pre-test spreadsheet problem.
• Post-test spreadsheet problem.
• Training modules - Treatment and Control.