Essay
Capturing Data from HTML Forms CGI and PERL
© 2001-2003 Raymond J. Kimball
Please print out and read the four documents linked to the top of this essay - PerlScript Code, Interactive Form Code, Trouble-Shooting, and Uploading to Web4students.
Trouble-Shooting - Common CGI/Perl Scripting Errors
Uploading PERL scripts and Forms to your Web4Students Site
Stating a Problem. Let's say you are asked by the Virginia Department of Motor Vehicles, or the Maryland Motor Vehicle Administration (MVA) to create a form to permit license holders to renew their licenses on-line. How would you do it?
First, you would create a form in HTML. You would load the form to your WWW server, where it could be downloaded and read by the visitor's browser. When the licensee or visitor fills out the form, the form is then submitted to the PERL data-capturing program located on the Motor Vehicle Administration's web site. The PERL program resides in a file in your server at the Virginia DMV or the Maryland MVA.
This brings us to a brief description of "client" and "server" computers and the World Wide Web. Here's how information gets stored and transmitted:
Within the PERL script you could provide a number of instructions to manipulate, sort, store, and do other things with the data, depending on the problems which needed to be solved. Among other things, you would use PERL to:
Now, you could do other things with the data as well. You could send it to another computer for storage in a database software program. You could send the information to a credit card company to validate an account. Scripts to perform these functions are available for free on the Internet. We will limit ourselves to the three functions described above.
Why is CGI/PERL useful? Forms have little value unless the information obtained from them can be stored, used, and manipulated. Data processing is one of the cornerstone uses of computers. Data transmission and retrieval certainly became important in e-commerce once Congress permitted commercial enterprises on the Internet in 1992.
HTML code is not capable of manipulating data. As advertised, it is merely a "text mark-up language." HTML can create the form, but it can't capture or process the data inserted into the form.
That's where PERL comes in. PERL is a separate programming language that provides the data processing tools HTML lacks. PERL is widely used with HTML forms because it uses the same set of protocols (rules) as HTML for transmitting the data between browser and server. That protocol is known as CGI - Common Gateway Interface.
What's CGI? CGI, or "Common Gateway Interface," is a protocol (a set of rules) that sets the communication standards between your client-side browser and the server. Both HTML and PERL use this common gateway to transfer information between client and server computers.
What's PERL script? We use PERL script to capture the HTML form information and transmit it to the server using the CGI protocol. PERL, a robust and flexible programming language which is ideal for capturing and manipulating information, is a common programming languages you may find on the web pages of many commercial clients to capture data from an HTML form. It's not the only one, but it's certainly a popular one, and one with which you should be familiar. For a first introduction to forms, it's relatively easy to use and understand.
In this survey course we introduce you to the PERL language, describe its basic features, and construct a basic form. As you progress, you can come back to this short introduction, absorb it a piece at a time, pick up a book about PERL programming, and start incorporating its power into your web pages if your server provides CGI-BIN capabilities to you as web designer.
You will not have to program PERL, but you should understand it for future reference if you are so inclined. Also, understanding PERL's basic programming concepts will help you write good HTML forms and learn about computer programming concepts. When you "marry up" the HTML code and the PERL script, you will convert your HTML form from a nice exercise in coding into a powerful, interactive, data-capturing tool!
There's other reason PERL is so popular, besides its immense flexibility. It's FREE and widely available on the Internet. For those of you who are starting out as "programmers" and designers, it's easy and inexpensive to find help on PERL, and it's easy to find PERL scripts for you to use and modify for the particular servers with which you may work. PERL scripts are already available on the web, and can be freely downloaded.
If you want to read more about PERL, try Elizabeth Castro's "PERL and CGI for the World Wide Web" by Peachpit Press. This book is by the same author and in the same format as your HTML textbook. It starts "from the beginning", and assumes you have had no prior experience with PERL, or other programming languages for that matter. I'd recommend you start with this one.
The PERL Program. Print out the PERL script linked from the button at the top of the page, and review it as you read this part of the essay.
The PERL program used in class performs the following functions. We'll look at the program line-by-line:
1. Calling the PERL compiler.
What's the first thing we do in an HTML document? We insert an <HTML> tag to let the browser know that this document is written in Hypertext Mark-up Language, which language the browser is designed to read. The same is true for PERL script. The first line starts with a "SHEBANG" line -- a #! set of special characters which alerts the PERL script: "-- SHEBANG! - that a script written for the PERL interpreter/compiler is running.
The next part of that first line:
#!/user/bin/perl
tells the computer the location of the PERL interpreter/compiler which will run the program. You need to specify this location in your PERL document, with the help of your ISP host.
The next few lines, which begin with the # symbol, are comment lines, similar in function to the <!-- --> comment tag you use in HTML. The reserved character #, standing alone tells the PERL compiler that this is a comment, not to be confused with or interpreted as a command for the PERL program to do something.
Library of PERL functions. After the comments, the first line of the code, require 'cgi-lib.pl' calls a library of PERL functions needed to run the program. Your instruction is, essentially, "we require the CGI library of commands connected with PERL (pl) to run the program." So, the perl compiler gets that library of functions. If it's not available, it won't run. If you don't include the line, the compiler won't know what library of functions to use.
You will need to upload the cgi-lib.pl file to your CGI bin. Go to the "Upload" essay at the top of this essay. Click on the cgi-lib.pl link to get the file. Save it to your disk, and upload it with the other files you create.
&ReadParse. ReadParse is one of the command functions within the CGI library you've just called. "Parse" means "to separate into its parts." . The script instructs the PERL program to call the ReadParse function. First, the program reads the information sent; then it and breaks apart the pairs of names and values you have caused the visitor to send upon clicking the submit button.
Names and Values. Where did this name/value pair come from? Well, that's where you come in. The name/value pair comes from your HTML form - and you have to specify it in your form box codes. You haven't done so yet, but now you need to make some additions to your form tags.
You have to modify your various HTML <form> and <input /> tags, etc. to create a name/value format. The visitor to your HTML document who fills out and submits this form will then cause this name/value pair to be submitted to the PERL script's variables when he/she presses the "submit" button.
You will modify your form to construct a name for the form element (text box or radio button) which matches the name of the variables specified in the PERL script. Your visitor sends this name and the corresponding value the user types into the form to the PERL script, rkgbook.cgi, located on the server.
Variables. Variables are "containers" which hold data. A cup can hold either coffee, tea, diet cherry Coke, Pepsi, orange juice, water - whatever liquid we want to pour into it. The cup can even hold solids - M&M's, plain or peanut - you get the idea.
In data processing the variable named "address" might return your address at work or my home address, depending what data we type into the form. We then manipulate and process the data by commanding the PERL script to run and process whatever data the variable (cup) may hold when each person submits the form information. The power of PERL is that you are able to write a rather short program that will accept data over and over from the same form. Using the variable names, the PERL program will store and process the submitted data, and do whatever you command in the script.
The named variable in a PERL script is a digital "cup" or "bowl" that holds the data sent by a particular form box. The PERL script we use here creates five variables, which are associated with the names of five corresponding form boxes in our HTML code.
| PERL Variable Name (Created by the script) |
HTML Formbox Name (You will add this name to your HTML code) |
Value User's data |
| $Gender = | $in{'Gender'}; | User's data |
| $LastNamel = | $in{'LastName'}; | User's data |
| $FirstName = | $in{'FirstName'}; | User's data |
| $Password = | $in{'Passowrd'}; | User's data |
| $State = | $in{'State'}; | User's data |
| $Courses = | $in{'Courses'} | User's data |
| $Comments = | $in{'Comments'} | User's data |
"Hidden" Input Tag and Variable. We also create an additional "hidden" variable called $Book. This variable allows us to "pour" the name of your individual log (NNbook.txt, where NN represents your first and last initials) file into the PERL script, so that the information submitted from your form is printed to your log file, not mine!
So, the data which the user types into HTML text input line named 'Email" will be placed in the PERL container {'Email'}, and then placed in the variable $Email for processing. Then we simply carry the contents of $Email around with us wherever we want, and can use it elsewhere in the program. We can print it to a monitor, store it, sort it, etc.
$Email, thus, is an "object" we have created, into which we place various data from elsewhere in the program, or from another program. We will use this "object", and the data temporarily stored in it, in other parts of the program to process information and perform tasks. This is the essence of object-oriented programming. We'll encounter this concept again when we learn JavaScript later in the semester.
Once we have read, parsed, and placed the data values into our PERL variables, we then command the PERL program to perform two functions.
First, we command PERL to send an HTML message back to the browser (see the series of print lines) that has just submitted the information, and to include in that HTML message the value contained in the $Name variable. This will be the name submitted by the user when he/she filled out the name box on the form.
Second, we store the information in a log file, using the "hidden" variable $Book. As previously discussed, $Book will contain the specified name of your personal text file - but only if you include the correct line of code!. You pass the name of your personal text file from your HTML form to the PERL script, just as the user (at your instruction) passes his/her information to PERL. You will add a line of code that passes the name of your particular text file (NNbook.txt) to the PERL script in a "hidden" name/value pair. The "value" (NNbook.txt) will be passed to variable $Book.
As with other variables, the values in $Book may vary, depending on whose form is being read - yours or mine. If my form is being read, $Book will contain the value RKbook.txt, because that's what I have instructed in my HTML document. If your form is being read, you will code your HTML form to return the value NNbook.txt to the PERL program, where NN represents your first and last initials.
Modifying your HTML Document. Now that you understand the basic PERL form, you need to modify your HTML code to match up the PERL script and HTML code. Here are the things you have to do:
1. Modify your <form> tag at the beginning of your form as follows:
<form action ="http://web4students.montgomerycollege.edu/courses/Fall2003/WDCE/CMP05624654/cgi-bin/tl_form.pl" method="post" name="NNBook" id="NNBook" value="NNBook.txt" onSubmit="return validForm(this)">
(where NN refers to your initials)
This will point the HTML document to the perl file that will run the script. You will use the POST method, which, as we have discussed, will permit you to print an infinite amount of data. The other method, GET, limits you to printing 256 ASCII characters on the browser's address line-and that's not what we want to do.
2. Create a "hidden" input box (i.e., one the browser won't see) by which you will send your text name to the $Book variable in the PERL script :
<input type="hidden" name="Book" id="Book" value="NNbook.txt" /> (where NN=your first and last initials.)
3. Insert the name and id attributes in five of your input tags, and give the input boxes a name value which corresponds to one of the five variable names specified in the PERL script, to wit:
| Input Type | Attribute | Value | Attribute | Value |
| Text | name | = "Gender" | id | = "Gender" |
| Text | name | = "LastName" | id | = "LastName" |
| Text | name | = "FirstName" | id | = "FirstName" |
| Radio | name | = "State" | id | = "State" |
| Textbox | name | = "Courses" | id | = "Courses" |
| Textbox | name | = "Comments" | id | = "Comments" |
Uploading Your Pages.
After you have comleted all your programming changes to the form, you will need to load the following files to the server, and place them in the correct folders:
Setting the Permissions. Read the essay on setting the file "Permissions" once you have uploaded the files to the server.
Testing Your Form; Looking at the Stored Information on the server.OK! That's it! Test your form, and see if it returns a response to your screen. Then, visit your log book and look at the returned answers later next week, to see what information people send you. Type in the URL for your web site (copy it from your browser address line, and then specify the location of your log file. The address will look something like this:
http://web4students.montgomerycollege.edu/courses/Fall2003/WDCE/CMP04925039/student03/NNBook.txt
where NN represents your first and last initials. If your code is working properly, you should see a record of the information that others send you.
If you can't find your log file where the text is stored, and you think your form is working, try this: Type in the name of the file exactly as you have listed it in the HIDDEN INPUT tag you added to your web site. Make sure the name in your HTML code and the file name you type into your URL match.
When you test other student's forms, the "Thank You" Message should be returned to your computer screen upon your submitting the form data.
As you may have guessed, the lab assignment is to create a form that works with the CGI script discussed in class. The form should include the controls identified. The specific questions you ask are up to you but they should work within the parameters of this program.
Post the form and Perl Script files on your web site, set the permissions, test the form, and make sure it works. It should print to your log file and return a message when you submit the information. Link in the form to your home page so others can test it. The form should work by 12 Noon October 6. You will have the next week and next Saturday's lab to work on the form. You are free to use the PERL script and files provided with this essay.
Once you've posted the form and the navigation link the last part of the assignment is to review other class members' assignment number two. You should look at the two students in the third and fourth places below your name in the alphabetical list and visit each of their sites sometime after Thursday morning. If the site does not contain the frame set or does not contain the forms and welcome pages then go on to another student further down the list. In all cases if the person you are to evaluate does not have their page posted in time simply skip that person and find the next one on the list. Just keep moving down the list each week and when you reach the bottom wrap around to the top.
The evaluations should be approximately one paragraph per person and address the specifics of the assignment for that week, as well as general aesthetics. The evaluations should be sent to the entire class so that others can see your evaluations of their sites and I can see that you've completed the evaluations. With each assignment's project grade, I will also indicate if I have seen your prior assignment's evaluations and give you credit for those as well. If you have any questions feel free to contact me.