Difference between revisions of "Documents/uHTTP Lib Forms"

From Nutwiki
Jump to: navigation, search
(Created page with "<div id="content"> = MicroHTTP Library: Basic Functions = This page gives you an overview of the application programming interface (API) provided by the uhttplib.html|Micr...")
 
 
Line 1: Line 1:
 
<div id="content">
 
<div id="content">
  
= MicroHTTP Library: Basic Functions =
+
= MicroHTTP Library: Common Gateway Interface =
  
This page gives you an overview of the application programming interface (API) provided by the [[uhttplib.html|MicroHTTP library]].
+
Like its predecessor, the [[uhttplib.html|MicroHTTP library]] supports CGI functions. This still works a in most simple way.
  
== Initialization ==
+
* The application registers a C function as a CGI script.
 +
* When this CGI script is requested, the webserver calls the registered C function.
 +
* The C function sends the content.
  
While Nut/OS tries to avoid initialization calls as far as possible, other operating systems and TCP/IP stacks may require them. Applications should therefore call
+
== HTML Forms With GET Method ==
  
<pre class="coding">int StreamInit(void);</pre>
+
CGI is most often used to request user input via HTML forms. The following form lets the user to enter his first and his family name:
before using any other function of the library. The Nut/OS version of this function simply returns 0 to indicate, that everything is just fine. And Nut/OS applications will run just fine without this call. However, this may change in later versions and it is highly recommended to include this call in any webserver application, even if you do not intend to run it on another operating system than Nut/OS.
+
  
As explained previously, the old library expected the application to establish the TCP connection. This has been changed in the MicroHTTP library to maintain portability. All this TCP stack related stuff is now handled in
+
<pre class="coding">&lt;html&gt;
 +
&lt;head&gt;
 +
&lt;title&gt;Form Sample&lt;/title&gt;
 +
&lt;/head&gt;
 +
&lt;body&gt;
 +
&lt;h1&gt;A Form&lt;/h1&gt;
 +
&lt;form action=&quot;getform.cgi&quot; method=&quot;get&quot;&gt;
 +
  &lt;p&gt;First name:&lt;br&gt;&lt;input name=&quot;firstname&quot; type=&quot;text&quot; size=&quot;30&quot; maxlength=&quot;30&quot;&gt;&lt;/p&gt;
 +
  &lt;p&gt;Family name:&lt;br&gt;&lt;input name=&quot;familyname&quot; type=&quot;text&quot; size=&quot;30&quot; maxlength=&quot;40&quot;&gt;&lt;/p&gt;
 +
  &lt;p&gt;&lt;input type=&quot;submit&quot; value=&quot;Send&quot;&gt;&lt;/p&gt;
 +
&lt;/form&gt;
 +
&lt;/body&gt;
 +
&lt;/html&gt;</pre>
 +
When the ''Send'' button is clicked, the browser will request the CGI script ''getform.cgi'' with the GET method. This means, that the values of the input fields are appended as arguments to the requested URL as ''name=value'' pairs. The webserver then passes this URL to the CGI function.
  
<pre class="coding">int StreamClientAccept(HTTP_CLIENT_HANDLER handler, const char *params);</pre>
+
To simplify CGI function programming, the MicroHTTP library provides a set of useful functions to parse the URL for parameters.
The first argument, the handler for incoming HTTP request, can be set to the standard handler ''HttpdClientHandler'' that is provided by the library. The argument ''params'' can be used to pass special arguments to the server routine, like the TCP port number. If a NULL pointer is passed instead, defaults will be used, for example, TCP port 80.
+
  
Note, that this function will only return on fatal errors, e.g. when the system is running out of memory. Once called, it will handle all HTTP requests from connecting clients. There is just one problem: It will send all data with ''Content-Type: text/plain''. Most webbrowsers will then display HTML source code instead of rendered content. Thus, before calling the function above, you should first call
+
* HttpArgParseFirst() returns the name of the first argument.
 +
* HttpArgParseNext() returns the name of the next argument.
 +
* HttpArgValue() returns the value of the current argument.
  
<pre class="coding">int MediaTypeInitDefaults(void);</pre>
+
When using these functions, processing the HTML form given above is quite simple. Here is the complete code:
This will setup a minimum set of mime types. For Nut/OS, they are configurable with the Configurator. On a PC, where plenty of memory is available, all major media types will be enabled by default. At the time of this writing, these are
+
  
* text/xml
+
<pre class="coding">int CgiGetForm(HTTPD_SESSION *hs)
* text/plain
+
{
* image/svg+xml
+
    char *arg;
* text/html
+
    char *val;
* image/png
+
    char *first = NULL;
* application/pdf
+
    char *last = NULL;
* image/jpeg
+
* application/x-javascript
+
* application/x-java-archive
+
* text/html
+
* image/gif
+
* text/css
+
* image/bmp
+
  
As a result, a minimal webserver can be implemented with three lines of code:
+
    for (arg = HttpArgParseFirst(&amp;hs-&gt;s_req); arg; arg = HttpArgParseNext(&amp;hs-&gt;s_req)) {
 +
        val = HttpArgValue(&amp;hs-&gt;s_req);
 +
        if (val) {
 +
            if (strcmp(arg, &quot;firstname&quot;) == 0) {
 +
                first = strdup(val);
 +
            }
 +
            else if (strcmp(arg, &quot;familyname&quot;) == 0) {
 +
                last = strdup(val);
 +
            }
 +
        }
 +
    }
 +
    SendResult(hs, first, last);
 +
    free(first);
 +
    free(last);
  
<pre class="coding">StreamInit();
+
    return 0;
MediaTypeInitDefaults();
+
}</pre>
StreamClientAccept(HttpdClientHandler, NULL);</pre>
+
The original Nut/OS HTTP library called CGI functions with a FILE stream pointer and a pointer to a request structure. MicroHTTP instead passes a single pointer to a ''HTTPD_SESSION'' structure, which, among other things, contains a HTTP_STREAM pointer and a slightly different request structure. Nevertheless, it shouldn't be too hard to port existing CGI functions to the new library.
This simply webserver will handle all GET requests for files. On Nut/OS, it will use the previously configured file system, which is UROM by default.
+
  
I will later explain how to implement all the fancy stuff like processing HTML forms, receiving file uploads or creating dynamic content. To be able to do this, the application must implement its own handlers, which need to deal with the TCP/IP stream of the established connection. The next chapter will give an introduction.
+
Please note, that the parser functions simply return pointers to a static buffer, which will be overridden when parsing the next argument. Therefore, the CGI function must make a local copy (strdup) of each string which will be used afterwards.
  
== TCP/IP Streams ==
+
Some of you may already have noticed, that the code is not as complete as it pretends to be. Indeed, the response to the webbrowser is missing. The MicroHTTP webserver expects, that the complete response is sent by the CGI function, including all header lines. This is done in the function ''SendResult'' that is called near the end of the CGI function. To simplify this task, two functions are provided by the library to cover standard responses.
  
Standard HTTP is based on TCP/IP streams. The server is listening on a TCP socket, to which the client (webbrowser) connects. As soon as the connection is established, both parties send and receive data via a stream socket.
+
* void HttpSendHeaderTop() sends initial header lines, HTTP version and the name of the server.
 +
* void HttpSendHeaderBottom() sends final header lines: Content type and length, connection type and content encoding, depending on the configuration of the library.
  
Nut/OS allows to attach stream sockets to standard C I/O-streams (FILE streams). This is a nice feature, because you can use C stdio functions like fprintf or fgets to send and receive data to and from the remote host. Actually, the old HTTP library didn't use any network specific API. Instead it expects, that the applications establishes the connection, attaches a FILE stream to the socket connection and passes a stream file pointer to the HTTP library functions.
+
Splitting HTTP header line transmission into two functions enables the CGI to send additional lines between both calls. The latter function will also send an empty line to terminate the HTTP header. The CGI may directly append the content thereafter. Here is the source code of the missing function:
  
While being convenient for the developer, it is less optimal for the target system. For example, reading a header line up to the next carriage-return/linefeed requires to call fgetc() for each character, which in turn calls a number of lower level functions. Quite a lot of overhead. As long as only a few header lines need to be processed by the embedded webserver application on a GET request, this may be acceptable. Performance problems were raised with the implementation of the POST method, specifically when posting large chunks of binary data. File uploading became unacceptable slow.
+
<pre class="coding">int SendResult(HTTPD_SESSION *hs, char *first, char *last)
 +
{
 +
    static const char head[] =
 +
        &quot;&lt;html&gt;&quot;
 +
        &quot;&lt;head&gt;&quot;
 +
        &quot;&lt;title&gt;Form Result&lt;/title&gt;&quot;
 +
        &quot;&lt;/head&gt;&quot;;
 +
    static const char body[] =
 +
        &quot;&lt;body&gt;&quot;
 +
        &quot;&lt;p&gt;Hello %s %s!&lt;/p&gt;&quot;
 +
        &quot;&lt;a href=\&quot;/index.html\&quot;&gt;back&lt;/a&gt;&quot;
 +
        &quot;&lt;/body&gt;&quot;
 +
        &quot;&lt;html&gt;&quot;;
  
To avoid this bottleneck, the new library implements its own input functions.
+
    HttpSendHeaderTop(hs, 200);
 +
    HttpSendHeaderBottom(hs, &quot;text&quot;, &quot;html&quot;, -1);
  
<pre class="coding">int StreamReadUntilChars(HTTP_STREAM *sp, const char *delim, const char *ignore, char *buf, int siz);
+
    s_puts(head, hs-&gt;s_stream);
int StreamReadUntilString(HTTP_STREAM *sp, const char *delim, char *buf, int siz);</pre>
+
    s_printf(hs-&gt;s_stream, body, first, last);
The first function reads data from a stream until any of the characters in the string delim appears. It further allows to skip all the characters contained in the string ignore. For example, reading a header line can be done by
+
    s_flush(hs-&gt;s_stream);
  
<pre class="coding">got = StreamReadUntilChars(stream, &quot;\n&quot;, &quot;\r&quot;, buffer, avail);</pre>
+
    return 0;
Note that HTTP header lines are terminated by a carriage return follow by a linefeed character. With the call above, all linefeeds are ignored and the read stops at the carriage return, which will be replaced by string terminator. As a result, the buffer will contain the pure header line without any line ending character. The parameter ''avail'' tells the function, how many bytes to read at maximum. With header lines, this will be typically limited to the buffer size. It is somewhat more interesting, when the function is used to retrieve the data of a POST request. The application will have parsed the ''Content-Length'' header of the POST request and therefore knows, how many bytes the body will contain. To keep track of this figure, the function returns the number of bytes consumed, which is typically more than the number of bytes stored in the buffer.
+
}</pre>
 +
Of course, this code could have been included into the CGI function. We use a separate function here, so we can reuse it when explaining the POST method in the next chapter.
  
The second function is similar, but reads from the stream until a specified string appears. This comes quite handy when reading multipart MIME contents, as they appear in file uploads. The following call may be used to read all data until the next boundary string, which had been given by the ''Content-Type'' header.
+
== HTML Forms With POST Method ==
  
<pre class="coding">got = StreamReadUntilString(stream, delim, data, avail);</pre>
+
The first problem you may encounter with the GET method is its size limitation. MicroHTTP is designed to run on tiny embedded systems with a 32 kilobytes of RAM or less. As it is often required to process all HTTP headers before parsing the URL, which always comes in the first line, there may not be enough memory space for large forms. Furthermore, even webbrowsers, which typically occupy megabytes or gigabytes of memory, do have a tight limit here. After the first release of HTTP routines in Nut/OS with POST method capabilities, application developers missed this feature immediately.
In the current implementation, the stream parameter of type HTTP_STREAM is no longer a FILE pointer, as it has been in the old library. Therefore, it cannot be used with C stdio functions. For reading, this is no big deal, because the new functions are much more convenient for handling HTTP streams. But it's a pity for writing to the stream, because the C standard functions were just fine and you are probably most familiar with them and don't want to learn, how to use new ones. Luckily, a set of output functions is provided, which is similar to the standard. They just have slightly different names and accept a HTTP_STREAM instead of a FILE stream pointer.
+
  
<pre class="coding">int s_write(const void *buf, size_t size, size_t count, HTTP_STREAM *sp);
+
To instruct the browser to use POST, we simply need to replace a single line in our HTML document.
int s_puts(const char *str, HTTP_STREAM *sp);
+
int s_printf(HTTP_STREAM *sp, const char *fmt, ...);
+
int s_flush(HTTP_STREAM *sp);</pre>
+
One additional function, which is not part of the C stdio, is
+
  
<pre class="coding">int s_vputs(HTTP_STREAM *sp, ...);</pre>
+
<pre class="coding">&lt;form action=&quot;getform.cgi&quot; method=&quot;get&quot;&gt;</pre>
This function writes a variable number of strings, which is quite useful with HTTP, where a mixture of constant and variable strings are often sent in one line. For example
+
must be replaced by
  
<pre class="coding">time_t now = time(NULL);
+
<pre class="coding">&lt;form action=&quot;postform.cgi&quot; method=&quot;post&quot;&gt;</pre>
s_vputs(stream, &quot;Date: &quot;, Rfc1123TimeString(gmtime(&amp;now)), &quot; GMT\r\n&quot;, NULL);</pre>
+
Actually, the CGI function doesn't require many modifications. The most obvious one is, that it needs to keep track of the content length. Futhermore, ''HttpArgParseFirst()'' is not required, ''HttpArgParseNext()'' is replaced by ''HttpArgReadNext()''. The value is retrieved by the same function as used with the GET method, ''HttpArgValue()''.
sends a complete ''Date'' header line. While the same could have been done with
+
  
<pre class="coding">s_printf(stream, &quot;Date: %s GMT\r\n&quot;, Rfc1123TimeString(gmtime(&amp;now)));</pre>
+
<pre class="coding">static int CgiPostForm(HTTPD_SESSION *hs)
the function ''s_vputs()'' is faster and much more simple, because no format string needs to be parsed. In any case, never forget to add a final NULL pointer to the list of arguments. Otherwise ''s_vputs()'' may send a lot of garbage to the server and may even crash your application.
+
{
 +
    char *arg;
 +
    char *val;
 +
    char *first = NULL;
 +
    char *last = NULL;
 +
    long avail;
 +
 
 +
    avail = hs-&gt;s_req.req_length;
 +
    while (avail) {
 +
        arg = HttpArgReadNext(hs, &amp;avail);
 +
        if (arg) {
 +
            val = HttpArgValue(&amp;hs-&gt;s_req);
 +
            if (val) {
 +
                if (strcmp(arg, &quot;firstname&quot;) == 0) {
 +
                    first = strdup(val);
 +
                }
 +
                else if (strcmp(arg, &quot;familyname&quot;) == 0) {
 +
                    last = strdup(val);
 +
                }
 +
            }
 +
        }
 +
    }
 +
    SendResult(hs, first, last);
 +
    free(first);
 +
    free(last);
 +
 
 +
    return 0;
 +
}</pre>
 +
For the response we can use the same function ''SendResult()'', that had been used in the previous chapter above.
 +
 
 +
== Webserver Code ==
 +
 
 +
The webserver is started in the same way as in the [[uhttplib_basic.html|minimal sample]]:
 +
 
 +
<pre class="coding">StreamInit();
 +
MediaTypeInitDefaults();
 +
HttpRegisterCgiFunction(&quot;getform.cgi&quot;, CgiGetForm);
 +
HttpRegisterCgiFunction(&quot;postform.cgi&quot;, CgiPostForm);
 +
HttpRegisterMediaType(&quot;cgi&quot;, NULL, NULL, HttpCgiFunctionHandler);
 +
StreamClientAccept(HttpdClientHandler, NULL);</pre>
 +
Two additional calls have been added to register the two CGI functions, one is used for the GET and another one of the POST method. In addition, we need to register a CGI media type, using the standard handler ''HttpCgiFunctionHandler'' of the MicroHTTP library. This handler will call any registered CGI function when the browser requests an URL with the extension ''cgi''. Further details about media type handlers will be discussed later.
  
 
== Next Step ==
 
== Next Step ==
  
You should now have the basis for [[uhttplib_forms.html|processing HTML forms]].
+
We can use CGIs to create all kind of dynamic content. However, in many cases only parts of a web page need to updated. Creating complete HTML pages within CGI functions may be an annoying job and requires to create a new runtime binary each time the web content is changed. [[uhttplib_ssi.html|Server side includes]] provide a smarter solution.
  
  
 
</div>
 
</div>

Latest revision as of 10:31, 13 July 2017

MicroHTTP Library: Common Gateway Interface

Like its predecessor, the MicroHTTP library supports CGI functions. This still works a in most simple way.

  • The application registers a C function as a CGI script.
  • When this CGI script is requested, the webserver calls the registered C function.
  • The C function sends the content.

HTML Forms With GET Method

CGI is most often used to request user input via HTML forms. The following form lets the user to enter his first and his family name:

<html>
<head>
<title>Form Sample</title>
</head>
<body>
<h1>A Form</h1>
<form action="getform.cgi" method="get">
  <p>First name:<br><input name="firstname" type="text" size="30" maxlength="30"></p>
  <p>Family name:<br><input name="familyname" type="text" size="30" maxlength="40"></p>
  <p><input type="submit" value="Send"></p>
</form>
</body>
</html>

When the Send button is clicked, the browser will request the CGI script getform.cgi with the GET method. This means, that the values of the input fields are appended as arguments to the requested URL as name=value pairs. The webserver then passes this URL to the CGI function.

To simplify CGI function programming, the MicroHTTP library provides a set of useful functions to parse the URL for parameters.

  • HttpArgParseFirst() returns the name of the first argument.
  • HttpArgParseNext() returns the name of the next argument.
  • HttpArgValue() returns the value of the current argument.

When using these functions, processing the HTML form given above is quite simple. Here is the complete code:

int CgiGetForm(HTTPD_SESSION *hs)
{
    char *arg;
    char *val;
    char *first = NULL;
    char *last = NULL;

    for (arg = HttpArgParseFirst(&hs->s_req); arg; arg = HttpArgParseNext(&hs->s_req)) {
        val = HttpArgValue(&hs->s_req);
        if (val) {
            if (strcmp(arg, "firstname") == 0) {
                first = strdup(val);
            }
            else if (strcmp(arg, "familyname") == 0) {
                last = strdup(val);
            }
        }
    }
    SendResult(hs, first, last);
    free(first);
    free(last);

    return 0;
}

The original Nut/OS HTTP library called CGI functions with a FILE stream pointer and a pointer to a request structure. MicroHTTP instead passes a single pointer to a HTTPD_SESSION structure, which, among other things, contains a HTTP_STREAM pointer and a slightly different request structure. Nevertheless, it shouldn't be too hard to port existing CGI functions to the new library.

Please note, that the parser functions simply return pointers to a static buffer, which will be overridden when parsing the next argument. Therefore, the CGI function must make a local copy (strdup) of each string which will be used afterwards.

Some of you may already have noticed, that the code is not as complete as it pretends to be. Indeed, the response to the webbrowser is missing. The MicroHTTP webserver expects, that the complete response is sent by the CGI function, including all header lines. This is done in the function SendResult that is called near the end of the CGI function. To simplify this task, two functions are provided by the library to cover standard responses.

  • void HttpSendHeaderTop() sends initial header lines, HTTP version and the name of the server.
  • void HttpSendHeaderBottom() sends final header lines: Content type and length, connection type and content encoding, depending on the configuration of the library.

Splitting HTTP header line transmission into two functions enables the CGI to send additional lines between both calls. The latter function will also send an empty line to terminate the HTTP header. The CGI may directly append the content thereafter. Here is the source code of the missing function:

int SendResult(HTTPD_SESSION *hs, char *first, char *last)
{
    static const char head[] =
        "<html>"
        "<head>"
        "<title>Form Result</title>"
        "</head>";
    static const char body[] =
        "<body>"
        "<p>Hello %s %s!</p>"
        "<a href=\"/index.html\">back</a>"
        "</body>"
        "<html>";

    HttpSendHeaderTop(hs, 200);
    HttpSendHeaderBottom(hs, "text", "html", -1);

    s_puts(head, hs->s_stream);
    s_printf(hs->s_stream, body, first, last);
    s_flush(hs->s_stream);

    return 0;
}

Of course, this code could have been included into the CGI function. We use a separate function here, so we can reuse it when explaining the POST method in the next chapter.

HTML Forms With POST Method

The first problem you may encounter with the GET method is its size limitation. MicroHTTP is designed to run on tiny embedded systems with a 32 kilobytes of RAM or less. As it is often required to process all HTTP headers before parsing the URL, which always comes in the first line, there may not be enough memory space for large forms. Furthermore, even webbrowsers, which typically occupy megabytes or gigabytes of memory, do have a tight limit here. After the first release of HTTP routines in Nut/OS with POST method capabilities, application developers missed this feature immediately.

To instruct the browser to use POST, we simply need to replace a single line in our HTML document.

<form action="getform.cgi" method="get">

must be replaced by

<form action="postform.cgi" method="post">

Actually, the CGI function doesn't require many modifications. The most obvious one is, that it needs to keep track of the content length. Futhermore, HttpArgParseFirst() is not required, HttpArgParseNext() is replaced by HttpArgReadNext(). The value is retrieved by the same function as used with the GET method, HttpArgValue().

static int CgiPostForm(HTTPD_SESSION *hs)
{
    char *arg;
    char *val;
    char *first = NULL;
    char *last = NULL;
    long avail;

    avail = hs->s_req.req_length;
    while (avail) {
        arg = HttpArgReadNext(hs, &avail);
        if (arg) {
            val = HttpArgValue(&hs->s_req);
            if (val) {
                if (strcmp(arg, "firstname") == 0) {
                    first = strdup(val);
                }
                else if (strcmp(arg, "familyname") == 0) {
                    last = strdup(val);
                }
            }
        }
    }
    SendResult(hs, first, last);
    free(first);
    free(last);

    return 0;
}

For the response we can use the same function SendResult(), that had been used in the previous chapter above.

Webserver Code

The webserver is started in the same way as in the minimal sample:

StreamInit();
MediaTypeInitDefaults();
HttpRegisterCgiFunction("getform.cgi", CgiGetForm);
HttpRegisterCgiFunction("postform.cgi", CgiPostForm);
HttpRegisterMediaType("cgi", NULL, NULL, HttpCgiFunctionHandler);
StreamClientAccept(HttpdClientHandler, NULL);

Two additional calls have been added to register the two CGI functions, one is used for the GET and another one of the POST method. In addition, we need to register a CGI media type, using the standard handler HttpCgiFunctionHandler of the MicroHTTP library. This handler will call any registered CGI function when the browser requests an URL with the extension cgi. Further details about media type handlers will be discussed later.

Next Step

We can use CGIs to create all kind of dynamic content. However, in many cases only parts of a web page need to updated. Creating complete HTML pages within CGI functions may be an annoying job and requires to create a new runtime binary each time the web content is changed. Server side includes provide a smarter solution.