Zinc is both a client and server HTTP library written and maintained by Sven van Caekenberghe. HTTP clients and servers are each others' mirror: An HTTP client sends a request and receives a response. An HTTP server receives a request and sends a response. Hence the fundamental Zn framework objects are used to implement both clients and servers.
This chapter focuses on the server-side features of Zinc and demonstrates through small, elegant and robust examples some possibilities of this powerful library. The client side is described in Chapter Zinc Client side
Getting an independent HTTP server up and running inside a Pharo image is surprisingly easy.
Don't try this just yet. To be able to see what is going on, it is better to enable logging, as follows:
This starts the default HTTP server, listening on port 1701. We use 1701 in the example because using a port below 1024 requires special OS level privileges, and ports like 8080 might already be in use. Visiting http://localhost:1701 with a browser yields the Zn welcome page. The Transcript produces output related to the server's activities, for example:
You can see the server starting and initializing its server socket on which it listens for incoming connections. When a connection comes in, it starts executing
its request-response loop. Then it gets a
GET request for
/ (the home page), to which it answers a 200 OK response with 997 bytes of HTML. The browser also
asks for a
favicon.ico, which the server supplies. The request-response loop is kept alive for some time and usually closes when the other end does. Although it
looks like an error, it actually is normal, expected behavior.
The example uses the default server: Zn manages a default server to
ease interactive experimentation. The server object is obtained by:
ZnServer default. The default server also survives image save
and restart cycles and needs to be stopped with
ZnServer stopDefault. The Transcript output will confirm what happens:
Due to its implementation, the server will print a debug notification:
Wait for accept timed out, every 5 minutes. Again, although it looks like an error, it is by design and normal, expected behavior.
The functional behavior of a
ZnServer is defined by an object called its delegate. A delegate implements the key method
handleRequest: which gets the
incoming request as parameter and has to produce a response as
result. The delegate only needs to reason in terms of a
ZnResponse. The technical side of being an HTTP server, like
the protocol itself, the networking and the (optional) multiprocessing, is handled by the
This allows us to write what is arguably the simplest possible HTTP server behavior:
Now go to http://localhost:1701 or do:
This server does not look at the incoming request. It always answers
200 OK with a
Hello World!. The
onRequestRespond: method accepts a block that takes a request and that should produce a response. It is implemented using the helper object
value: on a wrapped block.
Out of the box, a
ZnServer will have a certain functionality that is related to testing and debugging. The
ZnDefaultServerDelegate object implements this behavior. Assuming a server is running locally on port 1701, this is the list of URLs that are available.
/, equivalent to
/form-test-3are form test pages
The random handler normally returns 64 characters, you can specify your own size as well. For example,
/random/1024 will respond with a 1Kb random string.
The random pattern consists of hexadecimal digits and ends with a linefeed. The standard, slower UTF-8 encoding is used instead of the faster LATIN-1 encoding.
The bytes handler has a similar size option. Its output is in the form of a repeating BCDA pattern. When requesting equally sized byte patterns repeatably, some extra server side caching will improve performance.
The echo handler is used extensively by the unit tests. It not only lists the request headers as received by the server, but even the entity if there is one. In case of a non-binary entity, the textual contents will be included. This is really useful to debug PUT or POST requests.
In general, to help in debugging a server, enabling logging is important to learn what is going on. Breakpoints can be put anywhere in the server, but interrupting a running server can sometimes be a bit hard or produce strange results. This is because the server and its spawned handler subprocesses are different from the UI process.
When logging is enabled, the server will also keep track of the last request and response it processed. You can inspect these to find out what happened, even if there was no debugger raised.
Similar to the delegate, a
ZnServer also has an authenticator object whose function is to authenticate requests. An authenticator has to implement the
authenticateRequest:do: method whose first argument is the incoming request and second argument a block. This method has to produce a response, like
handleRequest: does. If the request is allowed, the block should be evaluated, which will produce the response. If the request is denied, the authenticator should generate a 401 Unauthorized response. One simple authenticator is available to add basic HTTP authentication:
Now, when you try to visit the server at http://localhost:1701 you will have to provide a username and password. Note that it is also possible to use
ZnEasy to send
a get request to this URL with these credentials.
ZnBasicAuthenticator or implementing an alternative authenticator is only one of several possibilities to address the problem of adding security to a web site or web application.
Log output consists of a log message preceded by a number of fixed fields. Here is an example of a server log.
The first two fields are the date and time in a fixed sized format. The next field is the id of the log entry. The next number is a fixed sized hash of the process ID. Note how 3 different processes are involved: the one starting the server (probably the UI process), the actual server listening process, and the client worker process spawned to handle the request.
ZnServer implement logging using a similar mechanism based on the announcements framework.
ZnLogEvents are subclasses of the
Announcement class and are
sent by an HTTP server or client containing logging information. A log event has a TimeStamp, an id, and a message.
To log something, a server or client uses its own log methods.
For example, a server receives a
logConnectionAccepted: message with the socket that will process the request as argument. In
ZnSingleThreadedServer, the implementation of
This logging mechnism can be easily customized by implementing subclasses of
ZnLogEvent. For example,
ZnConnectionAcceptedEvent is a subclass of
ZnLogEvent customized for connection acceptation.
You can also provide your own listener for
The following example shows how to log events in a file named
zn.log, next to the image.
The class side of
ZnServer is actually a factory to instantiate a particular concrete
ZnServer subclass, as can be seen in
hierarchy looks as follows.
ZnServer is an abstract class.
ZnSingleThreadedServer implements the core server functionality. It runs in one single process, which means it can only
handle one request at a time, making it easier to understand and debug.
ZnMultiThreadedServer spawns a new process on each incoming request, possibly
handling multiple request/response cycles on the same connection.
ZnManagedMultiThreadedServers keeps explicit track of which connections are alive so that
they can be stopped when the server stops instead of letting them die out.
Server instances can be started and stopped using
stop. By registering a server instance, by sending it
register, it becomes managed. That
means it will survive image save and restart. This only happens automatically with the
default server, for other server instances it needs to be enabled manually.
The main parameter a server needs is the port on which it will listen. Additionally, you can restrict the network interface the server should listen on by
bindingAddress: to some IP address. The default, which is
#[0 0 0 0], means to listen on all interfaces. With
#[127 0 0 1], the
server will not respond to requests over its normal network, but only to requests coming from the same host. This is often used to increase security while proxying.
When most people think about a web server, they imagine what is technically called static file serving. There is a directory full of HTML, image, CSS, and other files, somewhere on a machine, and the web server serves these files over HTTP to web browser clients anywhere on the network. This is indeed what Apache does in its most basic form.
Zn can do this by using a
ZnStaticFileServerDelegate. Given a directory and an optional prefix, this delegate will serve all files it finds in that directory, for example:
If we suppose the contents of
You can access these files with these URLs
The prefix is added in front of all files being served, the actual directory where the files reside is of course invisible to the end web user. If no prefix is specified, the files will be served directly.
Note how all other URLs result in a 404 Not found error. Note that while the
ZnStaticFileServerDelegate is very simple, it does have a couple of capabilities. Most importantly, it will do what most
people expect with respect to directories. Consider the following URLs:
The first URL above will result in a redirect to the second. The second URL will look for either an
index.htm file and serve that. Automatic
generation of an index page when there is no index file is not implemented.
As a static file server, the following features are implemented:
Here is a more complex example:
In the above example, we add the optional expiration and caching control based on default settings. Note that it is easy to combine static file serving with logging and authentication.
Dispatching or routing is HTTP application server speak for deciding what part of the software will handle an incoming request. This decision can be made on any of the properties of the request: the HTTP method, the URL or part of it, the query parameters, the meta headers and the entity body. Different applications will prefer different kinds of solutions to this problem.
Zinc HTTP Components is a general framework that offers all the necessary components to build your own dispatcher. Out of the box, there are the different
delegates that we discussed before. Most of these have hand coded dispatching in their
ZnDefaultServerDelegate can be configured to perform dispatching as it uses a prefix map internally that maps URI prefixes to internal methods. Configuration is by installing a block as the value to a prefix, which accepts the request and produces a response. Here is an example of using that capability:
This is taken from the configuration of what runs at http://zn.stfx.eu. A static web server is set up under the
zn prefix pointing to the directory
/home/ubuntu/zn. The prefix map of the default delegate is kept as is, with its standard functionality, but is modified, such that
znprefix is directly forwarded to the static file server
redirect-to-znprefix is set up which will issue a redirect to
/handler is linked to
redirect-to-zninstead of the default
Another option is to use
You configure the dispatcher using
map:to: methods. First argument is the prefix, second argument is a block taking two arguments: the incoming request and
an already instantiated response.
Proper character encoding and decoding is crucial in today's international world. Pharo Smalltalk encodes characters and strings using Unicode. The primary
internet encoding is UTF-8, but a couple of others are used as well. To translate between these two, a concrete
ZnCharacterEncoding subclass like
ZnUTF8Encoder is used.
ZnCharacterEncoding is an extension and reimplementation of regular
TextConverter. It only works on binary input and generated binary output and it adds the
ability to compute the encoded length of a source character, a crucial operation for HTTP. It is more correct and will throw proper exceptions when things go wrong.
Character encoding is mostly invisible. Here are some code snippets using the encoders directly, feel free to substitute any Unicode character to make the test more interesting.
There are no automatic conversions in Zinc, so no defaults are assumed. Instead you should specify a proper Content-Type header including the charset information. Otherwise Zinc has no chance of knowing what to use and the default NullEncoder will make your string wrong.
Consider the following example:
In the first case, a UTF-8 encoded string is POST-ed and correctly returned (in a UTF-8 encoded response).
In the second case, an ISO-8859-1 encoded string is POST-ed and correctly returned (in a UTF-8 encoded response).
In both cases the decoding was done correctly, using the specified charset (if that is missing, the ZnNullEncoder is used). Now, ö is not a perfect test example because its Unicode encoding value is 246 in decimal, U+00F6 in hex, still fits in 1 byte and hence survives null encoding/decoding (it would not be the case with € for example). That is why the following still works, although it is wrong to drop the charset.
Internet facing HTTP servers will come under attack by malicious clients. Good security is thus important. The first step is a correct and safe implementation of the HTTP protocol. Another way a server protects itself is by implementing some resource limits.
Zinc HTTP Components currently implements and enforces the following limits:
Of course these values may be customized if one needs to.
Also, Zn implements two important techniques used by HTTP servers when they send entity bodies to clients: Gzip encoding and chunked transfer encoding. The first one adds compression. The second one is used when the size of an entity is not known up front. Instead chunks of certain sizes are sent until the entity is complete.
All this is handled internally and invisibly. The main object dealing with content and transfer encoding is
ZnEntityReader. When necessary, the binary socket
stream is wrapped with either a
ZnChunkedReadStream and/or a
GZipReadStream. Zn also makes use of a
ZnLimitedReadStream to make sure there is no read
beyond the boundaries of one single request's body, provided the content length is set.
Seaside is a well known, cross platform, advanced Smalltalk web application framework. It does not provide its own HTTP server but
relies on an existing one by means of an adaptor. It works well with Zn, through the use of a
ZnZincServerAdaptor. It comes already included with certain Seaside distributions and on Pharo Smalltalk it is the default.
Starting this adaptor can be done using the Seaside Control panel in the normal way. Alternatively, the adaptor can be started programmatically.
Since Seaside does its own character conversions, the Zn adaptor is configured to work in binary mode for maximum efficiency. There is complete support for POST and PUT requests with entities in form URL, multipart or raw encoding.
There is even a special adaptor that combines being a Seaside adaptor with static file serving, which is useful if you don't like the WAFileLibrary machinery and prefer plain static files served directly.
As a last example of the use of Zinc HTTP, we now show the implementation of REST web services, both the client and the server parts. REST or Representational State Transfer is an architectural style most easily described as using HTTP verbs and URIs to deal with encoded resources. Some kind of framework is needed to successfully implement a non-trivial REST service. There is one available in the Zinc-REST-Server package, for example. Here we will implement a very small, simplified example by hand, for educational purposes.
The service will allow arbitrary JSON objects to be stored on the server, each identified by an URI allocated by the server. Here is the REST API exposed by the server:
A proper implementation should best use a couple of classes. However for brevity, the following implementation is written in a workspace, not using any classes. It requires STON (see Chapter STON) and starts by creating two global variables to hold the stored objects and the last ID used. The former is a standard dictionary mapping string URIs to objects.
The server implementation uses two helper objects: a
jsonEntityBuilder and a
mapper. Both make use of block closures.
jsonEntityBuilder block helps in transforming Smalltalk objects to a JSON entity. We use the STON writer and reader here because they are backwards compatible with JSON. We
use linefeeds to improve compatibility with internet conventions as well as pretty printing to help human interpretation of the data.
The mapper is a dynamically created array of associations (not a dictionary). Each association consists of two blocks. The first block is a condition: it tests a request and returns true when it matches. The second block is a handler that is evaluated with the incoming request to produce a response (if and only if the first condition matched).
The associations in the mapper follow exactly the list of the REST API as shown earlier. The server is set up with a block based delegate using the
onRequestRepond: method. Again, a more object-oriented implementation would use a proper delegate object here, but for this example, the block is sufficient.
The server logic thus becomes: find a matching entry in the mapper and invoke it. If no matching entry is found, we have a bad request. Error handling is of course rather limited in this small example.
Here is an example command line session using the Unix utility curl, interacting with the server.
It is trivial to use
ZnClient to have the same interaction. But we can do better: using a contentWriter and contentReader, we can customise the client to do the JSON conversions automatically.
Now we can hold the same conversation as above, only in this case in terms of real Smalltalk objects.
Zinc HTTP Components was written with the explicit goal of allowing users to explore the implementation. The test suite contains many examples that can serve as learning material. This carefulness while writing Zinc HTTP Components code now enable users to customize it to their need or to build on top of it. Zinc is indeed an extremely malleable piece of software.