Lisp web templating

roxbase is my minimal, rather simple yet quite versatile Template System for Common Lisp:

It isn't yet feature-complete and first and foremost intended for my own use. I have only 'tested' (used) it under SBCL.

Purpose

What do I mean by Template System? I wanted something similar to Edi Weitz' (of Hunchentoot fame) HTML-TEMPLATE, but more akin to ASP.NET than that Perl templating module which inspired aforementioned HTML-TEMPLATE.

So the basic idea is to embed Lisp code in text files (think HTML pages) and process those files so that each Lisp expression be replaced with the result of evaluating it.

How it works

The most obvious use for roxbase is within a web context. The system integrates well with Hunchentoot, this website for one runs on Hunchentoot and roxbase. However, you can use within any other kind of context. roxbase only provides some basic string processing logic as described below.

Once you have set it up, you can feed roxbase templates such as this:

<a href="<%= (my-url-function (random 50)) %>">
<%@ 'link-caption %>
</a>

Types of template expressions

There are two kinds of expressions that are being processed by roxbase:

Two and two makes <%= (+ 2 2) %>

 Let's loop <b>ten</b> times:
 <? (dotimes (val 10) ?>
 <div>
times <i>two</i>: <?= (* val 2) ?>
</div>
 <? ) ?>
 That's it, thanks! 

The process-template function first processes all mix-in expressions it finds, then all inline expressions. You can have your mix-in or inline expressions generate new mix-in or inline expressions on the fly, making for neat and possibly hard-to-debug, macro-like, recursive templating: if as the result of processing, new mix-in or inline expressions are present in the intermediate output, it repeats this process until the template has been completely processed (tail-recursively, of course).

(Beware of creating endless loops when dynamically generating new mix-in or inline expressions!)

Inline Expressions (InLexes)

The process-template function processes all inline expressions (InLexes) it finds in the template it is given. InLexes start with <% (unless you change the value of +inlex-start-tag+) and end with %> (unless you change the value of +inlex-end-tag+). In addition, <% must be followed directly by a character that identifies the kind of InLex you're writing. The following InLexes are already built into roxbase. Needless to say, you can define your own. Only the first three InLex handlers really come with their own specific evaluation logic, the others translate into more complex InLex expressions of <%= or <%%.

<%= (+ (random 20) (random 40)) %>

 <%%
 (defun rootpath (args)
 (gethash 'rox-rootpath args))
 %>
 Root path of this web app:
 <%= (rootpath) %> 

<%! (needs-fix) %>

 <%~ "code/my-page-tools"
"code/my-utils" %> 

<%? "foo" %>

<%& "foo" %>

<%$ "foo" %>

<%| "Accept-Content-Type" %>

<%@ "my-arg" %>

<%@ 'my-other-arg %>

<pre><%" "Sample code:
<hr />" %></pre>

alert("<%' "'Bobo'
said \"Hi\"." %>);

<a name="<%: "why not?" %>">
Anchor</a>

<%# "foot.htm"
("msg" "©2008")
("color" "blue") %>

Getting started

Before you can use it, roxbase needs to be initialized. Call the following function:

server-startup

• name — This could be the name of your package or any other name. If you want server-startup to start a Hunchentoot instance, this will be the name of your server.
• port — If you do not want server-startup to start a Hunchentoot instance, use NIL; otherwise, specify the port of your Hunchentoot instance.
• pathname-root — the absolute path of the directory serving as the base path for other relative path specifications used within templates or during a template processing request.
• script-path-handler: designates a function that accepts three arguments and returns the path (absolute or relative to pathname-root) to the file that should be processed by roxbase.
  • script-name — the request path for the processing request
  • args — the Arguments hash table for the processing request (template processing arguments are described further below)
  • subtemplate-p — T if this processing request was invoked recursively during a parent template processing.
• &key foldername-static — the path (relative to pathname-root) for files that should not be template-processed by roxbase.
• &key debug — unless port is NIL, sets the Hunchentoot variables *show-lisp-errors-p* and *show-lisp-backtraces-p* to the specified value (should be Boolean — NIL or T).
• &key scriptpackage — unless NIL, all <%% directives will be prepared with an (in-package) call before processing, where the package is either the value of scriptpackage or of name, if scriptpackage's value is T.

As described above, the port argument implicitly controls whether server-startup starts up and returns a new Hunchentoot server instance. If this is the case, roxbase sets up its own Hunchentoot request handlers that either simply return a file unprocessed if the request path fits the directory specification (relative to pathname-root) in foldername-static, otherwise, it invokes the specified script-path-handler function and processes the file represented by the path (relative to pathname-root) returned by the function call.

server-shutdown

After you're done with your template processing, you can call the server-shutdown function to clean things up. If your previous server-startup call initialized a new Hunchentoot server instance (that is still stored in the global variable *rox-server*), the service is stopped. Afterwards, the following global hash tables are cleared:

• *rox-inlex-cache* — contains cached compiled equivalents of successfully parsed inline expressions (InLexes)
• *rox-inlex-handlers* — contains all known InLex handlers (InLexes and InLex handlers are described further below); pre-populated with the roxbase system InLex handlers by server-startup
• *rox-settings* — pre-populated by server-startup with the following default values:
  • 'fn-html-condition-handler (a function designator, by default #'default-html-condition-handler, that produces output to indicate an error in place of a processed InLex and accepts three arguments:
    • condtype — either "EVAL", "COMPILE" or "RUNTIME", depending on when the InLex error was encountered
    • condsrc — the complete InLex that caused the error
    • condmsg — the error message)
  • and 'eval-compile-cache-handlers-p (the string of InLex handler identifiers whose expressions should be compiled and cached in *rox-inlex-cache*).

Invoking template processing requests

handle-request, create-args-table, process-template, load-template

If you're using roxbase with Hunchentoot, each non-static request is handled by handle-request, which returns a string that is the result of processing the file indicated by the script-path-handler you specified when you called server-startup. You can call handle-request yourself, it has the same signature as a script-path-handler. In case you ever need to do just that, pass the following arguments:

• script-name — pass any value that your script-path-handler will transform into the path (absolute or relative to pathname-root) of the file to be processed
• args — pass a hash table freshly created by the create-args-table function:
  • pathname-root — just pass the same path name you passed server-startup
  • script-path-handler — just pass the same function you passed server-startup.
  • parent-args — just pass NIL
  • root-args — just pass NIL.
• subtemplate-p — just pass NIL (this should only ever be T if called automatically from a parent template processing request)

If handle-request finds the specified file, it loads all its raw, unprocessed string contents into memory and passes it to the process-template and returns the result of that call; otherwise, it returns NIL and, when using Hunchentoot, sends a 404 status code prior to returning.

Unless called manually, handle-request is called either in response to a Hunchentoot web request or by the load-template macro, which is a syntactically compact way to recursively load sub-templates during a parent template processing request.