Doc2.0/Lisp.html

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<title>SWIG and Common Lisp</title>
<link rel="stylesheet" type="text/css" href="style.css">
</head>

<body bgcolor="#ffffff">
<H1><a name="Lisp"></a>25 SWIG and Common Lisp</H1>
<!-- INDEX -->
<div class="sectiontoc">
<ul>
<li><a href="#Lisp_nn2">Allegro Common Lisp</a>
<li><a href="#Lisp_nn3">Common Foreign Function Interface(CFFI)</a>
<ul>
<li><a href="#Lisp_nn4">Additional Commandline Options </a>
<li><a href="#Lisp_nn5">Generating CFFI bindings</a>
<li><a href="#Lisp_nn6">Generating CFFI bindings for C++ code</a>
<li><a href="#Lisp_nn7">Inserting user code into generated files</a>
</ul>
<li><a href="#Lisp_nn8">CLISP</a>
<ul>
<li><a href="#Lisp_nn9">Additional Commandline Options </a>
<li><a href="#Lisp_nn10">Details on CLISP bindings</a>
</ul>
<li><a href="#Lisp_nn11">UFFI </a>
</ul>
</div>
<!-- INDEX -->


<p>
      Common Lisp is a high-level, all-purpose, object-oriented,
      dynamic, functional programming language with long history. 
      Common Lisp is used in many fields, ranging from web development to
      finance, and also common in computer science education.
      There are more than 9 different implementations of common lisp which
      are available, all have different foreign function
      interfaces. SWIG currently supports only the Allegro Common
      Lisp, Common Foreign Function Interface(CFFI), CLisp and UFFI
      foreign function interfaces.  
</p>
<H2><a name="Lisp_nn2"></a>25.1 Allegro Common Lisp</H2>


<p>
  Allegro Common Lisp support in SWIG has been updated to include
  support for both C and C++. You can read about the interface
  <a href="Allegrocl.html#Allegrocl">here</a>
</p>

<H2><a name="Lisp_nn3"></a>25.2 Common Foreign Function Interface(CFFI)</H2>


<p>
      CFFI, the Common Foreign Function Interface, is a portable foreign
      function interface for ANSI Common Lisp systems, similar in
      spirit to UFFI. Unlike UFFI, CFFI requires only a small set of
      low-level functionality from the Lisp implementation, such as
      calling a foreign function by name, allocating foreign memory,
      and dereferencing pointers.  
</p>

<p>
      To run the cffi module of SWIG requires very little effort, you
      just need to run:
</p>
<div class="code"><pre>
swig -cffi -module <i>module-name</i>   <i>file-name</i> 

</pre></div>

<p>
      But a better was of using all the power of SWIG is to write SWIG
      interface files. Below we will explain how to write interface
      files and the various things which you can do with them.
</p>

<H3><a name="Lisp_nn4"></a>25.2.1 Additional Commandline Options </H3>


<p>
The following table list the additional commandline options available for the CLISP module. They can also be seen by using: 
</p>

<div class="code"><pre>
swig -cffi -help 
</pre></div>
<br/>

<table summary="CFFI specific options">
<tr>
 <th> CFFI specific options</th>
</tr>

<tr>
<td>-generate-typedef</td>
<td>If this option is given then defctype will be used to generate<br/>
      shortcuts according to the typedefs in the input.
</td>
</tr>

<tr>
<td>-[no]cwrap</td>
<td>Turn on or turn off generation of an intermediate C file when<br/>
    creating a C interface. By default this is only done for C++ code.
</td>
</tr>

<tr>
<td>-[no]swig-lisp</td>
<td>Turns on or off generation of code for helper lisp macro, functions,
      etc. which SWIG uses while generating wrappers. These macros, functions
      may still be used by generated wrapper code.
</td>
</tr>

</table>

<H3><a name="Lisp_nn5"></a>25.2.2 Generating CFFI bindings</H3>


As we mentioned earlier the ideal way to use SWIG is to use interface
    files. To illustrate the use of it, lets assume that we have a
    file named <i>test.h</i> with the following C code:

<div class="code"><pre>
#define y 5
#define x (y &gt;&gt;  1)

typedef int days;

struct bar {
  short p, q;
    char a, b;
    int *z[1000];
    struct bar * n;
};
  
struct   bar * my_struct;

struct foo {
    int a;
    struct foo * b[100];
  
};

int pointer_func(void (*ClosureFun)( void* _fun, void* _data, void* _evt ), int p);

int func123(div_t * p,int **q[100],int r[][1000][10]);

void lispsort_double (int n, double * array);

enum color { RED, BLUE, GREEN};
</pre></div>

Corresponding to this we will write a simple interface file:
<div class="code"><pre>
%module test

%include "test.h"

</pre></div>

The generated SWIG Code will be:

<div class="targetlang"><pre>
;;;SWIG wrapper code starts here

(cl:defmacro defanonenum (&amp;body enums)
   "Converts anonymous enums to defconstants."
  `(cl:progn ,@(cl:loop for value in enums
                        for index = 0 then (cl:1+ index)
                        when (cl:listp value) do (cl:setf index (cl:second value)
                                                          value (cl:first value))
                        collect `(cl:defconstant ,value ,index))))

(cl:eval-when (:compile-toplevel :load-toplevel)
  (cl:unless (cl:fboundp 'swig-lispify)
    (cl:defun swig-lispify (name flag cl:&amp;optional (package cl:*package*))
      (cl:labels ((helper (lst last rest cl:&amp;aux (c (cl:car lst)))
                    (cl:cond
                      ((cl:null lst)
                       rest)
                      ((cl:upper-case-p c)
                       (helper (cl:cdr lst) 'upper
                               (cl:case last
                                 ((lower digit) (cl:list* c #\- rest))
                                 (cl:t (cl:cons c rest)))))
                      ((cl:lower-case-p c)
                       (helper (cl:cdr lst) 'lower (cl:cons (cl:char-upcase c) rest)))
                      ((cl:digit-char-p c)
                       (helper (cl:cdr lst) 'digit 
                               (cl:case last
                                 ((upper lower) (cl:list* c #\- rest))
                                 (cl:t (cl:cons c rest)))))
                      ((cl:char-equal c #\_)
                       (helper (cl:cdr lst) '_ (cl:cons #\- rest)))
                      (cl:t
                       (cl:error "Invalid character: ~A" c)))))
        (cl:let ((fix (cl:case flag
                        ((constant enumvalue) "+")
                        (variable "*")
                        (cl:t ""))))
          (cl:intern
           (cl:concatenate
            'cl:string
            fix
            (cl:nreverse (helper (cl:concatenate 'cl:list name) cl:nil cl:nil))
            fix)
           package))))))

;;;SWIG wrapper code ends here


(cl:defconstant y 5)

(cl:defconstant x (cl:ash 5 -1))

(cffi:defcstruct bar
	(p :short)
	(q :short)
	(a :char)
	(b :char)
	(z :pointer)
	(n :pointer))

(cffi:defcvar ("my_struct" my_struct)
 :pointer)

(cffi:defcstruct foo
	(a :int)
	(b :pointer))

(cffi:defcfun ("pointer_func" pointer_func) :int
  (ClosureFun :pointer)
  (p :int))

(cffi:defcfun ("func123" func123) :int
  (p :pointer)
  (q :pointer)
  (r :pointer))

(cffi:defcfun ("lispsort_double" lispsort_double) :void
  (n :int)
  (array :pointer))

(cffi:defcenum color
	:RED
	:BLUE
	:GREEN)
</pre></div>

<p>
   The <i>SWIG wrapper</i> code refers to the special code which SWIG
    may need to use while wrapping C code. You can turn on/off the
    generation of this code by using the <i>-[no]swig-lisp</i>
    option. You must have noticed that SWIG goes one extra step to
    ensure that CFFI does not do automatic lispification of the C
    function names. The reason SWIG does this is because quite often
    developers want to build a nice CLOS based lispy API, and this one
    to one correspondence between C function names and lisp function
    name helps.
</p>
   
<p> Maybe you want to have your own convention for generating lisp
      function names for corresponding C function names, or you just
      want to lispify the names, also, before we forget you want to
      export the generated lisp names. To do this, we will use the
      SWIG <a
        href="Customization.html#Customization_features">feature directive</a>. 
Let's edit the interface file such that the C type "div_t*" is changed
      to Lisp type ":my-pointer", we lispify all names, 
      export everything, and do some more stuff.

</p>
<div class="code"><pre>
%module test

%typemap(cin) div_t* ":my-pointer";

%feature("intern_function","1");
%feature("export");

%feature("inline") lispsort_double;

%feature("intern_function", "my-lispify") lispsort_double;
%rename func123 renamed_cool_func;
%ignore "pointer_func";

%include "test.h"

</pre></div>

<p>
The <i>typemap(cin)</i> ensures that for all arguments which are input
    to C with the type "div_t*", the ":my-pointer" type be
    used. Similarly  <i>typemap(cout)</i> are used for all types which
    are returned from C.
</p>
<p>
The feature <i>intern_function</i> ensures that all C names are
      interned using the <b>swig-lispify</b> function. The "1" given
      to the feature is optional. The use of feature like
      <i>%feature("intern_function","1");</i> globally enables
      interning for everything. If you want to target a single
      function, or declaration then use the targeted version of
      feature, <i>%feature("intern_function", "my-lispify")
        lispsort_double;</i>, here we are using an additional feature
      which allows us to use our lispify function.
</p>
<p>The <i>export</i> feature allows us to export the symbols. The <i>inline</i>
      feature declaims the declared function as inline. The <i>rename</i>
      directive allows us to change the name(it is useful when
      generating C wrapper code for handling overloaded
      functions). The <i>ignore</i> directive ignores a certain
      declaration. 
</p>
<p>There are several other things which are possible, to see some
      example of usage of SWIG look at the Lispbuilder and wxCL
      projects. The generated code with 'noswig-lisp' option is:
</p>

<div class="targetlang"><pre>
(cl:defconstant #.(swig-lispify "y" 'constant) 5)

(cl:export '#.(swig-lispify "y" 'constant))

(cl:defconstant #.(swig-lispify "x" 'constant) (cl:ash 5 -1))

(cl:export '#.(swig-lispify "x" 'constant))

(cffi:defcstruct #.(swig-lispify "bar" 'classname)
	(#.(swig-lispify "p" 'slotname) :short)
	(#.(swig-lispify "q" 'slotname) :short)
	(#.(swig-lispify "a" 'slotname) :char)
	(#.(swig-lispify "b" 'slotname) :char)
	(#.(swig-lispify "z" 'slotname) :pointer)
	(#.(swig-lispify "n" 'slotname) :pointer))

(cl:export '#.(swig-lispify "bar" 'classname))

(cl:export '#.(swig-lispify "p" 'slotname))

(cl:export '#.(swig-lispify "q" 'slotname))

(cl:export '#.(swig-lispify "a" 'slotname))

(cl:export '#.(swig-lispify "b" 'slotname))

(cl:export '#.(swig-lispify "z" 'slotname))

(cl:export '#.(swig-lispify "n" 'slotname))

(cffi:defcvar ("my_struct" #.(swig-lispify "my_struct" 'variable))
 :pointer)

(cl:export '#.(swig-lispify "my_struct" 'variable))

(cffi:defcstruct #.(swig-lispify "foo" 'classname)
	(#.(swig-lispify "a" 'slotname) :int)
	(#.(swig-lispify "b" 'slotname) :pointer))

(cl:export '#.(swig-lispify "foo" 'classname))

(cl:export '#.(swig-lispify "a" 'slotname))

(cl:export '#.(swig-lispify "b" 'slotname))

(cffi:defcfun ("renamed_cool_func" #.(swig-lispify "renamed_cool_func" 'function)) :int
  (p :my-pointer)
  (q :pointer)
  (r :pointer))

(cl:export '#.(swig-lispify "renamed_cool_func" 'function))

(cl:declaim (cl:inline #.(my-lispify "lispsort_double" 'function)))

(cffi:defcfun ("lispsort_double" #.(my-lispify "lispsort_double" 'function)) :void
  (n :int)
  (array :pointer))

(cl:export '#.(my-lispify "lispsort_double" 'function))

(cffi:defcenum #.(swig-lispify "color" 'enumname)
	#.(swig-lispify "RED" 'enumvalue :keyword)
	#.(swig-lispify "BLUE" 'enumvalue :keyword)
	#.(swig-lispify "GREEN" 'enumvalue :keyword))

(cl:export '#.(swig-lispify "color" 'enumname))

</pre></div>

<H3><a name="Lisp_nn6"></a>25.2.3 Generating CFFI bindings for C++ code</H3>


<p>This feature to SWIG (for CFFI) is very new and still far from
    complete. Pitch in with your patches, bug reports and feature
    requests to improve it.
</p>
<p> Generating bindings for C++ code, requires <i>-c++</i> option to be
      present and it first generates C binding which will wrap the C++
      code, and then generates the 
      corresponding CFFI wrapper code. In the generated C wrapper
      code, you will often want to put your own C code, such as the
      code to include various files. This can be done by making use of
      "%{" and "%}" as shown below.
</p> 
<div class="code"><pre>
%{
 #include "Test/test.h"
%}
</pre></div>
<p>
Also, while parsing the C++ file and generating C wrapper code SWIG
    may need to be able to understand various symbols used in other
    header files. To help SWIG in doing this while ensuring that
    wrapper code is generated for the target file, use the "import"
    directive. The "include" directive specifies the target file for
    which wrapper code will be generated.
</p>
<div class="code"><pre>

%import "ancillary/header.h"

%include "target/header.h"

</pre></div>
Various features which were available for C headers can also be used
    here. The target header which we are going to use here is:
<div class="code"><pre>
namespace OpenDemo {
  class Test
    {
    public:
        float x;
        // constructors
        Test (void) {x = 0;}
        Test (float X) {x = X;}

        // vector addition
        Test operator+ (const Test&amp; v) const {return Test (x+v.x);}

      // length squared
        float lengthSquared (void) const {return this-&gt;dot (*this);}

        static float distance (const Test&amp; a, const Test&amp; b){return(a-b).length();}

        inline Test parallelComponent (const Test&amp; unitBasis) const {
          return unitBasis * projection;
        }

        Test setYtoZero (void) const {return Test (this-&gt;x);}

        static const Test zero;
    };


   inline Test operator* (float s, const Test&amp; v) {return v*s;}


    inline std::ostream&amp; operator&lt;&lt; (std::ostream&amp; o, const Test&amp; v)
    {
        return o &lt;&lt; "(" &lt;&lt; v.x &lt;&lt; ")";
    }


    inline Test RandomUnitVectorOnXZPlane (void)
    {
        return RandomVectorInUnitRadiusSphere().setYtoZero().normalize();
    }
}
</pre></div>
<p>The interface used is: </p>
<div class="code"><pre>
%module test
%include "test.cpp"
</pre></div>

SWIG generates 3 files, the first one is a C wrap which we don't show,
    the second is the plain CFFI wrapper which is as shown below:
<div class="targetlang"><pre>
(cffi:defcfun ("_wrap_Test_x_set" Test_x_set) :void
  (self :pointer)
  (x :float))

(cffi:defcfun ("_wrap_Test_x_get" Test_x_get) :float
  (self :pointer))

(cffi:defcfun ("_wrap_new_Test__SWIG_0" new_Test) :pointer)

(cffi:defcfun ("_wrap_new_Test__SWIG_1" new_Test) :pointer
  (X :float))

(cffi:defcfun ("_wrap_Test___add__" Test___add__) :pointer
  (self :pointer)
  (v :pointer))

(cffi:defcfun ("_wrap_Test_lengthSquared" Test_lengthSquared) :float
  (self :pointer))

(cffi:defcfun ("_wrap_Test_distance" Test_distance) :float
  (a :pointer)
  (b :pointer))

(cffi:defcfun ("_wrap_Test_parallelComponent" Test_parallelComponent) :pointer
  (self :pointer)
  (unitBasis :pointer))

(cffi:defcfun ("_wrap_Test_setYtoZero" Test_setYtoZero) :pointer
  (self :pointer))

(cffi:defcvar ("Test_zero" Test_zero)
 :pointer)

(cffi:defcfun ("_wrap_delete_Test" delete_Test) :void
  (self :pointer))

(cffi:defcfun ("_wrap___mul__" __mul__) :pointer
  (s :float)
  (v :pointer))

(cffi:defcfun ("_wrap___lshift__" __lshift__) :pointer
  (o :pointer)
  (v :pointer))

(cffi:defcfun ("_wrap_RandomUnitVectorOnXZPlane" RandomUnitVectorOnXZPlane) :pointer)
</pre></div>

The output is pretty good but it fails in disambiguating overloaded
    functions such as the constructor, in this case. One way of
    resolving this problem is to make the interface use the rename
    directiv, but hopefully there are better solutions.
 In addition SWIG also generates, a CLOS file


<div class="targetlang"><pre>
(clos:defclass test()
  ((ff :reader ff-pointer)))

(clos:defmethod (cl:setf x) (arg0 (obj test))
  (Test_x_set (ff-pointer obj) arg0))

(clos:defmethod x ((obj test))
  (Test_x_get (ff-pointer obj)))

(cl:shadow "+")
(clos:defmethod + ((obj test) (self test) (v test))
  (Test___add__ (ff-pointer obj) (ff-pointer self) (ff-pointer v)))

(clos:defmethod length-squared ((obj test) (self test))
  (Test_lengthSquared (ff-pointer obj) (ff-pointer self)))

(clos:defmethod parallel-component ((obj test) (self test) (unitBasis test))
  (Test_parallelComponent (ff-pointer obj) (ff-pointer self) (ff-pointer unitBasis)))

(clos:defmethod set-yto-zero ((obj test) (self test))
  (Test_setYtoZero (ff-pointer obj) (ff-pointer self)))
</pre></div>

<p>I agree that the CFFI C++ module needs lot more work. But I hope it
    provides a starting point, on which you can base your work of
    importing C++ libraries to Lisp. 
</p>
<p>
If you have any questions, suggestions, patches, etc., related to CFFI
      module feel free to contact us on the SWIG mailing list, and
      also please add a "[CFFI]" tag in the subject line.

<H3><a name="Lisp_nn7"></a>25.2.4 Inserting user code into generated files</H3>


<p>
It is often necessary to <a href="SWIG.html#SWIG_nn40">include user-defined code</a> 
into the automatically generated interface files. For example, when building
a C++ interface, example_wrap.cxx will likely not compile unless
you add a <tt>#include "header.h"</tt> directive. This can be done
using the SWIG <tt>%insert(section) %{ ...code... %}</tt> directive:
</p>

<div class="code">
<pre>
%module example

%{
#include "header.h"
%}

%include "header.h"

int fact(int n);
</pre>
</div>

<p>
Additional sections have been added for inserting into the
generated lisp interface file:
</p>
<ul>
  <li><tt>lisphead</tt> - inserts before type declarations</li>
  <li><tt>swiglisp</tt> - inserts after type declarations according to
    where it appears in the .i file</li>
</ul>
<p>
Note that the block <tt>%{ ... %}</tt> is effectively a shortcut for
<tt>%insert("header") %{ ... %}</tt>.
</p>


<H2><a name="Lisp_nn8"></a>25.3 CLISP</H2>


<p>
<a href="http://clisp.cons.org">CLISP</a> is a feature-loaded
      implementation of common lisp which is portable across most of the
      operating system environments and hardware. CLISP includes an
      interpreter, a compiler, a debugger, CLOS, MOP, a foreign
      language interface, i18n, regular expressions, a socket
      interface, and more. An X11 interface is available through CLX,
      Garnet and CLUE/CLIO. Command line editing is provided by
      readline. CLISP runs Maxima, ACL2 and many other Common Lisp
      packages.
</p>
<p>
      To run the clisp module of SWIG requires very little effort, you
      just need to execute:
</p>
<div class="code"><pre>
swig -clisp -module <i>module-name</i>   <i>file-name</i> 

</pre></div>

<p>
    Because of the high level nature of the CLISP FFI, the bindings
    generated by SWIG may not be absolutely correct, and you may need
    to modify them. The good thing is that you don't need to complex 
    interface file for the CLISP module. The CLISP module tries to
    produce code which is both human readable and easily modifyable.
</p>
<H3><a name="Lisp_nn9"></a>25.3.1 Additional Commandline Options </H3>


<p>
The following table list the additional commandline options available for the CLISP module. They can also be seen by using: 
</p>

<div class="code"><pre>
swig -clisp -help 
</pre></div>
<br/>
<table summary="CLISP specific options">
<tr>
<th>CLISP specific options</th>
</tr>

<tr>
<td>-extern-all</td>
<td>If this option is given then clisp definitions for all the functions<br/>
and global variables will be created otherwise only definitions for<br/>
	    externed functions and variables are created. 
</td>
</tr>

<tr>
<td>-generate-typedef</td>
<td>If this option is given then def-c-type will be used to generate<br/>
	    shortcuts according to the typedefs in the input.
</td>
</tr>

</table>

<H3><a name="Lisp_nn10"></a>25.3.2 Details on CLISP bindings</H3>


<p>
As mentioned earlier the CLISP bindings generated by SWIG may need
	some modifications. The clisp module creates a lisp file with
	the same name as the module name. This
	lisp file contains a 'defpackage' declaration, with the
	package name same as the module name. This package uses the
	'common-lisp' and 'ffi' packages. Also, package exports all
	the functions, structures and variables for which an ffi
	binding was generated.<br/>
     After generating the defpackage statement, the clisp module also
	sets the default language.

<div class="targetlang"><pre>
(defpackage :test
    (:use :common-lisp :ffi)
  (:export
   :make-bar
   :bar-x
   :bar-y
   :bar-a
   :bar-b
   :bar-z
   :bar-n
   :pointer_func
   :func123
   :make-cfunr
   :lispsort_double
   :test123))

(in-package :test)

(default-foreign-language :stdc)
</pre></div>
<p>
The ffi wrappers for functions and variables are generated as shown
      below. When functions have arguments of type "double * array",
      SWIG doesn't knows whether it is an 'out' argument or it is
      an array which will be passed, so SWIG plays it safe by
      declaring it as an '(array (ffi:c-ptr DOUBLE-FLOAT))'. For
      arguments of type "int **z[100]" where SWIG has more
      information, i.e., it knows that 'z' is an array of pointers to
      pointers of integers, SWIG defines it to be '(z (ffi:c-ptr
      (ffi:c-array (ffi:c-ptr (ffi:c-ptr ffi:int)) 100)))'  
</p>
<div class="code"><pre>
extern "C" {
int pointer_func(void (*ClosureFun)( void* _fun, void* _data, void* _evt ), int y);

int func123(div_t * x,int **z[100],int y[][1000][10]);

void lispsort_double (int n, double * array);

void test123(float x , double y);

}
</pre></div>
<div class="targetlang"><pre>
(ffi:def-call-out pointer_func
    (:name "pointer_func")
  (:arguments (ClosureFun (ffi:c-function (:arguments (arg0 (ffi:c-pointer NIL))
						      (arg1 (ffi:c-pointer NIL))
						      (arg2 (ffi:c-pointer NIL)))
					  (:return-type NIL)))
	      (y ffi:int))
  (:return-type ffi:int)
  (:library +library-name+))

(ffi:def-call-out func123
    (:name "func123")
  (:arguments (x (ffi:c-pointer div_t))
	      (z (ffi:c-ptr (ffi:c-array (ffi:c-ptr (ffi:c-ptr ffi:int)) 100)))
	      (y (ffi:c-ptr (ffi:c-ptr (ffi:c-array ffi:int (1000 10))))))
  (:return-type ffi:int)
  (:library +library-name+))


(ffi:def-call-out lispsort_double
    (:name "lispsort_double")
  (:arguments (n ffi:int)
	      (array (ffi:c-ptr DOUBLE-FLOAT)))
  (:return-type NIL)
  (:library +library-name+))

(ffi:def-call-out test123
    (:name "test")
  (:arguments (x SINGLE-FLOAT)
	      (y DOUBLE-FLOAT))
  (:return-type NIL)
  (:library +library-name+))

</pre></div>

<p>
The module also handles strutcures and #define constants as shown
      below. SWIG automatically adds the constructors and accessors
    created for the struct to the list of symbols exported by the
      package.
</p>
<div class="code"><pre>
struct bar {
    short x, y;
    char a, b;
    int *z[1000];
    struct bar * n;
};

#define max 1000
</pre></div>
<div class="targetlang"><pre>
(ffi:def-c-struct bar
    (x :type ffi:short)
  (y :type ffi:short)
  (a :type character)
  (b :type character)
  (z :type (ffi:c-array (ffi:c-ptr ffi:int) 1000))
  (n :type (ffi:c-pointer bar)))

(defconstant max 1000)

</pre></div>

<H2><a name="Lisp_nn11"></a>25.4 UFFI </H2>


</body>
</html>