Ignore unused arguments

[cffi.git] / doc / cffi-manual.texinfo

\input texinfo   @c -*- Mode: Texinfo; Mode: auto-fill -*-
@c %**start of header
@setfilename cffi.info
@settitle CFFI User Manual
@exampleindent 2

@c @documentencoding utf-8

@c Style notes:
@c
@c * The reference section names and "See Also" list are roman, not
@c   @code.  This is to follow the format of CLHS.
@c
@c * How it looks in HTML is the priority.

@c ============================= Macros =============================
@c The following macros are used throughout this manual.

@macro Function {args}
@defun \args\
@end defun
@end macro

@macro Macro {args}
@defmac \args\
@end defmac
@end macro

@macro Accessor {args}
@deffn {Accessor} \args\
@end deffn
@end macro

@macro GenericFunction {args}
@deffn {Generic Function} \args\
@end deffn
@end macro

@macro ForeignType {args}
@deftp {Foreign Type} \args\
@end deftp
@end macro

@macro Variable {args}
@defvr {Special Variable} \args\
@end defvr
@end macro

@macro Condition {args}
@deftp {Condition Type} \args\
@end deftp
@end macro

@macro cffi
@acronym{CFFI}
@end macro

@macro impnote {text}
@quotation
@strong{Implementor's note:} @emph{\text\}
@end quotation
@end macro

@c Info "requires" that x-refs end in a period or comma, or ) in the
@c case of @pxref.  So the following implements that requirement for
@c the "See also" subheadings that permeate this manual, but only in
@c Info mode.
@ifinfo
@macro seealso {name}
@ref{\name\}.
@end macro
@end ifinfo

@ifnotinfo
@alias seealso = ref
@end ifnotinfo

@c Typeset comments in roman font for the TeX output.
@iftex
@alias lispcmt = r
@end iftex
@ifnottex
@alias lispcmt = asis
@end ifnottex

@alias res = result

@c ============================= Macros =============================


@c Show types, functions, and concepts in the same index.
@syncodeindex tp cp
@syncodeindex fn cp

@copying
Copyright @copyright{} 2005 James Bielman <jamesjb at jamesjb.com> @*
Copyright @copyright{} 2005-2015 Lu@'{@dotless{i}}s Oliveira
  <loliveira at common-lisp.net> @*
Copyright @copyright{} 2005-2006 Dan Knapp <danka at accela.net> @*
Copyright @copyright{} 2005-2006 Emily Backes <lucca at accela.net> @*
Copyright @copyright{} 2006 Stephen Compall <s11 at member.fsf.org>

@quotation
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
``Software''), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:

The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.

@sc{The software is provided ``as is'', without warranty of any kind,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose and noninfringement.
In no event shall the authors or copyright holders be liable for any
claim, damages or other liability, whether in an action of contract,
tort or otherwise, arising from, out of or in connection with the
software or the use or other dealings in the software.}
@end quotation
@end copying
@c %**end of header

@dircategory Software development
@direntry
* CFFI Manual: (cffi-manual).           CFFI Manual.
@end direntry

@titlepage
@title CFFI User Manual
@c @subtitle Version X.X
@c @author James Bielman

@page
@vskip 0pt plus 1filll
@insertcopying
@end titlepage

@contents

@ifnottex
@node Top, Introduction, (dir), (dir)
@top cffi
@insertcopying
@end ifnottex

@menu
* Introduction::                What is CFFI?
* Installation::
* Implementation Support::
* Tutorial::                    Interactive intro to using CFFI.
* Wrapper generators::          CFFI forms from munging C source code.
* Foreign Types::
* Pointers::
* Strings::
* Variables::
* Functions::
* Libraries::
* Callbacks::
* The Groveller::
* Static Linking::
* Limitations::
* Platform-specific features::  Details about the underlying system.
* Glossary::                    List of CFFI-specific terms and meanings.
* Comprehensive Index::

@detailmenu
 --- Dictionary ---

Foreign Types

* convert-from-foreign::        Outside interface to backward type translator.
* convert-to-foreign::          Outside interface to forward type translator.
* defbitfield::                 Defines a bitfield.
* defcstruct::                  Defines a C structure type.
* defcunion::                   Defines a C union type.
* defctype::                    Defines a foreign typedef.
* defcenum::                    Defines a C enumeration.
* define-foreign-type::         Defines a foreign type specifier.
* define-parse-method::         Specifies how a type should be parsed.
@c * explain-foreign-slot-value::  <unimplemented>
* foreign-bitfield-symbols::    Returns a list of symbols for a bitfield type.
* foreign-bitfield-value::      Calculates a value for a bitfield type.
* foreign-enum-keyword::        Finds a keyword in an enum type.
* foreign-enum-value::          Finds a value in an enum type.
* foreign-slot-names::          Returns a list of slot names in a foreign struct.
* foreign-slot-offset::         Returns the offset of a slot in a foreign struct.
* foreign-slot-pointer::        Returns a pointer to a slot in a foreign struct.
* foreign-slot-value::          Returns the value of a slot in a foreign struct.
* foreign-type-alignment::      Returns the alignment of a foreign type.
* foreign-type-size::           Returns the size of a foreign type.
* free-converted-object::       Outside interface to typed object deallocators.
* free-translated-object::      Defines how to free a oreign object.
* translate-from-foreign::      Defines a foreign-to-Lisp object translation.
* translate-to-foreign::        Defines a Lisp-to-foreign object translation.
* with-foreign-object::         Allocates a foreign object with dynamic extent.
* with-foreign-objects::        Plural form of @code{with-foreign-object}.
* with-foreign-slots::          Accesses the slots of a foreign structure.

Pointers

* foreign-free::                Deallocates memory.
* foreign-alloc::               Allocates memory.
* foreign-symbol-pointer::      Returns a pointer to a foreign symbol.
* inc-pointer::                 Increments the address held by a pointer.
* incf-pointer::                Increments the pointer address in a place.
* make-pointer::                Returns a pointer to a given address.
* mem-aptr::                    The pointer to an element of an array.
* mem-aref::                    Accesses the value of an index in an array.
* mem-ref::                     Dereferences a pointer.
* null-pointer::                Returns a NULL pointer.
* null-pointer-p::              Tests a pointer for NULL value.
* pointerp::                    Tests whether an object is a pointer or not.
* pointer-address::             Returns the address pointed to by a pointer.
* pointer-eq::                  Tests if two pointers point to the same address.
* with-foreign-pointer::        Allocates memory with dynamic extent.

Strings

* *default-foreign-encoding*::  Default encoding for the string types.
* foreign-string-alloc::        Converts a Lisp string to a foreign string.
* foreign-string-free::         Deallocates memory used by a foreign string.
* foreign-string-to-lisp::      Converts a foreign string to a Lisp string.
* lisp-string-to-foreign::      Copies a Lisp string into a foreign string.
* with-foreign-string::         Allocates a foreign string with dynamic extent.
* with-foreign-strings::        Plural form of @code{with-foreign-string}.
* with-foreign-pointer-as-string::  Similar to CL's with-output-to-string.

Variables

* defcvar::                     Defines a C global variable.
* get-var-pointer::             Returns a pointer to a defined global variable.

Functions

* defcfun::                     Defines a foreign function.
* foreign-funcall::             Performs a call to a foreign function.
* foreign-funcall-pointer::     Performs a call through a foreign pointer.
* translate-camelcase-name::    Converts a camelCase foreign name to/from a Lisp name.
* translate-name-from-foreign::  Converts a foreign name to a Lisp name.
* translate-name-to-foreign::   Converts a Lisp name to a foreign name.
* translate-underscore-separated-name::  Converts an underscore_separated foreign name to/from a Lisp name.

Libraries

* close-foreign-library::       Closes a foreign library.
* *darwin-framework-directories*::  Search path for Darwin frameworks.
* define-foreign-library::      Explain how to load a foreign library.
* *foreign-library-directories*::  Search path for shared libraries.
* load-foreign-library::        Load a foreign library.
* load-foreign-library-error::  Signalled on failure of its namesake.
@c * reload-foreign-libraries::    Reload foreign libraries.
* use-foreign-library::         Load a foreign library when needed.

Callbacks

* callback::                    Returns a pointer to a defined callback.
* defcallback::                 Defines a Lisp callback.
* get-callback::                Returns a pointer to a defined callback.

@end detailmenu
@end menu




@c ===================================================================
@c CHAPTER: Introduction

@node Introduction, Installation, Top, Top
@chapter Introduction

@cffi{} is the Common Foreign Function Interface for @acronym{ANSI}
Common Lisp systems.  By @dfn{foreign function} we mean a function
written in another programming language and having different data and
calling conventions than Common Lisp, namely, C.  @cffi{} allows you
to call foreign functions and access foreign variables, all without
leaving the Lisp image.

We consider this manual ever a work in progress.  If you have
difficulty with anything @cffi{}-specific presented in the manual,
please contact @email{cffi-devel@@common-lisp.net,the developers} with
details.


@heading Motivation

@xref{Tutorial-Comparison,, What makes Lisp different}, for
an argument in favor of @acronym{FFI} in general.

@cffi{}'s primary role in any image is to mediate between Lisp
developers and the widely varying @acronym{FFI}s present in the
various Lisp implementations it supports.  With @cffi{}, you can
define foreign function interfaces while still maintaining portability
between implementations.  It is not the first Common Lisp package with
this objective; however, it is meant to be a more malleable framework
than similar packages.


@heading Design Philosophy

@itemize
@item
Pointers do not carry around type information. Instead, type
information is supplied when pointers are dereferenced.

@item
A type safe pointer interface can be developed on top of an
untyped one.  It is difficult to do the opposite.

@item
Functions are better than macros.  When a macro could be used
for performance, use a compiler-macro instead.
@end itemize


@c ===================================================================
@c CHAPTER: Installation

@node Installation, Implementation Support, Introduction, Top
@chapter Installation

@cffi{} can be obtained through one of the following means available
through its @uref{http://common-lisp.net/project/cffi/,,website}:

@itemize
@item
@uref{http://common-lisp.net/project/cffi/releases/?M=D,,official release
tarballs}

@item
@uref{http://common-lisp.net/gitweb?p=projects/cffi/cffi.git,,git
repository}

@c snapshots have been disabled as of
@c @item
@c @uref{http://common-lisp.net/project/cffi/tarballs/?M=D,,nightly-generated
@c snapshots}

@end itemize

In addition, you will need to obtain and install the following
dependencies:

@itemize
@item
@uref{http://common-lisp.net/project/babel/,,Babel}, a charset
encoding/decoding library.

@item
@uref{http://common-lisp.net/project/alexandria/,,Alexandria}, a
collection of portable public-domain utilities.

@item
@uref{http://www.cliki.net/trivial-features,,trivial-features}, a
portability layer that ensures consistent @code{*features*} across
multiple Common Lisp implementations.

@end itemize

Furthermore, if you wish to run the testsuite,
@uref{http://www.cliki.net/rt,,RT} is required.

You may find mechanisms such as
@uref{https://www.quicklisp.org/beta/,Quicklisp} (recommended)
or @uref{http://common-lisp.net/project/clbuild/,,clbuild} (for advanced
uses) helpful in getting and managing @cffi{} and its
dependencies.


@c ===================================================================
@c CHAPTER: Implementation Support

@node Implementation Support, Tutorial, Installation, Top
@chapter Implementation Support

@cffi{} supports various free and commercial Lisp implementations:
@acronym{ABCL}, Allegro CL, Clasp, @sc{clisp}, Clozure CL,
@acronym{CMUCL}, Corman CL, @acronym{ECL}, @acronym{GCL}, LispWorks,
@acronym{MCL}, @acronym{SBCL} and the Scieneer CL.

In general, you should work with the latest versions of each
implementation since those will usually be tested against recent
versions of CFFI more often and might include necessary features or
bug fixes. Reasonable patches for compatibility with earlier versions
are welcome nevertheless.

@section Limitations

Some features are not supported in all implementations.
@c TODO: describe these features here.
@c       flat-namespace too

@subheading Allegro CL

@itemize
@item
Does not support the @code{:long-long} type natively.
@item
Unicode support is limited to the Basic Multilingual Plane (16-bit
code points).
@end itemize

@subheading Clasp

@itemize
@item
Only supports a flat namespace.
@end itemize

@subheading CMUCL

@itemize
@item
No Unicode support. (8-bit code points)
@end itemize

@subheading Corman CL

@itemize
@item
Does not support @code{foreign-funcall}.
@end itemize

@subheading @acronym{ECL}

@itemize
@item
On platforms where ECL's dynamic FFI is not supported (ie. when
@code{:dffi} is not present in @code{*features*}),
@code{cffi:load-foreign-library} does not work and you must use ECL's
own @code{ffi:load-foreign-library} with a constant string argument.
@end itemize

@subheading Lispworks

@itemize
@item
Does not completely support the @code{:long-long} type natively in
32-bit platforms.
@item
Unicode support is limited to the Basic Multilingual Plane (16-bit
code points).
@end itemize

@subheading @acronym{SBCL}

@itemize
@item
Not all platforms support callbacks.

@end itemize


@c ===================================================================
@c CHAPTER: An Introduction to Foreign Interfaces and CFFI

@c This macro is merely a marker that I don't think I'll use after
@c all.
@macro tutorialsource {text}
@c \text\
@end macro

@c because I don't want to type this over and over
@macro clikicffi
http://www.cliki.net/CFFI
@end macro
@c TeX puts spurious newlines in when you use the above macro
@c in @examples &c.  So it is expanded below in some places.


@node Tutorial, Wrapper generators, Implementation Support, Top
@chapter An Introduction to Foreign Interfaces and @acronym{CFFI}

@c Above, I don't use the cffi macro because it breaks TeX.

@cindex tutorial, @cffi{}
Users of many popular languages bearing semantic similarity to Lisp,
such as Perl and Python, are accustomed to having access to popular C
libraries, such as @acronym{GTK}, by way of ``bindings''.  In Lisp, we
do something similar, but take a fundamentally different approach.
This tutorial first explains this difference, then explains how you
can use @cffi{}, a powerful system for calling out to C and C++ and
access C data from many Common Lisp implementations.

@cindex foreign functions and data
The concept can be generalized to other languages; at the time of
writing, only @cffi{}'s C support is fairly complete. Therefore, we
will interchangeably refer to @dfn{foreign functions} and @dfn{foreign
data}, and ``C functions'' and ``C data''.  At no time will the word
``foreign'' carry its usual, non-programming meaning.

This tutorial expects you to have a working understanding of both
Common Lisp and C, including the Common Lisp macro system.

@menu
* Tutorial-Comparison::         Why FFI?
* Tutorial-Getting a URL::      An FFI use case.
* Tutorial-Loading::            Load libcurl.so.
* Tutorial-Initializing::       Call a function in libcurl.so.
* Tutorial-easy_setopt::        An advanced libcurl function.
* Tutorial-Abstraction::        Why breaking it is necessary.
* Tutorial-Lisp easy_setopt::   Semi-Lispy option interface.
* Tutorial-Memory::             In C, you collect the garbage.
* Tutorial-Callbacks::          Make useful C function pointers.
* Tutorial-Completion::         Minimal get-url functionality.
* Tutorial-Types::              Defining new foreign types.
* Tutorial-Conclusion::         What's next?
@end menu


@node Tutorial-Comparison, Tutorial-Getting a URL, Tutorial, Tutorial
@section What makes Lisp different

The following sums up how bindings to foreign libraries are usually
implemented in other languages, then in Common Lisp:

@table @asis
@item Perl, Python, Java, other one-implementation languages
@cindex @acronym{SWIG}
@cindex Perl
@cindex Python
Bindings are implemented as shared objects written in C.  In some
cases, the C code is generated by a tool, such as @acronym{SWIG}, but
the result is the same: a new C library that manually translates
between the language implementation's objects, such as @code{PyObject}
in Python, and whatever C object is called for, often using C
functions provided by the implementation.  It also translates between
the calling conventions of the language and C.

@item Common Lisp
@cindex @acronym{SLIME}
Bindings are written in Lisp.  They can be created at-will by Lisp
programs.  Lisp programmers can write new bindings and add them to the
image, using a listener such as @acronym{SLIME}, as easily as with
regular Lisp definitions.  The only foreign library to load is the one
being wrapped---the one with the pure C interface; no C or other
non-Lisp compilation is required.
@end table

@cindex advantages of @acronym{FFI}
@cindex benefits of @acronym{FFI}
We believe the advantages of the Common Lisp approach far outweigh any
disadvantages.  Incremental development with a listener can be as
productive for C binding development as it is with other Lisp
development.  Keeping it ``in the [Lisp] family'', as it were, makes
it much easier for you and other Lisp programmers to load and use the
bindings.  Common Lisp implementations such as @acronym{CMUCL}, freed
from having to provide a C interface to their own objects, are thus
freed to be implemented in another language (as @acronym{CMUCL} is)
while still allowing programmers to call foreign functions.

@cindex minimal bindings
Perhaps the greatest advantage is that using an @acronym{FFI} doesn't
obligate you to become a professional binding developer.  Writers of
bindings for other languages usually end up maintaining or failing to
maintain complete bindings to the foreign library.  Using an
@acronym{FFI}, however, means if you only need one or two functions,
you can write bindings for only those functions, and be assured that
you can just as easily add to the bindings if need be.

@cindex C abstractions
@cindex abstractions in C
The removal of the C compiler, or C interpretation of any kind,
creates the main disadvantage: some of C's ``abstractions'' are not
available, violating information encapsulation.  For example,
@code{struct}s that must be passed on the stack, or used as return
values, without corresponding functional abstractions to create and
manage the @code{struct}s, must be declared explicitly in Lisp.  This
is fine for structs whose contents are ``public'', but is not so
pleasant when a struct is supposed to be ``opaque'' by convention,
even though it is not so defined.@footnote{Admittedly, this is an
advanced issue, and we encourage you to leave this text until you are
more familiar with how @cffi{} works.}

Without an abstraction to create the struct, Lisp needs to be able to
lay out the struct in memory, so must know its internal details.

@cindex workaround for C
In these cases, you can create a minimal C library to provide the
missing abstractions, without destroying all the advantages of the
Common Lisp approach discussed above.  In the case of @code{struct}s,
you can write simple, pure C functions that tell you how many bytes a
struct requires or allocate new structs, read and write fields of the
struct, or whatever operations are supposed to be
public.@footnote{This does not apply to structs whose contents are
intended to be part of the public library interface.  In those cases,
a pure Lisp struct definition is always preferred.  In fact, many
prefer to stay in Lisp and break the encapsulation anyway, placing the
burden of correct library interface definition on the library.}
@ref{The Groveller} automates this and other processes.

Another disadvantage appears when you would rather use the foreign
language than Lisp.  However, someone who prefers C to Lisp is not a
likely candidate for developing a Lisp interface to a C library.


@node Tutorial-Getting a URL, Tutorial-Loading, Tutorial-Comparison, Tutorial
@section Getting a @acronym{URL}

@cindex c@acronym{URL}
The widely available @code{libcurl} is a library for downloading files
over protocols like @acronym{HTTP}.  We will use @code{libcurl} with
@cffi{} to download a web page.

Please note that there are many other ways to download files from the
web, not least the @sc{cl-curl} project to provide bindings to
@code{libcurl} via a similar @acronym{FFI}.@footnote{Specifically,
@acronym{UFFI}, an older @acronym{FFI} that takes a somewhat different
approach compared to @cffi{}.  I believe that these days (December
2005) @cffi{} is more portable and actively developed, though not as
mature yet.  Consensus in the free @sc{unix} Common Lisp community
seems to be that @cffi{} is preferred for new development, though
@acronym{UFFI} will likely go on for quite some time as many projects
already use it.  @cffi{} includes the @code{UFFI-COMPAT} package for
complete compatibility with @acronym{UFFI}.}

@uref{http://curl.haxx.se/libcurl/c/libcurl-tutorial.html,,libcurl-tutorial(3)}
is a tutorial for @code{libcurl} programming in C.  We will follow
that to develop a binding to download a file.  We will also use
@file{curl.h}, @file{easy.h}, and the @command{man} pages for the
@code{libcurl} function, all available in the @samp{curl-dev} package
or equivalent for your system, or in the c@acronym{URL} source code
package.  If you have the development package, the headers should be
installed in @file{/usr/include/curl/}, and the @command{man} pages
may be accessed through your favorite @command{man} facility.


@node Tutorial-Loading, Tutorial-Initializing, Tutorial-Getting a URL, Tutorial
@section Loading foreign libraries

@cindex loading @cffi{}
@cindex requiring @cffi{}
First of all, we will create a package to work in.  You can save these
forms in a file, or just send them to the listener as they are.  If
creating bindings for an @acronym{ASDF} package of yours, you will
want to add @code{:cffi} to the @code{:depends-on} list in your
@file{.asd} file.  Otherwise, just use the @code{asdf:load-system} function to
load @cffi{}.

@tutorialsource{Initialization}
@lisp
(asdf:load-system :cffi)

;;; @lispcmt{Nothing special about the "CFFI-USER" package.  We're just}
;;; @lispcmt{using it as a substitute for your own CL package.}
(defpackage :cffi-user
  (:use :common-lisp :cffi))

(in-package :cffi-user)

(define-foreign-library libcurl
  (:darwin (:or "libcurl.3.dylib" "libcurl.dylib"))
  (:unix (:or "libcurl.so.3" "libcurl.so"))
  (t (:default "libcurl")))

(use-foreign-library libcurl)
@end lisp

@cindex foreign library load
@cindex library, foreign
Using @code{define-foreign-library} and @code{use-foreign-library}, we
have loaded @code{libcurl} into Lisp, much as the linker does when you
start a C program, or @code{common-lisp:load} does with a Lisp source
file or @acronym{FASL} file.  We special-cased for @sc{unix} machines
to always load a particular version, the one this tutorial was tested
with; for those who don't care, the @code{define-foreign-library}
clause @code{(t (:default "libcurl"))} should be satisfactory, and
will adapt to various operating systems.


@node Tutorial-Initializing, Tutorial-easy_setopt, Tutorial-Loading, Tutorial
@section Initializing @code{libcurl}

@cindex function definition
After the introductory matter, the tutorial goes on to present the
first function you should use.

@example
CURLcode curl_global_init(long flags);
@end example

@noindent
Let's pick this apart into appropriate Lisp code:

@tutorialsource{First CURLcode}
@lisp
;;; @lispcmt{A CURLcode is the universal error code.  curl/curl.h says}
;;; @lispcmt{no return code will ever be removed, and new ones will be}
;;; @lispcmt{added to the end.}
(defctype curl-code :int)

;;; @lispcmt{Initialize libcurl with FLAGS.}
(defcfun "curl_global_init" curl-code
  (flags :long))
@end lisp

@impnote{By default, CFFI assumes the UNIX viewpoint that there is one
C symbol namespace, containing all symbols in all loaded objects.
This is not so on Windows and Darwin, but we emulate UNIX's behaviour
there.  @ref{defcfun} for more details.}

Note the parallels with the original C declaration.  We've defined
@code{curl-code} as a wrapping type for @code{:int}; right now, it
only marks it as special, but later we will do something more
interesting with it.  The point is that we don't have to do it yet.

@cindex calling foreign functions
Looking at @file{curl.h}, @code{CURL_GLOBAL_NOTHING}, a possible value
for @code{flags} above, is defined as @samp{0}.  So we can now call
the function:

@example
@sc{cffi-user>} (curl-global-init 0)
@result{} 0
@end example

@cindex looks like it worked
Looking at @file{curl.h} again, @code{0} means @code{CURLE_OK}, so it
looks like the call succeeded.  Note that @cffi{} converted the
function name to a Lisp-friendly name.  You can specify your own name
if you want; use @code{("curl_global_init" @var{your-name-here})} as
the @var{name} argument to @code{defcfun}.

The tutorial goes on to have us allocate a handle.  For good measure,
we should also include the deallocator.  Let's look at these
functions:

@example
CURL *curl_easy_init( );
void curl_easy_cleanup(CURL *handle);
@end example

Advanced users may want to define special pointer types; we will
explore this possibility later.  For now, just treat every pointer as
the same:

@tutorialsource{curl_easy handles}
@lisp
(defcfun "curl_easy_init" :pointer)

(defcfun "curl_easy_cleanup" :void
  (easy-handle :pointer))
@end lisp

Now we can continue with the tutorial:

@example
@sc{cffi-user>} (defparameter *easy-handle* (curl-easy-init))
@result{} *EASY-HANDLE*
@sc{cffi-user>} *easy-handle*
@result{} #<FOREIGN-ADDRESS #x09844EE0>
@end example

@cindex pointers in Lisp
Note the print representation of a pointer.  It changes depending on
what Lisp you are using, but that doesn't make any difference to
@cffi{}.


@node Tutorial-easy_setopt, Tutorial-Abstraction, Tutorial-Initializing, Tutorial
@section Setting download options

The @code{libcurl} tutorial says we'll want to set many options before
performing any download actions.  This is done through
@code{curl_easy_setopt}:

@c That is literally ..., not an ellipsis.
@example
CURLcode curl_easy_setopt(CURL *curl, CURLoption option, ...);
@end example

@cindex varargs
@cindex foreign arguments
We've introduced a new twist: variable arguments.  There is no obvious
translation to the @code{defcfun} form, particularly as there are four
possible argument types.  Because of the way C works, we could define
four wrappers around @code{curl_easy_setopt}, one for each type; in
this case, however, we'll use the general-purpose macro
@code{foreign-funcall} to call this function.

@cindex enumeration, C
To make things easier on ourselves, we'll create an enumeration of the
kinds of options we want to set.  The @code{enum CURLoption} isn't the
most straightforward, but reading the @code{CINIT} C macro definition
should be enlightening.

@tutorialsource{CURLoption enumeration}
@lisp
(defmacro define-curl-options (name type-offsets &rest enum-args)
  "As with CFFI:DEFCENUM, except each of ENUM-ARGS is as follows:

    (NAME TYPE NUMBER)

Where the arguments are as they are with the CINIT macro defined
in curl.h, except NAME is a keyword.

TYPE-OFFSETS is a plist of TYPEs to their integer offsets, as
defined by the CURLOPTTYPE_LONG et al constants in curl.h."
  (flet ((enumerated-value (type offset)
           (+ (getf type-offsets type) offset)))
    `(progn
       (defcenum ,name
         ,@@(loop for (name type number) in enum-args
              collect (list name (enumerated-value type number))))
       ',name)))                ;@lispcmt{for REPL users' sanity}

(define-curl-options curl-option
    (long 0 objectpoint 10000 functionpoint 20000 off-t 30000)
  (:noprogress long 43)
  (:nosignal long 99)
  (:errorbuffer objectpoint 10)
  (:url objectpoint 2))
@end lisp

With some well-placed Emacs @code{query-replace-regexp}s, you could
probably similarly define the entire @code{CURLoption} enumeration.  I
have selected to transcribe a few that we will use in this tutorial.

If you're having trouble following the macrology, just macroexpand the
@code{curl-option} definition, or see the following macroexpansion,
conveniently downcased and reformatted:

@tutorialsource{DEFINE-CURL-OPTIONS macroexpansion}
@lisp
(progn
  (defcenum curl-option
    (:noprogress 43)
    (:nosignal 99)
    (:errorbuffer 10010)
    (:url 10002))
  'curl-option)
@end lisp

@noindent
That seems more than reasonable.  You may notice that we only use the
@var{type} to compute the real enumeration offset; we will also need
the type information later.

First, however, let's make sure a simple call to the foreign function
works:

@example
@sc{cffi-user>} (foreign-funcall "curl_easy_setopt"
               :pointer *easy-handle*
               curl-option :nosignal :long 1 curl-code)
@result{} 0
@end example

@code{foreign-funcall}, despite its surface simplicity, can be used to
call any C function.  Its first argument is a string, naming the
function to be called.  Next, for each argument, we pass the name of
the C type, which is the same as in @code{defcfun}, followed by a Lisp
object representing the data to be passed as the argument.  The final
argument is the return type, for which we use the @code{curl-code}
type defined earlier.

@code{defcfun} just puts a convenient fa@,cade on
@code{foreign-funcall}.@footnote{This isn't entirely true; some Lisps
don't support @code{foreign-funcall}, so @code{defcfun} is implemented
without it.  @code{defcfun} may also perform optimizations that
@code{foreign-funcall} cannot.}  Our earlier call to
@code{curl-global-init} could have been written as follows:

@example
@sc{cffi-user>} (foreign-funcall "curl_global_init" :long 0
                            curl-code)
@result{} 0
@end example

Before we continue, we will take a look at what @cffi{} can and can't
do, and why this is so.


@node Tutorial-Abstraction, Tutorial-Lisp easy_setopt, Tutorial-easy_setopt, Tutorial
@section Breaking the abstraction

@cindex breaking the abstraction
@cindex abstraction breaking
In @ref{Tutorial-Comparison,, What makes Lisp different}, we mentioned
that writing an @acronym{FFI} sometimes requires depending on
information not provided as part of the interface.  The easy option
@code{CURLOPT_WRITEDATA}, which we will not provide as part of the
Lisp interface, illustrates this issue.

Strictly speaking, the @code{curl-option} enumeration is not
necessary; we could have used @code{:int 99} instead of
@code{curl-option :nosignal} in our call to @code{curl_easy_setopt}
above.  We defined it anyway, in part to hide the fact that we are
breaking the abstraction that the C @code{enum} provides.  If the
c@acronym{URL} developers decide to change those numbers later, we
must change the Lisp enumeration, because enumeration values are not
provided in the compiled C library, @code{libcurl.so.3}.

@cffi{} works because the most useful things in C libraries ---
non-static functions and non-static variables --- are included
accessibly in @code{libcurl.so.3}.  A C compiler that violated this
would be considered a worthless compiler.

The other thing @code{define-curl-options} does is give the ``type''
of the third argument passed to @code{curl_easy_setopt}.  Using this
information, we can tell that the @code{:nosignal} option should
accept a long integer argument.  We can implicitly assume @code{t}
@equiv{} 1 and @code{nil} @equiv{} 0, as it is in C, which takes care
of the fact that @code{CURLOPT_NOSIGNAL} is really asking for a
boolean.

The ``type'' of @code{CURLOPT_WRITEDATA} is @code{objectpoint}.
However, it is really looking for a @code{FILE*}.
@code{CURLOPT_ERRORBUFFER} is looking for a @code{char*}, so there is
no obvious @cffi{} type but @code{:pointer}.

The first thing to note is that nowhere in the C interface includes
this information; it can only be found in the manual.  We could
disjoin these clearly different types ourselves, by splitting
@code{objectpoint} into @code{filepoint} and @code{charpoint}, but we
are still breaking the abstraction, because we have to augment the
entire enumeration form with this additional
information.@footnote{Another possibility is to allow the caller to
specify the desired C type of the third argument.  This is essentially
what happens in a call to the function written in C.}

@cindex streams and C
@cindex @sc{file}* and streams
The second is that the @code{CURLOPT_WRITEDATA} argument is completely
incompatible with the desired Lisp data, a
stream.@footnote{@xref{Other Kinds of Streams,,, libc, GNU C Library
Reference}, for a @acronym{GNU}-only way to extend the @code{FILE*}
type.  You could use this to convert Lisp streams to the needed C
data.  This would be quite involved and far outside the scope of this
tutorial.}  It is probably acceptable if we are controlling every file
we might want to use as this argument, in which case we can just call
the foreign function @code{fopen}.  Regardless, though, we can't write
to arbitrary streams, which is exactly what we want to do for this
application.

Finally, note that the @code{curl_easy_setopt} interface itself is a
hack, intended to work around some of the drawbacks of C.  The
definition of @code{Curl_setopt}, while long, is far less cluttered
than the equivalent disjoint-function set would be; in addition,
setting a new option in an old @code{libcurl} can generate a run-time
error rather than breaking the compile.  Lisp can just as concisely
generate functions as compare values, and the ``undefined function''
error is just as useful as any explicit error we could define here
might be.


@node Tutorial-Lisp easy_setopt, Tutorial-Memory, Tutorial-Abstraction, Tutorial
@section Option functions in Lisp

We could use @code{foreign-funcall} directly every time we wanted to
call @code{curl_easy_setopt}.  However, we can encapsulate some of the
necessary information with the following.

@lisp
;;; @lispcmt{We will use this type later in a more creative way.  For}
;;; @lispcmt{now, just consider it a marker that this isn't just any}
;;; @lispcmt{pointer.}
(defctype easy-handle :pointer)

(defmacro curl-easy-setopt (easy-handle enumerated-name
                            value-type new-value)
  "Call `curl_easy_setopt' on EASY-HANDLE, using ENUMERATED-NAME
as the OPTION.  VALUE-TYPE is the CFFI foreign type of the third
argument, and NEW-VALUE is the Lisp data to be translated to the
third argument.  VALUE-TYPE is not evaluated."
  `(foreign-funcall "curl_easy_setopt" easy-handle ,easy-handle
                    curl-option ,enumerated-name
                    ,value-type ,new-value curl-code))
@end lisp

Now we define a function for each kind of argument that encodes the
correct @code{value-type} in the above.  This can be done reasonably
in the @code{define-curl-options} macroexpansion; after all, that is
where the different options are listed!

@cindex Lispy C functions
We could make @code{cl:defun} forms in the expansion that simply call
@code{curl-easy-setopt}; however, it is probably easier and clearer to
use @code{defcfun}.  @code{define-curl-options} was becoming unwieldy,
so I defined some helpers in this new definition.

@smalllisp
(defun curry-curl-option-setter (function-name option-keyword)
  "Wrap the function named by FUNCTION-NAME with a version that
curries the second argument as OPTION-KEYWORD.

This function is intended for use in DEFINE-CURL-OPTION-SETTER."
  (setf (symbol-function function-name)
          (let ((c-function (symbol-function function-name)))
            (lambda (easy-handle new-value)
              (funcall c-function easy-handle option-keyword
                       new-value)))))

(defmacro define-curl-option-setter (name option-type
                                     option-value foreign-type)
  "Define (with DEFCFUN) a function NAME that calls
curl_easy_setopt.  OPTION-TYPE and OPTION-VALUE are the CFFI
foreign type and value to be passed as the second argument to
easy_setopt, and FOREIGN-TYPE is the CFFI foreign type to be used
for the resultant function's third argument.

This macro is intended for use in DEFINE-CURL-OPTIONS."
  `(progn
     (defcfun ("curl_easy_setopt" ,name) curl-code
       (easy-handle easy-handle)
       (option ,option-type)
       (new-value ,foreign-type))
     (curry-curl-option-setter ',name ',option-value)))

(defmacro define-curl-options (type-name type-offsets &rest enum-args)
  "As with CFFI:DEFCENUM, except each of ENUM-ARGS is as follows:

    (NAME TYPE NUMBER)

Where the arguments are as they are with the CINIT macro defined
in curl.h, except NAME is a keyword.

TYPE-OFFSETS is a plist of TYPEs to their integer offsets, as
defined by the CURLOPTTYPE_LONG et al constants in curl.h.

Also, define functions for each option named
set-`TYPE-NAME'-`OPTION-NAME', where OPTION-NAME is the NAME from
the above destructuring."
  (flet ((enumerated-value (type offset)
           (+ (getf type-offsets type) offset))
         ;; @lispcmt{map PROCEDURE, destructuring each of ENUM-ARGS}
         (map-enum-args (procedure)
           (mapcar (lambda (arg) (apply procedure arg)) enum-args))
         ;; @lispcmt{build a name like SET-CURL-OPTION-NOSIGNAL}
         (make-setter-name (option-name)
           (intern (concatenate
                    'string "SET-" (symbol-name type-name)
                    "-" (symbol-name option-name)))))
    `(progn
       (defcenum ,type-name
         ,@@(map-enum-args
            (lambda (name type number)
              (list name (enumerated-value type number)))))
       ,@@(map-enum-args
          (lambda (name type number)
            (declare (ignore number))
            `(define-curl-option-setter ,(make-setter-name name)
               ,type-name ,name ,(ecase type
                                   (long :long)
                                   (objectpoint :pointer)
                                   (functionpoint :pointer)
                                   (off-t :long)))))
       ',type-name)))
@end smalllisp

@noindent
Macroexpanding our @code{define-curl-options} form once more, we
see something different:

@lisp
(progn
  (defcenum curl-option
    (:noprogress 43)
    (:nosignal 99)
    (:errorbuffer 10010)
    (:url 10002))
  (define-curl-option-setter set-curl-option-noprogress
    curl-option :noprogress :long)
  (define-curl-option-setter set-curl-option-nosignal
    curl-option :nosignal :long)
  (define-curl-option-setter set-curl-option-errorbuffer
    curl-option :errorbuffer :pointer)
  (define-curl-option-setter set-curl-option-url
    curl-option :url :pointer)
  'curl-option)
@end lisp

@noindent
Macroexpanding one of the new @code{define-curl-option-setter}
forms yields the following:

@lisp
(progn
  (defcfun ("curl_easy_setopt" set-curl-option-nosignal) curl-code
    (easy-handle easy-handle)
    (option curl-option)
    (new-value :long))
  (curry-curl-option-setter 'set-curl-option-nosignal ':nosignal))
@end lisp

@noindent
Finally, let's try this out:

@example
@sc{cffi-user>} (set-curl-option-nosignal *easy-handle* 1)
@result{} 0
@end example

@noindent
Looks like it works just as well.  This interface is now reasonably
high-level to wash out some of the ugliness of the thinnest possible
@code{curl_easy_setopt} @acronym{FFI}, without obscuring the remaining
C bookkeeping details we will explore.


@node Tutorial-Memory, Tutorial-Callbacks, Tutorial-Lisp easy_setopt, Tutorial
@section Memory management

According to the documentation for @code{curl_easy_setopt}, the type
of the third argument when @var{option} is @code{CURLOPT_ERRORBUFFER}
is @code{char*}.  Above, we've defined
@code{set-curl-option-errorbuffer} to accept a @code{:pointer} as the
new option value.  However, there is a @cffi{} type @code{:string},
which translates Lisp strings to C strings when passed as arguments to
foreign function calls.  Why not, then, use @code{:string} as the
@cffi{} type of the third argument?  There are two reasons, both
related to the necessity of breaking abstraction described in
@ref{Tutorial-Abstraction,, Breaking the abstraction}.

The first reason also applies to @code{CURLOPT_URL}, which we will use
to illustrate the point.  Assuming we have changed the type of the
third argument underlying @code{set-curl-option-url} to
@code{:string}, look at these two equivalent forms.

@lisp
(set-curl-option-url *easy-handle* "http://www.cliki.net/CFFI")

@equiv{} (with-foreign-string (url "http://www.cliki.net/CFFI")
     (foreign-funcall "curl_easy_setopt" easy-handle *easy-handle*
                      curl-option :url :pointer url curl-code))
@end lisp

@noindent
The latter, in fact, is mostly equivalent to what a foreign function
call's macroexpansion actually does.  As you can see, the Lisp string
@code{"@clikicffi{}"} is copied into a @code{char} array and
null-terminated; the pointer to beginning of this array, now a C
string, is passed as a @cffi{} @code{:pointer} to the foreign
function.

@cindex dynamic extent
@cindex foreign values with dynamic extent
Unfortunately, the C abstraction has failed us, and we must break it.
While @code{:string} works well for many @code{char*} arguments, it
does not for cases like this.  As the @code{curl_easy_setopt}
documentation explains, ``The string must remain present until curl no
longer needs it, as it doesn't copy the string.''  The C string
created by @code{with-foreign-string}, however, only has dynamic
extent: it is ``deallocated'' when the body (above containing the
@code{foreign-funcall} form) exits.

@cindex premature deallocation
If we are supposed to keep the C string around, but it goes away, what
happens when some @code{libcurl} function tries to access the
@acronym{URL} string?  We have reentered the dreaded world of C
``undefined behavior''.  In some Lisps, it will probably get a chunk
of the Lisp/C stack.  You may segfault.  You may get some random piece
of other data from the heap.  Maybe, in a world where ``dynamic
extent'' is defined to be ``infinite extent'', everything will turn
out fine.  Regardless, results are likely to be almost universally
unpleasant.@footnote{``@i{But I thought Lisp was supposed to protect
me from all that buggy C crap!}''  Before asking a question like that,
remember that you are a stranger in a foreign land, whose residents
have a completely different set of values.}

Returning to the current @code{set-curl-option-url} interface, here is
what we must do:

@lisp
(let (easy-handle)
  (unwind-protect
    (with-foreign-string (url "http://www.cliki.net/CFFI")
      (setf easy-handle (curl-easy-init))
      (set-curl-option-url easy-handle url)
      #|@lispcmt{do more with the easy-handle, like actually get the URL}|#)
    (when easy-handle
      (curl-easy-cleanup easy-handle))))
@end lisp

@c old comment to luis: I go on to say that this isn't obviously
@c extensible to new option settings that require C strings to stick
@c around, as it would involve re-evaluating the unwind-protect form
@c with more dynamic memory allocation.  So I plan to show how to
@c write something similar to ObjC's NSAutoreleasePool, to be managed
@c with a simple unwind-protect form.

@noindent
That is fine for the single string defined here, but for every string
option we want to pass, we have to surround the body of
@code{with-foreign-string} with another @code{with-foreign-string}
wrapper, or else do some extremely error-prone pointer manipulation
and size calculation in advance.  We could alleviate some of the pain
with a recursively expanding macro, but this would not remove the need
to modify the block every time we want to add an option, anathema as
it is to a modular interface.

Before modifying the code to account for this case, consider the other
reason we can't simply use @code{:string} as the foreign type.  In C,
a @code{char *} is a @code{char *}, not necessarily a string.  The
option @code{CURLOPT_ERRORBUFFER} accepts a @code{char *}, but does
not expect anything about the data there.  However, it does expect
that some @code{libcurl} function we call later can write a C string
of up to 255 characters there.  We, the callers of the function, are
expected to read the C string at a later time, exactly the opposite of
what @code{:string} implies.

With the semantics for an input string in mind --- namely, that the
string should be kept around until we @code{curl_easy_cleanup} the
easy handle --- we are ready to extend the Lisp interface:

@lisp
(defvar *easy-handle-cstrings* (make-hash-table)
  "Hashtable of easy handles to lists of C strings that may be
safely freed after the handle is freed.")

(defun make-easy-handle ()
  "Answer a new CURL easy interface handle, to which the lifetime
of C strings may be tied.  See `add-curl-handle-cstring'."
  (let ((easy-handle (curl-easy-init)))
    (setf (gethash easy-handle *easy-handle-cstrings*) '())
    easy-handle))

(defun free-easy-handle (handle)
  "Free CURL easy interface HANDLE and any C strings created to
be its options."
  (curl-easy-cleanup handle)
  (mapc #'foreign-string-free
        (gethash handle *easy-handle-cstrings*))
  (remhash handle *easy-handle-cstrings*))

(defun add-curl-handle-cstring (handle cstring)
  "Add CSTRING to be freed when HANDLE is, answering CSTRING."
  (car (push cstring (gethash handle *easy-handle-cstrings*))))
@end lisp

@noindent
Here we have redefined the interface to create and free handles, to
associate a list of allocated C strings with each handle while it
exists.  The strategy of using different function names to wrap around
simple foreign functions is more common than the solution implemented
earlier with @code{curry-curl-option-setter}, which was to modify the
function name's function slot.@footnote{There are advantages and
disadvantages to each approach; I chose to @code{(setf
symbol-function)} earlier because it entailed generating fewer magic
function names.}

Incidentally, the next step is to redefine
@code{curry-curl-option-setter} to allocate C strings for the
appropriate length of time, given a Lisp string as the
@code{new-value} argument:

@lisp
(defun curry-curl-option-setter (function-name option-keyword)
  "Wrap the function named by FUNCTION-NAME with a version that
curries the second argument as OPTION-KEYWORD.

This function is intended for use in DEFINE-CURL-OPTION-SETTER."
  (setf (symbol-function function-name)
          (let ((c-function (symbol-function function-name)))
            (lambda (easy-handle new-value)
              (funcall c-function easy-handle option-keyword
                       (if (stringp new-value)
                         (add-curl-handle-cstring
                          easy-handle
                          (foreign-string-alloc new-value))
                         new-value))))))
@end lisp

@noindent
A quick analysis of the code shows that you need only reevaluate the
@code{curl-option} enumeration definition to take advantage of these
new semantics.  Now, for good measure, let's reallocate the handle
with the new functions we just defined, and set its @acronym{URL}:

@example
@sc{cffi-user>} (curl-easy-cleanup *easy-handle*)
@result{} NIL
@sc{cffi-user>} (setf *easy-handle* (make-easy-handle))
@result{} #<FOREIGN-ADDRESS #x09844EE0>
@sc{cffi-user>} (set-curl-option-nosignal *easy-handle* 1)
@result{} 0
@sc{cffi-user>} (set-curl-option-url *easy-handle*
                                "http://www.cliki.net/CFFI")
@result{} 0
@end example

@cindex strings
For fun, let's inspect the Lisp value of the C string that was created
to hold @code{"@clikicffi{}"}.  By virtue of the implementation of
@code{add-curl-handle-cstring}, it should be accessible through the
hash table defined:

@example
@sc{cffi-user>} (foreign-string-to-lisp
            (car (gethash *easy-handle* *easy-handle-cstrings*)))
@result{} "http://www.cliki.net/CFFI"
@end example

@noindent
Looks like that worked, and @code{libcurl} now knows what
@acronym{URL} we want to retrieve.

Finally, we turn back to the @code{:errorbuffer} option mentioned at
the beginning of this section.  Whereas the abstraction added to
support string inputs works fine for cases like @code{CURLOPT_URL}, it
hides the detail of keeping the C string; for @code{:errorbuffer},
however, we need that C string.

In a moment, we'll define something slightly cleaner, but for now,
remember that you can always hack around anything.  We're modifying
handle creation, so make sure you free the old handle before
redefining @code{free-easy-handle}.

@smalllisp
(defvar *easy-handle-errorbuffers* (make-hash-table)
  "Hashtable of easy handles to C strings serving as error
writeback buffers.")

;;; @lispcmt{An extra byte is very little to pay for peace of mind.}
(defparameter *curl-error-size* 257
  "Minimum char[] size used by cURL to report errors.")

(defun make-easy-handle ()
  "Answer a new CURL easy interface handle, to which the lifetime
of C strings may be tied.  See `add-curl-handle-cstring'."
  (let ((easy-handle (curl-easy-init)))
    (setf (gethash easy-handle *easy-handle-cstrings*) '())
    (setf (gethash easy-handle *easy-handle-errorbuffers*)
            (foreign-alloc :char :count *curl-error-size*
                           :initial-element 0))
    easy-handle))

(defun free-easy-handle (handle)
  "Free CURL easy interface HANDLE and any C strings created to
be its options."
  (curl-easy-cleanup handle)
  (foreign-free (gethash handle *easy-handle-errorbuffers*))
  (remhash handle *easy-handle-errorbuffers*)
  (mapc #'foreign-string-free
        (gethash handle *easy-handle-cstrings*))
  (remhash handle *easy-handle-cstrings*))

(defun get-easy-handle-error (handle)
  "Answer a string containing HANDLE's current error message."
  (foreign-string-to-lisp
   (gethash handle *easy-handle-errorbuffers*)))
@end smalllisp

Be sure to once again set the options we've set thus far.  You may
wish to define yet another wrapper function to do this.


@node Tutorial-Callbacks, Tutorial-Completion, Tutorial-Memory, Tutorial
@section Calling Lisp from C

If you have been reading
@uref{http://curl.haxx.se/libcurl/c/curl_easy_setopt.html,,
@code{curl_easy_setopt(3)}}, you should have noticed that some options
accept a function pointer.  In particular, we need one function
pointer to set as @code{CURLOPT_WRITEFUNCTION}, to be called by
@code{libcurl} rather than the reverse, in order to receive data as it
is downloaded.

A binding writer without the aid of @acronym{FFI} usually approaches
this problem by writing a C function that accepts C data, converts to
the language's internal objects, and calls the callback provided by
the user, again in a reverse of usual practices.

The @cffi{} approach to callbacks precisely mirrors its differences
with the non-@acronym{FFI} approach on the ``calling C from Lisp''
side, which we have dealt with exclusively up to now.  That is, you
define a callback function in Lisp using @code{defcallback}, and
@cffi{} effectively creates a C function to be passed as a function
pointer.

@impnote{This is much trickier than calling C functions from Lisp, as
it literally involves somehow generating a new C function that is as
good as any created by the compiler.  Therefore, not all Lisps support
them.  @xref{Implementation Support}, for information about @cffi{}
support issues in this and other areas.  You may want to consider
changing to a Lisp that supports callbacks in order to continue with
this tutorial.}

@cindex callback definition
@cindex defining callbacks
Defining a callback is very similar to defining a callout; the main
difference is that we must provide some Lisp forms to be evaluated as
part of the callback.  Here is the signature for the function the
@code{:writefunction} option takes:

@example
size_t
@var{function}(void *ptr, size_t size, size_t nmemb, void *stream);
@end example

@impnote{size_t is almost always an unsigned int.  You can get this
and many other types using feature tests for your system by using
cffi-grovel.}

The above signature trivially translates into a @cffi{}
@code{defcallback} form, as follows.

@lisp
;;; @lispcmt{Alias in case size_t changes.}
(defctype size :unsigned-int)

;;; @lispcmt{To be set as the CURLOPT_WRITEFUNCTION of every easy handle.}
(defcallback easy-write size ((ptr :pointer) (size size)
                              (nmemb size) (stream :pointer))
  (let ((data-size (* size nmemb)))
    (handler-case
      ;; @lispcmt{We use the dynamically-bound *easy-write-procedure* to}
      ;; @lispcmt{call a closure with useful lexical context.}
      (progn (funcall (symbol-value '*easy-write-procedure*)
                      (foreign-string-to-lisp ptr :count data-size))
             data-size)         ;@lispcmt{indicates success}
      ;; @lispcmt{The WRITEFUNCTION should return something other than the}
      ;; @lispcmt{#bytes available to signal an error.}
      (error () (if (zerop data-size) 1 0)))))
@end lisp

First, note the correlation of the first few forms, used to declare
the C function's signature, with the signature in C syntax.  We
provide a Lisp name for the function, its return type, and a name and
type for each argument.

In the body, we call the dynamically-bound
@code{*easy-write-procedure*} with a ``finished'' translation, of
pulling together the raw data and size into a Lisp string, rather than
deal with the data directly.  As part of calling
@code{curl_easy_perform} later, we'll bind that variable to a closure
with more useful lexical bindings than the top-level
@code{defcallback} form.

Finally, we make a halfhearted effort to prevent non-local exits from
unwinding the C stack, covering the most likely case with an
@code{error} handler, which is usually triggered
unexpectedly.@footnote{Unfortunately, we can't protect against
@emph{all} non-local exits, such as @code{return}s and @code{throw}s,
because @code{unwind-protect} cannot be used to ``short-circuit'' a
non-local exit in Common Lisp, due to proposal @code{minimal} in
@uref{http://www.lisp.org/HyperSpec/Issues/iss152-writeup.html,
@acronym{ANSI} issue @sc{Exit-Extent}}.  Furthermore, binding an
@code{error} handler prevents higher-up code from invoking restarts
that may be provided under the callback's dynamic context.  Such is
the way of compromise.}  The reason is that most C code is written to
understand its own idiosyncratic error condition, implemented above in
the case of @code{curl_easy_perform}, and more ``undefined behavior''
can result if we just wipe C stack frames without allowing them to
execute whatever cleanup actions as they like.

Using the @code{CURLoption} enumeration in @file{curl.h} once more, we
can describe the new option by modifying and reevaluating
@code{define-curl-options}.

@lisp
(define-curl-options curl-option
    (long 0 objectpoint 10000 functionpoint 20000 off-t 30000)
  (:noprogress long 43)
  (:nosignal long 99)
  (:errorbuffer objectpoint 10)
  (:url objectpoint 2)
  (:writefunction functionpoint 11)) ;@lispcmt{new item here}
@end lisp

Finally, we can use the defined callback and the new
@code{set-curl-option-writefunction} to finish configuring the easy
handle, using the @code{callback} macro to retrieve a @cffi{}
@code{:pointer}, which works like a function pointer in C code.

@example
@sc{cffi-user>} (set-curl-option-writefunction
            *easy-handle* (callback easy-write))
@result{} 0
@end example


@node Tutorial-Completion, Tutorial-Types, Tutorial-Callbacks, Tutorial
@section A complete @acronym{FFI}?

@c TeX goes insane on @uref{@clikicffi{}}

With all options finally set and a medium-level interface developed,
we can finish the definition and retrieve
@uref{http://www.cliki.net/CFFI}, as is done in the tutorial.

@lisp
(defcfun "curl_easy_perform" curl-code
  (handle easy-handle))
@end lisp

@example
@sc{cffi-user>} (with-output-to-string (contents)
             (let ((*easy-write-procedure*
                     (lambda (string)
                       (write-string string contents))))
               (declare (special *easy-write-procedure*))
               (curl-easy-perform *easy-handle*)))
@result{} "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01//EN\"
@enddots{}
Now fear, comprehensively</P>
"
@end example

Of course, that itself is slightly unwieldy, so you may want to define
a function around it that simply retrieves a @acronym{URL}.  I will
leave synthesis of all the relevant @acronym{REPL} forms presented
thus far into a single function as an exercise for the reader.

The remaining sections of this tutorial explore some advanced features
of @cffi{}; the definition of new types will receive special
attention.  Some of these features are essential for particular
foreign function calls; some are very helpful when trying to develop a
Lispy interface to C.


@node Tutorial-Types, Tutorial-Conclusion, Tutorial-Completion, Tutorial
@section Defining new types

We've occasionally used the @code{defctype} macro in previous sections
as a kind of documentation, much what you'd use @code{typedef} for in
C.  We also tried one special kind of type definition, the
@code{defcenum} type.  @xref{defcstruct}, for a definition macro that
may come in handy if you need to use C @code{struct}s as data.

@cindex type definition
@cindex data in Lisp and C
@cindex translating types
However, all of these are mostly sugar for the powerful underlying
foreign type interface called @dfn{type translators}.  You can easily
define new translators for any simple named foreign type.  Since we've
defined the new type @code{curl-code} to use as the return type for
various @code{libcurl} functions, we can use that to directly convert
c@acronym{URL} errors to Lisp errors.

@code{defctype}'s purpose is to define simple @code{typedef}-like
aliases.  In order to use @dfn{type translators} we must use the
@code{define-foreign-type} macro.  So let's redefine @code{curl-code}
using it.

@lisp
(define-foreign-type curl-code-type ()
  ()
  (:actual-type :int)
  (:simple-parser curl-code))
@end lisp

@code{define-foreign-type} is a thin wrapper around @code{defclass}.
For now, all you need to know in the context of this example is that
it does what @code{(defctype curl-code :int)} would do and,
additionally, defines a new class @code{curl-code-type} which we will
take advantage of shortly.

The @code{CURLcode} enumeration seems to follow the typical error code
convention of @samp{0} meaning all is well, and each non-zero integer
indicating a different kind of error.  We can apply that trivially to
differentiate between normal exits and error exits.

@lisp
(define-condition curl-code-error (error)
  (($code :initarg :curl-code :reader curl-error-code))
  (:report (lambda (c stream)
             (format stream "libcurl function returned error ~A"
                            (curl-error-code c))))
  (:documentation "Signalled when a libcurl function answers
a code other than CURLE_OK."))

(defmethod translate-from-foreign (value (type curl-code-type))
  "Raise a CURL-CODE-ERROR if VALUE, a curl-code, is non-zero."
  (if (zerop value)
      :curle-ok
      (error 'curl-code-error :curl-code value)))
@end lisp

@noindent
The heart of this translator is new method
@code{translate-from-foreign}.  By specializing the @var{type}
parameter on @code{curl-code-type}, we immediately modify the behavior
of every function that returns a @code{curl-code} to pass the result
through this new method.

To see the translator in action, try invoking a function that returns
a @code{curl-code}.  You need to reevaluate the respective
@code{defcfun} form so that it picks up the new @code{curl-code}
definition.

@example
@sc{cffi-user>} (set-curl-option-nosignal *easy-handle* 1)
@result{} :CURLE-OK
@end example

@noindent
As the result was @samp{0}, the new method returned @code{:curle-ok},
just as specified.@footnote{It might be better to return
@code{(values)} than @code{:curle-ok} in real code, but this is good
for illustration.}  I will leave disjoining the separate
@code{CURLcode}s into condition types and improving the @code{:report}
function as an exercise for you.

The creation of @code{*easy-handle-cstrings*} and
@code{*easy-handle-errorbuffers*} as properties of @code{easy-handle}s
is a kluge.  What we really want is a Lisp structure that stores these
properties along with the C pointer.  Unfortunately,
@code{easy-handle} is currently just a fancy name for the foreign type
@code{:pointer}; the actual pointer object varies from Common Lisp
implementation to implementation, needing only to satisfy
@code{pointerp} and be returned from @code{make-pointer} and friends.

One solution that would allow us to define a new Lisp structure to
represent @code{easy-handle}s would be to write a wrapper around every
function that currently takes an @code{easy-handle}; the wrapper would
extract the pointer and pass it to the foreign function.  However, we
can use type translators to more elegantly integrate this
``translation'' into the foreign function calling framework, using
@code{translate-to-foreign}.

@smalllisp
(defclass easy-handle ()
  ((pointer :initform (curl-easy-init)
            :documentation "Foreign pointer from curl_easy_init")
   (error-buffer
    :initform (foreign-alloc :char :count *curl-error-size*
                             :initial-element 0)
    :documentation "C string describing last error")
   (c-strings :initform '()
              :documentation "C strings set as options"))
  (:documentation "I am a parameterization you may pass to
curl-easy-perform to perform a cURL network protocol request."))

(defmethod initialize-instance :after ((self easy-handle) &key)
  (set-curl-option-errorbuffer self (slot-value self 'error-buffer)))

(defun add-curl-handle-cstring (handle cstring)
  "Add CSTRING to be freed when HANDLE is, answering CSTRING."
  (car (push cstring (slot-value handle 'c-strings))))

(defun get-easy-handle-error (handle)
  "Answer a string containing HANDLE's current error message."
  (foreign-string-to-lisp
   (slot-value handle 'error-buffer)))

(defun free-easy-handle (handle)
  "Free CURL easy interface HANDLE and any C strings created to
be its options."
  (with-slots (pointer error-buffer c-strings) handle
    (curl-easy-cleanup pointer)
    (foreign-free error-buffer)
    (mapc #'foreign-string-free c-strings)))

(define-foreign-type easy-handle-type ()
  ()
  (:actual-type :pointer)
  (:simple-parser easy-handle))

(defmethod translate-to-foreign (handle (type easy-handle-type))
  "Extract the pointer from an easy-HANDLE."
  (slot-value handle 'pointer))
@end smalllisp

While we changed some of the Lisp functions defined earlier to use
@acronym{CLOS} slots rather than hash tables, the foreign functions
work just as well as they did before.

@cindex limitations of type translators
The greatest strength, and the greatest limitation, of the type
translator comes from its generalized interface.  As stated
previously, we could define all foreign function calls in terms of the
primitive foreign types provided by @cffi{}.  The type translator
interface allows us to cleanly specify the relationship between Lisp
and C data, independent of where it appears in a function call.  This
independence comes at a price; for example, it cannot be used to
modify translation semantics based on other arguments to a function
call.  In these cases, you should rely on other features of Lisp,
rather than the powerful, yet domain-specific, type translator
interface.


@node Tutorial-Conclusion,  , Tutorial-Types, Tutorial
@section What's next?

@cffi{} provides a rich and powerful foundation for communicating with
foreign libraries; as we have seen, it is up to you to make that
experience a pleasantly Lispy one.  This tutorial does not cover all
the features of @cffi{}; please see the rest of the manual for
details.  In particular, if something seems obviously missing, it is
likely that either code or a good reason for lack of code is already
present.

@impnote{There are some other things in @cffi{} that might deserve
tutorial sections, such as free-translated-object, or structs.  Let us
know which ones you care about.}


@c ===================================================================
@c CHAPTER: Wrapper generators

@node Wrapper generators, Foreign Types, Tutorial, Top
@chapter Wrapper generators

@cffi{}'s interface is designed for human programmers, being aimed at
aesthetic as well as technical sophistication.  However, there are a
few programs aimed at translating C and C++ header files, or
approximations thereof, into @cffi{} forms constituting a foreign
interface to the symbols in those files.

These wrapper generators are known to support output of @cffi{} forms.

@table @asis
@item @uref{http://www.cliki.net/Verrazano,Verrazano}
Designed specifically for Common Lisp.  Uses @acronym{GCC}'s parser
output in @acronym{XML} format to discover functions, variables, and
other header file data.  This means you need @acronym{GCC} to generate
forms; on the other hand, the parser employed is mostly compliant with
@acronym{ANSI} C.

@item @uref{http://www.cliki.net/SWIG,SWIG}
A foreign interface generator originally designed to generate Python
bindings, it has been ported to many other systems, including @cffi{}
in version 1.3.28.  Includes its own C declaration munger, not
intended to be fully-compliant with @acronym{ANSI} C.
@end table

First, this manual does not describe use of these other programs; they
have documentation of their own.  If you have problems using a
generated interface, please look at the output @cffi{} forms and
verify that they are a correct @cffi{} interface to the library in
question; if they are correct, contact @cffi{} developers with
details, keeping in mind that they communicate in terms of those forms
rather than any particular wrapper generator.  Otherwise, contact the
maintainers of the wrapper generator you are using, provided you can
reasonably expect more accuracy from the generator.

When is more accuracy an unreasonable expectation?  As described in
the tutorial (@pxref{Tutorial-Abstraction,, Breaking the
abstraction}), the information in C declarations is insufficient to
completely describe every interface.  In fact, it is quite common to
run into an interface that cannot be handled automatically, and
generators should be excused from generating a complete interface in
these cases.

As further described in the tutorial, the thinnest Lisp interface to a
C function is not always the most pleasant one.  In many cases, you
will want to manually write a Lispier interface to the C functions
that interest you.

Wrapper generators should be treated as time-savers, not complete
automation of the full foreign interface writing job.  Reports of the
amount of work done by generators vary from 30% to 90%.  The
incremental development style enabled by @cffi{} generally reduces
this proportion below that for languages like Python.

@c Where I got the above 30-90% figures:
@c 30%: lemonodor's post about SWIG
@c 90%: Balooga on #lisp.  He said 99%, but that's probably an
@c      exaggeration (leave it to me to pass judgement :)
@c -stephen


@c ===================================================================
@c CHAPTER: Foreign Types

@node Foreign Types, Pointers, Wrapper generators, Top
@chapter Foreign Types

Foreign types describe how data is translated back and forth between C
and Lisp. @cffi{} provides various built-in types and allows the user to
define new types.

@menu
* Built-In Types::
* Other Types::
* Defining Foreign Types::
* Foreign Type Translators::
* Optimizing Type Translators::
* Foreign Structure Types::
* Allocating Foreign Objects::

Dictionary

* convert-from-foreign::
* convert-to-foreign::
* defbitfield::
* defcstruct::
* defcunion::
* defctype::
* defcenum::
@c * define-type-spec-parser::
* define-foreign-type::
* define-parse-method::
@c * explain-foreign-slot-value:
* foreign-bitfield-symbols::
* foreign-bitfield-value::
* foreign-enum-keyword::
* foreign-enum-value::
* foreign-slot-names::
* foreign-slot-offset::
* foreign-slot-pointer::
* foreign-slot-value::
* foreign-type-alignment::
* foreign-type-size::
* free-converted-object::
* free-translated-object::
* translate-from-foreign::
* translate-to-foreign::
* translate-into-foreign-memory::
* with-foreign-slots::
@end menu

@node Built-In Types, Other Types, Foreign Types, Foreign Types
@section Built-In Types

@ForeignType{:char}
@ForeignType{:unsigned-char}
@ForeignType{:short}
@ForeignType{:unsigned-short}
@ForeignType{:int}
@ForeignType{:unsigned-int}
@ForeignType{:long}
@ForeignType{:unsigned-long}
@ForeignType{:long-long}
@ForeignType{:unsigned-long-long}

These types correspond to the native C integer types according to the
@acronym{ABI} of the Lisp implementation's host system.

@code{:long-long} and @code{:unsigned-long-long} are not supported
natively on all implementations. However, they are emulated by
@code{mem-ref} and @code{mem-set}.

When those types are @strong{not} available, the symbol
@code{cffi-sys::no-long-long} is pushed into @code{*features*}.

@ForeignType{:uchar}
@ForeignType{:ushort}
@ForeignType{:uint}
@ForeignType{:ulong}
@ForeignType{:llong}
@ForeignType{:ullong}

For convenience, the above types are provided as shortcuts for
@code{unsigned-char}, @code{unsigned-short}, @code{unsigned-int},
@code{unsigned-long}, @code{long-long} and @code{unsigned-long-long},
respectively.

@ForeignType{:int8}
@ForeignType{:uint8}
@ForeignType{:int16}
@ForeignType{:uint16}
@ForeignType{:int32}
@ForeignType{:uint32}
@ForeignType{:int64}
@ForeignType{:uint64}

Foreign integer types of specific sizes, corresponding to the C types
defined in @code{stdint.h}.

@c @ForeignType{:size}
@c @ForeignType{:ssize}
@c @ForeignType{:ptrdiff}
@c @ForeignType{:time}

@c Foreign integer types corresponding to the standard C types (without
@c the @code{_t} suffix).

@c @impnote{These are not implemented yet. --luis}

@c @impnote{I'm sure there are more of these that could be useful, let's
@c add any types that can't be defined portably to this list as
@c necessary. --james}

@ForeignType{:float}
@ForeignType{:double}

On all systems, the @code{:float} and @code{:double} types represent a
C @code{float} and @code{double}, respectively. On most but not all
systems, @code{:float} and @code{:double} represent a Lisp
@code{single-float} and @code{double-float}, respectively. It is not
so useful to consider the relationship between Lisp types and C types
as isomorphic, as simply to recognize the relationship, and relative
precision, among each respective category.

@ForeignType{:long-double}

This type is only supported on SCL.

@ForeignType{:pointer &optional type}

A foreign pointer to an object of any type, corresponding to
@code{void *}.  You can optionally specify type of pointer
(e.g. @code{(:pointer :char)}).  Although @cffi{} won't do anything
with that information yet, it is useful for documentation purposes.

@ForeignType{:void}

No type at all. Only valid as the return type of a function.

@node Other Types, Defining Foreign Types, Built-In Types, Foreign Types
@section Other Types

@cffi{} also provides a few useful types that aren't built-in C
types.

@ForeignType{:string}

The @code{:string} type performs automatic conversion between Lisp and
C strings. Note that, in the case of functions the converted C string
will have dynamic extent (i.e.@: it will be automatically freed after
the foreign function returns).

In addition to Lisp strings, this type will accept foreign pointers
and pass them unmodified.

A method for @ref{free-translated-object} is specialized for this
type. So, for example, foreign strings allocated by this type and
passed to a foreign function will be freed after the function
returns.

@lisp
CFFI> (foreign-funcall "getenv" :string "SHELL" :string)
@result{} "/bin/bash"

CFFI> (with-foreign-string (str "abcdef")
        (foreign-funcall "strlen" :string str :int))
@result{} 6
@end lisp

@ForeignType{:string+ptr}

Like @code{:string} but returns a list with two values when convert
from C to Lisp: a Lisp string and the C string's foreign pointer.

@lisp
CFFI> (foreign-funcall "getenv" :string "SHELL" :string+ptr)
@result{} ("/bin/bash" #.(SB-SYS:INT-SAP #XBFFFFC6F))
@end lisp

@ForeignType{:boolean &optional (base-type :int)}

The @code{:boolean} type converts between a Lisp boolean and a C
boolean. It canonicalizes to @var{base-type} which is @code{:int} by
default.

@lisp
(convert-to-foreign nil :boolean) @result{} 0
(convert-to-foreign t :boolean) @result{} 1
(convert-from-foreign 0 :boolean) @result{} nil
(convert-from-foreign 1 :boolean) @result{} t
@end lisp

@ForeignType{:bool}

The @code{:bool} type represents the C99 @code{_Bool} or C++
@code{bool}. Its size is usually 1 byte except on OSX where it's an
@code{int}.

@ForeignType{:wrapper base-type &key to-c from-c}

The @code{:wrapper} type stores two symbols passed to the @var{to-c}
and @var{from-c} arguments. When a value is being translated to or
from C, this type @code{funcall}s the respective symbol.

@code{:wrapper} types will be typedefs for @var{base-type} and will
inherit its translators, if any.

Here's an example of how the @code{:boolean} type could be defined in
terms of @code{:wrapper}.

@lisp
(defun bool-c-to-lisp (value)
  (not (zerop value)))

(defun bool-lisp-to-c (value)
  (if value 1 0))

(defctype my-bool (:wrapper :int :from-c bool-c-to-lisp
                                 :to-c bool-lisp-to-c))

(convert-to-foreign nil 'my-bool) @result{} 0
(convert-from-foreign 1 'my-bool) @result{} t
@end lisp

@node Defining Foreign Types, Foreign Type Translators, Other Types, Foreign Types
@section Defining Foreign Types

You can define simple C-like @code{typedef}s through the
@code{defctype} macro. Defining a typedef is as simple as giving
@code{defctype} a new name and the name of the type to be wrapped.

@lisp
;;; @lispcmt{Define MY-INT as an alias for the built-in type :INT.}
(defctype my-int :int)
@end lisp

With this type definition, one can, for instance, declare arguments to
foreign functions as having the type @code{my-int}, and they will be
passed as integers.

@subheading More complex types

@cffi{} offers another way to define types through
@code{define-foreign-type}, a thin wrapper macro around
@code{defclass}. As an example, let's go through the steps needed to
define a @code{(my-string &key encoding)} type. First, we need to
define our type class:

@lisp
(define-foreign-type my-string-type ()
  ((encoding :reader string-type-encoding :initarg :encoding))
  (:actual-type :pointer))
@end lisp

The @code{:actual-type} class option tells CFFI that this type will
ultimately be passed to and received from foreign code as a
@code{:pointer}. Now you need to tell CFFI how to parse a type
specification such as @code{(my-string :encoding :utf8)} into an
instance of @code{my-string-type}.  We do that with
@code{define-parse-method}:

@lisp
(define-parse-method my-string (&key (encoding :utf-8))
  (make-instance 'my-string-type :encoding encoding))
@end lisp

The next section describes how make this type actually translate
between C and Lisp strings.

@node Foreign Type Translators, Optimizing Type Translators, Defining Foreign Types, Foreign Types
@section Foreign Type Translators

Type translators are used to automatically convert Lisp values to or
from foreign values.  For example, using type translators, one can
take the @code{my-string} type defined in the previous section and
specify that it should:

@itemize
@item
convert C strings to Lisp strings;
@item
convert Lisp strings to newly allocated C strings;
@item
free said C strings when they are no longer needed.
@end itemize

In order to tell @cffi{} how to automatically convert Lisp values to
foreign values, define a specialized method for the
@code{translate-to-foreign} generic function:

@lisp
;;; @lispcmt{Define a method that converts Lisp strings to C strings.}
(defmethod translate-to-foreign (string (type my-string-type))
  (foreign-string-alloc string :encoding (string-type-encoding type)))
@end lisp

@noindent
From now on, whenever an object is passed as a @code{my-string} to a
foreign function, this method will be invoked to convert the Lisp
value. To perform the inverse operation, which is needed for functions
that return a @code{my-string}, specialize the
@code{translate-from-foreign} generic function in the same manner:

@lisp
;;; @lispcmt{Define a method that converts C strings to Lisp strings.}
(defmethod translate-from-foreign (pointer (type my-string-type))
  (foreign-string-to-lisp pointer :encoding (string-type-encoding type)))
@end lisp

@noindent
When a @code{translate-to-foreign} method requires allocation of
foreign memory, you must also define a @code{free-translated-object}
method to free the memory once the foreign object is no longer needed,
otherwise you'll be faced with memory leaks.  This generic function is
called automatically by @cffi{} when passing objects to foreign
functions. Let's do that:

@lisp
;;; @lispcmt{Free strings allocated by translate-to-foreign.}
(defmethod free-translated-object (pointer (type my-string-type) param)
  (declare (ignore param))
  (foreign-string-free pointer))
@end lisp

@noindent
In this specific example, we don't need the @var{param} argument, so
we ignore it. See @ref{free-translated-object}, for an explanation of
its purpose and how you can use it.

A type translator does not necessarily need to convert the value.  For
example, one could define a typedef for @code{:pointer} that ensures,
in the @code{translate-to-foreign} method, that the value is not a
null pointer, signalling an error if a null pointer is passed.  This
would prevent some pointer errors when calling foreign functions that
cannot handle null pointers.

@strong{Please note:} these methods are meant as extensible hooks
only, and you should not call them directly.  Use
@code{convert-to-foreign}, @code{convert-from-foreign} and
@code{free-converted-object} instead.

@xref{Tutorial-Types,, Defining new types}, for another example of
type translators.

@node Optimizing Type Translators, Foreign Structure Types, Foreign Type Translators, Foreign Types
@section Optimizing Type Translators

@cindex type translators, optimizing
@cindex compiler macros for type translation
@cindex defining type-translation compiler macros
Being based on generic functions, the type translation mechanism
described above can add a bit of overhead.  This is usually not
significant, but we nevertheless provide a way of getting rid of the
overhead for the cases where it matters.

A good way to understand this issue is to look at the code generated
by @code{defcfun}. Consider the following example using the previously
defined @code{my-string} type:

@lisp
CFFI> (macroexpand-1 '(defcfun foo my-string (x my-string)))
;; @lispcmt{(simplified, downcased, etc...)}
(defun foo (x)
  (multiple-value-bind (#:G2019 #:PARAM3149)
      (translate-to-foreign x #<MY-STRING-TYPE @{11ED5A79@}>)
    (unwind-protect
        (translate-from-foreign
         (foreign-funcall "foo" :pointer #:G2019 :pointer)
         #<MY-STRING-TYPE @{11ED5659@}>)
      (free-translated-object #:G2019 #<MY-STRING-TYPE @{11ED51A79@}>
                              #:PARAM3149))))
@end lisp

@noindent
In order to get rid of those generic function calls, @cffi{} has
another set of extensible generic functions that provide functionality
similar to @acronym{CL}'s compiler macros:
@code{expand-to-foreign-dyn}, @code{expand-to-foreign} and
@code{expand-from-foreign}. Here's how one could define a
@code{my-boolean} with them:

@lisp
(define-foreign-type my-boolean-type ()
  ()
  (:actual-type :int)
  (:simple-parser my-boolean))

(defmethod expand-to-foreign (value (type my-boolean-type))
  `(if ,value 1 0))

(defmethod expand-from-foreign (value (type my-boolean-type))
  `(not (zerop ,value)))
@end lisp

@noindent
And here's what the macroexpansion of a function using this type would
look like:

@lisp
CFFI> (macroexpand-1 '(defcfun bar my-boolean (x my-boolean)))
;; @lispcmt{(simplified, downcased, etc...)}
(defun bar (x)
  (let ((#:g3182 (if x 1 0)))
    (not (zerop (foreign-funcall "bar" :int #:g3182 :int)))))
@end lisp

@noindent
No generic function overhead.

Let's go back to our @code{my-string} type.  The expansion interface
has no equivalent of @code{free-translated-object}; you must instead
define a method on @code{expand-to-foreign-dyn}, the third generic
function in this interface.  This is especially useful when you can
allocate something much more efficiently if you know the object has
dynamic extent, as is the case with function calls that don't save the
relevant allocated arguments.

This exactly what we need for the @code{my-string} type:

@lisp
(defmethod expand-from-foreign (form (type my-string-type))
  `(foreign-string-to-lisp ,form))

(defmethod expand-to-foreign-dyn (value var body (type my-string-type))
  (let ((encoding (string-type-encoding type)))
    `(with-foreign-string (,var ,value :encoding ',encoding)
       ,@@body)))
@end lisp

@noindent
So let's look at the macro expansion:

@lisp
CFFI> (macroexpand-1 '(defcfun foo my-string (x my-string)))
;; @lispcmt{(simplified, downcased, etc...)}
(defun foo (x)
  (with-foreign-string (#:G2021 X :encoding ':utf-8)
    (foreign-string-to-lisp
     (foreign-funcall "foo" :pointer #:g2021 :pointer))))
@end lisp

@noindent
Again, no generic function overhead.

@subheading Other details

To short-circuit expansion and use the @code{translate-*} functions
instead, simply call the next method.  Return its result in cases
where your method cannot generate an appropriate replacement for it.
This analogous to the @code{&whole form} mechanism compiler macros
provide.

The @code{expand-*} methods have precedence over their
@code{translate-*} counterparts and are guaranteed to be used in
@code{defcfun}, @code{foreign-funcall}, @code{defcvar} and
@code{defcallback}.  If you define a method on each of the
@code{expand-*} generic functions, you are guaranteed to have full
control over the expressions generated for type translation in these
macros.

They may or may not be used in other @cffi{} operators that need to
translate between Lisp and C data; you may only assume that
@code{expand-*} methods will probably only be called during Lisp
compilation.

@code{expand-to-foreign-dyn} has precedence over
@code{expand-to-foreign} and is only used in @code{defcfun} and
@code{foreign-funcall}, only making sense in those contexts.

@strong{Important note:} this set of generic functions is called at
macroexpansion time.  Methods are defined when loaded or evaluated,
not compiled.  You are responsible for ensuring that your
@code{expand-*} methods are defined when the @code{foreign-funcall} or
other forms that use them are compiled.  One way to do this is to put
the method definitions earlier in the file and inside an appropriate
@code{eval-when} form; another way is to always load a separate Lisp
or @acronym{FASL} file containing your @code{expand-*} definitions
before compiling files with forms that ought to use them.  Otherwise,
they will not be found and the runtime translators will be used
instead.

@node Foreign Structure Types, Allocating Foreign Objects, Optimizing Type Translators, Foreign Types
@section Foreign Structure Types

For more involved C types than simple aliases to built-in types, such
as you can make with @code{defctype}, @cffi{} allows declaration of
structures and unions with @code{defcstruct} and @code{defcunion}.

For example, consider this fictional C structure declaration holding
some personal information:

@example
struct person @{
  int number;
  char* reason;
@};
@end example

@noindent
The equivalent @code{defcstruct} form follows:

@lisp
(defcstruct person
  (number :int)
  (reason :string))
@end lisp

@c LMH structure translation
By default, @ref{convert-from-foreign} (and also @ref{mem-ref}) will
make a plist with slot names as keys, and @ref{convert-to-foreign} will
translate such a plist to a foreign structure.  A user wishing to define
other translations should use the @code{:class} argument to
@ref{defcstruct}, and then define methods for
@ref{translate-from-foreign} and
@ref{translate-into-foreign-memory} that specialize on this class,
possibly calling @code{call-next-method} to translate from and to the
plists rather than provide a direct interface to the foreign object.
The macro @code{translation-forms-for-class} will generate the forms
necessary to translate a Lisp class into a foreign structure and vice
versa.
@c Write separate function doc section for translation-forms-for-class?
@c Examples, perhaps taken from the tests?

Please note that this interface is only for those that must know about
the values contained in a relevant struct.  If the library you are
interfacing returns an opaque pointer that needs only be passed to
other C library functions, by all means just use @code{:pointer} or a
type-safe definition munged together with @code{defctype} and type
translation.  To pass or return a structure by value to a function, load
the cffi-libffi system and specify the structure as @code{(:struct
@var{structure-name})}.  To pass or return the pointer, you can use
either @code{:pointer} or @code{(:pointer (:struct
@var{structure-name}))}.

@subheading Optimizing translate-into-foreign-memory

Just like how @ref{translate-from-foreign} had
@code{expand-from-foreign} to optimize away the generic function call
and @ref{translate-to-foreign} had the same in
@code{expand-to-foreign}, @ref{translate-into-foreign-memory} has
@code{expand-into-foreign-memory}.

Let's use our @code{person} struct in an example. However, we are
going to spice it up by using a lisp struct rather than a plist to
represent the person in lisp.

First we redefine @code{person} very slightly.

@lisp
(defcstruct (person :class c-person)
  (number :int)
  (reason :string))
@end lisp

By adding @code{:class} we can specialize the @code{translate-*}
methods on the type @code{c-person}.

Next we define a lisp struct to use instead of the plists.

@lisp
(defstruct lisp-person
  (number 0 :type integer)
  (reason "" :type string))
@end lisp

And now let's define the type translators we know already:

@lisp
(defmethod translate-from-foreign (ptr (type c-person))
  (with-foreign-slots ((number reason) ptr (:struct person))
    (make-lisp-person :number number :reason reason)))

(defmethod expand-from-foreign (ptr (type c-person))
  `(with-foreign-slots ((number reason) ,ptr (:struct person))
     (make-lisp-person :number number :reason reason)))

(defmethod translate-into-foreign-memory (value (type c-person) ptr)
  (with-foreign-slots ((number reason) ptr (:struct person))
    (setf number (lisp-person-number value)
          reason (lisp-person-reason value))))
@end lisp

At this point everything works, we can convert to and from our
@code{lisp-person} and foreign @code{person}. If we macroexpand

@lisp
(setf (mem-aref ptr '(:struct person)) x)
@end lisp

we get something like:

@lisp
(let ((#:store879 x))
  (translate-into-foreign-memory #:store879 #<c-person person>
                                 (inc-pointer ptr 0))
  #:store879)
@end lisp

Which is good, but now we can do better and get rid of that generic
function call to @code{translate-into-foreign-memory}.

@lisp
(defmethod expand-into-foreign-memory (value (type c-person) ptr)
  `(with-foreign-slots ((number reason) ,ptr (:struct person))
     (setf number (lisp-person-number ,value)
           reason (lisp-person-reason ,value))))
@end lisp

Now we can expand again so see the changes:

@lisp
;; this:
(setf (mem-aref ptr '(:struct person)) x)

;; expands to this
;; (simplified, downcased, etc..)
(let ((#:store887 x))
  (with-foreign-slots ((number reason) (inc-pointer ptr 0) (:struct person))
    (setf number (lisp-person-number #:store887)
          reason (lisp-person-reason #:store887))) #:store887)
@end lisp

And there we are, no generic function overhead.

@subheading Compatibility note

Previous versions of CFFI accepted the
``bare'' @var{structure-name} as a type specification, which was
interpreted as a pointer to the structure.  This is deprecated and
produces a style warning.  Using this deprecated form means that
@ref{mem-aref} retains its prior meaning and returns a pointer.  Using
the @code{(:struct @var{structure-name})} form for the type,
@ref{mem-aref} provides a Lisp object translated from the
structure (by default a plist).  Thus the semantics are consistent with all
types in returning the object as represented in Lisp, and not a pointer,
with the exception of the ``bare'' structure compatibility retained.
In order to obtain the pointer, you should use the function @ref{mem-aptr}.

See @ref{defcstruct} for more details.

@node Allocating Foreign Objects, convert-from-foreign, Foreign Structure Types, Foreign Types
@section Allocating Foreign Objects

@c I moved this because I moved with-foreign-object to the Pointers
@c chapter, where foreign-alloc is.

@xref{Allocating Foreign Memory}.


@c ===================================================================
@c CONVERT-FROM-FOREIGN

@page
@node convert-from-foreign, convert-to-foreign, Allocating Foreign Objects, Foreign Types
@heading convert-from-foreign
@subheading Syntax
@Function{convert-from-foreign foreign-value type @res{} value}

@subheading Arguments and Values

@table @var
@item foreign-value
The primitive C value as returned from a primitive foreign function or
from @code{convert-to-foreign}.

@item type
A @cffi{} type specifier.

@item value
The Lisp value translated from @var{foreign-value}.
@end table

@subheading Description

This is an external interface to the type translation facility.  In
the implementation, all foreign functions are ultimately defined as
type translation wrappers around primitive foreign function
invocations.

This function is available mostly for inspection of the type
translation process, and possibly optimization of special cases of
your foreign function calls.

Its behavior is better described under @code{translate-from-foreign}'s
documentation.

@subheading Examples

@lisp
CFFI-USER> (convert-to-foreign "a boat" :string)
@result{} #<FOREIGN-ADDRESS #x097ACDC0>
@result{} T
CFFI-USER> (convert-from-foreign * :string)
@result{} "a boat"
@end lisp

@subheading See Also
@seealso{convert-to-foreign} @*
@seealso{free-converted-object} @*
@seealso{translate-from-foreign}


@c ===================================================================
@c CONVERT-TO-FOREIGN

@page
@node convert-to-foreign, defbitfield, convert-from-foreign, Foreign Types
@heading convert-to-foreign
@subheading Syntax
@Function{convert-to-foreign value type @res{} foreign-value, alloc-params}

@subheading Arguments and Values

@table @var
@item value
The Lisp object to be translated to a foreign object.

@item type
A @cffi{} type specifier.

@item foreign-value
The primitive C value, ready to be passed to a primitive foreign
function.

@item alloc-params
Something of a translation state; you must pass it to
@code{free-converted-object} along with the foreign value for that to
work.
@end table

@subheading Description

This is an external interface to the type translation facility.  In
the implementation, all foreign functions are ultimately defined as
type translation wrappers around primitive foreign function
invocations.

This function is available mostly for inspection of the type
translation process, and possibly optimization of special cases of
your foreign function calls.

Its behavior is better described under @code{translate-to-foreign}'s
documentation.

@subheading Examples

@lisp
CFFI-USER> (convert-to-foreign t :boolean)
@result{} 1
@result{} NIL
CFFI-USER> (convert-to-foreign "hello, world" :string)
@result{} #<FOREIGN-ADDRESS #x097C5F80>
@result{} T
CFFI-USER> (code-char (mem-aref * :char 5))
@result{} #\,
@end lisp

@subheading See Also
@seealso{convert-from-foreign} @*
@seealso{free-converted-object} @*
@seealso{translate-to-foreign}


@c ===================================================================
@c DEFBITFIELD

@page
@node defbitfield, defcstruct, convert-to-foreign, Foreign Types
@heading defbitfield
@subheading Syntax
@Macro{defbitfield name-and-options &body masks}

masks ::= [docstring] @{ (symbol value) @}* @*
name-and-options ::= name | (name &optional (base-type :int))

@subheading Arguments and Values

@table @var
@item name
The name of the new bitfield type.

@item docstring
A documentation string, ignored.

@item base-type
A symbol denoting a foreign type.

@item symbol
A Lisp symbol.

@item value
An integer representing a bitmask.
@end table

@subheading Description
The @code{defbitfield} macro is used to define foreign types that map
lists of symbols to integer values.

If @var{value} is omitted, it will be computed as follows: find the
greatest @var{value} previously used, including those so computed,
with only a single 1-bit in its binary representation (that is, powers
of two), and left-shift it by one.  This rule guarantees that a
computed @var{value} cannot clash with previous values, but may clash
with future explicitly specified values.

Symbol lists will be automatically converted to values and vice versa
when being passed as arguments to or returned from foreign functions,
respectively. The same applies to any other situations where an object
of a bitfield type is expected.

Types defined with @code{defbitfield} canonicalize to @var{base-type}
which is @code{:int} by default.

@subheading Examples
@lisp
(defbitfield open-flags
  (:rdonly #x0000)
  :wronly               ;@lispcmt{#x0001}
  :rdwr                 ;@lispcmt{@dots{}}
  :nonblock
  :append
  (:creat  #x0200))
  ;; @lispcmt{etc@dots{}}

CFFI> (foreign-bitfield-symbols 'open-flags #b1101)
@result{} (:WRONLY :NONBLOCK :APPEND)

CFFI> (foreign-bitfield-value 'open-flags '(:rdwr :creat))
@result{} 514   ; #x0202

(defcfun ("open" unix-open) :int
  (path :string)
  (flags open-flags)
  (mode :uint16)) ; unportable

CFFI> (unix-open "/tmp/foo" '(:wronly :creat) #o644)
@result{} #<an fd>

;;; @lispcmt{Consider also the following lispier wrapper around open()}
(defun lispier-open (path mode &rest flags)
  (unix-open path flags mode))
@end lisp

@subheading See Also
@seealso{foreign-bitfield-value} @*
@seealso{foreign-bitfield-symbols}


@c ===================================================================
@c DEFCSTRUCT

@page
@node defcstruct, defcunion, defbitfield, Foreign Types
@heading defcstruct
@subheading Syntax
@Macro{defcstruct name-and-options &body doc-and-slots @res{} name}

name-and-options ::= structure-name | (structure-name &key size)

doc-and-slots ::= [docstring] @{ (slot-name slot-type &key count offset) @}*

@subheading Arguments and Values

@table @var
@item structure-name
The name of new structure type.

@item docstring
A documentation string, ignored.

@item slot-name
A symbol naming the slot.  It must be unique among slot names in this
structure.

@item size
Use this option to override the size (in bytes) of the struct.

@item slot-type
The type specifier for the slot.

@item count
Used to declare an array of size @var{count} inside the
structure.  Defaults to @code{1} as such an array and a single element
are semantically equivalent.

@item offset
Overrides the slot's offset. The next slot's offset is calculated
based on this one.
@end table

@subheading Description
This defines a new @cffi{} aggregate type akin to C @code{struct}s.
In other words, it specifies that foreign objects of the type
@var{structure-name} are groups of different pieces of data, or
``slots'', of the @var{slot-type}s, distinguished from each other by
the @var{slot-name}s.  Each structure is located in memory at a
position, and the slots are allocated sequentially beginning at that
point in memory (with some padding allowances as defined by the C
@acronym{ABI}, unless otherwise requested by specifying an
@var{offset} from the beginning of the structure (offset 0).

In other words, it is isomorphic to the C @code{struct}, giving
several extra features.

There are two kinds of slots, for the two kinds of @cffi{} types:

@table @dfn
@item Simple
Contain a single instance of a type that canonicalizes to a built-in
type, such as @code{:long} or @code{:pointer}.  Used for simple
@cffi{} types.

@item Aggregate
Contain an embedded structure or union, or an array of objects.  Used
for aggregate @cffi{} types.
@end table

The use of @acronym{CLOS} terminology for the structure-related
features is intentional; structure definitions are very much like
classes with (far) fewer features.

@subheading Examples
@lisp
(defcstruct point
  "Point structure."
  (x :int)
  (y :int))

CFFI> (with-foreign-object (ptr 'point)
        ;; @lispcmt{Initialize the slots}
        (setf (foreign-slot-value ptr 'point 'x) 42
              (foreign-slot-value ptr 'point 'y) 42)
        ;; @lispcmt{Return a list with the coordinates}
        (with-foreign-slots ((x y) ptr point)
          (list x y)))
@result{} (42 42)
@end lisp

@lisp
;; @lispcmt{Using the :size and :offset options to define a partial structure.}
;; @lispcmt{(this is useful when you are interested in only a few slots}
;; @lispcmt{of a big foreign structure)}

(defcstruct (foo :size 32)
  "Some struct with 32 bytes."
                        ; @lispcmt{<16 bytes we don't care about>}
  (x :int :offset 16)   ; @lispcmt{an int at offset 16}
  (y :int)              ; @lispcmt{another int at offset 16+sizeof(int)}
                        ; @lispcmt{<a couple more bytes we don't care about>}
  (z :char :offset 24)) ; @lispcmt{a char at offset 24}
                        ; @lispcmt{<7 more bytes ignored (since size is 32)>}

CFFI> (foreign-type-size 'foo)
@result{} 32
@end lisp

@lisp
;;; @lispcmt{Using :count to define arrays inside of a struct.}
(defcstruct video_tuner
  (name :char :count 32))
@end lisp

@subheading See Also
@seealso{foreign-slot-pointer} @*
@seealso{foreign-slot-value} @*
@seealso{with-foreign-slots}


@c ===================================================================
@c DEFCUNION

@page
@node defcunion, defctype, defcstruct, Foreign Types
@heading defcunion
@subheading Syntax
@Macro{defcunion name &body doc-and-slots @res{} name}

doc-and-slots ::= [docstring] @{ (slot-name slot-type &key count) @}*

@subheading Arguments and Values

@table @var
@item name
The name of new union type.

@item docstring
A documentation string, ignored.

@item slot-name
A symbol naming the slot.

@item slot-type
The type specifier for the slot.

@item count
Used to declare an array of size @var{count} inside the
structure.
@end table

@subheading Description
A union is a structure in which all slots have an offset of zero.  It
is isomorphic to the C @code{union}.  Therefore, you should use the
usual foreign structure operations for accessing a union's slots.

@subheading Examples
@lisp
(defcunion uint32-bytes
  (int-value :unsigned-int)
  (bytes :unsigned-char :count 4))
@end lisp

@subheading See Also
@seealso{foreign-slot-pointer} @*
@seealso{foreign-slot-value}


@c ===================================================================
@c DEFCTYPE

@page
@node defctype, defcenum, defcunion, Foreign Types
@heading defctype
@subheading Syntax
@Macro{defctype name base-type &optional documentation}

@subheading Arguments and Values

@table @var
@item name
The name of the new foreign type.

@item base-type
A symbol or a list defining the new type.

@item documentation
A documentation string, currently ignored.
@end table

@subheading Description
The @code{defctype} macro provides a mechanism similar to C's
@code{typedef} to define new types. The new type inherits
@var{base-type}'s translators, if any. There is no way to define
translations for types defined with @code{defctype}.  For that,
you should use @ref{define-foreign-type}.

@subheading Examples
@lisp
(defctype my-string :string
  "My own string type.")

(defctype long-bools (:boolean :long)
  "Booleans that map to C longs.")
@end lisp

@subheading See Also
@seealso{define-foreign-type}


@c ===================================================================
@c DEFCENUM

@page
@node defcenum, define-foreign-type, defctype, Foreign Types
@heading defcenum
@subheading Syntax
@Macro{defcenum name-and-options &body enum-list}

enum-list ::= [docstring] @{ keyword | (keyword value) @}*
name-and-options ::= name | (name &optional (base-type :int))

@subheading Arguments and Values

@table @var
@item name
The name of the new enum type.

@item docstring
A documentation string, ignored.

@item base-type
A symbol denoting a foreign type.

@item keyword
A keyword symbol.

@item value
An index value for a keyword.
@end table

@subheading Description
The @code{defcenum} macro is used to define foreign types that map
keyword symbols to integer values, similar to the C @code{enum} type.

If @var{value} is omitted its value will either be 0, if it's the
first entry, or it it will continue the progression from the last
specified value.

Keywords will be automatically converted to values and vice-versa when
being passed as arguments to or returned from foreign functions,
respectively. The same applies to any other situations where an object
of an @code{enum} type is expected.

Types defined with @code{defcenum} canonicalize to @var{base-type}
which is @code{:int} by default.

@subheading Examples
@lisp
(defcenum boolean
  :no
  :yes)

CFFI> (foreign-enum-value 'boolean :no)
@result{} 0
@end lisp

@lisp
(defcenum numbers
  (:one 1)
  :two
  (:four 4))

CFFI> (foreign-enum-keyword 'numbers 2)
@result{} :TWO
@end lisp

@subheading See Also
@seealso{foreign-enum-value} @*
@seealso{foreign-enum-keyword}


@c ===================================================================
@c DEFINE-FOREIGN-TYPE

@page
@node define-foreign-type, define-parse-method, defcenum, Foreign Types
@heading define-foreign-type
@subheading Syntax
@Macro{define-foreign-type class-name supers slots &rest options @res{} class-name}

options ::= (@code{:actual-type} @var{type}) | @
    (@code{:simple-parser} @var{symbol}) | @
    @emph{regular defclass option}

@subheading Arguments and Values

@table @var
@item class-name
A symbol naming the new foreign type class.

@item supers
A list of symbols naming the super classes.

@item slots
A list of slot definitions, passed to @code{defclass}.
@end table

@subheading Description

@c TODO rewrite

The macro @code{define-foreign-type} defines a new class
@var{class-name}. It is a thin wrapper around @code{defclass}. Among
other things, it ensures that @var{class-name} becomes a subclass of
@var{foreign-type}, what you need to know about that is that there's
an initarg @code{:actual-type} which serves the same purpose as
@code{defctype}'s @var{base-type} argument.

@c TODO mention the type translators here
@c FIX FIX

@subheading Examples
Taken from @cffi{}'s @code{:boolean} type definition:

@lisp
(define-foreign-type :boolean (&optional (base-type :int))
  "Boolean type. Maps to an :int by default. Only accepts integer types."
  (ecase base-type
    ((:char
      :unsigned-char
      :int
      :unsigned-int
      :long
      :unsigned-long) base-type)))

CFFI> (canonicalize-foreign-type :boolean)
@result{} :INT
CFFI> (canonicalize-foreign-type '(:boolean :long))
@result{} :LONG
CFFI> (canonicalize-foreign-type '(:boolean :float))
;; @lispcmt{@error{} signalled by ECASE.}
@end lisp

@subheading See Also
@seealso{defctype} @*
@seealso{define-parse-method}


@c ===================================================================
@c DEFINE-PARSE-METHOD

@page
@node define-parse-method, foreign-bitfield-symbols, define-foreign-type, Foreign Types
@heading define-parse-method
@subheading Syntax
@Macro{define-parse-method name lambda-list &body body @res{} name}

@subheading Arguments and Values

@table @var
@item type-name
A symbol naming the new foreign type.

@item lambda-list
A lambda list which is the argument list of the new foreign type.

@item body
One or more forms that provide a definition of the new foreign type.
@end table

@subheading Description


@c TODO: update example. The boolean type is probably a good choice.

@subheading Examples
Taken from @cffi{}'s @code{:boolean} type definition:

@lisp
(define-foreign-type :boolean (&optional (base-type :int))
  "Boolean type. Maps to an :int by default. Only accepts integer types."
  (ecase base-type
    ((:char
      :unsigned-char
      :int
      :unsigned-int
      :long
      :unsigned-long) base-type)))

CFFI> (canonicalize-foreign-type :boolean)
@result{} :INT
CFFI> (canonicalize-foreign-type '(:boolean :long))
@result{} :LONG
CFFI> (canonicalize-foreign-type '(:boolean :float))
;; @lispcmt{@error{} signalled by ECASE.}
@end lisp

@subheading See Also
@seealso{define-foreign-type}


@c ===================================================================
@c EXPLAIN-FOREIGN-SLOT-VALUE

@c @node explain-foreign-slot-value
@c @heading explain-foreign-slot-value
@c @subheading Syntax
@c @Macro{explain-foreign-slot-value ptr type &rest slot-names}

@c @subheading Arguments and Values

@c @table @var
@c @item ptr
@c ...

@c @item type
@c ...

@c @item slot-names
@c ...
@c @end table

@c @subheading Description
@c This macro translates the slot access that would occur by calling
@c @code{foreign-slot-value} with the same arguments into an equivalent
@c expression in C and prints it to @code{*standard-output*}.

@c @emph{Note: this is not implemented yet.}

@c @subheading Examples
@c @lisp
@c CFFI> (explain-foreign-slot-value ptr 'timeval 'tv-secs)
@c @result{} ptr->tv_secs

@c CFFI> (explain-foreign-slot-value emp 'employee 'hire-date 'tv-usecs)
@c @result{} emp->hire_date.tv_usecs
@c @end lisp

@c @subheading See Also


@c ===================================================================
@c FOREIGN-BITFIELD-SYMBOLS

@page
@node foreign-bitfield-symbols, foreign-bitfield-value, define-parse-method, Foreign Types
@heading foreign-bitfield-symbols
@subheading Syntax
@Function{foreign-bitfield-symbols type value @res{} symbols}

@subheading Arguments and Values

@table @var
@item type
A bitfield type.

@item value
An integer.

@item symbols
A potentially shared list of symbols.
@code{nil}.
@end table

@subheading Description
The function @code{foreign-bitfield-symbols} returns a possibly shared
list of symbols that correspond to @var{value} in @var{type}.

@subheading Examples
@lisp
(defbitfield flags
  (flag-a 1)
  (flag-b 2)
  (flag-c 4))

CFFI> (foreign-bitfield-symbols 'flags #b101)
@result{} (FLAG-A FLAG-C)
@end lisp

@subheading See Also
@seealso{defbitfield} @*
@seealso{foreign-bitfield-value}


@c ===================================================================
@c FOREIGN-BITFIELD-VALUE

@page
@node foreign-bitfield-value, foreign-enum-keyword, foreign-bitfield-symbols, Foreign Types
@heading foreign-bitfield-value
@subheading Syntax
@Function{foreign-bitfield-value type symbols @res{} value}

@subheading Arguments and Values

@table @var
@item type
A @code{bitfield} type.

@item symbol
A Lisp symbol.

@item value
An integer.
@end table

@subheading Description
The function @code{foreign-bitfield-value} returns the @var{value} that
corresponds to the symbols in the @var{symbols} list.

@subheading Examples
@lisp
(defbitfield flags
  (flag-a 1)
  (flag-b 2)
  (flag-c 4))

CFFI> (foreign-bitfield-value 'flags '(flag-a flag-c))
@result{} 5  ; #b101
@end lisp

@subheading See Also
@seealso{defbitfield} @*
@seealso{foreign-bitfield-symbols}


@c ===================================================================
@c FOREIGN-ENUM-KEYWORD

@page
@node foreign-enum-keyword, foreign-enum-value, foreign-bitfield-value, Foreign Types
@heading foreign-enum-keyword
@subheading Syntax
@Function{foreign-enum-keyword type value &key errorp @res{} keyword}

@subheading Arguments and Values

@table @var
@item type
An @code{enum} type.

@item value
An integer.

@item errorp
If true (the default), signal an error if @var{value} is not defined
in @var{type}.  If false, @code{foreign-enum-keyword} returns
@code{nil}.

@item keyword
A keyword symbol.
@end table

@subheading Description
The function @code{foreign-enum-keyword} returns the keyword symbol
that corresponds to @var{value} in @var{type}.

An error is signaled if @var{type} doesn't contain such @var{value}
and @var{errorp} is true.

@subheading Examples
@lisp
(defcenum boolean
  :no
  :yes)

CFFI> (foreign-enum-keyword 'boolean 1)
@result{} :YES
@end lisp

@subheading See Also
@seealso{defcenum} @*
@seealso{foreign-enum-value}


@c ===================================================================
@c FOREIGN-ENUM-VALUE

@page
@node foreign-enum-value, foreign-slot-names, foreign-enum-keyword, Foreign Types
@heading foreign-enum-value
@subheading Syntax
@Function{foreign-enum-value type keyword &key errorp @res{} value}

@subheading Arguments and Values

@table @var
@item type
An @code{enum} type.

@item keyword
A keyword symbol.

@item errorp
If true (the default), signal an error if @var{keyword} is not
defined in @var{type}.  If false, @code{foreign-enum-value} returns
@code{nil}.

@item value
An integer.
@end table

@subheading Description
The function @code{foreign-enum-value} returns the @var{value} that
corresponds to @var{keyword} in @var{type}.

An error is signaled if @var{type} doesn't contain such
@var{keyword}, and @var{errorp} is true.

@subheading Examples
@lisp
(defcenum boolean
  :no
  :yes)

CFFI> (foreign-enum-value 'boolean :yes)
@result{} 1
@end lisp

@subheading See Also
@seealso{defcenum} @*
@seealso{foreign-enum-keyword}


@c ===================================================================
@c FOREIGN-SLOT-NAMES

@page
@node foreign-slot-names, foreign-slot-offset, foreign-enum-value, Foreign Types
@heading foreign-slot-names
@subheading Syntax
@Function{foreign-slot-names type @res{} names}

@subheading Arguments and Values

@table @var
@item type
A foreign struct type.

@item names
A list.
@end table

@subheading Description
The function @code{foreign-slot-names} returns a potentially shared
list of slot @var{names} for the given structure @var{type}. This list
has no particular order.

@subheading Examples
@lisp
(defcstruct timeval
  (tv-secs :long)
  (tv-usecs :long))

CFFI> (foreign-slot-names '(:struct timeval))
@result{} (TV-SECS TV-USECS)
@end lisp

@subheading See Also
@seealso{defcstruct} @*
@seealso{foreign-slot-offset} @*
@seealso{foreign-slot-value} @*
@seealso{foreign-slot-pointer}


@c ===================================================================
@c FOREIGN-SLOT-OFFSET

@page
@node foreign-slot-offset, foreign-slot-pointer, foreign-slot-names, Foreign Types
@heading foreign-slot-offset
@subheading Syntax
@Function{foreign-slot-offset type slot-name @res{} offset}

@subheading Arguments and Values

@table @var
@item type
A foreign struct type.

@item slot-name
A symbol.

@item offset
An integer.
@end table

@subheading Description
The function @code{foreign-slot-offset} returns the @var{offset} in
bytes of a slot in a foreign struct type.

@subheading Examples
@lisp
(defcstruct timeval
  (tv-secs :long)
  (tv-usecs :long))

CFFI> (foreign-slot-offset '(:struct timeval) 'tv-secs)
@result{} 0
CFFI> (foreign-slot-offset '(:struct timeval) 'tv-usecs)
@result{} 4
@end lisp

@subheading See Also
@seealso{defcstruct} @*
@seealso{foreign-slot-names} @*
@seealso{foreign-slot-pointer} @*
@seealso{foreign-slot-value}


@c ===================================================================
@c FOREIGN-SLOT-POINTER

@page
@node foreign-slot-pointer, foreign-slot-value, foreign-slot-offset, Foreign Types
@heading foreign-slot-pointer
@subheading Syntax
@Function{foreign-slot-pointer ptr type slot-name @res{} pointer}

@subheading Arguments and Values

@table @var
@item ptr
A pointer to a structure.

@item type
A foreign structure type.

@item slot-names
A slot name in the @var{type}.

@item pointer
A pointer to the slot @var{slot-name}.
@end table

@subheading Description
Returns a pointer to the location of the slot @var{slot-name} in a
foreign object of type @var{type} at @var{ptr}. The returned pointer
points inside the structure. Both the pointer and the memory it points
to have the same extent as @var{ptr}.

For aggregate slots, this is the same value returned by
@code{foreign-slot-value}.

@subheading Examples
@lisp
(defcstruct point
  "Pointer structure."
  (x :int)
  (y :int))

CFFI> (with-foreign-object (ptr '(:struct point))
        (foreign-slot-pointer ptr '(:struct point) 'x))
@result{} #<FOREIGN-ADDRESS #xBFFF6E60>
;; @lispcmt{Note: the exact pointer representation varies from lisp to lisp.}
@end lisp

@subheading See Also
@seealso{defcstruct} @*
@seealso{foreign-slot-value} @*
@seealso{foreign-slot-names} @*
@seealso{foreign-slot-offset}


@c ===================================================================
@c FOREIGN-SLOT-VALUE

@page
@node foreign-slot-value, foreign-type-alignment, foreign-slot-pointer, Foreign Types
@heading foreign-slot-value
@subheading Syntax
@Accessor{foreign-slot-value ptr type slot-name @res{} object}

@subheading Arguments and Values

@table @var
@item ptr
A pointer to a structure.

@item type
A foreign structure type.

@item slot-name
A symbol naming a slot in the structure type.

@item object
The object contained in the slot specified by @var{slot-name}.
@end table

@subheading Description
For simple slots, @code{foreign-slot-value} returns the value of the
object, such as a Lisp integer or pointer. In C, this would be
expressed as @code{ptr->slot}.

For aggregate slots, a pointer inside the structure to the beginning
of the slot's data is returned. In C, this would be expressed as
@code{&ptr->slot}. This pointer and the memory it points to have the
same extent as @var{ptr}.

There are compiler macros for @code{foreign-slot-value} and its
@code{setf} expansion that open code the memory access when
@var{type} and @var{slot-names} are constant at compile-time.

@subheading Examples
@lisp
(defcstruct point
  "Pointer structure."
  (x :int)
  (y :int))

CFFI> (with-foreign-object (ptr '(:struct point))
        ;; @lispcmt{Initialize the slots}
        (setf (foreign-slot-value ptr '(:struct point) 'x) 42
              (foreign-slot-value ptr '(:struct point) 'y) 42)
        ;; @lispcmt{Return a list with the coordinates}
        (with-foreign-slots ((x y) ptr (:struct point))
          (list x y)))
@result{} (42 42)
@end lisp

@subheading See Also
@seealso{defcstruct} @*
@seealso{foreign-slot-names} @*
@seealso{foreign-slot-offset} @*
@seealso{foreign-slot-pointer} @*
@seealso{with-foreign-slots}


@c ===================================================================
@c FOREIGN-TYPE-ALIGNMENT

@page
@node foreign-type-alignment, foreign-type-size, foreign-slot-value, Foreign Types
@heading foreign-type-alignment
@subheading Syntax
@c XXX: This is actually a generic function.
@Function{foreign-type-alignment type @res{} alignment}

@subheading Arguments and Values

@table @var
@item type
A foreign type.

@item alignment
An integer.
@end table

@subheading Description
The function @code{foreign-type-alignment} returns the
@var{alignment} of @var{type} in bytes.

@subheading Examples
@lisp
CFFI> (foreign-type-alignment :char)
@result{} 1
CFFI> (foreign-type-alignment :short)
@result{} 2
CFFI> (foreign-type-alignment :int)
@result{} 4
@end lisp

@lisp
(defcstruct foo
  (a :char))

CFFI> (foreign-type-alignment '(:struct foo))
@result{} 1
@end lisp

@subheading See Also
@seealso{foreign-type-size}


@c ===================================================================
@c FOREIGN-TYPE-SIZE

@page
@node foreign-type-size, free-converted-object, foreign-type-alignment, Foreign Types
@heading foreign-type-size
@subheading Syntax
@c XXX: this is actually a generic function.
@Function{foreign-type-size type @res{} size}

@subheading Arguments and Values

@table @var
@item type
A foreign type.

@item size
An integer.
@end table

@subheading Description
The function @code{foreign-type-size} return the @var{size} of
@var{type} in bytes.  This includes any padding within and following
the in-memory representation as needed to create an array of
@var{type} objects.

@subheading Examples
@lisp
(defcstruct foo
  (a :double)
  (c :char))

CFFI> (foreign-type-size :double)
@result{} 8
CFFI> (foreign-type-size :char)
@result{} 1
CFFI> (foreign-type-size '(:struct foo))
@result{} 16
@end lisp

@subheading See Also
@seealso{foreign-type-alignment}


@c ===================================================================
@c FREE-CONVERTED-OBJECT

@page
@node free-converted-object, free-translated-object, foreign-type-size, Foreign Types
@heading free-converted-object
@subheading Syntax
@Function{free-converted-object foreign-value type params}

@subheading Arguments and Values

@table @var
@item foreign-value
The C object to be freed.

@item type
A @cffi{} type specifier.

@item params
The state returned as the second value from @code{convert-to-foreign};
used to implement the third argument to @code{free-translated-object}.
@end table

@subheading Description

The return value is unspecified.

This is an external interface to the type translation facility.  In
the implementation, all foreign functions are ultimately defined as
type translation wrappers around primitive foreign function
invocations.

This function is available mostly for inspection of the type
translation process, and possibly optimization of special cases of
your foreign function calls.

Its behavior is better described under @code{free-translated-object}'s
documentation.

@subheading Examples

@lisp
CFFI-USER> (convert-to-foreign "a boat" :string)
@result{} #<FOREIGN-ADDRESS #x097ACDC0>
@result{} T
CFFI-USER> (free-converted-object * :string t)
@result{} NIL
@end lisp

@subheading See Also
@seealso{convert-from-foreign} @*
@seealso{convert-to-foreign} @*
@seealso{free-translated-object}


@c ===================================================================
@c FREE-TRANSLATED-OBJECT

@c TODO: update

@page
@node free-translated-object, translate-from-foreign, free-converted-object, Foreign Types
@heading free-translated-object
@subheading Syntax
@GenericFunction{free-translated-object value type-name param}

@subheading Arguments and Values

@table @var
@item pointer
The foreign value returned by @code{translate-to-foreign}.

@item type-name
A symbol naming a foreign type defined by @code{defctype}.

@item param
The second value, if any, returned by @code{translate-to-foreign}.
@end table

@subheading Description
This generic function may be specialized by user code to perform
automatic deallocation of foreign objects as they are passed to C
functions.

Any methods defined on this generic function must EQL-specialize the
@var{type-name} parameter on a symbol defined as a foreign type by
the @code{defctype} macro.

@subheading See Also
@seealso{Foreign Type Translators} @*
@seealso{translate-to-foreign}


@c ===================================================================
@c TRANSLATE-FROM-FOREIGN

@c TODO: update

@page
@node translate-from-foreign, translate-to-foreign, free-translated-object, Foreign Types
@heading translate-from-foreign
@subheading Syntax
@GenericFunction{translate-from-foreign foreign-value type-name @
                                        @res{} lisp-value}

@subheading Arguments and Values

@table @var
@item foreign-value
The foreign value to convert to a Lisp object.

@item type-name
A symbol naming a foreign type defined by @code{defctype}.

@item lisp-value
The lisp value to pass in place of @code{foreign-value} to Lisp code.
@end table

@subheading Description
This generic function is invoked by @cffi{} to convert a foreign value to
a Lisp value, such as when returning from a foreign function, passing
arguments to a callback function, or accessing a foreign variable.

To extend the @cffi{} type system by performing custom translations, this
method may be specialized by @sc{eql}-specializing @code{type-name} on a
symbol naming a foreign type defined with @code{defctype}.  This
method should return the appropriate Lisp value to use in place of the
foreign value.

The results are undefined if the @code{type-name} parameter is
specialized in any way except an @sc{eql} specializer on a foreign type
defined with @code{defctype}.  Specifically, translations may not be
defined for built-in types.

@subheading See Also
@seealso{Foreign Type Translators} @*
@seealso{translate-to-foreign} @*
@seealso{free-translated-object}


@c ===================================================================
@c TRANSLATE-TO-FOREIGN

@c TODO: update

@page
@node translate-to-foreign, translate-into-foreign-memory, translate-from-foreign, Foreign Types
@heading translate-to-foreign
@subheading Syntax
@GenericFunction{translate-to-foreign lisp-value type-name @
                                      @res{} foreign-value, alloc-param}

@subheading Arguments and Values

@table @var
@item lisp-value
The Lisp value to convert to foreign representation.

@item type-name
A symbol naming a foreign type defined by @code{defctype}.

@item foreign-value
The foreign value to pass in place of @code{lisp-value} to foreign code.

@item alloc-param
If present, this value will be passed to
@code{free-translated-object}.
@end table

@subheading Description
This generic function is invoked by @cffi{} to convert a Lisp value to a
foreign value, such as when passing arguments to a foreign function,
returning a value from a callback, or setting a foreign variable.  A
``foreign value'' is one appropriate for passing to the next-lowest
translator, including the low-level translators that are ultimately
invoked invisibly with @cffi{}.

To extend the @cffi{} type system by performing custom translations, this
method may be specialized by @sc{eql}-specializing @code{type-name} on a
symbol naming a foreign type defined with @code{defctype}.  This
method should return the appropriate foreign value to use in place of
the Lisp value.

In cases where @cffi{} can determine the lifetime of the foreign object
returned by this method, it will invoke @code{free-translated-object}
on the foreign object at the appropriate time.  If
@code{translate-to-foreign} returns a second value, it will be passed
as the @code{param} argument to @code{free-translated-object}.  This
can be used to establish communication between the allocation and
deallocation methods.

The results are undefined if the @code{type-name} parameter is
specialized in any way except an @sc{eql} specializer on a foreign type
defined with @code{defctype}.  Specifically, translations may not be
defined for built-in types.

@subheading See Also
@seealso{Foreign Type Translators} @*
@seealso{translate-from-foreign} @*
@seealso{free-translated-object}


@c ===================================================================
@c TRANSLATE-INTO-FOREIGN-MEMORY

@page
@node translate-into-foreign-memory, with-foreign-slots, translate-to-foreign, Foreign Types
@heading translate-into-foreign-memory
@subheading Syntax
@GenericFunction{translate-into-foreign-memory lisp-value type-name pointer}

@subheading Arguments and Values

@table @var
@item lisp-value
The Lisp value to convert to foreign representation.

@item type-name
A symbol or list @code{(:struct @var{structure-name})} naming a foreign type defined by @code{defctype}.

@item pointer
The foreign pointer where the translated object should be stored.
@end table

@subheading Description
Translate the Lisp value into the foreign memory location given by
pointer.  The return value is not used.

@c ===================================================================
@c WITH-FOREIGN-SLOTS

@page
@node with-foreign-slots,  , translate-into-foreign-memory, Foreign Types
@heading with-foreign-slots
@subheading Syntax
@Macro{with-foreign-slots (vars ptr type) &body body}

@subheading Arguments and Values

@table @var
@item vars
A list with each element a symbol, or list of length two with the
first element @code{:pointer} and the second a symbol.

@item ptr
A foreign pointer to a structure.

@item type
A structure type.

@item body
A list of forms to be executed.
@end table

@subheading Description
The @code{with-foreign-slots} macro creates local symbol macros for each
var in @var{vars} to reference foreign slots in @var{ptr} of @var{type}.
If the var is a list starting with @code{:pointer}, it will bind the
pointer to the slot (rather than the value). It is similar to Common
Lisp's @code{with-slots} macro.

@subheading Examples
@lisp
(defcstruct tm
  (sec :int)
  (min :int)
  (hour :int)
  (mday :int)
  (mon  :int)
  (year :int)
  (wday :int)
  (yday :int)
  (isdst  :boolean)
  (zone   :string)
  (gmtoff :long))

CFFI> (with-foreign-object (time :int)
        (setf (mem-ref time :int)
              (foreign-funcall "time" :pointer (null-pointer) :int))
        (foreign-funcall "gmtime" :pointer time (:pointer (:struct tm))))
@result{} #<A Mac Pointer #x102A30>
CFFI> (with-foreign-slots ((sec min hour mday mon year) * (:struct tm))
        (format nil "~A:~A:~A, ~A/~A/~A"
                hour min sec (+ 1900 year) mon mday))
@result{} "7:22:47, 2005/8/2"
@end lisp

@subheading See Also
@seealso{defcstruct} @*
@seealso{defcunion} @*
@seealso{foreign-slot-value}


@c ===================================================================
@c CHAPTER: Pointers

@node Pointers, Strings, Foreign Types, Top
@chapter Pointers

All C data in @cffi{} is referenced through pointers.  This includes
defined C variables that hold immediate values, and integers.

To see why this is, consider the case of the C integer.  It is not
only an arbitrary representation for an integer, congruent to Lisp's
fixnums; the C integer has a specific bit pattern in memory defined by
the C @acronym{ABI}.  Lisp has no such constraint on its fixnums;
therefore, it only makes sense to think of fixnums as C integers if
you assume that @cffi{} converts them when necessary, such as when
storing one for use in a C function call, or as the value of a C
variable.  This requires defining an area of memory@footnote{The
definition of @dfn{memory} includes the @acronym{CPU} registers.},
represented through an effective address, and storing it there.

Due to this compartmentalization, it only makes sense to manipulate
raw C data in Lisp through pointers to it.  For example, while there
may be a Lisp representation of a @code{struct} that is converted to C
at store time, you may only manipulate its raw data through a pointer.
The C compiler does this also, albeit informally.

@menu
* Basic Pointer Operations::
* Allocating Foreign Memory::
* Accessing Foreign Memory::

Dictionary

* foreign-free::
* foreign-alloc::
* foreign-symbol-pointer::
* inc-pointer::
* incf-pointer::
* make-pointer::
* mem-aptr::
* mem-aref::
* mem-ref::
* null-pointer::
* null-pointer-p::
* pointerp::
* pointer-address::
* pointer-eq::
* with-foreign-object::
* with-foreign-objects::
* with-foreign-pointer::
@end menu

@node Basic Pointer Operations, Allocating Foreign Memory, Pointers, Pointers
@section Basic Pointer Operations

Manipulating pointers proper can be accomplished through most of the
other operations defined in the Pointers dictionary, such as
@code{make-pointer}, @code{pointer-address}, and @code{pointer-eq}.
When using them, keep in mind that they merely manipulate the Lisp
representation of pointers, not the values they point to.

@deftp {Lisp Type} foreign-pointer
The pointers' representations differ from implementation to
implementation and have different types.  @code{foreign-pointer}
provides a portable type alias to each of these types.
@end deftp


@node Allocating Foreign Memory, Accessing Foreign Memory, Basic Pointer Operations, Pointers
@section Allocating Foreign Memory

@cffi{} provides support for stack and heap C memory allocation.
Stack allocation, done with @code{with-foreign-object}, is sometimes
called ``dynamic'' allocation in Lisp, because memory allocated as
such has dynamic extent, much as with @code{let} bindings of special
variables.

This should not be confused with what C calls ``dynamic'' allocation,
or that done with @code{malloc} and friends.  This sort of heap
allocation is done with @code{foreign-alloc}, creating objects that
exist until freed with @code{foreign-free}.


@node Accessing Foreign Memory, foreign-free, Allocating Foreign Memory, Pointers
@section Accessing Foreign Memory

When manipulating raw C data, consider that all pointers are pointing
to an array.  When you only want one C value, such as a single
@code{struct}, this array only has one such value.  It is worthwhile
to remember that everything is an array, though, because this is also
the semantic that C imposes natively.

C values are accessed as the @code{setf}-able places defined by
@code{mem-aref} and @code{mem-ref}.  Given a pointer and a @cffi{}
type (@pxref{Foreign Types}), either of these will dereference the
pointer, translate the C data there back to Lisp, and return the
result of said translation, performing the reverse operation when
@code{setf}-ing.  To decide which one to use, consider whether you
would use the array index operator @code{[@var{n}]} or the pointer
dereference @code{*} in C; use @code{mem-aref} for array indexing and
@code{mem-ref} for pointer dereferencing.


@c ===================================================================
@c FOREIGN-FREE

@page
@node foreign-free, foreign-alloc, Accessing Foreign Memory, Pointers
@heading foreign-free
@subheading Syntax
@Function{foreign-free ptr @res{} undefined}

@subheading Arguments and Values

@table @var
@item ptr
A foreign pointer.
@end table

@subheading Description
The @code{foreign-free} function frees a @code{ptr} previously
allocated by @code{foreign-alloc}. The consequences of freeing a given
pointer twice are undefined.

@subheading Examples

@lisp
CFFI> (foreign-alloc :int)
@result{} #<A Mac Pointer #x1022E0>
CFFI> (foreign-free *)
@result{} NIL
@end lisp

@subheading See Also
@seealso{foreign-alloc} @*
@seealso{with-foreign-pointer}


@c ===================================================================
@c FOREIGN-ALLOC

@page
@node foreign-alloc, foreign-symbol-pointer, foreign-free, Pointers
@heading foreign-alloc
@subheading Syntax
@Function{foreign-alloc type &key initial-element initial-contents (count 1) @
                        null-terminated-p @res{} pointer}

@subheading Arguments and Values

@table @var
@item type
A foreign type.

@item initial-element
A Lisp object.

@item initial-contents
A sequence.

@item count
An integer. Defaults to 1 or the length of @var{initial-contents} if
supplied.

@item null-terminated-p
A boolean, false by default.

@item pointer
A foreign pointer to the newly allocated memory.
@end table

@subheading Description
The @code{foreign-alloc} function allocates enough memory to hold
@var{count} objects of type @var{type} and returns a
@var{pointer}. This memory must be explicitly freed using
@code{foreign-free} once it is no longer needed.

If @var{initial-element} is supplied, it is used to initialize the
@var{count} objects the newly allocated memory holds.

If an @var{initial-contents} sequence is supplied, it must have a
length less than or equal to @var{count} and each of its elements
will be used to initialize the contents of the newly allocated
memory.

If @var{count} is omitted and @var{initial-contents} is specified, it
will default to @code{(length @var{initial-contents})}.

@var{initial-element} and @var{initial-contents} are mutually
exclusive.

When @var{null-terminated-p} is true,
@code{(1+ (max @var{count} (length @var{initial-contents})))} elements
are allocated and the last one is set to @code{NULL}. Note that in
this case @var{type} must be a pointer type (ie. a type that
canonicalizes to @code{:pointer}), otherwise an error is signaled.

@subheading Examples
@lisp
CFFI> (foreign-alloc :char)
@result{} #<A Mac Pointer #x102D80>     ; @lispcmt{A pointer to 1 byte of memory.}

CFFI> (foreign-alloc :char :count 20)
@result{} #<A Mac Pointer #x1024A0>     ; @lispcmt{A pointer to 20 bytes of memory.}

CFFI> (foreign-alloc :int :initial-element 12)
@result{} #<A Mac Pointer #x1028B0>
CFFI> (mem-ref * :int)
@result{} 12

CFFI> (foreign-alloc :int :initial-contents '(1 2 3))
@result{} #<A Mac Pointer #x102950>
CFFI> (loop for i from 0 below 3
            collect (mem-aref * :int i))
@result{} (1 2 3)

CFFI> (foreign-alloc :int :initial-contents #(1 2 3))
@result{} #<A Mac Pointer #x102960>
CFFI> (loop for i from 0 below 3
            collect (mem-aref * :int i))
@result{} (1 2 3)

;;; @lispcmt{Allocate a char** pointer that points to newly allocated memory}
;;; @lispcmt{by the :string type translator for the string "foo".}
CFFI> (foreign-alloc :string :initial-element "foo")
@result{} #<A Mac Pointer #x102C40>
@end lisp

@lisp
;;; @lispcmt{Allocate a null-terminated array of strings.}
;;; @lispcmt{(Note: FOREIGN-STRING-TO-LISP returns NIL when passed a null pointer)}
CFFI> (foreign-alloc :string
                     :initial-contents '("foo" "bar" "baz")
                     :null-terminated-p t)
@result{} #<A Mac Pointer #x102D20>
CFFI> (loop for i from 0 below 4
            collect (mem-aref * :string i))
@result{} ("foo" "bar" "baz" NIL)
CFFI> (progn
        (dotimes (i 3)
          (foreign-free (mem-aref ** :pointer i)))
        (foreign-free **))
@result{} nil
@end lisp

@subheading See Also
@seealso{foreign-free} @*
@seealso{with-foreign-object} @*
@seealso{with-foreign-pointer}


@c ===================================================================
@c FOREIGN-SYMBOL-POINTER

@page
@node foreign-symbol-pointer, inc-pointer, foreign-alloc, Pointers
@heading foreign-symbol-pointer
@subheading Syntax
@Function{foreign-symbol-pointer foreign-name &key library @res{} pointer}

@subheading Arguments and Values

@table @var
@item foreign-name
A string.

@item pointer
A foreign pointer, or @code{nil}.

@item library
A Lisp symbol or an instance of @code{foreign-library}.
@end table

@subheading Description
The function @code{foreign-symbol-pointer} will return a foreign
pointer corresponding to the foreign symbol denoted by the string
@var{foreign-name}.  If a foreign symbol named @var{foreign-name}
doesn't exist, @code{nil} is returned.

ABI name manglings will be performed on @var{foreign-name} by
@code{foreign-symbol-pointer} if necessary. (eg: adding a leading
underscore on darwin/ppc)

@var{library} should name a foreign library as defined by
@code{define-foreign-library}, @code{:default} (which is the default)
or an instance of @code{foreign-library} as returned by
@code{load-foreign-library}.

@strong{Important note:} do not keep these pointers across saved Lisp
cores as the foreign-library may move across sessions.

@subheading Examples

@lisp
CFFI> (foreign-symbol-pointer "errno")
@result{} #<A Mac Pointer #xA0008130>
CFFI> (foreign-symbol-pointer "strerror")
@result{} #<A Mac Pointer #x9002D0F8>
CFFI> (foreign-funcall-pointer * () :int (mem-ref ** :int) :string)
@result{} "No such file or directory"

CFFI> (foreign-symbol-pointer "inexistent symbol")
@result{} NIL
@end lisp

@subheading See Also
@seealso{defcvar}


@c ===================================================================
@c INC-POINTER

@page
@node inc-pointer, incf-pointer, foreign-symbol-pointer, Pointers
@heading inc-pointer
@subheading Syntax
@Function{inc-pointer pointer offset @res{} new-pointer}

@subheading Arguments and Values

@table @var
@item pointer
@itemx new-pointer
A foreign pointer.

@item offset
An integer.
@end table

@subheading Description
The function @code{inc-pointer} will return a @var{new-pointer} pointing
@var{offset} bytes past @var{pointer}.

@subheading Examples

@lisp
CFFI> (foreign-string-alloc "Common Lisp")
@result{} #<A Mac Pointer #x102EA0>
CFFI> (inc-pointer * 7)
@result{} #<A Mac Pointer #x102EA7>
CFFI> (foreign-string-to-lisp *)
@result{} "Lisp"
@end lisp

@subheading See Also
@seealso{incf-pointer} @*
@seealso{make-pointer} @*
@seealso{pointerp} @*
@seealso{null-pointer} @*
@seealso{null-pointer-p}


@c ===================================================================
@c INCF-POINTER

@page
@node incf-pointer, make-pointer, inc-pointer, Pointers
@heading incf-pointer
@subheading Syntax
@Macro{incf-pointer place &optional (offset 1) @res{} new-pointer}

@subheading Arguments and Values

@table @var
@item place
A @code{setf} place.

@item new-pointer
A foreign pointer.

@item offset
An integer.
@end table

@subheading Description
The @code{incf-pointer} macro takes the foreign pointer from
@var{place} and creates a @var{new-pointer} incremented by
@var{offset} bytes and which is stored in @var{place}.

@subheading Examples

@lisp
CFFI> (defparameter *two-words* (foreign-string-alloc "Common Lisp"))
@result{} *TWO-WORDS*
CFFI> (defparameter *one-word* *two-words*)
@result{} *ONE-WORD*
CFFI> (incf-pointer *one-word* 7)
@result{} #.(SB-SYS:INT-SAP #X00600457)
CFFI> (foreign-string-to-lisp *one-word*)
@result{} "Lisp"
CFFI> (foreign-string-to-lisp *two-words*)
@result{} "Common Lisp"
@end lisp

@subheading See Also
@seealso{inc-pointer} @*
@seealso{make-pointer} @*
@seealso{pointerp} @*
@seealso{null-pointer} @*
@seealso{null-pointer-p}


@c ===================================================================
@c MAKE-POINTER

@page
@node make-pointer, mem-aptr, incf-pointer, Pointers
@heading make-pointer
@subheading Syntax
@Function{make-pointer address @res{} ptr}

@subheading Arguments and Values

@table @var
@item address
An integer.

@item ptr
A foreign pointer.
@end table

@subheading Description
The function @code{make-pointer} will return a foreign pointer
pointing to @var{address}.

@subheading Examples

@lisp
CFFI> (make-pointer 42)
@result{} #<FOREIGN-ADDRESS #x0000002A>
CFFI> (pointerp *)
@result{} T
CFFI> (pointer-address **)
@result{} 42
CFFI> (inc-pointer *** -42)
@result{} #<FOREIGN-ADDRESS #x00000000>
CFFI> (null-pointer-p *)
@result{} T
CFFI> (typep ** 'foreign-pointer)
@result{} T
@end lisp

@subheading See Also
@seealso{inc-pointer} @*
@seealso{null-pointer} @*
@seealso{null-pointer-p} @*
@seealso{pointerp} @*
@seealso{pointer-address} @*
@seealso{pointer-eq} @*
@seealso{mem-ref}


@c ===================================================================
@c MEM-APTR

@page
@node mem-aptr, mem-aref, make-pointer, Pointers
@heading mem-aptr
@subheading Syntax
@Accessor{mem-aptr ptr type &optional (index 0)}

@subheading Arguments and Values

@table @var
@item ptr
A foreign pointer.

@item type
A foreign type.

@item index
An integer.

@item new-value
A Lisp value compatible with @var{type}.
@end table

@subheading Description
The @code{mem-aptr} function finds the pointer to an element of the array.

@lisp
(mem-aptr ptr type n)

;; @lispcmt{is identical to:}

(inc-pointer ptr (* n (foreign-type-size type)))
@end lisp

@subheading Examples

@lisp
CFFI> (with-foreign-string (str "Hello, foreign world!")
        (mem-aptr str :char 6))
@result{} #.(SB-SYS:INT-SAP #X0063D4B6)
@end lisp

@c ===================================================================
@c MEM-AREF

@page
@node mem-aref, mem-ref, mem-aptr, Pointers
@heading mem-aref
@subheading Syntax
@Accessor{mem-aref ptr type &optional (index 0)}

(setf (@strong{mem-aref} @emph{ptr type &optional (index 0)) new-value})

@subheading Arguments and Values

@table @var
@item ptr
A foreign pointer.

@item type
A foreign type.

@item index
An integer.

@item new-value
A Lisp value compatible with @var{type}.
@end table

@subheading Description
The @code{mem-aref} function is similar to @code{mem-ref} but will
automatically calculate the offset from an @var{index}.

@lisp
(mem-aref ptr type n)

;; @lispcmt{is identical to:}

(mem-ref ptr type (* n (foreign-type-size type)))
@end lisp

@subheading Examples

@lisp
CFFI> (with-foreign-string (str "Hello, foreign world!")
        (mem-aref str :char 6))
@result{} 32
CFFI> (code-char *)
@result{} #\Space

CFFI> (with-foreign-object (array :int 10)
        (loop for i below 10
              do (setf (mem-aref array :int i) (random 100)))
        (loop for i below 10 collect (mem-aref array :int i)))
@result{} (22 7 22 52 69 1 46 93 90 65)
@end lisp

@subheading Compatibility Note

For compatibility with older versions of CFFI, @ref{mem-aref} will
produce a pointer for the deprecated bare structure specification, but
it is consistent with other types for the current specification form
@code{(:struct @var{structure-name})} and provides a Lisp object
translated from the structure (by default a plist).  In order to obtain
the pointer, you should use the new function @ref{mem-aptr}.

@subheading See Also
@seealso{mem-ref} @*
@seealso{mem-aptr}

@c ===================================================================
@c MEM-REF

@page
@node mem-ref, null-pointer, mem-aref, Pointers
@heading mem-ref
@subheading Syntax
@Accessor{mem-ref ptr type &optional offset @res{} object}

@subheading Arguments and Values

@table @var
@item ptr
A pointer.

@item type
A foreign type.

@item offset
An integer (in byte units).

@item object
The value @var{ptr} points to.
@end table

@subheading Description
@subheading Examples

@lisp
CFFI> (with-foreign-string (ptr "Saluton")
        (setf (mem-ref ptr :char 3) (char-code #\a))
        (loop for i from 0 below 8
              collect (code-char (mem-ref ptr :char i))))
@result{} (#\S #\a #\l #\a #\t #\o #\n #\Null)
CFFI> (setq ptr-to-int (foreign-alloc :int))
@result{} #<A Mac Pointer #x1047D0>
CFFI> (mem-ref ptr-to-int :int)
@result{} 1054619
CFFI> (setf (mem-ref ptr-to-int :int) 1984)
@result{} 1984
CFFI> (mem-ref ptr-to-int :int)
@result{} 1984
@end lisp

@subheading See Also
@seealso{mem-aref}


@c ===================================================================
@c NULL-POINTER

@page
@node null-pointer, null-pointer-p, mem-ref, Pointers
@heading null-pointer
@subheading Syntax
@Function{null-pointer @res{} pointer}

@subheading Arguments and Values

@table @var
@item pointer
A @code{NULL} pointer.
@end table

@subheading Description
The function @code{null-pointer} returns a null pointer.

@subheading Examples

@lisp
CFFI> (null-pointer)
@result{} #<A Null Mac Pointer>
CFFI> (pointerp *)
@result{} T
@end lisp

@subheading See Also
@seealso{null-pointer-p} @*
@seealso{make-pointer}


@c ===================================================================
@c NULL-POINTER-P

@page
@node null-pointer-p, pointerp, null-pointer, Pointers
@heading null-pointer-p
@subheading Syntax
@Function{null-pointer-p ptr @res{} boolean}

@subheading Arguments and Values

@table @var
@item ptr
A foreign pointer that may be a null pointer.

@item boolean
@code{T} or @code{NIL}.
@end table

@subheading Description
The function @code{null-pointer-p} returns true if @var{ptr} is a null
pointer and false otherwise.

@subheading Examples

@lisp
CFFI> (null-pointer-p (null-pointer))
@result{} T
@end lisp

@lisp
(defun contains-str-p (big little)
  (not (null-pointer-p
        (foreign-funcall "strstr" :string big :string little :pointer))))

CFFI> (contains-str-p "Popcorns" "corn")
@result{} T
CFFI> (contains-str-p "Popcorns" "salt")
@result{} NIL
@end lisp

@subheading See Also
@seealso{null-pointer} @*
@seealso{pointerp}


@c ===================================================================
@c POINTERP

@page
@node pointerp, pointer-address, null-pointer-p, Pointers
@heading pointerp
@subheading Syntax
@Function{pointerp ptr @res{} boolean}

@subheading Arguments and Values

@table @var
@item ptr
An object that may be a foreign pointer.

@item boolean
@code{T} or @code{NIL}.
@end table

@subheading Description
The function @code{pointerp} returns true if @var{ptr} is a foreign
pointer and false otherwise.

@subheading Implementation-specific Notes
In Allegro CL, foreign pointers are integers thus in this
implementation @code{pointerp} will return true for any ordinary integer.

@subheading Examples

@lisp
CFFI> (foreign-alloc 32)
@result{} #<A Mac Pointer #x102D20>
CFFI> (pointerp *)
@result{} T
CFFI> (pointerp "this is not a pointer")
@result{} NIL
@end lisp

@subheading See Also
@seealso{make-pointer}
@seealso{null-pointer-p}


@c ===================================================================
@c POINTER-ADDRESS

@page
@node pointer-address, pointer-eq, pointerp, Pointers
@heading pointer-address
@subheading Syntax
@Function{pointer-address ptr @res{} address}

@subheading Arguments and Values

@table @var
@item ptr
A foreign pointer.

@item address
An integer.
@end table

@subheading Description
The function @code{pointer-address} will return the @var{address} of
a foreign pointer @var{ptr}.

@subheading Examples

@lisp
CFFI> (pointer-address (null-pointer))
@result{} 0
CFFI> (pointer-address (make-pointer 123))
@result{} 123
@end lisp

@subheading See Also
@seealso{make-pointer} @*
@seealso{inc-pointer} @*
@seealso{null-pointer} @*
@seealso{null-pointer-p} @*
@seealso{pointerp} @*
@seealso{pointer-eq} @*
@seealso{mem-ref}


@c ===================================================================
@c POINTER-EQ

@page
@node pointer-eq, with-foreign-object, pointer-address, Pointers
@heading pointer-eq
@subheading Syntax
@Function{pointer-eq ptr1 ptr2 @res{} boolean}

@subheading Arguments and Values

@table @var
@item ptr1
@itemx ptr2
A foreign pointer.

@item boolean
@code{T} or @code{NIL}.
@end table

@subheading Description
The function @code{pointer-eq} returns true if @var{ptr1} and
@var{ptr2} point to the same memory address and false otherwise.

@subheading Implementation-specific Notes
The representation of foreign pointers varies across the various Lisp
implementations as does the behaviour of the built-in Common Lisp
equality predicates. Comparing two pointers that point to the same
address with @code{EQ} Lisps will return true on some Lisps, others require
more general predicates like @code{EQL} or @code{EQUALP} and finally
some will return false using any of these predicates. Therefore, for
portability, you should use @code{POINTER-EQ}.

@subheading Examples
This is an example using @acronym{SBCL}, see the
implementation-specific notes above.

@lisp
CFFI> (eql (null-pointer) (null-pointer))
@result{} NIL
CFFI> (pointer-eq (null-pointer) (null-pointer))
@result{} T
@end lisp

@subheading See Also
@seealso{inc-pointer}


@c ===================================================================
@c WITH-FOREIGN-OBJECT

@page
@node with-foreign-object, with-foreign-pointer, pointer-eq, Pointers
@heading with-foreign-object, with-foreign-objects
@subheading Syntax
@Macro{with-foreign-object (var type &optional count) &body body}

@anchor{with-foreign-objects}
@Macro{with-foreign-objects (bindings) &body body}

bindings ::= @{(var type &optional count)@}*

@subheading Arguments and Values

@table @var
@item var
A symbol.

@item type
A foreign type, evaluated.

@item count
An integer.
@end table

@subheading Description
The macros @code{with-foreign-object} and @code{with-foreign-objects}
bind @var{var} to a pointer to @var{count} newly allocated objects
of type @var{type} during @var{body}. The buffer has dynamic extent
and may be stack allocated if supported by the host Lisp.

@subheading Examples

@lisp
CFFI> (with-foreign-object (array :int 10)
        (dotimes (i 10)
          (setf (mem-aref array :int i) (random 100)))
        (loop for i below 10
              collect (mem-aref array :int i)))
@result{} (22 7 22 52 69 1 46 93 90 65)
@end lisp

@subheading See Also
@seealso{foreign-alloc}


@c ===================================================================
@c WITH-FOREIGN-POINTER

@page
@node with-foreign-pointer,  , with-foreign-object, Pointers
@heading with-foreign-pointer
@subheading Syntax
@Macro{with-foreign-pointer (var size &optional size-var) &body body}

@subheading Arguments and Values

@table @var
@item var
@itemx size-var
A symbol.

@item size
An integer.

@item body
A list of forms to be executed.
@end table

@subheading Description
The @code{with-foreign-pointer} macro, binds @var{var} to @var{size}
bytes of foreign memory during @var{body}. The pointer in @var{var}
is invalid beyond the dynamic extend of @var{body} and may be
stack-allocated if supported by the implementation.

If @var{size-var} is supplied, it will be bound to @var{size} during
@var{body}.

@subheading Examples

@lisp
CFFI> (with-foreign-pointer (string 4 size)
        (setf (mem-ref string :char (1- size)) 0)
        (lisp-string-to-foreign "Popcorns" string size)
        (loop for i from 0 below size
              collect (code-char (mem-ref string :char i))))
@result{} (#\P #\o #\p #\Null)
@end lisp

@subheading See Also
@seealso{foreign-alloc} @*
@seealso{foreign-free}


@c ===================================================================
@c CHAPTER: Strings

@node Strings, Variables, Pointers, Top
@chapter Strings

As with many languages, Lisp and C have special support for logical
arrays of characters, going so far as to give them a special name,
``strings''.  In that spirit, @cffi{} provides special support for
translating between Lisp and C strings.

The @code{:string} type and the symbols related below also serve as an
example of what you can do portably with @cffi{}; were it not
included, you could write an equally functional @file{strings.lisp}
without referring to any implementation-specific symbols.

@menu
Dictionary

* *default-foreign-encoding*::
* foreign-string-alloc::
* foreign-string-free::
* foreign-string-to-lisp::
* lisp-string-to-foreign::
* with-foreign-string::
* with-foreign-strings::
* with-foreign-pointer-as-string::
@end menu


@c ===================================================================
@c *DEFAULT-FOREIGN-ENCODING*

@page
@node *default-foreign-encoding*, foreign-string-alloc, Strings, Strings
@heading *default-foreign-encoding*
@subheading Syntax

@Variable{*default-foreign-encoding*}

@subheading Value type

A keyword.

@subheading Initial value

@code{:utf-8}

@subheading Description

This special variable holds the default foreign encoding.

@subheading Examples

@lisp
CFFI> *default-foreign-encoding*
:utf-8
CFFI> (foreign-funcall "strdup" (:string :encoding :utf-16) "foo" :string)
"f"
CFFI> (let ((*default-foreign-encoding* :utf-16))
        (foreign-funcall "strdup" (:string :encoding :utf-16) "foo" :string))
"foo"
@end lisp

@subheading See also

@seealso{Other Types} (@code{:string} type) @*
@seealso{foreign-string-alloc} @*
@seealso{foreign-string-to-lisp} @*
@seealso{lisp-string-to-foreign} @*
@seealso{with-foreign-string} @*
@seealso{with-foreign-pointer-as-string}


@c ===================================================================
@c FOREIGN-STRING-ALLOC

@page
@node foreign-string-alloc, foreign-string-free, *default-foreign-encoding*, Strings
@heading foreign-string-alloc
@subheading Syntax
@Function{foreign-string-alloc string &key encoding null-terminated-p @
                               start end @res{} pointer}

@subheading Arguments and Values

@table @emph
@item @var{string}
A Lisp string.

@item @var{encoding}
Foreign encoding. Defaults to @code{*default-foreign-encoding*}.

@item @var{null-terminated-p}
Boolean, defaults to true.

@item @var{start}, @var{end}
Bounding index designators of @var{string}. 0 and @code{nil}, by
default.

@item @var{pointer}
A pointer to the newly allocated foreign string.
@end table

@subheading Description
The @code{foreign-string-alloc} function allocates foreign memory
holding a copy of @var{string} converted using the specified
@var{encoding}. @var{Start} specifies an offset into @var{string} and
@var{end} marks the position following the last element of the foreign
string.

This string must be freed with @code{foreign-string-free}.

If @var{null-terminated-p} is false, the string will not be
null-terminated.

@subheading Examples

@lisp
CFFI> (defparameter *str* (foreign-string-alloc "Hello, foreign world!"))
@result{} #<FOREIGN-ADDRESS #x00400560>
CFFI> (foreign-funcall "strlen" :pointer *str* :int)
@result{} 21
@end lisp

@subheading See Also
@seealso{foreign-string-free} @*
@seealso{with-foreign-string}
@c @seealso{:string}


@c ===================================================================
@c FOREIGN-STRING-FREE

@page
@node foreign-string-free, foreign-string-to-lisp, foreign-string-alloc, Strings
@heading foreign-string-free
@subheading Syntax
@Function{foreign-string-free pointer}

@subheading Arguments and Values

@table @var
@item pointer
A pointer to a string allocated by @code{foreign-string-alloc}.
@end table

@subheading Description
The @code{foreign-string-free} function frees a foreign string
allocated by @code{foreign-string-alloc}.

@subheading Examples

@subheading See Also
@seealso{foreign-string-alloc}


@c ===================================================================
@c FOREIGN-STRING-TO-LISP

@page
@node foreign-string-to-lisp, lisp-string-to-foreign, foreign-string-free, Strings
@heading foreign-string-to-lisp
@subheading Syntax
@Function{foreign-string-to-lisp ptr &key offset count max-chars @
                                 encoding @res{} string}

@subheading Arguments and Values

@table @var
@item ptr
A pointer.

@item offset
An integer greater than or equal to 0. Defauls to 0.

@item count
Either @code{nil} (the default), or an integer greater than or equal to 0.

@item max-chars
An integer greater than or equal to 0.
@code{(1- array-total-size-limit)}, by default.

@item encoding
Foreign encoding. Defaults to @code{*default-foreign-encoding*}.

@item string
A Lisp string.
@end table

@subheading Description
The @code{foreign-string-to-lisp} function converts at most
@var{count} octets from @var{ptr} into a Lisp string, using the
defined @var{encoding}.

If @var{count} is @code{nil} (the default), characters are copied
until @var{max-chars} is reached or a @code{NULL} character is found.

If @var{ptr} is a null pointer, returns @code{nil}.

Note that the @code{:string} type will automatically convert between
Lisp strings and foreign strings.

@subheading Examples

@lisp
CFFI> (foreign-funcall "getenv" :string "HOME" :pointer)
@result{} #<FOREIGN-ADDRESS #xBFFFFFD5>
CFFI> (foreign-string-to-lisp *)
@result{} "/Users/luis"
@end lisp

@subheading See Also
@seealso{lisp-string-to-foreign} @*
@seealso{foreign-string-alloc}
@c @seealso{:string}


@c ===================================================================
@c LISP-STRING-TO-FOREIGN

@page
@node lisp-string-to-foreign, with-foreign-string, foreign-string-to-lisp, Strings
@heading lisp-string-to-foreign
@subheading Syntax
@Function{lisp-string-to-foreign string buffer bufsize &key start @
                                 end offset encoding @res{} buffer}

@subheading Arguments and Values

@table @emph
@item @var{string}
A Lisp string.

@item @var{buffer}
A foreign pointer.

@item @var{bufsize}
An integer.

@item @var{start}, @var{end}
Bounding index designators of @var{string}. 0 and @code{nil}, by
default.

@item @var{offset}
An integer greater than or equal to 0. Defauls to 0.

@item @var{encoding}
Foreign encoding. Defaults to @code{*default-foreign-encoding*}.
@end table

@subheading Description
The @code{lisp-string-to-foreign} function copies at most
@var{bufsize}-1 octets from a Lisp @var{string} using the specified
@var{encoding} into @var{buffer}+@var{offset}. The foreign string will
be null-terminated.

@var{Start} specifies an offset into @var{string} and
@var{end} marks the position following the last element of the foreign
string.

@subheading Examples

@lisp
CFFI> (with-foreign-pointer-as-string (str 255)
        (lisp-string-to-foreign "Hello, foreign world!" str 6))
@result{} "Hello"
@end lisp

@subheading See Also
@seealso{foreign-string-alloc} @*
@seealso{foreign-string-to-lisp} @*
@seealso{with-foreign-pointer-as-string}


@c ===================================================================
@c WITH-FOREIGN-STRING

@page
@node with-foreign-string, with-foreign-pointer-as-string, lisp-string-to-foreign, Strings
@heading with-foreign-string, with-foreign-strings
@subheading Syntax
@Macro{with-foreign-string (var-or-vars string &rest args) &body body}

@anchor{with-foreign-strings}
@Macro{with-foreign-strings (bindings) &body body}

var-or-vars ::= var | (var &optional octet-size-var)
bindings ::= @{(var-or-vars string &rest args)@}*

@subheading Arguments and Values

@table @emph
@item @var{var}, @var{byte-size-var}
A symbol.

@item @var{string}
A Lisp string.

@item @var{body}
A list of forms to be executed.
@end table

@subheading Description
The @code{with-foreign-string} macro will bind @var{var} to a newly
allocated foreign string containing @var{string}. @var{Args} is passed
to the underlying @code{foreign-string-alloc} call.

If @var{octet-size-var} is provided, it will be bound the length of
foreign string in octets including the null terminator.

@subheading Examples

@lisp
CFFI> (with-foreign-string (foo "12345")
        (foreign-funcall "strlen" :pointer foo :int))
@result{} 5

CFFI> (let ((array (coerce #(84 117 114 97 110 103 97)
                           '(array (unsigned-byte 8)))))
        (with-foreign-string (foreign-string array)
          (foreign-string-to-lisp foreign-string)))
@result{} "Turanga"
@end lisp

@subheading See Also
@seealso{foreign-string-alloc} @*
@seealso{with-foreign-pointer-as-string}


@c ===================================================================
@c WITH-FOREIGN-POINTER-AS-STRING

@page
@node with-foreign-pointer-as-string,  , with-foreign-string, Strings
@heading with-foreign-pointer-as-string
@subheading Syntax
@Macro{with-foreign-pointer-as-string (var size &optional size-var @
                                      &rest args) &body body @res{} string}

@subheading Arguments and Values

@table @var
@item var
A symbol.

@item string
A Lisp string.

@item body
List of forms to be executed.
@end table

@subheading Description
The @code{with-foreign-pointer-as-string} macro is similar to
@code{with-foreign-pointer} except that @var{var} is used as the
returned value of an implicit @code{progn} around @var{body}, after
being converted to a Lisp string using the provided @var{args}.

@subheading Examples

@lisp
CFFI> (with-foreign-pointer-as-string (str 6 str-size :encoding :ascii)
        (lisp-string-to-foreign "Hello, foreign world!" str str-size))
@result{} "Hello"
@end lisp

@subheading See Also
@seealso{foreign-string-alloc} @*
@seealso{with-foreign-string}


@c ===================================================================
@c CHAPTER: Variables

@node Variables, Functions, Strings, Top
@chapter Variables

@menu
Dictionary

* defcvar::
* get-var-pointer::
@end menu


@c ===================================================================
@c DEFCVAR

@page
@node defcvar, get-var-pointer, Variables, Variables
@heading defcvar
@subheading Syntax
@Macro{defcvar name-and-options type &optional documentation @res{} lisp-name}

@var{name-and-options} ::= name | (name &key read-only (library :default)) @*
@var{name} ::= lisp-name [foreign-name] | foreign-name [lisp-name]

@subheading Arguments and Values

@table @var
@item foreign-name
A string denoting a foreign function.

@item lisp-name
A symbol naming the Lisp function to be created.

@item type
A foreign type.

@item read-only
A boolean.

@item documentation
A Lisp string; not evaluated.
@end table

@subheading Description
The @code{defcvar} macro defines a symbol macro @var{lisp-name} that looks
up @var{foreign-name} and dereferences it acording to @var{type}.  It
can also be @code{setf}ed, unless @var{read-only} is true, in which
case an error will be signaled.

When one of @var{lisp-name} or @var{foreign-name} is omitted, the
other is automatically derived using the following rules:

@itemize
@item
Foreign names are converted to Lisp names by uppercasing, replacing
underscores with hyphens, and wrapping around asterisks.
@item
Lisp names are converted to foreign names by lowercasing, replacing
hyphens with underscores, and removing asterisks, if any.
@end itemize

@subheading Examples

@lisp
CFFI> (defcvar "errno" :int)
@result{} *ERRNO*
CFFI> (foreign-funcall "strerror" :int *errno* :string)
@result{} "Inappropriate ioctl for device"
CFFI> (setf *errno* 1)
@result{} 1
CFFI> (foreign-funcall "strerror" :int *errno* :string)
@result{} "Operation not permitted"
@end lisp

Trying to modify a read-only foreign variable:

@lisp
CFFI> (defcvar ("errno" +error-number+ :read-only t) :int)
@result{} +ERROR-NUMBER+
CFFI> (setf +error-number+ 12)
;; @lispcmt{@error{} Trying to modify read-only foreign var: +ERROR-NUMBER+.}
@end lisp

@emph{Note that accessing @code{errno} this way won't work with every
implementation of the C standard library.}

@subheading See Also
@seealso{get-var-pointer}


@c ===================================================================
@c GET-VAR-POINTER

@page
@node get-var-pointer,  , defcvar, Variables
@heading get-var-pointer
@subheading Syntax
@Function{get-var-pointer symbol @res{} pointer}

@subheading Arguments and Values

@table @var
@item symbol
A symbol denoting a foreign variable defined with @code{defcvar}.

@item pointer
A foreign pointer.
@end table

@subheading Description
The function @code{get-var-pointer} will return a @var{pointer} to the
foreign global variable @var{symbol} previously defined with
@code{defcvar}.

@subheading Examples

@lisp
CFFI> (defcvar "errno" :int :read-only t)
@result{} *ERRNO*
CFFI> *errno*
@result{} 25
CFFI> (get-var-pointer '*errno*)
@result{} #<A Mac Pointer #xA0008130>
CFFI> (mem-ref * :int)
@result{} 25
@end lisp

@subheading See Also
@seealso{defcvar}


@c ===================================================================
@c CHAPTER: Functions

@node Functions, Libraries, Variables, Top
@chapter Functions

@menu
@c * Defining Foreign Functions::
@c * Calling Foreign Functions::

Dictionary

* defcfun::
* foreign-funcall::
* foreign-funcall-pointer::
* foreign-funcall-varargs::
* foreign-funcall-pointer-varargs::
* translate-camelcase-name::
* translate-name-from-foreign::
* translate-name-to-foreign::
* translate-underscore-separated-name::
@end menu

@c @node Calling Foreign Functions
@c @section Calling Foreign Functions

@c @node Defining Foreign Functions
@c @section Defining Foreign Functions


@c ===================================================================
@c DEFCFUN

@page
@node defcfun, foreign-funcall, Functions, Functions
@heading defcfun
@subheading Syntax
@Macro{defcfun name-and-options return-type &body [docstring] arguments [&rest] @
               @res{} lisp-name}

@var{name-and-options} ::= @var{name} | (@var{name} &key @var{library} @var{convention}) @*
@var{name} ::= @var{lisp-name} [@var{foreign-name}] | @var{foreign-name} [@var{lisp-name}] @*
@var{arguments} ::= @{ (@var{arg-name} @var{arg-type}) @}* @*

@subheading Arguments and Values

@table @var
@item foreign-name
A string denoting a foreign function.

@item lisp-name
A symbol naming the Lisp function to be created.

@item arg-name
A symbol.

@item return-type
@itemx arg-type
A foreign type.

@item convention
One of @code{:cdecl} (default) or @code{:stdcall}.

@item library
A symbol designating a foreign library.

@item docstring
A documentation string.
@end table

@subheading Description
The @code{defcfun} macro provides a declarative interface for defining
Lisp functions that call foreign functions.

When one of @var{lisp-name} or @var{foreign-name} is omitted, the
other is automatically derived using the following rules:

@itemize
@item
Foreign names are converted to Lisp names by uppercasing and replacing
underscores with hyphens.
@item
Lisp names are converted to foreign names by lowercasing and replacing
hyphens with underscores.
@end itemize

If you place the symbol @code{&rest} in the end of the argument list
after the fixed arguments, @code{defcfun} will treat the foreign
function as a @strong{variadic function}. The variadic arguments
should be passed in a way similar to what @code{foreign-funcall} would
expect. Unlike @code{foreign-funcall} though, @code{defcfun} will take
care of doing argument promotion. Note that in this case
@code{defcfun} will generate a Lisp @emph{macro} instead of a
function and will only work for Lisps that support
@code{foreign-funcall.}

If a foreign structure is to be passed or returned by value (that is,
the type is of the form @code{(:struct ...)}), then the cffi-libffi system
must be loaded, which in turn depends on
@uref{http://sourceware.org/libffi/,libffi}, including the header files.
Failure to load that system will result in an error.
Variadic functions cannot at present accept or return structures by
value.

@subheading Examples

@lisp
(defcfun "strlen" :int
  "Calculate the length of a string."
  (n :string))

CFFI> (strlen "123")
@result{} 3
@end lisp

@lisp
(defcfun ("abs" c-abs) :int (n :int))

CFFI> (c-abs -42)
@result{} 42
@end lisp

Function without arguments:

@lisp
(defcfun "rand" :int)

CFFI> (rand)
@result{} 1804289383
@end lisp

Variadic function example:

@lisp
(defcfun "sprintf" :int
  (str :pointer)
  (control :string)
  &rest)

CFFI> (with-foreign-pointer-as-string (s 100)
        (sprintf s "%c %d %.2f %s" :char 90 :short 42 :float pi
                 :string "super-locrian"))
@result{} "A 42 3.14 super-locrian"
@end lisp

@subheading See Also
@seealso{foreign-funcall} @*
@seealso{foreign-funcall-pointer} @*
@seealso{foreign-funcall-varargs} @*
@seealso{foreign-funcall-pointer-varargs}


@c ===================================================================
@c FOREIGN-FUNCALL

@page
@node foreign-funcall, foreign-funcall-pointer, defcfun, Functions
@heading foreign-funcall
@subheading Syntax
@Macro{foreign-funcall name-and-options &rest arguments @res{} return-value}

@var{arguments} ::= @{ @var{arg-type} @var{arg} @}* [@var{return-type}] @*
@var{name-and-options} ::= @var{name} | (@var{name} &key @var{library} @var{convention}) @*

@subheading Arguments and Values

@table @var
@item name
A Lisp string.

@item arg-type
A foreign type.

@item arg
An argument of type @var{arg-type}.

@item return-type
A foreign type, @code{:void} by default.

@item return-value
A lisp object.

@item library
A lisp symbol; not evaluated.

@item convention
One of @code{:cdecl} (default) or @code{:stdcall}.
@end table

@subheading Description
The @code{foreign-funcall} macro is the main primitive for calling
foreign functions.

If a foreign structure is to be passed or returned by value (that is,
the type is of the form @code{(:struct ...)}), then the cffi-libffi system
must be loaded, which in turn depends on
@uref{http://sourceware.org/libffi/,libffi}, including the header files.
Failure to load that system will result in an error.
Variadic functions cannot at present accept or return structures by
value.

@emph{Note: The return value of foreign-funcall on functions with a
:void return type is still undefined.}

@subheading Implementation-specific Notes
@itemize
@item
Corman Lisp does not support @code{foreign-funcall}. On
implementations that @strong{don't} support @code{foreign-funcall}
@code{cffi-sys::no-foreign-funcall} will be present in
@code{*features*}. Note: in these Lisps you can still use the
@code{defcfun} interface.
@end itemize

@subheading Examples

@lisp
CFFI> (foreign-funcall "strlen" :string "foo" :int)
@result{} 3
@end lisp

Given the C code:

@example
void print_number(int n)
@{
    printf("N: %d\n", n);
@}
@end example

@lisp
CFFI> (foreign-funcall "print_number" :int 123456)
@print{} N: 123456
@result{} NIL
@end lisp

@noindent
Or, equivalently:

@lisp
CFFI> (foreign-funcall "print_number" :int 123456 :void)
@print{} N: 123456
@result{} NIL
@end lisp

@lisp
CFFI> (foreign-funcall "printf" :string (format nil "%s: %d.~%")
                       :string "So long and thanks for all the fish"
                       :int 42 :int)
@print{} So long and thanks for all the fish: 42.
@result{} 41
@end lisp

@subheading See Also
@seealso{defcfun} @*
@seealso{foreign-funcall-pointer}


@c ===================================================================
@c FOREIGN-FUNCALL-POINTER

@page
@node foreign-funcall-pointer, foreign-funcall-varargs, foreign-funcall, Functions
@heading foreign-funcall-pointer
@subheading Syntax
@Macro{foreign-funcall-pointer pointer options &rest arguments @res{} return-value}

@var{arguments} ::= @{ @var{arg-type} @var{arg} @}* [@var{return-type}] @*
@var{options} ::= (&key @var{convention}) @*

@subheading Arguments and Values

@table @var
@item pointer
A foreign pointer.

@item arg-type
A foreign type.

@item arg
An argument of type @var{arg-type}.

@item return-type
A foreign type, @code{:void} by default.

@item return-value
A lisp object.

@item convention
One of @code{:cdecl} (default) or @code{:stdcall}.
@end table

@subheading Description
The @code{foreign-funcall} macro is the main primitive for calling
foreign functions.

@emph{Note: The return value of foreign-funcall on functions with a
:void return type is still undefined.}

@subheading Implementation-specific Notes
@itemize
@item
Corman Lisp does not support @code{foreign-funcall}. On
implementations that @strong{don't} support @code{foreign-funcall}
@code{cffi-sys::no-foreign-funcall} will be present in
@code{*features*}. Note: in these Lisps you can still use the
@code{defcfun} interface.
@end itemize

@subheading Examples

@lisp
CFFI> (foreign-funcall-pointer (foreign-symbol-pointer "abs") ()
                               :int -42 :int)
@result{} 42
@end lisp

@subheading See Also
@seealso{defcfun} @*
@seealso{foreign-funcall}


@c ===================================================================
@c FOREIGN-FUNCALL-VARARGS

@page
@node foreign-funcall-varargs, foreign-funcall-pointer-varargs, foreign-funcall-pointer, Functions
@heading foreign-funcall-varargs
@subheading Syntax
@Macro{foreign-funcall-varargs name-and-options (fixed-arguments) &rest arguments @res{} return-value}

@var{fixed-arguments} ::= @{ @var{arg-type} @var{arg} @}* [@var{return-type}] @*
@var{arguments} ::= @{ @var{arg-type} @var{arg} @}* [@var{return-type}] @*
@var{name-and-options} ::= @var{name} | (@var{name} &key @var{library} @var{convention}) @*

@subheading Arguments and Values

@table @var
@item name
A Lisp string.

@item arg-type
A foreign type.

@item arg
An argument of type @var{arg-type}.

@item return-type
A foreign type, @code{:void} by default.

@item return-value
A lisp object.

@item library
A lisp symbol; not evaluated.

@item convention
One of @code{:cdecl} (default) or @code{:stdcall}.
@end table

@subheading Description
The @code{foreign-funcall-varargs} macro is the main primitive for
calling foreign variadic functions. It behaves similarily to
@code{foreign-funcall} except @code{fixed-arguments} are distinguished
from the remaining arguments.

@subheading Examples

@lisp
CFFI> (with-foreign-pointer-as-string (s 100)
        (setf (mem-ref s :char) 0)
        (foreign-funcall-varargs
         "sprintf" (:pointer s :string) "%.2f")
         :double (coerce pi 'double-float) :int))
@result{} 3.14
@end lisp


@c ===================================================================
@c FOREIGN-FUNCALL-POINTER-VARARGS

@page
@node foreign-funcall-pointer-varargs, translate-camelcase-name, foreign-funcall-varargs, Functions
@heading foreign-funcall-pointer-varargs
@subheading Syntax
@Macro{foreign-funcall-pointer-varargs pointer options (fixed-arguments) &rest arguments @res{} return-value}

@var{fixed-arguments} ::= @{ @var{arg-type} @var{arg} @}* [@var{return-type}] @*
@var{arguments} ::= @{ @var{arg-type} @var{arg} @}* [@var{return-type}] @*
@var{options} ::= (&key @var{convention}) @*

@subheading Arguments and Values

@table @var
@item pointer
A foreign pointer.

@item arg-type
A foreign type.

@item arg
An argument of type @var{arg-type}.

@item return-type
A foreign type, @code{:void} by default.

@item return-value
A lisp object.

@item convention
One of @code{:cdecl} (default) or @code{:stdcall}.
@end table

@subheading Description
The @code{foreign-funcall-pointer-varargs} macro is the main primitive
for calling foreign variadic functions. It behaves similarily to
@code{foreign-funcall-pointer} except @code{fixed-arguments} are
distinguished from the remaining arguments.

@subheading Examples

@lisp
CFFI> (with-foreign-pointer-as-string (s 100)
        (setf (mem-ref s :char) 0)
        (foreign-funcall-pointer-varargs
         (foreign-symbol-pointer "sprintf") () (:pointer s :string "%.2f")
         :double (coerce pi 'double-float) :int))
@result{} 3.14
@end lisp


@c ===================================================================
@c TRANSLATE-CAMELCASE-NAME

@page
@node translate-camelcase-name, translate-name-from-foreign, foreign-funcall-pointer-varargs, Functions
@heading translate-camelcase-name
@subheading Syntax
@Function{translate-camelcase-name name &key upper-initial-p special-words @res{} return-value}

@subheading Arguments and Values

@table @var
@item name
Either a symbol or a string.

@item upper-initial-p
A generalized boolean.

@item special words
A list of strings.

@item return-value
If @var{name} is a symbol, this is a string, and vice versa.
@end table

@subheading Description
@code{translate-camelcase-name} is a helper function for
specializations of @code{translate-name-from-foreign} and
@code{translate-name-to-foreign}. It handles the common case of
converting between foreign camelCase names and lisp
names. @var{upper-initial-p} indicates whether the first letter of the
foreign name should be uppercase. @var{special-words} is a list of
strings that should be treated atomically in translation. This list is
case-sensitive.

@subheading Examples

@lisp
CFFI> (translate-camelcase-name some-xml-function)
@result{} "someXmlFunction"
CFFI> (translate-camelcase-name some-xml-function :upper-initial-p t)
@result{} "SomeXmlFunction"
CFFI> (translate-camelcase-name some-xml-function :special-words '("XML"))
@result{} "someXMLFunction"
CFFI> (translate-camelcase-name "someXMLFunction")
@result{} SOME-X-M-L-FUNCTION
CFFI> (translate-camelcase-name "someXMLFunction" :special-words '("XML"))
@result{} SOME-XML-FUNCTION
@end lisp

@subheading See Also
@seealso{translate-name-from-foreign} @*
@seealso{translate-name-to-foreign} @*
@seealso{translate-underscore-separated-name}


@c ===================================================================
@c TRANSLATE-NAME-FROM-FOREIGN

@page
@node translate-name-from-foreign, translate-name-to-foreign, translate-camelcase-name, Functions
@heading translate-name-from-foreign
@subheading Syntax
@Function{translate-name-from-foreign foreign-name package &optional varp @res{} symbol}

@subheading Arguments and Values

@table @var
@item foreign-name
A string denoting a foreign function.

@item package
A Lisp package

@item varp
A generalized boolean.

@item symbol
The Lisp symbol to be used a function name.
@end table

@subheading Description
@code{translate-name-from-foreign} is used by @ref{defcfun} to handle
the conversion of foreign names to lisp names. By default, it
translates using @ref{translate-underscore-separated-name}. However,
you can create specialized methods on this function to make
translating more closely match the foreign library's naming
conventions.

Specialize @var{package} on some package. This allows other packages
to load libraries with different naming conventions.

@subheading Examples

@lisp
CFFI> (defcfun "someXmlFunction" ...)
@result{} SOMEXMLFUNCTION
CFFI> (defmethod translate-name-from-foreign ((spec string)
                                              (package (eql *package*))
                                              &optional varp)
        (let ((name (translate-camelcase-name spec)))
          (if varp (intern (format nil "*~a*" name)) name)))
@result{} #<STANDARD-METHOD TRANSLATE-NAME-FROM-FOREIGN (STRING (EQL #<Package "SOME-PACKAGE">))>
CFFI> (defcfun "someXmlFunction" ...)
@result{} SOME-XML-FUNCTION
@end lisp

@subheading See Also
@seealso{defcfun} @*
@seealso{translate-camelcase-name} @*
@seealso{translate-name-to-foreign} @*
@seealso{translate-underscore-separated-name}


@c ===================================================================
@c TRANSLATE-NAME-TO-FOREIGN

@page
@node translate-name-to-foreign, translate-underscore-separated-name, translate-name-from-foreign, Functions
@heading translate-name-to-foreign
@subheading Syntax
@Function{translate-name-to-foreign lisp-name package &optional varp @res{} string}

@subheading Arguments and Values

@table @var
@item lisp-name
A symbol naming the Lisp function to be created.

@item package
A Lisp package

@item varp
A generalized boolean.

@item string
The string representing the foreign function name.
@end table

@subheading Description
@code{translate-name-to-foreign} is used by @ref{defcfun} to handle
the conversion of lisp names to foreign names. By default, it
translates using @ref{translate-underscore-separated-name}. However,
you can create specialized methods on this function to make
translating more closely match the foreign library's naming
conventions.

Specialize @var{package} on some package. This allows other packages
to load libraries with different naming conventions.

@subheading Examples

@lisp
CFFI> (defcfun some-xml-function ...)
@result{} "some_xml_function"
CFFI> (defmethod translate-name-to-foreign ((spec symbol)
                                            (package (eql *package*))
                                            &optional varp)
        (let ((name (translate-camelcase-name spec)))
          (if varp (subseq name 1 (1- (length name))) name)))
@result{} #<STANDARD-METHOD TRANSLATE-NAME-TO-FOREIGN (STRING (EQL #<Package "SOME-PACKAGE">))>
CFFI> (defcfun some-xml-function ...)
@result{} "someXmlFunction"
@end lisp

@subheading See Also
@seealso{defcfun} @*
@seealso{translate-camelcase-name} @*
@seealso{translate-name-from-foreign} @*
@seealso{translate-underscore-separated-name}


@c ===================================================================
@c TRANSLATE-UNDERSCORE-SEPARATED-NAME

@page
@node translate-underscore-separated-name,  , translate-name-to-foreign, Functions
@heading translate-underscore-separated-name
@subheading Syntax
@Function{translate-underscore-separated-name name @res{} return-value}

@subheading Arguments and Values

@table @var
@item name
Either a symbol or a string.

@item return-value
If @var{name} is a symbol, this is a string, and vice versa.
@end table

@subheading Description
@code{translate-underscore-separated-name} is a helper function for
specializations of @ref{translate-name-from-foreign} and
@ref{translate-name-to-foreign}. It handles the common case of
converting between foreign underscore_separated names and lisp names.

@subheading Examples

@lisp
CFFI> (translate-underscore-separated-name some-xml-function)
@result{} "some_xml_function"
CFFI> (translate-camelcase-name "some_xml_function")
@result{} SOME-XML-FUNCTION
@end lisp

@subheading See Also
@seealso{translate-name-from-foreign} @*
@seealso{translate-name-to-foreign} @*
@seealso{translate-camelcase-name}


@c ===================================================================
@c CHAPTER: Libraries

@node Libraries, Callbacks, Functions, Top
@chapter Libraries

@menu
* Defining a library::
* Library definition style::

Dictionary

* close-foreign-library::       Close a foreign library.
* *darwin-framework-directories*::  Search path for Darwin frameworks.
* define-foreign-library::      Explain how to load a foreign library.
* *foreign-library-directories*::  Search path for shared libraries.
* load-foreign-library::        Load a foreign library.
* load-foreign-library-error::  Signalled on failure of its namesake.
* use-foreign-library::         Load a foreign library when needed.
@end menu


@node Defining a library, Library definition style, Libraries, Libraries
@section Defining a library

Almost all foreign code you might want to access exists in some kind
of shared library.  The meaning of @dfn{shared library} varies among
platforms, but for our purposes, we will consider it to include
@file{.so} files on @sc{unix}, frameworks on Darwin (and derivatives
like Mac @acronym{OS X}), and @file{.dll} files on Windows.

Bringing one of these libraries into the Lisp image is normally a
two-step process.

@enumerate
@item
Describe to @cffi{} how to load the library at some future point,
depending on platform and other factors, with a
@code{define-foreign-library} top-level form.

@item
Load the library so defined with either a top-level
@code{use-foreign-library} form or by calling the function
@code{load-foreign-library}.
@end enumerate

@xref{Tutorial-Loading,, Loading foreign libraries}, for a working
example of the above two steps.


@node Library definition style, close-foreign-library, Defining a library, Libraries
@section Library definition style

Looking at the @code{libcurl} library definition presented earlier,
you may ask why we did not simply do this:

@lisp
(define-foreign-library libcurl
  (t (:default "libcurl")))
@end lisp

@noindent
Indeed, this would work just as well on the computer on which I tested
the tutorial.  There are a couple of good reasons to provide the
@file{.so}'s current version number, however.  Namely, the versionless
@file{.so} is not packaged on most @sc{unix} systems along with the
actual, fully-versioned library; instead, it is included in the
``development'' package along with C headers and static @file{.a}
libraries.

The reason @cffi{} does not try to account for this lies in the
meaning of the version numbers.  A full treatment of shared library
versions is beyond this manual's scope; see @ref{Versioning,, Library
interface versions, libtool, @acronym{GNU} Libtool}, for helpful
information for the unfamiliar.  For our purposes, consider that a
mismatch between the library version with which you tested and the
installed library version may cause undefined
behavior.@footnote{Windows programmers may chafe at adding a
@sc{unix}-specific clause to @code{define-foreign-library}.  Instead,
ask why the Windows solution to library incompatibility is ``include
your own version of every library you use with every program''.}

@impnote{Maybe some notes should go here about OS X, which I know
little about.  --stephen}


@c ===================================================================
@c CLOSE-FOREIGN-LIBRARY

@page
@node close-foreign-library, *darwin-framework-directories*, Library definition style, Libraries
@heading close-foreign-library
@subheading Syntax
@Function{close-foreign-library library @res{} success}

@subheading Arguments and Values

@table @var
@item library
A symbol or an instance of @code{foreign-library}.

@item success
A Lisp boolean.
@end table

@subheading Description

Closes @var{library} which can be a symbol designating a library
define through @code{define-foreign-library} or an instance of
@code{foreign-library} as returned by @code{load-foreign-library}.

@c @subheading Examples
@c @xref{Tutorial-Loading,, Loading foreign libraries}.

@subheading See Also

@seealso{define-foreign-library} @*
@seealso{load-foreign-library} @*
@seealso{use-foreign-library}


@c ===================================================================
@c *DARWIN-FRAMEWORK-DIRECTORIES*

@page
@node *darwin-framework-directories*, define-foreign-library, close-foreign-library, Libraries
@heading *darwin-framework-directories*
@subheading Syntax

@Variable{*darwin-framework-directories*}

@subheading Value type

A list, in which each element is a string, a pathname, or a simple
Lisp expression.

@subheading Initial value

A list containing the following, in order: an expression corresponding
to Darwin path @file{~/Library/Frameworks/},
@code{#P"/Library/Frameworks/"}, and
@code{#P"/System/Library/Frameworks/"}.

@subheading Description

The meaning of ``simple Lisp expression'' is explained in
@ref{*foreign-library-directories*}.  In contrast to that variable,
this is not a fallback search path; the default value described above
is intended to be a reasonably complete search path on Darwin systems.

@subheading Examples

@lisp
CFFI> (let ((lib (load-foreign-library '(:framework "OpenGL"))))
        (foreign-library-pathname lib))
@result{} #P"/System/Library/Frameworks/OpenGL.framework/OpenGL"
@end lisp

@subheading See also

@seealso{*foreign-library-directories*} @*
@seealso{define-foreign-library}


@c ===================================================================
@c DEFINE-FOREIGN-LIBRARY

@page
@node define-foreign-library, *foreign-library-directories*, *darwin-framework-directories*, Libraries
@heading define-foreign-library

@subheading Syntax

@Macro{define-foreign-library name-and-options @{ load-clause @}* @res{} name}

name-and-options ::= name | (name &key convention search-path)
load-clause ::= (feature library &key convention search-path)

@subheading Arguments and Values

@table @var
@item name
A symbol.

@item feature
A feature expression.

@item library
A library designator.

@item convention
One of @code{:cdecl} (default) or @code{:stdcall}

@item search-path
A path or list of paths where the library will be searched if not found in
system-global directories. Paths specified in a load clause take priority over
paths specified as library option, with *foreign-library-directories* having
lowest priority.
@end table

@subheading Description

Creates a new library designator called @var{name}.  The
@var{load-clause}s describe how to load that designator when passed to
@code{load-foreign-library} or @code{use-foreign-library}.

When trying to load the library @var{name}, the relevant function
searches the @var{load-clause}s in order for the first one where
@var{feature} evaluates to true.  That happens for any of the
following situations:

@enumerate 1
@item
If @var{feature} is a symbol present in @code{common-lisp:*features*}.

@item
If @var{feature} is a list, depending on @code{(first @var{feature})},
a keyword:

@table @code
@item :and
All of the feature expressions in @code{(rest @var{feature})} are
true.

@item :or
At least one of the feature expressions in @code{(rest @var{feature})}
is true.

@item :not
The feature expression @code{(second @var{feature})} is not true.
@end table

@item
Finally, if @var{feature} is @code{t}, this @var{load-clause} is
picked unconditionally.
@end enumerate

Upon finding the first true @var{feature}, the library loader then
loads the @var{library}.  The meaning of ``library designator'' is
described in @ref{load-foreign-library}.

Functions associated to a library defined by
@code{define-foreign-library} (e.g. through @code{defcfun}'s
@code{:library} option, will inherit the library's options.  The
precedence is as follows:

@enumerate 1
@item
@code{defcfun}/@code{foreign-funcall} specific options;

@item
@var{load-clause} options;

@item
global library options (the @var{name-and-options} argument)
@end enumerate


@subheading Examples

@xref{Tutorial-Loading,, Loading foreign libraries}.


@subheading See Also

@seealso{close-foreign-library} @*
@seealso{load-foreign-library}


@c ===================================================================
@c *FOREIGN-LIBRARY-DIRECTORIES*

@page
@node *foreign-library-directories*, load-foreign-library, define-foreign-library, Libraries
@heading *foreign-library-directories*
@subheading Syntax

@Variable{*foreign-library-directories*}

@subheading Value type

A list, in which each element is a string, a pathname, or a simple
Lisp expression.

@subheading Initial value

The empty list.

@subheading Description

You should not have to use this variable.

Most, if not all, Lisps supported by @cffi{} have a reasonable default
search algorithm for foreign libraries.  For example, Lisps for
@sc{unix} usually call
@uref{http://www.opengroup.org/onlinepubs/009695399/functions/dlopen.html,,
@code{dlopen(3)}}, which in turn looks in the system library
directories.  Only if that fails does @cffi{} look for the named
library file in these directories, and load it from there if found.

Thus, this is intended to be a @cffi{}-only fallback to the library
search configuration provided by your operating system.  For example,
if you distribute a foreign library with your Lisp package, you can
add the library's containing directory to this list and portably
expect @cffi{} to find it.

A @dfn{simple Lisp expression} is intended to provide functionality
commonly used in search paths such as
@acronym{ASDF}'s@footnote{@xref{Using asdf to load systems,,, asdf,
asdf: another system definition facility}, for information on
@code{asdf:*central-registry*}.}, and is defined recursively as
follows:@footnote{See @code{mini-eval} in @file{libraries.lisp} for
the source of this definition.  As is always the case with a Lisp
@code{eval}, it's easier to understand the Lisp definition than the
english.}

@enumerate
@item
A list, whose @samp{first} is a function designator, and whose
@samp{rest} is a list of simple Lisp expressions to be evaluated and
passed to the so-designated function.  The result is the result of the
function call.

@item
A symbol, whose result is its symbol value.

@item
Anything else evaluates to itself.
@end enumerate

The result of evaluating the @dfn{simple Lisp expression} should yield
a @emph{designator} for a @emph{list} of @emph{pathname designators}.

@strong{Note}: in Common Lisp, @code{#p"/foo/bar"} designates the
@emph{bar} file within the @emph{/foo} directory whereas
@code{#p"/foo/bar/"} designates the @emph{/foo/bar} directory. Keep
that in mind when customising the value of
@code{*foreign-library-directories*}.


@subheading Examples

@example
$ ls
@print{} liblibli.so    libli.lisp
@end example

@noindent
In @file{libli.lisp}:

@lisp
(pushnew #P"/home/sirian/lisp/libli/" *foreign-library-directories*
         :test #'equal)

(load-foreign-library '(:default "liblibli"))
@end lisp

@noindent
The following example would achieve the same effect:

@lisp
(pushnew '(merge-pathnames #p"lisp/libli/" (user-homedir-pathname))
          *foreign-library-directories*
          :test #'equal)
@result{} ((MERGE-PATHNAMES #P"lisp/libli/" (USER-HOMEDIR-PATHNAME)))

(load-foreign-library '(:default "liblibli"))
@end lisp

@subheading See also

@seealso{*darwin-framework-directories*} @*
@seealso{define-foreign-library}


@c ===================================================================
@c LOAD-FOREIGN-LIBRARY

@page
@node load-foreign-library, load-foreign-library-error, *foreign-library-directories*, Libraries
@heading load-foreign-library
@subheading Syntax
@Function{load-foreign-library library-designator @res{} library}

@subheading Arguments and Values

@table @var
@item library-designator
A library designator.

@item library-designator
An instance of @code{foreign-library}.
@end table

@subheading Description

Load the library indicated by @var{library-designator}. A @dfn{library
designator} is defined as follows:

@enumerate
@item
If a symbol, is considered a name previously defined with
@code{define-foreign-library}.

@item
If a string or pathname, passed as a namestring directly to the
implementation's foreign library loader.  If that fails, search the
directories in @code{*foreign-library-directories*} with
@code{cl:probe-file}; if found, the absolute path is passed to the
implementation's loader.

@item
If a list, the meaning depends on @code{(first @var{library})}:

@table @code
@item :framework
The second list element is taken to be a Darwin framework name, which
is then searched in @code{*darwin-framework-directories*}, and loaded
when found.

@item :or
Each remaining list element, itself a @dfn{library designator}, is loaded in
order, until one succeeds.

@item :default
The name is transformed according to the platform's naming convention
to shared libraries, and the resultant string is loaded as a @dfn{library
designator}.  For example, on @sc{unix}, the name is suffixed with
@file{.so}.
@end table
@end enumerate

If the library is already loaded it will be reloaded.

If the load fails, signal a @code{load-foreign-library-error}.

@strong{Please note:} For system libraries, you should not need to
specify the directory containing the library.  Each operating system
has its own idea of a default search path, and you should rely on it
when it is reasonable.

@subheading Implementation-specific Notes
On ECL platforms where its dynamic FFI is not supported (ie. when
@code{:dffi} is not present in @code{*features*}),
@code{cffi:load-foreign-library} does not work and you must use ECL's
own @code{ffi:load-foreign-library} with a constant string argument.

@subheading Examples

@xref{Tutorial-Loading,, Loading foreign libraries}.

@subheading See Also

@seealso{close-foreign-library} @*
@seealso{*darwin-framework-directories*} @*
@seealso{define-foreign-library} @*
@seealso{*foreign-library-directories*} @*
@seealso{load-foreign-library-error} @*
@seealso{use-foreign-library}


@c ===================================================================
@c LOAD-FOREIGN-LIBRARY-ERROR

@page
@node load-foreign-library-error, use-foreign-library, load-foreign-library, Libraries
@heading load-foreign-library-error

@subheading Syntax

@Condition{load-foreign-library-error}

@subheading Class precedence list

@code{load-foreign-library-error}, @code{error},
@code{serious-condition}, @code{condition}, @code{t}

@subheading Description

Signalled when a foreign library load completely fails.  The exact
meaning of this varies depending on the real conditions at work, but
almost universally, the implementation's error message is useless.
However, @cffi{} does provide the useful restarts @code{retry} and
@code{use-value}; invoke the @code{retry} restart to try loading the
foreign library again, or the @code{use-value} restart to try loading
a different foreign library designator.

@subheading See also

@seealso{load-foreign-library}


@c ===================================================================
@c USE-FOREIGN-LIBRARY

@page
@node use-foreign-library,  , load-foreign-library-error, Libraries
@heading use-foreign-library

@subheading Syntax

@Macro{use-foreign-library name}

@subheading Arguments and values

@table @var
@item name
A library designator; unevaluated.
@end table


@subheading Description

@xref{load-foreign-library}, for the meaning of ``library
designator''.  This is intended to be the top-level form used
idiomatically after a @code{define-foreign-library} form to go ahead
and load the library. @c ; it also sets the ``current foreign library''.
Finally, on implementations where the regular evaluation rule is
insufficient for foreign library loading, it loads it at the required
time.@footnote{Namely, @acronym{CMUCL}.  See
@code{use-foreign-library} in @file{libraries.lisp} for details.}

@c current foreign library is a concept created a few hours ago as of
@c this writing.  It is not actually used yet, but probably will be.

@subheading Examples

@xref{Tutorial-Loading,, Loading foreign libraries}.


@subheading See also

@seealso{load-foreign-library}


@c ===================================================================
@c CHAPTER: Callbacks

@node Callbacks, The Groveller, Libraries, Top
@chapter Callbacks

@menu
Dictionary

* callback::
* defcallback::
* get-callback::
@end menu


@c ===================================================================
@c CALLBACK

@page
@node callback, defcallback, Callbacks, Callbacks
@heading callback
@subheading Syntax
@Macro{callback symbol @res{} pointer}

@subheading Arguments and Values

@table @var
@item symbol
A symbol denoting a callback.

@item pointer
@itemx new-value
A pointer.
@end table

@subheading Description
The @code{callback} macro is analogous to the standard CL special
operator @code{function} and will return a pointer to the callback
denoted by the symbol @var{name}.

@subheading Examples

@lisp
CFFI> (defcallback sum :int ((a :int) (b :int))
        (+ a b))
@result{} SUM
CFFI> (callback sum)
@result{} #<A Mac Pointer #x102350>
@end lisp

@subheading See Also
@seealso{get-callback} @*
@seealso{defcallback}


@c ===================================================================
@c DEFCALLBACK

@page
@node defcallback, get-callback, callback, Callbacks
@heading defcallback
@subheading Syntax
@Macro{defcallback name-and-options return-type arguments &body body @res{} name}

name-and-options ::= name | (name &key convention)
arguments ::= (@{ (arg-name arg-type) @}*)

@subheading Arguments and Values

@table @var
@item name
A symbol naming the callback created.

@item return-type
The foreign type for the callback's return value.

@item arg-name
A symbol.

@item arg-type
A foreign type.

@item convention
One of @code{:cdecl} (default) or @code{:stdcall}.
@end table

@subheading Description
The @code{defcallback} macro defines a Lisp function that can be called
from C. The arguments passed to this function will be converted to the
appropriate Lisp representation and its return value will be converted
to its C representation.

This Lisp function can be accessed by the @code{callback} macro or the
@code{get-callback} function.

@strong{Portability note:} @code{defcallback} will not work correctly
on some Lisps if it's not a top-level form.

@subheading Examples

@lisp
(defcfun "qsort" :void
  (base :pointer)
  (nmemb :int)
  (size :int)
  (fun-compar :pointer))

(defcallback < :int ((a :pointer) (b :pointer))
  (let ((x (mem-ref a :int))
        (y (mem-ref b :int)))
    (cond ((> x y) 1)
          ((< x y) -1)
          (t 0))))

CFFI> (with-foreign-object (array :int 10)
        ;; @lispcmt{Initialize array.}
        (loop for i from 0 and n in '(7 2 10 4 3 5 1 6 9 8)
              do (setf (mem-aref array :int i) n))
        ;; @lispcmt{Sort it.}
        (qsort array 10 (foreign-type-size :int) (callback <))
        ;; @lispcmt{Return it as a list.}
        (loop for i from 0 below 10
              collect (mem-aref array :int i)))
@result{} (1 2 3 4 5 6 7 8 9 10)
@end lisp

@subheading See Also
@seealso{callback} @*
@seealso{get-callback}


@c ===================================================================
@c GET-CALLBACK

@page
@node get-callback,  , defcallback, Callbacks
@heading get-callback
@subheading Syntax
@Accessor{get-callback symbol @res{} pointer}

@subheading Arguments and Values

@table @var
@item symbol
A symbol denoting a callback.

@item pointer
A pointer.
@end table

@subheading Description
This is the functional version of the @code{callback} macro. It
returns a pointer to the callback named by @var{symbol} suitable, for
example, to pass as arguments to foreign functions.

@subheading Examples

@lisp
CFFI> (defcallback sum :int ((a :int) (b :int))
        (+ a b))
@result{} SUM
CFFI> (get-callback 'sum)
@result{} #<A Mac Pointer #x102350>
@end lisp

@subheading See Also
@seealso{callback} @*
@seealso{defcallback}


@c ===================================================================
@c CHAPTER: The Groveller

@node The Groveller, Limitations, Callbacks, Top
@chapter The Groveller

@cffi{}-Grovel is a tool which makes it easier to write @cffi{}
declarations for libraries that are implemented in C.  That is, it
grovels through the system headers, getting information about types
and structures, so you don't have to.  This is especially important
for libraries which are implemented in different ways by different
vendors, such as the @sc{unix}/@sc{posix} functions.  The @cffi{}
declarations are usually quite different from platform to platform,
but the information you give to @cffi{}-Grovel is the same.  Hence,
much less work is required!

If you use @acronym{ASDF}, @cffi{}-Grovel is integrated, so that it
will run automatically when your system is building.  This feature was
inspired by SB-Grovel, a similar @acronym{SBCL}-specific project.
@cffi{}-Grovel can also be used without @acronym{ASDF}.

@section Building FFIs with CFFI-Grovel

@cffi{}-Grovel uses a specification file (*.lisp) describing the
features that need groveling.  The C compiler is used to retrieve this
data and write a Lisp file (*.cffi.lisp) which contains the necessary
@cffi{} definitions to access the variables, structures, constants, and
enums mentioned in the specification.

@c This is most similar to the SB-Grovel package, upon which it is
@c based.  Unlike SB-Grovel, we do not currently support defining
@c regular foreign functions in the specification file; those are best
@c defined in normal Lisp code.

@cffi{}-Grovel provides an @acronym{ASDF} component for handling the
necessary calls to the C compiler and resulting file management.

@c See the included CFFI-Unix package for an example of how to
@c integrate a specification file with ASDF-built packages.

@menu
* Groveller Syntax::            How grovel files should look like.
* Groveller ASDF Integration::  ASDF components for grovel files.
* Groveller Implementation Notes:: Implementation notes.
* Wrapper for Inline/Static Functions and Macros:: Wrapper
@end menu

@node Groveller Syntax, Groveller ASDF Integration, The Groveller, The Groveller
@section Specification File Syntax

The specification files are read by the normal Lisp reader, so they
have syntax very similar to normal Lisp code.  In particular,
semicolon-comments and reader-macros will work as expected.

There are several forms recognized by @cffi{}-Grovel:

@deffn {Grovel Form} progn &rest forms

Processes a list of forms. Useful for conditionalizing several
forms. For example:
@end deffn

@lisp
#+freebsd
(progn
  (constant (ev-enable "EV_ENABLE"))
  (constant (ev-disable "EV_DISABLE")))
@end lisp

@deffn {Grovel Form} include &rest files

Include the specified files (specified as strings) in the generated C
source code.
@end deffn

@deffn {Grovel Form} in-package symbol

Set the package to be used for the final Lisp output.
@end deffn

@deffn {Grovel Form} ctype lisp-name size-designator

Define a @cffi{} foreign type for the string in @var{size-designator},
e.g. @code{(ctype :pid "pid_t")}.
@end deffn

@deffn {Grovel Form} constant (lisp-name &rest c-names) &key type documentation optional

Search for the constant named by the first @var{c-name} string found
to be known to the C preprocessor and define it as @var{lisp-name}.

The @var{type} keyword argument specifies how to grovel the constant:
either @code{integer} (the default) or @code{double-float}. If
@var{optional} is true, no error will be raised if all the
@var{c-names} are unknown. If @var{lisp-name} is a keyword, the actual
constant will be a symbol of the same name interned in the current
package.
@end deffn

@deffn {Grovel Form} feature lisp-feature-name c-name &key feature-list

Adds @var{lisp-feature-name} to the list @var{feature-list} if the @var{c-name}
string is known to the C preprocessor. @var{feature-list} defaults
to @code{cl:*features*}.
@end deffn

@deffn {Grovel Form} define name &optional value

Defines an additional C preprocessor symbol, which is useful for
altering the behavior of included system headers.
@end deffn

@deffn {Grovel Form} cc-flags &rest flags

Adds @var{cc-flags} to the command line arguments used for the C compiler
invocation.
@end deffn

@deffn {Grovel Form} pkg-config-cflags pkg &key optional

Adds @var{pkg} to the command line arguments for the external program
@code{pkg-config} and runs it to retrieve the relevant include flags
used for the C compiler invocation. This syntax can be used instead of
hard-coding paths using @code{cc-flags}, and ensures that include
flags are added correctly on the build system. Assumes
@code{pkg-config} is installed and working.  @var{pkg} is a string
that identifies an installed @code{pkg-config} package. See the
pkg-config manual for more information. If @var{optional} is true,
failure to execute @code{pkg-config} does @emph{not} abort
compilation.
@end deffn

@deffn {Grovel Form} cstruct lisp-name c-name slots

Define a @cffi{} foreign struct with the slot data specfied.  Slots
are of the form @code{(lisp-name c-name &key type count (signed t))}.
@end deffn

@deffn {Grovel Form} cunion lisp-name c-name slots

Identical to @code{cstruct}, but defines a @cffi{} foreign union.
@end deffn

@deffn {Grovel Form} cstruct-and-class c-name slots

Defines a @cffi{} foreign struct, as with @code{cstruct} and defines a
@acronym{CLOS} class to be used with it.  This is useful for mapping
foreign structures to application-layer code that shouldn't need to
worry about memory allocation issues.
@end deffn

@deffn {Grovel Form} cvar namespec type &key read-only

Defines a foreign variable of the specified type, even if that
variable is potentially a C preprocessor pseudo-variable.  e.g.
@code{(cvar ("errno" errno) errno-values)}, assuming that errno-values
is an enum or equivalent to type @code{:int}.

The @var{namespec} is similar to the one used in @ref{defcvar}.
@end deffn

@deffn {Grovel Form} cenum name-and-opts &rest elements

Defines a true C enum, with elements specified as @code{((lisp-name
&rest c-names) &key optional documentation)}.
@var{name-and-opts} can be either a symbol as name, or a list
@code{(name &key base-type define-constants)}. If @var{define-constants}
is non-null, a Lisp constant will be defined for each enum member.
@end deffn

@deffn {Grovel Form} constantenum name-and-opts &rest elements

Defines an enumeration of pre-processor constants, with elements
specified as @code{((lisp-name &rest c-names) &key optional
documentation)}.
@var{name-and-opts} can be either a symbol as name, or a list
@code{(name &key base-type define-constants)}. If @var{define-constants}
is non-null, a Lisp constant will be defined for each enum member.

This example defines @code{:af-inet} to represent the value held by
@code{AF_INET} or @code{PF_INET}, whichever the pre-processor finds
first.  Similarly for @code{:af-packet}, but no error will be
signalled if the platform supports neither @code{AF_PACKET} nor
@code{PF_PACKET}.
@end deffn

@lisp
(constantenum address-family
  ((:af-inet "AF_INET" "PF_INET")
   :documentation "IPv4 Protocol family")
  ((:af-local "AF_UNIX" "AF_LOCAL" "PF_UNIX" "PF_LOCAL")
   :documentation "File domain sockets")
  ((:af-inet6 "AF_INET6" "PF_INET6")
   :documentation "IPv6 Protocol family")
  ((:af-packet "AF_PACKET" "PF_PACKET")
   :documentation "Raw packet access"
   :optional t))
@end lisp

@deffn {Grovel Form} bitfield name-and-opts &rest elements

Defines a bitfield, with elements specified as @code{((lisp-name &rest
c-names) &key optional documentation)}.  @var{name-and-opts} can be either a
symbol as name, or a list @code{(name &key base-type)}.  For example:
@end deffn

@lisp
(bitfield flags-ctype
  ((:flag-a "FLAG_A")
    :documentation "DOCU_A")
  ((:flag-b "FLAG_B" "FLAG_B_ALT")
    :documentation "DOCU_B")
  ((:flag-c "FLAG_C")
    :documentation "DOCU_C"
    :optional t))
@end lisp


@c ===================================================================
@c SECTION: Groveller ASDF Integration

@node Groveller ASDF Integration, Groveller Implementation Notes, Groveller Syntax, The Groveller
@section ASDF Integration

An example software project might contain four files; an
@acronym{ASDF} file, a package definition file, an implementation
file, and a @cffi{}-Grovel specification file.

The @acronym{ASDF} file defines the system and its dependencies.
Notice the use of @code{eval-when} to ensure @cffi{}-Grovel is present
and the use of @code{(cffi-grovel:grovel-file name &key cc-flags)}
instead of @code{(:file name)}.

The @file{example-software.asd} file would look like that:

@lisp
;;; @lispcmt{CFFI-Grovel is needed for processing grovel-file components}
(defsystem "example-software"
  :defsystem-depends-on ("cffi-grovel")
  :depends-on ("cffi")
  :serial t
  :components
  ((:file "package")
   (:cffi-grovel-file "example-grovelling")
   (:cffi-wrapper-file "example-wrappers")
   (:file "example")))
@end lisp

The @file{package.lisp} file would contain one or several
@code{defpackage} forms, to remove circular dependencies and make
building the project easier.  Note that you may or may not want to
@code{:use} your internal package.

@impnote{Note that it's a not a good idea to @code{:use} when names may
clash with, say, CL symbols.
Or you could use @code{uiop:define-package} and its @code{:mix} option.}

@lisp
(defpackage #:example-internal
  (:use)
  (:nicknames #:exampleint))

(defpackage #:example-software
  (:export ...)
  (:use #:cl #:cffi #:exampleint))
@end lisp

The internal package is created by Lisp code output from the C program
written by @cffi{}-Grovel; if your specification file is
@file{exampleint.lisp}, the @file{exampleint.cffi.lisp} file will contain the
@cffi{} definitions needed by the rest of your project.
@xref{Groveller Syntax}.

@node Groveller Implementation Notes,  Wrapper for Inline/Static Functions and Macros, Groveller ASDF Integration, The Groveller
@section Implementation Notes

@cffi{}-Grovel will generate many files that not only architecture-specific,
but also implementation-specific, and should not be distributed.
ASDF will generate these files in its output cache;
if you build with multiple architectures (e.g. with NFS/AFS home
directories) or implementations, it is critical for avoiding clashes
to keep this cache in an implementation-dependent directory (as is the
default).

For @code{foo-internal.lisp}, the resulting @code{foo-internal.c},
@code{foo-internal}, and @code{foo-internal.cffi.lisp} are all
platform-specific, either because of possible reader-macros in
foo-internal.lisp, or because of varying C environments on the host
system.  For this reason, it is not helpful to distribute any of those
files; end users building @cffi{}-Grovel based software will need
@code{cffi}-Grovel anyway.

@impnote{For now, after some experimentation with @sc{clisp} having no
long-long, it seems appropriate to assert that the generated @code{.c}
files are architecture and operating-system dependent, but
lisp-implementation independent.  This way the same @code{.c} file
(and so the same @code{.grovel-tmp.lisp} file) will be shareable
between the implementations running on a given system.}

@c TODO: document the new wrapper stuff.

@node Wrapper for Inline/Static Functions and Macros,  , Groveller Implementation Notes, The Groveller
@section Wrapper for Inline/Static Functions and Macros

In a shared library, information in static/inlined functions and
macros are already removed during the compilation.  Wrapper file
enables to write an uninlined function wrapping the call to them.

A wrapper file compilation/loading proceeds as follows: 
Unlike groveller which generates C code that emits lisp files
containing cffi definitions, it generates C code, compiles it as a
shared library, loads the library, generate the cffi definitions (as
lisp code) and then loads the lisp code.

It has asdf integration similar to groveller. 

@lisp
(defsystem "example-software"
  :defsystem-depends-on ("cffi-grovel")
  :depends-on ("cffi")
  :serial t
  :components
  ((:file "package")
   (:cffi-grovel-file "example-grovelling")
   (:cffi-wrapper-file "example-wrappers")  ;; <<--- this part
   (:file "example")))
@end lisp

@deffn {Wrapper Form} defwrapper name-and-options return-type &rest args
@end deffn

@example
static inline int foo(int i) @{
  return 1+i;
@};
#define bar(i) (1+(i))
@end example

@lisp
(in-package :mypackage)
(defwrapper ("foo" foo) :int
  (i :int))
(defwrapper ("bar" bar) :int
  (i :int))
@end lisp

Other forms are similar to grovel files.

@deffn {Wrapper Form} progn &rest forms

Processes a list of forms. Useful for conditionalizing several
forms. For example:
@end deffn

@lisp
#+freebsd
(progn
  (constant (ev-enable "EV_ENABLE"))
  (constant (ev-disable "EV_DISABLE")))
@end lisp

@deffn {Wrapper Form} include &rest files

Include the specified files (specified as strings) in the generated C
source code.
@end deffn

@deffn {Wrapper Form} in-package symbol

Set the package to be used for the final Lisp output.
@end deffn

@deffn {Wrapper Form} flags &rest flags

Adds @var{cc-flags} to the command line arguments used for the C compiler
invocation.
@end deffn

@deffn {Wrapper Form} proclaim &rest proclaimations
@end deffn
@deffn {Wrapper Form} declaim &rest declaimations
@end deffn



@c ===================================================================
@c CHAPTER: Static Linking

@node Static Linking, Limitations, The Groveller, Top
@chapter Static Linking

On recent enough versions of supported implementations (currently,
GNU CLISP 2.49, CMUCL 2015-11, and SBCL 1.2.17), and with a recent
enough ASDF (3.1.2 or later), you can create a statically linked
Lisp executable image that includes all the C extensions
(wrappers and any other objects output by @code{compile-op})
as well as your Lisp code --- or a standalone application executable.
This makes it easier to deliver your code as a single file.

To dump a statically linked executable image, use:

@lisp
(asdf:load-system :cffi-grovel)
(asdf:operate :static-image-op :example-software)
@end lisp

To dump a statically linked executable standalone application, use:

@lisp
(asdf:load-system :cffi-grovel)
(asdf:operate :static-program-op :example-software)
@end lisp

See @uref{https://common-lisp.net/project/asdf/,,the ASDF
manual} for documentation about @code{image-op} and @code{program-op}
which are the parent operation classes that behave similarly except
they don't statically link C code.

@impnote{There is also an operation @code{:static-runtime-op} to create the
statically linked runtime alone, but it's admittedly not very useful
except as an intermediate step dependency towards building
@code{:static-image-op} or @code{:static-program-op}.}



@c ===================================================================
@c CHAPTER: Limitations

@node Limitations, Platform-specific features, The Groveller, Top
@chapter Limitations

These are @cffi{}'s limitations across all platforms; for information
on the warts on particular Lisp implementations, see
@ref{Implementation Support}.

@itemize @bullet
@item
The tutorial includes a treatment of the primary, intractable
limitation of @cffi{}, or any @acronym{FFI}: that the abstractions
commonly used by C are insufficiently expressive.
@xref{Tutorial-Abstraction,, Breaking the abstraction}, for more
details.

@end itemize


@node Platform-specific features, Glossary, Limitations, Top
@appendix Platform-specific features

Whenever a backend doesn't support one of @cffi{}'s features, a
specific symbol is pushed onto @code{common-lisp:*features*}.  The
meanings of these symbols follow.

@table @var
@item cffi-sys::flat-namespace
This Lisp has a flat namespace for foreign symbols meaning that you
won't be able to load two different libraries with homograph functions
and successfully differentiate them through the @code{:library}
option to @code{defcfun}, @code{defcvar}, etc@dots{}

@item cffi-sys::no-foreign-funcall
The macro @code{foreign-funcall} is @strong{not} available.  On such
platforms, the only way to call a foreign function is through
@code{defcfun}.  @xref{foreign-funcall}, and @ref{defcfun}.

@item cffi-sys::no-long-long
The C @code{long long} type is @strong{not} available as a foreign
type.

However, on such platforms @cffi{} provides its own implementation of
the @code{long long} type for all of operations in chapters
@ref{Foreign Types}, @ref{Pointers} and @ref{Variables}. The
functionality described in @ref{Functions} and @ref{Callbacks} will
not be available.

32-bit Lispworks 5.0+ is an exception. In addition to the @cffi{}
implementation described above, Lispworks itself implements the
@code{long long} type for @ref{Functions}. @ref{Callbacks} are still
missing @code{long long} support, though.

@item cffi-sys::no-stdcall
This Lisp doesn't support the @code{stdcall} calling convention.  Note
that it only makes sense to support @code{stdcall} on (32-bit) x86
platforms.

@end table


@node Glossary, Comprehensive Index, Platform-specific features, Top
@appendix Glossary

@table @dfn
@item aggregate type
A @cffi{} type for C data defined as an organization of data of simple
type; in structures and unions, which are themselves aggregate types,
they are represented by value.

@item foreign value
This has two meanings; in any context, only one makes sense.

When using type translators, the foreign value is the lower-level Lisp
value derived from the object passed to @code{translate-to-foreign}
(@pxref{translate-to-foreign}).  This value should be a Lisp number or
a pointer (satisfies @code{pointerp}), and it can be treated like any
general Lisp object; it only completes the transformation to a true
foreign value when passed through low-level code in the Lisp
implementation, such as the foreign function caller or indirect memory
addressing combined with a data move.

In other contexts, this refers to a value accessible by C, but which
may only be accessed through @cffi{} functions.  The closest you can
get to such a foreign value is through a pointer Lisp object, which
itself counts as a foreign value in only the previous sense.

@item simple type
A @cffi{} type that is ultimately represented as a builtin type;
@cffi{} only provides extra semantics for Lisp that are invisible to C
code or data.
@end table

@node Comprehensive Index,  , Glossary, Top
@unnumbered Index
@printindex cp

@bye