FIX: NOBUG_LOG env parsing error

[nobug.git] / README

NoBug Reference Manual
======================
Christian_Thäter,_Benny_Lyons

[abstract]
*****************************************************************
NoBug is a debugging library for instrumenting C and C++ programs
inspired by ideas originating from Design-by-Contract.
*****************************************************************

[quote]
__________________________________________________________________
Everyone makes mistakes, but with NoBug you won't make them twice!
__________________________________________________________________

Contents
--------

Overview
--------

The following features are provided by NoBug:

  * Three different check levels: from detailed to final no-overhead
  * Scope tags: tell whenever a function or loop is considered to be bug free
  * Precondition, Postcondition and Invariant checks and generic assertions
  * Data structures can be dumped
  * Application activities can be logged
  * Runtime customizable logging via an environment variable
  * Different logging targets to stderr, syslog, debugger, ...
  * Annotation of your sourcecode about known bugs, things to do, etc.
  * Tracking resources (files, locks, etc.) used by your program; help in
    detecting misuse
  * Detecting potential deadlocks
  * Simulate errors by injecting faults
  * Coverage checking to find out if all (instrumentated) errors are properly handled
  * Additionally, the NoBug project is used to maintain a script and
    some tools to setup testsuites

In contrast to traditional debuggers, NoBug is a non-interactive debugger that
is linked to your application doing hard-coded tests in an efficient way
without much overhead. Depending on the build level you choose, NoBug features
will be an integral part of your application making it possible to gather
debugging data for each stage of the application development, including
collecting debugging info for finally deployed applications.


.What NoBug can't do

NoBug is a (macro-)library, it is not a C/C++ language extension. This
means that code must be called at runtime to benefit from the set up
contracts. What's not tested is likely slipping through the net. Being
part of the program itself, it is affected by memory corruption.
Certain kinds of misuse may introduce new bugs (assert expressions with
side effects for example).


Support
~~~~~~~

When you need help with NoBug, have some ideas about potential features or
think you found a bug, then you can contact the NoBug community and developers
by sending mail to the NoBug mailinglist. Subscription is available at
http://lists.pipapo.org/cgi-bin/mailman/listinfo/nobug[].

Building and Installing
-----------------------

Supported Platforms
~~~~~~~~~~~~~~~~~~~

NoBug has been developed on Linux, using GCC. It should be possible to port
it to any other POSIX compliant operating system. Platform/compiler
specific things are kept optional. Currently Linux with a GCC that conforms to
C99 is supported for both 32 and 64 bit architectures. For non-GCC compilers,
some features are not available or degraded.

[frame="topbot",options="header", cols="<,<,<,<"]
|=================================================================================
|CPU    |OS          |Compiler    |State
|x86_64 |Debian      |gcc4        |supported footnote:[Reference Platform]
|x86    |other Linux |gcc4        |supported footnote:[Please report distro specific problems]
|armel  |maemo5      |gcc4        |supported footnote:[check fails in SDK (emulator bug)]
|x86*   |Mac OS X    |gcc4        |supported
|x86    |OpenSolaris |gcc4        |supported
|x86    |OpenSolaris |suncc       |mostly    footnote:[some features are not
available, needs more testing]
|       |*BSD        |            |planned   footnote:[Need volunteer for testing]
|=================================================================================

NoBug has few mandatory dependencies on other software and libraries,
some things such as valgrind support are optional and should be automatically
detected by `./configure`. Nevertheless it
requires 'pkg-config' to be installed or you'll get some weird errors at
bootstrapping (`autoreconf`) already.


Release Tarballs
~~~~~~~~~~~~~~~~

Releases are available at:
 http://www.pipapo.org/nobug-releases/[]

Gpg signed tarballs are being used for distribution. The first step involves
checking the signature:

 $ gpg nobug-VERSION.tar.gz.gpg

This will produce a nobug-VERSION.tar.gz and report if the signature could be
validated.

Since they are built with GNU Autotools, the usual build and install procedure
will work:

 $ tar xzvf nobug-VERSION.tar.gz
 $ cd nobug-VERSION
 $ mkdir -p build
 $ cd build
 $ ../configure
 $ make
 $ make check           # optional, runs the testsuite
 $ make install         # to be done as root,
                        # depending on distribution and setup


Development Version via Git
~~~~~~~~~~~~~~~~~~~~~~~~~~~

You can obtain a development version using Git.  The Git repository can be
cloned from `git://git.pipapo.org/nobug`.

Clone the Git repository with:

 $ git clone git://git.pipapo.org/nobug

After cloning the repository, then bootstrap the Autotools:

 $ cd nobug
 $ autoreconf -i                # creates the configure file

Then the usual

 $ cd build && ../configure && make && make install

(as above) will work. Careful users may run

 $ make check

to run a testsuite before installing.


Keeping Git Up To Date
^^^^^^^^^^^^^^^^^^^^^^

To update to any new revision, just enter the nobug dir and

 $ git pull

After that you can build as above

 $ cd build && ../configure && make && make install

This default pull will update from the 'master' branch which is meant to be an
on-going stable version (latest release and bugfixes).

Major new releases are assembled in the 'devel' branch, generally this is not
considered production ready.

All other branches are volatile and may be deleted or rebased anytime without
further notice.


What Is Installed
~~~~~~~~~~~~~~~~~

Currently, NoBug installs the following:

  * A single nobug.h headerfile. Include this in your code.
  * Static libraries. Statically link these to your application:
    - `libnobug.a` for singlethreaded programs.
    - `libnobugmt.a` for multithreaded programs.
  * Dynamic Libraries. Dynamically link these to your application:
    - `libnobug.so` for singlethreaded programs.
    - `libnobugmt.so` for multithreaded programs.
    - associated libtool descriptors (`libnobug*.la`)
  * Pkgconfig control files:
    - `nobug.pc` for singlethreaded programs.
    - `nobugmt.pc` for multithreaded programs.
  * The `nobug_rbdump` utility to inspect NoBug ringbuffers.


Generating This Documentation
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

There are Makefile targets for generating the documentation, either one of the
following does what you might expect:

 $ make nobug_manual.txt nobug_manual.html nobug_manual.pdf

Alternatively, you can generate all the documentation in one go as follows:

  $ make doc

Building the documentation has quite some more dependencies than building
NoBug itself. Unless you are a packager you may prefer to refer to the online
documentation or the shipped 'README' which is the complete NoBug reference
manual in text form. Generating the documentation requires: gawk, asciidoc,
graphviz and LaTeX. Check the `[header]` section of doc/latex.conf for required
packages.

Using NoBug
-----------

Your application will have to include the header file 'nobug.h' before NoBug
can be used. Prior including this, a Build-level has to be choosen,
Build-levels are discussed later, c.f., xref:buildlevel[buildlevel].
Build-levels are used to define the amount of information NoBug provides to
you. Maximum information is generally required while developing an application
and the ALPHA build-level is most apropriate during this phase; whereas the
released phase of an application will usually only require sparse information,
for which the RELEASE build-level has been conceived.

A build-level must always be specified, otherwise the compiler will complain
while attempting to compile your application.  You can specifiy a build level in
one of two ways: use a define statement in one of your modules, or pass the
build-level using the -D flag to your compiler.  Assuming we'd like to select
the ALPHA build-level in your application, then your module would assume the
following form:

 #define EBUG_ALPHA
 #include <nobug.h>

Subsequently you'll have to link the appropriate library to your application.

A number of different libraries are available to link depending on whether you
require to statically or dynamically link, or whether your application is multi
or single threaded.  The link order is important on your link line.  Link the NoBug
library 'after' your application's modules.  Here's how to statically link,
single-threaded applications:

[source,sh]
----------------
gcc -o mybinary  $(WHATEVER_FLAGS) \
       mymodules.o ... -lnobug
----------------

However, for more flexibility in selecting a build-level, you might wish to
define various targets in your makefile, one for each build-level.  In such
cases, the -D flag in your makefile is most appropriate; so your link line for
an ALPHA build with multi-threaded support would look like the following:

[source,sh]
----------------
gcc -o mybinary -DEBUG_ALPHA $(WHATEVER_FLAGS) \
       mymodules.o -lnobugmt
----------------

The NoBug libraries must be initialised before they can be used. To initialise
it call the `NOBUG_INIT` macro, which must be used before any NoBug features
can be used or any thread is created. This is discussed in more detail in the
xref:multithreading[multithreading] chapter.

So putting all this together, our application using NoBug might look something
like the following:


[source,C]
----------------
#define EBUG_ALPHA   /* If we don't use the -D Flag to cc */
#include <nobug.h>   /* Include the NoBug API */

int main()
{
        NOBUG_INIT;  /* Initialise NoBug libs */

        ...
}
----------------



Many aspects of NoBug can be configured by overriding macros before 'nobug.h' is
included.


Autotools Support
~~~~~~~~~~~~~~~~~

A project using NoBug can use autoconf to check for execinfo and
valgrind:

[source,sh]
----------------
AC_CHECK_HEADER([execinfo.h], AC_DEFINE(HAVE_EXECINFO_H))
PKG_CHECK_MODULES(VALGRIND, [valgrind],
                            [AC_DEFINE(HAVE_VALGRIND_H)])
----------------

For Multithreaded programs, you should also check for the availability of pthreads
and flavour with the `ACX_PTHREAD` maxro, see:

+http://ac-archive.sourceforge.net/ac-archive/acx_pthread.html[]+

When the resulting `HAVE_PTHREAD`, `HAVE_EXECINFO_H` and
`HAVE_VALGRIND_H` are defined by the configure script, the
relevant features become available.

`NOBUG_USE_PTHREAD`, `NOBUG_USE_VALGRIND` and `NOBUG_USE_EXECINFO` will be
defined depeding on the information gathered by configuration. If you do not
want to use any of these features in NoBug, you can override these macros by
setting them to `0` before including 'nobug.h'.

If `NVALGRIND` is defined, there will be no support for valgrind.

There are many other macros which can be set and overridden by the user to
control behavior. Your help would be appreciated in expanding this documentation
if you find some features useful; or simply contact any of the authors.


.Using Nobug from a Project using autoconf/automake

Here is a rather elaborate snippet how to put this all together into a
`configure.ac` file:

[source,sh]
----------------
# define autoheader templates for the config macros
AH_TEMPLATE(EBUG_ALPHA,
        [Define to 1 for selecting NoBug ALPHA build level])
AH_TEMPLATE(EBUG_BETA,
        [Define to 1 for selecting NoBug BETA build level])
AH_TEMPLATE(NDEBUG,
        [Define to 1 for selecting NoBug RELEASE build level])

# set configure options up
AC_ARG_ENABLE(alpha, AC_HELP_STRING([--enable-alpha],
                [select NoBug ALPHA build level]),
        nobug_level=alpha
        AC_DEFINE(EBUG_ALPHA),
[
AC_ARG_ENABLE(beta, AC_HELP_STRING([--enable-beta],
                [select NoBug BETA build level]),
        nobug_level=beta
        AC_DEFINE(EBUG_BETA),
[
AC_ARG_ENABLE(release, AC_HELP_STRING([--enable-release],
                [select NoBug RELEASE build level]),
        nobug_level=release
        AC_DEFINE(NDEBUG),

# default to ALPHA
        nobug_level=alpha
        AC_DEFINE(EBUG_ALPHA)
)])])

# check for required and optional packages
ACX_PTHREAD
AC_CHECK_HEADER([execinfo.h], AC_DEFINE(HAVE_EXECINFO_H))
PKG_CHECK_MODULES(VALGRIND, [valgrind],
        AC_DEFINE(HAVE_VALGRIND_H))

# finally check for nobug itself, multithreaded here
PKG_CHECK_MODULES(NOBUGMT_EXAMPLE, [nobugmt >= 201006.1],
        AC_DEFINE(HAVE_NOBUGMT_H),
        AC_MSG_ERROR([NoBug pkg-config metadata missing])
)
----------------

Additional Tools
~~~~~~~~~~~~~~~~

Various peripheral tools can be used by NoBug depending on the requirements
of the application and the detail desired by the user.  Such tools can provide
additional, detailed information on the application and its behaviour.
However, some applications may not require such detail and the associated
overhead in information, and users may decide to omit excess information by
excluding such tools.

At the moment NoBug supports the optional inclusion of gdb, valgrind and support
for multi-threaded applications and the information that can be provided by
these tools.  However, support for other tools may be supplied in the future,
e.g. the dbx debugger on OpenSolaris.


Link Appropriate Library
~~~~~~~~~~~~~~~~~~~~~~~~

Finally, the appropriate library (for either single or multi-threaded
applications) is linked to the project.

   *libnobug*:: Link-in this library for single threaded applications.
   *libnobugmt*:: Link with this library for multi-threaded applications.

NoBug installed static and dynamic libraries. When your application
uses multiple dynamic libraries which use NoBug or you build a dynamic
library, then you have to link against the dynamic library.

You can use the `pkg-config` tool to gather information about NoBug in
your build system. For example you can query the NoBug version which is
installed on your system:

 $ pkg-config --modversion nobug
 201006.1

Initialization
--------------

Global Initialization
~~~~~~~~~~~~~~~~~~~~~

Before anything from NoBug can be used, NoBug must be initialised. This
is archived by calling the `NOBUG_INIT()` macro before using any other of the
NoBug facilities.

`NOBUG_INIT` can be called more than once, subsequent calls will be a no-op,
thus initialising in main and in libraries won't interfere with one another.

Care must be taken when one already using NoBug features from dynamic
initialized things in C++, one has to ensure that NoBug gets initialized first
possibly by pulling up a singleton at very first.


Destroying NoBug
~~~~~~~~~~~~~~~~

Since NoBug is intended to be available throughout its whole lifetime,
destroying it is not to be advised. Nevertheless, there is a destroy function

 void nobug_destroy (void);

to shutdown NoBug, and this frees all resources associated with it.
This is mostly used in the NoBug testsuite itself to check for leaks,
and it might be useful for other programs which employ some kind of
leak checker.


Init logging Flags
~~~~~~~~~~~~~~~~~~

If you want to use environment variable controlled debuging, then you have to
initialize each defined flag with

  NOBUG_INIT_FLAG(flagname)

or

  NOBUG_INIT_FLAG_LIMIT(flagname, default)

or one of the C++ compatibility macros.

This is documented later in the xref:logconfig[logging configuration] chapter.


Threads
~~~~~~~

In Multithreaded programs you should assign an identifier to each
thread. A thread identifier is a string which will be automatically
appended with an underscore and an incrementing integer. It is is
created using the following:

  NOBUG_THREAD_ID_SET(name)

For example, calling `NOBUG_THREAD_ID_SET("worker")` will result in a thread
identifier 'worker_1'.

If you don't set an identifier, then NoBug will automatically assign
one.  This is further documented in the xref:multithreading[multi threading]
section of this manual.


.Initialization example

Thus the boilerplate code to pulling up NoBug in a multithreaded program looks
like (omit `NOBUG_THREAD_ID_SET()` in single threaded programs):
[source,C]
-------------------------------------------------------
#include <nobug.h>
NOBUG_DEFINE_FLAG(example);

...

int main()
{
    NOBUG_INIT();
    NOBUG_THREAD_ID_SET("main");
    NOBUG_INIT_FLAG(example);

    ...
}
-------------------------------------------------------

Debugging Information Granularity: The Build Levels
---------------------------------------------------

There are three different levels of debugging information available: alpha, beta
and release.  One of these levels must be specified before compiling, otherwise
an error while compiling will occur.


  *ALPHA*::
        This debugging level is envisaged for the development phase of a project
        where exhaustive testing and logging are required. All NoBug features
        are enabled.
  *BETA*::
        This debugging level is more appropriate for projects beyond the
        development phase and ready for field-testing with users willing to
        test the software. The most expensive debugging features are disabled
        at this level.
  *RELEASE*::
        This level is for final, end-users. Logging is mostly retained but
        assertions and other checking is disabled.

.Select a Build Level
A logging level can be selected by either using a define before including
`nobug.h`, or by passing the appropriate level using the -D switch
to the compiler:

    *ALPHA*::
       `-DEBUG_ALPHA` / `#define EBUG_ALPHA`

    *BETA*::
       `-DEBUG_BETA` / `#define EBUG_BETA`

    *RELEASE*::
       `-DNDEBUG` / `#define NDEBUG`

If a log level has not been selected, NoBug will abort the
compilation with an error.

Logging Concepts
----------------

Nearly all NoBug Macros emit some xref:logformat[log message]. NoBug gives
the user fine grained control over these log messages to display only
interesting information without loosing details.

Log messages can be routed to various destinations.  The following destintaions
are available:

  *RINGBUFFER*::
        The underlying storage backend. Messages are appended to the
        end of the buffer, overwriting older messages at the front of
        the buffer. NoBug comes with a highly efficient ringbuffer
        implementation. This ringbuffer is temporary by default but
        can be made persistent on disk which can be inspected with the
        'nobug_rbdump' tool.

  *CONSOLE*::
        This is either just stderr, or, if running under a supported
        debugger, the debuggers facilities to print messages will be used.

  *FILE*::
        The user can open files for log messages.

  *SYSLOG*::
        Messages are sent to the standard system logging daemon.

  *APPLICATION*::
        There are hooks which allow the programmer to catch logmessages and
        display them in an application which are defined by the application.

Each log message has a priority describing its severity in the same way as
syslog messages do.

All non-fatal messages are associated with a programmer defined flag
describing the source of the message (subsystem, module, ...). This is
referred to as 'channels' in other logging systems. xref:logflags[Flags] make
it possible to configure logging in much detail at runtime.

Putting this all together: A user can define which source/flag will be logged at
what priority level and to which destination. To make this all easier, NoBug
tries to provide reasonable defaults.

Configuration
~~~~~~~~~~~~~

.Log Levels

Each log macro has an explicit or implicit log-level which
correspondends to syslog levels. Logging is only emitted when the
message is more severe or the same as a defined limit.

INDEX Default levels for logging;loggingleveldefaults; table showing defaults log levels
[frame="topbot",options="header"]
|===============================================================================
|            |ALPHA    |BETA    |RELEASE
|ringbuffer footnote:[ringbuffer must always be most verbose] |TRACE    |INFO    |NOTICE
|console     |INFO     |NOTICE  |-1 footnote:[no log to console in release]
|file        |TRACE    |NOTICE  |WARNING
|syslog      |-1 footnote:[no syslog for test runs]      |NOTICE  |WARNING
|application |INFO     |WARNING |ERROR
|===============================================================================

Depending on the build level, there is a default logging target and a default
limit which is selected when the user doesn't specify one.

.The following default limits are

  In *ALPHA* builds::
     `NOBUG_LOG_LIMIT_ALPHA` is used which defaults to `LOG_INFO`
  In *BETA* builds::
     `NOBUG_LOG_LIMIT_BETA` is used and defaults to `LOG_WARNING`
  In *RELEASE* builds::
     `NOBUG_LOG_LIMIT_RELEASE` is used and defaults to `LOG_CRIT`

.The default targets are

  In *ALPHA* builds::
    `NOBUG_LOG_TARGET_ALPHA` defaulting to `NOBUG_TARGET_CONSOLE`
  In *BETA* builds::
     `NOBUG_LOG_TARGET_BETA` defaulting to `NOBUG_TARGET_FILE`
  In *RELEASE* builds::
     `NOBUG_LOG_TARGET_RELEASE` defaulting to `NOBUG_TARGET_SYSLOG`

You can override all these values with other limits or targets before
including 'nobug.h'. As an alternative, `NOBUG_LOG_LIMIT` and
`NOBUG_LOG_TARGET` can be defined to override all defaults at once.

Log Flags
~~~~~~~~~

Flags are used to inform NoBug about subsystems/modules or even finer
grained sections of the code. These are referred to as 'channels' in other
logging libraries.

Flags are generally used as follows:

    . Declare the flag.
    . Define the flag.
    . Initialise the flag.

To declare a flag, it is suggested to do so in a header file:

 NOBUG_DECLARE_FLAG(flagname)

The flag should then be defined most appropriately in some implementation file
by using one of the following macros:

 NOBUG_DEFINE_FLAG(flagname)

or:

 NOBUG_DEFINE_FLAG_LIMIT(flagname, limit)

Moreover, macros are available that accept a 'parent' flag as a parameter, which is then
used to initialize the defaults from another flag:

 NOBUG_DEFINE_FLAG_PARENT(flagname, parent)

or

 NOBUG_DEFINE_FLAG_PARENT_LIMIT(flagname, parent, limit)

This can be used to create hierarchies of flags.


C\++ support, C++ logflags
^^^^^^^^^^^^^^^^^^^^^^^^^^

Additional macros are available for applications written in C++:

 NOBUG_CPP_DEFINE_FLAG(name)
 NOBUG_CPP_DEFINE_FLAG_PARENT(name, parent)
 NOBUG_CPP_DEFINE_FLAG_LIMIT(name, default)
 NOBUG_CPP_DEFINE_FLAG_PARENT_LIMIT(name, parent, default)

These macros statically initialize the flags when they are defined, there is no
need to call `NOBUG_INIT_FLAG()` (see below).


Logging Flag Initialization
^^^^^^^^^^^^^^^^^^^^^^^^^^^

After a flag has been declared and defined, it has to be initialised:

 NOBUG_INIT_FLAG(flagname)

or

 NOBUG_INIT_FLAG_LIMIT(flagname, default)

Use either of these macros once at the begining your program for each flag.
This macros will parse the '$NOBUG_LOG' envirionment variable at runtime
initializing the given flag dynamically.

For flags defined with `NOBUG_DEFINE_FLAG(flagname)` the defaults are
initialized as in the xref:loggingleveldefaults[table above], while
`NOBUG_DEFINE_FLAG_LIMIT(flagname, level)` is used to initialize the default
target (depending on build level) to `level`.


Force declarations only
^^^^^^^^^^^^^^^^^^^^^^^

When `NOBUG_DECLARE_ONLY` defined to be `1` then all flag definitions here become
declarations only. When this is defined to be `0` (which is the default) then
all definitions behave as described. This can be used to construct a
headerfile which only contains definitions, but, by default, yield only
declarations. This provides one convenient single point to maintain flag
configurations.

.flags.h
[source,c]
----
#include <nobug.h>

/*
 if not included from flags.c then declare the flags,
 else define them
 */
#ifndef FLAGS_C
#define NOBUG_DECLARE_ONLY 1
#endif

/* use only DEFINE_FLAG here */
NOBUG_DEFINE_FLAG(example);

/*
 Reset it to 0 to cause no trouble
 */
#ifndef FLAGS_C
#undef NOBUG_DECLARE_ONLY
#define NOBUG_DECLARE_ONLY 0
#endif
----

.flags.c
[source,C]
----
#define FLAGS_C
#include "flags.h"
...
----

Controlling what gets logged
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The `NOBUG_INIT_FLAG...` series of macros parse the environment variable
'$NOBUG_LOG'. This enviromnet variable is thus used to configure what is
logged at runtime. Its syntax is as following:

[source,prolog]
----
logdecl_list --> logdecl, any( ',' logdecl_list).

logdecl --> flag, opt(limitdecl, any(targetdecl)).

flag --> "identifier of a flag".

limitdecl --> ':', "LIMITNAME".

targetdecl --> '@', "targetname", opt(targetopts).

targetopts --> '(', "options for target", ')', opt(targetopts).
----

Roughly speaking, '$NOBUG_LOG' contains a comma separated list of declarations
for flags, which is the name of the flag followed by a limit which is all
written in uppercase letters and preceeded by a colon, followed by target
declarations which are the names of targets, introduced by an at sign. Target
declarations can have options, which are described in the next section. Limit
and target declarations are optional and defaults are selected from the
xref:loggingleveldefaults[table above]. The defaults presented here are
currently an approximation of what might be viable. The default values used
here may be redefined in a future release.

Targets and Options
^^^^^^^^^^^^^^^^^^^

The Following options are available:

`@ringbuffer`::
  `(file=`'filename'`)`:::
    set 'filename' for the backing ringbuffer
  `(size=`'nnn'`)`:::
    set size of the ringbuffer to 'nnn' bytes, rounded-off up to the next page
    boudary
  `(append)`:::
    don't erase existing ringbuffer, start where it left off
  `(keep)`:::
    keep file after application ends
  `(temp)`:::
    unlink file instantly at creation

The default ringbuffer is a temporary file in '/tmp' with the `temp` option.
This means it will not be available and accessible for inspection, but it also
won't leave any stale data behind when the application ends.


`@console`::
  `(fd=`'n'`)`:::
    redirect console output to fd 'n'

When running under a debugger, NoBug tries to use debugger facilities
to print console messages.

`@file`::
  `(name=`'filename'`)`:::
    log to 'filename'
  `(append)`:::
    append to (existing) logfile

`@syslog`::
  `(ident=`'name'`)`:::
    global prefix for syslog
  `(cons)`:::
    log to system console if syslog is down
  `(pid)`:::
    include the process identifier in the log
  `(perror)`:::
    log to stderr as well (Not available on all systems)


.How $NOBUG_LOG is used
[source,sh]
----
# set limit to the default target and default limit
# (see table above)
NOBUG_LOG='flag,other'

# set the limit of the default target to DEBUG
NOBUG_LOG='flag:DEBUG'

# set console and syslog limits for flag to DEBUG
NOBUG_LOG='flag:DEBUG@console@syslog'

# trace 'other' to a persistent ringbuffer
NOBUG_LOG=\
 'other:TRACE@ringbuffer(file=log.rb)(size=8192)(keep)'
----
Predefined Flags
~~~~~~~~~~~~~~~~

There are some debugging flags which are predefined by NoBug.

.NOBUG_ON

The flag `NOBUG_ON` is always enabled at LOG_DEBUG level. This is
static and can not be changed.

.NOBUG_ANN

The flag `NOBUG_ANN` is used for the source annotations. This is
static and can not be changed. It differs from `NOBUG_ON` as in
never logging to syslog and only define a LOG_WARNING limit for the
application callback.

.nobug

Actions on NoBug itself will be logged under the `nobug` flag itself.
To see whats going on you can enable it with `NOBUG_LOG=nobug:TRACE`.
This is particulary useful to check if `NOBUG_INIT_FLAG()` is called on all flags.


.Using log flags (example.c)
[source,c]
----
#include <nobug.h>

NOBUG_DEFINE_FLAG (test);

int main()
{
   NOBUG_INIT;
   NOBUG_INIT_FLAG (test);

   TRACE (test, "Logging enabled");
   TRACE (NOBUG_ON, "Always on");
   return 0;
}
----

.test it:
[source,sh]
----
$ cc -DEBUG_ALPHA -lnobug example.c
$ ./a.out
0000000003: TRACE: example.c:11: -: main: Always on

$ NOBUG_LOG=test:TRACE ./a.out
0000000003: TRACE: example.c:10: -: main: Logging enabled
0000000004: TRACE: example.c:11: -: main: Always on
----

Log Line Format
~~~~~~~~~~~~~~~

Logging follows a rather rigid format and is not configurable. This is
intentional to make it easier to compare and process logs afterwards. NoBug is
designed with the goal to effectively do very fine grained logging and then
analyze this logs later, either manually or with further tools.

A typical log looks like:

 0000000003: TRACE: test.c:10: thread_1: main: message
 0000000003! TRACE: test.c:10: thread_1: main: continued
 0000000005: NOTICE: test.c:15: thread_1: main: next

The components are delimited by ": " or in one case by "! " are as following:

 . *Sequence Number* +
   Each logging event increments a counter, for preformance (and privacy)
   reasons we do not use timestamps here. The sequence number is represented by
   ten decimal digits, overflows are not specially handled. There is a special
   case that when the final message (see below) spans more log lines, then the
   delimiter after the sequence number is a "! " and not ": ". The sequence
   numbers in a log can be sparse because not every event is send to the log,
   the left out lines are still available in the ringbuffer.

 . *NoBug Facility* +
   Describes what NoBug macro produced the output. This can be a simple
   annotation or logging as well as assertions or one of the more complex
   NoBug facilities.

 . *File and Linenumber* +
   The filename is only printed without it's path. This is important for some
   reasons. First it makes the log vastly more readable and second depending
   on source location and build system and configuration thereof paths are not
   easily reproducible, this makes it hard to use simple tools like grep or
   diff on logs generated on different systems. For some messages the filename
   and/or the linenumber is not available or meaningless, a "-" is used then.

 . *Thread Identifier* +
   Shows which thread generated this message. The single threaded version of
   NoBug uses always "-".

 . *Function Name* +
   Gives the function in which the logging occured. For a lot macros it is
   possible to pass a context (file, line and function name) around to make it
   possible to delegate validation and logging to some special functions while
   still being able to present the context where the call originated. NoBug
   only uses the function name as identifier here no pretty printed prototype
   or demangled C++ method. The reason for this is that neither is well
   specified an may yield to non unique results which may even interfere with
   the logging format syntax and thus make it very unreliable to write tools
   to process logs. Some contexts dont provide a function name, a "-" is used
   then.

 . *Message* +
   The rest of the log line is the user-defined and optional message part.
   There are no constraints on the format and one should constrain himself to
   print only meaningful textual information. Embedded newlines will always be
   translated into a new logging line as shown in the example above.

Macros Overview
---------------

The NoBug interface is almost completely implemented using
preprocessor macros. This is required because NoBug uses the
+$$__FILE__$$+, +$$__LINE__$$+ and +$$__func__$$+ macros to
log information on the current file, line number and function.
Moreover, all the flat namespace uppercase identifiers make it easy
to recognise the macros in source code.

All macros are available without condition with a `NOBUG_...` prefix. Many
macros (the common cases) are also available without this prefix as a
convenience, however macros without this prefix must not have been previously
defined. When `NOBUG_DISABLE_SHORTNAMES` is defined before including
'nobug.h', then only the `NOBUG_` prefixed macros are available and the short
forms will never be defined.

A set of macros are provided by NoBug that are postfixed by `..._IF`. These
macros have the following form:

  * `..._IF(when, ...)`

They perform the desired action only if `when` is true. For example:

  * `REQUIRE_IF(foo != NULL, foo->something == constrained)`

The assertion will only be performed if `foo` is non `NULL`.

NoBug also contains a facility to pass the xref:NOBUGCONTEXT[source context]
(file, line, function) around, this can be used to write functions which
handle things where one is more interested in the context of the caller rather
than the location where the macro appears.

These macros are postfixed with `..._CTX` and take an extra context
parameter (usually at last but before the logging format specifier and
any variable argument list). The context parameter must be of type
`const struct nobug_context`.

When the `_CTX` context form is used together with the conditional `_IF`
form then the suffix of the macros is always `..._IF_CTX`.

Macros that can accept a context have no short form and must always be
prefixed with `NOBUG_...`.

Parameters types
~~~~~~~~~~~~~~~~

We use names for parameters which describe their type. These names are
orthogonal through all macro definitions.

[cols="1,5",frame="topbot"]
|==============================================================================
|`when`    |Assertion is only performed if expression `when` is true at runtime
|`expr`    |Test without side effects
|`flag`    |Destination flag for control logging at runtime
|`type`    |Data type to be checked as a single identifier name
|`pointer` |Pointer to type
|`lvl`     |Log level
|`depth`   |Depth for invariants and dumps
|`context` |Source context of type `struct nobug_context`
|`...`     |printf-like format string followed by its arguments
|==============================================================================

Source Contexts
~~~~~~~~~~~~~~~

 NOBUG_CONTEXT
 NOBUG_CONTEXT_NOFUNC

NoBug passes information about the source location of a given statement in
`const struct nobug_context` structures. These can be generated with
`NOBUG_CONTEXT` or `NOBUG_CONTEXT_NOFUNC`. The later one doesn't define a
function name and must be used when the function context is not available
like in static initialization etc..

Macros for Assertions
---------------------

The assertion set of macros provide a convenient status check on whether
to continue running, or abort execution as some condition was not fulfilled.
Assertion failures are fatal and must abort the application immediately by
calling the xref:ABORT[NOBUG_ABORT] macro which in turn may call a user defined
xref:abortcallback[abort hook].

.CHECK

 CHECK(expr, ...)
 CHECK_IF(when, expr, ...)

This assertion is never optimized out. Its main purpose is in implementing
test suites where one would like to assert tests independent of the build level.

.REQUIRE

 REQUIRE(expr, ...)
 REQUIRE_IF(when, expr, ...)
 NOBUG_REQUIRE_CTX(expr, context,...)
 NOBUG_REQUIRE_IF_CTX(when, expr, context, ...)

Precondition (input) check. Use these macros to validate the input a
function receives. The checks are enabled in *ALPHA* and *BETA* builds,
but have not effect in *RELEASE* builds.

.ENSURE

 ENSURE(expr, ...)
 ENSURE_IF(when, expr, ...)
 NOBUG_ENSURE_CTX(expr, context, ...)
 NOBUG_ENSURE_IF_CTX(when, expr, context, ...)

Postcondition (progress/output) check. Use these macros to validate the
data a function produces (example: return value). `ENSURE` is enabled
unconditionally in *ALPHA* builds and have no effect in *BETA* builds for
scopes which are tagged as `CHECKED`.

The `ENSURE_IF` variants are enabled in *ALPHA* and *BETA* builds.

In *RELEASE* builds these checks are optimized out.

.ASSERT

 ASSERT(expr, ...)
 ASSERT_IF(when, expr, ...)
 NOBUG_ASSERT_CTX(expr, context, ...)
 NOBUG_ASSERT_IF_CTX(when, expr, context, ...)

Generic check. Use these macros when you want to validate something
which doesn't fall into one of the above categories. Generally to be
avoided! The checks are enabled in *ALPHA* and *BETA* builds and
have no effect in *RELEASE* builds.

.assert

 assert(expr)

NoBug overrides the standard `assert` macro, using `NOBUG_ASSERT`.
This is just a compatibility feature, its use is not suggested.

.INVARIANT

 INVARIANT(type, pointer, depth)
 INVARIANT_IF(when,type, pointer, depth)
 INVARIANT_ASSERT(expr, ...)

Checking invariants. You can provide more complex checking functions
which test the validity of datastructures. Invariants are only enabled
in *ALPHA* builds for scopes which are not tagged as `CHECKED` and
otherwise optimized out.

 TODO: describe how to create invariant checks

Macros That Log Information
---------------------------

Logging targets a flag (except for `ECHO`) and is done at a log-level related to syslog levels.

.ECHO

 ECHO(...)

Never optimized out, logs at LOG_NOTICE level. Its main purpose is for implementing
testsuites where one would like to print and log messages independent of the build level

.ALERT

 ALERT(flag, ...)
 ALERT_IF(when, flag, ...)
 NOBUG_ALERT_CTX(flag, context, ...)
 NOBUG_ALERT_IF_CTX(when, flag, context, ...)

This is the most critical condition that might be registered by
an application. Situations might arise when the application
encounters such a serious error that can only be adequately treated
by, for example, safely shutting down the application.

.CRITICAL

 CRITICAL(flag, ...)
 CRITICAL_IF(when, flag, ...)
 NOBUG_CRITICAL_CTX(flag, context, ...)
 NOBUG_CRITICAL_IF_CTX(when, flag, context, ...)

An error which can not be handled occured but the application does not need to be
shutdowen, perhaps waiting for an operator to fix the cause.

.ERROR

 ERROR(flag, ...)
 ERROR_IF(when, flag, ...)
 NOBUG_ERROR_CTX(flag, context, ...)
 NOBUG_ERROR_IF_CTX(when, flag, context, ...)

This is used when an application registers an error
and appropriate action will have to be taken.

.WARN

 WARN(flag, ...)
 WARN_IF(when, flag, ...)
 NOBUG_WARN_CTX(flag, context, ...)
 NOBUG_WARN_IF_CTX(when, flag, context, ...)

When an application encounters a rare and unexpected
situation, these macros can be used.

.INFO

 INFO(flag, ...)
 INFO_IF(when, flag, ...)
 NOBUG_INFO_CTX(flag, context, ...)
 NOBUG_INFO_IF_CTX(when, flag, context, ...)

It may be benificial to output information at various
locations throughout the code, e.g., messages on programm
progress.

.NOTICE

 NOTICE(flag, ...)
 NOTICE_IF(when, flag, ...)
 NOBUG_NOTICE_CTX(flag, context, ...)
 NOBUG_NOTICE_IF_CTX(when, flag, context, ...)

Same as the INFO() macros, except more verbose.

.TRACE

 TRACE(flag, ...)
 TRACE_IF(when, flag, ...)
 NOBUG_TRACE_CTX(flag, context, ...)
 NOBUG_TRACE_IF_CTX(when, flag, context, ...)

The same as the NOTICE() macros, except very fine-grained information.
a common use case is to put just `TRACE(debugflag)` just at the begin of every
non-trivial function. This allows to watch fine grained application progress in the log.

NOTE: `TRACE` corresponds to `LOG_DEBUG`, because using `DEBUG` could be ambiguous.

.LOG

 NOBUG_LOG_CTX(flag, lvl, context, ...)
 NOBUG_LOG_IF_CTX(when, flag, lvl, context, ...)

Generic logging macro which takes the level explicitly.
Avoid this, unless you implement your own logging facilities.


NOTE: there is no logging macro for `LOG_EMERG`, this is only used by the assertions as a fatal message

.LOG_BASELIMIT

 NOBUG_LOG_BASELIMIT_ALPHA
 NOBUG_LOG_BASELIMIT_BETA
 NOBUG_LOG_BASELIMIT_RELEASE
 NOBUG_LOG_BASELIMIT

Anything more detailed than this base limits will be optimized out.
This is used to reduce the logging overhead for *RELEASE* builds.
By default the limit is set to `LOG_DEBUG` for *ALPHA* and *BETA*
builds, so all logging is retained and `LOG_NOTICE` in *RELEASE*
builds to log the application progress only coarsely.

These macros can be defined before including 'nobug.h' to some other
log level (as defined in 'syslog.h').

Dumping Data Structures
-----------------------

One can write functions to dump complex data structures using the NoBug
facilities. This is done by writing a custom function for each
data structure to be dumped, which may recursively call other dumping
functions. There are macros that can log within such a dumper function
and to initialise a dump of a given data structure.

A dump function has the prototype:

[source,c]
-------------------------------------------------------
void
nobug_NAME_dump (const struct NAME* POINTER,
                 const int DEPTH,
                 const struct nobug_context context,
                 void* EXTRA);
-------------------------------------------------------

where 'NAME' is the identifier for what you want to dump, 'POINTER' is a pointer
to the data to be dumped, 'DEPTH' is an integer which will be decremented when
recursing into the data structure dumper (your dump function does that, see
below) to limit the recursion depth, 'context' is a source context generated by
nobug when you call DUMP() and 'EXTRA' is a pointer transparently passed around
that you can use to store some additional state. The 'context' variable must
be named `context` because the `DUMP_LOG()` macro relies on this.

.DUMP

 DUMP(flag, type, pointer, depth, extra)
 DUMP_IF(when, flag, type, pointer, depth, extra)

These macros call a data structure dump of the object (`pointer`) in question.
`DUMP` has only effect in *ALPHA* and *BETA* builds, `DUMP_IF` is also
enabled for RELEASE builds.

`extra` is a void* which is transparently passed around and can be used to
pass a particular state around. NoBug does not alter this.

.DUMP_LOG

 DUMP_LOG(...)
 DUMP_LOG_IF(when, ...)

Any output from `DUMP` handlers should be done by these macros.

.DUMP_LEVEL

 #define NOBUG_DUMP_LEVEL ...

Dumping is by default done at `LOG_DEBUG` level, this can be overridden by
defining `NOBUG_DUMP_LEVEL` to some other logging level.

.How to use the DUMP facilities

[source,c]
-------------------------------------------------------
struct STRUCTNAME
{
  int INTEGER_MEMBER;
  char * STRING_MEMBER;
  struct STRUCTNAME* next;
}
-------------------------------------------------------

Thereafter, define a funcion as follows:

[source,c]
-------------------------------------------------------
void
nobug_STRUCTNAME_dump (const struct STRUCTNAME* self,
                       const int depth,
                       const struct nobug_context context,
                       void* extra)
{
  /* check for self != NULL and that the depth limit
     was not exceeded in recursive data structures */
  if (self && depth)
  {
    /* you may or may not do something with the extra
       parameter here, extra is transparently
       passed around */
    (void) extra;

    /* use DUMP_LOG not LOG to print the data */
    DUMP_LOG("STRUCTNAME %p: int is %d, string is %s", self,
                             self->INTEGER_MEMBER,
                             self->STRING_MEMBER);

    /* now recurse while decrementing depth */
    nobug_STRUCTNAME_dump (self->next,
                           depth-1,
                           context,
                           extra);
  }
}
-------------------------------------------------------

Now you can use the DUMP() macros within the code

[source,c]
-------------------------------------------------------
example()
{
  struct STRUCTNAME foo;
  init(&foo);

  /* extra can be anything,
     NULL is suggested when you don't use it */
  DUMP (my_flag, STRUCTNAME, &foo, 2, NULL);
}
-------------------------------------------------------

Source Code Annotations
-----------------------

Obsolete, buggy or precarious code can be marked in the code itself in
comments at the location where the code occurs.  However these code
locations can be easily overlooked, especially in large projects. NoBug
provides macros to tag code with such annotations. This provides an additional
instrument to alert or remind the programmer that there is still dubious
code embedded in the application. This tagging scheme not only informs
the programmer at compile time that there is code in the application
that needs attending to, but long after compilitation a test team
might become aware of dubious code due to runtime messages.


.DEPRECATED

 DEPRECATED(...)

Use this macro to identify code that is depreciated
and should not be used in future or productive code.

.UNIMPLEMENTED

 UNIMPLEMENTED(...)

Use this macro to identify code that is functionally
not yet complete and should not be used. This is convenient
for code under development or being reviewed.

.FIXME

 FIXME(...)

Use this macro to mark a known and unfixed bug.

.TODO

 TODO(...)

Enhancement or non-critical bug to be done soon.

.PLANNED

 PLANNED(...)

Future enhancement, optimization to similar which has no side effect on the current program.

.NOTREACHED

 NOTREACHED(...)

Code which must never be reached.

.ELSE_NOTREACHED

 ELSE_NOTREACHED(...)

This macro is the same as `else NOTREACHED()`, but completely
optimized out in release builds.


Annotation Semantics
~~~~~~~~~~~~~~~~~~~~

The action that should be taken when an annotated source line is reached
depends on the build level.

[frame="topbot",options="header"]
|================================================
|              |ALPHA |BETA         |RELEASE
|DEPRECATED    |
log footnote:[will only log once for each source line (not on each hit)] |
nothing footnote:[optimized out, irrelevant] |
won't compile footnote:[aborts compilation with an error message]

|UNIMPLEMENTED |abort footnote:[spew a log message, then abort the program] |
abort |won't compile

|FIXME         |log   |won't compile |won't compile
|TODO          |log   |log          |won't compile
|PLANNED       |log   |nothing      |nothing
|NOTREACHED    |abort |abort        |removed footnote:[optmized out, size]
|================================================

Scope Checks
------------

The programmer can tag any scope as `UNCHECKED` or `CHECKED`. In *ALPHA* and *BETA*
builds, a global `UNCHECKED` is implied. In *RELEASE* builds, `UNCHECKED` scopes are
not allowed.

[source,C]
-------
int
myfunc()
{
   /* the outer scope of this function is not validated yet*/
   UNCHECKED;

   if (...)
   {
      /* everything in this scope is considered ok */
      CHECKED;
      ...
   }
   return ...;
}
-------

.Assertions active depending on Build level and Scope

[cols="2h,3*3",frame="topbot",options="header",autowidth="something"]
|============================================================================================
|          |ALPHA                                     |BETA                          |RELEASE
|UNCHECKED |Preconditions, Postconditions, Invariants |Preconditions, Postconditions |compiling will abort
|CHECKED   |Preconditions, Postconditions             |Preconditions                 |
|============================================================================================

Fault injection
---------------

NoBug has some macros which can be used to simulate erroneous behaviour,
in other words, we can inject errors, or faults, into the code
using NoBug.

.INJECT_GOODBAD

 INJECT_GOODBAD(expr, good, bad)

Substitutes to an expression and returns 'good' when expr is false and
'bad' when expr is true. In BETA and RELEASE builds 'good' is always returned.

.INJECT_FAULT

 INJECT_FAULT(expr, bad)

Substitutes to a statement which executes 'bad'
when expr is true. This is only available in ALPHA builds and
is optimitsed out in BETA and RELEASE builds.

.Setting the logging level for fault injections
In both cases, when a fault is injected, it will be logged at
`NOBUG_INJECT_LEVEL` (default: `LOG_NOTICE`). This can be defined
before including 'nobug.h' to override it.

Fault coverage checking
-----------------------

CAUTION: Fault coverage checking is an experimental feature!

NoBug can automatically inject faults at instrumented points and permute
through all potential error paths of an application by restarting it with the
state from a former run. It can be used to provide information on whether a
particular error is adequately treated in an application. Fault coverage
checking is only available in ALPHA builds; it is optimized out in all other
builds.

NOTE: This kind of testing is very expensive on performance.

How Does Automatic Fault Injection Work?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

First, NoBug checks if the `NOBUG_COVERAGE` environment variable is set. If it
is set, it must contain a space, comma or a semicolon separated list of
filenames which are the logs from a previous run. These logs are then parsed
storing the old state in a lookup tree.

The application will then proceed to run as usual. When an instrumented
coverage point is hit, its status is checked against all states that have been
recorded previously. Any failure point that is encountered for a first time
causes a fault to be injected; the last seen but previously failed point will
now be passed, all other fault injection points behave as in their previous
run. This ensures that each successive run of the application changes only one
injection point and, thus, permutes through all possible code paths.

Fault injection points are identified by a 64bit hash over the backtrace
(return addresses on the stack) leading to it. This means each unique way to
reach an injection point is recorded. Parameters and threads are intentionally
not considered in this calculation.


How to Invoke Fault-Coverage Checking
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Each fault-injection point emits a logging message about its identity and
state, this logging uses the normal NoBug logging facilities. Thus, one can
control logging with the `NOBUG_LOG` environment variable. Furthermore, fault
coverage checking is only active when the `NOBUG_COVERAGE` environment
variable has been set to point to log files which are the results from a
previous run. NoBug comes with a `tests/coverage.sh` example script which
permutes through all possible error paths.

Coverage checking Macros
~~~~~~~~~~~~~~~~~~~~~~~~

.COVERAGE_FAULT

 COVERAGE_FAULT(flag, ...)

Injects the statement at `...` when simulating a failure. Only active in
ALPHA builds.

.COVERAGE_GOODBAD

 COVERAGE_GOODBAD(flag, good, bad)

Substitutes to an expression and injects `bad` when simulating a failure and `good`
otherwise. Only active in ALPHA builds.

.COVERAGE_LEVEL

 #define NOBUG_COVERAGE_LEVEL ...

Define the logging level in which fault-coverage logging is emitted.

NOTE: Supressing log output with this level will not supress fault injection,
      actually the opposite is true since every newly seen failure path gets injected.
      This might be changed in future releases.

.Disabling and enabling fault-coverage checks

 NOBUG_COVERAGE_DISABLE
 NOBUG_COVERAGE_ENABLE

Sometimes fault injection yields false positives, errors which may never happen in real life
(and are possibly enforced with an ENSURE afterwards). For this cases coverage fault injection can be
disabled temporarly and reenabled later. Disabling/enabling may nest and must properly match.

Resource Tracking
-----------------

With little effort, NoBug can watch all kinds of resources a program uses. This
becomes useful for resources which are distributed over a multithreaded
program. Resource tracking includes logging actions on resource and checking
locking policies over acquired resources. Resource logging is active in ALPHA
and BETA builds when NOBUG_RESOURCE_LOGGING is defined to 1 (the default).
The resource tracker which supervises locking policies is only enabled in
ALPHA builds.

Concepts
~~~~~~~~

Resources are an abstract entity for NoBug, which has little knowledge about the
kinds of resources; it only keeps records of resources and the code using
these resources and ensures basic constraints. More detailed checks on resource
usage have to be done with other NoBug facilities.

Resources are identified by an arbitrary identifier which is just a
pointer. Additionally a name, the type and the source locations which
registered the resource are stored.

Code which requiring to use a resource, calls an 'ENTER' macro, supplying
an identifier and state. The state can be altered. Thereafter a 'LEAVE' macro is
used when the the code is finished with the resources.

When a resource is used, one has to pass one of these states:

  `NOBUG_RESOURCE_WAITING`::
        For resources which might need to be blocked (locks),  enter with a
        WAITING state first, as soon at the resource is acquired, change the
        state to one of the following.
  `NOBUG_RESOURCE_EXCLUSIVE`::
        Acquired the resource exclusively. The resource must not be acquired
        again, not even from the same thread.
  `NOBUG_RESOURCE_RECURSIVE`::
        The resource might be entered multiple times from the same
        thread with this state.
  `NOBUG_RESOURCE_SHARED`::
        The resource might be entered multiple times from any thread
        with this state.

["graphviz", "resource-transitions.eps"]
---------------------------------------------------------------------
strict digraph G
{
        edge [fontname=Courier fontsize=10]

        start [shape=ellipse]

        node [shape=box]

        start -> Waiting [label="ENTER()"]
        start -> Exclusive [label="ENTER()"]
        start -> Recursive [label="ENTER()"]

        Waiting -> Exclusive [label="STATE()"]
        Waiting -> Recursive [label="STATE()"]

        Recursive -> Recursive [label="ENTER()\nSTATE()"]

        Waiting -> end [label="LEAVE()"]
        Exclusive -> end [label="LEAVE()"]
        Recursive -> end [label="LEAVE()"]

        end [shape=ellipse]
}
---------------------------------------------------------------------

Notes
~~~~~

The Resource Tracker relies on proper announce/forget and enter/leave
are properly paired. The programmer should ensure that this is correctly 
done, otherwise the results are unpredictable.

Resource tracking macros
~~~~~~~~~~~~~~~~~~~~~~~~

Unless the user defines `NOBUG_RESOURCE_LOGGING` to 0 each of the above macros
will emit a log message at `NOBUG_RESOURCE_LOG_LEVEL` which defaults to
`LOG_DEBUG`.

.RESOURCE_HANDLE

 RESOURCE_HANDLE(name)
 RESOURCE_HANDLE_INIT(name)
 RESOURCE_USER(name)
 RESOURCE_USER_INIT(name)

Define and initialize handles for to track resources.

 `name`::
     identifer to be used for the handle

There are two kinds of handles. Resource themself are abstracted with a
`RESOURCE_HANDLE` while uses of a resources are tracked by `RESOURCE_USER`
handle. These macros takes care that the declaration is optimized out
in the same manner as the rest of the resource tracker would be disabled.
You can still instantiate handles as `struct nobug_resource_record*` or
`struct nobug_resource_user*` in structures which must have a constant size
unconditional of the build level. The two `*_INIT` macros can be used to initialize
resource handles and are optimized out when the resource tracker gets disabled.

.RESOURCE_ANNOUNCE

 RESOURCE_ANNOUNCE(flag, type, name, identifier, handle){}
 NOBUG_RESOURCE_ANNOUNCE_RAW(flagptr, type, name, ptr, handle){}
 NOBUG_RESOURCE_ANNOUNCE_RAW_CTX(flagptr, type, name,
                                 ptr, handle, context){}

Publishes resources.

 `flag`::
     the NoBug flag name which turns logging on for this macro
 `type`::
     a string which should denote the domain of the resource,
     examples are "file", "mutex", "lock", "database" and so on
 `name`::
     the actual name of a named resource this as string. `type` and
     `name` must be available through the entire lifetime of the resource, using
     literal strings is recommended
 `identifier`::
     a pointer which must be unique for this resource, any
     kind of pointer will suffice, it is only used for identification. In
     multithreaded applications the thread identifier becomes an additional
     identifier
 `handle`::
     a `NOBUG_RESOURCE_HANDLE` which will be initialized to point to
     the newly created resource.

Resources must be unique, it is a fatal error when a resource it tried to be
announced more than one time.

`RESOURCE_ANNOUNCE()` acts like the head of a C loop statement, it ties to the following
(block-) statement which will be handled atomic.
This statement must not be left by break, return or any other kind of jump.

.RESOURCE_FORGET

 RESOURCE_FORGET(flag, handle){}
 NOBUG_RESOURCE_FORGET_RAW(flagptr, handle){}
 NOBUG_RESOURCE_FORGET_RAW_CTX(flagptr, handle, context){}

Removes resources that have become unavailable from the registry.

`flag`::
    the NoBug flag which turns logging on for this macro
`handle`::
    the `NOBUG_RESOURCE_HANDLE` used to track this resource

The resource must still exist and no users must be attached to it, else a fatal
error is raised.

`RESOURCE_FORGET()` acts like the head of a C loop statement, it ties to the following
(block-) statement which will be handled atomic.
This statement must not be left by break, return or any other kind of jump.

.RESOURCE_RESETALL

 RESOURCE_RESETALL(flag)
 NOBUG_RESOURCE_RESETALL_RAW(flagptr)
 NOBUG_RESOURCE_RESETALL_RAW_CTX(flagptr, context)

Sometimes the resource tracker can give false positives when it finds a locking order violation
while the programmer knows that this will never happen in the real program, because for example
this is only used at initialization or shutdown and never overlaps. This macro can then be used
to reset all whats learnt about all resources and start over.

`flag`::
    the NoBug flag which turns logging on for this macro

.RESOURCE_RESET

 RESOURCE_RESET(flag, handle)
 NOBUG_RESOURCE_RESET_RAW(flagptr, handle)
 NOBUG_RESOURCE_RESET_RAW_CTX(flagptr, handle, context)

Sometimes the resource tracker can give false positives when it finds a locking order violation
while the programmer knows that this will never happen in the real program, because for example
this is only used at initialization or shutdown and never overlaps. This macro can then be used
to reset all whats learnt about a single resources and start over.

`flag`::
    the NoBug flag which turns logging on for this macro
`handle`::
    the `NOBUG_RESOURCE_HANDLE` used to track this resource

.RESOURCE_ENTER

 RESOURCE_ENTER(flag, announced, user, state, handle){}
 NOBUG_RESOURCE_ENTER_CTX(flag, resource, user, state,
                          handle, context){}

Acquire a resource.

`flag`::
    nobug flag which turns logging on for this macro
`announced`::
    the handle set by `RESOURCE_ANNOUNCE`
`user`::
    a literal string defining the purpose
`state`::
    the state to enter
`handle`::
    a `NOBUG_RESOURCE_HANDLE` which will be initialized to the
    entering node

`RESOURCE_ENTER()` acts like the head of a C loop statement, it ties to the following
(block-) statement which will be handled atomic.
This statement must not be left by break, return or any other kind of jump.

.RESOURCE_WAIT

 RESOURCE_WAIT(flag, resource, user, handle){}
 NOBUG_RESOURCE_WAIT_CTX(flag, resource, user, handle, context){}

This is just an alias for

 RESOURCE_ENTER(flag, resource, user,
                NOBUG_RESOURCE_WAITING, handle)

.How to use it
[source,c]
----
RESOURCE_WAIT(flag, resource, user, handle);
if (lock_my_resource() == ERROR)
  NOBUG_RESOURCE_LEAVE(flag, handle);
else
  RESOURCE_STATE(flag, NOBUG_RESOURCE_EXCLUSIVE, handle);
----

.RESOURCE_TRY

 RESOURCE_TRY(flag, resource, user, handle){}
 NOBUG_RESOURCE_TRY_CTX(flag, resource, user, handle, context){}

This is just an alias for

 RESOURCE_ENTER(flag, resource, user,
                NOBUG_RESOURCE_TRYING, handle)

Trying on a resource is similar to waiting but will not trigger a deadlock check. This can be used
when a deadlock is expected at runtime and one handles this otherwise (by a timed wait or something like that).

.RESOURCE_STATE

 RESOURCE_STATE(flag, entered, state){}
 NOBUG_RESOURCE_STATE_CTX(flag, state, entered, context){}
 NOBUG_RESOURCE_STATE_RAW(flagptr, state, entered){}
 NOBUG_RESOURCE_STATE_RAW_CTX(flagptr, state, entered, context){}

Changes resource's state.

`flag`::
    is nobug flag which turns logging on for this macro
`state`::
    the new state Note that only certain state transitions are
    allowed, see discussion/diagram above
`entered`::
    the handle set by `RESOURCE_ENTER`

`RESOURCE_STATE()` acts like the head of a C loop statement, it ties to the following
(block-) statement which will be handled atomic.
This statement must not be left by break, return or any other kind of jump.

.RESOURCE_LEAVE

 RESOURCE_LEAVE(flag, handle){}
 NOBUG_RESOURCE_LEAVE_RAW(flagptr, handle){}
 NOBUG_RESOURCE_LEAVE_RAW_CTX(flagptr, handle, context){}

Disconnect from a resource identified with its handle.

`flag`::
    nobug flag which turns logging on for this macro
`handle`::
    the handle you got while entering the resource

`RESOURCE_LEAVE()` acts like the head of a C loop statement, it ties to the following
(block-) statement which will be handled atomic.
This statement must not be left by break, return or any other kind of jump.

.How to use it
[source,c]
----
NOBUG_RESOURCE_LEAVE(flag, handle)
  {
    unlock_my_resource();
  }
----

.RESOURCE_ASSERT_STATE

 RESOURCE_ASSERT_STATE(resource, state)
 RESOURCE_ASSERT_STATE_IF(when, resource, state)
 NOBUG_RESOURCE_ASSERT_STATE_CTX(resource, state, context)
 NOBUG_RESOURCE_ASSERT_STATE_IF_CTX(when, resource,
                                    state, context)

Assert that we have a resource in a given state. For multithreaded programms the topmost
state of the calling thread is checked, for non threadeded programs the most recent state on
resource is used.

`when`::
    Condition which must be true for testing the assertion
`resource`::
    Resource handle
`state`::
    The expected state

.RESOURCE_DUMP

 NOBUG_RESOURCE_DUMP(flag, handle)
 NOBUG_RESOURCE_DUMP_IF(when, flag, handle)

Dump the state of a single resource.

`when`::
    Condition which must be true to dump the resource
`flag`::
    Nobug flag for the log channel
`handle`::
    handle of the resource to be dumped

.RESOURCE_DUMPALL

 NOBUG_RESOURCE_DUMPALL(flag)
 NOBUG_RESOURCE_DUMPALL_IF(when, flag)

Dump the state of all resources.

`when`::
    Condition which must be true to dump the resources
`flag`::
    Nobug flag for the log channel

.RESOURCE_LIST

 NOBUG_RESOURCE_LIST(flag)
 NOBUG_RESOURCE_LIST_IF(when, flag)

List all registered resources.

`when`::
    Condition which must be true to list the resources
`flag`::
    Nobug flag for the log channel

.How to use the Resourcetracker
[source,c]
----
NOBUG_DEFINE_FLAG_LIMIT(test, LOG_DEBUG);

void example()
{
  /* define a mutex and announce it */
  pthread_mutex_t my_mutex = PTHREAD_MUTEX_INITIALIZER;
  RESOURCE_HANDLE(resource);

  /* 'example' is just a pointer to this function
     which suffices as unique identifier */
  RESOURCE_ANNOUNCE(test, "mutex", "my_mutex",
                    example, resource);

  /* the following would be done in a
     different thread in a real program */

  /* define a handle */
  RESOURCE_HANDLE(enter);

  /* announce that we want to use the resource
     &enter also suffices as unique pointer,
     which is all we need as identifer here */
  RESOURCE_WAIT(flag, resource, &enter, enter)
    {
      /* lock() might block */
      pthread_mutex_lock (&my_mutex);
      /* assume no errors, got it, change the state */
      RESOURCE_STATE(test, NOBUG_RESOURCE_EXCLUSIVE, enter);
    }

  /***************************************
  ** program does something useful here **
  ***************************************/

  /* we don't need it anymore */
  RESOURCE_LEAVE(test, enter)  /* << no semicolon! */
    pthread_mutex_unlock (&my_mutex);

  /* back in the main thread */

  /* remove the resource from the public registry */
  RESOURCE_FORGET(test, resource);
}
----

Deadlock Detection
~~~~~~~~~~~~~~~~~~

The Resource Tracker is able to detect potential deadlocks. This is done by
learning the relations between locks (precedence) and watching the order in
which resources are acquired. It has some heuristics to detect certain
patterns which are deadlock free. A possible deadlock results in a log message
and a fatal abort. Note that only waiting on resources can lead to a deadlock.
Deadlock detection is implemented in the Resource Tracker and active in ALPHA
builds and optimized out on any other build level.

For details about the deadlock detection algorithm see
xref:deadlockdetection[Appendix: Resource Tracking Alorithm].

Callbacks
---------

NoBug provides callbacks, applications can use these
to present logging information in some custom way or hook some special processing in.
The callbacks are initialized to NULL and never modified by NoBug, it is the sole responsibility
of the user to manage them.

CAUTION: There are certain constraints what and what not can be done in callbacks
         documented below which must be followed.

.logging callback prototype

 typedef void (*nobug_logging_cb)(const struct nobug_flag* flag,
                                  int priority,
                                  const char *log,
                                  void* data)

 `flag`::
    Flag structure which defines the logging configuration for this event
 `priority`::
    Log level of the current event
 `log`::
    Pointing to the current log line in the ringbuffer or `NULL`
 `data`::
    Global pointer defined by the user, passed arround (see below)

.abort callback prototype

 typedef void (*nobug_abort_cb)(void* data)

 `data`::
    Global data defined by the user, passed arround (see below)

.passing data to callbacks

 void* nobug_callback_data

This global variable is initialized to `NULL` and will never be touched by NoBug. One can use it
to pass extra data to the callback functions.

.callback when logging

 nobug_logging_cb nobug_logging_callback

This callback gets called when something gets logged.
NoBug will still hold its mutexes when calling this hook, calling NoBug logging or resource tracking
functions from here recursively will deadlock and must be avoided.
The `log` parameter points to the logging message in the ringbuffer.
Unlike other logging targets it is not automatically limited to the log level configured
in the flag but called unconditionally. The callback should implement its own limiting.

When one wants to do complex calls which may include recursion into logging and resource tracking
functions, the intended way is to pass contextual information possibly including a __copy__ of the
`log` parameter in xref:THREADDATA[NOBUG_THREAD_DATA] to the postlogging callback (see below).
Other internal NoBug facilties, like the ringbuffer etc, are protected by the mutexes and may be accessed
from this function.

.callback after logging

 nobug_logging_cb nobug_postlogging_callback

This callback gets called after something got logged. The `log` parameter is always NULL and all
NoBug mutexes are released. This means that this function may call any complex things, including
calling logging and resource tracking, but may not call internal NoBug facilities.
Contextual created in the `nobug_logging_callback` and stored in xref:THREADDATA[NOBUG_THREAD_DATA] can be
retrieved here and may need to be cleaned up here.

.callback for aborting

 nobug_abort_cb nobug_abort_callback

This callback gets called when the application shall be terminated due an error.
It can be used to hook exceptions or similar things in. When it returns, `abort()`
is called.

IMPORTANT: Errors detected by NoBug are always fatal. If one handles and possible
           throws an exception here, the application must shut down as soon as possible.
           Most causes for aborts are optimitzed out in `RELEASE` builds.

Tool Macros
-----------

.NOBUG_FLAG_RAW

 NOBUG_FLAG_RAW(ptr)

Using this macro one can pass a direct pointer to a flag where a name would
be expected. This is sometimes convinient when flag pointers are passed around
in management strutures and one wants to tie logging to dynamic targets.

[source,c]
----
NOBUG_DEFINE_FLAG(myflag);
...
struct nobug_flag* ptr = &NOBUG_FLAG(myflag);
TRACE(NOBUG_FLAG_RAW(ptr), "Passed flag by pointer")
----

.Backtraces

 BACKTRACE
 NOBUG_BACKTRACE_CTX(context)

The backtrace macro logs a stacktrace using the NoBug facilities.
This is automatically called when NoBug finds an error and is due
to abort. But one might call it manually too.

.Aborting

 NOBUG_ABORT_

This is the default implementation for aborting the program, it first syncs all ringbuffers to disk, then
calls the abort callback if defined and then `abort()`.

 NOBUG_ABORT

If not overridden, evaluates to `NOBUG_ABORT_`. One can override this before including
`nobug.h` to customize abortion behaviour. This will be local to the translation unit then.

.NOBUG_ALPHA_COMMA

 NOBUG_ALPHA_COMMA(something)
 NOBUG_ALPHA_COMMA_NULL

Sometimes it is useful to have initializer code only in *ALPHA* builds, for example when you
conditionally include resource handles only in *ALPHA* versions. An initializer can then
use this macros to append a comma and something else only in *ALPHA* builds as in:

[source,C]
----
struct foo = {"foo", "bar" NOBUG_ALPHA_COMMA_NULL };
----

Becomes the following in *ALPHA* builds

[source,C]
----
struct foo = {"foo", "bar", NULL};
----

and

[source,C]
----
struct foo = {"foo", "bar"};
----

in *BETA* and *RELEASE* builds.

.NOBUG_IF_*

 NOBUG_IF_ALPHA(...)
 NOBUG_IF_NOT_ALPHA(...)
 NOBUG_IF_BETA(...)
 NOBUG_IF_NOT_BETA(...)
 NOBUG_IF_RELEASE(...)
 NOBUG_IF_NOT_RELEASE(...)

This macros allow one to conditionally include the code in '(...)' only if the
criteria on the build level is met. If not, nothing gets substituted. Mostly used
internally, but can also be used for custom things.

Multithreading
--------------

It is important that NoBug protects certain operations with locks in
multithreaded programs. You have to ensure that 'HAVE_PTHREAD_H' is defined by
the configuration system and use the 'libnobugmt' library for linking. It is
particular important that libraries using NoBug are compiled with
'HAVE_PTHREAD_H' enabled when they are intended to be used in multithreaded
programs.

When Multithreading is used, log messages contain a identifier of the
originating thread. This identifier should be set by

.NOBUG_THREAD_ID_SET

 NOBUG_THREAD_ID_SET(name)

`name`::
        New name for the thread

Nobug will assemble a unique identifier by appending a underscore and a
number to name, for example `NOBUG_THREAD_ID_SET("gui")` will result in a
identifier like 'gui_5'. When you don't set a thread identifier, then NoBug
assigns one automatically with the name 'thread' preprended if needed. Thread
identifiers may be reset with a new call to this macro.

.NOBUG_THREAD_ID_GET

 NOBUG_THREAD_ID_GET

Will return a const char* of the thread id in multithreaded programs and
a pointer to a literal empty string in singlethreaded programs.

.NOBUG_THREAD_DATA

 NOBUG_THREAD_DATA

Evaluates to a variable of type `void*` which can be used to store
thread local information. This is useable for xref:callbacks[callbacks] which may
prepare context information to be reused later.

This macro is also available in singlethreaded programs, refering to a
single global variable.

Nobug initializes this variable to `NULL` and then touches it never again.

Dumping Persistent Ringbuffers
------------------------------

NoBug installs the `nobug_rbdump` tool for dumping the content of a persistent
ringbuffer. It is invoked with the filename of the ringbuffer, the content is then
printed to stdout.

 $ NOBUG_LOG='test:TRACE@ringbuffer(file=test.rb)(keep)' ./a.out
 0000000004: TRACE: example.c:11: -: main: Always on
 $ nobug_rbdump test.rb
 0000000003: TRACE: example.c:10: -: main: Logging
 0000000003! TRACE: example.c:10: -: main:  enabled

Best Practices
--------------

NOTE: this section is very work in progress

.Workflow

 1. Development
      * Write a testsuite, build your program with -O0 -g -DEBUG_ALPHA and run
        the testsuite under valgrind control. Hack until the program meets the
        specifications defined by the testsuite.
 2. Beta Test
      * Build with desired optimization level and -g -DEBUG_BETA and give the
        program to your beta testers.
 3. Release
      * Build it with optimization and without -g -DEBUG_*

.What and when to check

  * Add REQUIRE checks on your interfaces (incoming parameters). Especially if
    an argument might not cover the whole range of the underlying type.
  * Don't waste your and your CPU's time with unnecessary checks. The testsuite
    should validate your program. NoBug aids in debugging. You can add
    Postconditions (ENSURE) and Invariants when you have a bug somewhere and
    want to nail it down.
  * Added checks don't need to be removed.
  * When you use the CHECKED/UNCHECKED features then don't forget C scoping
    rules, tag things as CHECKED from the leaves to the root.

.Tips & Tricks

  * TRACE(flagname) at the begin of every nontrivial function will easily log
    the progress of your application.
  * Trying a RELEASE build will abort on certain conditions (known BUG, TODO's,
    UNCHECKED code), you can use this to find these spots.

Appendix
--------

The Resource Tracking Algorithm
-------------------------------

Each resource registers a global 'resource_record'.

Every new locking path discovered is stored as 'resource_node' structures which refer to the associated
'resource_record'.

Threads keep a trail of 'resource_user' structures for each resource entered. This 'resource_user' struct
refer to the 'resource_nodes' and thus indirectly to the associated 'resource_record'.

The deadlock checker uses this information to test if the acqusition of a new resource would yield a
potential deadlock.

Entering Resources
~~~~~~~~~~~~~~~~~~

In multithreaded programs, whenever a thread wants to wait for a 'resource_record'
the deadlock checker jumps in.

The deadlock checking algorithm is anticipatory as it will find and abort on conditions which may lead
to a potential deadlock by violating the locking order learned earlier.

Each thread holds a stack (list) of each 'resource_user' it created. Leaving
a resource will remove it from this stacklist.

Each 'resource_record' stores the trail which other 'resource_records' are already entered. This relations
are implemented with the 'resource_node' helper structure.

////
TODO: insert diagram here
  2-3
1
  3-4-2

1-3-2-4

3-4-2

1-4-2

////

First we find out if there is already a node from the to be acquired resource back to
the topmost node of the current threads user stack.

[source,c]
---------------------------------------------------------------------
struct nobug_resource_user* user = NULL;
struct nobug_resource_node* node = NULL;

if (!llist_is_empty (&tls->res_stack))
 {
  user = LLIST_TO_STRUCTP (llist_tail (&tls->res_stack),
                           struct nobug_resource_user,
                           res_stack);

  struct nobug_resource_node templ =
   {
    ...
    user->current->resource,
    ...
   };

  node = (struct nobug_resource_node*)
   llist_ufind (&resource->nodes,
                &templ.node,
                nobug_resource_node_resource_cmpfn,
                NULL);
 }
...
---------------------------------------------------------------------

Deadlock checking is only done when the node is entered in `WAITING` state and only
available in multithreaded programs.

[source,c]
---------------------------------------------------------------------
if (state == NOBUG_RESOURCE_WAITING)
 {
#if NOBUG_USE_PTHREAD
  ...
---------------------------------------------------------------------

If node was found above, then this locking path is already validated and no deadlock can happen,
else, if this stack already holds a resource (user is set) we have to go on with checking.

[source,c]
---------------------------------------------------------------------
if (!node && user)
 {
  ...
---------------------------------------------------------------------

If not then its checked that the resource to be entered is not on any parent trail of the current topmost resource,
if it is then this could be a deadlock which needs to be further investigated.

[source,c]
---------------------------------------------------------------------
LLIST_FOREACH (&user->current->resource->nodes, n)
 {
  for (struct nobug_resource_node* itr =
         ((struct nobug_resource_node*)n)->parent;
       itr;
       itr = itr->parent)
   {
    if (itr->resource == resource)
     {
       ...
---------------------------------------------------------------------

if the resource was on the trail, we search if there is a common ancestor before the resource
on the trail and the threads current chain,
if yes then this ancestor protects against deadlocks and we can continue.

[source,c]
---------------------------------------------------------------------
for (struct nobug_resource_node* itr2 = itr->parent;
     itr2;
     itr2 = itr2->parent)
 {
  LLIST_FOREACH_REV (&tls->res_stack, p)
   {
    struct nobug_resource_user* user =
     LLIST_TO_STRUCTP (p,
                       struct nobug_resource_user,
                       res_stack);
    if (user->current->resource == itr2->resource)
     goto done;
   }
  ...
---------------------------------------------------------------------

If no ancestor found, we finally abort with a potential deadlock condition.

[source,c]
---------------------------------------------------------------------
nobug_resource_error = "possible deadlock detected";
return NULL;
---------------------------------------------------------------------


Leaving Resources
~~~~~~~~~~~~~~~~~

store the tail and next aside, we need it later

[source,c]
---------------------------------------------------------------------
#if NOBUG_USE_PTHREAD
struct nobug_resource_user* tail =
 LLIST_TO_STRUCTP (llist_tail (&user->thread->res_stack),
                   struct nobug_resource_user,
                   res_stack);
struct nobug_resource_user* next =
 LLIST_TO_STRUCTP (llist_next (&user->res_stack),
                   struct nobug_resource_user,
                   res_stack);
...
---------------------------------------------------------------------

remove user struct from thread stack
The res_stack is now like it is supposed to look like with the 'user' removed.
We now need to fix the node tree up to match this list.

[source,c]
---------------------------------------------------------------------
llist_unlink_fast_ (&user->res_stack);
...
---------------------------------------------------------------------

When the the user node was not the tail or only node of the thread stack, we have to check
(and possibly construct) a new node chain for it. No valdation of this chain needs to be done,
since it was already validated when entering the resources first.

[source,c]
---------------------------------------------------------------------
if (user != tail && !llist_is_empty (&user->thread->res_stack))
 {
  struct nobug_resource_user* parent = NULL;
  if (llist_head (&user->thread->res_stack)!= &next->res_stack)
   {
    parent =
     LLIST_TO_STRUCTP (llist_prev (&next->res_stack),
                       struct nobug_resource_user,
                       res_stack);
   }
  ...
---------------------------------------------------------------------

iterate over all users following the removed node, finding nodes pointing to this users or
create new nodes.

[source,c]
---------------------------------------------------------------------
LLIST_FORRANGE (&next->res_stack, &user->thread->res_stack, n)
 {
  struct nobug_resource_user* cur =
   LLIST_TO_STRUCTP (n,
                     struct nobug_resource_user,
                     res_stack);

  struct nobug_resource_record* resource =
   cur->current->resource;
  ...
---------------------------------------------------------------------

find the node pointing back to parent, create a new one if not found, rinse repeat

[source,c]
---------------------------------------------------------------------
struct nobug_resource_node templ =
 {
  ...
  parent?parent->current:NULL,
  ...
 };

struct nobug_resource_node* node =
 (struct nobug_resource_node*)
 llist_ufind (&resource->nodes,
              &templ.node,
              nobug_resource_node_parent_cmpfn,
              NULL);

if (!node)
 {
  node = nobug_resource_node_new (resource,
                                  parent?parent->current:NULL);
  if (!node)
   {
    nobug_resource_error = "internal allocation error";
    return 0;
   }
 }

parent = cur;
---------------------------------------------------------------------


Reference Index
---------------

[dotted]

License
-------

 NoBug
 Copyright (C)
 2006, 2007, 2008, 2009, 2010   Christian Thäter <ct@pipapo.org>

 This program is free software; you can redistribute it and/or
 modify it under the terms of the GNU General Public License
 as published by the Free Software Foundation; either version 2
 of the License, or (at your option) any later version.

 This program is distributed in the hope that it will be useful,
 but WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
 GNU General Public License for more details.

 You should have received a copy of the GNU General Public
 License along with this program; if not, write to the Free
 Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
 Boston, MA 02110-1301 USA.

.License Rationale

NoBug is released under the "GNU General Public License version 2 or
any later" to protect its freedom. If one wants to use NoBug in a
propietary program, please contact the main author for
acknowledging relicensing terms.

For BSD license style Free Software, this means you can not distribute
binaries linking NoBug without making its source available. To make
this compatible, it is suggested that you dual-license your software
with your prefered BSD like license and the GPL. As long as it uses
NoBug, the GPL will take over and you have to make the source
available, while one can ship a BSD or LGPL Licensed headerfile which
defines all NoBug macros as empty macros and remove libnobug from the
linking, then NoBug isn't used anymore and you may apply BSD license
terms for resulting binaries.


.Contributor Agreement

Improvements and patches must be licensed as "GPL v2 or any later" to
be acceptable. Further a contributor must either assign his copyright
to the main NoBug author or agree with the possibility that NoBug can
be relicensed for propietary use:

 Independent of the GPL license as stated above, The main author
 of NoBug explicitly reserve the right to relicense NoBug under
 different, even propietary terms. Any contributor agrees to
 such a possiblility by sending his contribution to be included
 into the official releases.

 This agreement is bilateral, every contributor who worked on a
 substantial part of NoBug has the right to relicense it after
 negotiation with the NoBug main author. Exact terms of such
 relicensing are worked out on a per case base.

The intention is that anyone who worked on NoBug should be able to
benefit from his work. This means one should be able to use it at his
workplace, to gain a job or as well as relicense it for a customer.
Unlike other projects which simply ask for transfering the copyright
to the main author, NoBug tries to make it possible to retain the
copyright by anyone who helped the project.

This additional agreement has no impact on the GPL, it's sole purpose
is to define relicensing policies between the NoBug main author and
contributors. When you recieve NoBug it will be licensed under
GPL unless you personally acknowledged other terms with the NoBug main
author (or any other main contributor).

If anyone feels he is not credited in the 'AUTHORS' file or in any
copyright notice, please contact the main author for inclusion.

Credits
-------

Design, implementation::
 Christian Thaeter $$<ct@pipapo.org>$$ +

Testsuite::
 Christian Thaeter $$<ct@pipapo.org>$$ +

Documentation::
 Christian Thaeter $$<ct@pipapo.org>$$ +
 Benny Lyons $$<benny.lyons@uniserv.com>$$ +
 François KUBLER $$<ih8tehuman@free.fr>$$ +
 Andreas Hermann Braml $$<andreas@braml.org>$$ +

Autoconf build system improvements::
 Peter Simons $$<simons@cryp.to>$$ +
 Luca Barbato $$<lu_zero@gentoo.org>$$ +
 Michael Ploujnikov $$<ploujj@gmail.com>$$ +

Ringbuffer dump tool::
 Simeon Voelkel $$<simeon_voelkel@arcor.de>$$ +

Logo::
 Pablo Lizardo $$<pablo_lizardo@yahoo.com.ar>$$ +