Building a Lua extension library for Linux in C/C++

Lua is an embeddable scripting language, widely used by application designers to provide ways in which the application can be extended at runtime. Although there are many things that can be done in plain Lua, there are some important ones that can't. For example, there is notoriously no built-in function in Lua for providing timed delays of less than one second. In such cases, you can create an extension module in C or C++.

Lua being what it is, there various ways to provide extensions. Most obviously, you can just add the required functionality to the Lua source, and build it. Lua is easy and quick to build on Linux. If you're embedding Lua in some other application, you're probably building it from source anyway. However, an elegant method that doesn't (necessarily) involve modifying the Lua code is to create an extension as a shared object (.so file), and load it into Lua. The method isn't conceptually difficult, but it can be fiddly because of the various different ways that Lua is built and distributed.

Note:
This article applies to Lua 5.3, and will need at least slight modifications for other versions

I should point out that I'm not going to explain how to use the Lua C API. This is well documented in Programming in Lua, and the basic API hasn't changed all that much since the book was last updated.

The example I'm demonstrating is an extension to add Raspberry Pi-specific functions to Lua, although there are many other applications. I won't be presenting all the code (it will appear on GitHib in due course), just enough of an outline to understand how it is structured.

Basic principles

Lua can load Lua source or compiled Lua bytecode from a file; it can also load compiled C/C++ code from a shared object (or a DLL in Windows). The Lua function require() can be used for all these extension types, although there are other methods that can be used.

require() uses a pattern-based search path for files. For C/C++ libraries, the path is set in the global variable package.cpath. As with everything in Lua, this path will depend on how Lua was built, and on certain environment variables. The pattern will typically be something like this:

/usr/local/lib/lua/5.3/?.so;/usr/local/lib/lua/5.3/loadall.so;./?.so

By default, it won't look in any of the standard system locations although you can, of course, change the value of package.cpath. Notice that the current directory is included, and that is the directory that I will use in this example.

When (if) the library is found, the Lua loader looks for, and executes, a function called luaopen_xxx in the library. where the xxx matches the module name. Since I am creating a library called luapi, In need to provide the function luaopen_luapi(). This function will pack C function pointers into a Lua table, along with the function names, as I will demonstrate later.

The code

The following C code is complete and working example as presented although in practice there will probably be a lot more functions. There's nothing particularly difficult or extraordinary about structuring the C code -- the module might fit into one C source, or it might need many: organization is the same as for any other C library. I'm assuming that my module fits into one C source, and it will be called luapi.c.

#include <unistd.h>
#include "lua.h"
#include "lauxlib.h"

int luapi_wait_msec (lua_State *L)
  {
  int msec = (int) luaL_checknumber (L, -1);
  lua_pop (L, 1);
  usleep (msec * 1000);
  return 0;
  }

/* ... more functions ... */


int luaopen_luapi (lua_State *L)
  {
  lua_newtable(L);
  lua_pushcfunction (L, luapi_wait_msec);
  lua_setfield (L, -2, "wait_msec");
  /* ... more push operations ... */
  return 1;
  }

The one non-boilerplate function shown -- called wait_msec in Lua, and implemented as luapi_wait_msec in C -- pops the Lua argument off the stack, and makes a call to usleep. There's no particular naming rules for the C functions, because the mapping between Lua names and C functions is made in the luaopen_luapi() method.

The protocol for pushing the table onto the Lua stack is a bit fiddly, but it's boilerplate code -- it will always be the same. The lua_newtable() method pushes an empty table onto the stack. lua_pushcfunction() pushes (essentially) the function pointer. Then lua_setfield sets its argument ("wait_sec") and the object at the top of the stack (the C function) into the table whose position is -2 -- that is, second from the top of the stack. This process removes the C function from the stack, and just leaves the table. So further functions are added to the table using the same function calls and indexes.

Building the shared library

First, compile luapi.c to an object in the usual way:

$ gcc -fPIC -o luapi.o -c luapi.c -I$/path/to/lua/include/directory

You may not need to specify where the Lua include files are, if they are in a standard, system location. If you have built Lua from source, the include files are in the src/ directory of the build.

Depending on how Lua was built, you may or may not need to specify -fPIC (position-independent code), and perhaps other code generation flags.

As an aside, I suggest that this modern tendency to configure builds to create position-independent code is a real nuisance when it comes to building libraries. Unfortunately, many platforms now insist on it.

Now link with liblua.so to create the shared library:

$ gcc -fpic -shared -o luapi.so luapi.o \
           -L/path/to/lua/lib/directory -llua

If you don't have liblua.so, you (or some package maintainer) probably configured Lua to build as a static library, liblua.a. If you have liblua.a then you can generate liblua.so like this:

$ gcc -shared -o liblua.so liblua.a

What you don't want to do is to link your library with liblua.a. Although this will (probably) work, it will duplicate a heap of the Lua code in your library.

Running Lua with the module library

If you're running a system installation of Lua, and liblua.so is in a standard location, there's nothing extra to do here. If you have liblua.so in a non-standard location, or you had to build it yourself, you probably will need to specify its location using LD_LBRARY_PATH:

$ LD_LIBRARY_PATH=/path/to/lua/lib/directory lua

Loading the module library in Lua

Ensure that your library's .so file is in a location specified in the package.cpath variable. Then:

pi = require ("luapi");
pi.set_gpio (24, true); # Or whatever

Notes

It's possible to build an extension module in C++, rather than C, but it's certainly easier to do this if you've built Lua from source using a C++ compiler. If your Lua is built using C -- and the version in most Linux repositories almost certainly will be -- then the function names created by the C++ compiler won't match the names of the Lua API functions in the Lua library. To work around this problem, you need to configure the C++ compiler to use C naming conventions for all the methods in the Lua API. In at least some Lua versions, a way to do this is to #include the Lua headers like this:

extern "C" {
#include "lua.h"
...
}

If you're building Lua from source, it's probably just as easy to ignore the procedure I've described, and write your extension directly into the source. Have a look at linit.c to see how the built-in modules are loaded, and at the end of ldblib.c (for example) to see how a built-in library is registered.

Of course, you lose generality by working this way, but you get the convenience of a Lua that has your extensions built right in.

Summary

Building an extension module for Lua in C is not particulary difficult, but the variety of different way of doing it, the variety of ways that Lua is built, and the incompatibilities between the different Lua versions, make it hard to find specific instructions. I hope that by understanding what is actually going on in detail, it will be easier to find a method that suits your particular set-up.