next »

[eclectocrat]

Editors Note
There is no editor. This half finished Lua C API tutorial was rotting on my hard drive, and I'm far too busy to keep working on it, so I thought that it'd serve the community better here. The writing style is spastic, and the code has only been given a cursory check, there may be some bugs. Regardless, I think that for those who are interested in understanding the Lua C API, this will be a useful introduction. Consider this a WIP, I'll fix any mistakes that you point out, and I'll eventually add a few sections.

You've been warned.

Embedding Lua for Fun and Profit (Preview)
By Jeremy Jurksztowicz, AKA eclectocrat

Introduction
Lua is a very fast, very small, register based, interpreted programming language. It has a long  history of use in mission critical applications and AAA games, but has only recently come into the mainstream amongst indie game developers. I started learning Lua about 1 year ago, while embedding it into the Mysterious Castle game engine. Through trial and error, a lot of persistence and a little luck, I've managed to embed Lua into a large C++ project and develop a toolbox of code and techniques to easily manage the interface between C/C++ and Lua. I will present this toolbox here, but make no claim that what I present is the optimal, or even correct way of embedding Lua. The only claim that I can make confidently is that it works very well for my project.

For this tutorial I'm going to assume that you understand the Lua language fairly well, although you can do without understanding the more esoteric components, like environments and coroutines. It would be a good idea to have a copy of the Lua C API reference handy, but I'm going to assume  that you've never worked with it before, so I'll try and be explicit. Although I'm coming from a C++ angle, most of the tutorial should be easy to grasp for C programmers. However, the very last  section will go deep into C++ country and a firm grasp of templates is a must for understanding the code presented there.

[eclectocrat]

Understanding the Lua stack
I will not make any attempt to introduce you to the Lua language, there are many good tutorials available online. Here I will present the main structure of the Lua C API in brief, to setup a basis for understanding the rest of the tutorial.

A nice thing about the Lua language is that the interpreter uses no global variables whatsoever. This means that you can have multiple Lua environments running on different threads without any conflict. This is also one of the major reasons that embedding Lua is so simple, you really only have to keep track of a single variable per execution environment: lua_State.

I'll spare you the technical details, mostly because I'm not really aware of them, but in essence each lua_State maintains a stack of lua variables. Lua variables come in the following (mostly) self explanatory types: nil, boolean, number, string, function, userdata, thread, and table. I'll get to the specifics later on, just remember that lua_State contains a stack of such variables, and the only way in which you interact with Lua is through that stack. Lets create a lua_State and play with it's stack, so that we understand what's going on.

Code:

lua_State * ls = lua_newstate(0, 0);

lua_pushinteger(ls, 0);
lua_pushstring(ls, "this is a zero terminated string.")
lua_pushboolean(ls, true);

// Now the stack looks like this:
// 3. true
// 2. "this is a zero terminated string."
// 1. 0

Anyone who's used a stack before will tell you that it's most common to refer to the top of the stack. In this case the top of the stack is element 3. If you like you can refer to it by calling lua_gettop, which returns the number of elements in the stack, but there is a much easier way. You can refer to elements from the top down by using negative indices. So the top element can be indexed with -1, the second from the top with -2, and so on. As you'll see, you will refer to stack elements with negative indices quite frequently.

Code:

bool boolval = lua_toboolean(ls, -1); // true
const char * stringval = lua_tostring(ls, -2); // "this is a zero terminated string."
int intval = lua_tointeger(ls, -3); // 0

// It's worth noting here that only nil translates into a false value, the number 0 translates
// into true, something that most C programmers might be surprised at.
boolval = lua_toboolean(ls, -3); // true

Ok, so we have some stack elements, we can access them, what else can we do?

Code:

lua_pop(ls, 1); // pop the top element
lua_insert(ls, 1); // move the top element to the bottom of the stack
lua_pushvalue(ls, -1); // pushes a copy of the top element

// Now the stack looks like this:
// 3. 0
// 2. 0
// 1. "this is a zero terminated string."

lua_settop(ls, 0); // removes all elements from the stack

There you go, that is the core gist of the stack. Please consult your friendly neghborhood Lua manual for more detailed info, and of course there's much more detail.

Functions
Pushing and Popping strings and integers is all fun and good, but what if you want to actually do something useful? The most common thing a developer wants to do is to expose a C/C++ function or a C++ class to a Lua script. In order to understand how to expose classes, we have to start with functions, so lets dive right in.

Code:

// All functions exposed to Lua must be of the following structure:
int some_lua_function (lua_State * ls)
{
	// Do work...
	return number_of_return_values;
}

We interface with Lua exclusively with the stack. In the above function the stack will start off with the parameters that have been passed, and it's your responsibility to push the return values onto the stack and tell Lua how many there are. Here is a more concrete example:

Code:

int move_player (lua_Stack * ls)
{
	const int xoffset = lua_tointeger(ls, 1);
	const int yoffset = lua_tointeger(ls, 2);
	
	const Point p = player->move(xoffset, yoffset);
	
	lua_pushinteger(ls, p.x);
	lua_pushinteger(ls, p.y);
	return 2;
}

Ok, we have a lot going on here so lets take it slow. The first thing to notice is that whenever Lua calls your function it will give you a stack containing only the function parameters. The first part of the function turns the parameters into C/C++ stuff we can use (later on I'll show you how to be safe by calling luaL_checktype). The second part of the function is the work being done by your engine. I'll leave that to your imagination. The third part of the function shows how we return values to Lua, via the ever so versatile stack. The final line of the function indicates to Lua how many return values to receive.

So now that we have defined a function, here's how we get Lua to recognize it:

Code:

lua_register(ls, "MovePlayer", move_player);

Hmmm... that was easy. Now I can call MovePlayer in my Lua scripts! You now know enough to expose your engine to Lua scripts, but please stay tuned, I'm going to show you how to check parameters, handle errors, and expose classes.

[eclectocrat]

A quick note about the Lua C API
Some people may be scratching their head, wondering why in the 7 hells would anyone want to manually manage the Lua stack. It looks complex, it looks error prone, why not use a wrapper library? Sure, go ahead. If you want to use a library, and you find one suitable to your needs, then by all means use one. Just read the next few lines before you decide.

Stack management lends itself to encapsulation, so much of the complexity can be made invisible. Manual stack management is very fast, and Lua is a very fast language, a match made in heaven. Manual stack management doesn't require any additional libraries or API's. Finally, stack management is really not that complex or difficult. Yes, you need to spend some time with the API, you need to put forth a bit of effort, but you'd need to put effort into learning a wrapper library anyways. In the end you'll have a blazingly fast engine that's easy to program and fully under your control. Besides all that, I'll show you a cool trick that will simplify stack management enormously. Just
keep reading.

[eclectocrat]

Tables
Tables! Tables! Tables! If you don't understand tables, go read the Lua documentation and come back. Tables are everything. There is no Lua without tables. Here is a brief introduction on how to interface with Lua tables using the C API.

Code:

lua_newtable(ls); // creates an empty table on top of the stack
lua_pushstring(ls, "tablekey"); // pushes a value onto the stack that we will use as a table key
lua_pushinteger(ls, 1337); // this will be the value indexed with the above key

// Now the stack looks like this:
// -1. 1337
// -2. "tablekey"
// -3. <table>

// This uses the top element at -1 for the value, uses the element at -2 for the key, and writes to
// the table at the indicated position, in this case the element at -3. Both the value and the key
// are popped off of the stack, leaving the new table on top.
lua_settable(ls, -3);

// The above C code's equivalent Lua code is:
// {}["tablekey"] = 1337

The code is pretty self explanatory for anyone familiar with Lua tables, so I'm going to move quickly onto the next important issue. Where is the empty table that we created? It doesn't have a name, it exists only on the stack. In order to keep the table we have to create a permanent reference to it, but how? Here comes the delicious Lua gravy. Almost everything in Lua is a table, even the global environment! So in order to turn our new table into a global variable, we just put it in the global table, in the same way that we put '1337' into the new table! So continuing where we left off:

Code:

lua_pushstring(ls, "mynewtable");

// Now the stack looks like this:
// -1. "mynewtable"
// -2. <table>

// If you look at the code snippet above, you'll notice that the value should be at the top of the
// stack, with the key underneath it, so lets shuffle the stack a little.
lua_insert(ls, -2); // move the top element to -2, shifting the previous -2 to the top

// Now the stack looks like this:
// -1. <table>
// -2. "mynewtable"

// Now do the same thing as we did above, but refer to the global table with a special pseudo-index.
lua_settable(ls, LUA_GLOBALSINDEX);

Ta-Da! We have created a new global variable! Lua tables can use anything as keys, including other tables, but you'll most often be using strings as keys. The Lua C API has convenient functions to simplify getting and setting table 'fields', appropriately named lua_getfield and lua_setfield. With these useful functions the 2 code snippets above can be condensed to:

Code:

lua_newtable(ls);
lua_pushinteger(ls, 1337);
lua_setfield(ls, -2, "tablekey");
lua_setfield(ls, LUA_GLOBALSINDEX, "mynewtable");

Lets speed thing along and show you a very practical example of how to expose an engine singleton to Lua. In this cas we're going to expose an application singleton which shows an alert panel with some text.

Code:

// Somewhere or another we set this C++ class up.
AlertPanel * alert = ...;

int alert_show (lua_State * ls)
{
	string message; // C++ string
	if(lua_gettop(ls) > 0 && lua_isstring(ls, -1))
	{
		message = lua_tostring(ls, 1);
	}
	else
	{
		// Lets try to get a default message.
		lua_getfield(ls, LUA_GLOBALSINDEX, "Alert");
		luaL_checktype(ls, -1, LUA_TTABLE);
		lua_getfield(ls, -1, "default");
		if(lua_isstring(ls, -1))
			message = lua_tostring(ls, -1);
	}
	
	if(!message.empty())
	{
		alert->setMessage(message);
		alert->show();
	}
	return 0;
}

int alert_hide (lua_State * ls)
{
	alert->hide();
	return 0;
}

// Initialize Lua interface in some function...
lua_newtable(ls);

lua_pushcfuntion(ls, alert_show);
lua_setfield(ls, -2, "show");

lua_pushcfuntion(ls, alert_hide);
lua_setfield(ls, -2, "hide");

lua_pushstring(ls, "Alert");
lua_insert(ls, -2);
lua_settable(ls, LUA_GLOBALSINDEX);

Before you quit in disgust let me assue you there is a far simpler and more convenient way to do the above, but I want to make sure you understand the underlying mechanism. So what have we done? Look at the following Lua code to see how we'd use Alert:

Code:

-- Configure the Alert singleton with a default message.
Alert.default = "An unknown error occured!"

-- In case of an error the user should be aware of, Alert them!
local result,err = pcall(some_function, some_parameters)
if not result then
	Alert.show(err)
end

That looks a lot like a class. In fact for a lot of engines it may be enough to just expose a bunch of singletons created in this way. But to aleviate any concerns over the convoluted table management in the above code, let me show you a much simpler way of doing the same thing:

Code:

// After declaring the Alert C functions...
static const luaL_reg Alert_funcs[] =
{
    {"show", alert_show},
    {"hide", alert_hide},
    {0, 0}
};

// Initialize Lua interface in some function...
luaL_openlib(ls, "Alert", Alert_funcs, 0);
lua_pop(ls, 1);

Much easier, no? Now you know what is happening under the hood, how you can create a nifty table with functions and variables, and how to do it quickly and painlessly. Something else you can do with your new knowledge is to modify existing Lua tables. For instance you might want to add a few functions to the rather spartan 'io' library, perhaps 'create_dir' and 'destroy_dir'. Remember that you aren't modifying a library, but enhancing a built in facility of your game engine (Counter arguments in 3... 2... 1...).

[eclectocrat]

Userdata, Metatables, and Classes
Lua does not have any built in notion of classes. However with a few of Lua's incredibly powerful mechanisms you can create a first class class system. Go to lua.org to  find a wonderful selection of tutorials on how to support classes and inheritance. For my part, I'm going to show you how to expose C++ classes to Lua such that destructors can play nicely with Lua's garbage collection.

In order to expose a C++ class that will be garbage collected by Lua, we're going to use the userdata type, which in the Lua C API is represented by a void pointer. Rather than try to explain it, here's a complete example for us to analyse:

Code:

int monster_create (lua_State * ls)
{
	// Monster is a C++ class defined somewhere...
	Monster * monster = new (lua_newuserdata(ls, sizeof(Monster)) Monster;
	luaL_getmetatable(ls, "MetaMonster");
    lua_setmetatable(ls, -2);
    
	return 1;
}

int monster_destroy (lua_State * ls)
{
	luaL_checktype(ls, 1, LUA_TUSERDATA);
	Monster * monster = reinterpret_cast<Monster*>(lua_touserdata(ls, 1));
	monster->~Monster();
	
	return 0;
}

int monster_rawr (lua_State * ls)
{
	luaL_checktype(ls, 1, LUA_TUSERDATA);
	Monster * monster = reinterpret_cast<Monster*>(lua_touserdata(ls, 1));
	monster->rawr(); // prints a menacing "rawr! I's gun eet u!" message
	
	return 0;
}

static const luaL_reg Monster_funcs[] =
{
    {"create", monster_create},
    {0, 0}
};

static const luaL_reg Monster_methods[] =
{
	{"__gc", monster_destroy},
	{"rawr", monster_rawr},
    {0, 0}
};

// Initialize Lua interface in some function...
luaL_newmetatable(ls, "MetaMonster");
lua_pushstring(ls, "__index");
lua_pushvalue(ls, -2);
lua_settable(ls, -3);

luaL_openlib(ls, 0, Monster_methods, 0);
luaL_openlib(ls, "Monster", Monster_funcs, 0);

That's quite a lot to take in, but before we analyze it, take a quick look at how our class will be used in a Lua script:

Code:

local badguy = Monster.create()

-- Oh noes! You've upset the badguy!
badguy:rawr()

There you have a C++ class making an appearance in a Lua script. Now lets see how we got here, step by step, starting with monster_create.

The first thing we do is create a C++ class instance using the placement new operator. If you're not familiar with placement new, what it basically does is create a class instance at the memory given to it. In this case we are passing the memory that we've gotten from the Lua interpreter. This leaves a new userdata instance on top of the stack. The next two lines are where the magic happens.  I'm not going to attempt to explain metatables to you, so the following gross distortion will have to do. I'm basically setting the class instance's virtual function table, or list of functions that it supports. I'm using a metatable called "MetaMonster", which I create further down the code  snippet. Finally I return 1 to indicate to Lua that the userdata on the stack is to be returned to the caller of monster_create.

The next function we've defined is monster_destroy. This function will be called automagically by Lua, after the last reference to a monster instance is released. After making sure that the parameter makes sense, we cast the userdata to our C++ class type, and call it's destructor. Don't call the delete operator, Lua will handle the raw memory. You can zero out the memory if it makes you feel better.

The monster_rawr function is where we expose a member function to Lua. In this case it's the super intimidating rawr function, which takes no parameters and returns no results. When exposing a member function, the first parameter will always be the userdata itself, which we've created as a C++ class instance in monster_create. To support additional parameters or return values, just be aware that parameters will start from stack index 2, like this:

Code:

int monster_jump (lua_State * ls)
{
	luaL_checktype(ls, 1, LUA_TUSERDATA);
	Monster * monster = reinterpret_cast<Monster*>(lua_touserdata(ls, 1));
	
	// The member function parameters will start at stack index 2...
	luaL_checktype(ls, 2, LUA_TNUMBER);
	const int howhigh = lua_tointeger(ls, 2);
	const bool result = monster->jump(howhigh);
	
	lua_settop(ls, 0);
	lua_pushboolean(ls, result);
	return 1;
}

A C++ class is now usable from Lua! But there is something a little off, something not quite right about using userdata throughout a Lua scipt. What happens if you want to add a field to your class? You'd have to do it through C++ and expose the new field via another Lua function. That can be kind of messy, and seems to defeat the purpose of using a nice flexible language like Lua. So I'm going to show you how to take it to the next level and expose a C++ class as a Lua table, which you can add functions and variables to. The key to acheiving this is metatables. We already have a monster metatable with the supported member functions, now we just have to play with the fiddly bits so that when we call one of those functions we have access to the table and the C++ object. First off lets revisit monster creation, and return a table rather than a userdata:

Code:

int monster_create (lua_State * ls)
{
	// Create a table and set it's metatable.
	lua_newtable(ls);
	luaL_getmetatable(ls, "MetaMonster");
	lua_setmetatable(ls, -2);

	// Monster is a C++ class defined somewhere...
	Monster * monster = new (lua_newuserdata(ls, sizeof(Monster)) Monster;
	luaL_getmetatable(ls, "MetaMonster");
	lua_setmetatable(ls, -2);

	// Now the stack looks like this:
	// 2. <userdata> (our C++ class instance)
	// 1. <table> (with a "MetaMonster" metatable)
    
    lua_setfield(ls, -2, "core_");
	return 1;
}

The difference here is that we start off by creating a table and setting it's metatable before we create the userdata. Next, we give the userdata the same metatable, and put it in the new table instance as the core_ field. Maychance you can see where this is going? Our member functions will now accept the Lua table as the first parameter, extract the core_ field to get at the C++ class instance and use the two in tandem to do magic.

A quick note about the metatable. We assign the metatable twice, once to the table, and once to the userdata. We will use the member functions through the table, but only userdata objects will have the __gc metamethod called when they're ready to die. So rather than use two seperate metatables, one for member functions and one for the destructor, I just packed them together.

Now take a look at the new destroy and jump functions to see what happened.

Code:

// The __gc metamethod will only be called on userdata.
int monster_destroy (lua_State * ls)
{
	luaL_checktype(ls, 1, LUA_TUSERDATA);
	
	Monster * monster = reinterpret_cast<Monster*>(lua_touserdata(ls, 1));
	monster->~Monster();
	return 0;
}

int monster_jump (lua_State * ls)
{
	luaL_checktype(ls, 1, LUA_TTABLE);
	lua_getfield(ls, 1, "core_");
	luaL_checktype(ls, -1, LUA_TUSERDATA);
	
	Monster * monster = reinterpret_cast<Monster*>(lua_touserdata(ls, -1));
	
	// The member function parameters will start at stack index 2...
	luaL_checktype(ls, 2, LUA_TNUMBER);
	const int howhigh = lua_tointeger(ls, 2);
	
	// We can include the Lua table in this nonsensical function...
	int jumpbonus = 0;
	lua_getfield(ls, 1, "jumpbonus");
	if(lua_isnumber(ls, -1))
		jumpbonus = lua_tointeger(ls, -1);
	
	const int result = monster->jump(howhigh + jumpbonus);
	lua_pushboolean(ls, result);
	return 1;	
}

The first thing our member function does is check that it's first parameter is a table. It then extracts the core_ field and casts it to the appropriate C++ class instance. At this point I'd like to mention that we can always use more error checking, especially in the debug stages of a project. It would be a disasterous logic fail if you somehow replaced core_ with another userdata instance or other silly crap. You might want to avoid name clashes by renaming core_ to something more unique, like core_CANTTOUCHTHIS, or even better, core_nameofcppclass.

Moving on. Our member function now has a stack with a table on the bottom, and a pointer to the C++ class instance on top. Feel free to go crazy. In practice, I'm a little wary of accessing Lua members from C++ member functions because it presents the same problems as using straight-up userdata as class instances, it requires you to change C/C++ source and recompile. Nevertheless you've got the option, and there are sensible ways in which you can mix the two.

My personal opinion is that C++ should focus on whatever features it provides, and Lua should use those features to achieve your goals. Use C++ for the building blocks, use Lua for the design.

Code:

badguy = Monster.create()

-- badguy gets a special attack...
badguy.pounce = function (self, howhigh)
	self.jumpbonus = howhigh or 2
	self:jump()
	self:rawr()
end

[eclectocrat]

References
Up to now, I've been working from the position that objects are instantiated in Lua and kept there. However, it's easy to foresee times when you may want your engine to manage Lua objects. In these cases you need to have some sort of reference, which you can use to put a Lua object on the stack. While it's not too tricky to build your own reference system (I'll leave that as an exercise for the reader... I've always wanted to write that!), Lua provides a very nice prerolled one.

Code:

// Given that we want to maintain a reference to a Lua object on top of the stack...
const int objectRef = luaL_ref(ls, indexofatable);

// ...stuff happens, maybe the following code is in another function...

// Push the object onto the top of the stack.
lua_rawgeti(ls, indexofatable, objectRef);

// ...again, the following code could be anywhere...
luaL_unref(ls, indexofatable, objectRef);

// Now the reference is released, don't use it again.

Wha...!? What's this 'indexofatable' business? Well, Lua references have to live somewhere, so you need to indicate which table to put the reference in. If you've got a bunch of 'Monster' instances that you want to keep handy, then you might create a 'MonsterRef' table in the global namespace to keep track of 'Monster' instances. Yeah... I know... that's kind of lame. I thought references were supposed to be easy and fast. Well, Lua has that covered with a special table; the registry.

Code:

const int objectRef = luaL_ref(ls, LUA_REGISTRYINDEX);

The registry is accessed by a special stack pseudoindex, and acts as a nice tidy place to keep your references. It also has the advantage of being accessed very quickly, since a name lookup to find the table is unnecessary. While on this topic, let me assure you that you have absolutely no reason whatsoever to concern yourself with the performance of the reference system. When I first started with Lua, I was pretty worried about the performance hit of looking up references everytime Lua objects were accessed from C++. My concerns were totally unfounded. Reference lookup is really fast, it's basically just an array access.

This brings me to a broader point about the performance of Lua: it's very good. As a C++ programmer used to coding realtime media pipelines, I was naturally concerned about using this unusual opaque stack interface. I thought that if I wanted access to a Lua object, I should just be given a pointer and an interface. While I have found ways to push Lua to it's performance limits (in procedural world generation and AI, two CPU heavy tasks), profiling has shown that my extremely Lua heavy game loop only spends 2% of it's processing time in Lua calls. So go ahead and use the reference system as much as you like. It's highly unlikely to impact performance.

Good stuff, here's an example of how we might use references in our game.

Code:

int world_insert (lua_State* ls)
{
	// Get the self parameter, an instance of the C++ class "World"
	luaL_checktype(ls, 1, LUA_TTABLE);
	lua_getfield(ls, 1, "core_");
	luaL_checktype(ls, -1, LUA_TUSERDATA);
	
	World * world = reinterpret_cast<World*>(lua_touserdata(ls, -1));
	lua_pop(ls, 1);
	
	// Get the monster parameter.
	luaL_checktype(ls, 2, LUA_TTABLE);
	lua_getfield(ls, 2, "core_");
	luaL_checktype(ls, -1, LUA_TUSERDATA);
	
	Monster * monster = reinterpret_cast<Monster*>(lua_touserdata(ls, -1));
	lua_pop(ls, 1);
	
	// Get the X and Y coords.
	luaL_checktype(ls, 3, LUA_TNUMBER);
	luaL_checktype(ls, 4, LUA_TNUMBER);
	const Vec2i loc(
		lua_tointeger(ls, 3),
		lua_tointeger(ls, 4));
	
	const bool result = world->insert(monster, loc);
	
	// We need to store the reference somewhere, in this example, we'll
	// store it within the actual monster instance, in an integer property
	// named 'ref'.
	if(result && monster->ref() == LUA_NOREF)
	{	
		// We've successfully inserted the monster, lets keep a reference to
		// it, so that even if the Lua script lets go of it's handle, we can
		// still access it through our "World" class instance.
		
		// We need the object that we want to reference on top of the stack,
		// so we'll just pop off everything on top of our monster instance,
		// which is at stack position 2.
		lua_settop(ls, 2);
		
		const int ref = luaL_ref(ls, LUA_REGISTRYINDEX);
		monster->setRef(ref);
	}
	
	lua_pushboolean(ls, result);
	return 1;
}

int world_remove (lua_State* ls)
{
	luaL_checktype(ls, 1, LUA_TTABLE);
	lua_getfield(ls, 1, "core_");
	luaL_checktype(ls, -1, LUA_TUSERDATA);
	
	World * world = reinterpret_cast<World*>(lua_touserdata(ls, -1));
	lua_pop(ls, 1);
	
	luaL_checktype(ls, 2, LUA_TTABLE);
	lua_getfield(ls, 2, "core_");
	luaL_checktype(ls, -1, LUA_TUSERDATA);
	
	Monster * monster = reinterpret_cast<Monster*>(lua_touserdata(ls, -1));
	lua_pop(ls, 1);
	
	monster = world->remove(monster);
	if(monster)
	{
		luaL_unref(ls, LUA_REGISTRYINDEX, monster->ref());
		monster->setRef(LUA_NOREF);
	}
	return 0;
}

There's a fair bit going on here. We're working with a hypothetical C++ class called "World". World objects keep monsters and display them in the render loop. Somewhere in the World implementation it holds on to the Monster pointer that we give it in the 'insert' call, but in order to hold on to the Lua table and prevent the garbage collector from destroying it, we need to keep a reference somewhere. I'd prefer to keep Lua out of my C++ class implementations completely, so I'm going to handle references right here in the C++/Lua binding layer. Unfortunately I still need to keep the Lua handle somewhere, so I've opted to keep it in the C++ class instance itself. Lua references are just plain old ints, so I don't need to import Lua into the class file, only make a public int property called 'ref'. When I create a new monster instance, I'll set the ref property to LUA_NOREF to indicate that this instance has not been referenced.

Code:

Monster * monster = new (lua_newuserdata(ls, sizeof(Monster)) Monster;
	
// We have not created a reference yet.
monster->setRef(LUA_NOREF);

When I want to keep a monster instance somewhere in my C++ code, I'll create a reference to it, and when the instance is no longer needed by my engine, I'll release the reference. Note that there is no need to release the reference in the __gc metamethod, because that method will never be called while a reference exists.

Dealing with references can be error prone, especially of you do it in many different functions, so you may want to write an RAII C++ class to handle it for you. I only keep references to one kind of object in my game, and those objects are referenced and unreferenced in two well defined functions. I find that to be the best strategy for resource management.

PS. Lua also supports weak references. Consult your Lua documentation for details.

[eclectocrat]

Making it Tidy
Let me preface this section with the disclaimer that it's entirely unnecessary, and many people may find it in bad taste. The Lua C API tutorial ends here, and if you're satisfied with the raw C API, then be on your way with my blessing. What follows are a bunch of template specializations and poorly names preprocessor macros mashed together to make a slightly too clever mini-language for binding C++ and Lua. I'll repeat what I said at the beginning: this works well for me, your mileage may vary.

There's a truism in programming that states that repeatedly writing tedious code will increase bugs. You may have noticed a lot of tedious code in the above snippets, and here's where I'm going to fix that, with the most hated of all C/C++ facilities, the preprocessor.

The preprocessor is given a really bad rap. Like goto, it is portrayed as anathema to good software development. I understand the need to make people aware of the dangers, but absolutes have no place in reality and I've actually seen a few well utilized gotos in my life. Knowing the dangers of a tool we are far better prepared to handle it safely. So in this next section be aware that macro name clashes will KILL PUPPIES. The macro names that I use do not clash in any way with my code, but they may clash with yours.

I'm not going to explain all the ins and outs of what is going on under the hood, because we already studied the Lua C API in the sections above. Without further ado, here is the core outline of a macro mini-language for binding C++ code with Lua.

Here's the first component of the language, how to declare functions.

Code:

#define Lfunc(NAME) \
	int NAME (lua_State * ls) { \
		int _stackpos = 1; (void)_stackpos; \
		bool _opts = true; (void)_opts; \
		try {

Notice the two variables: _stackpos and _opts. We will be using these to help us extract function parameters from the stack. The next thing you'll notice is that we are opening a try block. If you want Lua to play nice with C++ stack unwinding and exception handling, then you have to enable C++ exceptions in luaconf.h and compile the Lua sources as C++ code. While you're at it, take a good long look at luaconf.h because it's very well documented and you can find some neat things there.

For passing data back and forth, we'll be using the amazing power of templates. The compiler will determine the type of parameters and return values for us.

Code:

template<typename T>
T parm_get (lua_State *, int&);

template<typename T>
T opts_get (lua_State *, int&, T const&, bool&);

template<typename T>
int ret_push (lua_State *, T const&);

//
// for example, bool
//
typedef bool Lparm_Lbool;
template<> bool parm_get (lua_State * ls, int& stackpos)
{
	luaL_checktype(ls, stackpos, LUA_TBOOLEAN);
	return lua_toboolean(ls, stackpos++);
}
template<> bool opts_get (lua_State * ls, int& stackpos, bool const& defaultval, bool& look)
{
	if(look && lua_isboolean(ls, stackpos))
		return lua_toboolean(ls, stackpos++);

	look = false;
	return defaultval;
}
template<> int ret_push (lua_State * ls, bool const& b)
{
	lua_pushboolean(ls, b);
	return 1;
}

By specializing the template instantiations of the above types, we can add support for any data that can be translated to and from Lua values.

Code:

typedef Vec2i Lparm_LVec2i;
template<> Vec2i parm_get (lua_State * ls, int& stackpos)
{
	luaL_checktype(ls, stackpos, LUA_TNUMBER);
	luaL_checktype(ls, stackpos+1, LUA_TNUMBER);
	
	return Vec2i(
		lua_tonumber(ls, stackpos++),
		lua_tonumber(ls, stackpos++));
}
template<> Vec2i opts_get (lua_State * ls, int& stackpos, Vec2i const& defaultval, bool& look)
{
	if(look && lua_isnumber(ls, stackpos) && lua_isnumber(ls, stackpos+1))
		return Vec2i(
			static_cast<int>(lua_tointeger(ls, stackpos++),
			static_cast<int>(lua_tointeger(ls, stackpos++));

	look = false;
	return defaultval;
}
// ... more types here ...

In the above example we take two integers and turn them into a Vec2i. We can likewise take a Lua table containing a core_ field and turn it into a C++ class instance. Just be careful to keep the stackpos parameter pointing to the next unused stack element.

We also provide support for optional parameters with default values. Any copy constructible C++ type will do, just be sure to set 'look' to false if the parameter is not present, so that we stop looking for additional parameters that aren't there.

Now take a look how we use what we've built to automate the parameter extraction and returning values.

Code:

#define Lparm(TYPE, NAME) \
	Lparm##_##TYPE NAME = parm_get<Lparm##_##TYPE>(ls, _stackpos);
	
#define Lopts(TYPE, NAME, DEFVAL) \
	Lparm##_##TYPE NAME = opts_get<TYPE>(ls, _stackpos, DEFVAL, _opts);

#define Lcatch \
	} catch (std::exception const& err) { \
		std::cerr << err.what() << std::endl; \
	} catch (...) { \
		std::cerr << "unknown exception." << endl; \
	}

#define Lret0 \
	Lcatch \
	lua_settop(ls, 0); \
	return 0; \
	}
	
#define Lretv(RET) \
	Lcatch \
	lua_settop(ls, 0); \
	return ret_push(RET); \
	}

By using the typedefs declared above, we can safely and easily extract all the parameters we want, and return anything we want, without any explicit Lua stack management. Just look at an example:

Code:

Lfunc(MovePointer)
	Lparm(Vec2i, distance)
	Lopts(Lfloat, speed, 1.0)
	
	const Vec2i pos = Game::movePointer(distance, speed);

	Lretv(pos)

Pretty tidy! We can easily access the distance and speed variables in our function body. Using some more preprocessor magic, we can expand our little mini-language to support classes. Rather than go over the boring details, I'll show you how it looks, and if you like what you see, you can copy-pasta the implementation below.

Code:

Lfunc(Entity_new)
	Linst(Entity)
	Lret1
	
// Yes, there are THREE (3) underscores
Lfunc(Entity___gc)
	luaL_checktype(ls, 1, LUA_TUSERDATA);
	Entity * self = (Entity*)lua_touserdata(ls, 1);
	self->~Entity();
	Lret0
	
Lmeth(Entity, get_location)
	Lretv(self->location())
	
Lmeth(Entity, move)
	Lparm(LVec2i, dist)
	const Vec2i pos = self->move(dist);
	Lretv(pos)
	
// This function shows how you can manually parse a Lua table
Lmeth(Entity, load_sprite)
	Lparm(Lstring, ident)
	
	if(self->sprite(ident))
	{
		cerr << "Unable to load sprite '" << ident 
			 << "', a sprite is already using that identifier." << endl;
	}
	else
	{
		// Manually extract the table parameter.
		luaL_checktype(ls, _stackpos, LUA_TTABLE);
		vector<string> textures;
		textures.reserve(lua_objlen(ls, _stackpos));

		lua_pushnil(ls);
		while(lua_next(ls, _stackpos))
		{
			// key is at -2, val is at -1
			textures.push_back(lua_tostring(ls, -1));
			lua_pop(ls, 1);
		}
		
		auto_ptr<Sprite> sprite(new Sprite);
		for(vector<string>::iterator i = textures.begin(); 
			i != textures.end(); ++i)
		{
			sprite->addFrame(*i);
		}
		
		if(self->insertSprite(ident, sprite.get()))
			sprite.release();	
	}
	Lret0
	
Lmeth(Entity, set_sprite)
	Lparm(Lstring, ident)
	self->activateSprite(ident);
	Lret0
	
// Define the interface to 'Entity'
Linterface_start(Entity)
	Linterface(Entity, __gc)
	Linterface(Entity, move)
	Linterface(Entity, get_location)
	Linterface(Entity, load_sprite)
	Linterface(Entity, set_sprite)
Linterface_end

void LuaExport (lua_State * ls)
{
	// Export the 'Entity' interface
	Linterface_class(Entity)
}

[eclectocrat]

Conclusion
I've presented the basic framework of how you might interface C/C++ and Lua. You can go a lot deeper if you like, perhaps inheriting from C++ classes in Lua, or even inheriting from Lua classes in C++. I personally don't see the point, but I'm sure you can find some good reasons to go deep. In any event, I hope you learned enough about the Lua stack to approach the C API with some confidence.

Feel free to comment or ask questions at:

   jurksztowicz (at the email domain) gmail (dot) com
   
Please buy my game at:

   www.mysteriouscastle.com

I'm also often at the TIGSource forums waxing lyrical. Cya!

[eclectocrat]

Full Source

Code:

#ifndef _ark_LuaBind_h
#define _ark_LuaBind_h

#include "lua/lua.h"
#include "lua/lauxlib.h"
#include "lua/lualib.h"
#include <string>

namespace ark {
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////
template<typename T>
T parm_get (lua_State *, int&);

template<typename T>
T opts_get (lua_State *, int&, T const&, bool&);

template<typename T>
int ret_push (lua_State *, T const&);

//
// bool
//
typedef bool Lparm_Lbool;
template<> bool parm_get (lua_State * ls, int& stackpos)
{
	luaL_checktype(ls, stackpos, LUA_TBOOLEAN);
	return lua_toboolean(ls, stackpos++);
}
template<> bool opts_get (lua_State * ls, int& stackpos, bool const& defaultval, bool& look)
{
	if(look && lua_isboolean(ls, stackpos))
		return lua_toboolean(ls, stackpos++);

	look = false;
	return defaultval;
}
template<> int ret_push (lua_State * ls, bool const& b)
{
	lua_pushboolean(ls, b);
	return 1;
}

//
// int
//
typedef int Lparm_Lint;
template<> int parm_get (lua_State * ls, int& stackpos)
{
	luaL_checktype(ls, stackpos, LUA_TNUMBER);
	return static_cast<int>(lua_tointeger(ls, stackpos++));
}
template<> int opts_get (lua_State * ls, int& stackpos, int const& defaultval, bool& look)
{
	if(look && lua_isnumber(ls, stackpos))
		return static_cast<int>(lua_tointeger(ls, stackpos++));

	look = false;
	return defaultval;
}
template<> int ret_push (lua_State * ls, int const& n)
{
	lua_pushinteger(ls, n);
	return 1;
}

//
// float
//
typedef float Lparm_Lfloat;
template<> float parm_get (lua_State * ls, int& stackpos)
{
	luaL_checktype(ls, stackpos, LUA_TNUMBER);
	return static_cast<float>(lua_tonumber(ls, stackpos++));
}
template<> float opts_get (lua_State * ls, int& stackpos, float const& defaultval, bool& look)
{
	if(look && lua_isnumber(ls, stackpos))
		return static_cast<float>(lua_tonumber(ls, stackpos++));

	look = false;
	return defaultval;
}
template<> int ret_push (lua_State * ls, float const& f)
{
	lua_pushnumber(ls, f);
	return 1;
}

//
// string
//
typedef std::string Lparm_Lstring;
template<> std::string parm_get (lua_State * ls, int& stackpos)
{
	luaL_checktype(ls, stackpos, LUA_TSTRING);
	return std::string(lua_tostring(ls, stackpos++));
}
template<> std::string opts_get (lua_State * ls, int& stackpos, std::string const& defaultval, bool& look)
{
	if(look && lua_isstring(ls, stackpos))
		return std::string(lua_tostring(ls, stackpos++));

	look = false;
	return defaultval;
}
template<> int ret_push (lua_State * ls, std::string const& s)
{
	lua_pushstring(ls, s.c_str());
	return 1;
}
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////
#define Lfunc(NAME) \
	int NAME (lua_State * ls) { \
		int _stackpos = 1; (void)_stackpos; \
		bool _opts = true; (void)_opts; \
		try {
	
#define Lself(TYPE) \
	luaL_checktype(ls, 1, LUA_TTABLE); \
	lua_getfield(ls, 1, "_core"); \
	lua_insert(ls, 2); \
	TYPE * self = (TYPE*)lua_touserdata(ls, 2); (void)self; \
	_stackpos+=2; \
	
#define Lmeth(TYPE, NAME) \
	int TYPE##_##NAME (lua_State* ls) { \
		int _stackpos = 1; (void)_stackpos; \
		bool _opts = true; (void)_opts; \
		Lself(TYPE) \
		try {
		
#define Linst(TYPE) \
	lua_newtable(ls); \
	luaL_getmetatable(ls, #TYPE); \
	lua_setmetatable(ls, -2); \
	TYPE * inst = new (lua_newuserdata(ls, sizeof(TYPE))) TYPE; \
	(void*)inst; \
	luaL_getmetatable(ls, #TYPE); \
	lua_setmetatable(ls, -2); \
	lua_setfield(ls, -2, "_core");
	
#define Lparm(TYPE, NAME) \
	Lparm##_##TYPE NAME = parm_get<Lparm##_##TYPE>(ls, _stackpos);
	
#define Lopts(TYPE, NAME, DEFVAL) \
	Lparm##_##TYPE NAME = opts_get<TYPE>(ls, _stackpos, DEFVAL, _opts);

#define Lcatch \
	} catch (std::exception const& err) { \
		std::cerr << err.what() << std::endl; \
	} catch (...) { \
		std::cerr << "unknown exception." << endl; \
	}
	
#define Lret0 \
	Lcatch \
	lua_settop(ls, 0); \
	return 0; \
	}
	
#define Lret1 \
	Lcatch \
	lua_insert(ls, 1); \
	lua_settop(ls, 1); \
	return 1; \
	}
	
#define Lret2 \
	Lcatch \
	lua_insert(ls, 1); \
	lua_insert(ls, 2); \
	lua_settop(ls, 2); \
	return 2; \
	}

#define Lretn(N) \
	Lcatch \
	for(int i=1; i <= N; ++i)\
		lua_insert(ls, i); \
	lua_settop(ls, N); \
	return N; \
	}
	
#define Lretv(RET) \
	Lcatch \
	lua_settop(ls, 0); \
	return ret_push(RET); \
	}
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////	
#define Linterface_start(TYPE) \
	static const luaL_reg TYPE##_interface[] = {
	
#define Linterface(TYPE, FUNC) \
	{#FUNC, TYPE##_##FUNC},
	
#define Linterface_end \
	{0, 0}};
	
#define Linterface_class(TYPE) \
	lua_createtable(ls, 0, 1); \
	lua_pushcfunction(ls, TYPE##_new); \
	lua_setfield(ls, -2, "new"); \
	lua_setfield(ls, LUA_GLOBALSINDEX, #TYPE); \
	\
    luaL_newmetatable(ls, #TYPE); \
    lua_pushstring(ls, "__index"); \
    lua_pushvalue(ls, -2); \
    lua_settable(ls, -3); \
	\
    luaL_openlib(ls, 0, TYPE##_interface, 0);
	
#define Linterface_group(NAME) \
	luaL_openlib(ls, #NAME, NAME##_interface, 0);

} // END namespace ark
		
#endif // _ark_LuaBind_h

[kamac]

Good job!

[tung]

Wow, very nice! Coupling a table with a class instance was a practical point I hadn't considered before, and I really like the idea of using macros and templates to reduce boilerplate.

[Mysticus]

I did a short tutorial on this a while back (

), but your tutorial beats mine by a long way!

Good tutorial! 

[verticalvertex]

Excellent tutorial.

[rubbersausage]

Just want to say thanks alot! And I'm really happy you released it even dough it's just a preview.

This tutorial really went well with how I think and all questions arised within me got directly answered when I just kept on reading.

Thanks to you I'm hacking away with C++ and Lua now 

You should update it to Lua 5.2 some day, since that's probably what most newbies will be learning.

Thanks again!