C++ Callbacks
Tuesday, March 16, 2010 4:39:04 PM
Back to the topic at hand is callbacks and closures. Closures are usually found in functional languages. When imperative languages like Borland C++ Builder say they have closures, they usually mean something much simpler in that a pointer can also hold a reference to the object instance. So when you invoke the function reference, it knows what object to pass as the function's first parameter. FYI, all non-static member functions have an implicit "this" pointer as a first parameter. That is the parameter that is causing so much trouble with callbacks since we need both the function pointer and the "this" pointer to the object.
So what are the ways to use callbacks that point to methods? There are four ways that are most common. There are certainly other ways of doing this. For now, these are the simplest and quickest.
1. Hack your own closure.
I've done this when porting Borland C++ Builder code to GCC for example. What you do is build a class (or struct) that holds two pointers. One is the function pointer and the other is the object pointer. As to invoking the function, a simple way is to use variable arguments. You need at least one argument for the prototype, but that's no problem since you need to pass the object pointer. Unfortunately, this is prone to errors and you'll have issues with return values.
With the advent of templates, you have much more flexibility and could come up with a solution. In fact, the next two solutions are pre-built versions of this.
2. Use Boost
I don't use Boost, but a lot of people like it. I'll just repeat an example posted by Sc4Freak in this reddit thread.
using namespace boost; function<int (int)> callback(bind(&TestClass::Foo, &TestObject, _1)); //Call TestObject.Foo(5). callback(5); //Change callback to a free function. callback = SomeRandomFunction; //Call SomeRandomFunction(8). callback(8);
You need specific functionality for callbacks to work.
- Creation
- Assignment of method
- Assignment of function
- Invocation
Creation is done by using a template. You specify the return type and the argument types within the angle brackets. Then you use bind to have the object and its method together. Not sure about the _1. Might be the number of arguments.
Invocation is straightforward as is assignment of a function. I haven't seen assignment of a method though. Perhaps it works the same way, but use bind again, like this.
callback = bind(&TestClass::Foo, &TestObject, _1); callback(5);
Can someone verify?
3. Use a library such as PlusCallback
I really like this one. It's simply a header file that you include. I may actually use it if I port Project V to other compilers.
Here is the link to the examples.
From the example, suppose we have the following code.
#include <callback.hpp>
struct Dog
{
void Bark(int volume);
};
struct Cat
{
void Meow(int volume);
};
Dog spot, rover; //We have two dogs
Cat felix; //and one cat.
//Define a normal function.
void Func(int a);
Creation
//Define a callback to a function returning void and taking //one int parameter. cb::Callback1<void, int> speak;
Assignment of method
//Point this callback at spot's Bark method. speak.Reset(&spot, &Dog::Bark);
Assignment of function
//Callbacks can be set to free functions. speak = Func;
Invocation
speak(50);
It also supports creation of closure instances with cb::Make1(&felix, &Cat::Meow). The examples show this used within STL containers. I'm not sure how it knows what the argument types are though. Maybe there's some template magic going on since the insert function would require a specific type.
4. Use Borland C++ Builder
This is my favourite option for my own projects because C++ Builder has closures built-in. But when at work or on other projects, you may already be using another compiler. So while this option isn't always possible, it's great when it is.
Creating a closure.
// It's a regular typedef. // Only difference is you add the keyword __closure typedef bool (__closure __fastcall *NCompareValue)(NEntry *A, NEntry *B); NCompareValue myfunc;
This is code from my type system in Project V (only the typedef). You can add custom functions for comparing values of specific types in the NEntry's. What's fantastic about this is that there is no new syntax. The __closure keyword just says to also store object instance.
Assignment of method.
class NTypeModule
{
protected:
// A built in value comparison.
bool __fastcall DefaultNCompareValues(NEntry *A, NEntry *B);
};
NTypeModule mymodule;
// This is how you assign a closure.
myfunc = mymodule->DefaultNCompareValues;
Assignment of function.
You cannot do this. A closure is a pointer to a method only in Borland. This isn't really a big deal since you can always write a method to call the static function.
(edit: There is actually an undocumented way to do this, but I can't remember the details. I have used it. From what I recall, there is a function that will create a closure from a static function pointer and you can then use that.)
Invocation
if (myfunc(node1,node2))
{
// Do stuff here...
}
That's it. There is ZERO new syntax. The only difference is using the __closure keyword when defining the typedef. There is ZERO change anywhere else. Closures are used everywhere in VCL for events (VCL is a C++ library for Windows GUI and structures). It's great being able to use C++ for event handlers and then simply assigning those methods to the proper event. No need for checking what event was triggered with switch statements or any need to use message maps.
But hey, it seems like a pointer that can also hold the object instance in standard C++ has been too much to ask for in these past three decades.
Obviously, the need is there. Of all the things that OOP is supposed to be about, specifically message passing, the simplest of closures should have been available from the beginning. Hopefully, the next revision of C++ will have closures. Unfortunately, it's a little late to the game. Now, if properties could eventually make it in there... well, by that time, Toronto will have won the Stanley Cup.


Unregistered user # Thursday, March 18, 2010 2:20:09 AM
Unregistered user # Friday, March 19, 2010 10:06:50 AM
Unregistered user # Friday, March 19, 2010 10:20:10 AM
Vorlath # Friday, March 19, 2010 11:08:47 PM
I know exactly what you mean. I wish I had someone to blame. I can't even blame JPM since I came up with it on my own. Since discovering dataflow, I can't look at other languages the same way either. It opens up a whole different way of seeing things. What I was fighting against, and what a lot of people are fighting against, is that a lot of problems are meant to be solved using dataflow, but the language isn't quite suited for it. However, I feel you can still use OOP within the framework in certain areas. Even in Project V, I can't completely eliminate global state required by OOP. The code or network itself is a state. Its existence I mean. So sequential programming still has a place.
Dan Pologeadpologea # Wednesday, April 21, 2010 3:33:05 PM
The callback from a classical system is a function (executable code) “injected” into the system (usually into another object) that could be “called back” later on.
What a “callback function” could be into a FBP system? In this case the callback is a component that has the input and output ports connected at runtime to another component that is using its services in the same manner a client is using a server. The “callback component” is the server in this case. The big difference is that in a FBP system the client component does not call the server component (function). The components are evaluated automatically by the execution system and IPs (information packets / parameters) are transported automatically from one component to another as they are produced and consumed by these.
In a FBP system a “callback” could be a component that can be wired with a high flexibility. For instance a component injected into a FBP network could connect its ports to other different / various components. This has strong similarities with actor or entity systems. This potential to wire the ports in any configuration possible is very powerful comparing with what a classical «callback» can do because the classical callback has only one point of entry (and exit). It has to return into the same place like any other “classical function”. Into a FBP network a component could be injected and connected in as many ways as the types of its ports allow. The injected component (“callback function”)
does not has to “return” into the only one place where it was called from because there is no call in the first place.
This is a huge advantage that usual callbacks can not accomplish.
The closures appeared because of the necessity to construct new functions during the execution of a program and its free variables to be bounded in the lexical environment. But these “bounds” in FBP are connections created at runtime and the “free variables” are in fact “state variables” of the processing component. By the way, in a “classical system” you can have “state variables” (of an object) and into a FBP system you can have transitory variables stored into connections.
Now, at first glance it appears curious why we need two pointers for the callback: the function pointer and the "this" pointer to the object. Why the life is more complicated than necessary? In my view regarding FBP, we don't need two pointers and also the connection point is not one but multiple (through several input and output ports connected in any configuration possible). It is possible to group all these input and output ports together into a single “plug” (composite port) in order to simplify connecting.
The “method” from OOP is a subcomponent in FBP. This subcomponent is an instance whatever it really is (pure function, data or a mixture). That means if we have a pure functional component in FBP, this is “instantiated” in a component environment. How a function could be instantiated you might ask. What are its state variables? Well, its state variables are the connections that are specific to that instantiation of that functional component model. The correspondence in a classical environment is a specific function call. This “specificity” means the calling environment that is usually saved on a stack (parameters).
And last but not least, in my view, into a FBP environment you don't have global or static variables anymore. Each variable is a component that is defined into another parent component. All connections with outer world have to pass through component's membrane (i.e. ports). It means you can “cut” the functional object (i.e. component) from its place and reuse it into another place (as a model or as an instance).
The problem with two pointers specified above comes from the fact that the “method” need “this” pointer. That means more than the method in general really uses. In FBP is like saying that a subcomponent depends on all the content of the parent component instead of its input ports only. By the way, this is another reason why you have encountered situations where you can not reuse a method from a class / object just because it is bound to the whole object through “this” pointer. This is a dependency that complicates the logic but simplifies the implementation.
Vorlath # Wednesday, April 21, 2010 4:54:48 PM
The second kind of callback is similar to the above, but where you can pass around the implementation itself. So instead of creating a connection to the inputs and outputs of a component, you would use meta-inputs and meta-outputs which sets and retrieves the implementation at runtime. This is similar to what we commonly use in contemporary programming languages where we manually select what interface implementations we use. It's like setting or retrieving a reference to the implementation.
The third kind of callback is much more powerful and much more difficult to use. In another layer of your network, you can develop a controller program. It's usually done in small parts. But this layer allows you to modify and build anything that is possible in Project V. For example, you can develop a meta-layer that will add a new connection to your ADDITION component when all inputs are filled up and generate the appropriate internal components.
That's a simple example. But you can do other things such as rewire a network at runtime. So you can indeed move components around and connect all inputs and outputs to as many different components, even if they are on other machines.
Most programming languages mix these two layers together. I think that's the wrong way to go. For example, the Actor's model is a primary culprit of this. The actor itself can produce other actors. That's wrong. Languages like Erlang work this way and IMO is a terrible, terrible flaw in the design.
More than that, anything that creates or controls how different parts of your system interacts should not be part of the main program. It should be separate. So if you're using OOP, 'new' (and delete if you have it) should never be part of your main program (if the language was designed properly). Unfortunately, OOP has no mechanism for controlling creation and deletion of objects on a higher level. No thought has gone into this and so the program itself MUST control the objects themselves.
To be clear, I'm not at all suggesting you not use 'new' and 'delete'. I'm saying it could have been better designed.
This is a problem that has already been solved in real life and see no reason to re-invent a broken wheel such as the Actor's model. In the real world, management is a hierarchy. And the workers always work directly on the end result. Not to say that management doesn't do any work. Quite the contrary. What I'm saying is that there are separate layers of computing. The top layers are valid programs just like the main layer. It's just that the top layers are management of the lower layers. It works well in the real world and I see absolutely no reason at all why I should counter thousands of years of evidence and knowledge.
Think of it this way. As a programmer, you are on a different level than the code. You enter the code and the compiler is the tool that is on a higher level that can construct the objects and functions that you entered. But you lose this functionality at runtime. Even if you have things such as eval(), it's not the same. The higher level is gone. eval() or any such construct is WITHIN the same code as everything else. You can no longer have a view of everything and make changes the way you could at the development/compiling stage.
Project V changes all that. All the power you have at the development stage will be available to you at runtime (and I'm not talking only about the ability to compile programs either which is almost besides the point). As I said, having a separate layer is a core principle that has existed for thousands of years whenever you need to produce something requiring large amounts of resources. Think about it. Even Java has a separate layer for garbage collection. That's this principle at work. It is separate. Think of JIT. Again the same principle. Unfortunately, the rest (like bytecode and the language itself) is contrary to this principle.
Vorlath # Wednesday, April 21, 2010 5:12:41 PM
Unregistered user # Thursday, April 29, 2010 7:12:35 PM
Dan Pologeadpologea # Thursday, April 29, 2010 8:45:54 PM
Originally posted by anonymous:
I am hearing this quite often. My fellow programmers did say the same thing: "Well, I don't care what library or language to use as long as I get the job done". The problem was just that the job got done ALWAYS after the deadline and process was painful. It was so hard to demonstrate that there were simpler ways to do it but radically different. It seems this "radically different" is very disturbing. If you look carefully to all concepts spread in the programming world you will realize that most of them are bullshit and what is good takes into account physical laws that nobody can escape.
Personally I have noticed that when all this "knowledge" is getting larger my believes are oriented toward the same set of intertwined concepts: "Matter, Energy, Laws" (the Holy Trinity). Each one of these aspects has a correspondence in software. The software is just a simulation of the real world and it has to obey to the physical laws.
My conclusion is: there is a tendency, if you are really good you are good to do more other things, if you are really bad you are bad to all. If you have a concept in programming that is really good than it has the tendency to be good in many other programming subdomains. I am not saying there is a language that fits all but there are for sure powerful principles that fits all. Among them is also DATA FLOW.
Remember the holy trinity: Matter, Energy, Laws. You can make a kind of association: Matter -> Data, Energy -> Hardware, Laws -> Functions (restrictions). A software component is a mixture of Matter (data) and Laws (functions) but it lacks the energy that pushes the matter through the sieve (laws). The energy is given from the real physical component that is the hardware (microprocessor). But the hardware is more than that. It represent also data and laws. The software is just it's extension but it lacks a very important aspect: the energy. If the data could move "by itself" trough the pipes it would be great but it can't. The matter moves "by itself". In fact it is about forces, pressure, energy distribution or whatever else. In software we need hardware for this.
To conclude: Flow Based Programming seems to take into account the natural physical laws. It starts from the knowledge that software is a simulation of the real physical world and takes it into account instead of fight it against.
Vorlath # Thursday, April 29, 2010 11:40:01 PM
I'm not certain this is the best way to go. C++ is very imperative and yet people are saying it's trying to do too much. That it tries to be everything to everyone. I think you need to compartmentalize and stick with one paradigm within each compartment. And make sure that the interactions between compartments are fluid and dynamic. This is harder than it seems and is where most of the problem come from as Dan correctly points out.
I do agree with this:
If you limit a dataflow network to be linear (the outputs of a component can only link to a single following component) and you limit the network to one single data packet, then yes indeed, you will simulate imperative sequential programming.