Traits
Monday, 24. November 2008, 18:59:23
On reddit, there's this pdf paper about traits. It's something that the authors would like to see in Object Oriented Programming Languages.
The easiest way to describe these traits is to say that they are similar to interfaces that come with their own implementation, but may list a set of required external methods. Consider it a double ended interface where you select individual methods for each of the externally required pure functions instead of actually providing an all-in-one implementation package. Traits cannot manipulate local state other than with get and setters (through external methods). This supposedly lets one mix and match traits and then select what specific implementation you want for each of the externally required trait methods.
The problem I have with the paper isn't necessarily the ideas presented. But rather in how they reach their conclusions. I understand that programming techniques are rather subjective, but some logical argument as to why traits are useful would have been nice. Instead, we get opinions. And perhaps this is something that I've failed to demonstrate in past article is that my suggestions on helper objects are not based on wishful thinking, but rather on logical deductions.
I will provide the argument as to why traits are seen as an advantage compared to inheritance alone. What do you know that takes an action and can redirect it to any other object? A switchboard. Switchboards are powerful things. It provides a gateway for connecting different parts of a greater system. The paper in question does accurately state that traits provides for the glue in your software in a much better way than inheritance.
The easiest way to describe these traits is to say that they are similar to interfaces that come with their own implementation, but may list a set of required external methods. Consider it a double ended interface where you select individual methods for each of the externally required pure functions instead of actually providing an all-in-one implementation package. Traits cannot manipulate local state other than with get and setters (through external methods). This supposedly lets one mix and match traits and then select what specific implementation you want for each of the externally required trait methods.
The problem I have with the paper isn't necessarily the ideas presented. But rather in how they reach their conclusions. I understand that programming techniques are rather subjective, but some logical argument as to why traits are useful would have been nice. Instead, we get opinions. And perhaps this is something that I've failed to demonstrate in past article is that my suggestions on helper objects are not based on wishful thinking, but rather on logical deductions.
I will provide the argument as to why traits are seen as an advantage compared to inheritance alone. What do you know that takes an action and can redirect it to any other object? A switchboard. Switchboards are powerful things. It provides a gateway for connecting different parts of a greater system. The paper in question does accurately state that traits provides for the glue in your software in a much better way than inheritance.
OK, that's the good part. What's wrong with it? Plenty. Before I get into that, please note that using traits would be a step up. If you're still reluctant to go to full helper objects (a 70s concept), then at least use traits. It's a step forward back to the 70s (and if you do eventually decide to go all the way to the 70's, you'll finally be able to move into the future again).
Traits are only switchboards for the programmer. They are useless as switchboards at runtime. So be 100% certain that you understand the consequences. Nothing wrong with this. You just need to understand what properties are available when. At design time, you can select what functions you want for the implementation of each trait method. Once they are selected, they remain for the duration of the runtime. The downside of this is that it again tries to define an uncoupled entity by only using methods.
I've said it before and I'll say it again. Encapsulation is a joke. It's a farce. Can you separate data a lot of the time? Sure. But there's a right way to do it. More importantly, you cannot encapsulate ALL the time. That's why encapsulation is a joke. If you have a method, you're going to have arguments. Those arguments are going to have types. And whatever works on those types will have to break encapsulation. On top of that, those types define in absolute terms what a request will operate on. Using methods doesn't reduce coupling, it actually relegates would-be high level constructs (defined as arguments) to become glorified primitive types where their internals MUST be known to all.
Think about it. If you know the internals, then you can make that object interact with another object. But that breaks encapsulation and your object basically becomes a public data structure (which I actually like because I think it's fruitless to fight a losing battle). The other way is to make the internals of other objects known to this argument object. Again, that breaks encapsulation although only one object's internals are used instead of two. The problem is that it permanently couples the argument object to the objects it operates on.
Here's a better way to explain it. Consider objects A, B and C. A and B are the two objects that must interact. C is the group of objects (typically arguments, but not always) used to enable the interaction between A and B.
It is impossible to ensure encapsulation on all three of A, B and C. IMPOSSIBLE. Let's say A requests an action on B. That means A's encapsulation is secure as it can take its internal data and send it along as C as arguments. Unfortunately, this means that A is coupled to B and C. And that B is likewise coupled to C. FAIL! And we didn't even get into the situation of how B was going to use C without breaking its encapsulation.
A point that may have been lost to readers is that if you devolve to primitive types, then you fail. The consequences of devolving to primitive types means that you can never be too far from the lowest common denominator. Primitive types are GLOBAL types known to all. As such, they have no encapsulation to speak of and are great for universal communication because every object knows about them.
Now, do you see where I'm going with this?
You need a global universal data entity with ZERO encapsulation. There is NOTHING in current programming language theory that even discusses communication between objects in a practical matter rather than pushing a biased agenda. By building compound and complex universal data structures used for inter-object communication, you can start moving your code to higher levels. Finally, you don't have to be one step away from built-in primitive types.
But hold on. We don't actually work with raw data directly. No, we use helper objects that know how to manipulate this data. Helper objects are similar to traits, except that they DO operate on raw data. You can merge two or more helper objects to work on the same data (or even merged data from two or more data structures).
Having a helper object isn't simply for convenience. It has a logical purpose. You don't want to be manipulating raw data directly. But neither do you want a universal lock on the meaning of that data. Remember that a primitive type lets each and every object manipulate it in whatever way it wishes. So we need to retain that concept. That's why each object may use whatever helper object it wishes to manipulate incoming or outgoing raw data.
All of a sudden, you've elevated the communications channel from built-in primitive types to compound and complex data types while still providing unique and separate interfaces to that data. What this means is that you no longer need to go back down to primitive types once you have these tools in place. You can code at a higher level. The communication channel is SO much more powerful now. And the supreme requirement that each object be able to view the data as it pleases is retained.
I didn't mention all this to explain the virtues of helper objects (though you may take it that way). No, this was necessary to explain the shortfalls of traits. Traits cannot do any of this. They must stay close to the low level primitive types. Eventually, your implementation will HAVE to devolve to manipulating primitive types. It's the only universal communication types available in your languages, unless you build your own. And that's where helper objects provide an interface to your own custom communication types. Once these are set up, you continue to develop at a high level.
That's the key right there. Interfaces and traits force you to stay close to primitive types. It's too low level. And ironically, using custom RAW data types with helper objects allows you to program at a higher level. This is 100% contrary to what most people would expect or believe.
Write software using objects, interfaces, implementations, mixins and traits, but where you can NEVER touch a primitive type and never use a getter or setter. It's impossible. There is a real logical reason for this. As said before, the only universal types available are primitive ones. So you have to use these at some point for communicating between objects. Unless you start creating higher level communication types that have the EXACT same power as primitive types, you will always be a limited distance from built-in primitive types. There can be no escape. And that means no high level programming for you no matter what you claim.
So forget helper objects and all that for a moment. That was simply used to contrast the power missing from traits. The main proposed advantage to traits is providing this lightweight glue that would enable more freedom of expression. IOW, reusability. Here again, traits falls into the trap of the false singularity. Where everything is X. Nothing is ever only one thing. Can't happen. The universe works in duality. This includes computing.
I said at the beginning that a trait can work as a switchboard. Sounds like the perfect mix and match concept. But there's one thing missing. Duality. You cannot connect two objects of the same makeup. This is a strange notion to most people. But it's 100% inescapable. Two switchboards are useless without wires to connect them. Two functions are useless because their shells are of the same makeup. The way two functions can work together is where the internals of one function calls the other. The insides are of a different makeup than the shell of the function. Duality.
That's not the only duality. Duality is everywhere. For example, functions are quite useless without data. Write me a program where you only use functions and don't have any data. That code won't do anything. None of your functions could take arguments and none of them would return anything. This is the essential duality that Object Oriented Programming has been tirelessly trying to kill without much success.
In order for any two objects to communicate, they need data. It's a shame that most programmers see the function as the glue between objects. It most definitely is NOT. The glue is and always will be the data. Proof of this can be seen with the Internet. Or with telephone networks. Or just talking to another person. Data is king. That's why traits will never be as beneficial as the authors of the linked article are hoping it to be. Traits, like much of everything else in OOP, only ever treats everything as X. In fact, it went as far as to dictate that it does not deal with state data at all.
If you really want to advance as a programmer, try out the A, B, C example on your own. Try and make two objects interact without coupling, without resorting to primitive types and without breaking encapsulation. I can tell you that's a tall order. But try it. What is the best balance for you? You will learn more from this exercise than anything else you've come across.
Then try creating a more powerful construct for communication between objects. When communication channels become more powerful, so does the dialogue. It's a universal truth that I'm amazed has been brushed aside in software development theory. I'm hoping that this explains in enough detail that these suggestions and topics aren't wishful thinking or personal preferences on my part. They are based on seeing the disastrous logical consequences of diminishing one part of the communication process in favor of another. I also hope that I've adequately shown how MORE data means higher level programming. However, make sure to understand that the interfaces must match the level of data. They must BOTH rise together. So you can have the fanciest of interfaces, traits and classes, but if you don't have enhanced communication data structures, then the power you derive from the stated techniques will always be bound to the power of the primitive types available to your programming language. Raise the power of both data communication and the interfaces to that data, and you'll see the true power of symbiotic synergies.
WIDGETIZE
Anonymous # 25. November 2008, 00:57
Your recent articles are reminding me of another old concept called "separation of concerns" which is also quite lost in modern time.
The design for testing crowd is pushing for more separation of concerns where the main concerns that need to be separated are data, logic, and wiring. Data is obvious, Logic is helpers and business logic, Wiring is creation of objects and telling them about each other.
Sean Conner # 25. November 2008, 05:32
I also suspect you are using a different definition of "encapsulation" than most people. I have a HTML parser I wrote (object A) that pulls data from a stream of data (object B). The HTML parser (A) doesn't care how the stream (B) works, just that it can get data from it. I would say that both my objects are pretty well encapsulated from each other, and that changes to the internal implementation of one (the stream object, only now I'm pulling data directly from a network socket instead of a block in memory, for example) doesn't effect the operation of the other. I think you're taking the definition of "encapsulation" to an extreme.
Vorlath # 25. November 2008, 16:08
Awesome! You've made my day because that's exactly what I'm talking about. That's the reference to the 70s I was talking about.
To spc476:
You want an example? Ok. I'll show you the heart of Project V. I've cut out many of the class definitions and will only show the NEntry class. This is data only. In my "helper object methodology", you essentially need 5 parts. You have one raw data structure. You usually have two or more helper objects. And then you have two or more other objects that uses those helper objects. Note that if you only have one of each is fine as it's already setup for expansion. I'll only show the raw data and the helper objects.
Say you have object A and B that want to communicate. Then A would use helper class X while object B would use helper object Y. The helper objects would both operate on data N. Also, it often happens that X and Y are the same helper object.
A->X->N->Y->B
edit: I should mention that there are three ways to use this.
1. A can call B directly with N as a parameter.
2. A higher level object (preferably another helper object) will request A to do something and then request B to do something on the same data.
3. Object A can make a repository available through public externally available helper objects. So third part objects would use these helper objects internally to access the repository. This happens with socket engines where you store data in buffers and external objects process that data (with those helper objects) into messages that can be acted upon.
The following class is an example of raw data N. This is what gets passed around.
This class looks busy, but it only has 6 elements. Those elements are:
1. Name
2. Unique SHA256 ID
3. Forward sets
4. Reverse sets
5. Flags
6. Optional Value
The methods found within this class are methods that do not need external data type other than global ones. Note that UnicodeString is now global. IOW, using only global types that it uses internally, the NEntry methods can perform the requested action all on its own without the use of another object.
Another note is that these actions should be extremely simple and should be functionality that would be required everywhere. More complicated functionality would be found in helper objects which I show here.
(Note: TypeList is a forward set and RTypeList is a reverse set.)
That is generic functionality used by my type system. However, you can use NEntry's in any way you wish. So you can build your own helper objects to have custom and specific uses. Which brings me to my next helper object.
Take a look at the Flatten() method. Looks innocuous. What this method does is create as simple a hierarchy as possible for use by the type comparison module. Look up Yegge's Universal Design Pattern for why I would do this. It basically compresses all overriden values into a single location. However, it's much more complicated than what Stevey describes because I have added flexibility. Sets (or name/value lists) can have namespaces.
These compressed type hierarchies are called XTypes. That's their meta type anyhow. These have custom functionality that I COULD have put in the PVHelper class above. But there's no need for XTypes anywhere but in the compiler section of Project V (mainly type comparisons). But lately, I've started using it in the GUI to retrieve information because XTypes have information readily available.
It would not make sense to put this in the main PVHelper class because XTypes have specific members and properties that only XTypes have. Using these methods (or even making them available) to plain NEntry's would not be wise and could corrupt your data. So if you create an PVXType helper object, you would only do so if you're working with XTypes.
It doesn't end there. NEntry's are also used to store values. But I'm not going to directly manipulate data structures that can hold anything at all. Remember that values can hold things that I cannot predict, so using something like a Variant is no good. What you do is create a helper object for each value type.
So I've shown about 7 helper objects. I've got a few more and I'll be adding more as I add more built-in types. And what I like is that I can add functionality in a group of methods that are similar in nature.
Note the Create() method. It's one of the few methods that are NOT virtual because each value type may need different information. Sometimes, it already knows much of the information needed. So it will actually use the main PVHelper's Create() method to start off and then fill in its own data.
Here's the implementation for Create() for creating an Int32 value.
All those methods are helper methods from the PVHelper class.
On this blog, I've shown plenty of examples that use helper objects. I know I've shown the socket buffer class. I can't remember the other ones.
Anyhow, different systems (like the type comparison module) will now use the PVXType helper class internally. But what is passed to it is a NEntry object (or objects). NEntry is now a global data structure used for communication between different modules. With helper classes, I don't need to go back down to only using primitive types. So I'm able to continue programming at a higher level. Other parts of Project V use similar techniques.
Oh yeah, as a side-note, I just recently added ID and String interning. NEntry's automatically have this functionality even if you access it directly. That's why I really can't live without properties. But that's another story completely. I just wish C++ had properties as a standard feature.
Vorlath # 25. November 2008, 16:54
See how I go against this notion in NEntry:
I'm fairly certain this goes against encapsulation principles. And I'm proud of that fact. The other way is a losing battle.
(emphasis mine).
That last part is the kicker. Does it devolve back down to built-in types? That's my point. The only way to NOT break encapsulation is to go back down to built-in types. But is it really encapsulation if you can retrieve data from it? The data returned must be understood and you'll have to break its encapsulation to use it... unless you use built-in types. And you end up back to low level programming.
Encapsulation isn't really the issue though. The issue I'm talking about in this blog entry is the ability to elevate communication between modules which enables more power and flexibility in creating and managing software.
Here's the $64,000 question. Do you use any data types other than built-in ones for communicating between objects? Note that simply sending a third object is not enough. How do you communicate with ANY object? If you send a third object, how does that object and the recipient object talk to each other? Continue until you actually make two objects interact directly. I bet it devolves to built-in types. It must and that means encapsulation has been broken.
I'm not taking encapsulation to an extreme. I'm talking about what happens in an implementation to make two objects interact. You're going to have to break open one of the objects at some point. There's no avoiding it. So I don't fight it anymore. If you end up breaking open objects anyways, then having bare objects is no longer taboo. All people have been doing is wrapping raw data just to be unwrapped at a later point. I simply expand on this concept and do it right. What freaks most people out is that I dispense with the wrapper, yet it has absolutely no use in normal programming anyhow other than to complicate matters.
However, dispensing with the wrapper means that different systems can see the data in a different manner. That's how real life works. That's how two people can see the same object differently. It's a VERY powerful concept.
Vorlath # 25. November 2008, 21:49
I suppose you could look at my "helper object methodology" as a way to provide encapsulation from argument lists or parts of it (between higher level systems) by providing more generic and uniform tools. Those tools would break the encapsulation of any objects it manipulates, whereas in the past, the main entity would directly break the encapsulation of the items in the argument list. Just accessing the argument list itself could be considered breaking encapsulation.
It's kind of funny. We have raw data in argument lists all the time. But if we try to use a data structure instead of an argument list, people tend to want to encapsulate it and it causes no end of aggravation. I simply leave it bare and use helper objects.
I should thank you for getting me to consider this point of view. I think it's a failure on programmers' part to notice when a data structure is simply an aggregate argument list (or parts of it). And an aggregate argument list, even in data structure format, should remain raw. I wonder how many times people write full OO objects when those objects are simply data used for communication between other higher level objects.
Yeah, this picture is actually quite interesting. With argument lists, both the sender and receiver can manipulate the individual items in any way they wish. So custom helper objects allows the sender and receiver to retain that flexibility while at the same time provide separation of concerns and a good level of encapsulation from the raw data itself.
I'll have to give this some more thoughts. Anyways, thanks again. This is exactly why I write this blog.
Sean Conner # 27. November 2008, 20:03
I started noticing this back in the early 90s with the differences between Windows and AmigaOS. The AmigaOS (at least for graphical operations) you tend to fill in a large structure and pass that to routines, whereas with Windows, you tend to pass in a large number of parameters (which I felt would have been better passed in a structure or structures).
It wasn't until just a few years ago as I was reading up on VAX Assembly language did I have an "Aha!" moment. The VAX has two instructions for calling a generalized subroutine, CALLS and CALLG. With CALLS, you push the parameters onto the stack, whereas with CALLG you point to a static structure in memory for the parameters. The called routine doesn't care how it's called since it gets a pointer to the parameters regardless of how it's called.
It wasn't much of a step from that to realizing that passing a structure, passing arguments or passing a message are all pretty much the same thing. Unfortunately, there are no languages that take advantage of this.
Sean Conner # 27. November 2008, 20:36
Then again, I don't do C++ (nor do I have an interest doing C++).
Back in the early 90s I implemented my own lanaguage (interpreter) that had user definable types (and operator overloading, but that's beside the point) and I actually used it in a class project at school. The project was a Unix shell, and it only took maybe an hour or two to implement a new type: a Unix command (either a single program, or several strung together via pipes). You could manipulate a Unix command much like you could any other type---store them in variables (not the result of running the command, the command itself), string them together, etc. Worked flawlessly too.
I never used it though. The language syntax was just too annoying for my own tastes (the language was based on Forth, and I still have this love/hate relationship with Forth).
Vorlath # 28. November 2008, 20:35
I can see why the PVHelper looks like a wrapper, but it has access to other things than NEntry. It has access to the type comparison module and to other tools. If I were to really expand on this idea, I could tell PVHelper which settings I wanted (as in what type comparison module I want and what interning modules I want to use). And perhaps I should have used a different example where the data is more diverse. NEntry's act as different datatypes, but that's not apparent by just looking at the list of methods. It's impossible to completely wrap all of NEntry's functionality. There'll always be something new that you can do with them. So helper objects provide the perfect way to add more functionality without breaking what's already there. With a wrapper, I'd have to change the existing API.