Skip navigation.

The whole Enchilada

A sequence of thought

Changed division semantics

I've changed the (indexing) semantics of division to allow for all kinds of list slicings.

The new division semantics may appear to be strange at first but is carefully crafted to make it possible to do an efficient transpose of a list.

Let's say we want to transpose the following '3x4 list':

[1; 2; 3
 4; 5; 6
 7; 8; 9
10;11;12]

To do this we first right-multiply the list with the number of columns (3):

3 [ 1; 2; 3
    4; 5; 6
    7; 8; 9
   10;11;12] *

36 [ 1; 2; 3
     4; 5; 6
     7; 8; 9
    10;11;12
     1; 2; 3
     4; 5; 6
     7; 8; 9
    10;11;12
     1; 2; 3
     4; 5; 6
     7; 8; 9
    10;11;12]

After that, we left-divide the result with the number of columns:

36 [ 1; 2; 3
     4; 5; 6
     7; 8; 9
    10;11;12
     1; 2; 3
     4; 5; 6
     7; 8; 9
    10;11;12
     1; 2; 3
     4; 5; 6
     7; 8; 9
    10;11;12] 3 /

giving us the transpose we want:

36 [ 1; 4; 7;10
     2; 5; 8;11
     3; 6; 9;12] 1

To understand the new division semantics, it is best to picture it as being a digital line-drawing algorithm with x's being the pixels of a line.

The previous version of division would draw the following line:

   [ x; 2; 3
     x; 5; 6
     x; 8; 9
     x;11;12
     x; 2; 3
     x; 5; 6
     x; 8; 9
     x;11;12
     x; 2; 3
     x; 5; 6
     x; 8; 9
     x;11;12]

while the new version draws the line from top-left to bottom-right:

   [ x; 2; 3
     x; 5; 6
     x; 8; 9
     x;11;12
     1; x; 3
     4; x; 6
     7; x; 9
    10; x;12
     1; 2; x
     4; 5; x
     7; 8; x
    10;11; x]

What makes the new division interesting is that the same trick can be achieved on huge lists. For instance the following code transposes a 50000x50000 matrix:

50000 2500000000 ~ * 50000 /

producing the result:

125000000000000 [0;50000;100000;150000;200000;250000;300000;
350000;400000;450000;500000;550000;600000;650000;700000;750000
 ... {2500000000} ... 2499299999;2499349999;2499399999;2499449999;
2499499999;2499549999;2499599999;2499649999;2499699999;2499749999;
2499799999;2499849999;2499899999;2499949999;2499999999] 1

You can find the newest version (0_5_5) on the Enchilada website.

More efficiency

Fixed a *lot* of bugs this weekend and refactored certain operators to be *much* more efficient.

I've also written a cheat sheet. Here you can find imperative/applicative constructs and translate them into Enchilada constructs. I've also updated the documention with seperate sections. Also notice that iota is redefined to start indexing at 0 (thanks to Stevan Apter).

Most bugs where releated to (lazy) alpha-conversion (not surprising). But I consider the efficient implementation of multiply and division the most interesting piece in this release (v0_5_4). Consider the following code:

100000000000000000 ~ 5000000000000000 / [0;5000000000000000;10000000000000000;15000000000000000 20000000000000000;25000000000000000;30000000000000000 35000000000000000;40000000000000000;45000000000000000 50000000000000000;55000000000000000;60000000000000000 65000000000000000;70000000000000000;75000000000000000 80000000000000000;85000000000000000;90000000000000000 95000000000000000] 1

This divides an (iota) list of size 1*10^17 with size 5*10^15 instantly without cheating!

Here is another one:

1000000 ~ 10000000 ~ 100000000 ~ + + 4440000 / [0;3440000;7880000;2320000;6760000;11200000;15640000;20080000; 24520000;28960000;33400000;37840000;42280000;46720000;51160000; 55600000;60040000;64480000;68920000;73360000;77800000;82240000; 86680000;91120000;95560000] 1

Give it a try yourself and manipulate the biggests lists you can imagine.

Lambda the ultimate?

Enchilada now has full lambda abstractions, disguised as eager macros. But alas, with lambda abstractions I had to implement all the hairy stuff, such as possible name capturing.

Name capturing is one of the reasons what makes Lisp macros so hard to understand. But with eager macros I made sure that this problem doesn't occur. Name clashes are automatically detected by Enchilada and resolved via alpha-conversion:

{a==[a] {c b=c b}} (no name clash)

{a=={c=c [a]}}

{a==[a] {a b=a b}} (name clash)

{a=={aa=aa [a]}}

{a==[a] {a aa b=a aa b}} (double clash)

{a=={aaaa aa=aaaa aa [a]}}

So a clashing name gets doubled, again and again, until there are no more name clashes with free variables.

Enchilada lambdas only operate on lists (because they are monadic operators). But because Enchilada lists are sequences of expressions, the manipulation of lambdas by other lambdas is not far away. For instance, compare the lazy version:

1 2 [{a b=b a}] {a b c=[a] [ b] c ^}

1 2 {aa bb=[aa] [bb] [{a b=b a}] ^}

1 {aa=[aa] [2] [{a b=b a}] ^}

[1] [2] [{a b=b a}] ^

[1] [2] {a b=b a}

[1] {a=[2] a}

[2] [1]

With the eager version:

1 2 [{a b==b a}] {a b c==[a] [ b] c ^}

1 2 {aa bb==[bb] [aa]}

1 {aa==[2] [aa]}

[2] [1]

My feeling is that Enchilada macros are easier to read then lambda abstractions and that they make a better Lisp macro because they don't allow name capturing.

You can find the newest version here. Then run the following command:

java -cp enchilada_v0_5_2.jar org.enchilada.v0_5.impl.Test

minumum redefined

Fixed some small bugs related to chop, and I've rephrased the minumum operator to be a filtering operator.

The new minimum operator is targeted to be used to filter things on precedence.

Let's say you want to select a list of numbers that have less or equals precedence then another number:

8 ~ [4] * &

[1;2;3;4;5;6;7;8] [4] * &

[1;2;3;4;5;6;7;8] [4;4;4;4;4;4;4;4] &

[ ; ; ;4;5;6;7;8]

Then if you chop the remaining list you get the final result (empty expression omitted):

[1;2;3;4;5;6;7;8] [4] * & \

[1;2;3;4;5;6;7;8] [4;4;4;4;4;4;4;4] & \

[ ; ; ;4;5;6;7;8] \

[4;5;6;7;8]

You can download the new version here

Then execute:

java -cp enchilada_v0_5_1.jar org.enchilada.v0_5.impl.Test

New release: v0_5

I've just finished a new major release v0_5.

A lot of things have been changed since the last release. The macro operator proved to be so versatile that I've dropped a whole range of typical stack operators. Duplicate, enlist, swap: they are all gone.

I've also included three new operators, force, chop and reverse and some other goodies. Next to that, I've written a whole new document that explains Enchilada semantics in much more detail.

You can download the latest version here.

Then execute:

java -cp enchilada_v0_5.jar org.enchilada.v0_5.impl.Test

Macros and shuffling

One of the big complaints about concatenative languages is that certain computations require a lot of 'stack shuffling'

For instance, one can re-arrange 4 operands with just dup, enlist and lift. But for a lot of programmers such re-arrangement programs look very much like noice.

Stevan Apter and Billy Tanksley have had similar concerns on the concatenative list. What they came up with is the shuffle notation. Inspired by their notation I've created a new construct that is even more powerful: the macro operator.

Let's start with some small examples to get an idea of how one could use the macro operator:

dup ==

1 {a=a a}

1 1

drop1 ==

1 {a=}

.

drop2 ==

1 {a}

.

swap ==

1 2 {a b=b a}

1 {a=2 a}

2 1

enlist ==

1 {a=[a]}

[1]

Note that the macro operator acts like a (curried) monadic operator, taking one operand at each reduction step.

Together with the macro operator, I had to introduce symbols. Symbols are, like in any other language, nothing more then sequences of characters. With the new symbols however, we have to redefine all alternative names for the operators, ie they must be prefixed with a dot, for instance:

1 2 .cat

1 2 +

3

5 .enlist

5 ^

[5]

The macro operator is made out of two parts, seperated by ='. On the left we have a list of symbols, and on the right we have a expression that contains symbols, matches those on the left.

It's illegal to have an empty left or to have duplicate symbols on the left. It's also illegal to have symbols on the right that don't match symbols in the left. The following examples are all illegal:

{}

{a a=a}

{a=b}

Here are some more elaborate examples that will show the expressive power of macros:

1 2 {one two=one one .cat two two .cat .cat}

1 2 {one two=one one + two two + +}

1 {one=one one + 2 2 + +}

1 1 + 2 2 + +

2 2 2 + +

2 4 +

6

[{c};{b}{a}] 0 .order

[{a};{b};{c}]

1 2 3 {a b c=[c;b;a]}

1 2 {a b=[3;b;a]}

1 {a=[3;2;a]}

[3;2;1]

You can download the latest version here

Then run the following command:

java -cp enchilada_v0_4_2.jar org.enchilada.v0_4.impl.Test2

Sticky operators and operands => dataflow

Today I want to share my thoughts on parallism in concatenative languages. But I'll do this not by argument but by adding a new feature to the Enchilada language: sticky operators and operands.

Let us first consider the following dataflow pipeline. Say we want to add 1 to a list of numbers (2,3) and then multiply the results by 2:

----> 1 + ----> 2 * | ---->

2-3-> 1 + ----> 2 * | ---->

--2-> 3 1 + --> 2 * | ---->

----> 1 + -4--> 2 * | ---->

----> 2 1 + --> 4 2 * | -->

----> 1 + -3--> 2 * | ----> 8 (parallel step)

----> 1 + ----> 3 2 * | --> 8

----> 1 + ----> 2 * | ----> 6 8

Notice how the pipeline stages are fixed. Now how could we express such pipeline in Enchilada? By prefixing a operator or operand with the (') symbol.

2 3 '1 '+ '2 '* '|

2 '1 '+ 4 '2 '* '|

'1 '+ 3 '2 '* 8 8 '|

'1 '+ '2 '* 6 6 '| 8

'1 '+ '2 '* '| 6 8

By the fixing operands and operators, one could image enormous parallel pipelines that can be reduced concurrently.

Still, I'm not sure about the notation/syntax of stickiness. Any suggestions to improve on that would be very welcome.

Dual lists and permutations

This minor release takes a gaint step. Dual lists are fully implemented and follow the syntax as proposed by Sjoerd Visscher.

Next to that I've implemented the turn(~) operator that allows the turning of dual lists. Additionally, the order(<) operator and the min(&) operator are introduced that allow the sorting, ordering and comparison on numbers, lists and expressions.

Because of the new operators, I've re-assigned the drop(~) operator to ` and I have replaced the pop(<) operator by the new order operator.

Now let's introduce the new dual list syntax. Every element of a dual list is a pair of expressions. Both expressions of the pair are seperated by the = symbol. For instance, the following expression reduces to:

[1=2;3=4] [5;6;7] [=8;=9;=10] + +

[1=2;3=4] [5;6;7;=8;=9;=10] +

[1=2;3=4;5;6;7;=8;=9;=10]

But if we replace the + operator with the max(|) operator we get:

[1=2;3=4] [5;6;7] [=8;=9;=10] | |

[1=2;3=4] [5=8;6=9;7=10] |

[1 5=2 8;3 6=4 9;7=10]

Ordering can be achieved with the dyadic order(<) operator. When two lists are ordered, all expression pairs of both lists are compared and sorted in order:

[4;3;2;1] [5;7;6;8] <

[1;2;3;4;5;6;7;8]

[1;3;4;5;6] [2] <

[1;2;3;4;5;6]

[2=3;1=2;2=2;2=1;1=1] 0 <

[1=1;1=2;2=1;2=2;2=3]

Elements in an expression are compared by following a fixed precedence:

  1. dyadic operators take precedence over monadic operators.
  2. monadic operators take precedence over lists.
  3. operators take precedence based on the ordering of their syntactic symbols.
  4. lists take precedence over numbers.
  5. bigger numbers take precedence of smaller numbers.
  6. non empty expressions take precedence over empty expressions.

The precedence rules can be succinctly stated by the reduction of the following expression:

[4;3;[3];[4];!;@;#;$;%;^;&;*;-;+;:;";|;\;~;`;<;>;?;] 0 <

[;3;4;[3];[4];\;`;";^;$;!;-;#;~;+;?;@;|;&;%;*;<;>;:]

When we combine ordering with dual list, a lot of interesting things can be done, most notably the permutation of lists and operands:

10 20 30 40 ^>>> ~ [4;3;2;1] | 0 < ~ !

10 20 30 40 ^ > > > ~ [4;3;2;1] | 0 < ~ !

10 20 30 [40] > > > ~ [4;3;2;1] | 0 < ~ !

10 20 [30;40] > > ~ [4;3;2;1] | 0 < ~ !

10 [20;30;40] > ~ [4;3;2;1] | 0 < ~ !

[10;20;30;40] ~ [4;3;2;1] | 0 < ~ !

[=10;=20;=30;=40] [4;3;2;1] | 0 < ~ !

[4=10;3=20;2=30;1=40] 0 < ~ !

[1=40;2=30;3=20;4=10] ~ !

[40=1;30=2;20=3;10=4] !

40 30 20 10

You can download the latests version here. Then run the following command:

java -cp enchilada_v0_4_1.jar org.enchilada.v0_4.impl.Test2

Reduction order and non-determinism

One interesting Enchilada feature is that Enchilada expressions can be reducted independently and in any order at any recursion level.

For instance, the following expression(s) can be reduced like this:

[1 2 + [5 6 +] !] ! +

[1 2 + [11] !] ! +

[3 [11] !] ! +

[3 11] ! +

3 11 +

14 +

or it can be reduced alternatively:

[1 2 + [5 6 +] !] ! +

[3 [5 6 +] !] ! +

[3 5 6 +] ! +

3 5 6 + +

3 11 +

14

or in parallel:

[1 2 + [5 6 +] !] ! +

[3 [11] !] ! +

3 [11] ! +

3 11 +

14

But there are two exceptions to this rule: the equals(?) and hash($) operator must always have their operands in a canonical (un-reducable) form. Otherwise computations could work out non-deterministically.

To see how this problem might occur consider the following run:

[1 2 +;5 6 +] [3;11] ?

[3;5 6 +] [3;11] ?

[3;11] [3;11] ?

1

alternatively, if we would allow non-canonical forms to be tested for equality:

[1 2 +;5 6 +] [3;11] ?

0

So if we test two lists for equality, all expressions that are contained by those lists should be reduced (recursively) to their canonical form. The same applies for the hash.

But one could image a version of Enchilada that allows such non-determinism. Then the order of reductions can drive computations to alternatives but without explicit controlling such alternatives from within the computation. One could even decide to build McCarthy's amb operator on top of the equals operator.

release 0.4

This release includes a fresh new parser that can accept both vanilla and canonical input (and a lot more).

The first noticable additions is that the output of the interpreter can be put in several modes (thanks to Stevan Apter). The default mode is without tracing and with simple syntax.

The following example shows that the parser can accept the inter-mixing of vanilla, keyword and canonical syntax:

> [()()] [();()] [cat;*;div;divide]

2 3 [+;*;/;/]

Slightly the same example but now with tracing and medium verbosity:

> \otm [()()] [();()] [cat;*;div;divide] !

0: 2 3 [cat;mul;div;div] lft

1: 2 3 cat mul div div

2: 5 mul div div

You can persistently set tracing (or any other mode) if you don't posfix the mode with an expression.

> \ot (o)utput (t)racing

> \ovt (o)utput (v)erbose (t)racing

> \om (o)utput (m)edium-verbosity

> \ost (o)utput (s)tandard (t)racing

If you want to see what are the possible symbols, keywords etc, you just create syntactical nonsense, giving you all possibilities:

> (

org.enchilada.v0_4.impl.io.ParseException: Encountered "(" at line 1, column 1.

Was expecting one of:

<EOF>

"[" ...

<NUM> ...

"\n" ...

"\\o" ...

"_" ...

etc, etc, etc

The new release also (slightly) redefines multiplication and division so that is it more in line with the Cartesian product. I've updated the documentation accordingly, together with the discussion of negative signed lists.

If you want to run the latest version please download the jar.

Then run the following command-line:

java -cp enchilada_v0_4.jar org.enchilada.v0_4.impl.Test2

December 2009
S M T W T F S
November 2009January 2010
1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30 31