completely fucking fucked

I’m developing a game in Ruby using SDL. The name of the game is completely fucking fucked. The idea is that I want to be able to create a game architecture that lets me change whatever I want whenever I want. This should allow me to cook up some truly post-modern gaming experiences. Walking on the walls, walking on air, fast color changes, tempo changes, whatever I want, whenever I want.

So how is the game engine that I’m building different from all other game engines?

Well, the motivating idea is that if we loosely model a game as a set of interacting finite state machines then what we have to do (at most) is describe how all the states interact. That’s about n^2 things you have to specify (where n is the total number of states). Not so bad, especially considering that you won’t usually have to specify them all.

The cool thing about writing games this way is that you don’t have to worry about anything other than what you are looking at at the moment. The structure is completely local.

For instance, I want to add a block object. I already have a mario object. What do I have to do to give the block object its behavior?

Well, I have to prepare the following:

Predicates for determining whether mario enters from the bottom, from the top, from the left, or from the right.
Predicates for determining whether mario is in the block.
Operations who function based on predicates above. For instance:

if (enteredFromBottom(mario)) and isInsideVertical(mario) and isInsideHorizontal(mario)) then
putBottom(mario)
makeFall(mario)
end

The idea being that I have to write the above once and then never, ever, EVER, think about it again. Or, if this is not the case, I have to update the framework so that it is the case.

Another THING motivating the design is the desire complete informational transparency. What do I mean?

For instance, in the model (where all the entity state is contained) I should have access to the keyState (what keys are up and what keys are down). Now, this seems to violate modularity (shouldn’t we want to have the model and the controller separate, you ask?) The answer is yes and no.

Yes, if we had just one model and one controller then we would want them to be separate. But when I think about the game I really think about it as a large number of models + one controller.

I mean, why can’t I have a model whose purpose is to describe the low-level model+controller system? This model would describe what happens, for instance, when a state-transition occurs. Obviously information related to state-transition (changes in the model’s representations) is not itself an aspect of the model. Thus it must be meta-information. Similarly, the controller is interested in processing key events – it should not be concerned with the current keystate – this, too, is meta-information. Therefore, since we have two nice pieces of meta-information we can write a model to use them. BadaBAM.

What benefits does informational transparency give you? Well for one thing it makes shit way easier to write. If you want a value you just get the damn value. You don’t have to ask around for the damn value.

Another design decision is to not use threads/message systems. I’m doing this in ruby and threads will only slow this motherUFF down as well as adding infinite complexity. I model animations as functions which know how to update even in the face of irregular update requests (parameterization, baby).

This post was not lucid but I don’t care. Hope you got something from it. I should be releasing a pre-alpha of CFF within the next few days. Does anyone even fucking read this?

new examples of usefulness of meta-programming compiler

example 1:

meta: ExpandComparators

if a < b < c < d < e
puts “a < b < c < d < e”
end

meta: End

becomes:

if ((a < b) && (b < c)  && (c < d) && (d < e))
puts “a < b < c < d < e”
end

obviously useful if you are doing this sort of comparison a lot.

Another example:

ruby compileMeta.rb –fixToS  myProgram.rb (this doesn’t even need to be run on a “rbm” – ruby meta – file)

will turn, for example:

puts “testVal: “  + 1 + “!=” + 2

into:

puts “testVal: ” + 1.to_s + “!=” +  2.to_s

—

Now, that last example might not seem like much, but consider:

1.) It is an unambiguous error (i.e. there is nothing else that could POSSIBLY be going on)

2.) I make it a lot

3.) This can be just one of many possible unambiguous and common errors that I make – I can have a flag like –fixMyErrors so that the call to the ruby interpreter becomes:

ruby compileMeta.rb –fixMyErrors myErrorFile.rb myProgram.rb

myErrorFile.rb can also do more intelligent things like check parens.

—

One final possible counter-argument is that this functionality could be replaced with a good IDE. Nonsense! First, show me a good IDE. Second, now that you’ve shown me Emacs, does it have a good, easy-to-use way of implementing these kinds of changes? Is it ruby-specific? Oh, I have to know Lisp? So I have to learn Lisp to write Ruby?

Anyway, I am continuing to try to justify this time-sink, as you can tell.

case study: string tokenization

Ok, so I’m writing this meta-ruby compiler but I’ve returned to ground zero because my tools were not sharp enough. In particular, I want transparent representations. Here’s an example:

Let’s say you have the following string of Ruby code:

a = “why, I ‘never’ \\\” would \\\” do ‘that” +   “said’ the man”

My goal is to create a “data structure” containing “tokens” so that I have a more efficient way of accessing information about this string. For instance, here are some questions I might like to ask:

Where are the string literals?
What variable is being assigned to?
What operators (if any) are being used?

Etc.

The number of questions I could ask about the string are infinite, but it seems like there is a “sweet spot” which it would be nice if I could access in O(1).

This brings me to the idea of a new data structure – the “flagged” list. It’s just a list, but each element has a “flag”. So the tokenized version of the above might look like:

["a"->symbol, " "->space, "="->operator, " "->space, "why, I 'never' \\\" would \\\" do 'that"->doubleQuoteContent, " "->space, "+"->operator, "   "->space, "said' the man"->doubleQuoteContent]

The goals for such a representation are:

1.) Fast query

E.g. I can say “flaggedList.getAll(“space”)” or “flaggedList.removeAll(“doubleQuoteContent”)” or “flaggedList.getAllAfter(“=”)” or whatever. The entire point is to make the structure extremely flexible.

2.) Fast extension.

Ok, this is a subtle point, but it’s important because I might want to build yet another compiler on top of my compiler. For instance, I might want to do something KERAZZY like make operators delimiters (so that everything between operators becomes absorbed into a token). Needing to create a new class for each type of token will just hurt my little old hands. Seriously, though, it’s better to just add a line to a list somewhere (operatorDelimiter) instead of creating a whole new class.

I don’t know why programmers are so afraid of flagging with strings. Maybe because it’s error prone? Bah humbug.

3.) Information Persistence

Another goal is to persist all information. None of the detail of the first line is lost in its flagged representation (e.g. even spaces are maintained). I can drop detail out if I want, but it is not dropped by default.

Anyway, there are other reasons, but they should be obvious. Or you can think of them. Whatever! The structure just appeals to me intuitively! Is it isomorphic to some other structure?
—

I guess the high-level claim here is that the difference between expressing something via one line of code vs. via two lines of code is enormous if you are planning on expressing that thing many many times. The difference between dropping a little information and dropping no information is enormous if you intend to extend the system ad infinitum. The difference between flagging with strings and flagging with classes is enormous if you want to maintain cognitive state (i.e. if you don’t want to forget why you wanted to add a flag while actually adding it).

You could summarize the design philosophy behind this approach to tokenization as: “Always sugar. Always.”

typing and fast conversion

Let’s say that you have the following situation:

You have an object that you want to model. Let’s say it’s a cube.

Now, what’s a cube made of? Well, vertices, edges, faces, “content.”

So you go to work and you implement your cube.

Later you find out that you need the geometric information related to the various parts of your cube. You now need to talk about Points, Line Segments, Bounded Planes, etc. Further, you need to be able to do “operations” no the geometric space which means you need heuristic characteristics: Vectors, orientation, “inner vs. outer,” etc.

Later you find out that you need to import data in a standard array-like format (cube described by array of “faces” which are really just arrays of “points” which are just arrays of numbers)

You might need to switch between each of these three domains fairly often. What the hell is the solution? Is there any programming language in which you can do this easily?

It seems like our ability to express how types are related is woefully, woefully, lacking. Now, you could say “make your Vertex a point.” But what type of point? An n-dimensional point? What if I want to write a 3D geometry engine? Then I have to think about how to relate the n-dimensional point class to the 3D point class.

The issue is that when we talk about types in everyday life we use vastly more relations than is-a. For instance, I could say:

“A vertex of a graph is an n-D point when the graph is embedded in n-D space.”
-or-
“A face is a bounded plane except that the orientation of the edges bounding a face matters.”

Computer science can be seen as “implementing math.” Imagine how stilted math would be if we had to perform these tedious conversions all the time. The crux is the following:

Conversions between types is fast and trivial when types are bound to computational contexts.

If I transition from the “graph theory” context to the “geometry” context then conversion between primitive types should be trivial (read: automated). I should be able to indicate which context I’m using.

continuation * nontrivial

So that previous post kind of set me to work on developing a set of meta-programming tools for ruby. This is very experimental. Really I’m just having fun writing my first compiler which tokenizes ruby with extra symbols and implements changes to the code following a set of rules. The goal is to set it up so that one can write “meta-programming strategies” (kind of like LISP macros, but different in a way that I’ll explain).

Here are the two examples I’ve implemented so far:

First, a string literal reverser:

meta: StringLiteralReverse

a = “abcdef”

b = “he said \”yo!\”"

endMeta

outputs to

a = “fedcba”

b = “\”!oy\” dias eh”

Cool, huh?

Next example is more sophisticated and highlights the “power” of these types of mini meta-programs w/r/t Lisp macros:

meta: BindingsCapture

a = 1

b = “hello!”

c = [what's your name?]

endMeta

meta: BindingsCapture

eachBinding(sym)

if (sym.class == Array)

sym << “yo!”

elsif (sym.class == String)

sym += “yo!”

else

sym += Char(‘y’) + Char(‘o’)

end

symCopy = sym

endMeta

outputs to:

a = 1

b = “hello!”

c = [what's your name?]

if a.class == Array

a << “yo!”

elsif a.class == String

a += “yo!”

else

a += Char(‘y’) + Char(‘o’)

end

aCopy = a

etc.

I’m totally eschewing the normal Compiler methodology (Parser -> Tokenizer -> Syntax Tree -> etc.) because really all I’m doing amounts to is implementing a language for writing FSM’s which use regular expressions to search in code. One of the reasons I want to write this ruby extension is that I like writing programs which use globally defined functional predicates. For example:

$startsWithColon = Proc.new {|x| x.class == String && x[0..0] = “:”}

$endsWithPeriod = Proc.new {|x| x.class == String && x[-1..-1] = “.”}

But I also like writing combination methods, for example:

$andPreds = Proc.new {|*x| Proc.new {|*y| val = true ; x.each do {|proc| val = val && proc[*y]}}}

But let’s say I’ve made a lot of the above predicates (startsWithColon, endsWithPeriod, etc.) and I want to form the and predicates (so that I can easily refer to, for instance startsWithColonEndsWithPeriod). This obviously requires some scripting (since it’s just a matter of calling the functions correctly. I want to be able to write something like:

meta: CombineProcs

setProcs(procs) // startsWithBlah, endsWithBlah

setCombinators(combinators) // and, or, xor, etc.

writeCombinations // writes out the combinations in the form startsWithBlahAndEndsWithBlah, etc. – will write lots

endMeta

You might ask what is the value with this. I say there are two values:

1.) Ease of reference (it is annoying to combine predicates, especially in “pseudo-functional” languages like Ruby)

2.) Verifiability (because it is done automatically I can rest assured that only the basic definitions need to be correct)

Then we can start combining things. For instance, how do we get the “procs” and “combinators” variables above? Well, we simply write a meta flag around our (named) predicates and they are automatically collected (nice!) E.g. we might write something like:

meta: BindingsCapture -> procs // puts the bindings in a named array

$startsWithBlah = Proc.new blah

$endsWithBlah = Proc.new blah

endMeta

etc.

Since I eschew the normal methodologies associated with tokenization (which creates the false dichotomy of symbol/operator) I am free to play games with variable names and crap.

I think that this sort of play has been largely missing in the programming world and I will tell you why – there is a strong understanding of the computer-related elements of semantics/syntax, but there is a somewhat weak understanding of the relation of those elements to their human-oriented correlates. How does a variable assignment correspond to what is going on in my brain? What is the semantic value of a variable name? Etc. Time will prove how useful (or unuseful) this approach is.

Trivial * Impossible

Here’s some code I wish I could write:

vals = []

meta: InputValues -> vals {

bind(a, 1)

bind(b, 2)

bind(c, 3)

}

print vals // [:a, :b, :c]

valsPP = []

meta: ModSymbol -> valsPP {

vals.each do |val|

newSymb = appendSymbol(val, “PP”)

bind(newSymb, eval(val) + 1)

end

}

puts valsPP // [:aPP, :bPP, :cPP]

puts aPP + bPP // 5

Too bad every programming language sucks. If I weren’t at work, this post would be longer.

What Problems Can Be Solved by Programming?

Programming – the profession, or even the social phenomenon, since much programming is done non-professionally by open source volunteers – has stabilized into a certain relationship between human programmers exercising expertise and creativity, and programming constructs which automate certain aspects of machine instruction. The most important question in programming right now is why this relationship has stabilized in this particular place, and how it might be moved to a new equilibrium.

To begin with I would like to back up my claim that the relationship has stabilized. Any comment on the programming phenomenon as a whole must of course be somewhat guarded. It is too large an arena to comprehend entirely. However, a certain level of confidence in its basic status can be achieved, based on the following observations:

1) Productive or useful innovations in programming can be and are disseminated quickly, often through open-source. Distributed version control is one recent innovation which has become significantly used within a short time of first being well implemented. This pattern can also be discerned in the rapid spread of programming languages, or programming paradigms such as “design patterns.” Programming is a field in which genuine innovations rapidly become available to all.

2) The problems which programmers face are persistent. Whereas a book outlining specific technological tools is quickly outdated in programming, a book outlining broad technical challenges, such as algorithms, data representation, security, or programmer collaboration, never seems to become obsolete. This phenomenon has been made a slogan within programming commentary – “there is no silver bullet.” In other words, no matter what technologies are used, certain problems can not be solved without serious application of human energy.

Between these two points, a picture of programming emerges of whose broad outlines we can be fairly confident: programmers eagerly take up new tools to help them in their task of instructing machines; but the state of available tools today can not be utilized by any programmer whatsoever to achieve qualitatively superior results to any other programmer who is competent in the field. In other words, the present relationship between programmer and programmable machine is in fact stable, and is dictated by the nature of programmatic constructs rather than individual programmer skill.

This second point above may at first seem like an eternal truth or truism, because it is such a commonplace within modern thought about programming, and because in any other industry it would be obvious (e.g., an artist’s tools can not make an artwork by themselves). But it is anything but a truism. Even the existence of any problem which is intractably difficult for programmatic automation, but yields to persistent human creative thought, was not obvious to early AI theorists. We presently seem to have settled upon a certain border beyond which computation can not pass – but our own existence demonstrates that there is some way past that border. Therefore we can reject immediately the attitude that this border is the obvious or natural sticking point for our relationship to programmable machines.

A second attitude is to suppose that there is some silver bullet, but it is out of our reach as programmers. That is, humans can think because they have something which can only be granted by computation many orders of magnitude greater than what we have access to – namely, the computational power of evolution. This explanation has the flavor of a post-hoc justification for our present status. There is no specific reason to believe that fundamentally new programmatic constructs require new kinds of computation. A serious inquiry into the problem might conceivably turn up such an impasse, but the possibility of failure is never an excuse not to move ahead.

The shape of our inquiry must therefore be guided by this question: What causes the current boundaries of our relationship to programmable machines, and how can those causes be addressed? This question will be taken up in my next post on the topic.

Post-Programming

In common parlance, “post-” is a prefix used to denote that a structure that had formally held a position of cultural legitimacy has been deconstructed and that its skeleton, its syntactic structure, has been appropriated, given new meaning. To create a structure which is “post-X” the development of X as a system must cease, its elements used as the foundation of a new structure which is in some sense “deeper.” The concept of “post” is a very subtle one. I hope I don’t make too many mistakes in trying to describe my thoughts on the matter.

The standard example is Post-Modernism. As a cultural critical form, Po-Mo is built on the grave of modernism, the movement whose central tenet was that “we,” the western world, had reached the pinnacle of our cultural ascent. The claims, cultural criticisms, and theory composing the content of that movement crumbled, but its skeleton remained, to be re-cast in a new context of moral relativism and value subjectivity.

Post-rock, as a musical movement, uses the motifs of rock without any sort of nod to their foundations. Think about the motifs as the “notes” of Post-rock. Post-rock’s contribution is in creating structures and theory to allow one to move around in the linguistic/deductive framework that these motifs generate.

The recurring theme is that the semantic content of a deductive or abstraction-based system is left in the dust, but the syntactic and symbolic content remain to be used as a springboard for new and hopefully more powerful abstractions.

From a broader perspective, the “post” phenomenon (though it often goes unnamed as such) is part of the evolutionary progress of culture. Every generation is presented with a new set of challenges, new complexity which is, for them, difficult to deal with. We have spent time figuring out, for instance, how to use cell phones – what is the proper etiquette, when is it safe to use a cellphone, etc. Though these questions aren’t difficult, for our generation, a generation which has “grown up beside cell phones,” it is impossible to forget the essential semantic content of our moral assertions. One should not use a cell phone during a meeting because it is rude. Our knowledge of an abstract cultural/ethical decree such as “put your cell phone in silent mode during a meeting” exists along side the experience that generated the requirement for such. We can recall times when cell phones were used during meetings, lectures, etc. and how annoying the phenomenon was. For us, the abstraction has content that is non-hypothetical. It is the difference between knowing that murder is wrong because mother and society tell you, and knowing that murder is wrong because you have murdered.

But what of the next generation?

The next generation will take our ethics of cellphone usage at face value, as primitives. They will not know in an essential, human, experiential sense “why” it is wrong to use cellphones in groups because they simply will not have come face-to-face with the phenomenon. For them, the abstract content of the moral code will be preserved – they will have a skeleton on which to base future ethical decisions, theory, etc. This is the nature of cultural abstraction.

The example of the ethics of cellphone usage is intentionally prosaic. I want to convey the commonness of the “post” phenomenon as a primary method by which our society deals with complexity. Once all memory of rude cellphone hijinx is gone forever, we will have progressed into a “post-cellphone” age, an age where we no longer need think at any great length about why we know what we know.

Of course, I could go into the frailty of these abstractions and tie that in to, for instance, the frailty of our current economic/political systems. But I won’t. Not now, anyway.

The point of this post is to start detailing the nature of “post-programming” which is a term that hopefully I (Chris) and my two buds (Emma and Jason) will explore through descriptions of our exploits in a new world of programming – a world wherein knowledge of the fundament of modern computing (compilers, assembly, C, the lambda calculus) is becoming increasingly sparce – a world wherein more “abstract” knowledge (Java, Ruby, .NET, Frameworks) is becoming increasingly commonplace.

The danger of such a description is that it will be based on over-simplification and stereotypes. Portraying Java as an abstract phoenix risen from the ashes of nitty-gritty C/C++ does a great deal to occlude the nuance of both languages. Therefore, I will resort to a description of one of my personal experiences in the world of post-programming.

I started an internship at a company (SECOM) in Japan this year. My group programs in C#. Here’s how I write a program in C#:

design design design…

Start programming…

Oh, I need to store some data. I have vague knowledge of this thing called XML, it seems like it’ll be useful. Maybe?

Google “XML” – read wikipedia page, read thescripts.com and various tutorials to get an idea of what tools to use, best practices w/ C#, etc.

Oh good. That was easy. IMPLEMENT. Oh, now I have to print to .pdf.

Google “print to pdf C#” – find iTextMate. Download package, read quick tutorial, write wrapper library, print to .pdf.

Oh good. That was easy. Oh, I need to do event handling.

Google “C# event handling.”

I need to make a resizable text-box – Google “C# resizable textbox” – I need to use XPath – Google “XPath C#.”

Etc. You get the idea. There’s no thinking here. I’m just putting together extremely high-level pieces. No knowledge is required. Point-and-click.

The idea that I would even begin to dive into the depths of any of these topics scares and bores the junk out of me. I don’t want to learn about the XML specification. That’s boring as hell. I just want to use the damn thing.

I can hardly expect that this is an uncommon sentiment.

At my job, when I am using C#/.NET, I am a post-programmer in the flesh. I am working completely within the framework (read: confines) of the C# .NET framework. Thinking about the deeper issues at hand can only bring grief and boredom, so I keep my eye on the prize (successful implementation) and try to deal with these high-level concepts as primitives. If something goes wrong, well, shit.

As with any cultural label, “post-programming” is a vast generalization and over-simplification. There are, of course, real “hackers” out there who have been down in the shit, who continue to write good code, who are brilliant. Yada yada. Even they are “post-programmers” in the sense that they don’t write assembler code. They don’t write in C most of the time. Even they are sitting on nice, inherited frameworks. They just aren’t as high up in the clouds as others.

As with any cultural phenomenon, post-programming can manifest as either friend and foe. I didn’t have to learn XML, which is good, because frankly I don’t give a damn – but even without learning its inner workings I was able to derive a great deal of utility from the power of its innate structure. On the other hand, the question arises if the skills that I learn from performing this sort of programming are useful in a general sense – will the structures on which “general” frameworks like .NET are based stand the tests of time? The post-programmer thrives on information which is easy to access and necessarily shallow, but the structures he derives in terms of this shallow language of (high-level) primitives are bound to be as deep as the structures behind the primitives he/she uses. That is, of course, accepting that his primitives were good enough and the whole thing doesn’t collapse under its own heavy restrictions (e.g. Java code bloat).

It is my goal as a post-programmer to detail and create these structures – to give rigorous definitions to terms such as “abstraction,” “meta,” “framework,” “DSL,” etc. (definitions which have been, historically speaking, lacking). It is my goal to create publicly available utilities for working within the vast sea of information and complexity that we are now swimming in. It is my goal to become the post-post-post-programmer.

—

As a closing note, I hope I haven’t portrayed myself as a “blub” programmer. There is a distinct difference between “blub” and “post” although it is somewhat subtle. I further hope that I will not color my co-bloggers’ future posts with this taint. We all get down in the dirty. I study programming languages, compilers, theory of computation, blah blah. I am a math guy before I am a programmer. I am a dreamer before I am a programmer. I am a meat-stick before I am a programmer. That being said, I realize the utility of working within the system, even if I hate it. So the system it is.

Do you think post-modernism ever made anyone smile?