<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>Opensource on bramp.net</title>
    <link>https://blog.bramp.net/</link>
    <description>Recent content in Opensource on bramp.net</description>
    <generator>Hugo -- gohugo.io</generator>
    <language>en-GB</language>
    <lastBuildDate>Sat, 16 Dec 2017 12:50:31 -0800</lastBuildDate>
    <atom:link href="https://blog.bramp.net/tags/opensource/" rel="self" type="application/rss+xml" />
    
    <item>
      <title>Parsing with Antlr4 and Go</title>
      <link>https://blog.bramp.net/post/2017/12/16/parsing-with-antlr4-and-go/</link>
      <pubDate>Sat, 16 Dec 2017 12:50:31 -0800</pubDate>
      
      <guid>https://blog.bramp.net/post/2017/12/16/parsing-with-antlr4-and-go/</guid>
      <description><p><em>Originally <a href="https://blog.gopheracademy.com/advent-2017/parsing-with-antlr4-and-go/">published</a> as part of the Go Advent 2017 series</em></p>
<h2 id="what-is-antlr">What is ANTLR?</h2>
<p><a href="http://www.antlr.org">ANTLR</a> (ANother Tool for Language Recognition),
is an <a href="http://www.antlr.org/papers/allstar-techreport.pdf">ALL(*)</a>
<a href="https://en.wikipedia.org/wiki/Parser_generator">parser generator</a>. In
layman&rsquo;s terms, Antlr, creates parsers in a number of languages (Go,
Java, C, C#, Javascript), that can process text or binary input. The
generated parser provides a callback interface to parse the input in an
event-driven manner, which can be used as-is, or used to build parse
trees (a data structure representing the input).</p>
<p>ANTLR is used by a number of popular projects, e.g Hive and Pig use it
to parse Hadoop queries, Oracle and NetBeans uses it for their IDEs, and
Twitter even uses it to understand search queries. Support was recently
added so that ANTLR 4 can be used to generate parsers in pure Go. This
article will explain some of the benefits of ANTLR, and walk us through
a simple example.</p>
<h2 id="why-use-it">Why use it?</h2>
<p>It is possible to <a href="https://blog.gopheracademy.com/advent-2014/parsers-lexers/">hand write a
parser</a>, but
this process can be complex, error prone, and hard to change. Instead
there are many [parser generators](<a href="https://en.wikipedia.org/wiki/Compari">https://en.wikipedia.org/wiki/Compari</a>
son_of_parser_generators) that take a grammar expressed in an domain-
specific way, and generates code to parse that language. Popular parser
generates include <a href="https://www.gnu.org/software/bison/">bison</a> and
<a href="http://dinosaur.compilertools.net/yacc/">yacc</a>. In fact, there is a
version of yacc, goyacc, which is written in Go and was part of the main
go repo until it was moved to
<a href="https://godoc.org/golang.org/x/tools/cmd/goyacc">golang.org/x/tools</a>
last year.</p>
<h3 id="so-why-use-antlr-over-these">So why use ANTLR over these?</h3>
<ul>
<li>
<p>ANTLR has a <a href="http://www.antlr.org/tools.html">suite of tools</a>, and
<a href="http://tunnelvisionlabs.com/products/demo/antlrworks">GUIs</a>, that
makes writing and debugging grammars easy.</p>
</li>
<li>
<p>It uses a simple <a href="https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form">EBNF</a>
syntax to define the grammar, instead of a bespoke configuration
language.</p>
</li>
<li>
<p>ANTLR is an <a href="http://www.antlr.org/papers/allstar-techreport.pdf">Adaptive</a>
<a href="https://en.wikipedia.org/wiki/LL_parser">LL(*) parser</a>, ALL(*) for short,
whereas most other parser generators (e.g Bison and Yacc) are
<a href="https://en.wikipedia.org/wiki/LALR_parser">LALR</a>. The difference
between LL(*) and LALR is out of scope for this article, but
simply LALR works bottom-up, and LL(*) works top-down. This
has a bearing on how the grammar is written, making some languages
easier or harder to express.</p>
</li>
<li>
<p>The generated code for a LL(*) parser is more understandable than a
LALR parser. This is because LALR parsers are commonly table driven,
whereas LL(*) parsers encode the logic in its control flow, making
it more comprehensible.</p>
</li>
<li>
<p>Finally ANTLR is agnostic to the target language. A single grammar
can be used to generate parsers in Java, Go, C, etc. Unlike
Bison/Yacc which typically embeds target language code into the
grammar, making it harder to port.</p>
</li>
</ul>
<h2 id="installing-antlr-v4">Installing ANTLR v4</h2>
<p>ANTLR is a Java 1.7 application, that generates the Go code needed to
parse your language. During development Java is needed, but once the
parser is built only Go and the <a href="https://godoc.org/github.com/antlr/antlr4/runtime/Go/antlr">ANTLR runtime
library</a> is
required. The ANTLR site has
[documentation](<a href="https://github.com/antlr/antlr4/blob/master/doc/getting-">https://github.com/antlr/antlr4/blob/master/doc/getting-</a>
started.md) on how to install this on multiple platforms, but in brief,
you can do the following:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell"><span class="line"><span class="cl">$ wget http://www.antlr.org/download/antlr-4.7-complete.jar
</span></span><span class="line"><span class="cl">$ <span class="nb">alias</span> <span class="nv">antlr</span><span class="o">=</span><span class="s1">&#39;java -jar $PWD/antlr-4.7-complete.jar&#39;</span>
</span></span></code></pre></div><p>The <code>antlr</code> command is now available in your shell. If you prefer, the
.jar file can be placed into a <code>~/bin</code> directory, and the alias can be
stored in your <code>~/.bash_profile</code>.</p>
<h2 id="classic-calculator-example">Classic calculator example</h2>
<p>Let&rsquo;s start with the “hello world” for parsers, the calculator example.
We want to build a parser that handles simple mathematical expressions
such as <code>1 + 2 * 3</code>. The focus of this article is on how to use Go with
ANTLR, so the syntax of the ANTLR language won’t be explained in
detail, but the ANTLR site has [compressive documentation](https://githu
b.com/antlr/antlr4/blob/master/doc/grammars.md).</p>
<p>As we go along, the <a href="https://github.com/bramp/goadvent-antlr">source is available to all
examples</a>.</p>
<pre tabindex="0"><code>// Calc.g4
grammar Calc;

// Tokens
MUL: &#39;*&#39;;
DIV: &#39;/&#39;;
ADD: &#39;+&#39;;
SUB: &#39;-&#39;;
NUMBER: [0-9]+;
WHITESPACE: [ \r\n\t]+ -&gt; skip;

// Rules
start : expression EOF;

expression
   : expression op=(&#39;*&#39;|&#39;/&#39;) expression # MulDiv
   | expression op=(&#39;+&#39;|&#39;-&#39;) expression # AddSub
   | NUMBER                             # Number
   ;
</code></pre><p>The above is a simple grammar split into two sections, <em>tokens</em>, and
<em>rules</em>. The tokens are terminal symbols in the grammar, that is, they
are made up of nothing but literal characters. Whereas rules are non-
terminal states made up of tokens and/or other rules.</p>
<p>By convention this grammar must be saved with a filename that matches
the name of the grammar, in this case “Calc.g4” . To process this file,
and generate the Go parser, we run the <code>antlr</code> command like so:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell"><span class="line"><span class="cl">$ antlr -Dlanguage<span class="o">=</span>Go -o parser Calc.g4 
</span></span></code></pre></div><p>This will generate a set of Go files in the “parser” package and
subdirectory. It is possible to place the generated code in a different
package by using the <code>-package &lt;name&gt;</code> argument. This is useful if your
project has multiple parsers, or you just want a more descriptive
package name for the parser. The generated files will look like the
following:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell"><span class="line"><span class="cl">$ tree
</span></span><span class="line"><span class="cl">├── Calc.g4
</span></span><span class="line"><span class="cl">└── parser
</span></span><span class="line"><span class="cl">    ├── calc_lexer.go
</span></span><span class="line"><span class="cl">    ├── calc_parser.go
</span></span><span class="line"><span class="cl">    ├── calc_base_listener.go
</span></span><span class="line"><span class="cl">    └── calc_listener.go
</span></span></code></pre></div><p>The generated files consist of three main components, the Lexer, Parser,
and Listener.</p>
<p>The Lexer takes arbitrary input and returns a stream of tokens. For
input such as <code>1 + 2 * 3</code>, the Lexer would return the following tokens:
<code>NUMBER (1), ADD (+), NUMBER (2), MUL (*), NUMBER (3), EOF</code>.</p>
<p>The Parser uses the Lexer’s output and applies the Grammar’s rules.
Building higher level constructs, such as expressions that can be used
to calculate the result.</p>
<p>The Listener then allows us to make use of the the parsed input. As
mentioned earlier, yacc requires language specific code to be embedded
with the grammar. However, ANTLR separates this concern, allowing the
grammar to be agnostic to the target programming language. It does this
through use of listeners, which effectively allows hooks to be placed
before and after every rule is encountered in the parsed input.</p>
<h2 id="using-the-lexer">Using the Lexer</h2>
<p>Let&rsquo;s move onto an example of using this generated code, starting with
the Lexer.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go" data-lang="go"><span class="line"><span class="cl"><span class="c1">// example1.go</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="kn">package</span><span class="w"> </span><span class="nx">main</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="kn">import</span><span class="w"> </span><span class="p">(</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="s">&#34;fmt&#34;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="s">&#34;github.com/antlr/antlr4/runtime/Go/antlr&#34;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="s">&#34;./parser&#34;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="kd">func</span><span class="w"> </span><span class="nf">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// Setup the input</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">is</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">antlr</span><span class="p">.</span><span class="nf">NewInputStream</span><span class="p">(</span><span class="s">&#34;1 + 2 * 3&#34;</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// Create the Lexer</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">lexer</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">parser</span><span class="p">.</span><span class="nf">NewCalcLexer</span><span class="p">(</span><span class="nx">is</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// Read all tokens</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="k">for</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">		</span><span class="nx">t</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">lexer</span><span class="p">.</span><span class="nf">NextToken</span><span class="p">()</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">		</span><span class="k">if</span><span class="w"> </span><span class="nx">t</span><span class="p">.</span><span class="nf">GetTokenType</span><span class="p">()</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="nx">antlr</span><span class="p">.</span><span class="nx">TokenEOF</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">			</span><span class="k">break</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">		</span><span class="p">}</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">		</span><span class="nx">fmt</span><span class="p">.</span><span class="nf">Printf</span><span class="p">(</span><span class="s">&#34;%s (%q)\n&#34;</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">			</span><span class="nx">lexer</span><span class="p">.</span><span class="nx">SymbolicNames</span><span class="p">[</span><span class="nx">t</span><span class="p">.</span><span class="nf">GetTokenType</span><span class="p">()],</span><span class="w"> </span><span class="nx">t</span><span class="p">.</span><span class="nf">GetText</span><span class="p">())</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="p">}</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="p">}</span><span class="w">
</span></span></span></code></pre></div><p>To begin with, the generated parser is imported from the local
subdirectory <code>import &quot;./parser&quot;</code>. Next the Lexer is created with some
input:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go" data-lang="go"><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// Setup the input</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">is</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">antlr</span><span class="p">.</span><span class="nf">NewInputStream</span><span class="p">(</span><span class="s">&#34;1 + 2 * 3&#34;</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// Create the Lexer</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">lexer</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">parser</span><span class="p">.</span><span class="nf">NewCalcLexer</span><span class="p">(</span><span class="nx">is</span><span class="p">)</span><span class="w">
</span></span></span></code></pre></div><p>In this example the input is a simple string, <code>&quot;1 + 2 * 3&quot;</code> but there
are other [<code>antlr.InputStream</code>](<a href="https://godoc.org/github.com/antlr/antlr">https://godoc.org/github.com/antlr/antlr</a>
4/runtime/Go/antlr#InputStream)s, for example, the <a href="https://godoc.org/github.com/antlr/antlr4/runtime/Go/antlr#FileStream"><code>antlr.FileStream</code></a>
type can read directly from a file. The <code>InputStream</code> is then passed to
a newly created Lexer. Note the name of the Lexer is <code>CalcLexer</code> which
matches the grammar’s name defined in the Calc.g4.</p>
<p>The lexer is then used to consume all the tokens from the input,
printing them one by one. This wouldn’t normally be necessary but we do
this for demonstrative purposes.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go" data-lang="go"><span class="line"><span class="cl"><span class="w"> 	</span><span class="k">for</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">		</span><span class="nx">t</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">lexer</span><span class="p">.</span><span class="nf">NextToken</span><span class="p">()</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">		</span><span class="k">if</span><span class="w"> </span><span class="nx">t</span><span class="p">.</span><span class="nf">GetTokenType</span><span class="p">()</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="nx">antlr</span><span class="p">.</span><span class="nx">TokenEOF</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">			</span><span class="k">break</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">		</span><span class="p">}</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">		</span><span class="nx">fmt</span><span class="p">.</span><span class="nf">Printf</span><span class="p">(</span><span class="s">&#34;%s (%q)\n&#34;</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">			</span><span class="nx">lexer</span><span class="p">.</span><span class="nx">SymbolicNames</span><span class="p">[</span><span class="nx">t</span><span class="p">.</span><span class="nf">GetTokenType</span><span class="p">()],</span><span class="w"> </span><span class="nx">t</span><span class="p">.</span><span class="nf">GetText</span><span class="p">())</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="p">}</span><span class="w">
</span></span></span></code></pre></div><p>Each token has two main components, the TokenType, and the Text. The
TokenType is a simple integer representing the type of token, while the
Text is literally the text that made up this token. All the TokenTypes
are defined at the end of calc_lexer.go, with their string names stored
in the SymbolicNames slice:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go" data-lang="go"><span class="line"><span class="cl"><span class="c1">// calc_lexer.go</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="kd">const</span><span class="w"> </span><span class="p">(</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">CalcLexerMUL</span><span class="w">        </span><span class="p">=</span><span class="w"> </span><span class="mi">1</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">CalcLexerDIV</span><span class="w">        </span><span class="p">=</span><span class="w"> </span><span class="mi">2</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">CalcLexerADD</span><span class="w">        </span><span class="p">=</span><span class="w"> </span><span class="mi">3</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">CalcLexerSUB</span><span class="w">        </span><span class="p">=</span><span class="w"> </span><span class="mi">4</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">CalcLexerNUMBER</span><span class="w">     </span><span class="p">=</span><span class="w"> </span><span class="mi">5</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">CalcLexerWHITESPACE</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="mi">6</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="p">)</span><span class="w">
</span></span></span></code></pre></div><p>You may also note, that the Whitespace token is not printed, even though
the input clearly had whitespace. This is because the grammar was
designed to skip (i.e. discard) the whitespace <code>WHITESPACE: [ \r\n\t]+ -&gt; skip;</code>.</p>
<h2 id="using-the-parser">Using the Parser</h2>
<p>The Lexer on its own is not very useful, so the example can be modified
to also use the Parser and Listener:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go" data-lang="go"><span class="line"><span class="cl"><span class="c1">// example2.go</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="kn">package</span><span class="w"> </span><span class="nx">main</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="kn">import</span><span class="w"> </span><span class="p">(</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="s">&#34;./parser&#34;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="s">&#34;github.com/antlr/antlr4/runtime/Go/antlr&#34;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="kd">type</span><span class="w"> </span><span class="nx">calcListener</span><span class="w"> </span><span class="kd">struct</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="o">*</span><span class="nx">parser</span><span class="p">.</span><span class="nx">BaseCalcListener</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="p">}</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="kd">func</span><span class="w"> </span><span class="nf">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// Setup the input</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">is</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">antlr</span><span class="p">.</span><span class="nf">NewInputStream</span><span class="p">(</span><span class="s">&#34;1 + 2 * 3&#34;</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// Create the Lexer</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">lexer</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">parser</span><span class="p">.</span><span class="nf">NewCalcLexer</span><span class="p">(</span><span class="nx">is</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">stream</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">antlr</span><span class="p">.</span><span class="nf">NewCommonTokenStream</span><span class="p">(</span><span class="nx">lexer</span><span class="p">,</span><span class="w"> </span><span class="nx">antlr</span><span class="p">.</span><span class="nx">TokenDefaultChannel</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// Create the Parser</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">p</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">parser</span><span class="p">.</span><span class="nf">NewCalcParser</span><span class="p">(</span><span class="nx">stream</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// Finally parse the expression</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">antlr</span><span class="p">.</span><span class="nx">ParseTreeWalkerDefault</span><span class="p">.</span><span class="nf">Walk</span><span class="p">(</span><span class="o">&amp;</span><span class="nx">calcListener</span><span class="p">{},</span><span class="w"> </span><span class="nx">p</span><span class="p">.</span><span class="nf">Start</span><span class="p">())</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="p">}</span><span class="w">
</span></span></span></code></pre></div><p>This is very similar to before, but instead of manually iterating over
the tokens, the lexer is used to create a [<code>CommonTokenStream</code>](https://
godoc.org/github.com/antlr/antlr4/runtime/Go/antlr#CommonTokenStream),
which in turn is used to create a new <code>CalcParser</code>. This <code>CalcParser</code> is
then “walked”, which is ANTLR&rsquo;s event-driven API for receiving the
results of parsing the rules.</p>
<p>Note, the [<code>Walk</code>](<a href="https://godoc.org/github.com/antlr/antlr4/runtime/Go/">https://godoc.org/github.com/antlr/antlr4/runtime/Go/</a>
antlr#ParseTreeWalker.Walk) function does not return anything. Some may
have expected a parsed form of the expression to be returned, such as
some kind of <a href="https://en.wikipedia.org/wiki/Abstract_syntax_tree">AST</a>
(abstract syntax tree), but instead the Listener receives event as the
parsing occurs. This is similar in concept to
<a href="https://en.wikipedia.org/wiki/Simple_API_for_XML">SAX</a> style parsers
for XML. Event-based parsing can sometimes be harder to use, but it has
many advantages. For example, the parser can be very memory efficient as
previously parsed rules can be discarded once they are no longer needed.
The parser can also be aborted early if the programmer wishes to.</p>
<p>But so far, this example doesn’t do anything beyond ensuring the input
can be parsed without error. To add logic, we must extend the
<code>calcListener</code> type. The <code>calcListener</code> has an embedded
<code>BaseCalcListener</code>, which is a helper type, that provides empty methods
for all those defined in in the <code>CalcListener</code> interface. That interface
looks like:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go" data-lang="go"><span class="line"><span class="cl"><span class="c1">// parser/calc_listener.go</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="c1">// CalcListener is a complete listener for a parse tree produced by CalcParser.</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="kd">type</span><span class="w"> </span><span class="nx">CalcListener</span><span class="w"> </span><span class="kd">interface</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">antlr</span><span class="p">.</span><span class="nx">ParseTreeListener</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// EnterStart is called when entering the start production.</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nf">EnterStart</span><span class="p">(</span><span class="nx">c</span><span class="w"> </span><span class="o">*</span><span class="nx">StartContext</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// EnterNumber is called when entering the Number production.</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nf">EnterNumber</span><span class="p">(</span><span class="nx">c</span><span class="w"> </span><span class="o">*</span><span class="nx">NumberContext</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// EnterMulDiv is called when entering the MulDiv production.</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nf">EnterMulDiv</span><span class="p">(</span><span class="nx">c</span><span class="w"> </span><span class="o">*</span><span class="nx">MulDivContext</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// EnterAddSub is called when entering the AddSub production.</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nf">EnterAddSub</span><span class="p">(</span><span class="nx">c</span><span class="w"> </span><span class="o">*</span><span class="nx">AddSubContext</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// ExitStart is called when exiting the start production.</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nf">ExitStart</span><span class="p">(</span><span class="nx">c</span><span class="w"> </span><span class="o">*</span><span class="nx">StartContext</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// ExitNumber is called when exiting the Number production.</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nf">ExitNumber</span><span class="p">(</span><span class="nx">c</span><span class="w"> </span><span class="o">*</span><span class="nx">NumberContext</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// ExitMulDiv is called when exiting the MulDiv production.</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nf">ExitMulDiv</span><span class="p">(</span><span class="nx">c</span><span class="w"> </span><span class="o">*</span><span class="nx">MulDivContext</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// ExitAddSub is called when exiting the AddSub production.</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nf">ExitAddSub</span><span class="p">(</span><span class="nx">c</span><span class="w"> </span><span class="o">*</span><span class="nx">AddSubContext</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="p">}</span><span class="w">
</span></span></span></code></pre></div><p>There is an Enter and Exit function for each rule found in the grammar.
As the input is walked, the Parser calls the appropriate function on the
listener, to indicate when the rule starts and finishes being evaluated.</p>
<h2 id="adding-the-logic">Adding the logic</h2>
<p>A simple calculator can be constructed from this event driven parser by
using a stack of values. Every time a number is found, it is added to a
stack. Everytime an expression (add/multiple/etc) is found, the last two
numbers on the stack are popped, and the appropriate operation is
carried out. The result is then placed back on the stack.</p>
<p>Take the expression <code>1 + 2 * 3</code>,  the result could be either <code>(1 + 2) * 3 = 9</code>, or <code>1 + (2 * 3) = 7</code>. Those that recall the <a href="https://en.wikipedia.org/wiki/Order_of_operations">order of
operations</a>, will
know that multiplication should always be carried out before addition,
thus the correct result is 7. However, without the parentheses there
could be some ambiguity on how this should be parsed. Luckily the
ambiguity is resolved by the grammar. The precedence of multiplication
over addition was subtly implied within Calc.g4, by placing the <code>MulDiv</code>
expressed before the <code>AddSub</code> expression.</p>
<div class="text-center">
	<img src="parse-tree.svg">
</div>
<p>The code for a listener that implements this stack of value
implementation is relatively simple:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go" data-lang="go"><span class="line"><span class="cl"><span class="kd">type</span><span class="w"> </span><span class="nx">calcListener</span><span class="w"> </span><span class="kd">struct</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="o">*</span><span class="nx">parser</span><span class="p">.</span><span class="nx">BaseCalcListener</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">stack</span><span class="w"> </span><span class="p">[]</span><span class="kt">int</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="p">}</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="kd">func</span><span class="w"> </span><span class="p">(</span><span class="nx">l</span><span class="w"> </span><span class="o">*</span><span class="nx">calcListener</span><span class="p">)</span><span class="w"> </span><span class="nf">push</span><span class="p">(</span><span class="nx">i</span><span class="w"> </span><span class="kt">int</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">l</span><span class="p">.</span><span class="nx">stack</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="nb">append</span><span class="p">(</span><span class="nx">l</span><span class="p">.</span><span class="nx">stack</span><span class="p">,</span><span class="w"> </span><span class="nx">i</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="p">}</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="kd">func</span><span class="w"> </span><span class="p">(</span><span class="nx">l</span><span class="w"> </span><span class="o">*</span><span class="nx">calcListener</span><span class="p">)</span><span class="w"> </span><span class="nf">pop</span><span class="p">()</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="k">if</span><span class="w"> </span><span class="nb">len</span><span class="p">(</span><span class="nx">l</span><span class="p">.</span><span class="nx">stack</span><span class="p">)</span><span class="w"> </span><span class="p">&lt;</span><span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">		</span><span class="nb">panic</span><span class="p">(</span><span class="s">&#34;stack is empty unable to pop&#34;</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="p">}</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// Get the last value from the stack.</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">result</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">l</span><span class="p">.</span><span class="nx">stack</span><span class="p">[</span><span class="nb">len</span><span class="p">(</span><span class="nx">l</span><span class="p">.</span><span class="nx">stack</span><span class="p">)</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// Remove the last element from the stack.</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">l</span><span class="p">.</span><span class="nx">stack</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="nx">l</span><span class="p">.</span><span class="nx">stack</span><span class="p">[:</span><span class="nb">len</span><span class="p">(</span><span class="nx">l</span><span class="p">.</span><span class="nx">stack</span><span class="p">)</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="k">return</span><span class="w"> </span><span class="nx">result</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="p">}</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="kd">func</span><span class="w"> </span><span class="p">(</span><span class="nx">l</span><span class="w"> </span><span class="o">*</span><span class="nx">calcListener</span><span class="p">)</span><span class="w"> </span><span class="nf">ExitMulDiv</span><span class="p">(</span><span class="nx">c</span><span class="w"> </span><span class="o">*</span><span class="nx">parser</span><span class="p">.</span><span class="nx">MulDivContext</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">right</span><span class="p">,</span><span class="w"> </span><span class="nx">left</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">l</span><span class="p">.</span><span class="nf">pop</span><span class="p">(),</span><span class="w"> </span><span class="nx">l</span><span class="p">.</span><span class="nf">pop</span><span class="p">()</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="k">switch</span><span class="w"> </span><span class="nx">c</span><span class="p">.</span><span class="nf">GetOp</span><span class="p">().</span><span class="nf">GetTokenType</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="k">case</span><span class="w"> </span><span class="nx">parser</span><span class="p">.</span><span class="nx">CalcParserMUL</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">		</span><span class="nx">l</span><span class="p">.</span><span class="nf">push</span><span class="p">(</span><span class="nx">left</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="nx">right</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="k">case</span><span class="w"> </span><span class="nx">parser</span><span class="p">.</span><span class="nx">CalcParserDIV</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">		</span><span class="nx">l</span><span class="p">.</span><span class="nf">push</span><span class="p">(</span><span class="nx">left</span><span class="w"> </span><span class="o">/</span><span class="w"> </span><span class="nx">right</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="k">default</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">		</span><span class="nb">panic</span><span class="p">(</span><span class="nx">fmt</span><span class="p">.</span><span class="nf">Sprintf</span><span class="p">(</span><span class="s">&#34;unexpected op: %s&#34;</span><span class="p">,</span><span class="w"> </span><span class="nx">c</span><span class="p">.</span><span class="nf">GetOp</span><span class="p">().</span><span class="nf">GetText</span><span class="p">()))</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="p">}</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="p">}</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="kd">func</span><span class="w"> </span><span class="p">(</span><span class="nx">l</span><span class="w"> </span><span class="o">*</span><span class="nx">calcListener</span><span class="p">)</span><span class="w"> </span><span class="nf">ExitAddSub</span><span class="p">(</span><span class="nx">c</span><span class="w"> </span><span class="o">*</span><span class="nx">parser</span><span class="p">.</span><span class="nx">AddSubContext</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">right</span><span class="p">,</span><span class="w"> </span><span class="nx">left</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">l</span><span class="p">.</span><span class="nf">pop</span><span class="p">(),</span><span class="w"> </span><span class="nx">l</span><span class="p">.</span><span class="nf">pop</span><span class="p">()</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="k">switch</span><span class="w"> </span><span class="nx">c</span><span class="p">.</span><span class="nf">GetOp</span><span class="p">().</span><span class="nf">GetTokenType</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="k">case</span><span class="w"> </span><span class="nx">parser</span><span class="p">.</span><span class="nx">CalcParserADD</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">		</span><span class="nx">l</span><span class="p">.</span><span class="nf">push</span><span class="p">(</span><span class="nx">left</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nx">right</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="k">case</span><span class="w"> </span><span class="nx">parser</span><span class="p">.</span><span class="nx">CalcParserSUB</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">		</span><span class="nx">l</span><span class="p">.</span><span class="nf">push</span><span class="p">(</span><span class="nx">left</span><span class="w"> </span><span class="o">-</span><span class="w"> </span><span class="nx">right</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="k">default</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">		</span><span class="nb">panic</span><span class="p">(</span><span class="nx">fmt</span><span class="p">.</span><span class="nf">Sprintf</span><span class="p">(</span><span class="s">&#34;unexpected op: %s&#34;</span><span class="p">,</span><span class="w"> </span><span class="nx">c</span><span class="p">.</span><span class="nf">GetOp</span><span class="p">().</span><span class="nf">GetText</span><span class="p">()))</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="p">}</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="p">}</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="kd">func</span><span class="w"> </span><span class="p">(</span><span class="nx">l</span><span class="w"> </span><span class="o">*</span><span class="nx">calcListener</span><span class="p">)</span><span class="w"> </span><span class="nf">ExitNumber</span><span class="p">(</span><span class="nx">c</span><span class="w"> </span><span class="o">*</span><span class="nx">parser</span><span class="p">.</span><span class="nx">NumberContext</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">i</span><span class="p">,</span><span class="w"> </span><span class="nx">err</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">strconv</span><span class="p">.</span><span class="nf">Atoi</span><span class="p">(</span><span class="nx">c</span><span class="p">.</span><span class="nf">GetText</span><span class="p">())</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="k">if</span><span class="w"> </span><span class="nx">err</span><span class="w"> </span><span class="o">!=</span><span class="w"> </span><span class="kc">nil</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">		</span><span class="nb">panic</span><span class="p">(</span><span class="nx">err</span><span class="p">.</span><span class="nf">Error</span><span class="p">())</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="p">}</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">l</span><span class="p">.</span><span class="nf">push</span><span class="p">(</span><span class="nx">i</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="p">}</span><span class="w">
</span></span></span></code></pre></div><p>Finally this listener would be used like so:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go" data-lang="go"><span class="line"><span class="cl"><span class="c1">// calc takes a string expression and returns the evaluated result.</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="kd">func</span><span class="w"> </span><span class="nf">calc</span><span class="p">(</span><span class="nx">input</span><span class="w"> </span><span class="kt">string</span><span class="p">)</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// Setup the input</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">is</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">antlr</span><span class="p">.</span><span class="nf">NewInputStream</span><span class="p">(</span><span class="nx">input</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// Create the Lexer</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">lexer</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">parser</span><span class="p">.</span><span class="nf">NewCalcLexer</span><span class="p">(</span><span class="nx">is</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">stream</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">antlr</span><span class="p">.</span><span class="nf">NewCommonTokenStream</span><span class="p">(</span><span class="nx">lexer</span><span class="p">,</span><span class="w"> </span><span class="nx">antlr</span><span class="p">.</span><span class="nx">TokenDefaultChannel</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// Create the Parser</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">p</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">parser</span><span class="p">.</span><span class="nf">NewCalcParser</span><span class="p">(</span><span class="nx">stream</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// Finally parse the expression (by walking the tree)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="kd">var</span><span class="w"> </span><span class="nx">listener</span><span class="w"> </span><span class="nx">calcListener</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">antlr</span><span class="p">.</span><span class="nx">ParseTreeWalkerDefault</span><span class="p">.</span><span class="nf">Walk</span><span class="p">(</span><span class="o">&amp;</span><span class="nx">listener</span><span class="p">,</span><span class="w"> </span><span class="nx">p</span><span class="p">.</span><span class="nf">Start</span><span class="p">())</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="k">return</span><span class="w"> </span><span class="nx">listener</span><span class="p">.</span><span class="nf">pop</span><span class="p">()</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="p">}</span><span class="w">
</span></span></span></code></pre></div><p>Following the algorithm, the parsing of <code>1 + 2 * 3</code> would work like so.</p>
<ol>
<li>The numbers 2 and 3 would be visited first (and placed on the stack),</li>
<li>Then the MulDiv expression would be visited, taking the values 2 and
3, multiplying them, and placing the result, 6, back on the stack.</li>
<li>Then the number 1 would visited and pushed onto the stack.</li>
<li>Finally AddSub would be visited, popping the 1 and the 6 from the
stack, placing the result 7 back.</li>
</ol>
<p>The order the rules are visited is completely driven by the Parser, and
thus the grammar.</p>
<h2 id="more-grammars">More grammars</h2>
<p>Learning how to write a grammar may be daunting, but there are many
resources for help. The author of ANTLR, <a href="http://parrt.cs.usfca.edu/">Terence
Parr</a>, has <a href="https://pragprog.com/book/tpantlr2/the-definitive-antlr-4-reference">published a
book</a>,
with some of the content freely available on <a href="http://antlr.org">antlr.org</a>.</p>
<p>If you don’t want to write your own grammar, there are many <a href="https://github.com/antlr/grammars-v4">pre-written
grammars available</a>. Including
grammars for CSS, HTML, SQL, etc, as well many popular programming
languages. To make it easier, I have <a href="https://github.com/bramp/antlr4-grammars">generated
parsers</a> for all those
available grammars, making them as easy to use just by importing.</p>
<p>A quick example of using one of the pre-generated grammars:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go" data-lang="go"><span class="line"><span class="cl"><span class="kn">import</span><span class="w"> </span><span class="p">(</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="s">&#34;bramp.net/antlr4/json&#34;</span><span class="w"> </span><span class="c1">// The parser</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="s">&#34;github.com/antlr/antlr4/runtime/Go/antlr&#34;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="kd">type</span><span class="w"> </span><span class="nx">exampleListener</span><span class="w"> </span><span class="kd">struct</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// https://godoc.org/bramp.net/antlr4/json#BaseJSONListener</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="o">*</span><span class="nx">json</span><span class="p">.</span><span class="nx">BaseJSONListener</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="p">}</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="kd">func</span><span class="w"> </span><span class="nf">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// Setup the input</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">is</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">antlr</span><span class="p">.</span><span class="nf">NewInputStream</span><span class="p">(</span><span class="s">`
</span></span></span><span class="line"><span class="cl"><span class="s">		{
</span></span></span><span class="line"><span class="cl"><span class="s">			&#34;example&#34;: &#34;json&#34;,
</span></span></span><span class="line"><span class="cl"><span class="s">			&#34;with&#34;: [&#34;an&#34;, &#34;array&#34;]
</span></span></span><span class="line"><span class="cl"><span class="s">		}`</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// Create the JSON Lexer</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">lexer</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">json</span><span class="p">.</span><span class="nf">NewJSONLexer</span><span class="p">(</span><span class="nx">is</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">stream</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">antlr</span><span class="p">.</span><span class="nf">NewCommonTokenStream</span><span class="p">(</span><span class="nx">lexer</span><span class="p">,</span><span class="w"> </span><span class="nx">antlr</span><span class="p">.</span><span class="nx">TokenDefaultChannel</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// Create the JSON Parser</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">p</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">json</span><span class="p">.</span><span class="nf">NewJSONParser</span><span class="p">(</span><span class="nx">stream</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="c1">// Finally walk the tree</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">	</span><span class="nx">antlr</span><span class="p">.</span><span class="nx">ParseTreeWalkerDefault</span><span class="p">.</span><span class="nf">Walk</span><span class="p">(</span><span class="o">&amp;</span><span class="nx">exampleListener</span><span class="p">{},</span><span class="w"> </span><span class="nx">p</span><span class="p">.</span><span class="nf">Json</span><span class="p">())</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="p">}</span><span class="w">
</span></span></span></code></pre></div><h2 id="conclusion">Conclusion</h2>
<p>Hopefully this article has given you a taste of how to use ANTLR with Go.
The examples for this article are <a href="https://github.com/bramp/goadvent-antlr">found here</a>,
and the <a href="https://godoc.org/github.com/antlr/antlr4/runtime/Go/antlr">godoc for the ANTLR library is here</a>
which explains the various InputStream, Lexer, Parser, etc interfaces.</p>
</description>
    </item>
    
    <item>
      <title>Building a better “What&#39;s My IP?” site</title>
      <link>https://blog.bramp.net/post/2017/02/20/building-a-better-whats-my-ip-site/</link>
      <pubDate>Mon, 20 Feb 2017 12:50:31 -0800</pubDate>
      
      <guid>https://blog.bramp.net/post/2017/02/20/building-a-better-whats-my-ip-site/</guid>
      <description><p>Occasionally I’m curious to know what network my device is using, if it has a IPv6 address, and who owns the address space. For example, when in a coffee shop I’m curious to know their ISP, or when roaming internationally I’m always curious to understand which mobile operator’s IP address gets assigned to device.</p>
<p>Most &ldquo;<a href="https://www.google.com/search?q=What%E2%80%99s+my+IP+address">What’s my IP address</a>&rdquo; sites, will either only show you one of your IPv4, or IPv6. It won’t do a DNS lookup, and they rarely do a WHOIS lookup.  Doing all these things, shouldn’t be too hard, so I figured in a weekend I could hack together a site to do this.</p>
<p>This blog post explains the creation of <a href="http://ip.bramp.net">ip.bramp.net</a>.</p>
<div class="text-center">
  <img src="screenshot.png" alt="Screenshot of ip.bramp.net in action"></img>
</div>
<h2 id="how-to-get-both-ipv4-and-ipv6-address">How to get both IPv4 and IPv6 address?</h2>
<p>When navigating to a website your browser makes a connection, normally over one of IPv4, or IPv6. Which one is based on the DNS records available for the website’s domain, and the preference of your OS and browser. Thus your web server sees a single incoming connection, with a single remote address. This address is typically stored in a variable named REMOTE_ADDR. Most &ldquo;What’s my IP&rdquo; sites then display this variable back to the user as their IP address. However, as I’d like to see both IPv4 and IPv6 addresses, I need to somehow force the browser to make two requests, one over each.</p>
<p>There is no API to tell a browser to use IPv6 over IPv4, however, I use a trick with two domain names. Namely, I have ip4.bramp.net, and ip6.bramp.net. Both resolve to the same web server, but the former only has <a href="https://tools.ietf.org/html/rfc1035">DNS A records</a>, and the latter only <a href="https://tools.ietf.org/html/rfc3596">DNS AAAA records</a>. For example:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">$ dig ip4.bramp.net
</span></span><span class="line"><span class="cl">ip4.bramp.net.		300	IN	A	216.239.32.21
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">$ dig AAAA ip6.bramp.net
</span></span><span class="line"><span class="cl">ip6.bramp.net.		291	IN	AAAA	2001:4860:4802:32::15
</span></span></code></pre></div><p>This forces the connection to be over either IPv4, or IPv6. If a browser doesn’t support IPv6, then the connection is never made, and an error is returned. Interesting side note, some browsers uses a technique called <a href="https://en.wikipedia.org/wiki/Happy_Eyeballs">Happy Eyeballs</a>, which tries to connect over both concurrently, but abandons the slower or worse behaving of the two connections.</p>
<h2 id="how-do-you-make-two-requests-from-one-page">How do you make two requests from one page?</h2>
<p>To force the site to make requests to both of these domains, I issue two <a href="https://en.wikipedia.org/wiki/Ajax_(programming)">AJAX</a> queries. The typical flow looks like:</p>
<div class="text-center">
  <object data="diagram.svg" type="image/svg+xml" height="364" width="583" alt="diagram of AJAX calls">
    <img src="diagram.png" />
  </object>
</div>
<!--
```
user->ip.bramp.net: GET /
ip.bramp.net->user: <html...>
user->ip4.bramp.net: GET ip4.bramp.net/json
ip4.bramp.net->user: {address: 1.2.3.4}
user->ip6.bramp.net: GET ip6.bramp.net/json
ip6.bramp.net->user: {address: 2001:db8::1}
```
-->
<p>These AJAX queries return a simple JSON object, containing information about the requesting user. In my application an example response may look like:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-json" data-lang="json"><span class="line"><span class="cl"><span class="p">{</span>
</span></span><span class="line"><span class="cl">  <span class="nt">&#34;RemoteAddrFamily&#34;</span><span class="p">:</span> <span class="s2">&#34;IPv6&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">  <span class="nt">&#34;RemoteAddr&#34;</span><span class="p">:</span>       <span class="s2">&#34;2601:646:c200:b466:b446:ff32:b227:a53c&#34;</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span></code></pre></div><p>This response can then be used to update the page, to display the appropriate address.</p>
<p>An experienced reader may be aware of some security issues with making AJAX request to a different domain. In particular, there are subtle ways in which a malicious site could abuse your AJAX endpoints. This is easily fixed by using cross-origin resource sharing (<a href="https://en.wikipedia.org/wiki/Cross-origin_resource_sharing">CORS</a>) headers, in particular ip4.bramp.net, and ip6.bramp.net return the following:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">$ curl -v ip4.bramp.net/json
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">&lt; HTTP/1.1 <span class="m">200</span> OK
</span></span><span class="line"><span class="cl">&lt; Content-Type: application/json
</span></span><span class="line"><span class="cl">&lt; Access-Control-Allow-Origin: http://ip.bramp.net
</span></span></code></pre></div><p>The last header, explicitly allows requests from ip.bramp.net only, and thus forbids requests from other sites. Without this header, the AJAX request would be issued, but then rejected by the browser.</p>
<h2 id="what-about-the-reverse-dns-and-whois">What about the reverse DNS and WHOIS?</h2>
<p>I noted I wanted to display both the reverse DNS, and WHOIS records. This is something the browser doesn’t support, but the server side could. Thus as part of processing the /json AJAX request, the application makes various additional requests to remote DNS and WHOIS servers.</p>
<p>To reverse lookup a IP address, you need to issue a <a href="https://tools.ietf.org/html/rfc1035">PTR DNS request</a>. This is a special DNS request which requires the IP address to be formatted as a in-addr.arpa or ip6.arpa name. For example, the IP address 1.2.3.4, would become 4.3.2.1.in-addr.arpa. Then when a request is sent for that in-addr.arpa. name, the reverse DNS is returned.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">$ dig PTR 4.3.2.1.in-addr.arpa.
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">4.3.2.1.in-addr.arpa.	74312	IN	PTR	c-1-2-3-4.example.com.
</span></span></code></pre></div><p>The in-addr.arpa transformation, and lookup is commonly provided by <a href="https://linux.die.net/man/3/getaddrinfo">getaddrinfo(3)</a> function, which makes this easy to do.</p>
<p>The WHOIS lookup is a little bit more complex. Each domain is represented by a different WHOIS server, that can be determined by the ccTLD or TLD of the domain. However, with IP addresses, you must identify the Regional Internet Registry (<a href="https://en.wikipedia.org/wiki/Regional_Internet_registry">RIR</a>) that owns the IP space. Sadly there is not a trivial mapping, so instead I issue a WHOIS query to the Internet Assigned Numbers Authority (<a href="https://en.wikipedia.org/wiki/Internet_Assigned_Numbers_Authority">IANA</a>), who replies with the RIR which owns the IP address. From there, I can query the correct registry directly, typically one of <a href="https://en.wikipedia.org/wiki/AFRINIC">AFRINIC</a>, <a href="https://en.wikipedia.org/wiki/American_Registry_for_Internet_Numbers">ARIN</a>, <a href="https://en.wikipedia.org/wiki/Asia-Pacific_Network_Information_Centre">APNIC</a>, <a href="https://en.wikipedia.org/wiki/Latin_America_and_Caribbean_Network_Information_Centre">LACNIC</a>, or <a href="https://en.wikipedia.org/wiki/R%C3%A9seaux_IP_Europ%C3%A9ens_Network_Coordination_Centre">RIPE NCC</a>. Thus a typical WHOIS request looks like this:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">$ whois -h whois.iana.org 1.2.3.4
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">% IANA WHOIS server
</span></span><span class="line"><span class="cl">% <span class="k">for</span> more information on IANA, visit http://www.iana.org
</span></span><span class="line"><span class="cl">% This query returned <span class="m">1</span> object
</span></span><span class="line"><span class="cl">refer:        whois.apnic.net
</span></span><span class="line"><span class="cl">inetnum:      1.0.0.0 - 1.255.255.255
</span></span><span class="line"><span class="cl">organisation: APNIC
</span></span><span class="line"><span class="cl">status:       ALLOCATED
</span></span><span class="line"><span class="cl">whois:        whois.apnic.net
</span></span><span class="line"><span class="cl">changed:      2010-01
</span></span><span class="line"><span class="cl">source:       IANA
</span></span></code></pre></div><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">$ whois -h whois.apnic.net 1.2.3.4
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">% <span class="o">[</span>whois.apnic.net<span class="o">]</span>
</span></span><span class="line"><span class="cl">% Whois data copyright terms    http://www.apnic.net/db/dbcopyright.html
</span></span><span class="line"><span class="cl">% Information related to <span class="s1">&#39;1.2.3.0 - 1.2.3.255&#39;</span>
</span></span><span class="line"><span class="cl">inetnum:        1.2.3.0 - 1.2.3.255
</span></span><span class="line"><span class="cl">netname:        Example-prefix
</span></span><span class="line"><span class="cl">descr:          APNIC Example Project
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">...
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">% This query was served by the APNIC Whois Service version 1.69.1-APNICv1r0
</span></span></code></pre></div><p>Both the reverse DNS and WHOIS response is returned as part of the AJAX JSON response.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-json" data-lang="json"><span class="line"><span class="cl"><span class="p">{</span>
</span></span><span class="line"><span class="cl">  <span class="nt">&#34;RemoteAddrFamily&#34;</span><span class="p">:</span>  <span class="s2">&#34;IPv4&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">  <span class="nt">&#34;RemoteAddr&#34;</span><span class="p">:</span>        <span class="s2">&#34;1.2.3.4&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">  <span class="nt">&#34;RemoteAddrReverse&#34;</span><span class="p">:</span> <span class="s2">&#34;c-1-2-3-4.example.com.&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">  <span class="nt">&#34;RemoteAddrWhois&#34;</span><span class="p">:</span>   <span class="s2">&#34;
</span></span></span><span class="line"><span class="cl"><span class="s2">    % IANA WHOIS server
</span></span></span><span class="line"><span class="cl"><span class="s2">    % for more information on IANA, visit http://www.iana.org
</span></span></span><span class="line"><span class="cl"><span class="s2">    % This query returned 1 object
</span></span></span><span class="line"><span class="cl"><span class="s2">    refer:        whois.apnic.net
</span></span></span><span class="line"><span class="cl"><span class="s2">    inetnum:      1.0.0.0 - 1.255.255.255
</span></span></span><span class="line"><span class="cl"><span class="s2">    organisation: APNIC
</span></span></span><span class="line"><span class="cl"><span class="s2">    status:       ALLOCATED
</span></span></span><span class="line"><span class="cl"><span class="s2">    ...
</span></span></span><span class="line"><span class="cl"><span class="s2">  &#34;</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span></code></pre></div><h2 id="what-about-proxies">What about proxies?</h2>
<p>Many users are behind proxies, which connects to the webserver on the user’s behalf. Thus the REMOTE_ADDR is the address of the proxy, not the actual user.  Some proxies have a workaround for this, by placing the user’s real IP address in the <a href="https://en.wikipedia.org/wiki/X-Forwarded-For">X-Forwarded-For</a> (XFF), or newer <a href="https://tools.ietf.org/html/rfc7239">Forwarded</a> HTTP header. However, these headers are easily set by the user, so can not be trusted. Thus for the moment I ignore these headers, and will instead display the proxy’s IP address. It is conceivable to create a whitelist of proxies, that I would trust the XFF header, but for now I didn’t want that headache. Especially since the server side issues requests to external hosts, if I trusted the XFF header, an abusive user could use my site as a proxy, or even use my site as a relay to <a href="https://en.wikipedia.org/wiki/Denial-of-service_attack">denial of service</a> these remote servers.</p>
<h2 id="tying-this-all-together">Tying this all together</h2>
<p>Server side I use <a href="https://cloud.google.com/appengine/">App Engine</a>, and <a href="https://golang.org/">Go</a>. Why? Because I wanted to play with App Engine, and I’m a fan of Go right now. On the client side I use <a href="http://getbootstrap.com">Bootstrap</a> to make the page look nice, and <a href="https://angularjs.org">AngularJS</a>. AngularJS because I’m familiar with it, and because it is really easy to issue an AJAX requests and transform the result into a web page.</p>
<p>I like App Engine, because of the <a href="https://en.wikipedia.org/wiki/Platform_as_a_service">PaaS</a> model. It keeps my costs down, and I don’t need to setup a virtual machine, or create docker images. Instead I just write a single binary and upload it. However, App Engine does place some restrictions on what I can do, in particular limiting outbound connections to ones made via its own library. Thus I had to jump through a few hoops to make the reverse DNS and WHOIS requests. Instead of using <a href="https://linux.die.net/man/3/getaddrinfo">getaddrinfo(3)</a>, I had to issue DNS requests myself using App Engine’s socket library and my own UDP packets on port 53. Luckily the Go DNS library, <a href="https://github.com/miekg/dns">miekg/dns</a>, makes this relatively easy.  Similarly I had to implement the WHOIS lookup by hand, but again aided by a library, this time <a href="https://github.com/domainr/whois">domainr/whois</a>.</p>
<p>In conclusion, though the use of multiple domain names, some AJAX queries, and server side support. I was able to make &ldquo;(a better) What&rsquo;s My IP Address?&rdquo; site in under a weekend.</p>
<p>Check out the <a href="https://github.com/bramp/myip">full source on github</a>, or view the site at <a href="http://ip.bramp.net/">ip.bramp.net</a></p>
</description>
    </item>
    
    <item>
      <title>Peano Curves</title>
      <link>https://blog.bramp.net/post/2016/08/08/peano-curves/</link>
      <pubDate>Mon, 08 Aug 2016 21:35:35 -0700</pubDate>
      
      <guid>https://blog.bramp.net/post/2016/08/08/peano-curves/</guid>
      <description><p>My latest addition to the <a href="https://github.com/bramp/hilbert">hilbert go library</a>, <a href="https://en.wikipedia.org/wiki/Peano_curve">Peano Curves</a>. The original space-filing curve, similar to the Hilbert curve, but a little more complex.</p>
<figure><img src="/post/2016/08/08/peano-curves/peano_animation.gif"><figcaption>
      <h4>Animation of Peano curve with N in the range 1..5</h4>
    </figcaption>
</figure>

</description>
    </item>
    
    <item>
      <title>Introducing Prob.js</title>
      <link>https://blog.bramp.net/post/2016/08/08/introducing-prob.js/</link>
      <pubDate>Mon, 08 Aug 2016 21:33:33 -0700</pubDate>
      
      <guid>https://blog.bramp.net/post/2016/08/08/introducing-prob.js/</guid>
      <description><h2 id="generate-random-numbers-from-different-probability-distributions">Generate random numbers from different probability distributions</h2>
<p>There are multiple libraries for generating random numbers from a uniform, and sometimes normal distributions. However,
I needed code to generate them from a expontential or lognormal distribution. I had written code to do this ~10 years ago
in Java, but I needed a more modern Javascript solution.</p>
<p>Introducing <a href="https://github.com/bramp/prob.js">Prob.js</a>, a javascript library to generate random numbers from different probability distributions. Avaiable via both bower and NPM, prob.js can generate random numbers from the following distrubtions:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-javascript" data-lang="javascript"><span class="line"><span class="cl"><span class="nx">Prob</span><span class="p">.</span><span class="nx">uniform</span><span class="p">(</span><span class="nx">min</span><span class="p">,</span> <span class="nx">max</span><span class="p">)</span> <span class="c1">// Uniform distribution in range [min, max).
</span></span></span><span class="line"><span class="cl"><span class="nx">Prob</span><span class="p">.</span><span class="nx">normal</span><span class="p">(</span><span class="nx">μ</span><span class="p">,</span> <span class="nx">σ</span><span class="p">)</span>      <span class="c1">// Normal distribution with mean and standard deviation.
</span></span></span><span class="line"><span class="cl"><span class="nx">Prob</span><span class="p">.</span><span class="nx">exponential</span><span class="p">(</span><span class="nx">λ</span><span class="p">)</span>    <span class="c1">// Exponential distribution with lambda.
</span></span></span><span class="line"><span class="cl"><span class="nx">Prob</span><span class="p">.</span><span class="nx">lognormal</span><span class="p">(</span><span class="nx">μ</span><span class="p">,</span> <span class="nx">σ</span><span class="p">)</span>   <span class="c1">// Log-normal distribution defined as ln(normal(μ, σ)).
</span></span></span><span class="line"><span class="cl"><span class="nx">Prob</span><span class="p">.</span><span class="nx">poisson</span><span class="p">(</span><span class="nx">λ</span><span class="p">)</span>        <span class="c1">// Poisson distribution returning integers &gt;= 0.
</span></span></span><span class="line"><span class="cl"><span class="nx">Prob</span><span class="p">.</span><span class="nx">zipf</span><span class="p">(</span><span class="nx">s</span><span class="p">,</span> <span class="nx">N</span><span class="p">)</span>        <span class="c1">// Zipf&#39;s distribution returning integers in range [1, N].
</span></span></span><span class="line"><span class="cl"><span class="nx">After</span> <span class="nx">generating</span> <span class="nx">a</span> <span class="nx">distribution</span><span class="p">,</span> <span class="nx">the</span> <span class="nx">following</span> <span class="nx">methods</span> <span class="nx">are</span> <span class="nx">available</span><span class="o">:</span>
</span></span></code></pre></div><p>With a very simple to use API:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-javascript" data-lang="javascript"><span class="line"><span class="cl"><span class="kd">var</span> <span class="nx">r</span> <span class="o">=</span> <span class="nx">Prob</span><span class="p">.</span><span class="nx">exponential</span><span class="p">(</span><span class="mf">1.0</span><span class="p">);</span> <span class="c1">// Create a distribution.
</span></span></span><span class="line"><span class="cl"><span class="nx">r</span><span class="p">()</span>        <span class="c1">// Generates a number within the distribution.
</span></span></span><span class="line"><span class="cl"><span class="nx">r</span><span class="p">(</span><span class="nx">src</span><span class="p">)</span>     <span class="c1">// Generates a number using a `src` of random numbers. (See note below.)
</span></span></span><span class="line"><span class="cl"><span class="nx">r</span><span class="p">.</span><span class="nx">Min</span>      <span class="c1">// The min random number which could be returned by `r()` (inclusive).
</span></span></span><span class="line"><span class="cl"><span class="nx">r</span><span class="p">.</span><span class="nx">Max</span>      <span class="c1">// The max random number which could be returned by `r()` (exclusive).
</span></span></span><span class="line"><span class="cl"><span class="nx">r</span><span class="p">.</span><span class="nx">Mean</span>     <span class="c1">// The expected mean for this distribution.
</span></span></span><span class="line"><span class="cl"><span class="nx">r</span><span class="p">.</span><span class="nx">Variance</span> <span class="c1">// The expected variance for this distribution.
</span></span></span></code></pre></div><p>I created a <a href="https://bramp.github.io/prob.js/">quick demo site</a> that generates 1 million random numbers from each distribution in the browser, and plots the PDF as it goes. Same samples:</p>
<p><figure><img src="/post/2016/08/08/introducing-prob.js/normal.png"><figcaption>
      <h4>Normal (μ = 0, σ = 1.0)</h4>
    </figcaption>
</figure>

<figure><img src="/post/2016/08/08/introducing-prob.js/lognormal.png"><figcaption>
      <h4>Log-normal (μ = 0, σ = 1.0)</h4>
    </figcaption>
</figure>

<figure><img src="/post/2016/08/08/introducing-prob.js/zipf.png"><figcaption>
      <h4>Zipf (s = 1, N = 100)</h4>
    </figcaption>
</figure>
</p>
</description>
    </item>
    
    <item>
      <title>Introducing Hilbert. A Go library to map values onto a Hilbert curve.</title>
      <link>https://blog.bramp.net/post/2015/08/07/introducing-hilbert/</link>
      <pubDate>Fri, 07 Aug 2015 20:53:41 -0700</pubDate>
      
      <guid>https://blog.bramp.net/post/2015/08/07/introducing-hilbert/</guid>
      <description><p>A <a href="https://en.wikipedia.org/wiki/Hilbert_curve">Hilbert curve</a> is a space-filling (snakey) curve through a 2D space:</p>
<figure><img src="/post/2015/08/07/introducing-hilbert/hilbert.png"><figcaption>
      <h4>Image of 8 by 8 Hilbert curve</h4>
    </figcaption>
</figure>

<p>This can be very useful for mapping a 1D value, into a 2D space. For example, it is commonly used to <a href="https://xkcd.com/195/">map IP addresses into a 2D space</a>.</p>
<p>I recently created a library for <a href="https://golang.org/">Go</a> that can map to and from a curve. The project is <a href="http://github.com/bramp/hilbert">hosted on Github</a>, and can be used like so:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go" data-lang="go"><span class="line"><span class="cl"><span class="kn">import</span><span class="w"> </span><span class="s">&#34;github.com/bramp/hilbert&#34;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="c1">// Create a Hilbert curve for mapping to and from a 16 by 16 space.</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="nx">s</span><span class="p">,</span><span class="w"> </span><span class="nx">err</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">hilbert</span><span class="p">.</span><span class="nf">New</span><span class="p">(</span><span class="mi">16</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="c1">// Now map one dimension numbers in the range [0, N*N-1], to an x,y</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="c1">// coordinate on the curve where both x and y are in the range [0, N-1].</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="nx">x</span><span class="p">,</span><span class="w"> </span><span class="nx">y</span><span class="p">,</span><span class="w"> </span><span class="nx">err</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">s</span><span class="p">.</span><span class="nf">Map</span><span class="p">(</span><span class="nx">t</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="c1">// Also map back from (x,y) to t.</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="nx">t</span><span class="p">,</span><span class="w"> </span><span class="nx">err</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">s</span><span class="p">.</span><span class="nf">MapInverse</span><span class="p">(</span><span class="nx">x</span><span class="p">,</span><span class="w"> </span><span class="nx">y</span><span class="p">)</span><span class="w">
</span></span></span></code></pre></div><p>The project contains some demos, such as this cool animations:</p>
<figure><img src="/post/2015/08/07/introducing-hilbert/hilbert_animation.gif"><figcaption>
      <h4>Hilbert curve animation</h4>
    </figcaption>
</figure>

</description>
    </item>
    
    <item>
      <title>Draw UML Sequence Diagrams with Javascript</title>
      <link>https://blog.bramp.net/post/2013/03/23/draw-uml-sequence-diagrams-with-javascript/</link>
      <pubDate>Sat, 23 Mar 2013 00:00:00 +0000</pubDate>
      
      <guid>https://blog.bramp.net/post/2013/03/23/draw-uml-sequence-diagrams-with-javascript/</guid>
      <description><p>I’m happy to announce one of my projects, <a href="https://bramp.github.io/js-sequence-diagrams/">js-sequence-diagrams</a>. This uses Javascript to draw UML sequence diagrams in SVG format. Here is an example:</p>
<div class="text-center">
    <a href="https://bramp.github.io/js-sequence-diagrams/">
		<img src="sample-with-editor.png" alt="js-sequence-diagram example" width="865" height="333" />
	</a>
</div>
<p>You can alter the diagram in real time, and I even have a simple jQuery plugin to make this easy to use on your own sites.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-html" data-lang="html"><span class="line"><span class="cl"><span class="ni">&amp;lt;</span>script src=&#34;sequence-diagram-min.js&#34;<span class="ni">&amp;gt;&amp;lt;</span>/script<span class="ni">&amp;gt;</span>
</span></span><span class="line"><span class="cl"><span class="ni">&amp;lt;</span>div class=&#34;diagram&#34;<span class="ni">&amp;gt;</span>A-<span class="ni">&amp;gt;</span>B: Message<span class="ni">&amp;lt;</span>/div<span class="ni">&amp;gt;</span>
</span></span><span class="line"><span class="cl"><span class="ni">&amp;lt;</span>script<span class="ni">&amp;gt;</span>
</span></span><span class="line"><span class="cl">$(&#34;.diagram&#34;).sequenceDiagram({theme: <span class="ni">&amp;#39;</span>hand<span class="ni">&amp;#39;</span>});
</span></span><span class="line"><span class="cl"><span class="ni">&amp;lt;</span>/script<span class="ni">&amp;gt;</span>
</span></span></code></pre></div></description>
    </item>
    
  </channel>
</rss>
