When using third-party packages in Go, they are imported by a path that represents how to download that package from the Internet. For example, to use the popular structured logging library, Logrus, it would imported at the top of the Go program like so:

import (
  "github.com/sirupsen/logrus"
)

When go get is then executed, it fetches the Logrus source code from GitHub and places the code in the $GOPATH/src directory. Take a look for yourself:

$ tree $GOPATH/src
...
├── github.com
│   ├── Sirupsen
│   │   └── logrus
...

An astute reader may wonder, how exactly does go get know that github.com/sirupsen/logrus is a Git repository, and that it can be fetched via the git protocol from that URL. The go get binary could have some smarts in it, that knows about GitHub, and does the right thing. But that seems inflexible, and problematic if new sites want to be supported. Instead the Go developers built a layer of indirection that allows the go get tool to discover the correct source repo.

As outlined in the Remote Import Paths docs, the go get binary will make a normal HTTP request to https://github.com/sirupsen/logrus (falling back to http if needed) and look at the returned HTML for a <meta name="go-import" tag. This meta tag, can then redirect the go get binary to the correct source code repository for the package.

This meta tag can been seen with curl:

$ curl https://github.com/sirupsen/logrus | grep meta | grep go-import

<meta name="go-import"
  content="github.com/sirupsen/logrus git https://github.com/sirupsen/logrus.git">

That tag says, the package rooted at github.com/sirupsen/logrus can be fetched with git, at the URL https://github.com/sirupsen/logrus.git. The meta tag can express other source control systems, e.g Mercurial, Bazaar, Subversion.

GitHub is a very convenient place to host source code, but the GitHub URL is generic. Instead it is possible to use the <meta> tag to create vanity domains to host projects. For example, the package hosted at github.com/bramp/goredirects could instead be imported as bramp.net/goredirects. All that is needed is a static HTML page at bramp.net/goredirects, containing the following <meta> tag pointing at GitHub.

<meta name=go-import
  content="bramp.net/goredirects git https://github.com/bramp/goredirects.git">

Incase a user attempted to visit that page directly with their web browser, it is worthwhile placing more information about the project on the page, or simply making the page redirect.

To help make these redirect pages, I wrote a simple go tool, goredirects, that inspects all local repositories under a vanity domain directory in the local $GOPATH/src/ and outputs static HTML pages that can be hosted on that domain.

For example, create your new project on GitHub, but check out the project under $GOPATH/src/example.com/project. Then run the tool:

$ go install bramp.net/goredirects
$ goredirects example.com outputdir

The directory outputdir will now contain multiple directories and html files, one for each project under $GOPATH/src/example.com. These HTML files contain the appropriate goimports meta tag to redirect the download of source code from the vanity name, to GitHub. Just upload these files to your website, voilà you are done. Examples of these vanity redirect files can be found on bramp.net, e.g bramp.net/goredirects/index.html. This tool even works for packages with sub-packages under the main root.

Finally, it is possible to ensure that if someone finds your project via GitHub, that go get will always place it under your vanity domain. This be can be achieved with an import comment. Within the source code, ensure that at least one of the files in your page has a comment like so:

package project // import "example.com/project"

Then go get will enforce the correct/vanity URL to use, instead of the true location.

More helpful links on the topic:

• golang.org/cmd/go/#hdr-Import_path_checking
• golang.org/cmd/go/#hdr-Remote_import_paths
• golang.org/doc/go1.4#canonicalimports
• godoc.org/golang.org/x/tools/cmd/fiximports
• texlution.com/post/golang-canonical-import-paths/

My latest addition to the hilbert go library, Peano Curves. The original space-filing curve, similar to the Hilbert curve, but a little more complex.

Generate random numbers from different probability distributions

There are multiple libraries for generating random numbers from a uniform, and sometimes normal distributions. However, I needed code to generate them from a expontential or lognormal distribution. I had written code to do this ~10 years ago in Java, but I needed a more modern Javascript solution.

Introducing Prob.js, a javascript library to generate random numbers from different probability distributions. Avaiable via both bower and NPM, prob.js can generate random numbers from the following distrubtions:

Prob.uniform(min, max) // Uniform distribution in range [min, max).
Prob.normal(μ, σ)      // Normal distribution with mean and standard deviation.
Prob.exponential(λ)    // Exponential distribution with lambda.
Prob.lognormal(μ, σ)   // Log-normal distribution defined as ln(normal(μ, σ)).
Prob.poisson(λ)        // Poisson distribution returning integers >= 0.
Prob.zipf(s, N)        // Zipf's distribution returning integers in range [1, N].
After generating a distribution, the following methods are available:

With a very simple to use API:

var r = Prob.exponential(1.0); // Create a distribution.
r()        // Generates a number within the distribution.
r(src)     // Generates a number using a `src` of random numbers. (See note below.)
r.Min      // The min random number which could be returned by `r()` (inclusive).
r.Max      // The max random number which could be returned by `r()` (exclusive).
r.Mean     // The expected mean for this distribution.
r.Variance // The expected variance for this distribution.

I created a quick demo site that generates 1 million random numbers from each distribution in the browser, and plots the PDF as it goes. Same samples:

GitHub just announced that all github.io sites can enforce the use of HTTPS. Previously GitHub supported HTTPS on these sites, but you couldn’t force users to use HTTPS other than using a javascript redirect hack, or putting a CDN infront of GitHub. Now by checking a single box you can force users to the secure version:

Sadly I have far too many repositories, and I don’t recall which one uses pages. I figured it would be easy to hit GitHub’s API and enabled this. The API doesn’t directly support enabling HTTPS, so instead I wrote a quick script to list all repos with GitHub pages:

#!/usr/bin/env python
# Prints a list of all owned repositories with pages.
# by Andrew Brampton 2016 https://bramp.net
#
import requests

headers = {'Authorization': 'token XXXXX'} # Replace XXXX with a token from https://github.com/settings/tokens
params = {'type': 'owner', 'page': 1}

while True:
	r = requests.get('https://api.github.com/user/repos', params=params, headers=headers)
	repos = r.json()
	if not repos:
		break

	for repo in repos:
		if repo['has_pages']:
			print "%32s %s/settings" %(repo['name'], repo['html_url'])

	params['page'] += 1

Download turn-on-github-ssl.py

This print out something like so:

$ ./turn-on-github-ssl.py

                blog https://github.com/bramp/blog/settings
  ffmpeg-cli-wrapper https://github.com/bramp/ffmpeg-cli-wrapper/settings
js-sequence-diagrams https://github.com/bramp/js-sequence-diagrams/settings
           js-sudoku https://github.com/bramp/js-sudoku/settings
        nodewii-talk https://github.com/bramp/nodewii-talk/settings
             prob.js https://github.com/bramp/prob.js/settings
                test https://github.com/bramp/test/settings
              unsafe https://github.com/bramp/unsafe/settings
...

Now click on each URL, and just check the box.

After getting my js-sequence-diagrams project onto Hacker News, the popularity has gone viral.