Welcome to GoNB¶

GoNB is a Go notebook kernel. It allows one to easily run Go code in a Jupyter Notebook and variations.

In this tutorial we will walk through most of its features, and explain how it works.

See the README.md's Installation section. It also includes a Docker with Jupyter+GoNB pre-installed, that makes it trivial.

Hello World!¶

Whenever a cell is executed, gonb saves the cell content of the cell to a Go file, auto-imports missing dependencies (when it can guess), compiles and runs it. It may seem a lot, but its pretty fast (except maybe the first cell run that is usually a little slower) and feels interactive.

Easy, right ? Now when trying different things, to avoid having to write func main() at every cell, gonb provides a short cut: "%%". Anything after a "%%" will be wrapped inside a func main() { ... }.

So, let's redo our example above:

Globals and Updates¶

Imports, functions, constants, types and variables global declarations are memorized once executed, and carry over from one cell to another.

So one can work on different functions let's say on different cells.

Let's test it out:

Ok, now we have incr defined to any numeric type, we can use it in all our future cells.

Some quick tests:

Note: Only the various declarations are carried over from one cell to another, not the results of the execution, including updates to variables.

So for instance, if we initialize a variable startValue with 1, then increment it in one cell. Next time we execute a new cell, it will be again initialized to 1.

Now if we execute again, startValue is again initialized to 1:

If one wants to save results calculated from one cell to another, GoNB includes the github.com/janpfeifer/gonb/cache package that makes it trivial to save and load previously generated results.

Example: Below VeryExpensive is only called once for CachedValue, so you will notice that if you run the cell multiple times, it will display always the same number, while NonCachedValue will always call VeryExpensive again, and display another number. So the string "...calculating..." is printed twice only the first time.

The cache package has many more features, check out its documentation.

Now, we don't want to have NonCachedValue execute VeryExpensive at every cell, so let's remove its definition. See how to manage memorized definitions using %help, in the "Managing Memorized Definitions" section, it's displayed at the end of the tutorial.

Imports¶

A few things to remember from imports in gonb:

• Like all other global declarations, they are memorized and carry over from one cell to the other. Still it's good practice to have each cell import what it needs -- no harm in importing the same thing multiple times.
• gonb runs goimports before compiling the code: that means it will be able to automatically add import statements for builtin packages (in the examples above we never wrote import "fmt", and it just worked).
• gonb runs go get before compiling the code. This automatically fetches an external import dependency. That is convenient in most cases, but in case you want to get an external Go module at an specific version, you can do it manually with something like !*go get <github.com/user/my_go_module>@<my_version>. See below on running shell commands.

Let's create a simple example that imports a delighful progress-bar library. Notice it automatically fetches the lastest version of the library github.com/schollz/progressbar/v3 -- and the execution of the cell the first time may take a few seconds because of that.

Displaying HTML, Image and others¶

One of the things that makes working in Notebooks better than using a terminal is that one can display rich content, like dynamically generated images, plots, HTML, even videos and sound.

We'll follow with a few examples of what is already supported.

Improved Hello World!¶

gonb includes the accompanying library gonbUI that handles the interfacing to the Notebook through a very simple API:

Markdown¶

Markdown output is also supported -- and used for the %help command, see the bottom of the tutorial.

This also renders math formulas using latex, use $x^2$ for formulas inlined in text, or $$x^2$$ for formulas in their own line.

Example:

Fractals¶

Let's draw a fractal, using another fun package: github.com/benc-uk/gofract

SVG Drawing¶

From the amazing SVGo library, I really wish I was that creative. Below is Antony Stark's Shining example, demoed here

Plotting with Margaid (SVG)¶

A real pearl!

Since its latest update is not yet "released" (tagged in Git), we needed to get the version on the specific commit. See "Executing Shell Commands" below.

Animated Plots with UpdateHtml¶

Still using Margaid but now we animate a Sin(x) plot varying the frequency from 0.0 to 10.0, every 10 milliseconds. This demonstrates gonbui.UpdateHtml(id, html): it allows a transient HTML cell to be updated in the middle of the execution of a cell.

Note: This also works with markdown, use gonbui.UpdateMarkdown(id, markdown) instead.

Plotting with Gonum Plot (SVG and PNG)¶

Another great plotting library.

The version in SVG looks better though:

Plotting with Plotly¶

Plotly has really fancy plots, worth exploring for generating exquisite reports.

Currently supported using github.com/MetalBlueberry/go-plotly library.

Experimental: but if you bump into any issues please report, and we'll try to fix it.

Desktop App Development with Fyne¶

If using a Jupyter-Lab ran with access to X11, one can also experiment with desktop UI programs. Let's try the Hello World from Fyne, a popular, high quality graphical application toolkit for Go.

Important Note: Disabled by default: since this have many requirements, and won't run in a test environment. But feel free to copy&paste the code to a new cell and run it!

Note 1: Fyne needs to link many C++ libraries, so the firt time this is used it takes a few minutes to compile everything. But afterwards it becomes immediate and interactive.

Note 2: To compile it it requires some libraries present in the system. See Fyne's Getting Started.

Note 3: Remember to close the newly created small "Hello World!" window: GoNB is blocking while running a cell, and the cell only finishes to execute when the window is closed. Here we add a timeout just in case.

import (
    "log"
    "time"

    "fyne.io/fyne/v2/app"
    "fyne.io/fyne/v2/widget"
)

%%
a := app.New()
w := a.NewWindow("Hello World")
go func() {
    <- time.Tick(10 * time.Second)
    log.Printf("Timed out, exiting...")
    os.Exit(1)
}()

w.SetContent(widget.NewLabel("Hello World!"))
w.ShowAndRun()
fmt.Println("Goodbye!")

Init Functions¶

Since there is always only one definition per function name, it's not possible for each cell to have it's own init() function. Instead GoNB converts any function named func init_something() to init() before compiling and executing the cells. This way each cell can create its own func init_...() and have it called at every cell execution.

Example:

Now, we don't want to have init_a and init_b on all of our cells below. So we remove those definitions -- see how to manage memorized definitions using %help, in the "Managing Memorized Definitions" section, it's displayed at the end of the tutorial.

Flags¶

The %% command can also be used to set arguments to the execution of the cell. This makes it easy to configure different runs of the same code using flags. This is something handy when testing or developing code that is shared with a normal code that already used flags.

Also,%% not only wraps the code following it in a func main() { ... } but also automatically adds a call to flag.Parse().

Example:

Alternatively one can also set the arguments for execution with %args, if not using %%, as in:

Interacting with the DOM¶

The package gonbui/dom allows one to manipulate the DOM in the browser in arbitrary ways.

It is very useful if implementing new widgets. Things to check out for:

• dom.TransientJavascript: Executes Javascript in a transient cell output and then erases it (so it's not saved with the notebook).
• dom.Append: Appends HTML to a any element identified by an Id.
• dom.SetInnerHtml and dom.GetInnerHtml: Manipulate the innerHTML of any element.
• dom.Persist: Takes the innerHTML of a transient element, and then displays it normally with gonbui.DisplayHtml, so it can be saved with the notebook. Useful if using Javascript libraries (plotting for instance) that dyncamically generate content.

Check the library documentation for more!

Widgets¶

GoNB allows you to use the browser not only to output rich content (HTML, Widgets), but also interact with HTML UI elements.

It provides the widgets package with a few basic widgets for you to use, and a gonbui/comms package that you can use to create new widgets -- see docs/FrontEndCommunication.md file for more details on that.

Let's reuse our Sine curve plotting function below, and connect it to a slider. And then create a button to stop the interaction when we are done.

(To allow this notebook to run automatically, we add a timeout -- if your playing with the tutorial with a live Jupyter Notebook, feel free to change it or remove the timeout)

User Input¶

There are different ways to provide input to a program in GoNB. We list them below and introduce a new one:

1. Simply define constants or variables in the cell and use them. The cell itself is a good way to enter input.
2. Add flags, and pass the values of flags after the %% command. For instance %% --x=10 will run your cell with the flag x set to 10. This is handy for instance to test a function with different values, each one in a different cell.
3. Read the input from an external file. One can edit the file in Jupyter or another text editor. It's easy to set os.Stdin to the desired file.
4. Use HTML widgets (see gonbui/widgets package, described above).
5. Jupyter Notebooks also provides a form of input request, where it displays an in-place text box. The gonbui package has a function to do that, the results of which can be read in from the stdin afterwards. See the following example:

import (
    "fmt"
    "github.com/janpfeifer/gonb/gonbui"
)

%%
gonbui.RequestInput("Tell me a number: ", false)
var x int
_, err := fmt.Scan(&x)
if err != nil { panic(err) }
fmt.Printf("The number you typed was %d\n", x)

gonbui.RequestInput("Tell me a secret: ", true)
var secret string
_, err = fmt.Scan(&secret)
if err != nil { panic(err) }
fmt.Printf("Shh! Your secret was %q\n", secret)

The output would be something like:

Tell me a number:  42
The number you typed was 42
Tell me a secret:  ······
Shh! Your secret was "I🧡GoNB!"

Note: Not executed by default because it breaks the automatic tests, but try it out on a new cell!

Executing Shell Commands¶

There are two variations to execute shell commands. They differ only on the directory from where they are executed.

• ! prefix executes what comes next should be executed as a shell command, on the same directory where the kernel is executed -- typically the same directory where the notebook files is saved.

• !* prefix executes what comes next as a shell command, on the temporary directory used to compile the Go program when executing the cells. This includes the go.mod file, that can be manipulated for special use cases, like importing a specific version of a module, or to redirect a module to a local directory for development (see Replace section below)

Example:

You can also use a \ at the end of the line to extend the shell command to multiple lines.

Example:

If some shell program requires some input from the user, you can precede it with a %with_inputs or %with_password (for hidden input) and it will open a text field for typing some arbitrary text input. Example:

%with_password
!sudo -S apt update

Debugging A Cell¶

While GoNB doesn't (yet) talk the debug protocol with JupyterLab, it's easy to start a GUI debugger from a cell, if being executed on the same machine as the browser.

The common Go debugger recommendation is delve, and in particular its front-end gdlv. And to make it simpler GoNB includes a small wrapper script ndlv to set the directory and program name to the last cell executed. Copy or link that script somewhere in your PATH (maybe ${HOME}/bin if you have such directory set up).

To open the debugger, after executing a cell you want to debug, you create and execute a new cell with this single shell command:

!ndlv

Environment Variables¶

For convenience, GoNB defines the following environment variables -- available for the shell scripts (! and !*) and for the Go cells:

• GONB_DIR: the directory where commands are executed from. This can be changed with %cd.
• GONB_TMP_DIR: the directory where the temporary Go code, with the cell code, is stored and compiled. This is the directory where !* scripts are executed. It only changes when a kernel is restarted, and a new temporary directory is created.
• GONB_PIPE: is the named pipe directory used to communicate rich content (HTML, images) to the kernel. Only available for Go cells, and a new one is created at every execution. This is used by the gonbui functions described above, and doesn't need to be accessed directly.

Tests and Benchmarks¶

GoNB allows also executing the cell with go test, to demo or debug tests and benchmarks. For that just mark the cell with %test and if no other parameters are given, it will run the tests and benchmarks defined in the current cell.

If it is the only command in the cell, it will run all tests defined so far.

Alternatively, you can provide extra flags that are passed to the test binary. For instance, to execute all benchmarks defined so far, use:

%test -test.bench=. -test.run=Bechmark

Developing Go libraries with a notebook: go.mod and go.work¶

GoNB uses go.mod and understands go.work -- but won't create it by default.

"Replace" modules in go.mod¶

Using the !* command above we can easily "replace" a module to a local directory. This can be very handy for developing a library in a powerful IDE on the side, and using the GoNB notebook to execute tests and experiments. Changes in the library (in the IDE) when saved immediate take effect on the next cell execution.

The accompaining library gonbui was implemented mostly in this fashion using a notebook, that started with:

!*go mod edit -replace github.com/janpfeifer/gonb=/home/janpf/Projects/gonb

Check out the results with:

!*cat go.mod

"Use" modules in go.work¶

Another way to refer to modules that are being co-developed in local file is using Go workspaces. GoNB can be configure it automatically when the kernel is started: during GoNB installation use the gonb -install -go_work=<...> flag to set the default used directories of the go.work. Notice if set, it automatically include the "." directory to the list, if not already set.

Alternatively, one can also configure the kernel's go.work directly from the notebook itself. Example:

Note:

• GoNB uses go get to automatically fetch missing imports, but it (by design) ignores go.work use definitions. and attempt to fetch them anyway. The special command %goworkfix handles that by adding a replace entry in go.mod for all modules pointed to by go.work.
• The env variable GONB_DIR to refer to the current kernel directory (changed with %cd). This is just an example, instead here you would use the path to some library you have locally.

Tracked directories¶

GoNB tracks for changes in local files in target directories of replace rules in go.mod or use rules in go.work. This way auto-complete and contextual help stay up-to-date with changes on local files you may be editing on a separate editor.

See %track and %untrack to list and control tracking. For instance, in our tutorial, this is what GoNB is tracking:

Custom go build flags¶

Sometimes it can be desirable to change the Go flags used to build the cells being executed. For instance to check for coverage, or enable race condition detection, etc.

GoNB supports this in two ways:

1. You can tell GoNB to use arbitrary go build flags with %goflags <values ...>. %goflags without any values displays current settings, and %goflags "" will reset to the default (no extra flags).
2. In some cases it's not that one wants GoNB to build the binary with the flags, but simply run go build ... to see the results. This is supported by simply running !*go build ..., which is conveniently executed (the !* prefix) in the temporary directory where the cells code maintained by GoNB is located (in a main.go). E.g.: To look at optimization decisions, use !*go build -gcflags=-m.

See an example in the examples/tests/gcflags.ipynb file, which is used as an integration test.

More ... and Help¶

Some other features:

• Errors reported by the Go compiler are parsed and a mouse-over context is displayed. This facilitates understanding the error, since the line numbers reported by Go are not the same as in the cell being executed.
• Contextual Help: can be activated with "Shift+Tab"" (floating note) or "Control+I" (for a separate sub-window). Notice it may be slow if some new import needs to be fetched -- consider executing once some code with the imports before using the contextual help.

The library is new, and there is still development going on, with still some features in the pipeline (more on the displaying side, auto-complete). For those who enjoy coding, help in improving GoNB is very welcome!

Finally, there is also the %help command, which lists all the currently supported features: