Tag: golang

Tests Timing Out? Check Go Tests Parallelism

Go Tests Parallelism

The command go test ./... lists all packages in the current directory, builds tests executable for each package, and runs them. The run part is interesting.

By default, the number of packages executed in parallel is equal to the value of GOMAXPROCS variable. On my laptop, it is 12 which is the number of CPU cores available.

The other parallelism is inside a package. If a test uses t.Parallel, go test -parallel <num of tests> determines the parallelism.

Why is it important to tune the degree of parallelism

Too many tests running in parallel would time out/crash for want of resources or it could expose a bug. There might be a leakage in the socket, buffer, and so on. If your machine is busy and you see tests timing out often, do try to tune the parallelism.

Tip

Always use go test ./... because it caches tests result for a package. So you save time, CPU in upcoming runs given your changes are not spreading to all packages.

References

  • go help test
  • go help testflag
  • go help build

Credit to StackOverflow
GOMAXPROCS

package main

import (
    "runtime"
    "fmt"
)

func getGOMAXPROCS() int {
    return runtime.GOMAXPROCS(0)
}

func main() {
    fmt.Printf("GOMAXPROCS is %d\n", getGOMAXPROCS())
}

Written with StackEdit.

What is SemVer or Semantic Versioning?

In the Linux world, kernel releases follow MAJOR.MINOR.PATCH version format.

Semver is a similar concept. It advises to follow a simple set of rules:

  1. Major version is changed for incompatible changes.
  2. Minor versions are changed for new functionality
  3. Patch version is changed after bug fixes.

Versions are always incremented and reset to zero:

  1. Patch version is set to zero if the Minor version is changed
  2. Patch and Minor versions are set to zero if the Major version is changed.

References

https://semver.org/

golang: NewReader vs NewBufferString

NewReader

func bytes.NewReader(b []byte) *bytes.Reader

bytes.NewReader on pkg.go.dev

NewReader returns a new Reader reading from b.

NewBufferString

func bytes.NewBufferString(s string) *bytes.Buffer

bytes.NewBufferString on pkg.go.dev

NewBufferString creates and initializes a new Buffer using string s as its initial contents. It is intended to prepare a buffer to read an existing string.

Difference

// A Reader implements the io.Reader, io.ReaderAt, io.WriterTo, io.Seeker,
// io.ByteScanner, and io.RuneScanner interfaces by reading from
// a byte slice.
// Unlike a Buffer, a Reader is read-only and supports seeking.
// The zero value for Reader operates like a Reader of an empty slice.

The bufferString creates a mutable buffer.

go: cannot find main module, but found .git config to create a module there, run: go mod

go: cannot find main module, but found .git/config to create a module there, run: go mod

This problem occurred in a monorepo opened in VSCode. The repo has many services placed in isolated directories with respective go.mod files.

I realized from various sources that the problem is from two places:

  1. Move the mono repo under $GOPATH/src.
  2. The ~/.zshrc or ~/.bashrc must be as follows: 
    export GOPATH=$HOME/go/
export GOROOT=/usr/local/go

export PATH="$GOROOT/bin:$PATH"
export PATH="$PATH:$GOPATH/bin"

Do not add any other variable that affects gopls such as GO111MODULE=on.

Written with StackEdit.

References

How to Add Multiple Directories in GOPATH with VSCode?

Since go versions keep moving up, maintaining GOPATH inside a Go installation path is a pain.

export GOPATH=/home/a-user/go/1.14.9

ls /home/a-user/go/1.14.0/src 
bitbucket.org github.com    go.uber.org   golang.org    gopkg.in

Adding multiple directories in GOPATH

  1. Create a directory e.g. for work 
    mkdir $HOME/mywork
  2. GOPATH assumes a structure in the directory, so create three directories inside. 
    ls $HOME/mywork
bin pkg src
  3. You will do all your personal development work inside $HOME/mywork/src. 
    ls $HOME/mywork/src
project-01
  4. Append .bashrc or .zshrc (append is a must, it won’t work otherwise because GOPATH is preset by goenv) 
    # keep it at the end of the file
export GOENV_ROOT="$HOME/.goenv"
export PATH="$GOENV_ROOT/bin:$PATH"
eval "$(goenv init -)"

export PATH="$GOROOT/bin:$PATH"
export PATH="$PATH:$GOPATH/bin"
export PATH="$PATH:$HOME/bin"

# add personal directory
export GOPATH="$GOPATH:$HOME/mywork/"
  5. Open the directory $HOME/mywork/src/project-01 in VSCode workspace.
  6. If you have the VSCode extension for Go installed, restart VSCode.

If go to implementation doesn’t work, ensure that go is installed properly with src directory.

ls $HOME/go/1.14.0/ 
bin pkg src

That’s all folks!

References

VSCode Plugins for Golang Development

  1. Rich Go language support for Visual Studio Code
  2. Theme: Dark Knight: A simple dark theme for Golang projects
  3. Git Blame & Git History
  4. Postfix templates for Golang
  5. Adds go to implementation on context menu
  6. All-language autocompleter — TabNine uses machine learning to help you write code faster.

This is a minimal list of Go development extensions in my opinion.

Strategy Pattern in Golang

Introduction

The strategy is a behavior pattern. It fits where multiple choices for an interface are available and the design would allow switching them dynamically.

Conditions

  • A set of choices with similarity in the interface
  • Dynamic update of a choice

Example

  1. A cache with evictions
    • Multiple eviction algorithms
    • Cache could switch to a different algorithm dynamically
  2. A duck type has a variety of flying behaviors. A duck can fly very fast, slow and no-fly.

Implementation

Define what a duck is.

// duck_type.go

package main

type duck struct {
	flyBehavior
}

Define fly behavior

// fly_behavior.go

package main

type flyBehavior interface {
	fly()
	nofly()
}

Define the fast duck fly behavior

//fast_duck.go

package main

import "fmt"

type duckF18 struct {
}

func (d *duckF18) fly() {
	fmt.Printf("%+v\n", "flying like a F18")
}

func (d *duckF18) nofly() {
	fmt.Printf("%+v\n", "not implemented")
}

Define no fly behavior

// nofly_duck.go

package main

import "fmt"

type duckNofly struct {
}

func (d *duckNofly) fly() {
	fmt.Printf("%+v\n", "not implemented")
}

func (d *duckNofly) nofly() {
	fmt.Printf("%+v\n", "no flying")
}

// main.go

package main

func createNewDuck(f flyBehavior) *duck {
	return &duck{
		flyBehavior: f,
	}
}

func (d *duck) setFlyBehavior(f flyBehavior) {
	d.flyBehavior = f
}

func main() {
	// f18 duck
	aDuck := createNewDuck(&duckF18{})
	aDuck.flyBehavior.fly()

    // duck switches to no fly
	aDuck.setFlyBehavior(&duckNofly{})
	aDuck.flyBehavior.fly()
}

References

  1. https://golangbyexample.com/strategy-design-pattern-golang/
  2. https://www.youtube.com/watch?v=9uDFHTWCKkQ

golang: How to Add Better Context to Errors

golang: How to Add Better Context to Errors

There are at least three ways to add more context to errors.

  1. fmt.Errorf
  2. errors package
  3. zap library
package main

import (
	"errors"
	"fmt"

	werr "github.com/pkg/errors"
	"go.uber.org/zap"
)

// 1. wraps with a message
func wrap(e error) error {
	return fmt.Errorf("another layer - %w", e)
}

// 2. wraps with callstack and message
func betterWrap(e error) error {
	return werr.Wrap(e, "a better layer")
}

// 3. wraps with the first callstack frame
func zapLogger(e error) {
	logger, _ := zap.NewProduction()
	defer logger.Sync()
	logger.Info("failed to fetch URL",
		// Structured context as strongly typed Field values.
		zap.Error(e))
}

func main() {
	e := errors.New("a test err")

	fmt.Printf("err= %+v\n", wrap(e))
	fmt.Printf("%+v\n", betterWrap(e))
	zapLogger(e)
}

Execution

 $ go run err_context.go

fmt.errorf

err= another layer - a test err

errors package

a test err
a better layer
main.betterWrap
        /Users/xxx/go-code/blog/err/err_context.go:18
main.main
        /Users/xxx/go-code/blog/err/err_context.go:35
runtime.main
        /Users/xxx/.goenv/versions/1.14.0/src/runtime/proc.go:203
runtime.goexit
        /Users/xxx/.goenv/versions/1.14.0/src/runtime/asm_amd64.s:1373

zap

{"level":"info","ts":1604374256.270431,"caller":"err/err_context.go:25",
"msg":"failed to fetch URL","error":"a test err"}

Conclusion

I like the errors package the most. It has a relevant context to troubleshoot a problem.

How to Check nil Interface in Golang?

It’s not a good idea to check an interface for nil.

Try the following code:

type user interface{}
type staff struct{}

func compareNil() {
    var generic user
	generic = nil

    // works as expected
	fmt.Printf("value=%v type=%T (generic==nil)=%v\n", generic, generic, generic == nil)

    generic = (*staff)(nil)

    // fails my expectation
	fmt.Printf("value=%v type=%T (generic==nil)=%v\n", generic, generic, generic == nil)
}

go playground: https://play.golang.org/p/7J9DeIjgNia

Output
value=<nil> type=*main.staff (generic==nil)=false
value=<nil> type=<nil> (generic==nil)=true
Why interface check for nil is special

An interface is a tuple of [Value, Type]. The nil interface is a tuple [nil, nil]. However, the above code is an interface containing [nil, *main.staff] which is not nil.

We can check for nil as follows:

func isNil(i interface{}) bool {                        
    return i == nil || reflect.ValueOf(i).IsNil() 
}

Here i==nil means i has [nil, nil] or has a nil value in [nil, *main.staff].

But if the interface points to a type that has no Nil value:

s := "hello"
generic = s
fmt.Printf("value=%v type=%T type=%v\n", generic, generic, reflect.ValueOf(generic).IsNil())

The code panics:

panic: reflect: call of reflect.Value.IsNil on string Value

goroutine 1 [running]:
reflect.Value.IsNil(...)
        /Users/xxx/.goenv/versions/1.14.0/src/reflect/value.go:1063
main.compareNil()
        /Users/xxx/go/1.14.0/src/mygo/interfaces/interfaces.go:48 +0x3b1
main.main()
        /Users/xxx/go/1.14.0/src/mygo/interfaces/interfaces.go:29 +0x142
exit status 2

Conclusion

The safest way to compare nil interfaces is switch on various types the interface can assume. Never check as myinterface == nil.

References