Adding Dependencies To The Go Project

In a previous post I wrote that if a language has a standard way of making projects, then that should be taught right away, along with automated tests. Then I wrote a post in which I went over making a simple project with Go.

I realized I skipped something important: adding external dependencies. I will re-use the project from the previous post. You should complete the steps in that post before starting this one.

The project already has two packages. This time I will add two files in one of our existing packages. Each file will use a different module. I will also include a test for each.

I picked a couple of project off of the Awesome Go github page.

The first one is stringy (link to module on pkg.go.dev here).


cd spearhead
emacs morestrings/use_stringy.go

Here is the contents of the file:


package morestrings

import (
    "fmt"
    "github.com/gobeam/stringy"
) 

func UseReplaceLast(root_string, old_s, new_s string ) string {
    fmt.Println("in morestrings/UseReplaceLast")
    replaceLast := stringy.New(root_string)
    return replaceLast.ReplaceLast(old_s, new_s)
}

I am only using the “ReplaceLast” function. Notice the line importing “github.com/gobeam/stringy”.

Next I run “go build”:


go build
morestrings/use_stringy.go:5:2: no required module provides package github.com/gobeam/stringy; to add it:
    go get github.com/gobeam/stringy

I will add code to spearhead.go to invoke my function:


fmt.Println("about to call morestrings.UseReplaceLast('this is my string, working with strings, i love strings', 'string', 'rope') : ", 
        morestrings.UseReplaceLast("this is my string, working with strings, i love strings", "string", "rope"))

Before I run “go get”, here is what is in the go.mod file:


more go.mod 
module info/shelfunit/spearhead

go 1.18

Now I run “go get”:


go get github.com/gobeam/stringy
go: added github.com/gobeam/stringy v0.0.5

Here is go.mod:


module info/shelfunit/spearhead

go 1.18

require github.com/gobeam/stringy v0.0.5 // indirect

There are no issues running “go build” and running the executable:


go build
ericm@latitude:~/github/goapps/spearhead$ ./spearhead 
Hello, world.
about to call numberstuff.AddOneToSum( 5, 6):  12
about to call numberstuff.SubtractOneFromSum( 5, 6 ):  10
about to call numberstuff.AddOneToProduct( 5, 6):  31
about to call numberstuff.SubtractOneFromProduct( 5, 6 ):  29
about to call morestrings.ReturnStringTwice("twice"):  twicetwice
in morestrings/UseReplaceLast
about to call morestrings.UseReplaceLast('this is my string, working with strings, i love strings', 'string', 'rope') :  this is my string, working with strings, i love ropes

The next step is to make a test file: morestrings/use_stringy_test.go


package morestrings

import (
    "fmt"
    "testing"
)

func TestUseReplaceLast(t *testing.T) {
    fmt.Println( "Starting TestUseReplaceLast" )
    cases := []struct {
        root, old, new, want string
    }{
        {"Hello, world, i love the world, dog meets world", "world", "food", "Hello, world, i love the world, dog meets food"},
        {"learning go, will go places, gonna be big",       "go",    "wa",   "learning go, will go places, wanna be big"},
        {"I love life, go live your life, go create life", "life",   "lice", "I love life, go live your life, go create lice"},
    }
    for _, c := range cases {
        fmt.Println( "c.root: ", c.root, ", c.old: ", c.old, ", c.new: ", c.new, ", c.want: ", c.want )
        got := UseReplaceLast(c.root, c.old, c.new)
        if got != c.want {
            t.Errorf("UseReplaceLast(%q, %q, %q) == %q, want %q", c.root, c.old, c.new, got, c.want)
        }
    }
    fmt.Println( "Ending TestUseReplaceLast" ) 
}

When I run “go test -v ./morestrings/” we get all the tests for this module. I will only share the output of the test I just added:


=== RUN   TestUseReplaceLast
Starting TestUseReplaceLast
c.root:  Hello, world, i love the world, dog meets world , c.old:  world , c.new:  food , c.want:  Hello, world, i love the world, dog meets food
in morestrings/UseReplaceLast
c.root:  learning go, will go places, gonna be big , c.old:  go , c.new:  wa , c.want:  learning go, will go places, wanna be big
in morestrings/UseReplaceLast
c.root:  I love life, go live your life, go create life , c.old:  life , c.new:  lice , c.want:  I love life, go live your life, go create lice
in morestrings/UseReplaceLast
Ending TestUseReplaceLast
--- PASS: TestUseReplaceLast (0.00s)

If I run “go test -v ./morestrings/use_stringy_test.go”, I get an error:


# command-line-arguments [command-line-arguments.test]
morestrings/use_stringy_test.go:19:16: undefined: UseReplaceLast
FAIL	command-line-arguments [build failed]
FAIL

I am sure some Golang experts are shaking their heads because I did something wrong. As far as I know, while the “go test” command can run tests using a directory as an argument, it does not work as well when you specify individual files.

To test only the test I just created, I can run this command:


go test -v ./morestrings -run TestUseReplaceLast 

I suppose if you wanted to run multiple tests in a specific file, you would need to give them all a name that is common to the tests in that file, but also unique to that group of tests. So you need to be careful when naming your tests. The argument after the “run” is a regular expression. If I wanted to run the tests from the last post in ./numberstuff/multiplication_enhancements_test.go, I would use this command:


go test -v ./numberstuff/ -run Test.*Product

To run the tests in numberstuff/addition_enhancements_test.go, I would use this command:


go test -v ./numberstuff/ -run Test.*Sum.*

The next package I will use is strutil (link to module on pkg.go.dev here). The file using this module is morestrings/use_strutil.go:


package morestrings

import (
    "fmt"
    "github.com/ozgio/strutil"
) 

func UseCountWords(root_string string ) int {
    fmt.Println("in morestrings/UseCountWords")
    return strutil.CountWords(root_string)
}

Again: note there is an import statement for our module. Here is the output of “go build”:


go build
morestrings/use_strutil.go:5:2: no required module provides package github.com/ozgio/strutil; to add it:
    go get github.com/ozgio/strutil

So I run the “go get” command:


go get github.com/ozgio/strutil
go: added github.com/ozgio/strutil v0.4.0

Now we have two modules in a block in go.mod:


more go.mod 
module info/shelfunit/spearhead

go 1.18

require (
    github.com/gobeam/stringy v0.0.5 // indirect
    github.com/ozgio/strutil v0.4.0 // indirect
)

I will add a couple of calls to our new function in spearhead.go:


fmt.Println("about to call morestrings.UseCountWords('this is my string with words'): ", morestrings.UseCountWords("this is my string with words"))
    root_string := "I am again called upon by the voice of my country to execute the functions of its Chief Magistrate"
    fmt.Println("about to call UseCountWords('I am again called upon by the voice of my country to execute the functions of its Chief Magistrate'): ",
        morestrings.UseCountWords(root_string))

And again I call “go build” and run the executable:


go build
ericm@latitude:~/github/goapps/spearhead$ ./spearhead 
Hello, world.
about to call numberstuff.AddOneToSum( 5, 6):  12
about to call numberstuff.SubtractOneFromSum( 5, 6 ):  10
about to call numberstuff.AddOneToProduct( 5, 6):  31
about to call numberstuff.SubtractOneFromProduct( 5, 6 ):  29
about to call morestrings.ReturnStringTwice("twice"):  twicetwice
in morestrings/UseReplaceLast
about to call morestrings.UseReplaceLast('this is my string, working with strings, i love strings', 'string', 'rope') :  this is my string, working with strings, i love ropes
in morestrings/UseCountWords
about to call morestrings.UseCountWords('this is my string with words'):  6
in morestrings/UseCountWords
about to call UseCountWords('I am again called upon by the voice of my country to execute the functions of its Chief Magistrate'):  19

I will make a test file in morestrings/use_strutil_test.go:


package morestrings

import (
    "fmt"
    "testing"
)

func TestUseCountWords(t *testing.T) {
    fmt.Println( "Starting TestUseCountWords" )
    cases := []struct {
        in_string string
        want int
    }{
        {"Among the vicissitudes incident to life no event could have filled me with greater anxieties than that of which the notification was transmitted by your order, and received on the 14th day of the present month", 36},
        {"I am again called upon by the voice of my country to execute the functions of its Chief Magistrate", 19},
        {"When it was first perceived, in early times, that no middle course for America remained between unlimited submission to a foreign legislature and a total independence of its claims, men of reflection were less apprehensive of danger from the formidable power of fleets and armies they must determine to resist than from those contests and dissensions which would certainly arise concerning the forms of government to be instituted over the whole and over the parts of this extensive country", 79},
    }
    for _, c := range cases {
        fmt.Println( "----- c.in_string: ", c.in_string, ", c.want: ", c.want) // , ", here it is: ", UseCountWords(c.in_string))
        got := UseCountWords(c.in_string)
        if got != c.want {
            t.Errorf("ERROR: UseCountWords(%q) == %d, want %d", c.in_string, got, c.want)
        }
    }
    fmt.Println( "Ending TestUseCountWords" )
}

Here is the result of go test -v ./morestrings -run TestUseCountWords:


=== RUN   TestUseCountWords
Starting TestUseCountWords
----- c.in_string:  Among the vicissitudes incident to life no event could have filled me with greater anxieties than that of which the notification was transmitted by your order, and received on the 14th day of the present month , c.want:  36
in morestrings/UseCountWords
----- c.in_string:  I am again called upon by the voice of my country to execute the functions of its Chief Magistrate , c.want:  19
in morestrings/UseCountWords
----- c.in_string:  When it was first perceived, in early times, that no middle course for America remained between unlimited submission to a foreign legislature and a total independence of its claims, men of reflection were less apprehensive of danger from the formidable power of fleets and armies they must determine to resist than from those contests and dissensions which would certainly arise concerning the forms of government to be instituted over the whole and over the parts of this extensive country , c.want:  79
in morestrings/UseCountWords
Ending TestUseCountWords
--- PASS: TestUseCountWords (0.00s)
PASS
ok  	info/shelfunit/spearhead/morestrings	0.002s

In case you are wondering, while reading through post, what those quotes are, it is my duty to inform you, as any good upstanding citizen would, that they are from the first three inaugural addresses of the heads of government of our great country, whose writing style, which leaves much to be desired by people living in the post-Hemingway era, given that inserting entire thoughts in the middle of sentences can introduce unneeded complexity, I have done my utmost to emulate.

Given the length and content of Washington’s second inaugural address, I get the impression he did not want a second term.

Image from Synaxarium of Euthymius of Athos, an 11-century Georgian manuscript, image from Wikimedia, assumed allowed under public domain.

Making a Go Project

In a post I mentioned that when learning a new programming language we should start out making projects and tests for our code.

One language I am interested in is Go. Here I will list the steps to make a Go project with some tests. I will also include the source code for each file. I know this is redundant since there is a tutorial for this on the Go language site, but I wanted to make one with multiple packages. [Note 1]

First cd to a directory where you will create your project.


go.dir$ mkdir spearhead
go.dir$ cd spearhead
go.dir/spearhead$ go mod init info/shelfunit/spearhead
go: creating new go.mod: module info/shelfunit/spearhead
go.dir/spearhead$ more go.mod 
module info/shelfunit/spearhead

go 1.18

NOTE: You must use slashes, not dots. I put in dots the first time, and it did not work.

First I will make a few functions that do some simple things with numbers. I will make a couple that add two numbers, and then add one to the total. I will make another function that subtracts one from the total. I will do the same for multiplication.


mkdir numberstuff
emacs -nw numberstuff/addition_enhancements.go

Here is the contents of the file:


package numberstuff 

func AddOneToSum( x, y int ) int {
    return x + y + 1
}

func SubtractOneFromSum( x, y int ) int {
    return x + y - 1
}

Now I will write a few tests for this. This will go in numberstuff/addition_enhancements_test.go. Since I am using Emacs, I will create the new buffer while in numberstuff/addition_enhancements.go. I will probably not have too many Emacs commands going forward. Also: I am not too familiar with the Go toolchain or the Emacs Go mode, so I will be running some tests and commands on the command line. When my Emacs-fu is more powerful, I will be able to do it all in Emacs.


package numberstuff

import (
    "fmt"
    "testing"
)

func Test_AddOneToSum( t *testing.T ) {
    fmt.Println("Testing AddOneToSum")
    result := AddOneToSum(3, 4)
    if result != 8 {
        t.Error("Incorrect result, expected 8, got ", result)
    }
}

func Test_AddOneToSumCases(t *testing.T) {
    fmt.Println("Testing AddOneToSumCases")
    cases := []struct {
        a, b, result int
    }{
        {3, 4, 8},
        {4, 5, 10},
        {5, 6, 12},
    }
    for _, c := range cases {
        got := AddOneToSum(c.a, c.b)
        if (got != c.result) {
            t.Errorf( "incorrect result: AddOneToCases(%x, %x) gave %x, wanted %x", c.a, c.b, got, c.result )
        }
    }
}

func Test_SubtractOneFromSum( t *testing.T ) {
    result := SubtractOneFromSum(3, 4)
    if result != 6 {
        t.Error("Incorrect result, expected 6, got ", result)
    }
}

Now I will do the same for multiplication. Here is numberstuff/multiplication_enhancements.go:


package numberstuff 

func AddOneToProduct( x, y int ) int {
    return (x * y) + 1
}

func SubtractOneFromProduct( x, y int ) int {
    return (x * y) - 1
}

Here is numberstuff/multiplication_enhancements_test.go:


package numberstuff

import (
    "fmt"
    "testing"
)

func Test_AddOneToProduct( t *testing.T ) {
    fmt.Println("Testing AddOneToProduct")
    result := AddOneToProduct(3, 4)
    if result != 13 {
        t.Error("Incorrect result, expected 13, got ", result)
    }
}

func Test_SubtractOneFromProduct( t *testing.T ) {
    result := SubtractOneFromProduct(3, 4)
    if result != 11 {
        t.Error("Incorrect result, expected 11, got ", result)
    }
}

Next I will invoke these from my main file, spearhead.go:


package main

import (
    "fmt"
    "info/shelfunit/spearhead/numberstuff"
) 

func main() {
  fmt.Println("Hello, world.")
    fmt.Println("about to call numberstuff.AddOneToSum( 5, 6): ", numberstuff.AddOneToSum( 5, 6 ) )
    fmt.Println("about to call numberstuff.SubtractOneFromSum( 5, 6 ): ", numberstuff.SubtractOneFromSum( 5, 6 ) )
    fmt.Println("about to call numberstuff.AddOneToProduct( 5, 6): ", numberstuff.AddOneToProduct( 5, 6 ) )
    fmt.Println("about to call numberstuff.SubtractOneFromProduct( 5, 6 ): ", numberstuff.SubtractOneFromProduct( 5, 6 ) )
}

Now run this from the command line.


go build

This will create an executable called “spearhead” next to “spearhead.go”. There are two ways I can run this program:


go run spearhead.go

Or just


./spearhead

Either way gives the following:


Hello, world.
about to call numberstuff.AddOneToSum( 5, 6):  12
about to call numberstuff.SubtractOneFromSum( 5, 6 ):  10
about to call numberstuff.AddOneToProduct( 5, 6):  31
about to call numberstuff.SubtractOneFromProduct( 5, 6 ):  29

To run the test, I need to go into the numberstuff directory:


cd numbertest
go test

Here is the result:


Testing AddOneToSum
Testing AddOneToSumCases
Testing AddOneToProduct
PASS
ok  	info/shelfunit/spearhead/numberstuff	0.003s

I put in some calls to fmt to print out stuff to the command line so I know it is working. For that, we have “go test -v”:


go test -v
=== RUN   Test_AddOneToSum
Testing AddOneToSum
--- PASS: Test_AddOneToSum (0.00s)
=== RUN   Test_AddOneToSumCases
Testing AddOneToSumCases
--- PASS: Test_AddOneToSumCases (0.00s)
=== RUN   Test_SubtractOneFromSum
--- PASS: Test_SubtractOneFromSum (0.00s)
=== RUN   Test_AddOneToProduct
Testing AddOneToProduct
--- PASS: Test_AddOneToProduct (0.00s)
=== RUN   Test_SubtractOneFromProduct
--- PASS: Test_SubtractOneFromProduct (0.00s)
PASS
ok  	info/shelfunit/spearhead/numberstuff	0.003s

If I want to run the test from the root of the project, run go test with the relative path to the project, with or without “-v” as desired:


go test ./numberstuff
ok  	info/shelfunit/spearhead/numberstuff	0.002s

Just to make sure I am not going crazy, let’s change one of the result assertions in numberstuff/addition_enhancements_test.go from 8 to 9:


func Test_AddOneToSum( t *testing.T ) {
    fmt.Println("Testing AddOneToSum")
    result := AddOneToSum(3, 4)
    if result != 9 {
        t.Error("Incorrect result, expected 9, got ", result)
    }
}

That gives us this:


go test  ./numberstuff
Testing AddOneToSum
--- FAIL: Test_AddOneToSum (0.00s)
    addition_enhancements_test.go:12: Incorrect result, expected 9, got  8
Testing AddOneToSumCases
Testing AddOneToProduct
FAIL
FAIL	info/shelfunit/spearhead/numberstuff	0.003s
FAIL

Change it back before moving forward.

To prove that I know what I am doing, I will add another package: morestrings. This is the same package that is used in How To Write Go, but I am going to make a different function.

I will make a file morestrings/duplicate_string.go:


package morestrings

func ReturnStringTwice(s string) string {
    return s + s
}

Next I will make a test file morestrings/duplicate_string_test.go:


package morestrings

import (
    "fmt"
    "testing"
)

func TestDuplicateString(t *testing.T) {
    fmt.Println( "Starting TestDuplicateString" )
    cases := []struct {
        in, want string
    }{
        {"Hello, world", "Hello, worldHello, world"},
        {"eating nun arrays funny", "eating nun arrays funnyeating nun arrays funny"},
        {"negative houses gauge freedom", "negative houses gauge freedomnegative houses gauge freedom"},
    }
    for _, c := range cases {
        fmt.Println( "c.in: ", c.in, ", c.want: ", c.want )
        got := ReturnStringTwice(c.in)
        if got != c.want {
            t.Errorf("ReturnStringTwice(%q) == %q, want %q", c.in, got, c.want)
        }
    }
}

Now we run the tests:


go test -v ./morestrings/ 
=== RUN   TestDuplicateString
Starting TestDuplicateString
c.in:  Hello, world , c.want:  Hello, worldHello, world
c.in:  eating nun arrays funny , c.want:  eating nun arrays funnyeating nun arrays funny
c.in:  negative houses gauge freedom , c.want:  negative houses gauge freedomnegative houses gauge freedom
--- PASS: TestDuplicateString (0.00s)
PASS
ok  	info/shelfunit/spearhead/morestrings	(cached)

Now we update spearhead.go:


package main

import (
    "fmt"
    "info/shelfunit/spearhead/morestrings"
    "info/shelfunit/spearhead/numberstuff"
) 

func main() {
    fmt.Println("Hello, world.")
    fmt.Println("about to call numberstuff.AddOneToSum( 5, 6): ", numberstuff.AddOneToSum( 5, 6 ) )
    fmt.Println("about to call numberstuff.SubtractOneFromSum( 5, 6 ): ", numberstuff.SubtractOneFromSum( 5, 6 ) )
    fmt.Println("about to call numberstuff.AddOneToProduct( 5, 6): ", numberstuff.AddOneToProduct( 5, 6 ) )
    fmt.Println("about to call numberstuff.SubtractOneFromProduct( 5, 6 ): ", numberstuff.SubtractOneFromProduct( 5, 6 ) )
    fmt.Println("about to call morestrings.ReturnStringTwice(\"twice\"): ", morestrings.ReturnStringTwice( "twice" ) )
}

We can run all the tests with this command:


go  test -v ./...

That is a period, a slash, and three periods. Sometimes the generator and browser make the three periods into one character.

I know we should be open to new ideas and new ways of doing things, but I wish “go test” just ran all the tests in the project tree. Just like Maven and Gradle. And Leiningen. And Clojure CLI tools And Ruby. And Elixir. Before anyone tries to play the “because Ken Effing Thompson sez so” card, I already do that with the ternary operator: he is the guy who put it in C, and he kept it out of Go.

Note 1: I have also been trying to do this with Elixir. For some reason, I have been able to make a few different modules, but I cannot get the tests to run. I also do not get any errors for the tests. I will step through the relevant chapter in Dave Thomas’ book and post about Elixir in the near future.

Image from Évangéliaire de Saint-Mihiel, a 10th century manuscript held at the Colleges of the Université Catholique de Lille; image from Wikimedia, assumed allowed under public domain.

2022-11-20: New Stuff I Learned: Grep and Databases

I recently learned a couple of new tricks, and I want to share a practice I engage in that I have gotten some compliments on.

I have worked on a few Java web applications, and instead of searching with the IDE, I use grep in cygwin. Sometimes the term I search for is not just in a Java file, but it is also in a Javascript file that has no line breaks. (Seriously, why do people make files like this?) This makes the results harder to read. One possibility is to pipe the results of the grep to another grep:

grep -riIn someString * | grep -v '.min.js' | grep -v '.js.map'

The disadvantage of this method is you lose the color in the output. I read the man page, and I found there is an exlusion flag: –exclude. Even better, you can use it more than once:

grep -riIn --exclude=*min.js --exclude=*.js.map someString * 

I also found out the -I option can exclude binary files, which were also polluting the output.

The database we use is Oracle. I started using the Explain Plan to analyze some queries. You can significanly reduce the cost of a query by using upper to compare strings. Going from this:

where some_column like 'some string%'

to this:

where upper( some_column ) like upper ( 'some string%' )

reduced the cost of a few queries from 20,000 to 40,000 to between 30 to 40. I thought that using “upper” would take longer, since it has to run that comparison on the whole table, and that using equals would be faster, but using “upper” is faster. I do not know if this also holds true for other databases. On Oracle “upper” seems to be faster than “lower”.

Another thing I want to mention is that I have gotten a few compliments on how I format insert statements. I line the column names up, and I put in comments in the values() part of the statement for the name of each column.

You have probably seen insert statements like this:

insert into us_state( state_id, state_name, capital,largest_city, abbreviation, population, median_age, persons_sq_mile,nickname, area_sq_mile, avg_elevation, north_latitude_deg, north_latitude_min, south_latitude_deg, south_latitude_min, east_longitude_deg, east_longitude_min, west_longitude_deg, west_longitude_min, created_date, created_by, modified_date, modified_by)
values (func_to_get_next_id(), 'Illinois', 'Springfield', 'Chicago', 'IL', 12882135, 39, 231, 'Prairie State', 57915, 600, 42, 30, 36, 58, 87, 30, 91, 31, sysdate(), user_id, sysdate(), user_id);

I have some that are worse than that. Imagine that statement with no spaces after the commas.

Compare that to this:

insert into us_state( 
    state_id,           state_name,         capital,            largest_city,       abbreviation,       population,         
    median_age,         persons_sq_mile,    nickname,           area_sq_mile,       avg_elevation,      north_latitude_deg, 
    north_latitude_min, south_latitude_deg, south_latitude_min, east_longitude_deg, east_longitude_min, west_longitude_deg, 
    west_longitude_min, created_date,       created_by,         modified_date,      modified_by 
) values (
    func_to_get_next_id(), -- state_id
    'Illinois',            -- state_name
    'Springfield',         -- capital
    'Chicago',             -- largest_city
    'IL',                  -- abbreviation
    12882135,              -- population
    39,                    -- median_age
    231,                   -- persons_sq_mile
    'Prairie State',       -- nickname
    57915,                 -- area_sq_mile
    600,                   -- avg_elevation,
    42,                    -- north_latitude_deg
    30,                    -- north_latitude_min
    36,                    -- south_latitude_deg
    58,                    -- south_latitude_min
    87,                    -- east_longitude_deg
    30,                    -- east_longitude_min
    91,                    -- west_longitude_deg
    31,                    -- west_longitude_min
    sysdate(),             -- created_date
    user_id,               -- created_by
    sysdate(),             -- modified_date
    user_id                -- modified_by
);

It is a little more typing up front, but it is less thinking later on [1].

I also reformat numbers in my notes. I keep notes as I work on stories/defects. If I get an id or a long number from a database, I will copy it, and then add dashes every third number:

4198275630/419-827-563-0
5832064197/583-206-419-7

Years ago I worked on a project where the IDs were a string based on the date down to the nanosecond with no dashes or separators. So I was dealing with very long strings of about 30 characters, half of which I did not need. I got tired of parsing long numbers in my head. More typing means less thinking. The fewer plates I have to spin in my head, the better. Granted, a lot of things that could reduce my coginitve load are usually out of my control, but I change what I can change.

You’re welcome.

Note 1: The formatting is lined up in the WordPress editor; sometimes it looks crooked in the final version. Things like this are why I sometimes think about going with a static site.

Image from Golden Gospels of Henry III, aka Codex Aureus of Speyer or Speyer Gospels (Speyerer Evangeliar), an 11th-centry manuscript housed in the Royal Site of San Lorenzo de El Escorial; image from Wikimedia assumed allowed under public domain.

Learning Languages Test-first and Project-first

Sometimes I wonder why don’t books and tutorials teach programming languages closer to the way web apps are taught and made: By making projects up front, and by incorporating tests right away.

I am going through some Pluralsight tutorials for Go and Elixir, and while there is a lot of information in them, in most of them the code is not well organized. Only one of the dozen or so Go tutorials that I have been through makes a project. Why not do that every time? That is how we make web apps, especially since after Rails most frameworks are created with generators. You can make projects with most languages, but for some reason it is not emphasized. Granted, Go is a language where you can run code outside of a project: You can just put code into “some_file.go”, and then type “go run some_file.go”, but in my opinion it is a bad practice.

Several years ago, I went through a Rails bootcamp. After I was done, another alumni and I started a small study group to go through the RSpec book by David Chelimsky et al. We only had a few meetings, and I do not remember how far in the book we got. We thought that testing was not covered enough in the bootcamp, and we thought it was odd that some languages (like Ruby) have a strong “test-first” culture but when it comes to learning the language tests are treated like an afterthought. If we tell people to write their apps tests-first (or at least simultaneously with their code), why not teach languages test-first? A lot of books wait until halfway through before talking about tests and/or projects, or leave it until the end.

Even Dave Thomas holds off on projects and testing. I thought one of the guys famous for being part of the agile/extreme/TDD movement would start out with tests right away. The pickaxe book and his Elixir book do not get to them until halfway through; it is chapter 13 out of 25 in the Elixir book. Chapter 13 starts with: “Let’s stop hacking and get serious.” Why not start out serious?

It also might help in learning a language. If you are going through a tutorial, I think it is better to have the different sections as files in a project as opposed to just one-off files or snippets floating in space.

It might seem like it is too much for beginners. But what is another chapter in a 200+ page book? And people should know it anyway. If we want people to be competent professionals, they should learn the proper way to do things from the beginning. I think we need to get rid of this idea that a smart developer can learn a language in a weekend. Yes, syntax can be picked up quickly. But to really be good at a language, you need to know the libraries, frameworks and tools. And how to structure your code so that other people can understand and use it.

To introduce people to projects and tests, a tutorial could make a package (or namespace or whatever your language calls them) that just has a few functions/methods that take a couple of numbers, sums them, and adds one to the sum. And another method that subtracts one from the sum. And you could have another package that does the same for multiplication.

I might re-do some of the Pluralsight tutorials by making them into projects. I noticed there is now one for the Gin web framework. Perhaps someone at Pluralsight read my post a few months back on things I do not like about Javascript. They have a tutorial on Go web services that includes a massive Angular app; it takes up more than 500 MB on my hard drive. I wrote it would have been easier (and better for learning Go) to just make a web app in Go.

I was looking for a way to make a project with Go, when I found a video that creates a project while using VS Code. This tutorial follows a page on the Go site. One thing I do not like about Go is that “go test” does not run tests in all the sub-directories by default. Maybe it does and I am just not as familiar with it. But “mvn test” or “gradle test” or “lein test” will run all the tests.

Keep in mind, I am not a TDD purist. I do not always write failing tests first. I have no issue with writing tests after code; Ken Kousen calls it GDD: Guilt Driven Development. I also have no problem with tests hitting the database. A presenter at Austin Ruby years ago pointed out that you can get mocks and stubs to do what you want. If your code has a lot of mocks and stubs, are your tests telling you that your code does what you think it should do, or are they telling you that you are good at mocking and stubbing? Some TDD people say it is fine to use an in-memory database. Why not use an actual database? You are still going outside your code either way.

If you are a TDD purist, do not bother arguing with me. You will not change my mind, and NOTHING ever changes the mind of a purist. Not even the fact that purists are more likely to turn people off to something than turn them on to it. I announced my Groovy Email Server on a list somewhere, and someone got pissy with me because the first thing they did was run tests, and they failed because there was no database connection. Even though the README clearly states that you need to set up a database first. Like Will Rogers said, some people read the sign, and some people just gots to touch the electric fence.

You’re welcome.

Image from León Antiphonary, a 10th-century manuscript hosted at Virtual Library of Bibliographic Heritage, image allowed under Creative Commons CC BY 4.0.

Random Thoughts On Javascript

Random thoughts on Javascript.

As Matthew Butterick put it, Lisp makes me smarter, while anything named *Script kills brain cells on contact.

Here is a comment from Hacker News that sums it up for me: ..they feel like walled gardens of arbitrary knowledge that only applies to their ecosystem rather than to some fundamental learning about software. When I learned about OOP I was able to understand OOP code in a broad range of applications, languages and frameworks, but learning how react works gives me no insight into anything but react, and it’s even abstracted enough from JavaScript that you could learn React without ever properly knowing how to write JavaScript (which is something I’ve encountered in a handful of new devs).

It does seem like a lot of Javascript developers are like small children: They only know Javascript, and cannot imagine anyone not knowing only Javascript. Like iPhone users but worse. They seem to constantly re-invent the wheel, and there seems to be no collective learning. If Javascript is so wonderful, why are there so many libraries, frameworks and entire languages that transpile to Javascript? If Javascript is so wonderful, why do people go through all that effort to avoid dealing with Javascript directly?

Whenever I tell people I want nothing to do with Javascript, I get the same song and dance that “you have to know Javascript.” First off, I do not use it at my current job, so for a lot of jobs you do not. But I do have to use apps made by Microsoft. Because I have to. I am tired of using garbage because I have to. That is one reason I tend not to call in to the Austin Clojure Meetup when they talk about ClojureScript. Lisp is something I want to learn. I spend enough time with things I have to use. Don’t pollute the stuff I want to use with the stuff I have to use.

And usually you get to the first stage of “you have to use X” because a lot of people are too lazy or too stupid to investigate alternatives. The second stage is the X that you have to use is the only alternative, and most people are too lazy or too stupid to know there ever was an alternative. If I wanted to do what everybody else does, I would not be interested in Lisp or Clojure. You use Lisp because you know the world is wrong.

Maybe everybody else uses it because everybody else is stupid. JS is like Windows: It is a status quo that everybody defends, nobody actually chose, and very few actually like.

I don’t buy the argument that developers should learn Javascript because it is a combination of Lisp (the good parts) and C (everything else). “It has the stuff you like with the stuff you do not like.” I would like my life to have more of the stuff that I like, and less of the stuff I do not like, not to mix them together. “It’s Lisp and C” is really a reason to learn Lisp.

I am tired of being treated like a dog, and people thinking that I will just eat whatever is put in front of me.

But I am not the only one. The README for Coast states “It uses a relational database and renders html on the server without javascript which allows you to ship your web applications faster.” The Pragmatic Bookshelf has a book called Programming Phoenix LiveView: Interactive Elixir Web Programming Without Writing Any JavaScript. Granted, they do not seem to be against JS. I think that LiveView is like GWT: You write code in Elixir (LiveView) or Java (GWT), and it gets translated to JS. Because JS is too complicated.

The PDFs for that book said that we need JS to make single-page apps and give users all the “modern” features of the web: like notifications and infinite scroll. Personally, I need fewer notifications and I hate infinite scroll. JS seems to give nothing but bloat. Granted, there are some things it is useful for: I think a lot of video players are done with JS. But beyond that, all JS has given us is bloat and complexity. I feel like I am spending more and more time on websites waiting for circles to stop spinning.

I think we need to get back to CRUD apps. The kind we used to make 10 years ago. Yes, you had to know several languages, like HTML, CSS, SQL, maybe some Javascript. But you could focus on a language that interested you, like Java, or Ruby, or Python. Now it seems like a lot of people want to make it all Javascript all the time, now it’s Javascript AND maybe a little bit of the thing you really wanted to learn. Two years ago I took a course on Pluralsight about making web services in Go. The course included an Angular app to view the database. The Angular app takes up 589 megabytes on my hard drive. I took the course to learn how to use Go, not deal with Javascript. It reminds me of Joe Armstrong’s line about OOP: I just wanted a banana, not the monkey holding the banana and the tree it is sitting in.

Why they did not have a prerequisite course about making a web application in Go is beyond me.

Javascript is like someone comes to a restaurant, and at first they seem outgoing and charismatic. Then you realize the guy is groping all the women, he’s hammered and you can’t get rid of him.

I haven’t gotten around to reading any of the PDFs I have going over the history of Lisp. Perhaps for a couple of decades the Lisp world was total chaos like JS is today. But now it isn’t. Why not stand on the shoulders of giants?

If my impression of Javascript is out of date, so be it. It wasn’t my idea to ignore history and push a language which was described by its own creator as having “a lot of stupid in it.” He is still jerking around the world by giving us a browser that shields you from ads so they can give you their ads. And if that isn’t shady enough, Brave is involved with corrupt-o-currency. This guy is the poster jerk for crank magnetism.

I am open to learning new things. Lisp. Go. Elixir. Functional programming. Graph databases. I probably need to learn more about relational databases. Just using Emacs will help you to pick up new things: Org mode, CIDER. I learned some Scheme and Racket and how to get them working in Emacs so I could go through Simply Scheme so I could eventually go through SICP.

I have noticed that when people judge you for not being open to new ideas or learning something new, what they are really doing is being judgemental because you are not interested in what they are interested in. Being open to new ideas does not mean being open to all ideas.

You’re welcome.

Image from the Melisende Psalter, a 12th century manuscript assumed to have been created at the Monastery of the Holy Sepulchre; housed in the British Library as Egerton MS 1139, assumed allowed under Public Domain.

2021-11 Emacs Meetup

There was a meeting of EmacsATX, the Austin Emacs meetup, last week. There was no set agenda.

I started off by mentioning that while I have not continued working through Rainer Konig’s Org tutorial, I was using Org a lot more. I am using it to keep track of tasks. I am converting a lot of text files into org files, since a lot of topics in the files appear and re-appear. I also mentioned that I had started using the Emacs Desktop to save sessions. A lot of people were not familiar with that term. I don’t know why Emacs calls the file that saves session information the “Desktop”. Perhaps that is a holdover from the PDP-11 days. I used to keep an instance of JEdit open, and I am slowly phasing that over to an instance that uses the desktop. I also made an alias with “–no-desktop” so other instances to not try to use the desktop file. Since a lot of email is happening on GMail and Outlook, and the Emacs browser is not the greatest, it might be hard to do everything in Emacs these days. But I am using it more and more.

I am also using Org to write this post.

Anyway, one guy ranted about MELPA. He might make a blog post about it. I know some people do not think that Emacs handles packages and dependencies very well, but from what I have read, it used to be worse than it is today.

A few people talked about using LSP, the Language Server Protocol with Emacs. It started with VSCode. I think the idea is to make it easier for an editor to work with different programming languages. I don’t know if this makes language modes unnecessary in Emacs.

There is nothing planned for the next meeting. I pointed out that since EmacsConf 2021 is between now and then, we could watch the videos on our own and talk about them. A few people seemed to like that idea. I have a task tree in an Org file to watch the videos from prior years. This was the first meeting in a few months in which Sacha Chua, the organizer of EmacsConf, was not present.

You’re welcome.

Image from Add Ms 28815, a 10th century manuscript housed at the British Library; image assumed allowed under Public Domain.

2021-01-10 Update

I recently went through an Elixir tutorial on Pluralsight.

I know it was just one tutorial, but I do not think I am sold on Elixir yet. Granted, I know the Erlang VM is good for multi-threading, and it is better to use abstractions for that then dealing with threads directly, like Java does. One count count against Go is that it seems like the multi-threading support is at a lower level. Then again, I think Go in general is lower level.

One thing I find interesting is that Elixir allows you to overload a function with the same number and type of arguments as an alternative to convoluted cond statements (Elixir has “if” and “else”, but you cannot have more than two choices; for “if..else if…else” you use “cond”.)

The example in the tutorial was printing out the song “100 bottles of beer on the wall.” You can use the same code to handle 100 down to 3, but you need to handle 2 and 1 slightly differently. So you could do this:

defp get_sentence(2) do
     # stuff here
end

defp get_sentence(1) do
     # stuff here
end

defp get_sentence(number) do
     # stuff here
end

I do not know if any other language allows that. Maybe Ruby does and I don’t remember or never encountered it.

Another feature that I like is doctests. You can put iex commands in your documentation (“iex” is the interactive Elixir shell), and you need to start them with the “iex>” prompt. When you run your automated tests, Elixir will also run the iex commands in your documentation, so your docs will always have up-to-date examples.

It was just one tutorial, so there was not much about actors or processes. My subscription is up in a few months, and while I am satisified, I do not think I will renew because I plan on getting up to speed on Java and Kotlin by then.

One thing I do not like is the inconsistency. You can put parentheses around a functions arguments. Or not. And like Go and the C family, if you use operators then the arguments are placed differently than they are for functions. An example is “2 + 3” as opposed to “2.add(3)” or “add(2, 3)”. I know the latter two are more typing, but I prefer consistency and clarity over conciseness.

I think I prefer braces over “do…end”.

And I do have a few concerns about the community of Elixir. Back in 2015, there were a lot of articles from mostly Ruby/Rails people who were gushing about how they loved Elixir. Some of their arguments were technical. But a lot of them said that Elixir is new and it all feels the way Ruby and Rails felt a decade before. Do you want to chase the new car smell, or do you want to solve problems?

A few people even pointed out that just as Dave Thomas wrote one of the first books on Ruby, he wrote one on Elixir. Would a guy who started a publishing company want you to learn a new language? Why wouldn’t he?

And speaking of problems: While not everyone has the same path, a lot of people went from Java/Spring/Hibernate to Ruby/Rails and now to Elixir/Phoenix. They have abandoned hard-to-maintain apps for the new hotness twice in a decade. And when they want to Ruby/Rails, some of the criticisms were that it was slow, did not handle multi-threading (I do not know if Ruby still has the global interpreter lock issue) and a lot of critics said that the magic of metaprogramming would cause problems in the long run.

The Ruby/Rails crowd said that developer speed is more important than runtime speed, that they could just send requests to a message broker, and the magic helped them get solutions out faster.

Now they are going to Elixir because it’s faster than Ruby, Elixir can handle multi-threading, and they are tired of dealing with logic spread out across an unmaintainable monolith. In other words, Elixir solves problems that they kept telling the Java crowd were not problems. From “U mad, bro?” to “Eff this” without even acknowledging the critics were right.

I am not against learning new things. But do we need to learn new things just to say we are learning new things? Maybe we need to do some of the old things better. I think a lot of Java apps would be easier to deal with if (like the one I work with) they were upgraded and refactored to use newer versions of Java/Spring/Hibernate. It would be painful, but not changing these apps is painful, and porting to a new language would be painful. And leaving a language for the newest, hottest thing just leaves a problem for someone else. I keep running into old Lispers and Smalltalkers talk about all the things they could do 20, 30 years ago, and I keep wishing there was a way to break the cycle.

Maybe Rails apps will always be impossible monoliths (I am out of the Rails scene, so I don’t know). Sometimes you are better off walking away, especially if the suits won’t allow refactoring for “business” reasons. (It seems like “business” is always used as a reason to say “No” to things that will make my life easier.)

There are some interesting things in Elixir/Erlang land. Is it possible to implement the “Let it crash” philosophy in other languages? A lot of people seem to chase the “new tech sugar high”, and it seems like Lispers had answers all along.

Plus, when I look at companies that use Elixir, it seems like a lot of social media/web 2.0/gig economy companies, or “$BLAH_BLAH for X” companies copying the latest money-losing unicorn out of “the Valley”. Not a lot of large corporations. I am not exactly Mr Corporate, but when I look at some of the lists of clients of Rails/Elixir consulting firms, I think to myself, “If all of these companies went under, I wouldn’t care and it would have no effect on my life.” A lot of hipsters may hate the idea of working for a bank, or an insurance company or a utility, and while trying to survive the yearly purge is draining, people actually use the services those companies provide. At least some of them in the case of banks.

If you are an Erlang/Elixir person reading this, you might be thinking “You got A wrong”, or “You should read B” or “You should talk to C”. Maybe. I have limited time. For now, if I pick a new direction, I would try Go before Elixir.

You’re welcome.


Here are some notes from 2015 or so that I looked at to write this, with comments and more recent links:

https://www.monterail.com/blog/2015/the-elixir-world
Has a link to a tweet that says:
#ElixirConf has a similar vibe to Ruby confs of ~10yrs ago. Excitement. Cool stuff being hacked on. Very thought provoking and energising.
Great
http://solnic.eu/2015/06/06/cutting-corners-or-why-rails-may-kill-ruby.html
https://solnic.codes/2015/06/06/cutting-corners-or-why-rails-may-kill-ruby/
Says monkey patching is killing Ruby

https://teamgaslight.com/blog/4-reasons-were-having-fun-programming-elixir
“What’s more, it’s just a blast in the same way that I felt when I first started playing with Ruby.”

http://www.creativedeletion.com/2015/04/19/elixir_next_language.html
“Like with Ruby 12 years ago, once again I feel that this new platform is so much better than the other solutions. And it is not just a gut feeling, there are rational arguments to back it up.”
What would be more rational is to not change languages every stinking decade.
“Another similarity is that Dave Thomas is also excited about Elixir and has written a book about the language.”
Are you being played here?

https://medium.com/@kenmazaika/why-im-betting-on-elixir-7c8f847b58#.d5skiqwyy

http://www.littlelines.com/blog/2014/07/08/elixir-vs-ruby-showdown-phoenix-vs-rails/
tl;dr Phoenix showed 10.63x more throughput over Rails when performing the same task, with a fraction of CPU load
Hmmm. When it was Ruby vs Java 10 years ago, performance was not that important. Now it is.

Blog post on Let It Crash: https://www.amberbit.com/blog/2019/7/26/the-misunderstanding-of-let-it-crash/

Another one, stating that a lot of Erlang/Elixir people do not understand “Let It Crash”: http://stratus3d.com/blog/2020/01/20/applying-the-let-it-crash-philosophy-outside-erlang/

 

Image from Codex Amiatinus, a 6th century Vulgate manuscript housed at the Laurentian Library in Florence. Image from Wikimedia. This image is assumed to be allowed under Fair Use.

 

June 2020 Update

This month I have been working through a tutorial on JUnit. There are a lot of new things in JUnit 5. I will summarize them in a later post when I am done.

You’re welcome.

2019-05-31 Update: Passwords

I have written about websites with passphrase generators and generating passwords and passphrases locally (particularly on this page and this post). I have decided to just use KeePassXC for password management.

I started using it on my personal laptop, and I really like it. I back the file up onto a thumb drive on a regular basis, or at least when I make changes. I have a few entries for other information, like my car’s license plate and VIN number in case I ever need them. I also have an entry for my shoe size, since I don’t buy shoes too often, but it is nice to have the info readily available when I need it.

The hard drive on my work laptop crashed, and I had to request access to everything again. I downloaded KeePassXC on my work laptop, and I use that and it has made everything easier. I also back it up to the company’s OneDrive, so I will not be locked out of anything again.

I know everybody wants to do everything online all the time, but I really like having my password database in a local file that I can control.

You’re welcome.

Image from the Gerona Beatus, a 10th century manuscript of ‘Commentary on the Apocalypse‘, written in the 8th century by Beatus of Liébana; image from Arachne, licensed under Creative Commons License (BY-NC-ND 3.0).

 

2019-01-09 Update

I recently attended the first meeting of the Austin Kotlin Meetup. I might keep going in the future. I am kind of on the fence about Kotlin. It is nicer than Java, at least Java as most companies use it today.

I get the impression there are a lot of teams using old versions of Java, and old versions of Spring. They might solve a lot of their pain by upgrading and not really have any need to go to a new language. I wish I knew a way to get teams to upgrade. When Java came out, it was the sleek language (vis a vis C++); now it is the old chestnut. Rails has gone through the same cycle. If companies have fallen behind on Java, and fallen behind on Rails, will going to Kotlin really make things better? Maybe Kotlin teams will get bogged down by legacy Kotlin in ten years. Do we need more languages? Why can’t we just be more intelligent about the languages we use? (And since languages seem to becoming more like Lisp, perhaps we should just go to some dialect of Lisp.)

At one point in the meeting, someone asked what makes a language a “functional language”. I thought a language is considered “functional” when a function can be sent as an argument to another function, or a function can return a new function. One person mentioned immutability. Another mentioned tail-call optimization.

I have been working a bit more on Simply Scheme. Chapters 9 and 10 dealt with lambdas and higher-order functions. I think I am understanding functional programming, but I do not feel like I am getting the enlightenment that people keep talking about. Perhaps this is because I have been exposed to these concepts for a while now.

One reason I doubt I am becoming enlightened is sometimes my answers are larger than other people’s. I can make something that works, but it is not always elegant and minimal.

Another reason I doubt my potential enlightenment is that I am not sure it is making me a better programmer at my job. We are running JDK 6, we are using XML to configure Spring, and there is logic in the views. I am not sure how understanding higher-order functions will help me improve things. There are a lot of things I cannot change. I think it should be re-written from the ground up. Perhaps I will get more insights when I finish Simply Scheme.

Frankly, I think a lot of people where I work do not understand a lot of this stuff and have not been exposed to a lot of these concepts. (Most people I talk to have never even heard of Lisp, or Smalltalk, or Erlang or Scala, or Ruby.) I was not exposed to it for a long time, and I did not understand it at first either. When you are surrounded by people who think that the Gang Of Four patterns are the height of technological sophistication, it is easy to think that is the way things should be done. Part of the issue is that the functional programming community does not explain a lot of this stuff very well. I do not think that state is bad, or that changing state is bad. And side effects are hard to avoid if you want a program to be useful. I/O is a side effect, and it is important. I think the real issue is that state changes and side effects can cause problems if they happen when you do not intend for them to happen, and it is better to explicitly control them.

I am considering taking another break from Simply Scheme and making some web apps in Clojure. Perhaps I should work on getting better in the technologies I want to work in, and go for enlightenment later.

The viability of web apps showed up on Hacker News a few times the past few weeks: “Start with a Website, Not a Mobile App” and “Ask HN: Is web development still a viable career choice?” I will look at these a bit and comment more later.

Going back to chapters 9 and 10 of Simply Scheme.: They showed a neat trick to use lambda to wrap a higher-order function that only takes one argument (like every, keep and accumulate, which are their versions of Clojure’s map, filter and reduce) and put it in an anonymous function so it can take two or more arguments. Pretty slick.

Note: To search the c2.com wiki, try the search page.

Image from “Beatus a Liebana, Commentarius in Apocalypsin”, a 9th century manuscript housed at the Bibliothèque nationale de France, aka Saint-Sever Beatus (more info in French at this page). Source gallica.bnf.fr / BnF; image assumed allowed under Fair Use.