2022-05-18 Update

It is time for another update.

I am done for the time being editing and making changes to my site. For now I will stick with WordPress, and I will leave the old posts intact.

I have been working more with Clojure. I have not done a lot with Programming Clojure, and at some point I plan on finishing it. I need to work on learning more about Clojure and its capabilities, but I think I want to make an app, even a small one. I think it can help some of the information stick more, and after a while just doing tutorials is a slog. Granted, maybe if I kept going for a few more pages, I would read about something that would give me an idea, but for now I will look into other things. I have mentioned this in the past, but I would like to get into the Clojure rules engine Clara. (Granted, there are a lot of things I would like to do.)

I have been looking into web frameworks for Clojure. I started looking into Luminus more. I went through the first few pages of the tutorial a few years ago, but recently when I tried to run the application it did not work. So I started over. There have been a few changes. My goal was to be able to add some code to get the same results as the Rails routes for a database table. I know a lot of people like to make REST apps, but from what I have seen, when people make REST apps they add data by sending JSON requests and typing them by hand. I find this to be tedious. I went through a tutorial on Pluralsight on making web services with Go, and typing requests in Postman was a drag. This was a few years ago, and I looked at Postman again more recently for work. There were limits on what you could do with the free version. Maybe making forms on HTML pages is more work, but I was put off by Postman.

So I started going through the Luminus tutorial and taking more thorough notes. I went to the Luminus channel on the Clojurians Slack, and someone made a reference to a new framework named Kit. The developer behind Luminus (who goes by “Yogthos” online) is one of the Kit developers. I get the feeling that Kit is intended to replace Luminus. Judging by the projects’ commit histories, Yogthos seems to be spending more time on Kit than Luminus lately. He has a post on his website about Kit; he never comes out and says that Kit is replacing Luminus, but that seems to be the takeaway. Another one of the Kit developers has a post on his site here.

Also, Kit uses the Clojure CLI tools. Luminus uses Leiningen, and gives you the option to use Boot as well (which I think is a hint that Luminus will get less attention going forward than Kit). The CLI tools seemed to have killed Boot stone dead. The most recent tag in the Boot repo was on 2019-04-13. (Granted, Luminus’s most recent tag was in 2015.) I have been seeing fewer mentions of Boot since the CLI tools came out. I think CLI really took away a lot of Boot’s momentum. I am getting the feeling that CLI is also taking some mindshare from Leiningen, although this is happening more slowly since Boot was not as popular as Leiningen. It looks like CLI tools are the future.

So I have also spent some time with the CLI tools. There is a site with some Clojure tutorials called Practicalli. They have a nice page on the CLI tools and a repo with a nice deps.edn config file. For the most part it is pretty easy, but it is a drag learning something else. I am not sure I have got the hang of using CIDER with the CLI tools. I had to run the app in one terminal, and fire up Emacs in another to use CIDER. With Leiningen, I could do it all in Emacs.

Another difference is Luminus manages state with Mount, while Kit uses Integrant. I need to spend some time studying Integrant.

I then “pivoted” to Coast, a web framework that tries to be the Rails for Clojure (site here, Github repo here). It has been around for a few years. I did not look at it because it uses the CLI tools, and for a long time I was not interested in learning about CLI tools. Coast also uses make, which strikes me as odd. Right now I am intersted in making a web app in order to quickly work with data, so I thought that since I am looking into CLI tools, why not try Coast. Plus it looks like Kit is still a work in progress (the devs pretty much said that on Slack), so I let it percolate for a while. Interesting note: I think Yogthos starting working on Kit not too long after the third edition of his book came out.

Coast uses Make for some tasks. That is make, or as cats on the internet would put it: The thing that runz ur see code. Although if you look at the Makefile, the Make commands are aliases to clj commands. So if you really wanted to stick with pure Clojure, you could just run the clj commands.

Recently there was a post on Hacker News about a new web framework for Go called Bud (Hacker News post here, Github repo here). One of the commenters told the Bud developer to try to avoid the word “framework” because a lot of people in the Go community hate the concept, even though some commenters thought that frameworks are useful. When I was in The Starter League, one of the instructors said he liked to handle sessions and authentication himself. I never understood this mentality. A lot of web apps are going to have common needs: handling sessions, routes, authentication, authorization, database connections. What you usually care about is what comes after. I would like to make some pages so I can create data so I can look into a rules engine.

I think you are more likely to make changes to an existing app than build one from scratch. So score one for the framework crowd. Plus, whether the anti-framework people like it or not, this question will come up until there is an answer. I bet nobody looks at Ruby and asks how to write a web app in Ruby.

Image from an 11th century manuscript known as Dionysiou 587, created and housed at the Monastery Agios Dionisiou on Mount Athos, assumed allowed under Public Domain.

Programming Clojure Chapter 04

I got done with chapter 4 of Programming Clojure. This chapter was about functional programming.

Interestingly, they did not spend much time in this chapter on the concept of higher-order functions. They did use them, but it was not a major part of their definition of “functional programming”. They talked more about purity, referential transparency, immutability, persistent data structures, recursion, and laziness and eagerness (for both collections/sequences and transformations).

I have noticed that when Lisp/Scheme programmers talk about “functional programming”, they tend to talk about functions: higher-order, purity, immutability, laziness. When Haskell or Scala programmers talk about “functional programming”, they talk about types as much as they talk about functions.

The section on lazy sequences blew my mind. They worked with Fibonacci numbers, and I made a lazy sequence of factorials. It is amazing that you can calculate some very large numbers very quickly without blowing the stack.

In case you were wondering, here is 200 factorial (split across a few lines so there is no horizontal scroll):

788657867364790503552363213932185062295135977687173263294742533
244359449963403342920304284011984623904177212138919638830257642
790242637105061926624952829931113462857270763317237396988943922
445621451664240254033291864131227428294853277524242407573903240
321257405579568660226031904170324062351700858796178922222789623
703897374720000000000000000000000000000000000000000000000000N

That’s 7.88657 times 10 to the 374th. My answer matches what is on Calculator Soup.

Here is my sequence:

(defn lazy-seq-fact
  ([]
   (concat [1N 1N 2N] (lazy-seq-fact 3N 2N)))
  ([a b]                                                                                                     
   (let [n (* a b)]
     (lazy-seq (cons n (lazy-seq-fact (inc a) n))))))

(nth (lazy-seq-fact) 200)

I am not quite sure I understand transducers or eduction as well as I would like. (I think they may be changing the meanings of the words “eduction” and “transducer” slightly.) I think one of the ideas behind transducers is that a transducer can be more efficient and do things in fewer steps. A threading macro with multiple functions will make a sequence or collection at each step. A transducer does not. They had a couple of examples that they converted from using the threading macro to using transducers and “into”. Only the first step in ->> (the starting collection, where it comes from) and the last step in ->> (the final collection, where to put it, which was a call to “vec”) are flipped when we go to “into”; the transformation steps are pretty much the same. In their example, they combined the steps into a function with “comp”. Also: transducers go right-to-left, which is unintuitive for me.

I think you use “eduction” when you want to transduce part of a collection and not have dangling resources.

I did not understand the threading macro until I had a need for it. Perhaps transducers and eduction will be the same.

I did find a way to use the partition function to work with moving averages. You need the members of your collection to be in order from most recent to oldest. The you can get a collection of moving averages with this:

(map #(/ (reduce + %1) 
         (count %1)) 
     (partition 4 
                1 
                coll-most-recent-to-oldest))

Chapter 5 is about spec. Clojure seems to have a lot of concepts that are new to me, even more than the little Scheme I have worked with.

You’re welcome.

Image from the Liuthar Gospels, a tenth century manuscript created in southern Germany, currently housed in the cathedral of Aachen, assumed allowed under public domain. I have read there are 30 full-page miniatures in the manuscript, but I have not been able to find a pdf or any other pictures than the one on this page.

2021-03-28 Update

There is not a whole lot to report this month. I have not had a lot of time to look at Clojure.

You’re welcome.

Image from Collected Fragments Volume II from the Abbey Library of St. Gall, written around the 8th century from the monastery of St. Gall. Image from e-Codices. This image is assumed to be allowed under Fair Use.

Focusing On Clojure

I have decided to focus on Clojure going forward.

I think of all the Lisps, Clojure right now has the most momentum. And I got a couple of inquiries about Clojure positions.

I have been going through the Clojure API, inspired by an article called “One Weird Trick To Become a Clojure Expert“. The article advocates going through the API alphabetically. Maybe I should have done that, but I think I might be better off going through a few books.

I know at some point I have to stop learning and start making something. But I think that going through the API might not be the best way. And the two are not mutually exclusive.

One of the books I will go through is the third edition of Programming Clojure. One of the co-authors is Alex Miller, who helps make Clojure at Cognitect. Clojure does things differently than other languages, and if anyone can explain it, it is the people at Cognitect. The same publishing company has a book called Getting Clojure, which is written by Russ Olsen, who also works at Cognitect.

Reading books by Cognitect employees is one way you can get Clojure.

You’re welcome.

Image from Ms DF III 3, a 10th century manuscript housed in the Strahov Monastery in the Czech Republic; image from Wikimedia, assumed allowed under Public Domain.

2019-06-30 Update

I am up to chapter 19 in Simply Scheme. This chapter is higher-order functions.

I looked into a couple of other Clojure web frameworks this month. Someone did a presentation on Duct at the Austin Clojure meetup a while back. I might have to look at James Reeves’ presentation on Integrant at Skills Matter, since Duct is built on Integrant. (That presentation is hard to search for; if you search for “Integrant”, you get anything to do with “integration”.)

There is a tutorial on Duct by someone at Circle Ci that I was kind of able to follow. Until I saw this code:

(defmethod ig/init-key :film-ratings.handler.film/create [_ {:keys [db]}]
  (fn [{[_ film-form] :ataraxy/result :as request}]
    (let [film (reduce-kv (fn [m k v] (assoc m (keyword k) v))
                          {}
                          (dissoc film-form "__anti-forgery-token"))
          result (boundary.film/create-film db film)
          alerts (if (:id result)
                   {:messages ["Film added"]}
                   result)]
      [::response/ok (views.film/film-view film alerts)])))

I am not too clear what that does. Perhaps knowing more about Integrant will help. (There are also other presentations on Duct at Skills Matter here and here.)

There is also Clojure on Coast (Github repo here, website here). I think the aim is to be Rails for Clojure. I might try this out too. I would really like to work with Clara Rules, and I think a web app might be the best way to enter data.

Or I might just stick with Luminus.

You’re welcome.

2019-03-14 Update: Simply Scheme, Simply Clojure, Simply Racket

I have been taking a break from Simply Scheme. I got stumped on one of the exercises. I started doing the exercises in Clojure for a couple of reasons. One is that is the most commercially viable Lisp out there right now. Another reason is that I want to run automated tests. I would like to be able to quickly run a function multiple times with different inputs. I was typing and re-typing the same calls over and over. Or I could just use tests.

I do not know how to do tests in Scheme. I was using Chicken Scheme for Simply Scheme. There are a few “eggs” for testing, but the instructions were not that great. Also some of them were not working on the version of Chicken that I was using for Simply Scheme. I use Chicken on Ubuntu and Cygwin, and not directly; they are a bit behind the official version. I did find out recently there is an SFRI for testing. It is implemented by Kawa, so perhaps I could have used Kawa. I do not plan on building any apps with Scheme, or using it long-term. It is just a vehicle for enlightenment. I do not know what the most common test libraries are, and I do not want to spend time on something that I might not be able to use later. Clojure has testing out of the box.

I also started looking into using Racket. I have thought about it before, since there are a lot of language modules for Racket. There is one for Simply Scheme. I also found out about an Emacs mode for Racket. I am just running some tests from some .rkt files, but from what I understand, this is intended to be a replacement for Dr Racket. I do not know how to make an app with Racket, but right now I do not need much. I think I might be better off going with Racket, even if I am using a Scheme language module. While Racket is pretty rare, it is used more than Scheme.

So I have started a project using Racket and the Simply Scheme library to do the Simply Scheme exercises. Maybe someday I will do SICP in Racket, and become a Little, Reasoned or Seasoned Racketeer.

You’re welcome.

Image from ImgFlip, assumed allowed under Fair Use.

2019-02-27 Update: Web Apps and Multimethods

I have started looking at the tutorial for Luminus, the Clojure web framework. I am thinking about making a web app that sends data to Clara, the Clojure rules engine.

I have started looking at the tutorial for Luminus, the Clojure web framework. I am thinking about making a web app that sends data to Clara, the Clojure rules engine.

I have also started a github sub-project simply-clojure, which is a port of Simply Scheme to Clojure. I used multimethods, based on the use seen in Clojure For the Brave and True.

One of the functions that comes with Simply Scheme is appearances, which “returns the number of times its first argument appears as a member of its second argument”. It works with “words” and “sentences”. A “word” in SS is any type of data with a single member; it could be a number or a string with one word. A “sentence” is basically a list.

(appearances 'star '(star trek star wars star gate))
3
(appearances 'a 'stargate)
2
(appearances 1 123145167)
2
(appearances 1 '(1 2 1 2 3 4 1))
2

It can handle various datatypes. For Clojure, I decided to use multimethods. I think they can only take one argument, so I cheated and used a map.

(defmulti my-appearances (fn [the-map] (:type the-map)))

(defmethod my-appearances :sentence
  [the-map]
  (count (filter #(= (:part the-map) %1) (helper/split-string-to-words (:whole the-map)))))

(defmethod my-appearances :word [the-map]
  (count (filter #(= (:part the-map) %1) (helper/split-word-to-letters (:whole the-map)))))

(defmethod my-appearances :vector [the-map]
  (count (filter #(= (:part the-map) %1) (:whole the-map)))

Here are the tests:

(test/deftest test-my-appearances
  (test/testing "Testing my-appearances"
    (test/is (= 2 (my-appearances {:type :word, :part "e" :whole "feelings"})))
    (test/is (= 2 (my-appearances {:type :sentence :part "hello" :whole "hello goodbye hello again goodbye again nothing again" })))
    (test/is (= 3 (my-appearances {:type :vector :part 2 :whole [1 2 3 2 4 2 5]})))))

Code highlighted at http://hilite.me/.

You’re welcome.

Image from León Bible of 920, a 10th century Bible manuscript housed in the Leon Cathedral; image from Wikimedia, assumed allowed under Fair Use.

2018-09-30 Update: Clojure Web and AI

I am still going through Purely Functional. It is interesting that there are so many ways to define a function in Clojure. I wonder if all functional languages are like that.

I have decided to spend time looking into making web apps in Clojure. I know that AI is big, and it would be nice for Clojure to be a force in that space, but things seem to have changed since I blogged about it in May.

The guy who wrote Guildsman is no longer updating it. I don’t know what is going on with dl4clj. (Granted, I haven’t emailed the guy.) The Deeplearning4J site said that was the official Clojure library for Deeplearning4J. It looks like they are now part of the Eclipse Project, and they totally redid their site. And in my opinion, not very well. There are lots of things that look like links on their FAQ page, but nothing happens when you click them. They have a gitter chat. I will try looking into that sometime. The search on gitter is not that great. I think you have to go day-by-day. Here is the link for the archive on 2018-09-26. Cortex still looks like it is or will be a ghost town. If only someone could make all this stuff less complicated.

I will get more into web apps. I know a lot of people think that mobile is the future, but I think web apps will be around for a while. When the iPad came out in 2010, all the Apple fanboys said that within five years we would all be using iPads for everything all the time. But that did not happen. At my job, we all use laptops. When I go to tech meetups, people still use laptops. Gamers are still into desktops.

Sidenote: One thing that I find odd and frustrating is that while 15-20 years ago a lot of technology people thought they were so smart and sophisticated for not trusting Microsoft (or IBM, or Oracle), yet ever since the iPhone came out, those same people won’t say the sun is out until Apple says so. Yet tech people used to make fun of the suits for not thinking the sun is out unless MS said so. Was I wrong to believe in open source?

For one thing, web browsers are on more devices. Phones, tablets and laptops/desktops can all use web apps, but laptops and desktops cannot use mobile apps. Plus I have a hard time typing on phones, and dealing with passwords on phones is very frustrating for me. I prefer dealing with passwords in a password manager or a spreadsheet or an encrypted file. I prefer keeping track of logins and passwords that way rather than deal with a bunch of apps on a phone.

And EVERYBODY makes an app. My bank. My insurance company. The company that processes my rent payments. Which I could make using the app from my credit card company. There is an app for all those stupid scooters all over Austin. Every mom-and-pop Indian place here has an app. My gym has an app. Everybody that Robert Llewellyn talks to on “Fully Charged” has an app. I know my phone is old, but I don’t have a lot of space. I would rather use that space for pictures than yet another app that I might only use one time.

Laptops have bigger screens, bigger keyboards, more hard drive space, more memory; since when did Americans want less?

So there was a thread on the Clojure Google group recently about web apps. A lot of libraries were mentioned. I will go with Luminus first. He walks you through what libraries you need, and you can pick and choose which ones you want to use. Others that were mentioned: Pedestal, Yada, Integrant, and Duct.

I know a lot of Clojure people (and Lispers in general; should we call them Lispters?) don’t like frameworks. They prefer libraries. I never liked the answer “go use libraries.” Nobody ever tells you what libraries to use. I think a lot of people want to be able to produce something quickly that they can see, and frameworks do that. People always like to contrast the “stringing together libraries” approach with Rails. I think that while something like Rails has a lot of “magic” and can do a lot of things for you, magic was not the only reason Rails took off.

What I think helped Rails is that it gave people a default answer. Back then, there was PHP: not a language anybody really loves, and you have to do everything yourself every damn time. “How do you make an app in PHP? From the ground up. Every stinking time.” And you have Java: Lots of frameworks, some of them viable and widely used, some niche. How do you know you are making a good choice and your framework won’t be abandoned later? “How do I make a web app in Java? Pick a framework. We got plenty: A, B, C, D, etc, etc. And then pick a servlet engine. We got a lot of those too!”

“How do I make a web app with Ruby? Use Rails. Go here and do this.” The shortest and most useful answer. Granted, there are/were other frameworks in Ruby, so if you don’t like Rails you have options. But I think having a default answer is the way to go. Choice is great when you understand the choices.

Some people want to make an app, and then go under the hood.

You’re welcome.

Image from ‘Moralia in Job’, a 10th century manuscript at the National Library of Spain, licensed under CC BY-NC-SA 4.0.

Clojure And Gradle Update

I am putting twitter-retriever-gradle on hold. I spent a couple of hours trying to send JVM system properties to the clojureRun task, and I couldn’t figure it out.

I raised an issue on the github repo for the Clojure Gradle plug-in. They made a suggestion, and it did not work out. I was using a few libraries that could use system properties for configuration. That is not the only way to do things, but with Gradle my other option was to have multiple copies of the same file all over the repo. I would rather not do that.

I might ask on the Gradle forum. But for now, I think I will move on to something else.

I think Gradle works well for Groovy and Java. It seems a bit harder to use for Clojure. Back to Purely Functional.

You’re welcome.

Image from ‘Biblia de Ávila [Manuscrito] : [Vetus et Novum Testamentum cum praefationibus et argumentis Sancti Iheronymi et aliorum]’, an 11th century manuscript at the National Library of Spain, licensed under CC BY-NC-SA 4.0.

Clojure With Gradle: twitter-retriever-gradle

I took my twitter-retriever project and reworked it to use Gradle. It is as twitter-retriever-gradle.

To a certain degree, this may just be about satisfying my curiosity. But then again, you might find yourself at a company that uses a lot of JVM languages and everyone knows gradle. Plus I think that a lot of people in the Clojure community are not too thrilled about having to deploy one monster uberjar. With Gradle, you can keep your jars separate.

Before I go any further, let’s go over some issues. I was not able to get the tests to work. The directory structure is different than leiningen, and I had to make a copy of a file. I was not able to use environ for some reason, so I had to hard-code the database info (a lot of Clojure libraries assume you are using lein or boot; to be fair, Clojure people usually are). I will try again; I might just go with cprop [Note 1].

This requires the nebula-clojure-plugin. The Nebula plugins are Gradle plugins made by Netflix. This plugin does not have a lot of documentation. I had to search the source code to see how to do some things. It adds the tasks “clojureRun”, “clojureRepl” and “clojureTest”. You can still call “clean” and “build” as before. I also enabled the application plugin, so “distZip” works.

First thing you have to do is move the files in src/clojure to src/main/clojure.

You can run it two different ways. One is directly with Gradle:

gradle clojureRun --fn='twitter-retriever.work/-main --user=GitHub --oauth=YourNameHere'

I think you have everything after the –fn in a single quote.

You can also use the application plugin. You specify the main class in build.gradle. I found out you can also put in a few command line args as well. For Clojure, your main function will not change, so you can add that along with clojure.main:

mainClassName = "clojure.main --main twitter-retriever.work"

Then run “gradle distZip”, and it will create a tar and a zip file in build/distributions. Expand these, and run the script in the “bin” directory. You can add further command line args when you run it:

bin/twitter-retriever-gradle --user=TweetsToGet --oauth=YourID

For database access, I am using conman, which under its hood uses hugsql. You put your SQL statements in an SQL file which is read in. I put the file on the classpath at “twitter_retriever/sql/statements.sql” (which in this case was $PROJECT/src/main/clojure/twitter_retriever/sql/statements.sql). Gradle could find that when I ran it with the “gradle” command on the command line. But to run it from the expanded zip file that was produced by the distZip command, that did not work. I had to make a copy of the file at $PROJECT/resources/twitter_retriever/sql/statements.sql. Each location only worked for one run method, so I need two copies. I will see if I can get this to work with one file. cprop might help here.

I could not get the tests to run. (First off, the tests must go into src/test/clojure.) It kept having an issue finding statements.sql on the classpath. I tried putting it into src/test/resources/twitter_retriever/sql, but that did not help. Again, cprop might take care of this.

I did not try running a REPL with “gradle clojureRepl”. I was able to use a REPL and do a few things by calling M-x cider-jack-in in an emacs file. When I did, a little option popped up at the bottom asking me if I was using lein or gradle.

So the experiment was not a complete success, but there is some potential.

You’re welcome.

Note 1: I am leaning towards cprop for the regular twitter-retriever. Like the crop README says: What if you have 100 args? That’s 100 properties to put in the environment.

Image from ‘Forum iudicum, Tabulae Chronologicae, Canones Ecclesiae et Opuscula Grammaticae’, an 11th century manuscript at the National Library of Spain, licensed under CC BY-NC-SA 4.0.