Clojure

I am working through the tutorials on Purely Functional.

I got a bit hung up right away. I was trying to figure out how to reload files while working in the REPL as I updated the files. I followed this suggestion on StackOverflow by some guy named Dirk. At first, Dirk’s quirk did not work. Was Dirk a jerk? His burst was not voted first. SO at its worst? It made me so weak I could not speak. My REPL I could not tweak, putting an end to my learning streak. I do not want to sound pissed, but I do not wish to be dissed by Lisp. I am too tired for this.

Then I realized that while the files were reloaded, the new functions were not included in any aliases I used when I brought in the namespace with “require”. I could use the new functions, but I had to type out the ENTIRE namespace. O, the huge manatee! I did some googling, and I don’t think anyone mentions that.

You’re welcome.

2018-06-12_23.46.42 Update:

Just so it is easy to copy and paste, here are the commands:

(require '[clojure.tools.namespace.repl :refer [refresh]])
(refresh)

Or you could do this:

(require '[clojure.tools.namespace.repl :as repl])
(repl/refresh)

You might have a function called “refresh”, but you will probably not have a namespace “repl”.

Do this after you follow the directions in the StackOverflow answer. It is better to bring in the clojure.tools namespace in your profile than in every stinkin’ project.

Image from Bible de Moutier-Grandval, MS 10546, a 9th century manuscript housed at the British Library; image assumed allowed under Fair Use.

I am still going through Clojure For the Brave and True. He does some pretty mind-bending things in chapter 5 when he walks you through his pegthing program.

Check this out:

(defn add-pos
  "Pegs the position and performs connections"
  [board max-pos pos]
  (let [pegged-board (assoc-in board [pos :pegged] true)]
    (reduce (fn [new-board connection-creation-fn]
              (connection-creation-fn new-board max-pos pos))
            pegged-board
            [connect-right connect-down-left connect-down-right])))

In the call to “reduce”, he is sending it a collection of functions called “connect-right”, “connect-down-left” and “connect-down-right”. Then in the function in the reduce, there is a call to “connection-creation-fn”, which is passed to the anonymous function. So he is not actually calling a function called “connection-creation-fn”; that is a placeholder for the functions in the array being passed to “reduce”.

Granted, I knew before this what “reduce” is. But in this chapter he has a lot of functions calling functions. Granted, he does say that reducing over a collection of functions is not something you will do or see very often. Even though I have been looking at Clojure for a while, and this chapter wasn’t really anything new, it was a reminder how different functional programming can be.

You’re welcome.

I am still going through the the code review of twitter-retriever by Eric Normand of Purely Functional. And going through Clojure For the Brave and True.  Sometimes I come home from work and I just don’t feel like doing anything.

I am seeing a lot of nice core functions in Clojure For the Brave and True. It is helping me in my quest to post examples of as much of the core functions as I can.

I am still watching the code review in bits and pieces. I guess I don’t like being criticized, even though he is very supportive (not only of members of his site, but the Clojure community in general) and not being too critical, and I have incorporated a few of the things he has said. I had to do some funky things with tweet IDs to work Twitter timelines that seemed a bit confusing and might not be idiomatic Clojure; Twitter has a guide for that here. That said, I was able to incorporate some suggestions and reduced the lines of code, and I was able to remove one of the SQL calls and use some functions to get the same result.

He did mention that a lot of the defs should be in a let block. I showed this to the Austin Clojure group, and they had the same suggestion.

I still have a lot of println statements in there. I can run this on the command line or call some of the functions in the REPL, but testing some of this stuff is a bit beyond me. I have no idea how to mock/stub database calls or external calls in Clojure.

You’re welcome.

Image from a 9th century manuscript held in the St. Vitus Cathedral in Prague; image from Wikimedia, assumed allowed under Fair Use.

I am still slowly going through Clojure For the Brave and True. I will keep going for the time being until inspiration hits.

I did try another app that uses the WordPress API. It is called WP-API v2 Java Client. I was not able to get this to work either. I thought (briefly) about making a direct SQL call to insert the post. I turned on SQL logging and decided it might be a bad idea. I have heard that many SQL experts are not impressed with WordPress’s schema. Plus I would either have to refactor (slightly) to use MySQL, or figure out how to use two connections.

Eric Normand of Purely Functional put up his code review of twitter-retriever. Take a look and let me know if it was good. He also has a channel where he talks about functional programming in general. He does these in his backyard.

You’re welcome.

Image from Harley Golden Gospels, Harley MS 2788, a 9th century manuscript housed at the British Library; image assumed allowed under Fair Use.

I think I am done for the time being with the Twitter-retriever.

As of now, it can get the last 3200 tweets of a user (excluding retweets) and capture any tweets going forward. It prints them out in HTML to a file with each tweet in enclosing “li” tags.

I tried to get the WordPress REST API working. I installed WordPress on my laptop, but I had a hard time getting anything to work with the POST command. GET was easy. It is the authentication that stumped me. I got a couple to work with curl, but no POST commands to work with clj-http. I might try again later, but for now I am done. It seemed to be a rabbit-hole of google searches, adding yet another plugin, editing the .htaccess file, etc, etc. For now, what I have is good enough.

I might look into an alternative to environ, but for the time being I think it will do.

I don’t know what I will work on next. I might get back to Simply Scheme. I might go back to coding examples for the Clojure API. I might go through some lessons at Purely Functional. Maybe TensorFlow (like with guildsman or clojure-tensorflow or tensorflow-clj). Maybe Luminus. As of right now, I have not decided.

You’re welcome.

Image from St. Riquier Gospels, a Carolingian manuscript from the 8th century, housed at the Bibliothèque Municipale d’Abbeville; image from Wikipedia, assumed allowed under Fair Use.

I am still working on the twitter retriever.

I am almost ready to start looking at the wordpress API. I think I will start a local instance so I can work with the REST API and not clutter up this site.

I also looked a bit more at the Clojure library environ. What I like about it is that it gets all the environment variables at start time, and any old time you want you just call the env function, and it works. No fuss, no getting files in every namespace. But I am not so sure if it would work in a production environment. There are a lot of pages from people saying that putting usernames and passwords in the environment is a bad idea, and I am leaning that way too. I have not been able to get my app to use environment variables in .lein-env, profiles.clj or .boot-env when I run it with “lein run”.

I also found out that (after some trial and error) that I cannot compile my app with “lein uberjar”. I use environ for my database creds, and it looks for them during the uberjar process. I was not able to use the profiles.clj or .boot-env for this either. I had to put the database variables into the environment when I compile. (So maybe there is not security risk during production after all.)  Here is what I did in bash:

export DATABASE_URL="postgres://localhost:5433/twit_retrieve_dev_db?user=twit_getter&password=this-is-twitter"
export DB_DRIVER=org.postgresql.Driver

That is not a production database, so no big deal if the creds are here. But that seemed to do the trick. I was able to use the uberjar in another window without any problems (from what I remember).

Also: Eric Normand from Purely Functional got back to me. I told him I was nowhere near done. Partially due to not spending as much time as I should, partially parsing the infamous error messages, partially due to find and trying out libraries as I get this up and running. He wants to do a code review. I think I will clean up a few things before I say yes.

Before I do, I would like to copy and paste in here a few functions for the sake of posterity:

(defn month-string-to-num [month-str]
  (case month-str
    "Jan" "01"
    "Feb" "02"
    "Mar" "03"
    "Apr" "04"
    "May" "05"
    "Jun" "06"
    "Jul" "07"
    "Aug" "08"
    "Sep" "09"
    "Oct" "10"
    "Nov" "11"
    "Dec" "12"))

(defn create-time-from-map [time-from-map]
  (let [matches (re-matches #"(\w{3})\s(\w{3})\s(\d{2})\s(\d{2}\p{Punct}\d{2}\p{Punct}\d{2})\s\+\d{4}\s(\d{4})" 
                time-from-map)
        year        (nth matches 5)
        month       (month-string-to-num (nth matches 2))
        day         (nth matches 3)
        time-of-day (nth matches 4)
            ]
        (println year,"-", month, "-", day, " ", time-of-day  "seconds since midnight")
        (println "Here it is as a string: ", (str year,"-", month, "-", day, " ", time-of-day))
(str year,"-", month, "-", day, " ", time-of-day)))

Those should parse a string with a timestamp like “Tue Feb 11 15:03:45 +0000 2014” and convert it to “2014-02-11_15:03:45”. I then found clj-time.

I also think I found a couple of function calls you can try in the REPL to interactively work on regular expressions:

(re-find (re-matcher #"(\w{3})\s(\w{3}\s\d{2})\s(\d{2}\p{Punct}\d{2}\p{Punct}\d{2})\s\+\d{4}\s(\d{4})" "Sun Dec 20 18:17:33 +0000 2009"))
(re-matches #"(\w{3})\s(\w{3})\s(\d{2})\s(\d{2}\p{Punct}\d{2}\p{Punct}\d{2})\s\+\d{4}\s(\d{4})" "Sun Dec 20 18:17:33 +0000 2009")

I know those are pretty wide, but I found it helpful.

Here is just a little let:

(let [action (:action (:options arg-map))]
     (case action
       "create-user" (do
                       (println "You want to create a user")
                       (actions/create-user user-name my-creds))
       "retrieve-tweets" (do
                           (println "You want to retrieve tweets"))
       (do
         (println "Action ", action, " not specified.")
         (println "Possible actions: create-user retrieve-tweets"))))

You’re welcome.

Image from Athelstan Psalter, a 9th century manuscript housed at the British Library; image assumed allowed under Fair Use.