I am still working on the twitter retriever.
I am almost ready to start looking at the wordpress API. I think I will start a local instance so I can work with the REST API and not clutter up this site.
I also looked a bit more at the Clojure library environ. What I like about it is that it gets all the environment variables at start time, and any old time you want you just call the env function, and it works. No fuss, no getting files in every namespace. But I am not so sure if it would work in a production environment. There are a lot of pages from people saying that putting usernames and passwords in the environment is a bad idea, and I am leaning that way too. I have not been able to get my app to use environment variables in .lein-env, profiles.clj or .boot-env when I run it with “lein run”.
I also found out that (after some trial and error) that I cannot compile my app with “lein uberjar”. I use environ for my database creds, and it looks for them during the uberjar process. I was not able to use the profiles.clj or .boot-env for this either. I had to put the database variables into the environment when I compile. (So maybe there is not security risk during production after all.) Here is what I did in bash:
export DATABASE_URL="postgres://localhost:5433/twit_retrieve_dev_db?user=twit_getter&password=this-is-twitter" export DB_DRIVER=org.postgresql.Driver
That is not a production database, so no big deal if the creds are here. But that seemed to do the trick. I was able to use the uberjar in another window without any problems (from what I remember).
Also: Eric Normand from Purely Functional got back to me. I told him I was nowhere near done. Partially due to not spending as much time as I should, partially parsing the infamous error messages, partially due to find and trying out libraries as I get this up and running. He wants to do a code review. I think I will clean up a few things before I say yes.
Before I do, I would like to copy and paste in here a few functions for the sake of posterity:
(defn month-string-to-num [month-str]
(case month-str
"Jan" "01"
"Feb" "02"
"Mar" "03"
"Apr" "04"
"May" "05"
"Jun" "06"
"Jul" "07"
"Aug" "08"
"Sep" "09"
"Oct" "10"
"Nov" "11"
"Dec" "12"))
(defn create-time-from-map [time-from-map]
(let [matches (re-matches #"(\w{3})\s(\w{3})\s(\d{2})\s(\d{2}\p{Punct}\d{2}\p{Punct}\d{2})\s\+\d{4}\s(\d{4})"
time-from-map)
year (nth matches 5)
month (month-string-to-num (nth matches 2))
day (nth matches 3)
time-of-day (nth matches 4)
]
(println year,"-", month, "-", day, " ", time-of-day "seconds since midnight")
(println "Here it is as a string: ", (str year,"-", month, "-", day, " ", time-of-day))
(str year,"-", month, "-", day, " ", time-of-day)))
Those should parse a string with a timestamp like “Tue Feb 11 15:03:45 +0000 2014” and convert it to “2014-02-11_15:03:45”. I then found clj-time.
I also think I found a couple of function calls you can try in the REPL to interactively work on regular expressions:
(re-find (re-matcher #"(\w{3})\s(\w{3}\s\d{2})\s(\d{2}\p{Punct}\d{2}\p{Punct}\d{2})\s\+\d{4}\s(\d{4})" "Sun Dec 20 18:17:33 +0000 2009"))
(re-matches #"(\w{3})\s(\w{3})\s(\d{2})\s(\d{2}\p{Punct}\d{2}\p{Punct}\d{2})\s\+\d{4}\s(\d{4})" "Sun Dec 20 18:17:33 +0000 2009")
I know those are pretty wide, but I found it helpful.
Here is just a little let:
(let [action (:action (:options arg-map))]
(case action
"create-user" (do
(println "You want to create a user")
(actions/create-user user-name my-creds))
"retrieve-tweets" (do
(println "You want to retrieve tweets"))
(do
(println "Action ", action, " not specified.")
(println "Possible actions: create-user retrieve-tweets"))))
You’re welcome.
Image from Athelstan Psalter, a 9th century manuscript housed at the British Library; image assumed allowed under Fair Use.