Groovy

2021-02-07 Update

by

I completed an introductory Kotlin tutorial on Pluralsight.

Kotlin seems like an okay language. The one tutorial I went through did not touch on Kotlin Native, or Android, or web programming. I get the impression that Kotlin is mostly used for Android. I have little interest in mobile apps. And from what I read from a reliable source, Android apps are getting more complex (see here and here).

So I might put off learning more Kotlin for a while. Kotlin Native looks very interesting, but it is not covered at Pluralsight. Kotlin Native might be a less-painful way to do embedded programming. It would be really interesting if it could be used in robotics. I think that would open up robotics programming to a lot of developers.

WRT the language itself, I am kind of neutral. Every time the instructor said, “Here is something you can do in Kotlin that you cannot do in Java”, it was always something you could do in Groovy that you could not do in Java. Java is adding more features all the time.

I have started going through some of the “What’s new in Java X”, since honestly my Java skills are out of date.

There was a conference at CME about JVM languages about 10 years ago. One Groovy programmer said that as Java got more features, Groovy might become superfluous. To a certain degree, that seems to be happening.

Granted, it might appear that way to me since the Austin Groovy/Grails Meetup stopped meeting long before everything else stopped. And Ken Kousen does not mention Groovy a lot in his newsletter, and there has not been a new episode of the Groovy Podcast since 2020-08. I like Groovy, but why fight the tide? If Ken Kousen is spending time on other things, perhaps that is a sign of the times. I am already shaking one fist at the clouds by looking into Lisp.

One thing that I think held Groovy back is that you really have to know Java in order to use it. Knowing some C will probably help you with Ruby, but I think Ruby shields newbies from C better than Groovy does with Java. I think this is also true of Kotlin, but not as much.

Granted, from what I heard about the origins of Groovy, making Java easier without necessarily replacing seems like it was the point. But for someone who is new to JVM languages (or programming altogther), they kind of have to learn two languages at once. Plus when a lot of people get tired of Java, they tend to stay away from JVM languages altogether.

Maybe I should have found a way to keep the Austin Groovy group going.

I was hoping to get through as much Kotlin and Java on Pluralsight as I could before my subscription expires. But I might just renew it for a year so I do not put as much pressure on myself, and I can also spend time on other things.

Like Clojure. I got a few feelers about Clojure jobs, so I think I will get back to going through the functions in the core Clojure API. I made a list of functions that seem to take functions as arguments or return functions, or take predicates. I think I will look at some of those, since those are the functional parts of Clojure.

I found a tutorial on Clojure Spec (article here, Hacker News comments here), and some of it was new to me. It looks like Clojure is starting to encourage keywords to have namespaces.

You’re welcome.

Image from an 15th century manuscript housed at Stiftsarchiv, image from e-Codices, assumed allowed under Fair Use.

2018-02-11 Update

by

This week there was not much Clojure. I have the call with the Purely Functional guy next Monday. He has a conference this Thursday and Friday. I am not attending. I think a few people from Austin Clojure are going.

I spent more time on the Groovy Mail Server. I just don’t know when to quit. I cannot send mail to Proton Mail. I think they are using Postfix. Every time I try to send an RCPT command, they return “550 5.5.1 Protocol error”. If I am reading the Postfix source code correctly, that can happen if you send an invalid command (RCPT is valid) or if you send a new line either by itself or without a carriage return. I do not think I am sending a command in the wrong form. For one thing, just about every other server I try accepts my mail.

I also had problems with Outlook.com. I was getting a socket exception. From what I gathered from googling, they were ending the socket after the QUIT command. I was not having this problem with Outlook.com mails before. I added a try/catch block, and now I save the emails.

I don’t know how much longer I will continue the Groovy Mail Server. Part of me wants to complete it. On the other hand, since Clojure and Lisp are the ultimate direction I want to go in, maybe I should put it aside. On the first hand, it is always good to keep up with a few extra languages. I would also like to get through Simply Scheme and then SICP, because that would make me a really smart person.

If I really wanted to finish my mail server, I would probably have to get another VPS account and set up something like mail-in-a-box for TLS. I might need a few attempts to get TLS right, and I don’t want everybody banning my emails while I figure it out. There also might be an issue with TLS. I don’t know a whole lot about setting up certificates or how to set up a certificate keystore in Java or how to use keytool. I have Let’s Encrypt certs, but those only last for a few months. I don’t know if you can re-import a cert into a Java app while the app is running. Sometimes it seems like there is not a lot of overlap between security and development.

You’re welcome.

Image from an 11th-century manuscript housed at the Bamberg State Library (Wikipedia page here), manuscript description here, citation link here, image assumed allowed under Fair Use.

2018-02-05 Update

by

I know it’s a day late, but here is the weekly update.

I worked some more on the Groovy Email Server, even though I said I would not. Sometimes I hit a wall on something, say I am going to stop, but it just bugs me that I left things in a bad state. Then at some point I get back to it.

I was getting a lot of null strings, which was causing exceptions, and (after some changes I had to revert) sometimes infinite loops. It turns out I was not checking to see if the arguments to EHLO were proper domains. Apparently there are some bots that send bad arguments to some commands to try to take them over. At first I thought I was doing something wrong, but now I know it’s not me. I can exchange messages with Apache James, Gmail, Yahoo and Outlook.com, so if you are sending null strings you are doing something wrong. (I think Yahoo used to use QMail, but now I think they have moved to something else.)

I also have to work on delivery status notifications (RFC 3461). I got rejected by a server that told me I was not implementing the protocol correctly. I guess that SMTP extensions can really change things.

Not much Clojure. I have a call in a couple of weeks with the Purely Functional guy. He offered to come up with a lesson plan for people. I have noticed a few job posts in the Clojure mailing list and Brave Clojure requiring people to have paid Lisp/Clojure experience. One huge complaint in the Lisp world is there are not enough Lisp jobs. I want to email these companies and ask them where I can get the Lisp job that will enable me to apply for the Lisp job they are offering.

You’re welcome.

Image from Claricia Psalter, a late 12th/early 13th century manuscript housed at The Walters Art Museum, manuscript information here, image from World Document Library, image assumed allowed under Fair Use.

2018-01-28 Update

by

This week I did not do much Clojure. I did work on my Groovy mail server. I am starting to use it for my own mail, replacing Apache James. I will work on TLS next.

Update 2018-01-29_02.00.43:
I may put aside the Groovy mail server project, perhaps permanently. I thought I was on a roll, but now I am having issues sending. I cannot figure out what is wrong. I did make a change to my firewall, but I changed back and restarted the server. Still not able to send.

You’re welcome.

 

2018-01-21 Update

by

There is not too much on the Clojure front. There was a meetup at Cafe Express. I went to that, and I looked at some of Clojure’s special forms. I might work with them a bit and add them to my API example pages.

I am also working on the Groovy mail server. There are a lot of things that should be case-insensitive that were not, so I am working on that. I also need to beef up the docs, especially on setting up clamav. Hopefully I can start using it for myself pretty soon.

You’re welcome.

2017-07-16 Update

by

I took a couple of weeks off from work, so I did not get much done over the past two weeks.

At my company, “vacations” is now “PTO”: paid time off. I have noticed a few people I know at other companies who also use this terminology. When did “vacation” get replaced with a corporate term?

Anyway, I did go to the local Processing meetup. I did not stay the whole time. I noticed I did not have my wallet, so I left early to go look for it. It turns out I just left it at home.

I did a bit with Simply Scheme. I am now on chapter 6. I am wondering if/when I will get the “Lisp enlightenment” that will make me better in all things CS. It looks like there are things you can do in Scheme and Lisp that are just impossible in other languages. We shall see.

I did spend some time on the Groovy Email Server. I am keeping track of the ip addresses and host names of all incoming requests. I might be able to put this to good use someday.

You’re welcome.

Image from a 12th century manuscript housed at Bavarian State Library, webpage information here, image from World Document Library, image assumed allowed under Fair Use.

2017-05-21 Update

by

The big news in the JDK space this week is that Android now supports Kotlin. See this page, on the Android site and this post on Hacker News. So I will start learning Kotlin.

I don’t want to turn into one of those Scala weenies who goes on about how much they hate Java (see Note 1 below), but Android development was always pretty ghastly. Lots of XML, lots of anonymous inner classes (at least the last time I looked).

When Java started, it was basically: We took C++, and removed the hard parts you don’t like. Android is pretty much: We are using Java, but mostly the hard parts you don’t like.

I might look into React Native and ClojureScript at some point. But for now I am going through the Kotlin koans. Back when I was living in Chicago (back when Google told you to use Eclipse for Android), I spoke to a mobile developer who wrote apps for both Android and iDrone. He did not use any of the Javascript wrappers. He said the best way to do it was to just do it the way Google and Apple tell you to do it.

Maybe I will love Kotlin, but to be upfront I am not too happy with it at the moment. It is kind of a competitor to Groovy. I have spent a lot of time with Groovy, and I wish that it had more traction. But the world isn’t going to change for me. A lot of people at my job think every gosh darn thing has to be in a Word file. We make a web app for our client, why not make one for ourselves? But whatever.

There is no Kotlin group in Austin at the moment. Officially, there is a Groovy group, but there hasn’t been a meeting since January. The past two companies that hosted are no longer give the group the space. The group organizer works for OCI, the company that supports Grails. He works with clients all over Texas. I guess there is more Groovy out there than people think. Groovy is one of those languages where the user group is full of people who think the language is great, but their companies won’t even look at it. At the last meetup when he said he works with companies large and small all over Texas,  everyone got pretty excited. But he couldn’t give out any more info because of NDAs.

I know this sounds like I am not oozing with enthusiasm about leaving a language I like for one that I do not know very well. Frankly, at this point, I am not. But it looks like this is where the world is going. Kotlin had already gained a lot of momentum on Android before this (more than Groovy, certainly). If Google is willing to listen to its developers, then maybe Google is a vendor to pay attention to. Unlike Shiny-Wrapper-Microsoft (aka Apple).

Kotlin does have some similarities to Scala, which I do not like, so we will see how this all goes.

So, not a whole lot on the Clojure front.

I am still going through Simply Scheme.

You’re welcome.

Note 1: I always found Scala developers’ hatred of Java odd for two reasons: 1. The language you say you love was/is implemented in the language you say you hate, and 2. Why does Javascript get a complete pass from everybody?

Image from the Bamberg Apocalypse, a 11th-century manuscript housed at the Bamberg State Library (Wikipedia page here), manuscript description here, citation link here, image assumed allowed under Fair Use.

Thoughts On Native, GPU, Groovy, Java and Clojure

by

One interest of mine is math and scientific computing, although I admit right now my interest is far greater than my knowledge.

It seems like for a lot of math and scientific computing, you either need to do stuff in C, C++ or even Fortran, or use a library in a higher-level language that lets you interface with libraries in C/C++/Fortran. Another new trend is to use code that runs on the GPU, like OpenCL.

Lately I have been looking at a few Java libraries that interface with lower level code. I think it is a good idea to keep learning new things. I think AI and machine learning will become more important over time.

I contributed briefly to the SciRuby project a few years ago. One of their main subprojects is a Ruby wrapper around BLAS, ATLAS and LAPACK. I think some of the instructions for compiling BLAS, ATLAS and LAPACK might still have a few sentences written by yours truly. (It’s a very involved process.) Another one of their projects is a Ruby wrapper/interface around the GNU Scientific Library, or GSL.

One huge language in AI and data science these days is Python, with the NumPy and SciPy libraries. Again, some of the functionality is provided by linking to ATLAS and LAPACK. If it worked for them, maybe it will work for the JVM family of languages.

There is at least one project in Clojure that does this: Uncomplicate. It has sub-projects for OpenCL and BLAS and ATLAS. Along with Java and Groovy, Clojure is a language I am interested in. But for the time being I cannot use these projects. I think my graphics card is a bit too old and the drivers are too old for many of these libraries.

Uncomplicate is part of the reason I am looking at this area again. Somewhere on the site, the author points at the for development, it is okay to use pre-baked packages for ATLAS and LAPACK. One of these projects that I mention says for the best performance, you may want to hire someone to install ATLAS and LAPACK and other native libraries optimized for your production hardware. I did not know people made a living doing just that.

Another reason I am looking at this is GPU computing was mentioned in one of the side talks at the most recent ClojureConj. This is second- or third-hand, but apparently Goldman Sachs runs a nightly report analyzing the market (or some part of it, I am not too clear). It used to take nine hours to run. It was refactored to run on the GPU, and now it takes 20 minutes. I think all developers should become familiar with some of these technologies.

Will any of this gain traction in the Groovy space? I know things have been rough for Groovy for the past couple of years, especially after getting dropped by Pivotal. It seems like a lot of Groovy developers are happy making web apps with Grails. I like Grails, and it has worked out well for a lot of people. I will go through the Groovy Podcast backlog; perhaps Groovy developers are looking into this and I am just not aware of it.

Groovy developers can use the Java wrappers for native and GPU computing. No need to re-invent the wheel. Maybe going from Groovy to Java to native is not quite as fast as going from Python or Ruby to native, but I don’t think so. Even Ruby developers joke that Ruby is pretty slow. They say that sometimes the speed of the programmer is more important than the speed of the program. (But for some reason the Ruby devs who are moving to Elixir are now saying performance and concurrency are important after years of saying they were not. Whatevs.)

I do not have much experience in native or GPU programming. I am not too clear how NumPy and SciPy are used. I don’t want to swallow the Python ecosystem to get the answers to just a few questions, but: Are NumPy and SciPy programs (or bare-metal math/science apps in general) rewritten frequently? Or do they stay the same for years at a time? It seems to me if a GPU program does not change, you might be better off biting the bullet and doing things in C or C++.

But given the success of NumPy and SciPy, perhaps Groovy and Java can be used in this space. Maybe it’s not the end of the world if a bank’s nightly report takes 22 minutes. (How successful is SciPy? Not only are there conventions for general Python, there are also conferences for just SciPy). I doubt any technology will dethrone Python anytime soon, but it is an area to look at.

Having looked a small bit at some of these libraries, I do wonder how much can the unique features of Groovy be used. A lot of the tests deal with arrays of primitive types.  Someone on the Groovy mailing list wrote: I’ve found that you can really shrink down the lines of code just by using the basics that Groovy provides around collections, operator overloading, Groovy truthiness, etc.. Maybe that is enough reason to use Groovy for high performance computing (or at least higher performance). Can we make code calling native and GPU libraries better with features from Groovy, or will we just get Java without semicolons? Some of these functions do not want an ArrayList of objects, they want a fixed-length array of integers. Granted, you can make primitive arrays in Groovy:

def someArray = [[1, 2], [3, 4]] as int[][]

but I think that is more keystrokes than in Java.

Just looking at a small part of TensorFlow, here is a line from the original Java test:

final Tensor matrix = Tensor.create(new int[][] {{1, 2}, {3, 4}})

Here are two Groovy versions that both work (meaning the tests still pass):

final Tensor matrix = Tensor.create([[1, 2], [3, 4]] as int[][])
final Tensor matrix = Tensor.create([[1, 2], [3, 4]])

Granted, it’s one line, but I’d say that’s idiomatic Groovy. Perhaps this could be a (small) market opportunity for the Groovy community. Or maybe I’m just crazy.

If you google “groovy jni” the first hit is this page from Object Partners. It mentions JNA and BridJ which I think are used by a lot of the libraries mentioned below. Frankly, it sounds like a lot of work making this sort of thing happen.

Regardless, even if I have to stick with Java and Clojure, I will still keep an eye on these libraries and this space. I don’t think I can become an expert in all of these, but as I said, I think it is important for developers to keep an eye on this spae. I might start some pages about the different libraries, and perhaps share my thoughts on them. Below are a few notes on what I have found so far.

I started by googling “Java BLAS” or “Java GPU”, and at first I only got a few libraries. But as I kept looking, I kept finding more.

The first two I found where jblas and Aparapi. From the jblas website, jblas “is essentially a light-wight wrapper around the BLAS and LAPACK routines.” (And from what I can gather, the correct capitalization is “jblas”.)

Aparapi is a wrapper around GPU and OpenCL code. If you cannot get your GPU drivers installed properly, it will do everything on the CPU. Sometimes I get core dumps running the tests on my Ubuntu laptop. It is about six years old, and has 6 GB of memory, so perhaps that is the issue. But on my 8 GB Windows laptop two of the tests fail. I plan on getting a new laptop soon, so I will try this again when I do. I bet getting all this set up on an Apple laptop is really easy; for the first time in my life I am thinking about buying an Apple product.

TensorFlow has some Java APIs, although they warn that there may be API changes and breakages for languages other than Python and C. This can use either the CPU or the GPU, but for GPUs it can only work with NVidia cards.

The Lightweight Java Game Library works with OpenCL, OpenGL, Vulkan, EGL and a lot of other stuff I have never heard of. As the name suggests, it is primarily for game development, and the wrappers around the underlying native libraries and standards require more knowledge of those underlying native libraries and standards than most libraries.

There is ND4J (N-Dimensional Arrays for Java) and Deeplearning4j. I am not sure if they are related somehow, but they were both started by developers at a company called Skymind.

And then there is the granddaddy: ByteDeco. There is a lot of stuff here. Some of the contributors work on ND4J and Deeplearning4j, and also work at Skymind. This project provides Java interfaces to 21 different C/C++ libraries: video, math, science, AI, robotics, facial recognition, lots of good stuff here. And after looking at their list, yes, I think “Skymind” is a bit too close to “Skynet”. But at least they’re giving you a hint.

You’re welcome.

Image from an 11th-century manuscript housed at the Bamberg State Library (Wikipedia page here), manuscript description here, citation link here, image assumed allowed under Fair Use.

2016-04-21 Update

by

I added an SSL cert to this site with Let’s Encrypt. So now I am secure. The only issue is that for some reason the WordPress permalinks stopped working, so I had to go back to the default URL style. I spent about 20 minutes on the google, and all the sites gave the same suggestion that did not work. Perhaps it is time to look at Nginx.

I am also working on the Groovy Mail Server. I know I keep saying I will get to Clojure, and I promise I will. I got a few things working with SSL, so I decided to keep going just a little bit longer.

You’re welcome.

Image from the Carrow Psalter, a 13th century manuscript housed at The Walters Art Museum, manuscript information here, image from World Document Library, image assumed allowed under Fair Use.

Update To Groovy Validator

by

I made a change to the Groovy Validator project.

I changed the names of the annotations. For example, “IntAnnotation” is now “ValidInt”. The same change was made for the other types.

Here is the new README file:

This project has a few annotations that validate fields in POGOs, sort of like Grails constraints.I will attempt to make some annotations for properties in Groovy.Here is a POGO:

package info.shelfunit.properties.sample

class Book {

    int pages
    String title
    int year
}

It’s clean, and has no getters and setters. But what I do not like is there is no validation for your data. What if you want your String to be between 10 and 20 characters? What if you want your int field to be more than 100? And what’s to stop some dingo from trying to create a book object with less than 0 pages?

So I made some annotations that can do some validation for you.

package info.shelfunit.properties.sample

import validation.ValidInt
import validation.ValidString

class Book {

    @ValidInt( minValue = 30, maxValue = 400, throwEx = false )
    def pages
    @ValidString( minLength = 5, maxLength = 20, regEx = /^.*?[Gg]roovy.*$/  )
    String title
    int year
}

For POGOs, if a numeric field is declared as “def”, it will become null if the argument does not meet the validation constraints. If it is declared as a primitive, it will be set to 0 if the argument does not meet the validation constraints.

This project can also validate fields in immutable objects. In addition to using the annotations for the fields, you annotate the class with ImmutableValidator:

package info.shelfunit.properties.sample.immutable

import validation.ImmutableValidator
import validation.ValidInt
import validation.ValidLong
import validation.ValidString

@ImmutableValidator
class ImmutableObject002 {
    @ValidString( minLength = 5, maxLength = 10 )
    String firstString
    @ValidInt( minValue = 10, maxValue = 100 )
    int firstInt
    @ValidLong( maxValue = 100L, divisorSet = [ 5L, 7L ] )
    long firstLong
}

To process the annotations, put your properties in a Map, and add a boolean called “validation” and set it to true (since I couldn’t overload the Map constructor, I added a boolean):

def validatingImObject = new ImmutableObject002( 
    [ firstString: "Hi Again", firstInt: 11, firstLong: 22L ], true )

Adding the “throwEx” will throw an exception if the arguments do not meet the validation constraints. It is optional and is set to false by default. If an exception is thrown, it will print out the value and the constraints.

You might get a message like this:

"Hey" is a String with a length outside the range of 5 and 10 or does not match the regular expression ".*"

You can also use it with immutable objects annotated with the ImmutableValidator annotation. This would be a second boolean after the Map with your properties, since the first boolean controls validation:

def thirdImObject = new ImmutableObject002( 
[ firstString: "Hi Once Again", firstInt: 1234567, firstLong: 222L ], 
true, true )

In that case, you get a message with a line for each field. So you might get a message like this:

Groovy validation exception: 
"eeeeeeeeeee" is a String with a length outside the range of 5 to 10 characters or does not match the regular expression ".*" 
1234567 is an integer outside the range 10 to 100 or it is not divisible by anything in the set [1] 
222 is a long outside the range 0 to 100 or it is not divisible by anything in the set [5, 7]

If “throwException” is true for an immutable object and an exception is thrown, then the object will not be created.

This library can also handle final fields in mutable objects.

import groovy.transform.ToString
import validation.ValidInt
import validation.FinalFieldValidator

@ToString( includeNames = true )
@FinalFieldValidator
class Car {
    @ValidInt( minValue = 10, throwEx = false )
    int miles
    @ValidInt( minValue = 1990 )
    final int year
}

As with immutable validation, you need to use a map in the constructor to validate a final field.

def car = new Car( [ miles: 50, year: 2007 ], true, true )

Right now it only handles String, double, float, int and long. For String, it checks the string is checked against a minimum (“minLength”) and maximum (“maxLength”) length, and against a regular expression (“regEx”). For integers and longs, the field is checked against minimum (“minValue”) and maximum (“maxValue”) values, and a set of divisors (“divisorSet”). For double and float, the field is checked against minimum (“minValue”) and maximum (“maxValue”) values. There are defaults for all of these.

To use this project: Run

gradle distZip

and use build/libs/groovy-validator.jar in your project.

You’re welcome.

Image from “Psalterium Caroli Calvi ou Psalterium ad usum monasterii Sancti Dionysii, dit Psautier de Charles le Chauve”, a 9th century manuscript housed at the Bibliothèque nationale de France. Source gallica.bnf.fr / BnF; image assumed allowed under Fair Use.