2017-11-06 Update: Deeplearning4J

One thing I did this weekend was get permalinks working again on my blog. I got an SSL certificate from Let’s Encrypt several months ago, and I was not able to get permalinks to work. It bothered me for the longest time. Problem solved.

Not much on the Clojure API front.

I did look at Deeplearning4J, a deep learning framework written in Java. I cloned their repo of examples, and I got some of them working with gradle files, and learned a bit more about Gradle in the process. You can use a GPU, or a CPU. No messing with CUDA if you don’t want to.

There is even a Clojure wrapper for Deeplearning4J as well.

I had a post about using CPUs and GPUs from JVM languages for math/science/AI a while back. Since then, I have gotten both Neanderthal and Deeplearning4J to work on my system. I did mention Bytedeco, which is used by Deeplearning4J and a few other libraries. I looked at some of the Javadoc, and they use a LOT of static methods (see here for an example). I may leave Bytedeco for others and use some of the higher-level libraries, like Deeplearning4J. I think a lot of static methods is not idiomatic Java. I might consult “Effective Java” later for a second opinion.

I am going to include below one of the build.gradle files that I used to get some of the Deeplearning4J examples to work.

You’re welcome.

apply plugin: 'java'
apply plugin: 'maven'
// apply plugin: 'groovy'
apply plugin: 'codenarc'
// apply plugin: 'application'

repositories {
  mavenCentral()
  jcenter()
}

// this has to be at the top so runJava can read mainClassProp
ext {
    dl4jVersion        = '0.9.1'
    nd4jVersion        = '0.9.1'
    jacksonVersion     = '2.6.6'
    logbackVersion     = '1.1.7'
    nd4jBackend        = 'nd4j-native-platform'
    scalaBinaryVersion = '2.11'
    dl4jSparkVersion   = '0.9.1_spark_1'
    awsSdkVersion      = '1.11.109'
    jcommonVersion     = '1.0.23'
    mainClassProp      = 'org.deeplearning4j.transferlearning.vgg16.dataHelpers.FeaturizedPreSave'
}

/*
'org.deeplearning4j.emr.EmrSparkExample'
'org.deeplearning4j.transferlearning.vgg16.FitFromFeaturized'
'org.deeplearning4j.transferlearning/vgg16.dataHelpers.FeaturizedPreSave'
'org.deeplearning4j.rnn.SparkLSTMCharacterExample'
'org.deeplearning4j.stats.TrainingStatsExample'
'org.deeplearning4j.mlp.MnistMLPExample'
'org.deeplearning4j.mlp.MnistMLPDistributedExample'
*/

allprojects {
    dependencies {
        // testCompile 'junit:junit:4.10'
        // testCompile 'org.mockito:mockito-all:[1.8.5,)'
        // From Maven, it's GroupId:ArtifactId:Version 
    
        // logging
        // Using sfl4j allows us to replace different loggers later, if desired
        compile group: 'org.slf4j', name:'slf4j-api', version: '1.7.2'
        // sfl4j is just a facade for loggers. To get logging to work, tie it in with an implementation, e.g., logback
        compile group: 'ch.qos.logback', name:'logback-classic', version: "${logbackVersion}"
        compile group: 'ch.qos.logback', name:'logback-core',    version: "${logbackVersion}"
        
        compile "org.deeplearning4j:deeplearning4j-zoo:${dl4jVersion}"
        compile "com.fasterxml.jackson.core:jackson-databind:${jacksonVersion}"
        compile "com.fasterxml.jackson.core:jackson-annotations:${jacksonVersion}"
        compile "com.fasterxml.jackson.module:jackson-module-scala_2.11:${jacksonVersion}"

        compile "org.nd4j:${nd4jBackend}:${nd4jVersion}"

        compile "org.deeplearning4j:dl4j-spark_${scalaBinaryVersion}:${dl4jSparkVersion}"

        compile "org.deeplearning4j:dl4j-spark-parameterserver_${scalaBinaryVersion}:${dl4jSparkVersion}"

        compile "com.beust:jcommander:${jcommonVersion}"

		compile "com.amazonaws:aws-java-sdk-emr:${awsSdkVersion}"
		// scope: provided
		
		compile "com.amazonaws:aws-java-sdk-s3:${awsSdkVersion}"
		// scope provided
        // compile "org.nd4j:${nd4jBackend}:${nd4jVersion}"
        // for CUDA
        // compile "org.nd4j:nd4j-cuda-7.5-platform:${nd4jVersion}"
    }
}

task runJava( type: JavaExec ) {
    println( "-- Groovy version: " + groovy.lang.GroovySystem.getVersion() )
    dependsOn classes
    description = 'Run gradle.sample.SampleApp'
    // Java main class to execute.
    if ( rootProject.hasProperty( "mainClass" ) ) {
        main = mainClass
    } else {
        main = mainClassProp
    }
    // We need to set the classpath.
    classpath sourceSets.main.runtimeClasspath
    // Extra options can be set.
    systemProperty 'sysProp', 'notUsed'
    println System.properties['iterations'] 
    // systemProperty 'iterations', System.getProperty( 'iterations' )
    systemProperty 'iterations', System.properties[ 'iterations' ] 
    // you can use this if nothing is set via command line
    // systemProperties = System.getProperties()
    jvmArgs '-server'
    // We can pass arguments to the main() method
    // of gradle.sample.SampleApp.
    if ( rootProject.hasProperty( "mainArgs" ) ) {     
        args = mainArgs.tokenize() // args wants a List, so split by spaces if need be
    } 
}

Image from the Trebizond Gospels, a 12th-century Byzantine manuscript housed at The Walters Art Museum, manuscript information here, image from World Document Library, image assumed allowed under Fair Use.

1 thought on “2017-11-06 Update: Deeplearning4J”

  1. I want to get my inspiration from really significant phrases said by truly terrific men and women like “Whatever is reasonable is true, and whatever is true is reasonable”, would you know the place I’m able to find thematic compiltaions of those?

Comments are closed.

There are 674 words in this article.