2017-11-06 Update: Deeplearning4J

One thing I did this weekend was get permalinks working again on my blog. I got an SSL certificate from Let’s Encrypt several months ago, and I was not able to get permalinks to work. It bothered me for the longest time. Problem solved.

Not much on the Clojure API front.

I did look at Deeplearning4J, a deep learning framework written in Java. I cloned their repo of examples, and I got some of them working with gradle files, and learned a bit more about Gradle in the process. You can use a GPU, or a CPU. No messing with CUDA if you don’t want to.

There is even a Clojure wrapper for Deeplearning4J as well.

I had a post about using CPUs and GPUs from JVM languages for math/science/AI a while back. Since then, I have gotten both Neanderthal and Deeplearning4J to work on my system. I did mention Bytedeco, which is used by Deeplearning4J and a few other libraries. I looked at some of the Javadoc, and they use a LOT of static methods (see here for an example). I may leave Bytedeco for others and use some of the higher-level libraries, like Deeplearning4J. I think a lot of static methods is not idiomatic Java. I might consult “Effective Java” later for a second opinion.

I am going to include below one of the build.gradle files that I used to get some of the Deeplearning4J examples to work.

You’re welcome.

apply plugin: 'java'
apply plugin: 'maven'
// apply plugin: 'groovy'
apply plugin: 'codenarc'
// apply plugin: 'application'

repositories {
  mavenCentral()
  jcenter()
}

// this has to be at the top so runJava can read mainClassProp
ext {
    dl4jVersion        = '0.9.1'
    nd4jVersion        = '0.9.1'
    jacksonVersion     = '2.6.6'
    logbackVersion     = '1.1.7'
    nd4jBackend        = 'nd4j-native-platform'
    scalaBinaryVersion = '2.11'
    dl4jSparkVersion   = '0.9.1_spark_1'
    awsSdkVersion      = '1.11.109'
    jcommonVersion     = '1.0.23'
    mainClassProp      = 'org.deeplearning4j.transferlearning.vgg16.dataHelpers.FeaturizedPreSave'
}

/*
'org.deeplearning4j.emr.EmrSparkExample'
'org.deeplearning4j.transferlearning.vgg16.FitFromFeaturized'
'org.deeplearning4j.transferlearning/vgg16.dataHelpers.FeaturizedPreSave'
'org.deeplearning4j.rnn.SparkLSTMCharacterExample'
'org.deeplearning4j.stats.TrainingStatsExample'
'org.deeplearning4j.mlp.MnistMLPExample'
'org.deeplearning4j.mlp.MnistMLPDistributedExample'
*/

allprojects {
    dependencies {
        // testCompile 'junit:junit:4.10'
        // testCompile 'org.mockito:mockito-all:[1.8.5,)'
        // From Maven, it's GroupId:ArtifactId:Version 
    
        // logging
        // Using sfl4j allows us to replace different loggers later, if desired
        compile group: 'org.slf4j', name:'slf4j-api', version: '1.7.2'
        // sfl4j is just a facade for loggers. To get logging to work, tie it in with an implementation, e.g., logback
        compile group: 'ch.qos.logback', name:'logback-classic', version: "${logbackVersion}"
        compile group: 'ch.qos.logback', name:'logback-core',    version: "${logbackVersion}"
        
        compile "org.deeplearning4j:deeplearning4j-zoo:${dl4jVersion}"
        compile "com.fasterxml.jackson.core:jackson-databind:${jacksonVersion}"
        compile "com.fasterxml.jackson.core:jackson-annotations:${jacksonVersion}"
        compile "com.fasterxml.jackson.module:jackson-module-scala_2.11:${jacksonVersion}"

        compile "org.nd4j:${nd4jBackend}:${nd4jVersion}"

        compile "org.deeplearning4j:dl4j-spark_${scalaBinaryVersion}:${dl4jSparkVersion}"

        compile "org.deeplearning4j:dl4j-spark-parameterserver_${scalaBinaryVersion}:${dl4jSparkVersion}"

        compile "com.beust:jcommander:${jcommonVersion}"

		compile "com.amazonaws:aws-java-sdk-emr:${awsSdkVersion}"
		// scope: provided
		
		compile "com.amazonaws:aws-java-sdk-s3:${awsSdkVersion}"
		// scope provided
        // compile "org.nd4j:${nd4jBackend}:${nd4jVersion}"
        // for CUDA
        // compile "org.nd4j:nd4j-cuda-7.5-platform:${nd4jVersion}"
    }
}

task runJava( type: JavaExec ) {
    println( "-- Groovy version: " + groovy.lang.GroovySystem.getVersion() )
    dependsOn classes
    description = 'Run gradle.sample.SampleApp'
    // Java main class to execute.
    if ( rootProject.hasProperty( "mainClass" ) ) {
        main = mainClass
    } else {
        main = mainClassProp
    }
    // We need to set the classpath.
    classpath sourceSets.main.runtimeClasspath
    // Extra options can be set.
    systemProperty 'sysProp', 'notUsed'
    println System.properties['iterations'] 
    // systemProperty 'iterations', System.getProperty( 'iterations' )
    systemProperty 'iterations', System.properties[ 'iterations' ] 
    // you can use this if nothing is set via command line
    // systemProperties = System.getProperties()
    jvmArgs '-server'
    // We can pass arguments to the main() method
    // of gradle.sample.SampleApp.
    if ( rootProject.hasProperty( "mainArgs" ) ) {     
        args = mainArgs.tokenize() // args wants a List, so split by spaces if need be
    } 
}

Image from the Trebizond Gospels, a 12th-century Byzantine manuscript housed at The Walters Art Museum, manuscript information here, image from World Document Library, image assumed allowed under Fair Use.

2 thoughts on “2017-11-06 Update: Deeplearning4J”

  1. Hi, I found this via google alerts. I figured I would comment here. It looks like you ignored our suggestion on the website to reach out via the live chat. We could have cleared a lot of your concerns up here.
    I will try my best to help you here but my suggestion would be to interact with us directly when you can next time.

    We maintain that whole stack including javacpp. Sam, the author is in our support channel as well.
    The reason there are static methods everywhere is because c++ allows for one off methods.
    It’s not a problem of the author’s. Javacpp is a c++ transpilation tool. It has to do *something* to approximate that.

    Beyond that in general, yes please stick to higher level libraries where poossible.
    If you have other questions, please reach out via the live chat. Thanks!

  2. I want to get my inspiration from really significant phrases said by truly terrific men and women like “Whatever is reasonable is true, and whatever is true is reasonable”, would you know the place I’m able to find thematic compiltaions of those?

Comments are closed.

There are 655 words in this article.