Clojure

Here are the notes I took when Venkat Subramaniam was here for Austin JUG. I will mostly present them as I typed them.

Why Concurrency? Why do we care?

We now have mutli-core processors. Things that work well on one core may not work well on multiple cores.

The Java memory model controls how info goes back and forth from main memory and working memory.

His arguments in his classes are all final.

In the account class. Should the getBalance() be synchronized?

“synchronized” gets a lock and crosses the memory barrier, working and main memory.

Accessing a volatile variable, wait, join, a few other calls where you cross the memory barrier. We could compromise the invariant, like during a constructor.

So if you do not make getBalance synchronized, you might get an old value.

The code could have deadlock, or livelock (you wait, but nothing waits for you).

When you call synchronized, you might wait forever. There is no time out. So you use the Lock interface.

So for ReentrantLock you could call tryLock, or throw the Exception, then you have to unlock it.

You can order the accounts by ID to removed threat of dead lock.

95% of Java code is broken for concurrency. (Is he including code for Groovy, Scala, Clojure?)

Synchronize and suffer model.

Solutions: Software Transactional Memory (from Clojure) and Actors (Scala)

STM: We modify a variable (data) in our code. Shared mutability is bad. A few solutions: Avoid mutability. STM says let’s have a special type of variable: Managed mutable variable. It will only change in certain circumstances: a transaction.

(def balance (ref 0))

(println @balance) ; dereference the reference

Identity: Values are immutable. The stock price at 1:00 is different than at 1:10. But the 1:00 price will never change.

The identity is “google stock price”, points to a different value at different times.

We must protect the identity.

The problem in Java is that many times the problems happen without warning or error message. In STM, you get an error message.

(dosync(ref-set balance 100))

This changes it in a transaction. If there are conflicts, the transaction is aborted and retried. Up to 100,000 times.

Can we do it in Java? He used ref and dosync, so can we use them in Java?

clojure.lang.Ref. Look at his code, see what class he uses for Ref.

private Ref balance; // this was an int

The set method will use Lambda expression in Java 8. For now, we could do a Callable. LockingTransaction.runInTransaction

He is using an inline Callable.call()

What if you try to put more in the to account than is the from amount? It will throw an exception.

When to use STM? When you have multiple readers, multiple writers, and infrequent write collision. Not too many things are good for high-write collision.

You should not modify unmanaged variables in a transaction. Do not write to a file, print out messages, do not send email.

What about a long-running task

Another solution to shared mutability: isolated mutability.

Problem with encapsulation: Many threads can call the methods that change your private variables.

It is like voice mail: Many people can leave him a message, he gets a lot of messages, he goes through them one at a time. Actors have built-in message queue. Asynchronous messages.

ActorSystem.create();

actorSystem.actorOf(new Props(HollywoodActor.class));

pitt.tell("Wonka", null);

actorSystem.shutdown();

The Actor runs on a separate thread.

When do actors make sense?

final String PATH = "/usr/bin";

 private static int countFiles(final String path) {

    int count;
    File file = new File(path);
       if (file.listFiles() != null)
}

ActorSystem actorSystem = ActorSystem.create();
MasterCollection master = actorSystem.actorOf(Prop(MasterCollector.class))
master.tell()

The fields are immutable. The messages are not, so they should be immutable.

public void onReceive(final Object message) {}

You can have variables in an Actor, since only one thread will touch an Actor.

When does Actor make sense: Single threaded tasks sequentially

It punishes readers as much as writers. In STM, readers are never blocked. Writers are punished.

Purpose of Actor is not to deal with shared mutability, you want to prevent concurrent access, you want to serialize.

New concurrency API in Java 8.

You need Java 8 with Lambdas.

Look at java.util.streams.*

public static void printHighestPrice( Stream< String > symbols ) {

    symbols
        .map( StockUtil::getPrice ) // look that up
        .filter( stockInfo -> StockUtil.isPriceLessThat( 500 ) )
        .reduce( new StockInfo( "", 0.0 ), StockUtil::pickHigh )

}

http://download.java.net/lambda/b78/docs/api/java/util/stream/package-summary.html
:: is a method reference

map(StockUtil::getPrice)

can be done as

map(stock -> StockUtil.getPrice(stock))

Java is getting more functional

Synchronize and shared mutability can be avoided

JDK 5 concurrency is the assembly code of concurrency

Image from the Rheinau Psalter, a 13th century manuscript housed at Central Library of Zurich. Image from e-Codices. This image is assumed to be allowed under Fair Use.

 

I attended the No Fluff Just Stuff conference here in Chicago back in November 2011. I took  a few notes on some of the lectures. Here are my notes on some of the lectures on concurrency. I may post in a while with notes from more lectures, as well as some more thoughts on concurrency.

Venkat: concuurrency without pain in pure Java
Why Concurrency?
Java gave us a consistent API for multi-threading
Errors are worse on multi-core machines – multiple levels of cache in application
To get concurrency correct, you have to be God.
That model is the synchronize and suffer model
2 other models: Software Transaction Memory and Actors

public class Account {
    private int balance;
    public Account
}

Synchronized helps you cross the memory barrier synchronizing main and working memory
Locking and race condition is only part of the story
Do not synchronize the accounts, synchronize the transactions
You should also synchronize in the class using the Account classes

synchronized(from) {
    synchronized(to) {
        // do stuff
    }
}

There could be livelock or deadlock
But you could wait a long time.
But you cannot tell synchronized() to timeout.

So instead in Account, use a ReentrantLock
in Account.deposit method, use ReentrantLock.tryLock(long, TimeUnit)

This makes code more complex. synchronized blocks are pretty easy.
Don’t forget to call Reentrant.unlock()
But what if there is an exception before that? Put it in a finally.
This is why we don’t use raw JDBC

Deadlock: I have lock on A, I need a lock on B. You have a lock on B, you need a lock on A.
You could impose an order
If someone introduces a new method, all your locking could be ruined. It’s like a bathroom with a lot of doors. You got to lock every one.
Concurrency on JDK is like a mother-in-law: It’s waiting for you to fail.
95% of Java is broken for concurrency, and we don’t know it yet.

STM: Software Transaction Memory
Everything is immutable (almost everything)
Managed Mutable Identity is also in Clojure
Sharing is good, mutability is good, shared mutability is bad
Separates state from identity
identity points to state
Like stock price: $600 at 1:00 PM, $610 at 1:10
The price at 1PM on November 11, 2011 will always be $600
We can flip a switch to point identity to a new value
Immutability means no locks

the problem is not that it is failing, but that it is misbehaving quietly and you do not know it.
MMI fails loudly if you mess with it outside of a transaction
transaction thread runs in a cone of silence
final change is visible to outside world

If there is a collision, a transaction could repeat
Clojure is written in Java – it’s another JAR, and you can use it in Java

change int in Account to clojure.lang.Ref

private Ref balance
public Account(final int initialBalance) {
    balance = new Ref( initialBalance );
}

int will be replaced with java.lang.Integer
You can only put immutable stuff in there.
Also

public void deposit(final int amount) {
    LockingTransaction.runInTransaction( new Callable< Void >() {
        public Void call() throws Exception() {
        if (amount > 0) { balance.set( getBalance() + amount ) }
        return null;
        }
    });
}
public int getBalance()   { return (Integer) balance.deref(); }

Clojure:

(def balance (ref 0))
(println @balance)
(dosynch (ref-set balance 1) )
(println @balance)

You could still have race conditions. The AccountService that does the transfer still needs a transaction to wrap around the transactions in the Account classes
A transaction in STM could repeat 100,000 times – so do not print, send email

STM: we don’t see any locks (so no deadlocks)
2 drawbacks:
1. STM requires state be immutable – greate for Clojure – the only language where concurrency is safe
2. Code could ge slow or fail due to retries
Hard to be functional in Java
Another option: Isolated mutability – only one thread can touch it at a time
Actors and active objects
An actor can receive many messages at once, but they only work on one at a time
Akka library – now we use Scala’s implementation
Akka gives us STM and Actors

ActorRef ratcliff = Actors.actorOf(HollywoodActor.class).start();
ratcliff.sendOneWay( "Potter" );
ratcliff.stop();

HollywoodActor extends UntypedActor() {
    public void onReceive( final Object message )
}

So we passed a message. We did not call a method directly.
Actors do not hold Threads hostage
Actors could fail
Each Actor is sequential
Could deadlock with 2-way messaging

JDK concurrency APIs are assembly language of concurrency
—
But can Java web developers do this?

Ted Neward Guava:
Get rid of mutable state

Image from  Codex Amiatinus, a 6th century Vulgate manuscript housed at the Laurentian Library in Florence. Image from Wikimedia. This image is assumed to be allowed under Fair Use.