Saturday, December 15, 2007

Friday - Practical Concurrency - Rob Harrop

This was the best lecture I've ever been to. Ever! Ok, enough of my opinion and onto Rob.

Rob said a lot of his enterprise customers have a lot of really smart guys that make basic errors when it comes to concurrency.

He said that he was not going to discuss Java 1.3 and 1.4 because "there was no memory model basically".

He said "Don't just use synchronized and volatile!" Testing is hard. Some people try to use sleep() in tests and this kind of works sometimes and kind of doesn't other times.

What motivates concurrency? Multi-core architecture changes EVERYTHING. They recently bought an 8 core machine for about 3000 pounds (note: sorry I don't have the pound symbol handy, but I mean money not lbs).

Parallel (processing) - memory model is more important. "It's the memory model." JMM = Java Memory Model.

In Java writes to long and double are not atomic but everything else is.

Synchronized - made a point that it doesn't lock object (he made this point a few times- it is really scary that some people think that it does!).

Volatile- inefficient but guards double/long (and more as you get into Rob's later stuff)

Final- try to use final field when you can

Setter injection is NOT great for concurrent applications. (I assume this means use constructor injection in those cases?- didn't note that though. No reason given for this that I noted)

happens-before - 3 main ones are

* Program order rule
* Monitor lock
* Volatile variable rule

Other 2 he mentioned, but said they weren't important.

He said learn these 3 and then don't worry about it anymore (just be aware of them). Use higher level tools to handle it. Rob barely uses these 3 (or thinks of them) because of the JVM tools in Java 5.

Rob said "it was said (last night/Wed by Rob I think when he was talking about Spring Security) every time someone uses ACEGI, a fairy dies". "Well," he said, "everytime someone uses JMM, 10 fairies die." (Got a laugh.)

Shared state has to be volatile.

Safe publication
* Final - they don't guarantee state - so need to make all final
* Volatile - ...
* ...

Never use *any* kind of shared state (even for primitive) without volatile (strangely not all examples had volatile, but all had either final or volatile).

JMM makes NO guarantees in larger system. It may just cache a value of a member and not flush it to main memory (which is shared across cores/etc.) Default value of a field is guaranteed but that's it! (WOW! that was huge and better than the fairy statement even)

Using HashMap or non-sync'd collection and sync'ing on your own is a "recipe for disaster", he said. Can't just sync on add. Have to sync on get.

*Can't* do final read lock and final write lock and sync on write for add and sync on read for get. You need to sync on the write lock (or the map instance itself someone argued and Ron agreed.)

Through reflection can change *any* state. (Made the classic example of changing strings in string pool, but also mentioned something about the string pool itself not getting used necessarily across large system (I think), and about changing private members which was news to some in the audience.

Common Bug Patterns

Made point again you are NOT locking the object when you synchronize on an object (why would anyone think that?).

Rob introduced Atomics. They are fast and have their own memory model. Can work with an AtomicInteger against any number of threads.

Semiphores - he said you won't use these much but are good to have in the swiss army knife.

Latches -

an aside from Rob: Java states that Thread Group # is just an estimate. You can't use it because for example Log4J will spawn a thread.

Locks and conditions - in Java 6 is better

JVM can do things with Hashtable like sometimes determining it is only being used by one thread and unlock it! (but later he makes point of using ConcurrentHashMap as general fix-all for most cases)

final AtomicInteger

*have* to make it final or won't be seen by all threads. (And he said volatile even though wasn't in example. why?)

use .getAndSet() and .compareAndSet() methods on Atomics (I guess that makes it visible across system).

Latch - Use CountDownLatch (reminder to self- NOT ThreadGroup). CountDownLatch is one of the most important tools in *any* situation needing a latch.

eg.SimpleCache.java example. Same thread can acquire same lock. ReentrantReadWriteLock contains two locks internally. Really good solution if reads heavily outnumber writes. *DO NOT* forget finally block. Can only write if no one has write lock. Anyone can read so long as no one is writing.

Hashtable is locked on both so is not the same.

Made point previous example is about having heavy reads.

eg.LogWriter.java is example of flushing after some amount of time (every 6 seconds).

Concurrent Collections

Java 5 and 6 are excellent. Get it right easily using this.

* ConcurrentHashMap - best overall
- by default is 16 (buffer size?)
- allows concurrent write. "really interesting" he said (a good way)
* CopyOnWrite (List|Set) - not quite as used. Very inefficient if not heavy on writes.
* ConcurrentSkipList (Map|Set) - not useful. Have yet to find a situation where ConcurrentHashMap is not better.

These are an improvement over synchronized collections (like Hashtable I assume he means).

Efficient apps don't lock on everything. :)

Must assume some level of uncertainty.

eg.execution.Basic.java

Executor

eg.execution.CustomizedExecution.java

Always try to use bounded queue Should be able in most cases to use throttled approach. If you can't have to either abort or log (Rob stumbled a moment here and then backtracked a little to basically say to use the throttled approach, as he said he couldn't really think of an example where a throttled approach wouldn't work.)

see CompletionRunner.java - he said was very useful. Only shows stuff that came back with future timestamp (?).

* Use ConcurrentHashMap - the swiss army knife. CopyOnWrite for some stuff.

DON'T rely on StringPool String hash or your own solution.

You should NEVER expose your locks to the outside world. It is a recipe for disaster. (deadlock)

Don't call methods outside of your control whilst holding a lock.

Testing-

MultithreadedTC - written by Bill Pew - Rob said Bill was a multithreaded genius. (or something to that effect with "genius" in it)
www.cs.umd.edu/projects ...

Example in concurrent-samples/src/... eg.Runner.java (nifty)

Java 7

Java Fork/Join Framework by Doug Lea (sp)

(Rob - he talks fast!)

Rob using Java 6 preview on Mac.

Friday - Useful Spring Extensions - Costin Leau and Craig Wall

Costin spoke for 1/2 and Craig for 1/2.

Costin's part:

Spring Modules was started by Rob Harrop in 2005. Most code falls under Spring license (one noted exception was db4o which is GNU or commercial license depending on use). It is a community project (even though it is now led by SpringSource) that exists because everything can't belong to Spring Framework (due to nightmare it would cause for versioning and other logistics).

There are currently over 50 modules not counting sandbox.

Cache is the most popular module.

Other ones of note:
JSR-94 - aka Rules - there was a whole module (lecture) on this last year
aka JSR-170 - Java Content Repository also one of note
DataMap provides map view of DB
Ant (Costin seems to like Ant I assume?) makes Spring modules available to Ant. Supposedly there is one for Maven also (Maven 2?) - I need to look into that when I get back. That looked pretty neat.
Cache - just an abstraction to use 1 of 5 or so cache providers. The cache id is method name + args. Clustered cache available via OpenSymphony (but he said it was buggy), Coherence, EhCache (EzCache).

Note in the cache module configuration slide there is an error that someone pointed out - the method should not be returning null (there would be nothing to cache).

Workflow (he said "you'll probably love")- integration w/Flux, etc. (2 others). Allows access to Spring beans from workflow. No other workflow but Spring workflow supports:
* having either single or multiple workflows going on and can still have context w/in or across sessions
* wiring or IOC

TypeResolver
* no methods have to be defined in spring definition. Does it automatically(?).

JCR - repository - is a data API for accessing CMS (remote storage, file system, DB)
(versus JPA - which is for accessing OEM(?? I know I must have written that down wrong)

Most people using OpenSessionInView

Transaction Mgmt integrates w/Jackrabbit. Behind Hibernate. All is easy.

Transaction aware repository similar to Spring. Transaction aware Session repository(?)

Craig Walls took over:

XT - was originally called "Spring to the eXTreme" by Sergio Bossa.

XT Modeling and XT Ajax (latter is similar to Rails Ajax- maybe)

Craig had some problems in setup of his "Expectations" project. Expectations is Craig's project that has unit tests that help verify his understanding of all of the projects he wants to understand.

(Craig uses Maven 2 as do a number of other developers who were building code, and they seem to mostly/all use the src/main/java/... structure.)

XT Modeling can generate classes from interfaces. Craig mentioned that he wasn't really sure why/how useful that would be.

XT Ajax - minimal javascript required to write. It seems to use Scriptaculous. (Note: from others I've heard Scriptaculous changes the behavior of Javascript itself, which a number of people find "yucky".)

With one line you can add XT to a webpage, fire event in another, handle event in another. So basically 3 lines of javascript to do a basic Ajax page. If you don't mind Scriptaculous, that sounds pretty good to me.

The class he wrote as the controller extends AbstractAjaxHandler. ... Example is in his XT-Ajax svn project:
svn://svn.geekisp.com/habuma/trunk/expectations

db4o - OO DB also available for .NET. GNU and commercial licensed depending on use. Pure OO DB. Storing data is simple. When using Spring it hides the try/catch/finally. Use @Transactional or tx:advice for transaction. Retrieving data is a little harder.

Craig again mentioned that Spring was good for making SQL exceptions a little easier to do something with because of all of the Spring database exception class types.

(I think that Craig finds db4o interesting, but not many in the audience seemed to. I'm guessing that is because most projects would use MySQL, Oracle, PostgreSQL, or SQL server, and HSQL for small stuff/in memory stuff).

(Craig uses TextMate as do others in the lectures for text editing. Pretty much everyone was using a MacBook Pro and either Eclipse (~75%) or IntelliJ IDea 7 (~25%). Shouldn't be a surprise, but just in case you were curious!)

Spring Validation

Can use Commons Validator - struts uses Commons Validator also. Factory reads rules as XML. Email filter (validator) comes with it. Phone doesn't (not terribly surprising since there are so many different ways of writing phone#s across the world- seems like we all need a standard! :) ).

Valang - note that in this slide the first message is the default message. Valang supports i18n. (Note: I would have thought it would use the resource bundle's default properties (the one without locale or anything on end). I'm sure they do this for ease of use, but it is a little non-standard for i18n.)

(Unfortunately Craig lost the validation examples because they didn't get transferred from his work computer like he thought.)

There are also XML based validator and Annotations based validator.

"expression" lets you do conditionals. In example it is used for non-null check (but there is already a check for that I think).

Craig likes the in-future date check.

Craig was asked "What do you prefer XT or DWR?"

Craig said DWR is really remoting and have to do a little more javascript. So it seemed like he likes XT better.

Thursday, December 13, 2007

Thursday - Rod Johnson - State of the Art in Dependency Injection

Went over history of Spring, etc.

Spring took the Constructor injection from PicoContainer even though you could tell Rod doesn't like constructor injection. It's funny, because later in the lecture, Rod noted that EJB 3 didn't have constructor injection. One thing that is pretty bad about EJB 3 though is that he said in Vegas someone asked the EJB guys why it was that they didn't include constructor injection and they said because it was hard to test... I agree that is one of the stupidest things I've heard. But what do I know? He said outjection (in Seam) was not necessary because Spring does similar automatically (it isn't stateless as Seam claims). You could tell Rod didn't like Seam. I could tell that Rod respected Guice. They got some good ideas from it.

Rod said XML is the most powerful configuration option for Spring.

Spring Java Config (which he spent most of the lecture talking about) is close (or maybe equal).

Ops/Production folks would prefer XML because don't have to recompile.

Said context:component-scan is highly efficient and great for development. Not good for prod though (config tied to annotations).

@Resource - Rod wouldn't use it - should use @Autowired.

Quote from Rod (as he started into a long deal on Spring Java Config- this may not be word for word but is close) "Even though I'm known as the 'high priest of XML' for the past year or so I've been working on annotations projects."

So the second 1/2 of the lecture was all Spring Java Config? Ugh. Main advantages- can use constants, code level validation, refactoring/inheritance. Disadvantages: tied to code, still need to have user/pass etc. in diff file, should have XML for production.

Note - slide "Bootstrapping Java Configuration" is incorrect and outdated. Changed in M3 and has error in package name?

Best practice:
* code and config that is mostly static - use Java Config + Annotations
* dynamic code/production - use XML
* JDBC user/pass - use props

XML always overrides code config.

Thursday - Rob Harrop and Ben Hale - Data Access and I/O Strategies for Batch Processing

More sound issues. Ugh. About 5 or so minutes trying to get sound working.

Rob did most of it. Ben did about 5 min toward the end.

Rob was a huge fan of IntelliJ IDea he said, but is now an Eclipse user because of SpringSource App Suite and Mylen etc. "makes it bearable".

Main points:
* The time waster is NOT drive/actual file IO in hardware, it is what you do in the software after reading the data (or while).
* NIO is waste of time
* Collections are evil (use Iterator and read one line at a time into an object)
* DOM is evil (use XPP with StreamingWriter) - mostly for memory not time
* Use Buffered classes!
* Establish a data model class (data class). It does not need to be same as domain model.

It's all about memory, and specifically about avoiding use of T.. space and especially avoiding Survivor space. Showed lots of examples with JConsole (2nd bar in heap is bad, 3rd bar (survivor) is awful - is fine for first bar to be high).

DB:
* Autocommit is slow (as would be expected)
* Single txn equivalent time to doing chunk
* Batch chunk is fastest next to DB specific optimizations
* Latter is fastest

template.batchUpdate(...)

JDBC batching is fastest. Read chunk = pre-fetch. Larger chunk is NOT better (because of GC) - find the sweet spot. Use ORM or not? If you do prefetch and tune it then ORM is good

Partitioning is good. Parallelization only good depending on item interdependency. If one thing has to be put into DB before another, etc. etc. then parallelization can be bad. He spent 3 hours the other day with Ben trying to figure out a parallelization problem was was fresh in his mind.

By far Rob's and Craig's lectures were my favorite so far. Rob is super smart and great to listen to.

You can tell Ben is smart also (he does the batch project) but he unfortunately sounded a bit to sales-sy today talking about it.

Thursday - Spring Framework 2.5 - Jurgen Holler

What Jurgen said wasn't a lot different than the slides.

Things I thought were strange is that it supports RAR deployment (for non-webapps). I'm curious as to why RAR, when everything else in Java is zip based (jar, war). I'm guessing RAR is better?

It was really hard to understand Jergen because of the reverb in the room and usually only when he spoke quickly. I know that I would be able to understand him without the mic and big room though. Could partially just be the tone of his voice? I know enough about acoustics to power a small ant hill.

JMS and JCA-
or if you'd rather use JCA. Otherwise are the same inside the element.

The short of what he said: basically in Spring 2.5 everything works with JEE 5.

Jurgen recommends only annotating one contructor with @Autowired vs. multiple, even though it should work.

He also recommends to Autowire methods or contructor and not fields, because doing fields is less easily testable (requires reflection).

Something totally cool was the qualifier type="Emea" in spring config and using stuff like:
@Autowired
public void SomeConstructor(@Emea OrderService service)

Talked about component-scan for @Component/@Repository. Also neat.

Quote "in good old Spring style" used often.

He said component-scan works for some cases but context:annotation-config (lets you use JSR-250 annotations, etc.) = more mainstream.

He said @RequestMapping
public void handleSomeAction(@RequestParam("myParam") String p, ModelMap model)
is VERY testable. Seems like he recommends that.

The annotated portlet controller stuff was cool also.

Talked about JUnit (no longer have to extend any class for test etc.). bean(name). Load time weaving.

Mentioned that Load Time Weaving works with built-in for Tomcat(?) but need to use diff classes for Glassfish, Weblogic, ... and some applications servers not supported (standard JEE (??) and Websphere don't support Load Time Weaving if I understood him correctly).

Thursday - Spring in Action - Craig Walls

There was a late addition to the slides, namely the subversion URL for example code:
svn://svn.geekisp.com/habuma/trunk/Presentations/examples

Craig did a survey of people in the room and whether they were using Java/.Net/etc. and what version of Java and Spring they were using. Majority were Java 5 and Spring 2.

He described Spring as like an aquarium for POJOs (plain old java objects as Rod said later in the day was a term originating from Martin Fowler).

Craig also said that Spring had been a catalyst for change (creation of OSGi, and EJB 3 was lighterweight because of pressure of Spring).

Craig seemed excited about Spring Web Services - basically web services that are independent of SOAP, etc. unlike Axis (and I am a big fan of Axis, but I'm going to look into Spring Web Services now). Craig also seemed excited about OSGi.

Craig said that he "didn't know a lot about Spring Batch" and felt fortunate not to have been involved with it (he said jokingly).

Craig is an Eclipse user and is a big fan of Spring IDE for Eclipse, but he also likes IntelliJ IDea.

He said that he didn't agree with the quote last night about his lecture being the whole "Spring in Action" book in 90 minutes. He said he'd only hit 4 major points and wouldn't go into the advanced stuff. I am a huge fan of Craig and the book, so I was a little disappointed it was only going to be the basics, but I ended up liking it a lot despite that.

Craig went over Spring 2.5 and I thought the new stuff with Annotations that almost allow you to completely get away with not having any XML is neat (@Component, etc.). However read on...

He was asked about whether he thought using XML or annotations were best practice. Craig said that he preferred XML over annotations, because with annotations, Spring is tied to the code via the imports in the code. Craig was definitely all about making the code itself small (not taking XML into account) and only loosely coupled. Craig made a point that you can't have completely decoupled code, as that is useless (things have to relate to each other- if they act totally independently then it isn't really useful).

Craig mentioned the term "DMSU" (don't make stuff up).

He said that Spring MVC is good because it hides having to setup Application Context path, etc.

Craig was asked about lazy loading... He mentioned that the way it works in Spring is not totally that useful because if you have a bean "Quest" with lazy-init="true" and another bean "Knight" that refers to that bean, then when the Knight is instantiated (during init I assume) the Quest is instantiated. Craig said that 2 mos ago he pondered this issue for the first time even though he noted others had pondered it previously. And what he/they came up with to get around it is that you can use proxies that lazy load when accessed to get around this. Sounds like an excessive amount of work to me. I hope that this can be fixed in later versions of Spring (but I'm sure I don't fully understand the complexities involved with this).

Craig talked about AOP and the minstrel and knight example.

He said "around" advice is not good. Just use before and after instead. The reason is that "around" is buggy.

He said dynamic pointcuts are slower and you hardly ever need to use them.

Craig said that AspectJ is like Spring annotations. However even though he and other folks at the conference seem to look favorably towards AspectJ, he thought it was kind of wierd that the @Pointcut needs to define some random method to be used. I agree - seems like that would have made more sense to have been done as a class level annotation (with argument) instead (is that a stupid idea?).

He also mentioned that in his slides his AspectJ example of @After didn't say that @After runs regardless of whether the method succeeded unlike after-return (which only runs if method is successful).

He said you could also have multiple pointcuts @Before("foo() && bar()").

He said with AspectJ you could put pointcuts in anything (before/after stuff you didn't even write that doesn't use AspectJ). However he said Spring AOP can only do methods using Spring (because it uses Java something-or-other proxy 1.3).

Also per question, stated @Pointcut("execution(* *.embarkOnQuest")) will pointcut ALL classes with embarkonQuest method, but if you only want to do Knight (class) then you do @Pointcut("execution(* Knight.embarkOnQuest")). He said @Configurable enables AOP with objects not managed by Spring.

Spring JDBC advantages: you get many more exceptions than just SQLException (so easier to figure out what is wrong) and some are not checked exceptions (to make it easier).

New in Spring 2 - via use of namespaces it is MUCH less XML code. Dynamic language beans like JRuby, Groovy, Beanshell and through a patch can do Javascript.

Spring 2.5 supports JUnit 4 (4.4 as said later).

Wednesday Spring Experience Party

Free Rod Johnson bobblehead and Sam Adams.