Wednesday, December 19, 2012

A dozen things to know about AWS Simple Workflow in Eclipse and Maven

Amazon AWS Simple Workflow

AWS Simple Workflow(SWF) from Amazon is a unique workflow solution comparing to traditional workflow products such as JBPM and OSWorkflow. SWF is extremely scalable and engineer friendly(in that flow is defined with Java code) while it comes with limitations and lots of gotchas.

Always use Flow Framework

The very first thing to know is, it's almost impossible to build a SWF application correctly without Flow Framework. Even though the low level SWF RESTful service API is public and available in SDK, for most workflow with parallelism, timer or notification, consider all possibilities of how each event can interlace with another, it's beyond manageable to write correct code with low-level API to cover all use cases. For this matter SWF is quite unique comparing to other thin-client AWS technologies.

The SWF flow framework heavily depends on AspectJ for various purposes. If you are not familiar with AspectJ in Eclipse and Maven, this article talks about challenges you may encounter while working with Flow Framework in Eclipse and Maven based on AWS Java SDK 1.3.26, and helps you understand what's going on.

Understand what needs to be done

Before following the official tutorial, it's important to understand what needs to happen and what development environment should achieve. Here's a list of steps, some of which is done by engineer and some is done by development environment which is either Eclipse or Maven. Engineer should be able to write and run fully functioning workflow with Eclipse only or Maven only.

  1. Write Workflow interface that defines interaction point of workflow, including how workflow is started, what signal we want to send to affect running workflow execution and the read-only call to get state of execution. Write activity interface. This step is documented here in official tutorial.
  2. Have Eclipse or Maven do APT code generation for activity and workflow interfaces. Note that client code generation has nothing to do with load-time vs. compile-time weaving. Client code is always generated at compile time of workflow and activity interfaces. AspectJ weaving applies to the workflow implementation and generated activity client code. For a FooWorkflow interface, if AspectJ is configured property, 10 classes should be generated at compile-time. They are
    1. FooWorkflowCliet
    2. FooWorkflowClientImpl
    3. FooWorkflowClientFactory
    4. FooWorkflowClientFactoryImpl
    5. FooWorkflowClientExternal
    6. FooWorkflowClientExternalImpl
    7. FooWorkflowClientExternalFactory
    8. FooWorkflowClientExternalFactoryImpl
    9. FooWorkflowSelfClient
    10. FooWorkflowSelfClientImpl 
  3. For a FooActivities interface, 2 classes should be generated by AspectJ. Please verify if Eclipse or Maven correctly creates these generated java source and compile them to classes directory. If not, later part of this article will help troubleshoot.
    1. FooActivitiesClient
    2. FooActivitiesClientImpl
  4. Write implementation of Workflow interface using generated client code. The generated client code is used in Workflow implementation to call activity, run child workflow or restart itself, etc. Without correct client code this step can not be done.
  5. Now it comes to AspectJ weaving for Asynchronous and ExponentialRetry annotations. This step is to make sure workflow implementation is executed correctly at runtime by injecting additional code logic around the calls to methods annotated with two annotations. Normally it is the workflow implementation class and activity client code,, that calls annotated methods, so both workflow implementation and activity client code need AspectJ weaving. For this step user can choose between load-time weaving and compile-time weaving.
    1. For load-time weaving, workflow implementation is compiled like normal classes, the aop.xml file must exist in classpath and define the patterns of classes to apply advices and JVM must be launched with "-javaagent:/path/to/a/aspectjweaver.jar".
    2. For compile-time weaving, workflow implementation is compiled with AspectJ compiler and numerous inner classes are generated within workflow implementation and activity client code, around each call to annotated methods. There's no special requirement to JVM at runtime so the aop.xml file is not required.
  6. It's very important to write unit test for every workflow implementation. I found almost everyone, me included, makes mistake in the first version without unit test. Without weaving unit test can't run property, and it doesn't always fail either. It'd be hard to troubleshoot so it's important to verify previous step is done correctly before running unit tests. Following the tutorial, I also learned that the SWF junit classes does not work for junit 4.10 or newer version, 4.7 is the latest working version I tried.
  7. Launch WorkflowWorker and ActivityWorker with spring or not.


Client code generation

These steps are required in Ecilpse to enable APT code generation:

  • Workflow and activity interfaces are annotated with @Workflow and @Activities.
  • Annotation processing is enabled in Project properties/Java Compiler/Annotation Processing.
  • AWS Java SDK jar and build-tool jar are in classpath, this usually means com.amazonaws:aws-java-sdk and com.amazonaws:aws-java-sdk-flow-build-tools is in pom.xml

If all three steps are done code generation should happen. Java code should be created to .apt_generated. If .apt_generated is empty verify the steps above.


Compile time weaving

Eclipse does compile-time weaving when

  • Project is converted into an AspectJ project
  • AWS Java SDK java is in AspectJ build path

One way to verify if Eclipse is configured right to interpret annotation is to look for the markers on left side of source editor for each call to method with @Asynchronous and @ExponentialRetry annotations.

Note that the marks only indicate Eclipse sees the annotation and knows to apply aspects, but doesn't necessarily mean it does right things for these aspects. Another way verify compile-time weaving is to find compiled inner classes created by AspectJ. There should be several inner classes named AjcClosure#. If you see the AjcClosure# for WorkflowImp and ActivitiesClientImpl classes, it's a pretty positive sign that compile-time weaving is happening properly.

$ find target/classes -type f | grep Ajc

Load time weaving

Load time weaving means classes are compiled with standard Java compiler, while JVM is launched with "-javaagent" argument pointing to an aspectjweaver jar file at runtime. Load-time weaving require aop.xml to exist in classpath.

Eclipse, Maven consistency

With M2E eclipse plugin, the compile destination directory in Eclipse is read from Maven so both Eclipse and Maven compiles classes into the same directory. Therefore with M2E it becomes very important to keep Eclipse and Maven configured consistently. If not, after Maven compiles code Eclipse often doesn't know the classes are generated differently. As a result the same code in Eclipse can behavior differently.


Client code generation

Two things to be aware of to enable client code generation in Maven
  1. Add build tool dependency. Official Maven repository doesn't have it anymore so you need to install it from AWS Java SDK to local repository manually
  2. Configure APT maven plugin


To verify, look for the 10 client classes in target/classes after a Maven build.
I had various troubles with Maven compiler plugin 3.0. Compiler plugin 2.5.1 or 2.5 works correctly.

Note that AspectJ maven plugin can do APT code generation as well so it seems simpler to have one AspectJ maven plugin do both APT code generation and compile-time weaving. However this does not work as the plugin can only do only one thing to a class. This means it can generate client code for workflow and activity interface, weave workflow implementation, but it can't continue to weave generated activity client code. As a result, the application mostly works except that @ExponentialRetry will be ignored. This problem has been discussed here.

Here is a fully working example of module that does APT code generation for workflow interface without AspectJ weaving.

Compile-time weaving

An example

To verify, look for the AjcClosure inner classes in target/classes after a Maven build. It's important to weave classes in both src/main/java and target/generated-sources/annotation directories because the APT generated activity client code also needs AspectJ weaving.

This module is an example of compile-time weaved workflow implementation as well as activity APT code generation.

Load-time weaving

Load-time weaving means there's nothing special for compilation but JVM needs to be launched with "-javaagent:" argument. In turn, Maven surefire plugin needs to be customized in order to run unit tests properly. In the POM file













Tuesday, November 20, 2012

Something is seriously wrong with moviegoers nowadays

Skyfall, Lincoln, Wreck-it, Flight and Argo are probably the best of the year in each of its genre. But Twilight!!! Again!!! What the hell is happening to this generation of movie audience?

US box office result on Nov 18, 2012

Wednesday, October 31, 2012

Ron Perlman, Judge Dredd

I enjoyed both Dredd movies. The 1995 Judge Dredd movie is one of those I could watch again and again no matter how bad review it gets. In the 2012 remake Karl Urban is as brutal as Stallone. Now the question is, when will Ron Perlman play Dredd? It seems Ron is equipped with the perfect chin for Dredd.

By the way not being an expert of image processing, I spent an hour and half replacing Karl Urban's face with Ron Perlman using Gimp in order to prove one thing ---- I AM NOT GOOD AT IT AND I HAVE A LOT OF TIME!!!

Sunday, October 14, 2012

JAXRS client on Android


Not a while ago I realized that for many people on this planet including my parents, smart-phone application is the only way they understand to interact with Internet. Therefore with zero client side software development experience, I started learning Android app programming.

Only after several days of learning I found myself blocked for there doesn't seem to be a clear answer for a problem that is so essential -- how to create JAXRS client on Android application. After days of trying a working solution finally presented to me. This document talks about the solution that I found as well as, more importantly, the solutions that do not work.

It's really important to known what we are looking for before start coding. I found many discussions on internet where people provide irrelevant because they have no idea what they are looking for. A solution must satisfy all following criteria:

  1. A Java library that calls HTTP RESTful service, preferably support HTTPS, wth/without client certificate authentication. Must support interception to allow injection of additional code logic around calls.
  2. Compatible to JAXRS(JSR311). Library must provides API that returns Java proxy of JAXRS annotated interface, via which method invocation is interpreted as HTTP RESTful calls
  3. Runs on Android. To be specific, the library and all its dependencies' must fall into the subset of JavaSE API that Android platform supports.

Apache CXF

In my experience in past most RESTful applications are built on Apache CXF framework, which satisfies #1 and #2 nicely and requires minimal coding and configuration. Unfortunately it fails for #3 which is not very surprising.

Jersey based solution

This document is at the top of Google search result from keywords "JAXRS" and "Android" at the time I write this document. Steps in detail seem to imply that the author did have it work on an Android device. However this Jersey 1.x based solution does not satisfy #2. Jersey is the reference implementation of JSR311 for server side implementation, while the client side API is low-level and could be tedious for RESTful interfaces with large number of methods.

Jersey proxy client

It's kind of unbelievable that Jersey does not provide high-level client API that satisfies #2 while many of its competitors(like CXF) do. With a little research I found that there is an ongoing and young client proxy subproject under Jersey based on JAXRS 2.0(JSR-339). It's available in official Maven repository. I went ahead and added it into my Android app, which requires to upgrade all Jersey dependencies to 2.0 and JAXRS API to 2.0, at which point it started violating #3 and failed to run on Android. Worth a try though.

Finally something is working, RestEasy-mobile

This discussion might be the second item on the top of Google search result. A lot of things are covered, of which almost all are waste of time. It's amazing how much people want to jump in before they even understand the question.

You may see that the starter asked the same question, a JAXRS client that runs on Android, which is commented with "have you tried X, a simple enough JAXRS client?", "here's how to access JAXRS service in Java" and "What's API level 9?". The only thing that turns out to work after hours of trying is JBoss resteasy-mobile, someone mentioned it in discussion but no one seemed to follow up. A lesson learnt from here is, if a library does not explicitly claim it runs on Android, it usually doesn't.


Before the first HTTP call went through via resteasy-mobile I ran into a number of issues, which weren't problem when I built non-Android RESTful services in past.

  1. JAXB annotations are unsupported and must be removed. Even when I didn't plan to rely on them, having a useless JAXB annotation in POJO caused Andoid app to fail. This also implies that as long as interface is shared between client and server, server can not use jaxb or jackson-jaxb-json provider. I ended up using jackson-json provider.
  2. @Consumes must be defined in API. It's not required by CXF client, but required by resteasy-mobile as far as I can tell.
  3. @Produces must be defined in API probably for similar reason. In CXF when @Consumes and @Produces are undefined, library usually can intelligently match out the right data binding provider, while obviously, resteasy-mobile can not. (further confirmation needed)


After all the research, JBoss resteasy-mobile is the only solution I found working for me so far. I hope this document helps you avoid wasting efforts on dead ends like I did.

Solution HTTP client library JAXRS interface proxy Runs on Android
Apache CXF Yes Yes No
Jersey Client 1.x Yes No Yes
Jersey Client Proxy 2.x Yes Yes No
JBoss resteasy-mobile Yes Yes Yes
RESTlet edition for Android Yes No Yes
Resty Yes No No

Friday, August 31, 2012

Dumped Comcast, finally

I used to pay a lot more to Comcast for package with 20Mb/s download speed.

Updated at Nov 19, 2012

Monday, July 30, 2012

This is why you never take CNN seriously

I couldn't believe my eyes when I first time read it. What is the right word to describe it? Hilarious, ridiculous, or imbecilic. Then I realized, hey, this is CNN.

Monday, May 14, 2012

How does boat run on street?

I was walking with my wife and Yilin, my two-and-half-year-old daughter who is super excited about talking random things in a language mixed with English, Mandarin, baby talk and occasionally Spanish.

Near Space Needle on Broad Street, we saw one of those "Ride and Ducks" amphibious touring vehicles stopping at a traffic light, so I asked Yilin, "Look, how does that boat run on street?"

Yilin looked at the vehicle, confused, so she asked my wife, "Mama, how does that boat run on street?"

"Because that boat gets wheels on it, Yilin", my wife explained.

Yilin turned her face back to me and said, "Baba, because that boat gets wheels on it".

Friday, April 27, 2012

Happily ever after

During dinner I was reading a note from the daycare of Yilin, my two and half year old daughter:
We read a story about a farm which ended happily ever after and Yilin said "Cinderella was happily ever after!!" Have a great weekend!
It made my day.