Skip to main content

Datamung won Netflix Cloud Prize award for best datastore integration


On Nov 14th, in AWS re:invent conference day 2 keynote, Netflix announced 2013 Cloud Prize award winners. I was very lucky and my project, datamung, is one of the ten winners. Datamung received the award for best datastore integration.


Datamung is a Java open source web application that backs up RDS MySQL database into S3 using AWS Simple Workflow and EC2. It's a RESTful service and website on top of it, a single installation allows multiple AWS users to backup their database living in their own AWS accounts. The fact that it uses mysqldump command indicates that the backup result in S3 is a SQL file ready to use across AWS accounts, regions, VPC, or outside of AWS network. If you are interested please jump into the Datamung project wiki to find out more.



Ironically, in the same keynote session where winner list was announced, 40 minutes later Werner Vogels announced a cross-region RDS replication feature which targets the same problem that Datamung attempts to solve.


Netflix flew my family to Las Vegas to attend 2013 AWS re:invent conference and accept award. I was horned to meet the judges, some of which are basically my career changers. For example Martin Fowler, when I talk to people about things he wrote, I got job offers so I could pay mortgage and send daughter to school. I wouldn't restart on open source development after 3 quiet years without Adrian Cockcroft launching the Cloud Prize program, not mentioning Netflix system had me watch hundreds of movies via internet. Anyway, I had good time in sin city and so did my wife and daughter.



Lastly, Chinese food in Vegas is surprisingly good as it turns out. I even found an authentic, in fact the only authentic Xi'an restaurant after 10 years of searching in United States. Having grown up in Xi'an myself, I was pleased to taste childhood food, and suffer from stomach pain of overeating once again.


Labels:

Comments

Post a Comment

Popular posts from this blog

Publish Maven site with Amazon S3 and CloudFront

Amazon S3 now supports static website hosting . As a 10 years Maven user, I wonder how easy it is to deploy Maven generated site to Amazon S3 and let the rock-solid storage provider to host my project websites. There are several existing s3 wagon providers , which all seem to have the same problem, not supporting directory copy. This is understandable since before S3 new website hosting feature, I guess people mostly expect to deploy artifacts rather than website to S3. So my first task is to write an AWS S3 wagon that supports directory copy. With AWS Java SDK , task becomes as simple as one single class . I made my S3 wagon available in Maven central repository at org.cyclopsgroup:awss3-maven-wagon:0.1 . The source code is hosted in github:jiaqi/cym2/awss3 . The next thing is to create an S3 bucket in console . To avoid trouble, bucket name is set to the future website domain name according to this discussion . Website feature needs to be explicitly enabled. I also created an...
5 comments
Read more

Customize IdGenerator in JPA, gap between Hibernate and JPA annotations

JPA annotation is like a subset of Hibernate annotation, this means people will find something available in Hibernate missing in JPA. One of the important missing features in JPA is customized ID generator. JPA doesn't provide an approach for developer to plug in their own IdGenerator. For example, if you want the primary key of a table to be BigInteger coming from sequence, JPA will be out of solution. Assume you don't mind the mixture of Hibernate and JPA Annotation and your JPA provider is Hibernate, which is mostly the case, a solution before JPA starts introducing new Annotation is, to replace JPA @SequenceGenerator with Hibernate @GenericGenerator. Now, let the code talk. /** * Ordinary JPA sequence. * If the Long is changed into BigInteger, * there will be runtime error complaining about the type of primary key */ @Id @Column(name = "id", precision = 12) @GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "XyzIdGenerator") @SequenceGe...
4 comments
Read more

Spring, Angular and other reasons I like and hate Bazel at the same time

For several weeks I've been trying to put together an Angular application served Java Spring MVC web server in Bazel. I've seen the Java, Angular combination works well in Google, and given the popularity of Java, I want get it to work with open source. How hard can it be to run arguably the best JS framework on a server in probably the most popular server-side language with  the mono-repo of planet-scale ? The rest of this post walks through the headaches and nightmares I had to get things to work but if you are just here to look for a working example, github/jiaqi/angular-on-java is all you need. https://github.com/jiaqi/angular-on-java Java web application with Appengine rule Surprisingly there isn't an official way of building Java web application in Bazel, the closest thing is the Appengine rule  and Spring MVC seems to work well with it. 3 Java classes, a JSP and an appengine.xml was all I need. At this point, the server starts well but I got "No ...
Post a Comment
Read more