Skip to main content

Gitcon, runtime configuration via a Git repo

This weekend, aside from taking kid to football game and various classes, changing diapers and watching "Breaking bad", I've been working on an new small open source library, Gitcon, which allows Java application to decouple some configuration into a standalone source repository so that software owner can modify configuration without deploying application for every modification. It is a cheap approach to dynamic configuration.

By the way, I was very surprise to find out that the term "Gitcon" has not been reserved for "Git conference" so far.

Requirements


What led me to this project was a set of requirements:
  1. As a start, the configuration means properties file
  2. As a start, only Git source repository needs to be supported
  3. As a start, it only needs to gets configuration for once when application starts. The configuration is used to populate Spring application context where beans are mostly singletons. Software owner needs to restart application to pick up configuration change.
  4. The library must be very friendly to Spring
  5. It gets file from Git repo authenticated via SSH key.
  6. The SSH private key must be configurable
  7.  This is outside the scope of the library, but the SSH key comes from S3, which is authenticated by instance profile
  8. When SSH key is not specified, default one in user's home directory is applied. This is for development purpose.
  9. One Spring context may includes multiple properties coming from multiple repositories authenticated with different SSH keys.
  10. One properties file can include another under the same repository
 These requirements are the minimal goals that the first version of Gitcon must achieve in order to replace configurations of some existing applications.

Implementation


The design is fairly straightforward. All it needs is to cope with the limitations from Jsch that JGit relies on.


1. S3FileFactoryBean


S3FileFactoryBean is something that takes an IAM, a bucket and a key, downloads a file from S3 into local temporary directory, returns the file and deletes it when Spring context closes. Since the logic is generic to the purpose of Gitcon, it lives in a neutral package, kaufman-aws as artifact org.cyclopsgroup:kaufan-aws:0.0.2.

<bean id="gitSshKeyFile" class="org.cyclopsgroup.kaufman.aws.S3FileFactoryBean">
    <constructor-arg value="my-s3-bucket" />
    <constructor-arg value="my-app/git-ssh-private.key" />
</bean>

2. Clone, checkout Git repo with JGit


The easiest way to clone a Git repo is to use JGitLocalResourceRepository. With support from Eclipse JGit, this class defines Git repository URI, the access to Git repository and optionally the path to SSH private key. You may find out more details in Javadoc. An example:

<bean id="gitResourceRepo"
    class="org.cyclopsgroup.gitcon.spring.JGitLocalResourceRepository">
    <constructor-arg
        value="git@bitbucket.org:me/my-runtime-config.git" />
    <property name="sshIdentityFile" ref="gitSshKeyFile" />
</bean>

3. Create properties


The class GitconPropertiesFactoryBean simply creates Properties instance out of given ResourceRepository and a path to properties file. Since it creates Properties with ExtendedProperties, variable replacement and file inclusion is supported.

<bean id="appProperties"
    class="org.cyclopsgroup.gitcon.spring.GitconPropertiesBeanFactory">
    <constructor-arg ref="gitResourceRepo" />
    <constructor-arg value="myapp/myapp-prod.properties" />
</bean>

If you haven't tried, ExtendedProperties interprets properties file in a different way than Properties and exposes the content with a much more user friendly API. It is an ancient class from commons-collections, if a human is as old as this class, he would probably get a driver license by now.


4. Expand properties in Spring


This is done by a simple setup in Spring. It doesn't belong to Gitcon but I put it here so that the story is complete. The following config does the work.

<context:property-placeholder properties-ref="appProperties" />


To learn more


Like other CyclopsGroup projects, Gitcon Maven site is published to dist.cyclopsgroup.org/projects/gitcon. Very soon I will come up with a few wiki pages with official guides and examples.

Javadoc can be found here. Source code is in Github. The first version, org.cyclopsgroup:gitcon:0.0.1 has been published into Maven central repository.

If you have question, please feel free to contact me in Github, Facebook or Google+. Thank you for reading.
Labels:

Comments

Post a Comment

Popular posts from this blog

Spring, Angular and other reasons I like and hate Bazel at the same time

For several weeks I've been trying to put together an Angular application served Java Spring MVC web server in Bazel. I've seen the Java, Angular combination works well in Google, and given the popularity of Java, I want get it to work with open source. How hard can it be to run arguably the best JS framework on a server in probably the most popular server-side language with  the mono-repo of planet-scale ? The rest of this post walks through the headaches and nightmares I had to get things to work but if you are just here to look for a working example, github/jiaqi/angular-on-java is all you need. https://github.com/jiaqi/angular-on-java Java web application with Appengine rule Surprisingly there isn't an official way of building Java web application in Bazel, the closest thing is the Appengine rule  and Spring MVC seems to work well with it. 3 Java classes, a JSP and an appengine.xml was all I need. At this point, the server starts well but I got "No ...
Post a Comment
Read more

Customize IdGenerator in JPA, gap between Hibernate and JPA annotations

JPA annotation is like a subset of Hibernate annotation, this means people will find something available in Hibernate missing in JPA. One of the important missing features in JPA is customized ID generator. JPA doesn't provide an approach for developer to plug in their own IdGenerator. For example, if you want the primary key of a table to be BigInteger coming from sequence, JPA will be out of solution. Assume you don't mind the mixture of Hibernate and JPA Annotation and your JPA provider is Hibernate, which is mostly the case, a solution before JPA starts introducing new Annotation is, to replace JPA @SequenceGenerator with Hibernate @GenericGenerator. Now, let the code talk. /** * Ordinary JPA sequence. * If the Long is changed into BigInteger, * there will be runtime error complaining about the type of primary key */ @Id @Column(name = "id", precision = 12) @GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "XyzIdGenerator") @SequenceGe...
4 comments
Read more

Project Euler problem 220 - Heighway Dragon

This document goes through a Java solution for Project Euler problem 220 . If you want to achieve the pleasure of solving the unfamiliarity and you don't have a solution yet, PLEASE STOP READING UNTIL YOU FIND A SOLUTION. Problem 220 is to tell the coordinate after a given large number of steps in a Dragon Curve . The first thing came to my mind, is to DFS traverse a 50 level tree by 10^12 steps, during which it keeps track of a direction and a coordinate. Roughly estimate, this solution takes a 50 level recursion, which isn't horrible, and 10^12 switch/case calls. Written by a lazy and irresponsible Java engineer, this solution vaguely looks like: Traveler traveler = new Traveler(new Coordinate(0, 0), Direction.UP); void main() { try { traverse("Fa", 0); } catch (TerminationSignal signal) { print signal; } } void traverse(String plan, int level) { foreach(char c:plan) { switch(c) { case 'F': traveler.stepForward(); break; ca...
Post a Comment
Read more