Introducing Flixport, a tool that exports photos from Flickr

Overview

If you are like me with tens of thousands photos in Flickr which is about to stop 1TB free storage service, moving these photos out would be a very challenging task.

Flixport is an open source command line tool that bulk-copies photos from Flickr to Amazon S3, Google Storage or your local computer.

Flickr is terminating the free 1TB storage service in February 2019. Unless you convert to a premium user, Flickr will only keep 1000 photos and purge others at some point in February. To backup my own photos I evaluated all existing solution and none worked for me. Luckily, being a software engineer, I could write code to work around difficulties.

Please visit the Flixport page to find out more.

Why S3 and Google storage?

AWS S3, as of today, is the de facto online storage. People may hear Dropbox or Box more often but many of these consuming storage product are backed by another Cloud storage service, of which S3 is the clear leader. Since coding is not a problem for me, I got rid of the middle man and went straight to S3.

Because I work for Google, I also felt comfortable implementing a connection to Google Cloud storage. Sorry Azure, no Microsoft support for now.

Why not Dropbox, Google Drive or Google Photos?

These popular consumer products are obviously good next steps. In general for a command line to work with them, some OAuth-based authentication needs to happen and users will have to copy some long, obscure token from browser and paste it in command line. To offer the best user experience, it's better to run the tool from a website instead of as a command line.

Therefore, if I ever had bandwidth to work on the support of them, it'd not be part of the command line tool, but likely a web-based service that integrates with user's Google or Dropbox account.

Comments

Spring, Angular and other reasons I like and hate Bazel at the same time

For several weeks I've been trying to put together an Angular application served Java Spring MVC web server in Bazel. I've seen the Java, Angular combination works well in Google, and given the popularity of Java, I want get it to work with open source. How hard can it be to run arguably the best JS framework on a server in probably the most popular server-side language with the mono-repo of planet-scale ? The rest of this post walks through the headaches and nightmares I had to get things to work but if you are just here to look for a working example, github/jiaqi/angular-on-java is all you need. https://github.com/jiaqi/angular-on-java Java web application with Appengine rule Surprisingly there isn't an official way of building Java web application in Bazel, the closest thing is the Appengine rule and Spring MVC seems to work well with it. 3 Java classes, a JSP and an appengine.xml was all I need. At this point, the server starts well but I got "No ...

Customize IdGenerator in JPA, gap between Hibernate and JPA annotations

JPA annotation is like a subset of Hibernate annotation, this means people will find something available in Hibernate missing in JPA. One of the important missing features in JPA is customized ID generator. JPA doesn't provide an approach for developer to plug in their own IdGenerator. For example, if you want the primary key of a table to be BigInteger coming from sequence, JPA will be out of solution. Assume you don't mind the mixture of Hibernate and JPA Annotation and your JPA provider is Hibernate, which is mostly the case, a solution before JPA starts introducing new Annotation is, to replace JPA @SequenceGenerator with Hibernate @GenericGenerator. Now, let the code talk. /** * Ordinary JPA sequence. * If the Long is changed into BigInteger, * there will be runtime error complaining about the type of primary key */ @Id @Column(name = "id", precision = 12) @GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "XyzIdGenerator") @SequenceGe...

Project Euler problem 220 - Heighway Dragon

This document goes through a Java solution for Project Euler problem 220 . If you want to achieve the pleasure of solving the unfamiliarity and you don't have a solution yet, PLEASE STOP READING UNTIL YOU FIND A SOLUTION. Problem 220 is to tell the coordinate after a given large number of steps in a Dragon Curve . The first thing came to my mind, is to DFS traverse a 50 level tree by 10^12 steps, during which it keeps track of a direction and a coordinate. Roughly estimate, this solution takes a 50 level recursion, which isn't horrible, and 10^12 switch/case calls. Written by a lazy and irresponsible Java engineer, this solution vaguely looks like: Traveler traveler = new Traveler(new Coordinate(0, 0), Direction.UP); void main() { try { traverse("Fa", 0); } catch (TerminationSignal signal) { print signal; } } void traverse(String plan, int level) { foreach(char c:plan) { switch(c) { case 'F': traveler.stepForward(); break; ca...

Jiaqi's Blog

Search This Blog