Skip to main content

Introducing Flixport, a tool that exports photos from Flickr

Overview


If you are like me with tens of thousands photos in Flickr which is about to stop 1TB free storage service, moving these photos out would be a very challenging task.

Flixport is an open source command line tool that bulk-copies photos from Flickr to Amazon S3, Google Storage or your local computer.

Flickr is terminating the free 1TB storage service in February 2019. Unless you convert to a premium user, Flickr will only keep 1000 photos and purge others at some point in February. To backup my own photos I evaluated all existing solution and none worked for me. Luckily, being a software engineer, I could write code to work around difficulties.

Please visit the Flixport page to find out more.

Why S3 and Google storage?


AWS S3, as of today, is the de facto online storage. People may hear Dropbox or Box more often but many of these consuming storage product are backed by another Cloud storage service, of which S3 is the clear leader. Since coding is not a problem for me, I got rid of the middle man and went straight to S3.

Because I work for Google, I also felt comfortable implementing a connection to Google Cloud storage. Sorry Azure, no Microsoft support for now.

Why not Dropbox, Google Drive or Google Photos?


These popular consumer products are obviously good next steps. In general for a command line to work with them, some OAuth-based authentication needs to happen and users will have to copy some long, obscure token from browser and paste it in command line. To offer the best user experience, it's better to run the tool from a website instead of as a command line.

Therefore, if I ever had bandwidth to work on the support of them, it'd not be part of the command line tool, but likely a web-based service that integrates with user's Google or Dropbox account.
Labels:

Comments

Post a Comment

Popular posts from this blog

Publish Maven site with Amazon S3 and CloudFront

Amazon S3 now supports static website hosting . As a 10 years Maven user, I wonder how easy it is to deploy Maven generated site to Amazon S3 and let the rock-solid storage provider to host my project websites. There are several existing s3 wagon providers , which all seem to have the same problem, not supporting directory copy. This is understandable since before S3 new website hosting feature, I guess people mostly expect to deploy artifacts rather than website to S3. So my first task is to write an AWS S3 wagon that supports directory copy. With AWS Java SDK , task becomes as simple as one single class . I made my S3 wagon available in Maven central repository at org.cyclopsgroup:awss3-maven-wagon:0.1 . The source code is hosted in github:jiaqi/cym2/awss3 . The next thing is to create an S3 bucket in console . To avoid trouble, bucket name is set to the future website domain name according to this discussion . Website feature needs to be explicitly enabled. I also created an...
5 comments
Read more

4-states state machine for CSV parsing

Parsing CSV file is easy, it's nothing but splitting string with comma delimiter, which can be easily done in Java... The first thing came to my mind when I'm about to parse CSV file in Java is just like that. Now, reality is that following examples are all possible valid lines in a CSV file 1,Bender 2,"Bender" 3,"Bender, Bending" 4,"Ben""d""er" 5, Ben"der 6, Ben""der Line 7 might be arguable but anyway, two basic rules are If there's comma in field, use double quot to wrap field, otherwise double quot wrapper isn't required. Inside double quot, double quot is used to escape double quot. Suddenly the problem is complicated to something more than string splitting, however it can be simplified into a finite state machine with 4 states. States: 1. Ready for new field (initial state) 2. Field without double quot 3. Field with double quot 4. Escaping or end of double quot Transitions *Direction*|*Condition*|*Ac...
Post a Comment
Read more

1300ms to 160ms, tune Spring/Hibernate on slow MySQL

I write this article to remember the different behaviour various JDBC connection pool displays when they work with slow JDBC connection(to MySQL database, in this case). It starts with a typical Java application on Spring, Hibernate, Jetty, ApacheCXF and MySQL like following code. Version 1: without correct pooling //... service code @Transactional(isolation=Isolation.READ_COMMITTED) public void foo() { //... do something with database } //... connection pool configuration ... class = "com.mysql.jdbc.jdbc2.optional.MysqlConnectionPoolDataSource"; url = "jdbc:mysql://mysql.far-far-away.com/mysystem"; user = ... //... transaction management configuration in spring ... <tx:annotation-driven transaction-manager="transactionManager" order="100" /> <bean id="transactionManager" class="org.springframework.orm.hibernate3.HibernateTransactionManager"> <property name="sessionFactory" ref="mySessionFact...
2 comments
Read more