Create a self-contained .jar file with Maven

If you want to create a self-contained .jar file, with all library dependencies included, you can do it with Maven. Include this in your pom.xml:


Then when you run mvn package, you will get a target/*-jar-with-dependencies.jar file which you can run with java -jar.

Posted in Java | Comments Off on Create a self-contained .jar file with Maven

Running Jetty as a service in Ubuntu Linix

Jetty is a popular open source Java application server. In order to run it as a service on a Linux server, they recommend using a horribly overcomplicated and quite fragile script.

In Ubuntu server, there is a better way. Leverage Upstart and its declarative job definitions.

First install Jetty into /opt/jetty and create a jetty user:

useradd --user-group --shell /bin/false --home-dir /opt/jetty/temp jetty

Then create a file /etc/init/jetty.conf with content like this:

start on runlevel [2345]
stop on starting rc RUNLEVEL=[016]

chdir /opt/jetty

setuid jetty

reload signal SIGQUIT

exec /usr/local/lib/jvm/bin/java \
-Xmx512m \
-Djava.awt.headless=true \
-Djetty.state=/opt/jetty/jetty.state \
-Djetty.port=8080 \
-Djetty.home=/opt/jetty \ \
-jar /opt/jetty/start.jar \
jetty.port=8080 \
etc/jetty-logging.xml \
etc/jetty-started.xml \
etc/jetty-requestlog.xml \
OPTIONS=plus etc/jetty-plus.xml \

This will replace both /etc/init.d/jetty and /etc/default/jetty.

If you run service jetty reload, it will create a thread dump (like jstack) in /var/log/upstart/jetty.log.

You can even add respwan to have the JVM automatically restarted if it unexpectedly quits for some reason.

A similar approach can be used for other JVM based servers, like Apache Tomcat.

Posted in Java, Linux, Ubuntu | Comments Off on Running Jetty as a service in Ubuntu Linix

Don’t use large BLOBs in MySQL from Java

The MySQL database can store large chunks of binary data (up to 1 GB) using the BLOB data types.

However, this does not work well if you access the MySQL database from Java (or any other JVM based language) using the MySQL JDBC driver.

The JDBC API supports an efficient stream based way to handle BLOBs, but the MySQL JDBC driver does not implement this properly. It works, but it will buffer all data in memory in a way which is very inefficient and can make your JVM run out of memory if the BLOBs are large.

A good rule of thumb is to not store any objects larger than 64 KB in MySQL. Store such objects as files outside of the database instead. So never use the MEDIUMBLOB, LONGBLOB, MEDIUMTEXT or LONGTEXT data types.

(I am not sure whether this advise is only valid for JVM, or for usage of MySQL from other environments as well.)

Posted in database, Java | 3 Comments

Don’t use PipedOutputStream on Android

I was using in an Android app. The app performed horribly bad. After doing some profiling, it turned out that the call to PipedOutputStream.write(byte[]) that was really slow. After digging into the Android source code, I discovered that PipedOutputStream.write(byte[]) was not implemented, it just delegated to the default implementation in OutputStream which iterate through the array and call PipedOutputStream.write(byte) for each byte.

Since PipedOutputStream.write(byte) does some synchronization and Object.notifyAll() each time, it is really slow to do this 1000 times when you write a block of 1 KB data.

Just out of curiosity, I had a look on how PipedOutputStream is implemented in Oracle’s standard Java implementation. I haven’t actually done any benchmarks on it, but I can see from the source code that it is a completely different implementation which does implement writing a block properly and probably efficiently.

The bottom line is: Don’t use PipedOutputStream on Android. If you need similar functionality, implement it yourself or find a 3rd party library which does it.

Posted in Android, Java | 2 Comments

Running Adobe Lightroom 4.4 in Linux

I use Adobe Lightroom 4.4 for photo editing. There is one very annoying aspect of this program, it is not available for Linux (only for Windows and Max OS X).

In order to run Lightroom on my computer, I had to use VirtualBox and install Windows 7 in it. This works, but is quite clumsy and annoying. And since Lightroom is the only reason for me to run Windows, I would like to get rid of it.

So I decided to try out Wine. The current version of Wine does not support Lightroom out of the box, but I found some patches which seems make it work. This is how I did it, for Ubuntu 12.04 32-bit.

  1. Uninstall any Wine installation you might already have
  2. Prepare to build Wine from source (for some odd reason, it should be wine1.4 and not wine1.6):
    sudo apt-get build-dep wine1.4
  3. Download Wine 1.6 sources
  4. Unpack Wine 1.6 sources:
    tar -xjf ~/Downloads/wine-1.6.tar.bz2
  5. Download the necessary patches
  6. Unpack and apply the paches:
    tar -xzf ~/Downloads/wine-1.6-lr5-patches.tar.gz
    patch -p0 <0-Menu-wine-1.6.patch
    patch -p0 <1-ConditionVariables-wine-1.6.patch
    patch -p0 <2-InitOnceExecuteOnce-wine-1.6.patch
  7. Build and install the patched Wine 1.6:
    cd wine-1.6
    sudo make install
  8. Install winetricks:
    cd /usr/local/bin
    sudo wget
    sudo chmod +x winetricks
  9. Install cabextract:
    sudo apt-get install cabextract
  10. Prepare and install Lightroom 4.4.1 32-bit:
    winetricks win7
    wine ~/Downloads/Lightroom_4_LS11_win_4_4_1.exe
    winetricks winxp
    winetricks corefonts gdiplus ie7
  11. Download sRGB color profile
  12. Unpack and install sRGB color profile:
    tar -xzf ~/Downloads/lr-wine-1.5.17.tar.gz
    cp "sRGB Color Space Profile.icm" ~/.wine/drive_c/windows/system32/spool/drivers/color
  13. Now you can start Lightroom with this command:

    wine "C:\\Program Files\\Adobe\\Adobe Photoshop Lightroom 4.4.1\\lightroom.exe"

    I found most of the information here.

Posted in Linux, Ubuntu | 16 Comments

Using an embedded SQL database in Java web application

You have a Java web application needing a relational SQL database. According to JavaEE conventions, you declare the DataSource in web.xml:


and then fetch it in code with (DataSource)new InitialContext().lookup("java:comp/env/jdbc/DS").

Using Maven, Jetty and HSQLDB, you can very easily run the web application for local testing without having to setup neither a web container, nor a database. Add this to pom.xml:


And add a file src/etc/jetty-ds.xml to your project:

<!DOCTYPE Configure PUBLIC "-//Mort Bay Consulting//DTD Configure//EN" "">
<Configure id="wac" class="org.eclipse.jetty.webapp.WebAppContext">
  <New id="DS" class="">
    <Arg><Ref refid="wac"/></Arg>
      <New class="org.hsqldb.jdbc.JDBCDataSource">
        <Set name="DatabaseName">mem:db</Set>
        <Set name="User">SA</Set>
        <Set name="Password">""</Set>

Then run it locally with mvn jetty:run. The database will be in-memory and does not presist between runs. It is also possible to configure HSQLDB to persist data on disk, see documentation.

If you want to have access to in-memory database in your tests, add this dependency to pom.xml:


And create the DataSource in your test code like this:

import org.hsqldb.jdbc.JDBCDataSource;

        JDBCDataSource hsqldbDs = new JDBCDataSource();

Note: This will not add the DataSource to JNDI.

Posted in Java, JavaEE, web | 3 Comments

GIMP 2.8 in Ubuntu 12.04

This is no longer relevant, since the PPA have updated to GIMP 2.8.10, which has fixed the bug.

If you use Ubuntu 12.04 LTS, you only have access to the old GIMP 2.6 in the standard repositories. This is unfortunate since GIMP 2.8 have a lot of new useful features.

The standard way to get GIMP 2.8 is to use the a PPA, as described here. This used to work fine. However, then the PPA upgraded to GIMP 2.8.6, which contains a horrible UI bug (seems like neither the GIMP team, nor the PPA maintainer has any particularly strong QA process).

I find this bug unacceptable, and the PPA has not yet released any new update. So I chose to use GIMP 2.8.4 instead. This is not available from the PPA, so I decided to build it from source.

I found some instructions for building here. However, I was unable to build GEGL since the latest version uses a newer version of a standard library than what is available in Ubuntu 12.04.

Fortunately, you can still use BABL and GEGL from the PPA, and only build GIMP itself from source.

This is what you need to do:

  1. Install the PPA (if you haven’t done so already), but do not install GIMP.
  2. Uninstall any version of GIMP already installed (from standard repository or PPA):

    sudo apt-get purge gimp gimp-data gimp-gmic gimp-plugin-registry libgimp2.0 gimp-help-common gimp-help-en gimp-lensfun
  3. Install various dependencies:

    sudo apt-get -y build-dep gimp
    sudo apt-get purge libgegl-0.0-dev libbabl-0.0-0-dev libgegl-0.0 libbabl-0.0-0
    sudo apt-get -y install gegl libgegl-0.2-0 libgegl-dev libbabl-0.1-0 libbabl-dev
    sudo apt-get -y install libavformat-dev libavcodec-dev valac-0.16 graphviz ruby1.9.1-dev liblua5.1-0-dev libopenexr-dev libopenraw-dev libexiv2-dev libsdl1.2-dev libgs-dev libjasper-dev checkinstall
  4. Build GIMP 2.8.4 from source:

    cd /tmp
    mkdir gimp_build && cd gimp_build
    export PATH=/opt/gimp-2.8/bin:$PATH
    tar -xjf gimp-2.8.4.tar.bz2
    cd gimp-2.8.4
    ./configure --prefix=/opt/gimp-2.8
    sudo make install
  5. Make it possible to launch it from Unity:

    cp /opt/gimp-2.8/share/applications/gimp.desktop ~/.local/share/applications/

    and edit~/.local/share/applications/gimp.desktop to include this:

    Exec=/opt/gimp-2.8/bin/gimp-2.8 %U

Unfortunately, I have not managed to get the G’MIC GIMP plugin to work.

Posted in Linux, Ubuntu | Tagged | 2 Comments

Using AAC music files in Android

If you have an file with .aac extension, it is AAC encoded audio in an ADTS container. If you want to play such file on an Android device, you have problems. Android 2.x does not support this file format at all, and not even the latest version of Android supports reading metadata tags from it.

The solution is to repackage the audio in an MPEG-4 container to a file with .m4a extension. Android can both play and read metadata tags from such files.

This can be done with the MP4Box tool. You can also add metadata tags while repackaging:

MP4Box -itags "artist=${ARTIST}:name=${TITLE}:genre=${GENRE}" -add ${FILENAME}.aac#audio ${FILENAME}.m4a

If I understand it correctly, this will not recode the actual audio data, and thus not affect the quality in any way. The file size will also be almost the same.

Posted in Android, Linux | 1 Comment

How to design a RESTful protocol

What is REST?

In most cases, it is sufficient say that REST is a way to design a network protocol based on HTTP. I perfer to call it a RESTful protocol, but it can also be called RESTful API or RESTful Web Service. It is important to keep in mind that it technically is a network protocol, and not an API like an interface with some methods in a regular programming language.

REST is not RPC or RMI. You should not try to map an existing API in a regular programming language to a RESTful protocol directly method by method. Rather, you should map the domain model of your application to the protocol.

REST is not a religion. Be pragmatic and use what works for you. But be sure to understand the rules before you possibly break them, otherwise you may get unpleasant surprises.

How to do REST

The most important rule is to use HTTP properly and leverage it extensively. Read and understand the HTTP 1.1 specification in RFC2616.

Understand the concept of safe and idempotent methods:

  • safe – no observable side-effects
  • idempotent – doing same request several times have the same effect as doing it once

And use the appropriate HTTP methods:

  • GET and HEAD are safe and idempotent
  • PUT and DELETE are not safe but idempotent
  • POST does not have to be either safe or idempotent

Utilize the rich set of response codes defined in HTTP:

  • 200 OK
  • 201 Created
  • 202 Accepted
  • 303 See Other
  • 304 Not Modified
  • 400 Bad Request
  • 401 Unauthorized
  • 403 Forbidden
  • 404 Not Found
  • 405 Method Not Allowed
  • 406 Not Acceptable
  • 409 Conflict
  • 412 Precondition Failed
  • 415 Unsupported Media Type
  • 500 Internal Server Error
  • 503 Service Unavailable

Make use of HTTP headers:

  • Location
  • Allow
  • Authorization, WWW-Authenticate
  • Content-Type, Accept
  • ETag, If-Match, If-None-Match
  • Last-Modified, If-Modified-Since, If-Unmodified-Since
  • Cache-Control
  • etc

The request and response messages should simply contain your domain objects, probably encoded as JSON or XML (do not forget to set proper Content-Type). Do not wrap it in any envelope as SOAP does.

Do this: {"prop1":"foo","prop2":["one","two","three"]}

Do not do this: {"result":{"prop1":"foo","prop2":["one","two","three"]}}

Use HTTP response codes for errors/exceptions.

State and security

Try to avoid storing client session state on the server. You can push small session state to the client in URLs. Use HTTP Authentication, possibly with OAuth, to avoid having to keep track of “logged in users” on the server. Always use HTTPS with proper server certificates if you have any authentication, and consider using client certificates if it makes sense.


If you follow all advices so far, you will probably be able to design a good and useful protocol. But it will not be RESTful unless you also apply Hypermedia as the Engine of Application State.

This will give you a high degree of decoupling between client and server. It will be easy to evolve the protocol in a backwards compatible way, to allow newer clients make use of new features and still allow old clients to operate. And you can do this without any explicit versioning of the protocol.

The client should only need one entry-point URL, and will get all other URLs in responses from the server.

In HTML, you have semi-standardized link relations:
<link href="" rel="edit" />

Use something similar to allow the server to send back URLs to the client:

URLs should be opaque to the client. The client should accept both absolute and relative URLs, and resolve relative URLs according to RFC3986 (use a platform or 3rd party library for this). The client may sometimes append query parameters. The client should never inspect or manipulate URLs in any other way. (This is exactly how an ordinary web browser handles links in an HTML web page.)

Posted in web | Comments Off on How to design a RESTful protocol

How to add password protection to GRUB2

These instructions are tested with Ubuntu desktop 12.04, but will probably be useful in other Linux distros with GRUB2 as well.

The goal is to block everything except booting the default system. In paricular, it should not be possible for anyone to boot into recovery mode, since that will bypass normal login and give root access directly.

  1. Run grub-mkpasswd-pbkdf2 from a terminal and enter the desired password, copy the output.
  2. Edit /etc/grub.d/40_custom and add this to the end:
    set superusers="root"
    password_pbkdf2 root output from grub-mkpasswd-pbkdf2 goes here
    password bogus bogus
  3. Make /etc/grub.d/40_custom non-readable for users:
    chmod o-r /etc/grub.d/40_custom
  4. Edit /etc/grub.d/10_linux according to this diff:
    <   printf "menuentry '${title}' ${CLASS} {\n" "${os}" "${version}"
    >   if ${recovery} || ${in_submenu}; then
    >     printf "menuentry '${title}' ${CLASS} --users '' {\n" "${os}" "${version}"
    >   else
    >     printf "menuentry '${title}' ${CLASS} {\n" "${os}" "${version}"
    >   fi
    <     echo "submenu \"Previous Linux versions\" {"
    >     echo "submenu \"Previous Linux versions\" --users '' {"
  5. Edit /etc/grub.d/20_memtest86+ according to this diff:
    < menuentry "Memory test (memtest86+)" {
    > menuentry "Memory test (memtest86+)" --users "" {
    < menuentry "Memory test (memtest86+, serial console 115200)" {
    > menuentry "Memory test (memtest86+, serial console 115200)" --users "" {
    < #menuentry "Memory test (memtest86+, experimental multiboot)" {
    > #menuentry "Memory test (memtest86+, experimental multiboot)" --users "" {
    < #menuentry "Memory test (memtest86+, serial console 115200, experimental multiboot)" {
    > #menuentry "Memory test (memtest86+, serial console 115200, experimental multiboot)" --users "" {
  6. Run update-grub

(I am not sure if it is possible to abuse memtest86+, but better safe than sorry.)

See this page for more information.

Please note that this by itself does not give you a secure system. It should be combined password protection for BIOS setup and for booting from removable media (CD-ROM) and USB devices. And you should not allow login without password in the main Linux system.

None of these measures protect from serious tampering with the hardware, such as removing the internal HDD and connecting it as non-boot device to another computer.

Posted in Linux, security, Ubuntu | 2 Comments