Jersey 1.13 Released

July 13th, 2012 by Martin 2 comments »

Last week we released Jersey 1.13 after the long 4+ months! Since the team is now fully focusing on Jersey 2.0 and JAX-RS 2.0 development, this is mostly a bug-fixing release. Anyway, we did manage to address a number of interesting issues. As for any other Jersey 1.x release, the list of changes can be found in changes.txt file in the root of our repository.

The main changes were around JSON handling – Michal, one of our new team-mates, got rid of our custom JSON stream reader and replaced it with a Jackson-based one to make our JSON-JAXB binding more robust. He also made some fixes and improvements to Jersey integration with MOXy JAXB implementation. We received a few external code contributions – from Edouard Chevalier (fixing the interaction of GZIP encoding filter with ETags) and from Atlassian – thank you guys! Finally, I ported the proxy client to Jersey 1.x. It is available in the Jersey 1.x svn repository under experimental/proxy-client (you won’t find it mentioned in changes.txt given it is an experimental module).

Writing Web Applications That Can Run With Jersey 1.x As Well As Jersey 2.0

June 26th, 2012 by Martin 1 comment »

Now, that early builds of Jersey 2.0 are integrated into GlassFish application server main trunk (which Jakub is still planning to blog about) more people have to deal with the reality that Jersey 2.0 is not compatible with Jersey 1.x. All the internal Jersey API’s underwent a significant refactoring in Jersey 2.0. And people started asking if there is any way of designing the web application (WAR-based) so that it can run on GlassFish 3.x (which bundles Jersey 1.x) while also being able to run with GlassFish main trunk (that now bundles Jersey 2.0 milestone builds). So, I decided to spend my last week’s “weekend coding” on creating a simple jar that one can bundle in their war to enable this. As a result, I added a new maven module under jersey/ext (in Jersey 2.0 workspace) called servlet-portability. You can build it and add it to your application as a dependency. Now, here is what it does and what it doesn’t do:

  • It provides PortableServletContainer class you can use in web.xml instead of the Jersey version-specific ServletContainer class to register a Jersey servlet.
  • It enables your application to bundle Jersey 2.x-specific code along with Jersey 1.x-specific code and helps to ensure the right code gets loaded depending on which version of the Jersey runtime is on the runtime classpath.
  • It does not provide a compatibility layer – in other words, it does not enable you to use API’s from Jersey 1.x with Jersey 2.x runtime – i.e. in your web app you have to write the Jersey version-specific code twice – one class using Jersey 1.x API’s and another one using Jersey 2.x API’s.

Example Application

Here is how you can use it. I will build a simple application with the following requirements:

  1. Use package-scanning for discovering all the resources and providers that are using pure JAX-RS API (i.e. are not using any version-specific Jersey API’s)
  2. Expose one resource that will require different implementation depending on Jersey version

Getting Jersey 2.0 Source Code and Building It

To try this out let’s first get Jersey 2.0 source code. You can do it in one of the following ways:

  • Download the sources zip file, or
  • Clone Jersey git repository on java.net by executing the following in the terminal:
    git clone ssh://<your_java_net_userid>@git.java.net/jersey~code jersey

    , or

  • Clone Jersey git repository on github.com by executing the following in the terminal:
    git clone git://github.com/jersey/jersey.git

Now, to build the code execute the following in the source tree root:

mvn clean install -Dmaven.test.skip=true

Creating a New Web Application Project

We will use Jersey 2.0 web application maven archetype to quickly create a skeleton of a new Jersey web application. To do that, switch to the directory where you want your project to reside and execute the following command in that directory:

mvn archetype:generate -DarchetypeArtifactId=jersey-quickstart-webapp -DarchetypeGroupId=org.glassfish.jersey.archetypes -DinteractiveMode=false -DgroupId=com.example -DartifactId=simple-service -Dpackage=com.example -DarchetypeVersion=2.0-SNAPSHOT

You should see a new simple-service subdirectory got created that contains the project files. Now, open this project in your favorite IDE. You should see it is a simple project containing a single resource called MyResource (in com.example package) and registering Jersey servlet using web.xml descriptor.

Updating the pom.xml

Let’s update the pom.xml of the project to add a dependency on the portability module, change the scope of Jersey 2.0 servlet dependency to “provided” (so that the Jersey jars are not bundled in our war) and add Jersey 1.x servlet dependency. The resulting pom.xml should look as follows:


    4.0.0
    com.example
    simple-service
    <packaging>war</packaging>
    1.0-SNAPSHOT
    simple-service
    
        simple-service
        <plugins>
            <plugin>
                org.apache.maven.plugins
                maven-compiler-plugin
                true
                
                    1.6
                    1.6
                
            </plugin>
        </plugins>
    
    
        
            org.glassfish.jersey.ext
            jersey-servlet-portability
            ${jersey.version}
        
        
            org.glassfish.jersey.containers
            jersey-container-servlet-core
            ${jersey.version}
            provided
        
        
            com.sun.jersey
            jersey-servlet
            1.12
            provided
        
    
    <properties>
        2.0-SNAPSHOT
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

Adding Jersey Version-Specific Resource

Create two new sub-packages – com.example.jersey1 and com.example.jersey2. We will use these for version-specific classes. Let’s create a new resource in com.example.jersey2 package and name it InfoResource. Just for illustration, it will inject ResourceConfig class from Jersey 2 API, retrieve the value of “example.text” property and return it back to the client. Here is how the resource class looks like:

package com.example.jersey2;

import javax.ws.rs.GET;
import javax.ws.rs.core.Context;
import org.glassfish.jersey.server.ResourceConfig;

public class InfoResource {
    private @Context ResourceConfig rc;
    
    @GET
    public String get() {
        return (String) rc.getProperty("example.text");
    }
}

As you can see on line 5 it uses Jersey version-specific class. Let’s clone this into com.example.jersey1 package, changing just the import statement to import ResourceConfig from Jersey 1.x package. Here is how the InfoResource in com.example.jersey1 should look like:

package com.example.jersey1;

import javax.ws.rs.GET;
import javax.ws.rs.core.Context;
import com.sun.jersey.api.core.ResourceConfig;

public class InfoResource {
    private @Context ResourceConfig rc;
    
    @GET
    public String get() {
        return (String) rc.getProperty("example.text");
    }
}

One important thing to note here is I haven’t annotated InfoResource using the @Path annotation. This is to make sure it does not get picked up by the package scanning (otherwise it would result in Jersey 2-specific resource being loaded when only Jersey 1.x runtime is present or vice-versa). We will use the “explicit root resources” feature from Jersey 1.x and resource builder API from Jersey 2.0 to register this unannotated resource into the Jersey runtime. We need to implement a custom ResourceConfig class for that.

Implementing ResourceConfig

Add ExampleResourceConfig class to com.example.jersey2 package and configure it to scan “com.example” package for resources and providers and map InfoResource to “info” path using the resource builder API. Here is the source code:

package com.example.jersey2;

import java.util.ArrayList;
import org.glassfish.jersey.server.ResourceConfig;
import org.glassfish.jersey.server.model.Resource;
import org.glassfish.jersey.server.model.ResourceModelIssue;

public class ExampleResourceConfig extends ResourceConfig {
    public ExampleResourceConfig() {
        // look for resources in com.example package and its subpackages
        this.packages("com.example");
        
        // create a resource builder from the InfoResource class
        // (sending in dummy issues list - we don't expect introspection issues)
        Resource.Builder rb = Resource.builder(
            InfoResource.class,
            new ArrayList<ResourceModelIssue>()
        );
        // bind it to "info" path
        rb.path("info");
        // add it to the resource config
        this.addResources(rb.build());
    }
}

Add the Jersey 1.x version of the same class into the com.example.jersey1 package. It should look as follows:

package com.example.jersey1;

import com.sun.jersey.api.core.PackagesResourceConfig;

// extend PackagesResourceConfig to get the package scanning functionality
public class ExampleResourceConfig extends PackagesResourceConfig {
    public ExampleResourceConfig() {
        // pass name of the package to be scanned
        super("com.example");
        
        // add InfoResource as an explicit resource bound to "info" path
        this.getExplicitRootResources().put("info", InfoResource.class);
    }
}

Adding Link to InfoResource to the index.jsp

To be able to conveniently access the InfoResource, let’s add a link to the index.jsp page. Here is how the updated index.jsp looks like:

<html>
<body>
    <h2>Jersey RESTful Web Application!</h2>
    <p>Jersey resource
    <p>InfoResource
    <p>Visit Project Jersey website
    for more information on Jersey!
</body>
</html>

Updating web.xml – Putting It All Together

And finally we will use the PortableServletContainer to put this all together. Open web.xml and replace the use of ServletContainer with the use of PortableServletContainer. It will have two sets of init-parameters – one Jersey 1.x specific – prefixed with jersey1#, the other one Jersey 2.0 specific prefixed with jersey2#. The servlet will figure out automatically what version of Jersey runtime is present and use the right init-params subset. Btw, obviously you don’t have to add jersey1/2# prefix to the parameter name if that parameter should be present and have the same value regardless of Jersey version being used. Here is how the updated web.xml should look like:



    
        Jersey Web Application
        org.glassfish.jersey.servlet.portability.PortableServletContainer
        
            jersey1#javax.ws.rs.Application
            com.example.jersey1.ExampleResourceConfig
        
        
            jersey1#example.text
            Using Jersey 1.x
        
        
            jersey2#javax.ws.rs.Application
            com.example.jersey2.ExampleResourceConfig
        
        
            jersey2#example.text
            Using Jersey 2.0
        
        1
    
    
        Jersey Web Application
        /webapi/*
    

Trying It Out

Now you can build the application and deploy the war to GlassFish 3.1.2 which bundles Jersey 1. Assuming the GlassFish is running at locahost:8080, you should be able to access the application at http://localhost:8080/simple-service/. You can see that clicking on Jersey resource works (invokes MyResource.getIt() method returning “Got it!”) and clicking on InfoResource shows “Using Jersey 1.x”.

If you deploy the same thing on the latest GlassFish 4.0 promoted build that bundles Jersey 2, you should see that after clicking on InfoResource, “Using Jersey 2.0″ is displayed.

Resources

You can download a zip file with this example application at: http://java.net/projects/jersey/downloads/download/portability-example.zip

Another example can be seen in in Jersey 2.0 integration tests workspace: https://github.com/jersey/jersey/tree/master/tests/integration/portability-jersey-1

Fork Jersey 2.x on GitHub

May 24th, 2012 by Martin 2 comments »

While working full speed on Jersey 2.0, we are trying to be as transparent as possible. The JAX-RS itself is run in a transparent way – everybody can see the EG discussions by subscribing to users@jax-rs-spec.java.net or browsing the mailing list archives at http://jax-rs-spec.java.net. With Jersey 2.0 we are trying to track everything in the public Jersey JIRA – just click on the Agile tab and switch to the current milestone to see the planned tasks and the progress we are making.

One thing we haven’t been able to open up (due to firewall restrictions) is our internal code review process. We are using Gerrit to review every commit that goes into the public repository. We have hooks set up for the internally hosted Hudson to kick in for each review request and automatically verify that the proposed commit does not break the build if approved. All works great, but unfortunately isn’t available for external checkins (all checkins have to go through the internal Gerrit Git repository and only Gerrit pushes the changes to java.net once they are approved).

So, to make things more transparent and easier (and inviting) for potential external contributors, and people who want to keep an eye on what’s going on, I’ve been playing lately with making the Jersey 2.x sources available on GitHub as well. We’ve managed to set up a 3-way synchronization between GitHub, Gerrit and java.net which turns the GitHub Jersey 2.x workspace mirror into the first class citizen. The recently announced BuildHive free service from CloudBees helped to supply the pull request verification, so we are now able to handle GitHub pull requests in a similar way to our internal Gerrit review requests. While we will most likely still continue using our internal set-up to do the internal code reviews (due to better performance and integration with internal tools), we are now able to handle external pull requests in a nice manner. So you are now free to follow and/or fork Jersey on GitHub and eventually submit patches as pull request. Here is the link: http://github.com/jersey/jersey

Just remember, for us to be able to accept your contributions, you need to send in a signed OCA.

Proxy Client on Top of JAX-RS 2.0 Client API

May 4th, 2012 by Martin 6 comments »

UPDATE (7/3/2012): Proxy-client module is now part of the regular workspace (instead of incubator) and get’s pushed to maven along with other Jersey modules, so no need to build it manually anymore.

Several JAX-RS 1.x implementations are providing proxy client API as one of the ways to access web services. The basic idea is you can attach the standard JAX-RS annotations to an interface, and then implement that interface by a resource class on the server side while reusing the same interface on the client side by dynamically generating an implementation of that using java.lang.reflect.Proxy calling the right low-level client API methods.

Jersey 1.x client API (and now JAX-RS 2.0 client API) leverages the fact REST (and HTTP) defines the standard set of operations, so unlike for JAX-WS type of web services, which are verb-centric, for RESTful web services the interface is uniform and thus there is no need for generating client proxies. The standard client API is static (has fixed set of operations – get, put, post, delete, …). Proxy client API may be seen as evil, as it creates tighter coupling between services and clients. However, some people do find it useful and it provides better opportunities for reusing JAX-RS concepts between services and clients, including name-bound interceptors, for example.

Few weeks back I quickly “hacked” a proxy client factory for Jersey as part of Jersey 2.0 and made some small incremental improvements since then. It is based purely on the current draft of the standard JAX-RS 2.0 client API, so should be usable with any JAX-RS 2.0 implementation, not just Jersey. Currently it lives in the “incubator” subfolder of Jersey 2.0 workspace, so if you want to give it a try, you have to check it out and build it yourself. The project includes a test that illustrates the usage, and also a simple example.

Building the Proxy-Client Project

To build it, first clone the Jersey repository, build Jersey, then switch to the proxy-client folder and build the proxy-client project. The following shell commands do that (assuming you have git and mvn installed):

git clone ssh://[your-java.net-user-id]@git.java.net/jersey~code jersey
cd jersey
mvn install
cd incubator/proxy-client
mvn install

Using/Playing with the Proxy-Client

The project has just one class – WebResourceFactory. Here is how it can be used:

Let’s say you want to access a web resource available at a URI that could be represented by the following URI template:

http://acme.com/rest/books/{book-id}

Let’s assume this resource can produce and consume application/json and application/xml and supports GET, PUT and DELETE operations. There is also a parent resource – http://acme.com/rest/books/ which accepts GET (returns a list of books) and POST (creates a new book). Here is one example of how you can model it using the interfaces with JAX-RS annotations and then access these resources using those interfaces and WebResourceFactory class:

Interface representing the “books” resource:

@Path("books")
public interface BooksResource {
    @GET
    @Produces({MediaType.APPLICATION_XML, MediaType.APPLICATION_JSON})
    List<Book> getBooks();

    @POST
    @Consumes({MediaType.APPLICATION_XML, MediaType.APPLICATION_JSON})
    Response createBook(Book book);

    @Path("{bookid}")
    BookResource getBookResource(@PathParam("bookid") String bookId);
}

Interface representing “books/{bookid}” subresource:

public interface BookResource {
    @GET
    @Produces({MediaType.APPLICATION_XML, MediaType.APPLICATION_JSON})
    Book get();

    @PUT
    @Consumes({MediaType.APPLICATION_XML, MediaType.APPLICATION_JSON})
    @Produces({MediaType.APPLICATION_XML, MediaType.APPLICATION_JSON})
    Book update(Book book);

    @DELETE
    void delete();
}

Accessing it from your application:

// create a new JAX-RS 2.0 target pointing to the root of the web api
Target t = ClientFactory.newClient().target("http://acme.com/rest/");

// create a new client proxy for the BooksResource
BooksResource booksRsc = WebResourceFactory(BooksResource.class, t);

// get list of books
List<Book> books = booksRsc.getBooks();

// get book resource by ID
BookResource bookRsc = booksRsc.getBookResource(bookId);
// get book object
Book myBook = bookRsc.get();
// delete book
bookRsc.delete();

I hope you got the idea. As you can see, the proxy factory can handle path parameters and can automatically create proxies for sub-resources. It can also handle other kinds of parameters such as header parameters, query parameters, form parameters and cookie parameters.

Give it a try and let us know what you think.

Optimizing MacOS X Lion for SSD

April 1st, 2012 by Martin 146 comments »

Approx. 18 months back I bought my first SSD (120GB OCZ Vertex 2) for my (then) new iMac. I am using it as the primary disk, while I kept the original 1TB HDD as the secondary one. So far I’ve been very happy with it and thanks to applying some tweaks the drive seems still healthy as new. Recently I bought another SSD – this time a 240GB Vertext 3 for my MacBook – that one I am using as the only drive in the laptop. And as I found out, with the new features in Lion, some additional tweaks need to be applied – especially in case of a laptop. So, I’ve decided to write this blog – for my own benefit (to keep track of what I did in case I have to re-apply it in the future) as well as for the benefit of others who may run into this blog entry. Here I am providing the list of things I found on various web sites or came up with myself to minimize the risk the SSD wears out too soon. Some of them you may already be aware of. So, here is a list of content in case you want to skip to particular sections:

DISCLAIMER: Applying any of these tweakss is at your own risk. Make sure you back up your computer before trying any of these.

Don’t run benchmarks on your new SSD
Use Trim Enabler
Turn off local Time Machine snapshots [laptops only]
Turn off hibernation [laptops only]
Set noatime flag
Move user home directories to HDD [SSD+HDD only]
Use RAM disk or HDD for temporary files
Turn off sudden motion sensor [SSD+HDD only]
Turn off hard drive sleep [SSD+HDD only]
References/Other tweaks

I’ll try keeping this list up to date. Let me know if any other tweaks work well for you.

Don’t run benchmarks on your new SSD

Some people, right after they buy a new SSD, want to enjoy the speed and are eager to find out how much faster exactly their new SSD is than the old HDD. So they get this cool idea of running some extensive benchmarks to see the amazing performance numbers. Benchmarks usually write a lot of data to the disk (to test the write speed), wearing it out. So it is the best way how you can ruin your SSD even before you start using it. Don’t do it.

Use Trim Enabler

TRIM support is essential for keeping the SSD healthy. Unfortunately, MacOS supports TRIM only for Apple-supplied SSD’s. If you bought a 3rd party one like I did, you have to tweak the system to be able to turn on the TRIM support. Trim Enabler is a simple utility that does this for you. You can read more here or go to the TrimEnabler web site directly.

Turn off local Time Machine snapshots [laptops only]

If you are using the SSD in a laptop, and you turned the Time Machine backups on, you should know that OS X Lion does local snapshots at times when your Time Machine disk is not available. This is quite common – you are typically connecting your external TM disk only once in a while or you are using Time Capsule at home but take your laptop to the office every day for 8+ hours. You can confirm if the local TM backups are on by opening Time Machine Preferences:
Time Machine Preferences pane showing local backups are enabled

There is no GUI switch to turn these local backups off, but it can easily be done on the command line. Just start Terminal.app and execute the following command:

sudo tmutil disablelocal

Once you do this, the TM Preferences panel will immediately reflect it – the text will change from “Time Machine keeps local snapshots as space permits, and:” to “Time Machine keeps:”. To turn it back on, you can simply run the following in the Terminal:

sudo tmutil enablelocal

Also note, this feature gets turned on automatically whenever you turn off and on the Time Machine – so don’t forget to turn the local backups back off again whenever you do that.

I found out about how to turn off the local backups from this article: http://osxdaily.com/2011/09/28/disable-time-machine-local-backups-in-mac-os-x-lion/

Turn off hibernation [laptops only]

Another feature of Mac OS turned on by default on laptops is, that it saves all the memory to disk when entering sleep mode. This is to ensure your laptop does not lose your work if it runs out of battery while “sleeping”. The more RAM you have, the more gigabytes it writes to the disk every time you close the lid/put it to sleep. I typically do this at least twice a day – when leaving the office and when going to sleep in the evening. If you are in a similar situation and you have 8GB of RAM, that means your MacBook writes 16 to 24 GB of hibernation data to your SSD every day. Here is how you can turn this off – it will not only make your SSD’s life longer, but also significantly speed up the time it takes for your laptop to enter the sleep mode:

sudo pmset -a hibernatemode 0

I found it in this article: http://news.metaparadigma.de/osx86-enable-and-disable-hibernation-57/. Reading the man pages for pmset sheds some more light on the factory defaults and meaning of the hibernatemode values:

We do not recommend modifying hibernation settings. Any changes you make are not supported. If you choose to do so anyway, we recommend using one of these three settings. For your sake and mine, please don’t use anything other 0, 3, or 25.

hibernatemode = 0 (binary 0000) by default on supported desktops. The system will not back memory up to persistent storage. The system must wake from the contents of memory; the system will lose context on power loss. This is, historically, plain old sleep.

hibernatemode = 3 (binary 0011) by default on supported portables. The system will store a copy of memory to persistent storage (the disk), and will power memory during sleep. The system will wake from memory, unless a power loss forces it to restore from disk image.

hibernatemode = 25 (binary 0001 1001) is only settable via pmset. The system will store a copy of memory to persistent storage (the disk), and will remove power to memory. The system will restore from disk image. If you want “hibernation” – slower sleeps, slower wakes, and better battery life, you should use this setting.

Once you turn off hibernation, you can also remove the sleep image file that will free up several GB of disk space (depending on how much RAM you have):

sudo rm /var/vm/sleepimage

Set noatime flag

MacOS (like other unix-based systems) by default records last access time for every file. I.e. every time you read a file, a write is made on the filesystem to record this action. There is no point in doing it and no side effects if you disable that by mounting the root filesystem with noatime flag set. To do that create a file named for example “com.nullvision.noatime.plist” (you can pick any other name you wish) in the directory /Library/LaunchDaemons with the following content:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" 
        "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

    
        Label
        com.nullvision.noatime
        ProgramArguments
        
            mount
            -vuwo
            noatime
            /
        
        RunAtLoad
        <true/>
    

And execute the following command in Terminal:

sudo chown root:wheel /Library/LaunchDaemons/com.nullvision.noatime.plist

Restart the machine.

You can verify that it worked by executing the following in Terminal:

mount | grep " / "

You should get the following output (i.e. see noatime in the list in parentheses):

/dev/disk0s2 on / (hfs, local, journaled, noatime)

This tweak was taken from blogs.nullvision.com, which seems to not be available anymore, but I found a mirror of the old content here: http://www.icyte.com/saved/blogs.nullvision.com/441781
And the same trick (inspired by the same blog) is here: http://blog.philippklaus.de/2011/04/ssd-optimizations-on-mac-os-x/

Move user home directories to HDD [SSD+HDD only]

This tweak is only useful if you have both SSD as well as HDD in your Mac. I’m using this in my iMac. I moved all the content of /Users folder to the HDD and created a symbolic link from the SSD to it (so that I don’t have to change the home folder location in the user settings, as I read some applications don’t like it and may not work correctly). To do that execute the following commands in Terminal:

sudo ditto /Users /Volumes/your_hdd_name/Users
sudo mv /Users /Users.bak
sudo ln -s /Volumes/your_hdd_name/Users /Users

UPDATE (5/22/2012): To be safe, you should also go to System Preferences, click on Users & Groups, click the lock icon to unlock advanced editing (password prompt will appear). Once unlocked, you should be able to right-click on each user account and choose Advanced Options from the pop-up menu. Once in the Advanced Options dialog, change the Home directory of the user from “/Users/user-name” to the new location (e.g. “/Volumes/HDD/Users/user-name”).

Now, check if your home folders are showing up correctly in Finder. If so, restart your computer.

Finally, delete the back-up of your Users folder on the SSD by typing the following into the Terminal:

sudo rm -rf /Users.bak

Use RAM disk or HDD for temporary files

If you have enough RAM, you can dedicate (typically around 256 to 512 MB) of RAM to a RAM disk. RAM disk is a virtual disk that only resides in memory, so is suitable for storing data that need to live only until you shut down your computer. Temporary files are ideal for this. You can create a RAM disk during the boot time and redirect all the temporary files there. To do that, create a file named “MoveTempFoldersToRamDisk.sh” in your home directory and put the following content in:
Now, run the following in the Terminal:

chmod 755 ~/MoveTempFoldersToRamDisk.sh
~/MoveTempFoldersToRamDisk.sh

This creates two RAM disks on startup – one 256MB large for /private/tmp (command “RAMDisk /private/tmp 256″ in the middle of the above script) and another one 64MB large for /var/run. You can now delete ~/MoveTempFoldersToRamDisk.sh from your computer.
For the changes to take effect, you have to restart.

If you decide to undo this tweak in the future, you can do it simply by deleting /System/Library/StartupItems/RamFS directory from your Mac. E.g. by executing the following command in the Terminal:

sudo rm -rf /System/Library/StartupItems/RamFS

Again, restart is needed for this to take effect.

There are some small drawbacks to applying this tweak:

  • After applying it it takes a few seconds (2-3 on my machine) to shut down
  • It lowers the size of RAM usable for applications

If you are bothered by the above and have HDD in your Mac as well, you can consider moving the temporary files to HDD instead of the RAM disk. The steps are similar to moving the user home directories. E.g. to move /private/tmp, execute the following in the Terminal:

sudo ditto /private/tmp /Volumes/your_hdd_name/private/tmp
sudo rm -rf /private/tmp
sudo ln -s /Volumes/your_hdd_name/private/tmp /private/tmp

 
RAM disk portion of this tweak taken from here: http://blog.philippklaus.de/2011/04/ssd-optimizations-on-mac-os-x/
Originally suggested by blogs.nullvision.com (mirror at http://www.icyte.com/saved/blogs.nullvision.com/441781)

Turn off sudden motion sensor [no HDD only]

If SSD is the only drive in your Mac, there is no point in using the Sudden Motion Sensor. You can switch it off by executing the following in the Terminal:

sudo pmset -a sms 0

Taken from http://poller.se/2010/08/optimizing-mac-os-x-for-ssd-drives/

Turn off hard drive sleep [no HDD only]

Some websites mention SSD may freeze when the hard drive sleep feature is on, so it is recommended to turn it off. However, you probably don’t want to do this if you also have a HDD in your Mac. To switch the hard drive sleep off, go to System Preferences->Energy Saver and uncheck “Put the hard disk(s) to sleep when possible”.
Energy Saver preferences pane screeshot

 
Taken from http://poller.se/2010/08/optimizing-mac-os-x-for-ssd-drives/

References/Other tweaks

The tweaks that I presented are the tweaks that I thought are worth applying. None of them really limits any features. There are other tweaks, which I did not want to apply as I would be giving up on some functionality (such as disabling the Spotlight) or I was not comfortable with (e.g. disabling the swap files). You can find these and more on the following web sites:
http://poller.se/2010/08/optimizing-mac-os-x-for-ssd-drives/
http://blog.philippklaus.de/2011/04/ssd-optimizations-on-mac-os-x/
http://www.ocztechnologyforum.com/forum/showthread.php?62354

New Library for Reading and Writing Zip Files in Java

March 31st, 2012 by Martin 6 comments »

More than two years ago I posted this blog entry, where I showed a little utility for reading password-protected zip files in Java. Since then it seems it helped a lot of people and many asked if I could provide something for the other direction – i.e. writing encrypted files. Finally I’ve decided to add that as well as improve the original class for reading and recently I published a small open source project which provides both the ZipDecryptInputStream (for reading) as well as ZipEncryptOutputStream (for writing). The project is named ziputils and is available here: https://bitbucket.org/matulic/ziputils/overview.

The usage is very simple. ZipDecryptInputStream can be used in the same way as outlined in my old blog entry. ZipEncryptOutputStream which I added can be used in a very similar way:

import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;

// usage: java Main zip-file-name password filename1 [filename2 [filename3 [...]]]
public class Main {
    public static void main(String[] args) throws IOException {
        // create a stream that will encrypt the resulting zip file
        // using the password provided in the second command line argument
        ZipEncryptOutputStream zeos = new ZipEncryptOutputStream(new FileOutputStream(args[0]), args[1]);
        // create the standard zip output stream, initialize it with our encrypting stream
        ZipOutputStream zos = new ZipOutputStream(zeos);

        // write the zip file
        for (int i = 2; i < args.length; i++) {
            ZipEntry ze = new ZipEntry(args[i]);
            zos.putNextEntry(ze);
            InputStream is = new FileInputStream(args[i]);
            int b;
            while ((b = is.read()) != -1) {
                zos.write(b);
            }
            zos.closeEntry();
        }
        zos.close();
    }
}

You can find more about how to use the classes in the project javadoc. To download the ziputils jar, go to the project downloads page. Enjoy!

ooomailer 1.0 – More Flexible Out-of-office Replies

October 2nd, 2011 by Martin No comments »

In my day job I have to deal with a huge amount of e-mails each day – most of them are not urgent, but some are. When I go for vacation or on a business trip, I need to let people know I am not available and won’t respond until I get back (and provide an alternate contact for urgent matters). These days pretty much every mail server has a “vacation message” feature, that allows you to do exactly that – supports sending automated responses to all e-mails coming to your inbox, letting others know you won’t be able to read the message before a given date. However the implementation of that feature varies from server to server.

I have several e-mail accounts on several mail servers and I found myself in a situation when I needed more flexibility in the way I set up my vacation message. Being involved in several open source projects and subscribed to many open mailing lists, I needed to make sure I don’t spam those lists or users of those lists (who may not know me) with my vacation auto-replies. Even people I do want to notify don’t need to get my auto-reply to every message that reaches me – notification once in several days is good enough. And I needed to customize the vacation message for internal recipients (e.g. providing links to internal resources) and have it slightly more detailed than the auto-replies that go to external people.

So, a couple of months back, the weekend before my summer vacation I sat down and implemented a simple utility that provides me with this flexibility. Since then I used it several times, fixed a few issues, and Alexis (a colleague of mine) contributed some more improvements. At this point I am pretty confident the tool works smoothly, so decided to release version 1.0. Give it a try if you are in a similar situation – the tool and the source code are available at http://hg.alutam.com/ooomailer.

Jersey 1.9.1 Released

September 16th, 2011 by Martin No comments »

Over the past two weeks, I’ve been working with Pavel on finalizing and staging the bits for the hands-on-lab on OAuth, we are going to do at this year’s JavaOne. As part of that, I had to make a few more clean-ups in the Jersey OAuth client library, so we decided to make a branch for 1.9.1 and make those clean-ups along with some other small fixes there. Now, 2 weeks after 1.9, we released it. This is the release we’ll be using for JavaOne and although the release cycle was so short, it does have two nice additions worth highlighting.

  • Un-/marshalling collection types
    Until 1.9.1, JAXB un-/marshalling in Jersey worked only for Collection and List interfaces. I.e. if your resource method returned (or took as a parameter) Collection<Foo> or List<Foo> (where Foo was a JAXB bean), de-/serialization from/to XML/JSON would work, but if it returned LinkedList<Foo> or Set<Foo> or any other Collection subtype, it would not work. This is fixed in 1.9.1 and you can now return and retrieve any well-known interfaces that extend Collection (such as Set, Queue, etc.) and their implementations which have default public constructor.
  • PostReplaceFilter improvements
    PostReplaceFilter can be used to support clients which can’t send the full range of HTTP methods. It enables converting POST requests to other methods such as PUT or DELETE. If a POST request comes with a different method specified in X-HTTP-Method-Override header, the filter will replace POST in the request with that specified method. This has been in Jersey for a while, but only supported method overriding using the X-HTTP-Method-Override header. In 1.9.1 you can now use “_method” query parameter as well, and when overriding POST to GET the filter will convert all the form parameters to query parameters. Whether both header and query parameter are looked at by the filter (or only the header or only the query parameter) is configurable. Thanks to gk5885Fredy Nagy and Florian Hars for sharing their views and patches.
You can see the full list of changes in our changelog. For more info on Jersey see http://jersey.java.net.

Jersey and Cross-Site Request Forgery (CSRF)

September 14th, 2011 by Martin 9 comments »

About two weeks back we released Jersey 1.9. See Jakub’s blog for more info on what’s new. One thing Jakub didn’t mention is that Jersey 1.9 also includes a new server side filter for Cross Site Request Forgery prevention. I won’t go into the details on what CSRF is – please refer to the OWASP CSRF page for that. Unfortunately, the generally recommended prevention is to generate per-request or per-session tokens on the server side, which client then has to include in its subsequent requests. This is quite easy to implement and there are servlet filters for doing that, however it does require a session state to be maintained and thus is not very RESTful. I was trying to implement something that would not require a session. After some searching I found the following two papers which both suggest there is a solution which works, and is not based on sessions:

The main idea is to check the presence of a custom header (agreed-upon between the server and a client – e.g. X-CSRF or X-Requested-By) in all state-changing requests coming from the client. The value of the header does not really matter. It works, because the browser would not send custom headers unless the web page makes a request using XMLHttpRequest, which only allows requests to the same site.

So, in Jersey 1.9 we added a server-side filter which does exactly that. You can find it here: server-side CsrfProtectionFilter.java

And, to make it easy to build clients, a corresponding client filter (that attaches the custom header to all potentially state-changing requests) is there as well: client-side CsrfProtectionFilter.

This can be further extended based on the feedback – we may add a check for the Referrer header and eventually even implement the session-based solution as an available configuration option. Just let us know, if you have an opinion.

Reading Password-Protected ZIP Files in Java

October 31st, 2009 by Martin 71 comments »

On a recent “fun” project, I needed my application to be able to access password-protected zip files of a particular format. It was one of these features I thought will take me no time to implement. Anyway, to my surprise, neither JDK supports password-protected ZIP files, nor I was able to find a suitable Java open source library I could use for that purpose. So, I ended up writing the utility class on my own. I wrote an implementation of java.io.InputStream that filters the ZIP file data and turns a password-protected ZIP into an unprotected one on the fly – so the stream can be nicely chained with java.util.zip.ZipInputStream. Although the class is specifically targeted at the particular type of ZIP files I had to deal with (see the limitations below), maybe other people have to deal with the same type of files, or this class can provide a good start for others to turn it into a utility that would work with any type of ZIP (maybe I will do it myself some day – for now I don’t have time).
To implement this class I used the ZIP File Format Specification as the source of information. I also used the 7-zip project (C++) as a reference during the debugging to verify my understanding of the ZIP spec. and the CRC algorithm.
So, here is the class:

import java.io.IOException;
import java.io.InputStream;

public class ZipDecryptInputStream extends InputStream {
    private static final int[] CRC_TABLE = new int[256];
    // compute the table
    // (could also have it pre-computed - see http://snippets.dzone.com/tag/crc32)
    static {
        for (int i = 0; i < 256; i++) {
            int r = i;
            for (int j = 0; j < 8; j++) {
                if ((r & 1) == 1) {
                    r = (r >>> 1) ^ 0xedb88320;
                } else {
                    r >>>= 1;
                }
            }
            CRC_TABLE[i] = r;
        }
    }

    private static final int DECRYPT_HEADER_SIZE = 12;
    private static final int[] LFH_SIGNATURE = {0x50, 0x4b, 0x03, 0x04};

    private final InputStream delegate;
    private final String password;
    private final int keys[] = new int[3];

    private State state = State.SIGNATURE;
    private int skipBytes;
    private int compressedSize;
    private int value;
    private int valuePos;
    private int valueInc;

    public ZipDecryptInputStream(InputStream stream, String password) {
        this.delegate = stream;
        this.password = password;
    }

    @Override
    public int read() throws IOException {
        int result = delegate.read();
        if (skipBytes == 0) {
            switch (state) {
                case SIGNATURE:
                    if (result != LFH_SIGNATURE[valuePos]) {
                        state = State.TAIL;
                    } else {
                        valuePos++;
                        if (valuePos >= LFH_SIGNATURE.length) {
                            skipBytes = 2;
                            state = State.FLAGS;
                        }
                    }
                    break;
                case FLAGS:
                    if ((result & 1) == 0) {
                        throw new IllegalStateException("ZIP not password protected.");
                    }
                    if ((result & 64) == 64) {
                        throw new IllegalStateException("Strong encryption used.");
                    }
                    if ((result & 8) == 8) {
                        throw new IllegalStateException("Unsupported ZIP format.");
                    }
                    result -= 1;
                    compressedSize = 0;
                    valuePos = 0;
                    valueInc = DECRYPT_HEADER_SIZE;
                    state = State.COMPRESSED_SIZE;
                    skipBytes = 11;
                    break;
                case COMPRESSED_SIZE:
                    compressedSize += result << (8 * valuePos);
                    result -= valueInc;
                    if (result < 0) {
                        valueInc = 1;
                        result += 256;
                    } else {
                        valueInc = 0;
                    }
                    valuePos++;
                    if (valuePos > 3) {
                        valuePos = 0;
                        value = 0;
                        state = State.FN_LENGTH;
                        skipBytes = 4;
                    }
                    break;
                case FN_LENGTH:
                case EF_LENGTH:
                    value += result << 8 * valuePos;
                    if (valuePos == 1) {
                        valuePos = 0;
                        if (state == State.FN_LENGTH) {
                            state = State.EF_LENGTH;
                        } else {
                            state = State.HEADER;
                            skipBytes = value;
                        }
                    } else {
                        valuePos = 1;
                    }
                    break;
                case HEADER:
                    initKeys(password);
                    for (int i = 0; i < DECRYPT_HEADER_SIZE; i++) {
                        updateKeys((byte) (result ^ decryptByte()));
                        result = delegate.read();
                    }
                    compressedSize -= DECRYPT_HEADER_SIZE;
                    state = State.DATA;
                    // intentionally no break
                case DATA:
                    result = (result ^ decryptByte()) & 0xff;
                    updateKeys((byte) result);
                    compressedSize--;
                    if (compressedSize == 0) {
                        valuePos = 0;
                        state = State.SIGNATURE;
                    }
                    break;
                case TAIL:
                    // do nothing
            }
        } else {
            skipBytes--;
        }
        return result;
    }

    @Override
    public void close() throws IOException {
        delegate.close();
        super.close();
    }

    private void initKeys(String password) {
        keys[0] = 305419896;
        keys[1] = 591751049;
        keys[2] = 878082192;
        for (int i = 0; i < password.length(); i++) {
            updateKeys((byte) (password.charAt(i) & 0xff));
        }
    }

    private void updateKeys(byte charAt) {
        keys[0] = crc32(keys[0], charAt);
        keys[1] += keys[0] & 0xff;
        keys[1] = keys[1] * 134775813 + 1;
        keys[2] = crc32(keys[2], (byte) (keys[1] >> 24));
    }

    private byte decryptByte() {
        int temp = keys[2] | 2;
        return (byte) ((temp * (temp ^ 1)) >>> 8);
    }

    private int crc32(int oldCrc, byte charAt) {
        return ((oldCrc >>> 8) ^ CRC_TABLE[(oldCrc ^ charAt) & 0xff]);
    }

    private static enum State {
        SIGNATURE, FLAGS, COMPRESSED_SIZE, FN_LENGTH, EF_LENGTH, HEADER, DATA, TAIL
    }
}

These are the limitations:

  • Only the “Traditional PKWARE Encryption” is supported (spec. section VII)
  • Files that have the “compressed length” information at the end of the data section (rather than at the beginning) are not supported (see “general purpose bit flag”, bit 3 in section V, subsection J in the spec.)

And this is how you can use it in your code:

import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;

// usage: java Main [filename] [password]
public class Main {
    public static void main(String[] args) throws IOException {
        // password-protected zip file I need to read
        FileInputStream fis = new FileInputStream(args[0]);
        // wrap it in the decrypt stream
        ZipDecryptInputStream zdis = new ZipDecryptInputStream(fis, args[1]);
        // wrap the decrypt stream by the ZIP input stream
        ZipInputStream zis = new ZipInputStream(zdis);

        // read all the zip entries and save them as files
        ZipEntry ze;
        while ((ze = zis.getNextEntry()) != null) {
            FileOutputStream fos = new FileOutputStream(ze.getName());
            int b;
            while ((b = zis.read()) != -1) {
                fos.write(b);
            }
            fos.close();
            zis.closeEntry();
        }
        zis.close();
    }
}