Using Play with MongoDB

Welcome to the Play Framework/MongoDB playground

This site tries to give some additional support to the existing documentation, in case you want to use the MongoDB module with the playframework. Before going on here, you should already have read the excellent documentation, which is already available for the mongodb module.

Installing the most current mongodb module

If some of the features explained here do not work out of the box, it is very possible, that these are only running with my development tree of the mongodb module. I try to merge back my changes back with Andrew (the mongo module maintainer) whenever I have the feel that it is stable. If you do not want to wait that long, you will have to build the module manually. Actually this is quite easy:
cd $playDir/modules
bzr branch lp:~alr/play-mongo/embedded mongo-alr
cd mongo-alr
ant

Changelog

  • 2010 06 06: Added a @MongoField annotation, allowing you to freely choose the representation of an object in the db. Added findById(String id) and findById(ObjectId id) method to the MongoModel class (thanks to Brian Nesbitt for the patch). Removed all jackson library dependencies (which were used for the first version) and some not needed classes using it. Moved the annotations into its own package, thus breaking compatibility for a cleaner design. Also started a first test suite, which will be ready soon I hope and have several dozen unittests.
  • 2010 06 01: Added @MongoTransient annotation, SortedSets are handled correctly
  • 2010 05 24: Fixed a bug an the embedded branch preventing numbers from being used in the geospatial example

About

If you find bugs or useful hints, which you think should be placed here, contact me any time. If you have commercial offers for me to work on play and/or the mongo module in Germany, especially in and around Munich, feel free to contact me at rednaxela @ neesler . ten - you should work out the anti spam measurement, or contact me via xing. If you are interested in play or mongodb, feel free to ask me to give a talk, I will be glad to do so.

Setting up the module

The module setup is actually pretty easy. Either you install the module in your application via command line
play install mongo
or you build the current version as described in the Introduction and easily enable it in your application.conf file via
module.crud=${play.path}/modules/mongo-alr
Remember from the introduction, that the directory was renamed in order to not clash with the upstream mongo module.

Specifying the mongo server and database

After installing the module you need to specify the server, port and database to connect to.
mongo.host=localhost
mongo.port=27017
mongo.database=places
For further options like username and password, please read the excellent already provided documentation.

Working with Model classes

If you have already worked with play, you should be used working with the JPAModel based classes. The mongo module tries to resemble this behaviour wherever possible in order to have low switching costs between technologies. However as mongo is not SQL there are changes, especially when querying for data and when joining different data.

A simple model class

@MongoEntity("places")
public class Place extends MongoModel {

	public Place (String name, Double latitude, Double longitude) {
		this.name = name;
		this.location = new Location(latitude, longitude);
	}

	@Required
	public String name;
	
	@MongoEmbedded
	public Location location;
	
	public Double rating = new Double(0);
}
As you can easily see, every Place instance is stored in the places collection and contains a name, a location and a rating. The location refers to another object, but the @MongoEmbedded annotation forces the object to embedded into the place instead of being joined like it would be done in SQL. The required annotation is a play specific annotation, which can be used the validate the object in the controler.

Search by timestamp

In SQL searching by timestamp is pretty simple as there is a built-in type in SQL standard. In MongoDB it is also simple, like this.
@MongoEntity("posts")
public class Post extends MongoModel {

	public Date postedAt;
	...
	
	public Post previous() {
		return Post.find("byPostedAt", new BasicDBObject("$lt", postedAt)).order("by-PostedAt").first();
	}

	public Post next() {
		return Post.find("byPostedAt", new BasicDBObject("$gt", postedAt)).first();
	}

}
Using the "greater than" and "lower than" operators in conjunction with the posted date helps you to find the all the posts which fulfill this requirement. Therefore you need to change the order in the first case, otherwise you would get the oldest post first.

Searching by several criterias

Imagine you have a Post class, where every post is tagged with an arbitrary amount of tags. Now searching for all posts which are tagged "play" and "mongodb" is pretty easy
@MongoEntity("posts")
public class Post extends MongoModel {

	...
	@MongoEmbedded
	public Set tags;

	public static List findTaggedWith(String... tags) {
		// Equivalent mongodb query
		// db.posts.find({ tags : { $all : [ { name : "tagThree" }, { name : "cool" } ] } } ) 
		BasicDBList tagList = new BasicDBList();
		for (String tag : tags) {
			tagList.add(new BasicDBObject("name", tag));
		}
		DBObject allMatchObj = new BasicDBObject("$all", tagList);
		return Post.find("byTags", allMatchObj).fetch();
    	}
}
Now calling Post.findTaggedWith("play", "mongodb") will yield your expected result

Mongos geospatial feature

Please use the development branch for this
Recommended read before diving into play specifics is the MongoDB document about Geospatial Indexing.

Adding a geospatial index

In order to query for geospatial information, you have to put a geospatial index on the columns. When using play you can verify this on start up of the application.
@OnApplicationStart
public class EnsureGeoMongoIndex extends Job {

	@Override
	public void doJob() {
		DB db = MongoDB.db();
		DBCollection coll = db.getCollection("places");
		coll.ensureIndex(new BasicDBObject("location", "2d"));
	}
}

Using geospatial queries

Consider the following two entities.
@MongoEntity("locations")
public class Location extends MongoModel {

	public Location(Double latitude, Double longitude) {
		this.latitude = latitude;
		this.longitude = longitude;
	}
	
	public Double latitude;
	public Double longitude;
	
}
@MongoEntity("places")
public class Place extends MongoModel {

	public Place (String name, Double latitude, Double longitude) {
		this.name = name;
		this.location = new Location(latitude, longitude);
	}

	@Required
	public String name;
	
	@MongoEmbedded
	public Location location;
	
	public Double rating = new Double(0);
	
	
	public String toString() {
		return name + "@" + location.latitude + "/" + location.longitude;
	}
	
	public static List findNearBy(Double latitude, Double longitude, int limit) {
		// db.places.find( { loc : { $near : [50,50] } } )
		DBObject coordinateObj = new BasicDBObject();
		coordinateObj.put("latitude", latitude);
		coordinateObj.put("longitude", longitude);
		DBObject nearByObj = new BasicDBObject("$near", coordinateObj);
		// Ignore the first, this is your own place, it has always 0 distance
		return Place.find("byLocation", nearByObj).from(1).fetch(limit);
	}

	
	public List findNearBy() {
		return Place.findNearBy(location.latitude, location.longitude, 10);
	}

	public List findNearBy(int limit) {
		return Place.findNearBy(location.latitude, location.longitude, limit);
	}

	
}
The Place class includes all the necessary code for searching nearby places. Imagine the following controller code in order to search for a place based on its name
    public static void search(String q) {
    	notFoundIfNull(q);
    	Pattern pattern = Pattern.compile(q, Pattern.CASE_INSENSITIVE);
    	List places = Place.find("byName", pattern).fetch();
    	render(places);
    }
This page returns a list of places depending on the entered query. Clicking on a place in the result set might lead you to the following controller:
    public static void showPlace(String id) {
    	Place place = Place.find("by_id", id).first();
    	List nearbyPlaces = place.findNearBy(5);
    	render(nearbyPlaces, place);
    }
The template might look like this:
<ul>
#{list nearbyPlaces, as:'nearPlace'}
<li>#{a @Application.showplace(nearPlace._id)}${nearPlace.name}#{/a}, 
  distance is ${nearPlace.distanceTo(place).format("#.##")} km</li>
#{/list}
</ul>
The distanceTo method is a extension method, which calculates the distance to another place. Ideally this could also go into the model class, as it is very likely business logic, but for the sake of completeness it is just a extension, which calculates the distance in kilometres.
public class PlacesExtensions extends JavaExtensions {

	public static Double distanceTo(Place fromPlace, Place toPlace) {
		return new Double(distance(fromPlace.location.latitude, fromPlace.location.longitude,
				toPlace.location.latitude, toPlace.location.longitude));
	}


	private static double distance(double lat1, double lon1, double lat2, double lon2) {
		double theta = lon1 - lon2;
		double dist = Math.sin(deg2rad(lat1)) * Math.sin(deg2rad(lat2)) + Math.cos(deg2rad(lat1)) * 
			Math.cos(deg2rad(lat2)) * Math.cos(deg2rad(theta));
		
		dist = Math.acos(dist);
		dist = rad2deg(dist) * 60 * 1.1515 * 1.609344;
		
		return dist;
	}

	private static double deg2rad(double deg) {
		return (deg * Math.PI / 180.0);
	}

	private static double rad2deg(double rad) {
		return (rad * 180.0 / Math.PI);
	}

}

Map/Reduce in mongo

If you do not know what Map/Reduce is you might want to check Wikipedia first. If you already know, but cannot see the direct link to mongodb, remember mongos data structure. As mongo distributes its data and as there is no referential integrity, some ways must be found to efficiently aggregate data. At this point mongodb and map reduce cross its ways. For more information about it, feel free to read the mongo docs

A simple map reduce example

The following lines will introduce an absolute basic Map/Reduce example using play and mongo. Though Map/Reduce is offered by the java driver it is basically just a thin wrapper around the javascript based map/reduce mechanism in mongo. This means unfortunately to have some javascript code in the source. The example here creates a simple tag cloud from all tags. The play classes look like this:
@MongoEntity("tags")
public class Tag extends MongoModel {

	@Required
	public String name;

}
And a post class
@MongoEntity("posts")
public class Post extends MongoModel {
	...
	@MongoEmbedded
	public Set tags;
	...
}
This implies a basic problem - every post has its own set of tags. So this is not the SQL world, where a Tag table can just be counted. You need to step through every post and count each occurence of a tag. Extending the tag class is the solution
@MongoEntity("tags")
public class Tag extends MongoModel {

	@Required
	public String name;

	/*
	 * Map function: 
	 * function () { this.tags.forEach(function (z) {emit(z.name, {count:1});}); }
	 *
	 * Reduce function:
	 * r = function(key, values){ var total = 0; for ( var i=0; i < values.length; i++ ) { 
	 *           total += values[i].count } ; return { count : total }; }
	 * 
	 * Execute:
	 * res = db.posts.mapReduce(m, r)
	 * 
	 * Show data:
	 * db[res.result].find().sort({ _id : 1 })
	 * 
	 * Drop data:
	 * db[res.result].drop()
     */
	public static List getCloud() {
		String mapFunction = "function () { this.tags.forEach(function (z) {emit(z.name, {count:1});}); }";
		String reduceFunction = "function(key, values){ var total = 0; for ( var i=0;" +
			"i < values.length; i++ ) total += values[i].count; return { count : total }; }";
    	
		MapReduceOutput mapReduceOutput = MongoDB.db().getCollection(Post.getCollectionName())
			.mapReduce(mapFunction, reduceFunction, null, null);
    	
		List resultList = new ArrayList();
		DBCursor cursor = mapReduceOutput.getOutputCollection().find();
		for (Iterator iterator = cursor.iterator(); iterator.hasNext();) {
			DBObject dbObj = (DBObject) iterator.next();
			Map map = new HashMap(2);
			map.put("tag", dbObj.get("name"));
			DBObject value = (DBObject) dbObj.get("value");
			map.put("pound", value.get("count"));
			resultList.add(map);
		}
    	
		mapReduceOutput.drop();
    	
		return resultList;
	}
}
This looks a little bit confusing at first but is actually quite straight forward. I put the javascript functions in the comments for easier readability. The map function walks through the tags of every entry of a collection and emits its name and a count of 1. The reduce function takes for every key (which was the name of the tag in the emit function) its values array, which always contains only count : 1, sums this up and returns it. The MapReduceOutput class calls both functions on the Post collection and writes the output into a temporary collection. This collection is iterated and the data put into the list (as a hashmap per entry - it is chosen this way to retain compatibility with the yabe example). After getting the results the collection is dropped again.

Lots of stuff still missing

MongoDB offers nearly endless possibilities. So does the playframework. It is actually pretty easy to mix up these two - however there are still some things missing. Some are listed here along with hints, how it could be implemented. Feel free to built your own branch on launchpad and add any feature - the upstream is pretty responsive and very open for help.

MongoFixtures

Unfortunately the Fixtures class in the playframework uses hibernate specific code, so it cannot be used for the mongo module. Adding such a class with a similar functionality would help testability of the module and especially applications based on it a lot.

GridFS support

MongoDB offers excellent support for distributed file storage via its GridFS support. Incorporating this with play - especially for file uploads would be a very nice feature. It should be pretty simple. Just store any field which is a java.io.File inside of mongo.

Implementing some CRUD module

The playframework CRUD module lacks the problem as the Fixtures class, it has a hard dependency on hibernate models. I am not yet really sure on how to build this, but it would be a very good idea.

Test suite

In progress
A test suite - which is basically just a play app - has to be made available which proves all module features usable. This should be a first class goal.

Lazy loading

Implementing a nice lazy loading is quite important. Adding a @MongoLazy annotation to a field should mark it as loaded only when the setter is called. I think this could be done via Jboss AOP or even the javassist package. In any case it would be quite cool - as big data would not to be loaded when it is not needed.

Using JBoss AOP instead of javassist

Using javassist in the byte code enhancer resembles the big problem of using plain text to overwrite the methods with its real functionality. While it is of course possible and useful to cover these methods via unittests, I like the apporach of jboss aop far more where you can write all your aop related code in pure java - with the use of annotations. I have yet to check whether jboss aop would cope with play.

Implementing a @MongoTransient annotation

Incorporated into embedded tree
This is an absolute easy one. Like in JPA there should also exist a Transient annotation which holds values, which are not stored in the database.
public class Author extends MongoModel {
	...
	@MongoTransient
	public String transientString;	
	...
}

Implementing field name annotations

Incorporated into embedded tree
As mongo stores data as a document, its size always grows with big column names. So decoupling the column names from the names used in java would be really helpful. This could be easily added via annotations, like the one below.
public class Location extends MongoModel {
	@MongoField("long")
	public Double longitude;
	
	@MongoField("lat")
	public Double latitude;
}

Download examples

This section features downloadable play applications in order give you some hands on experience with the modules.
  • Yabe using mongodb as backend. Uses some queries and Map/Reduce. Download
  • BYOQ (Build your own qype). Uses the geospatial indexes and shows a bing map with markers. Also features a small rating calculator job. Download