The Channel logo


By | Dr Pan Pantziarka 13th November 2007 19:56

Get into data with Groovy

Part 2: Object grabber

Hands on In the first part of this two-part series we looked at how Groovy provides a simple and intuitive approach to accessing MySQL. Compared to Java, Groovy is less verbose and more focused on what the developer wants to do with the database.

Additionally, things like opening and closing database connections, writing boilerplate code to handle exceptions and other house-keeping activities are hidden from the developer.

However, there's more to Groovy's database abilities than syntactic sugar sweetening Java's JDBC architecture. Having used the Sql object, let's turn to Groovy's DataSet object.

Where the Sql object uses SQL to interact with the database, the DataSet hides SQL completely, and instead grabs rows of data, each of which is stuffed directly into a map - the data structure also known as a dictionary or associative array in other languages. A map stores data as key/value pairs, and in this particular case the keys are database fields and the values are data points.

A quick example will make all of this clear, and as before we'll work with our users table from the pers database. We create a DataSet as follows:

import groovy.sql.Sql
import groovy.sql.DataSet
def sql = Sql.newInstance("jdbc:mysql://", "pan","regdev", "com.mysql.jdbc.Driver")
def ds=sql.dataSet('users')

We connect to MySQL using Sql.newInstance and then use the dataSet method to create the DataSet. The first thing to note is that instead of a SQL query we just give the name of the table, and it's the complete table that is returned. We can take a look at the data using the rows method as follows:

x.each { println it }

Putting the previous code into a file called ds.groovy and running it from the command-line gives us the following result:

["user_name":"tom", "user_id":1, "email":""] ["user_name":"dick", "user_id":2, "email":""] ["user_name":"harry", "user_id":3, "email":""] ["user_name":"george", "user_id":4, "email":""]

In other words, each row contains a map of key: value pairs, where the key is the field name and the value is the content of that field for the record.

So far so good, but how much value is there in simply being able to grab complete tables from MySQL into a Groovy data structure? Plenty.

Firstly, we can access individual columns in a very straightforward manner. Want to grab all of the user names? Try this:

x.each {println it.user_name}

How about some filtering of data? Say we want to grab only those users who have a user_id > 2. Rather than doing a SELECT WHERE query, we can use the DataSet directly:

over_2 = x.findAll { it.user_id > 2 }
over_2.each { println it.user_name }

All of this without having to requery the data. And you can chain query clauses, say you want all users with a user_id >2 and a user_name not equal to harry:

not_harry = ds.findAll { it.user_id > 2 && it.user_name != 'harry' }
not_harry.each { println it.user_name }

Of course, under the surface there's still some SQL going on and we can look at it if we want to using the sql and parameters properties of the DataSet. For example, in the case of not_harry this maps to:

println not_harry.sql
println not_harry.parameters

Which gives us:

select * from users where (user_id > ? and user_name != ?)
[2, "harry"]

Can we go further with a DataSet? So far we've restricted ourselves to querying and filtering table data. What if we want to write data back to the database? The good news is that you can add new rows of data, the bad news is that deletes or updates are not yet implemented. We'll have to hope and wait for a future release that adds this functionality.

To illustrate the add row functionality we're going to implement a common enough scenario - we're going to populate our MySQL table with data that's been dumped from a spreadsheet or another RDBMS into a CSV file. It's the kind of data manipulation task that scripting languages traditionally excel at.

The first thing to do is work out how to grab the data from the CSV file and parse it correctly. Here's an extract from our users.csv file:


As should be clear, Groovy is big on iterators, and in this case we can just grab the file and iterate over each line, splitting it on the comma character. As a test we can run the following code:

new File('users.csv').splitEachLine(',') {fields ->
  println fields[0] + " " + fields[1] + " " + fields[2]

Here each line is parsed, tokenised on the comma and bound to a variable that's called fields. The above code will cycle through each line of the file and print the different fields for us. What we want to do next is add each line of data to the database, but instead of INSERT queries we're going to use our DataSet directly:

new File('users.csv').splitEachLine(',') {fields ->
    user_name: fields[0],
    user_id: fields[2],
    email: fields[1]
ds.each { println it.user_name }

Running the above code will add the rows from the CSV file and then dump the list of user_names to verify that it's worked.

In just a few lines of code we've managed to read a file, parse it, and then add the data to a database, with minimal amounts of house-keeping code or boilerplate Java.

Of course, the fact the DataSet only works on tables and not on more complex structures (such as the result of a JOIN) means you can't get away from using SQL altogether, but Groovy makes it easy to mix and match approaches. And in the case of complex queries, it's fairly straightforward to use the Sql object to create a database view and then to use the DataSet object to access that.

In all then, Groovy offers a set of high-level objects that make database interaction a relative breeze. ®

comment icon Read 1 comment on this article alert Send corrections


Frank Jennings

What do you do? Use manual typwriters or live in a Scottish croft? Our man advises
A rusty petrol pump at an abandoned gas station. Pic by Silvia B. Jakiello via shutterstock

Trevor Pott

Among other things, Active Directory needs an overhaul
Baby looks taken aback/shocked/affronted. Photo by Shutterstock

Kat Hall

Plans for 2 million FTTP connections in next four years 'not enough'
Microsoft CEO Satya Nadella


Suit-and-tie-wearing man tries to meditate, take deep breaths in faux yoga pose. Photo by Shutterstock
Emotional intelligence, not tech skills, is the way to woo suits
League of gentlemen poster - Tubbs and Edward at the local shop. Copyright BBC
One reselling man tells his tale of woe