tag:blogger.com,1999:blog-79668192063995664152024-02-20T08:50:42.963-08:00John's Blogi'll start and see what happens...johnrellishttp://www.blogger.com/profile/13318765113024624030noreply@blogger.comBlogger11125tag:blogger.com,1999:blog-7966819206399566415.post-58110223352038082282012-06-30T08:17:00.000-07:002012-06-30T08:21:36.159-07:00GORM - False Sense of SimplicityWhile maintaining some code recently, I came across what seemed to be some particularly evil code. It was a service method that used a Grails Criteria Query to fish out some objects from the DB. These were then converted to Data Transfer Objects (DTO) to be serialized for use in a GWT client. It went something like :<br />
<br />
<br />
<pre class="brush: java">def myObjects = Domain.findAllBySomething(thing)
....
....
....
myObjects.each{obj ->
def dto = new Dto()
dto.property = obj.innerDomain.property
dto.propertyTwo = obj.innerDomain.anotherDomain.property
dtos << dto
}
</pre>
<br />
It seems fairly innocent but try telling that the database. Each call on any of the properties within obj that are themselves domains resulted in another database call. Creating the DTO above resulted in about 3 queries just to set two properties. This might not sound too bad, but imagine if myObjects contains 1,000 or so objects.... not the best strategy. The actual code I was maintaining was typically returning about 1,000 results and this resulted in about 13,000 queries running on the database. This was a user driven event so it could be happening quite a lot. This was obviously terrible for performance. Of course there is an argument to use paging and caching of results etc. but that's a different blog post. Long story short, a colleague (who I won't name for social reasons) and I were able to get the 13,000 queries down to 25 for the entire service method by just configuring the query correctly.<br />
<br />
The main point here is the difference between eager and lazy fetching. A long time ago, Grails' default fetching strategy was eager. This basically meant that when you queried for domains, using dynamic finders or criteria, your object model would return fully populated. Unfortunately this could also result in you returning the entire database in one query. Not good.<br />
<br />
Around about Grails 1.0.4, the default fetching strategy was changed to lazy. This meant that your object model returned partially populated, that is, proxy objects with only their id's, were returned for properties that were themselves domains. When you called for a property on one of these proxies, Grails would request it from the database. All this happened pretty transparently to the user.<br />
<br />
Grails provides many ways of forcing earger fetching. It can be done in the <a href="http://grails.org/doc/latest/ref/Database%20Mapping/fetch.html" target="_blank">domain mappings </a>and in the <a href="http://grails.org/doc/latest/ref/Domain%20Classes/fetchMode.html" target="_blank">domain itself by setting the fetch mode</a>. It can also be done at a query level which I am more in favour of. Setting on the domain itself is too much of a commitment in my opinion as you can never imagine all use cases at time of coding. I will illustrate setting it in dynamic finders with the following domains :<br />
<br />
<br />
<pre class="brush: java">class Customer {
String name
Account account
static constraints = {
account unique: true, nullable: false, blank: false
name nullable: false, blank: false
}
}
class Account {
static belongsTo = [owner: Customer]
Long accountNumber
static constraints = {
accountNumber min: 0l, nullable: false, unique: true
}
}
class BankingTransaction {
Long transactionNumber
BigDecimal amount
Account fromAccount
Account toAccount
static constraints = {
transactionNumber min: 0l, unique: true, nullable: false
fromAccount nullable: false, validator: {Account thisFromAccount, BankingTransaction thisTransaction ->
if (thisFromAccount?.accountNumber == thisTransaction?.toAccount?.accountNumber) {
return false//cannot have a transaction to the same account... probably
}
}
toAccount nullable: false
amount nullable: false
}
BankingTransactionDTO toBankingTransactionDTO() {
def dto = new BankingTransactionDTO()
dto.with {//probably better ways of doing this but this illustrated the example
transactionNumber = this.transactionNumber
amount = this.amount
fromCustomersName = fromAccount.owner.name
toCustomersName = toAccount.owner.name
}
return dto
}
}
</pre>
<pre class="brush: java"></pre>
So to illustrate the lazy and eager fetching I wrote some tests that were bootstrapped on a MySQL database. First we need to set up some test data and an additional Grooy Sql connection to run some status queries on our DB.<br />
<br />
<br />
<pre class="brush: java">class BankingTransactionTests {
Sql sql
SessionFactory sessionFactory
BankingTransactionService bankingTransactionService
@Before
void setUp() {
BankingTransaction.withNewSession {session ->
100.times {num ->
new Customer(name: "Customer ${num}", account: new Account(accountNumber: num)).save(failOnError: true)
}
}
BankingTransaction.withNewSession {session ->
final allCustomers = Customer.list()
def generousCustomer = allCustomers[0]
allCustomers[1..-1].eachWithIndex {customer, index ->
new BankingTransaction(fromAccount: generousCustomer.account, toAccount: customer.account, transactionNumber: index, amount: index * 100).save(failOnError: true)
}
}
sql = new Sql(sessionFactory.currentSession.connection())
sql.metaClass.getNumberOfSelects = {->
return delegate.rows('SHOW STATUS LIKE "Com_select"')[0].Value as Long
}
}
....
}
</pre>
<br />
I won't go into the details, it is just an integration test, the code is on <a href="https://github.com/johnrellis/OrmSimplicity" target="_blank">GitHub </a>anyway for the forever curious. So first off lets do a simple test that queries for all BankingTransactions.<br />
<br />
<br />
<pre class="brush: java"> @Test
void testSimpleQuery() {
//get the current number of selects in this session
def numberOfQueriesBeforeTest = sql.numberOfSelects
//carry out the query
final transaction = BankingTransaction.list()//1 query
//now calculate the number of queries
def numberOfQueriesAfterTest = sql.numberOfSelects
final numberOfSelects = numberOfQueriesAfterTest - numberOfQueriesBeforeTest
//is it what we expect??
assert numberOfSelects == 1
}
</pre>
We run the test, and huzzah, test passes. As we would expect BankingTransaction.list() is a simple select query.
<br />
<br />
OK, now lets access some properties on the first BankingTransaction<br />
<br />
<br />
<pre class="brush: java"> @Test
void testSimplePropertyAccess() {
//get the current number of selects in this session
def numberOfQueriesBeforeTest = sql.numberOfSelects
//carry out the query
final transaction = BankingTransaction.list()[0]
def fromAccountNumber = transaction.fromAccount.accountNumber
def fromAccountCustomerName = transaction.fromAccount.owner.name
//now calculate the number of queries
def numberOfQueriesAfterTest = sql.numberOfSelects
final numberOfSelects = numberOfQueriesAfterTest - numberOfQueriesBeforeTest
//is it what we expect??
assert numberOfSelects == 1
}
</pre>
<br />
Uh oh, test fails with :
<br />
<br />
<pre class="brush: java">Assertion failed:
assert numberOfSelects == 1
| |
3 false
</pre>
<br />
Looking closely at the test we can see the three queries, the first is the the call the list(), the second is the call to fromAccount and the third, the call to fromAccount.owner.<br />
<br />
OK, so, lets fix the query by setting the fetch mode to join in the query.<br />
<br />
<br />
<pre class="brush: java"> @Test
void testSimplePropertyAccess() {
//get the current number of selects in this session
def numberOfQueriesBeforeTest = sql.numberOfSelects
def queryParams = [fetch: [fromAccount: 'join', 'fromAccount.owner': 'join']]
//carry out the query
final transaction = BankingTransaction.list(queryParams)[0]
def fromAccountNumber = transaction.fromAccount.accountNumber
def fromAccountCustomerName = transaction.fromAccount.owner.name
//now calculate the number of queries
def numberOfQueriesAfterTest = sql.numberOfSelects
final numberOfSelects = numberOfQueriesAfterTest - numberOfQueriesBeforeTest
//is it what we expect??
assert numberOfSelects == 1
}
</pre>
<br />
<br />
Test passes! Sweet! This is because we told GORM to fetch the associated domains using a join. I suppose we should look at a more real work example. Suppose we have a BankingTransactionService with the following method :
<br />
<pre class="brush: java">class BankingTransactionService {
Collection retrieveTransactions() {
def transactions = BankingTransaction.list()
return transactions.collect {it.toBankingTransactionDTO()}
}
}
</pre>
<br />
<br />
With the following test :
<br />
<br />
<pre class="brush: java"> @Test
void testBankingService() {
//get the current number of selects in this session
def numberOfQueriesBeforeTest = sql.numberOfSelects
//carry out the query
def dtos = bankingTransactionService.retrieveTransactions()
//now calculate the number of queries
def numberOfQueriesAfterTest = sql.numberOfSelects
final numberOfSelects = numberOfQueriesAfterTest - numberOfQueriesBeforeTest
//is it what we expect??
assert numberOfSelects == 1
}
</pre>
<br />
<br />
We run the test, and eh... not looking good :
<br />
<br />
<pre class="brush: java">Assertion failed:
assert numberOfSelects == 1
| |
201 false
</pre>
<br />
<br />
Hmmm 201 queries, this will not please the performance junkies at all. What is happening in toBankingTransactionDTO() is that as the objects are created a query happens each time a property that is a domain is accessed. These slowly build up in the first level cache reducing the amount of queries, but I think we can do better.
<br />
<br />
Let's go all out and reduce the number of queries to 1 for this service method. It's gonna be a long road.... not really :<br />
<br />
<br />
<pre class="brush: java"> Collection retrieveTransactions() {
final queryParams = [fetch: [
fromAccount: 'join',
toAccount: 'join',
'toAccount.owner': 'join',
'fromAccount.owner': 'join']
]
def transactions = BankingTransaction.list(queryParams)
return transactions.collect {it.toBankingTransactionDTO()}
}
</pre>
<br />
<br />
Test passes. There we go, 201 queries to 1. Not really that much work, just don't take the simplicity provided by GORM for granted or you will have some angry DBA's on your tail... you have been warned!
<br />
<br />
Slán
<br />
<br />
Code on <a href="https://github.com/johnrellis/OrmSimplicity" target="_blank">GitHub</a>.johnrellishttp://www.blogger.com/profile/13318765113024624030noreply@blogger.com7tag:blogger.com,1999:blog-7966819206399566415.post-4695238686160790662011-01-24T03:34:00.000-08:002011-01-24T03:44:29.018-08:00Quick Groovy Script! Google Geocoding!Nice little script for parsing the Google Ceocoding Web Service. (Groovy 1.7.5)<div><br /><pre class="brush: java"><br />/**<br />Simple Script to parse the google geocode web service<br />http://code.google.com/apis/maps/documentation/geocoding/<br />*/<br />def url = new URL('http://maps.googleapis.com/maps/api/geocode/xml?address=1600+Amphitheatre+Parkway,+Mountain+View,+CA&sensor=true')<br />def geoCodeResult = new XmlParser().parseText(url.getText())<br />def lat = geoCodeResult.result.geometry.location.lat.text()<br />def lng = geoCodeResult.result.geometry.location.lng.text()<br />println lat<br />println lng<br /></pre></div><br /><br />Outputs :<br /><div>37.4216227</div><div>-122.0840263</div><div><br /></div><div><a href="http://maps.google.com/maps?q=Google@37.4216227,-122.0840263" target="_blank">http://maps.google.com/maps?q=Google@37.4216227,-122.0840263</a></div>johnrellishttp://www.blogger.com/profile/13318765113024624030noreply@blogger.com3tag:blogger.com,1999:blog-7966819206399566415.post-67634406256290836782010-11-22T11:55:00.000-08:002011-02-18T08:06:34.402-08:00Grails Quick Tip : External ConfigsA pretty decent bloke once said that "<span class="Apple-style-span" style="font-family: georgia, 'bookman old style', 'palatino linotype', 'book antiqua', palatino, 'trebuchet ms', helvetica, garamond, sans-serif, arial, verdana, 'avante garde', 'century gothic', 'comic sans ms', times, 'times new roman', serif; font-size: medium; border-collapse: collapse; color: rgb(51, 0, 0); ">Simplicity is the ultimate sophistication</span>".... and that is what I like about this idea; it is so damn simple and the rewards gained from it are vast. Well maybe not vast.... but there's a few. The idea I am harping on about is, external configuration files in Grails.<div><br /></div><div>Ok so, the sales pitch. You have created the worlds greatest application, (not difficult with Grails) and you are on top of the world! Then suddenly, the emails you have configured to send out every minute to remind you how great you are, have stopped! What the???? The mail configuration settings have changed???? Uh oh, we are going to have to redeploy the application?? I am in the airport! Uploading my 30MB war file is going to take hours... weeks.... DAYS! I haven't tagged the code and the war might contain untested code!!!!</div><div><br /></div><div>Relax... breathe... there is an alternative!</div><div><br /></div><div>External configuration files in Grails allow you to extract the tidbits from Config.groovy that you feel are important. Things like the mail configuration that can change at any time. Others like your datasource password which you don't really want checked into git, subversion or whatever version control you may be using!</div><div><br /></div><div>Enabling external configuration files in your Grails application is as easy as uncommenting a few lines of code in the top of your Config.groovy, the one's that look like this :</div><div><br /><pre class="brush: java"><br />// grails.config.locations = [ "classpath:${appName}-config.properties",<br />// "classpath:${appName}-config.groovy",<br />// "file:${userHome}/.grails/${appName}-config.properties",<br />// "file:${userHome}/.grails/${appName}-config.groovy"]<br /></pre><br /><br /></div><div>For this example, let's work with this simple case :<br /></div><br /><pre class="brush: java"><br />grails.config.locations = ["file:${userHome}/.grails/${appName}Config.groovy"]<br /></pre><br /><div>Here we have defined an external config that will sit in a hidden grails folder in the user's home directory. userHome is a variable added to the binding of Config.groovy at runtime and works on windows and unix based boxes alike. Similarly, appName is also a variable that is added to the binding at runtime containing the name of your Grails app.</div><div><br /></div><div>This is all well and good but how does it work in the real world??? A very real scenario is that you will want to use external Config files in production and use the regular Config.groovy during development. No one wants to navigate outside their Grails application directory during development, we are not savages after all!</div><div><br /></div><div>So..... how do we configure grails to look at an external configuration file in production and still have the flexibility of using Config.groovy on development?? Well you can use the environments closure in Config.groovy like so :</div><br /><pre class="brush: java"><br />environments {<br />production {<br /> //all my production configs are externalized<br /> grails.config.locations = ["file:${userHome}/.grails/${appName}Config.groovy"]<br />}<br />development {<br />//the dev environment is none the wiser!<br />grails {<br /> mail {<br /> host = "mail.server.com"<br /> 'default' {<br /> from = "mail@me.net"<br /> }<br /> port = 25<br /> username = "mail@me.com"<br /> password = "password"<br /> props = ["mail.smtp.auth": "true",<br /> "mail.smtp.socketFactory.port": "25",<br /> //"mail.smtp.socketFactory.class":"java.net.ssl.SSLSocketFactory",<br /> "mail.smtp.socketFactory.fallback": "false"]<br /> }<br /> }<br />}<br />test {<br /> grails.gorm.failOnError = true<br />}<br />}<br /></pre><br /><br />You can also declare settings that apply to all environments outside of this closure. This has the added bonus of reducing the 'noise' in your production config which should be kept as clean and as crisp as possible.<div><br /></div><div>Now thinking back to our imaginary scenario at the beginning, our jet set developer can now just log into the server, change the offending setting and bounce the server! (Another step would be to database the settings to prevent bouncing but this is slightly off topic.)<br /><div><br /></div><div>It is also possible to refactor your dataSource config options to your external file... something along these lines will work :</div><div><br /></div><div><br /><pre class="brush: java"><br />dataSource {<br />pooled = true<br />driverClassName = "com.mysql.jdbc.Driver"<br />username = "root"<br />password = "password"<br />url = "jdbc:mysql://localhost/pretty_db"<br />}<br /></pre><br /></div><br />Huzzah! Now if you need to change db password, you do not need to redeploy a new war!</div><div><br /></div><div>Honestly, one of the more convenient advantages of doing this is the ability to work with the logging of your application in production. This can be a pain when you first deploy to an app server on a production server away from the cushy dev environment that Grails provides!</div><div><br /></div><div>It is worth noting too that you should set your external files up with permissions of 600, the owner of the file being the user that you run tomcat or your other container with.</div><div><br /></div><div>So one simple change has got us :</div><div><ul><li>Ability to change Config options without having to build a war</li><li>Separated our Datasource options from our application... I assume the DBA's will thank you!</li><li>Tuning and changing logging has now become a lot easier</li><li>Your war can now be transfered to different servers with different file systems and all you have to do is change a config file...</li></ul><div>In these recessionary times, you cannot argue with that value! Thanks to Tim Berglund for showing how easy this was at his talk at SpringOne2Gx this year :)</div></div><div><br /></div><div>This is a great post on using environment variables to achieve this :</div><div><br /></div><div><a href="http://phatness.com/2010/03/how-to-externalize-your-grails-configuration/">http://phatness.com/2010/03/how-to-externalize-your-grails-configuration/</a></div><div><br /></div><div>UPDATE : It looks like someone has created a plugin that allows you to reload the external config without a restart (I have not used this but should be reliable :) ):</div><div><br /></div><div><a href="http://www.grails.org/plugin/reloadable-config">http://www.grails.org/plugin/reloadable-config</a></div><div><br /></div><div>Hope this saves someone the hassle of re-warring for silly reason... it is quite the pain!</div><div><br /></div><div>John :)</div>johnrellishttp://www.blogger.com/profile/13318765113024624030noreply@blogger.com10tag:blogger.com,1999:blog-7966819206399566415.post-57537946296311594422010-07-30T08:43:00.000-07:002010-07-30T09:00:30.786-07:00Grails Quick Tip : Work DirectoryYesterday I was required to work on a branch of one of our projects and came across a little predicament : The branch had different plugin versions than the trunk I was currently working on.<div><br /></div><div>Grails stores all the plugin classes, files and resources in ~/.grails, the projects are then divided out by their Grails version and project name. In this case the Grails version and of course the project names matched. I was a little peeved that I might need to blow away plugins and re-download if I was switching branches etc. What is one to do at a time like this??? Cry??? No..... just add the following line to BuildConfig.groovy under the grails-app/conf directory :</div><div><br /></div><div>grails.work.dir="work"</div><div><br /></div><div>This will simply place all your project meta-data into your project base dir /work... No need to blow away your previously downloaded plugins. Tears averted! I think this was introduced in 1.1.</div><div><br /></div><div>One cool plus is that Intellij IDEA picks up on this (as with a lot of changes in BuildConfig.groovy these days) and allows you to continue with no pesky configuration changes! Also, this folder doesn't need to be in your grails-app, it can be anywhere, just use absolute paths.</div><div><br /></div><div>Hope this help avert some crying developers.... they aren't much fun (except when Captain Morgan is the cause) </div><div><br /></div><div>:) Thanks!</div>johnrellishttp://www.blogger.com/profile/13318765113024624030noreply@blogger.com2tag:blogger.com,1999:blog-7966819206399566415.post-22606126391014729432010-05-31T09:52:00.000-07:002010-12-01T04:12:07.214-08:00Relationship Advice : Grails One-To-ManyIf you are here investigating bigamy.... you have come to the wrong place.... We are actually going to have a quick once over on One-To-Many relationships in Grails and GORM (Let's abbreviate to OTM for the sake of my fingers). Here's what we'll cover:<br /><br /><ul><li>Starting a Relationship</li><li>Living and Working With Your Relationship</li><li>Finding out more about your Relationship (or querying :) )</li><li>What if you don't belongTo someone?</li></ul><br /><b>Starting a Relationship :</b><br /><br />First of all, what is an OTM? Let's let the code do the talking:<br /><pre class="brush: java"><br />class Parent{<br /> static hasMany = [children:Child]<br /> String name<br />}<br /><br />class Child {<br /> static belongsTo = [parent:Parent]<br /> String name<br />}<br /></pre><br /><br />Here is a very simple domain model that will serve as a nice concise example. We have a Parent domain that can 'haveMany' children... in turn these children belongTo a specific parent. This is quite a 'tightly coupled' OTM. There can be a few nuances that catch people out here, which of course we will cover... (we will take a brief look at the "loosely coupled" OTM too).<br /><br />The definition of the hasMany in the Parent class is actually a Map, and it should read like [collectionName:TypeContainedInCollection], so the parent has a collection with the name 'children', that will contain instances of the Child domain class.<br /><br />Similarly in the Child class, the 'belongsTo' property is also a Map, [instanceOfOwningClass:typeOfOwningClass]. The 'belongsTo' property forms what I referred to earlier as a 'tightly coupled' OTM. This basically entails that the children instances are 'owned' by their 'parent', this in turn has some interesting side effects that we will cover now.<br /><br /><b>Living and Working With Your Relationship :</b><br /><br />OK let's create a parent with some children :<br /><pre class="brush: java"><br />def parent = new Parent(name:'Dad')<br /><br />//add a new child instance<br />parent.addToChildren(new Child(name:'son'))<br /><br />//add another child instance using a map<br />parent.addToChildren([name:'daughter'])<br /><br />//save the parent<br />parent.save(flush:true,failOnError:true)<br /></pre><br />To add a Child to a Parent we use the 'addTo' method that is provided by Grails, it is basically addTo+Name of Association. When we use these methods, GORM will wire up our domain classes and because children have a 'belongsTo' property set, calling save() on the parent will also validate and save the children... sweetness :)<br /><br />OK now... what if we need to remove some children from our relationship.... easy....<br /><pre class="brush: java"><br />//retrieve our parent<br />def parent = Parent.findByName('Dad')<br /><br />//retrieve the child instance<br />def child = Child.findByName('son')<br /><br />//remove the child<br />parent.removeFromChildren(child)<br /><br />parent.save()<br /></pre><br />Oh.... wait.... NOT EASY!!! If we run the above code we get the following Exception:<br /><br />"not-null property references a null or transient value: Child.parent"<br /><br />Yeeeeeeeeesh.... not pretty... The knee jerk reaction here might to add a nullable constraint for the parent property in the Child class... but this would be wrong. The relationship we defined has a 'belongsTo' so in this case, if we remove a child.... we simply want it to be deleted automatically at the same time. To achieve this, our Parent domain must look like:<br /><br /><pre class="brush: java"><br />class Parent{<br /> static hasMany = [children:Child]<br /> String name<br /><br /> static mapping = {<br /> children cascade: "all-delete-orphan"<br /> }<br />}<br /></pre><br />Now if we re-run the previous code, the child instance will be removed from our relationship and also deleted!<br /><br /><br /><b>Finding out more about your Relationship :</b><br /><br />If we have a Parent instance, it is rather easy to query for our children.... we simply access them as a property:<br /><br />def parent = Parent.findByName(name:'Dad', [lazy:[children:false]])<br />parent.children.each{//do something}<br /><br />However, things are not so clear cut when coming from the other direction. Suppose we have the name of a child and we want to find all the parents that have children with that name, so basically querying the relationship from the many side:<br /><pre class="brush: java"><br />def name = 'son'<br /><br />//create criteria so we can perform a criteria query<br />def c = Parent.createCriteria()<br /><br />//perform a listDistinct as querying relationships can result in duplications<br />def parents = c.listDistinct{<br /> createAlias("children", "c")<br /> eq("c.name", name)<br />}<br /></pre><br />Thanks to Robert Fletcher for blogging on this (link to follow)....<br /><br /><b>What if you don't belongTo someone? :</b><br /><br />In brief, if you define an OTM without declaring a 'belongsTo' property as we have, you're Child instances can exist on their own, without the need of being attached to a parent. One side effect of this is that the cascading save behaviour we saw before does not apply here... you will need to call save() on each domain separately.<br /><br />So there we have it.... defining, creating and querying a One-To-Many relationship in Grails... I hope this has been helpful to anyone that comes across it! Happy trails!<br /><br />Resources :<br />http://adhockery.blogspot.com/2009/06/querying-by-association-redux.html<br />http://www.grails.org/GORM+-+Defining+relationships<br /><br />Constraining across relationships:<br />http://johnrellis.blogspot.com/2009/09/grails-constraints-across-relationships.htmljohnrellishttp://www.blogger.com/profile/13318765113024624030noreply@blogger.com70tag:blogger.com,1999:blog-7966819206399566415.post-45484698987954517492010-03-08T16:27:00.000-08:002010-06-01T04:01:01.459-07:00Grails inplace plugin tip - BuildConfig.groovyHey... it's late... so lets keep this one short!<br /><br /><div>(Update: check out this <a href="http://blog.springsource.com/2010/06/01/whats-a-plugin-oriented-architecture/">article </a>too! )</div><div><br />I have taken it upon myself to get some re-usable functionality out of our companies grails apps and turn them into plugins.... I thought this was a genius idea but alas I wasn't the person to come up with it... creating grails plugins to modularize your applications is common practice these days.<br /><br />One headache I came across immediately however was developing an application that depended on a plugin that was also in flux... lets call them MyApp and my-plugin....<br /><br />I found myself going nuts.... I would come across an error or a shortcoming in my-plugin code while developing MyApp (yes... i am man enough to admit it!!) and had to change the code in my-plugin, do a grails clean.... followed by a grails compile... followed by a grails package-plugin..... ..... FOLLOWED BY ..... a grails install-plugin in MyApp directory... followed by a soul crushing 'y' to confirm installation.... HOLY KEYSTROKES BATMAN!!!!!!!! sorry...<br /><br />Then I came across a nice solution while pestering Peter Ledbrook on his blog on how to best test plugins....<br /><br />He pointed me to BuildConfig.groovy in the grails-app/conf folder of MyApp... in there I put the magic words:<br /><br /><blockquote>grails.plugin.location."my-plugin" = "../my-plugin"</blockquote><br /><br />This is basically the path to my-plugin relative to MyApp. I have both in the same workspace directory.<br /><br />By doing this you only need to package your plugin once to ensure all files are generated.<br /><br />Now if I find I need to change my-plugin while working in MyApp i simply jump to the source in another IDEA window, make the changes and then restart MyApp.... happy days!!<br /><br />One word of warning though... I had to remove my-plugin from application.properties as from what I could gather... grails didn't know this reference would then relate back to the one BuildConfig.groovy.... this resulted in my-plugin being deleted.... thank the higher beings for version control!!!<br /><br />Hmmm now I need to figure out how to do a plugin repo for our internal plugins.... another late night maybe....<br /><br />Hope this helps.... thanks to Peter for his time!</div>johnrellishttp://www.blogger.com/profile/13318765113024624030noreply@blogger.com5tag:blogger.com,1999:blog-7966819206399566415.post-2403802866495264172010-02-15T09:36:00.000-08:002010-02-15T11:21:45.796-08:00Retrieve Grails Domain Errors From the 'messageSource'As you may know, we can add certain constraints to our Grails domains to ensure the integrity of our priceless data.<br /><br />Suppose we have a simple domain:<br /><br /><pre class="brush: java"><br />class AppUser{<br />String login<br />String password<br /><br />static constraints = {<br /> login unique:'true', blank:'false'<br /> password blank:'false'<br />}<br />}<br /></pre><br /><br />Here, we have set up our domain so that our login not only needs to be a non-blank value but it also has to be unique.<br /><br />When saving a new user in something like a service, we can take the following approach:<br /><br /><pre class="brush: java"><br />if (!user.save()) {<br /> log.error "Failed to Save User"<br /> user.errors.allErrors.each {log.error it.defaultMessage} <br />}<br /></pre><br /><br />So lets just concentrate on the unique constraint. If that constraint is broken, the 'user.save()' will evaluate to false. We then query the domain for its errors using 'users.errors.allErrors' and iterate over each error using the each{} method that is provided by Groovy to all collections. (We can throw an exception or take another action after this if you wish :) )<br /><br />OK.... so after checking the log, I was a little confused as my 'log.error it.defaultMessage' was printing something cryptic like:<br /><br /><pre class="brush: java"><br />Property [{0}] of class [{1}] with value [{2}] must be unique<br /></pre><br /><br />Hmmm.... not very nice... this is where the Grails messageSource bean comes into play. The messageSource can be used to translate our cryptic error message to something more digestible..<br /><br /><pre class="brush: java"><br /><br />class MyService{<br />def messageSource // inject the messageSource<br /><br />def saveUser(AppUser user) {<br /> if (!user.save()) {<br /> log.error "Failed to Save User"<br /> user.errors.allErrors.each {log.error messageSource.getMessage(it, null)} <br /> }<br />}<br />}<br /></pre><br />Now our error message will be read from messages.properties in grails-app/i18n and look something like:<br /><pre class="brush: java"><br />Property [login] of class [AppUser] with value [john] must be unique<br /></pre><br />Huzzah.... readable messages.... (Note that in the call to 'messageSource.getMessage(it, null)', the null refers to the locale. We won't cover that here...)<br /><br />If we check our messages.properties file mentioned previous we can see where this default message came from:<br /><br /><pre class="brush: java"><br />default.not.unique.message=Property [{0}] of class [{1}] with value [{2}] must be unique<br /></pre><br /><br />The beauty is... we can provide our own message to make the error message even cleaner for the user... we just need to add the following to message.properties :<br /><br /><pre class="brush: java"><br />com.company.project.AppUser.login.unique=Login must be a unique value!<br /></pre><br /><br />Basically we use the path to our class, followed by the field then followed by the constraint.<br /><br />OK, so one final thing was bugging me.... do I need to include code in every service method to iterate over the errors and get the correct messages when validation fails, this wouldn't be very dry and rather .... boring....<br /><br />Hmmm how could I solve this one???? I know... metaprogramming.... c'mon.. stay with me!!!<br /><br />The aim: add a method to all my domain classes that would 'collect' all the messages for each error and return them in a readable format. The best place to do this is probably the BootStrap class. So lets give it a try:<br /><br /><pre class="brush: java"><br />class BootStrap{<br />def messageSource //our old friend<br />def grailsApplication//even older friend :)<br /><br />def init = {servletContext -><br /> grailsApplication.domainClasses.each {domainClass ->//iterate over the domainClasses<br /> if (domainClass.clazz.name.contains("com.company.project")) {//only add it to the domains in my plugin<br /><br /> domainClass.metaClass.retrieveErrors = {<br /> def errorString = delegate?.errors?.allErrors?.collect{messageSource.getMessage(it,null)}?.join(' \n')<br /><br /> return errorString<br /> }<br /> }<br /> }<br />}//end of init<br />}<br /></pre><br /><br /><br />Phew... Taking it one step at a time, we first iterate over the applications domain classes. A check is performed to ensure that we only add the method to our own domains (optional) and the method retrieveErrors() is added to the meta class of each of our domains. We then have a fancy (or egotistical) one liner that collects all the errors of the delegate domain, evaluates all their error messages into a List and joins them with a line break. Let's break up the one liner some what:<br /><br /><pre class="brush: java"><br />def init = {servletContext -><br /> grailsApplication.domainClasses.each {domainClass ->//iterate over the domainClasses<br /> if (domainClass.clazz.name.contains("com.company.project")) {//only add it to the domains in my plugin<br /><br /> domainClass.metaClass.retrieveErrors = {<br /> def list = delegate?.errors?.allErrors?.collect{messageSource.getMessage(it,null)}<br /> return list?.join('\n')<br /> }<br /> }<br /> }<br />}//end of init<br /></pre><br /><br /><br />In the end we have replaced :<br /><pre class="brush: java"><br />if (!user.save()) {<br /> log.error "Failed to Save User"<br /> user.errors.allErrors.each {log.error messageSource.getMessage(it, null)}<br /></pre><br />with:<br /><br /><pre class="brush: java"><br />if (!user.save()) log.error "Failed to save user : ${user.retrieveErrors())}"<br /></pre><br /><br />One final tidbit.... If you find yourself needing the messageSource outside of a Grails artefact... you can try the following:<br /><br /><pre class="brush: java"><br />MessageSource messageSource = ApplicationHolder.application.mainContext.getBean('messageSource')<br /></pre><br /><br />The import requred is 'import org.springframework.context.MessageSource'<br /><br />Hope this helps!<br /><br />Info:<br /><br />http://grails.org/doc/latest/ref/Constraints/validator.html<br />http://grails.org/doc/latest/ref/Constraints/Usage.htmljohnrellishttp://www.blogger.com/profile/13318765113024624030noreply@blogger.com26tag:blogger.com,1999:blog-7966819206399566415.post-12908507992476651422009-11-07T15:36:00.000-08:002009-11-07T15:59:11.530-08:00Grails and JSON are pretty Groovy!Once again the incredible productivity boosts provided by Grails have become glaringly obvious.... what is it this time???? Dealing with JSON!!<br /><br />In my project I needed to save domain info to files and these domains contained domains so I thought the easiest way would be XML and JSON.... having done some work in XML in Grails I decided to see how I fared in JSON.<br /><br />Here's what happened:<br /><br /><pre class="brush: java"><br />import grails.converters.*<br /><br />def report = MyDomain.get(1)<br />def json = ""<br />JSON.use("deep") {<br />def converter = report as JSON<br />converter.prettyPrint = true<br />json = converter.toString()<br />}<br /><br /></pre><br /><br />First off I grab a domain instance and then convert it to JSON using grails.converters.JSON. Note that grails.converters.deep is deprecated so to render your domain "deeply" you must use the configuration name "deep". I also grab the converter so I can alter some settings, specifically prettyPrint. This will render your domain in nicely formatted JSON!<br /><br />So now I have a JSON string, how can I access its properties.... do I have to parse it by hand???? Fat chance...<br /><br /><pre class="brush: java"><br /><br />def jsonArray = JSON.parse(json)<br /><br />println jsonArray.myDomainProperty<br /><br /></pre><br /><br />So now we use JSON.parse() to convert our json String to an object that allows us to access our domain properties... sweet!<br /><br />So there we have it, domain to json and back to an object that we could, if we wanted to, convert back to a domain...<br /><br />Hope this saves people some time. Parsing and creating JSON???? Let Grails do the work...... again! :)johnrellishttp://www.blogger.com/profile/13318765113024624030noreply@blogger.com3tag:blogger.com,1999:blog-7966819206399566415.post-52254424723091286942009-09-30T07:28:00.000-07:002009-09-30T08:54:33.093-07:00Grails Constraints Across RelationshipsHoly Moley am I glad to see the back of this problem!<br /><br />This blog post is a summary of an issue I raised on the Grails User mailing list, <a href="http://www.nabble.com/Transient-Error-While-Saving-Parent-in-One-To-Many-td25668692.html">here</a>, and let me first give thanks to <a href="http://www.nabble.com/user/UserProfile.jtp?user=1304370">Dave Klein</a> and <a href="http://www.nabble.com/user/UserProfile.jtp?user=65108">Daniel Honig</a> who really helped in figuring out this issue and explaining it to me, even though the time difference meant I could not stay awake any longer :)<br /><br />OK so, suppose we have the following domains:<br /><br /><pre class="brush: java"><br />class Parent {<br /><br /> static hasMany = [children: Child]<br /><br /> String name;<br /><br /> static constraints = {<br /> }<br />}<br /><br /></pre><br /><br />and ...<br /><br /><pre class="brush: java"><br /><br />class Child {<br /> static belongsTo = [parent: Parent]<br /><br /> String name<br /><br /> static constraints = {<br /> name nullable: false, blank: false<br /> }<br />}<br /><br /></pre><br /><br />Here we can see typical one to many relationship which allows me (you) to do the following in say, a Grails Service:<br /><br /><pre class="brush: java"><br /><br />def parent = new Parent(name:"Father")<br />def billy= new Child(name:"Billy")<br />parent.addToChidren(billy)<br />parent.save()<br /><br /></pre><br /><br />So what happens here? We create a Parent, add a child to it, call save() on the parent and because the child "belongsTo" the parent, the child instance is saved too. That's just good parenting in my opinion!<br /><br />OK so, our Parent doesn't want to be confused and have 2 children with the same name, nothing wrong with this, it's just this parents preference! So the knee-jerk reaction is to add a "unique:true" constraint to the Child domain.... hmmm wait a minute, if we do this, this means that we can only have one child named "Billy" in the entire world, hmmm that wont work. So thinking about it again, we want the Child's name to be unique only for a specific parent, so maybe we do something like "unique:'parent'" on the name property of the Child domain. If we do this, our Child domain becomes:<br /><br /><pre class="brush: java"><br />class Child {<br /> static belongsTo = [parent: Parent]<br /><br /> String name<br /><br /> static constraints = {<br /> name nullable: false, blank: false, unique: 'parent'<br /> }<br />}<br /></pre><br /><br />OK so, now lets start afresh:<br /><br /><pre class="brush: java"><br />def parent = new Parent(name:"Father")<br />def billy= new Child(name:"Billy")<br />def robert= new Child(name:"Robert")<br />parent.addToChidren(billy)<br />parent.addToChidren(robert)<br />parent.save()<br /></pre><br /><br />Uh oh, something has gone wrong, I am now getting this madness when i try to save(): "InvalidDataAccessApiUsageException: object references an unsaved transient instance - save the transient instance before flushing: Parent".... Hmmm, something is not quite right here.<br /><br />After some very good debugging from the aforementioned Groovynauts, it was discovered that Grails needed the Parent to be persisted before proper validation could take place on the children. This is only a minor annoyance if we wrap the code in a withTransaction{} block, why you ask? If we use a withTransaction block we can save our Parent confident in the fact that if any constraints fail on our children (or something else goes wrong) we will not have Parents in our database missing their children. Grails will roll back and remove the Parent from the database for us. So here it is (place this in a service method for instance):<br /><br /><pre class="brush: java"><br />Parent.withTransaction{<br />def parent = new Parent(name:"Father")<br />parent.save(flush:true)<br /><br />def billy= new Child(name:"Billy")<br />def robert= new Child(name:"Robert")<br />parent.addToChidren(billy)<br />parent.addToChidren(robert)<br />parent.save(flush:true)<br />}<br /></pre><br /><br />And there we have it, no parent can have multiple children that have the same name! For more info on how this was all figured out, check the link to the user list above, also check <a href="http://grails.org/doc/latest/ref/Constraints/unique.html">here </a>for how to do something like this in a single domain and not across relationships.<br /><br />Oh and if you do attempt to add children with the same, a "org.springframework.dao.DataIntegrityViolationException" is thrown!<br /><br />Thanks again to the grails community! I hope someone finds this blog useful and doesn't lose as may hours (days?) as i did :)<br /><br />Oh and one more thing, if you add unique:'parent' after you have done some dev work, i.e. your database is already built, you may need to rebuild the database as MySql did not update the indexes for me, might save some more confusion!johnrellishttp://www.blogger.com/profile/13318765113024624030noreply@blogger.com6tag:blogger.com,1999:blog-7966819206399566415.post-52903924477261400742009-08-04T10:53:00.000-07:002009-08-07T04:45:30.024-07:00Pretty Printing Groovy's StreamingMarkupBuilderUnfortunately, Groovy's StreamingMarkupBuilder will not output indented xml due to its streaming nature. However, never fear, JTidy is here to save the day!<br /><br />JTidy is a project hosted <a href="http://jtidy.sourceforge.net/">here</a> on sourceforge and it is a pretty nice HTML/XML pretty printer. It also seems to check your HTML for errors, which is cool! Examples are lacking however so I thought I would share my findings with this example:<br /><br /><pre class="brush: java"><br />public class JTidyExample {<br /><br />def tidyMeUp(String singleLine) {<br />StringWriter writer = new StringWriter()<br />Tidy tidy = new Tidy()<br />tidy.identity {<br /> setEscapeCdata(false)//leave cdata untouched<br /> setIndentCdata(true)//indent the CData<br /> setXmlTags(true)//working with xml not html<br /> parse(new StringReader(singleLine), writer)<br />}<br />writer.toString()<br />}<br /><br />def createMarkUp() {<br />String cData = "<![CDATA[hello]]>"<br />StreamingMarkupBuilder xml = new StreamingMarkupBuilder();<br />def person1 = {<br /> person(id: 1) {<br /> firstName("John")<br /> lastName("Doe")<br /> data_labels {<br /> mkp.yieldUnescaped(cData)<br /> }<br /> }<br />}<br />def personList = {<br /> people {<br /> out << person1<br /> }<br />}<br />xml.bind(personList).toString()<br />}<br /><br />def static main(def args) {<br />def example = new JTidyExample()<br />def singleLine = example.createMarkUp()<br />println "Before: \n ${singleLine}"<br />println "After: \n ${example.tidyMeUp(singleLine)}"<br />}<br /><br />}<br /><br /></pre><br /><br />The following is output:<br /><br /><pre class="brush: java"><br />Before:<br /><br /><people><person id='1'><firstName>John</firstName><lastName>Doe</lastName><data_labels><![CDATA[hello]]></data_labels></person></people><br /><br />Tidy (vers 26-Sep-2004) Parsing "InputStream"<br />no warnings or errors were found<br /><br />After:<br /><br /><people><br /><person id='1'><br /><firstName>John</firstName><br /><lastName>Doe</lastName><br /><data_labels><br /> <![CDATA[hello]]><br /></data_labels><br /></person><br /></people><br /><br /></pre><br /><br />Here we can see that our XML is now nicely indented. Want some more??? Check out Scott Davis'<a href="http://www.amazon.co.uk/Groovy-Recipes-Greasing-Pragmatic-Programmers/dp/0978739299/ref=sr_1_2?ie=UTF8&qid=1249409501&sr=8-2"> Groovy Recipes</a>, it has a pretty sweet XML section!<br /><br />UPDATE: Check out Paul Kings comments where he suggests using groovy.xml.XmlUtil.serialize(xml.bind(personList)). Sweet!johnrellishttp://www.blogger.com/profile/13318765113024624030noreply@blogger.com6tag:blogger.com,1999:blog-7966819206399566415.post-1945083483416959102009-07-24T05:08:00.000-07:002009-08-04T10:52:48.040-07:00Grails Transactions In BriefOK so first ever blog post, lets keep things short!<br /><br />I was playing around with transactions in Grails as part of a project a while ago, bearing in mind I have no Spring experience, I was a little worried it might be complicated. Need I worry??? Lord no! Grails to the rescue once again, however there is one 'Gotcha' that I feel is worth putting out there, hence me writing this 'blog'.<br /><br />Basically transactions are a facet of Grails (obtained from Spring I think) that allow us to roll back database writes if an exception occurs mid-method making our databases cleaner than.... hmm.. don't want to write an offensive metaphor in my first post so lets leave it at clean :)<br /><br />Suppose we have the following method in our service:<br /> <pre class="brush: java"><br />static transactional = true<br /><br /> def serviceMethod(params1,params2){<br /><br /> def domain1 = new Domain()<br /> domain1.save()<br /><br /> def domain2 = new Domain()<br /> domain2.save()//Exception thrown here<br />}<br /><br /><br /> </pre><br /><br />So because this is transactional, when the exception is thrown while saving domain2, domain1 should be deleted from the database (well that was my assumption). However this is NOT the case. After some reading, it seems that the roll back will only occur if the exception thrown is of type <b>RuntimeException.</b> This means that we would have to catch an exception and then throw a RuntimeException, which I think makes this sort of transactional behavior pointless... if we know where exceptions are going to occur we could just roll back ourselves maybe.... I'm sure there's a reason :)<br /><br />However, all is not lost!! We can take finer grained control of transactions using the Domain.withTransaction{} closure where Domain is the name of a domain in our project. To use this, we must set the service's transactional property to <b>false</b>. Here's how it works:<br /><br /> <pre class="brush: java"><br />static transactional = false<br /><br />def serviceMethod(params1,params2){<br /><br />Domain.withTransaction{tx -><br /><br /> def domain1 = new Domain()<br /> domain1.save()<br /><br /> def domain2 = new Domain()<br /> domain2.save()//<i>Exception thrown here</i><br /><br /> }<br /><br />}<br /></pre><br /><br />Now when any exception is thrown at domain2.save() (probably before it), the transaction is rolled back and domain one is deleted. We can also roll back programmically by using tx.setRollbackOnly(). Actually saying domain one is deleted probably isn't correct because they are never persisted, just saved to the session... i think!<br /><br />Anyways, I hope this helps some people new to dealing with transactions in Grails. Feedback is welcome and also if anyone knows how to make my source code indent properly, by all means share :)<br /><br />Johnjohnrellishttp://www.blogger.com/profile/13318765113024624030noreply@blogger.com4