BDD(behavior driven development) with easyb

hi,
just wondering is there anyone who started using easyb?
which is behavior driven development framework. if you are not familiar with BDD here is my explanation.

as you heard and practicing TDD (test driven development), (if you follow test first approach) you keep your specification up front through test case.
for example -

public void testShouldCreateAnUserWhenItHasValidData() {…}

as you can see, you are actually writing test case for behavior(specification) for your expected code.
and based on that test case you are implemented your logic in code. this how TDD works. for more explanation google IT

in BDD, this process is more simplified, for example if you look at my previous example -
public void testShouldCreateAnUserWhenItHasValidData() {…}

you can find, i have written one scenario when user object has valid data.
same test can be in different scenarios such as, when user object has no valid data. or the caller is not authorized and so on.

so to make such thing clear in java code, it requires code like the following. ie.
public void testShouldCreateAnUserWhenItHasNoValidData();
public void testShouldCreateAnUserWhenItHasValidDataButCallerIsNotPermitted();
public void testShouldCreateAnUserWhenItHasValidDataButCallerIsPermitted();

here how BDD is proposing a new approach of making this thing more fluent through a simplified test framework.
like JUnit, easyb is also another test framework, where you are writing your test context, and behavior in groovy code.

actullay the beauty is test scenario are written following the user story convention
which is similar with the ideal convention
“as an Author
i want to write book
so that user can understand me”.
you can also generate user story from the groovy code which you can’t do with JUnit.
so you don’t need to maintain separate document for maintaing user stories.

so when you are preparing user story and you can use easyb and groovy to format your user story rather than using ms word, excel or notepad text file
ie.

import com.somewherein.bdd.UserService
import com.somewherein.bdd.UserServiceImpl
import com.somewherein.bdd.Userscenario “create a new user with valid data”, {

given “an user with the valid data”, {
user = new User()
userService = new UserServiceImpl()
state = false
}

when “creating a new user”, {
state = userService.createUser(user)
}

then “returned state should be true”, {
state.shouldBe true
}

and “newly created user should be found”, {
userService.exists(user)
}
}

when i run this test it says -

Running user service story (UserServiceStory.groovy)
Scenarios run: 1, Failures: 0, Pending: 0, Time Elapsed: 0.649 sec1 behavior run with no failures

so if i ask for generating the user story – it generate the following text

1 scenario executed successfullyStory: user service

scenario create a new user with valid data
given an user with the valid data
when creating a new user
then returned state should be true
then newly created user should be found

this type of practice is very common in ruby on rails based development.
in ruby we have several options, but RSpec is the early comer who showed how cool it could be.

anyway, this is something you should work try EiD vacation, happy test first development.

easyb makes it easy, man

here is an article from javalobby
Is easyb Easy? | Javalobby

you can use it with spring framework, here is the example -
best wishes,

debugging rails internal query execution

while we were working with somewhere in… ads project we came up with some debugging and performance mesuring tool, here in my post i will describe how you can use it for yourself.

query debugging –

query debugging tool logs every executed query from active record and keep them in memory and using assisting template code it display all executed query from the active page.

also it executes query with mysql “explain” keyword. so on the same window you can see mysql query execution plan.
it helped us to track down queries which were not hitting the right index.
this is very simple trick – go through the code below -

module DebugUtil
class QueryDebug
@@QUERIES = {}
def self.add(p_query, p_report)
@@QUERIES[p_query] = p_report
end

def self.queries
q = @@QUERIES
clean
return q
end

def self.clean
@@QUERIES = {}
end
end
end

QueryDebug class keeps all executed query and their explained resultset in to the static array. so later in template QueryDebug::queries is invoked to get all executed query for the current page.

here is how we trap the query execution from active record -

if defined?(QUERY_DEBUG_ENABLED) && QUERY_DEBUG_ENABLED
ActiveRecord::ConnectionAdapters::MysqlAdapter.class_eval do
alias __existing_execute_method execute

def execute(sql, name = nil)
if sql.match(/^SELECT/i)
report = []
@connection.query(“explain #{sql}”).each do |row|
report < < row
end
DebugUtil::QueryDebug.add(sql, report)
end
__existing_execute_method(sql, name)
end
end

Object.class_eval do
def raise_during_query_debug
raise DebugUtil::QueryDebug::queries.inspect
end
end
end

you can see we have used “QUERY_DEBUG_ENABLED” constant to ensure whether this is enabled by intention.
now see how we are rendering on our template.

query debug

1. 	checked
   < %= row.join(“	“) % >

we put this code in common layout. so it renders on every page. thats all

time based cache expiry for rails action cache

rails has excellent support for caching action, page, query and so on.
rails default behavior is more than expected for most of the project. though i was looking for some time based expiry function on “caches_action” functionality. unfortunately there wasn’t anything so here is a simple trick i have used to make it work with different url and time based expiration.

i added “caches_action :recent” on my controller and added the following protected method -

protected
def fragment_cache_key(p_args)
cache_key = “cache_key_#{request.path}#{request.headers["QUERY_STRING"]}”.gsub(/=/, “”)
action_cache_key = get_from_cache(cache_key)
if action_cache_key
return action_cache_key
else
action_cache_key = Digest::MD5::hexdigest(“#{rand}#{Time.now}”)
add_to_cache(cache_key, action_cache_key, {:expiry => 1.hours})
return action_cache_key
end

end

actually i generate key and stored them inside my memcached instance with an hour expiry limit.
so when memcache invalidates my cache my action cache is also get invalidated.

so thus rails default action cache work with time limit

don’t think this is all, i suppose to cleanup the previously created cache file so i won’t get unnecessary store consumption .

semantic-repository-0.5.2: deploymet update

hi,
as some of you know, semantic repository version 0.5.2 has some problem with IOException (too many files open)
which was stopping repository to perform further search. thats what was reason to get empty search result.
after searching a while, we found the problem (which was also mentioned in lucene FAQ document).

by default bash shell allow limited files to open, since we had many index files, lucene has to open them up during performing search.
so when it exceeds the limit of 1025 files (which is default on our production environment). our application threw the following exception -
Caused by: java.io.FileNotFoundException: /var/indexes/ads-index/_gm.tvd (Too many open files)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.(RandomAccessFile.java:212)
at org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.(FSDirectory.java:506)
at org.apache.lucene.store.FSDirectory$FSIndexInput.(FSDirectory.java:536)
at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:445)
at org.apache.lucene.index.TermVectorsReader.(TermVectorsReader.java:70)

after increasing this limit to 10,000 we didn’t find such problem exists. we had to apply “ulimit -n 10000″ command on bash shell to make it works.

TODO: possible bug, probable our system is opening up several index searcher.

about semantic repositroy –

semantic repository service is intended for indexing content from different sources and maintain multi indexes for different types of content and perform different types of search. yet another solr type indexing service on top of lucene but it will gradually support content versioning and more semantic search result.

lucene based semantic-repository 0.5.2: major performance improvement now 24,000 items imported in 19 minutes

when we started using semantic repository, we had only one lucene index to make our content search able,
later we came up with another integration with one php based service aawaj.

on aawaj service they had  more than 150,000 items to index. we tried with our current release 0.5.1 to index all contents but we ended with extremely performance outage. later we released another version 0.5.2, where we added queued request handling and threw index optimization over an restful service uri – /rest/service/optimize/

here is the simple benchmark report -

version – 0.5.1 – first 100 items  ended in – 13.611 seconds.
version – 0.5.2 – first 100 items  ended in – 5.6152 seconds.

the change is really different and significant, later today we had anoter import on our repository, interestingly it took 1 hour to index 150,000 items. which was bit surprising since we were unable to do it with 0.5.1

actually we added single thread executor which keeps everything in queue and execute one by one. so we could remove  synchronized method.

here is an example code -

private final Executor mIndexTaskExecutor =

Executors.newSingleThreadExecutor();

public void addDocument(final Document pDocument) {

mIndexTaskExecutor.execute(

new Runnable() {

public void run() {

getLuceneIndexTemplate().addDocument(pDocument);

}

}

);

}

about semantic repositroy –

semantic repository service is intended for indexing content from different sources and maintain multi indexes for different types of content and perform different types of search. yet another solr type indexing service on top of lucene but it will gradually support content versioning and more semantic search result.

internet explorer expected identifier, string or number

huh, IE is one of those curses which came down to earth from the hell.
anyway if you face such problem you should check the following stuffs -

1. this guy described about the first problem
2. here is what i have faced and resolved,

i had a javascript code like this -

var element = new Element(“a”, {class: “keyboard_option_link”, href: “javascript: void(0)”});

so IE minds, because i have used “class” the fix i have used -

var element = new Element(“a”, {“class”: “keyboard_option_link”, href: “javascript: void(0)”});

this things now working fine,
best wishes,

nginx on debian box

i had a tough time to configure nginx on my debian production environment.
the recent stable release from nginx is 0.6.x but on debian repository it was 0.4.x, so i had to build it from the source and install it.

since i had an old 0.4.x instance of nginx, installation wasn’t as smooth as i was expecting. here i would try to show how i have resolved those broken issues and made my way to run nginx to reverse proxy my backend mongrel instances.

i took several attemtps to remove the existing 0.4.x instance of nginx but i failed.
i used “aptitude remove nginx” i ended with the following error -

Reading package lists… Done
Building dependency tree
Reading state information… Done
The following packages will be REMOVED:
nginx
0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
Need to get 0B of archives.
After unpacking 582kB disk space will be freed.
Do you want to continue [Y/n]?
(Reading database … 78227 files and directories currently installed.)
Removing nginx …
Stopping nginx: nginx.
Stopping nginx: invoke-rc.d: initscript nginx, action “stop” failed.
dpkg: error processing nginx (–remove):
subprocess pre-removal script returned error exit status 1
Starting nginx: nginx.
Errors were encountered while processing:
nginx
E: Sub-process /usr/bin/dpkg returned an error code (1)

though this is not my real error code but it has similarity, i took it from the following url -

http://sudhanshuraheja.com/2007/09/remove-nginx-from-ubuntu-fiesty-fawn.html

this blog author had some suggestion, but that wasn’t working for me, so i tried in different way -
i executed “sudo apt-get build-dep nginx” i found this tips from one of the blog comments
the comment author explained in this way -

“this should install everything required to build the package (compiler, headers/libs, packaging tools). Usually on a fresh install I do this to get everything required to build zope.Then issue “apt-get source nginx” (you need deb-src sources in /etc/apt/sources.list). This will download nginx sources (original tarball, diff, and uncompressed sources with patches applied). Just cd in source dir, make your modifications and use “dpkg-buildpackage -rfakeroot -b” (this requires fakeroot package). In parent directory you should get new deb files ready to install, with start/stops scripts and your patches. Just take care of package update that will surely remove your nginx version.”

if you want service script to initiate nginx on startup follow the link -
http://blog.labratz.net/articles/2006/10/03/rails-deployment-apache-lighttpd-nginx-mongrel-cluster

best wishes,

ruby dynamic factory method implementation

i was looking for some way out to implement  factory method on ruby code. where i have a VersionControl::ServiceFactory which will take different implementation as factory method. ie. VersionControl::ServiceFactory::subversion, VersionControl::ServiceFactory::git and so on.

though all these “subversion, git, perforce” methods are not predefined, these will be added while new implemention is added. my skeleton (abstract) implementation was in VersionControl::Service, so whatever implemention (svn, git, perforce, other) comes ahead it will implement method from VersionControl::Service method. since ruby doesn’t support class abstraction so this was the only way  i got in head.

so my skeleton implemention was consist of the following code in summary -

module VersionControl

# log entry class for data object
class LogEntry; end

# local repository information
class Information; end

# abstract service class.
# define service which will be exposed for a
# normal version controlling service.
class Service

# retrieve recent logs from the current directory
# required parameters - *base project path* and other options.
def logs(p_path, p_options = {}); raise "not implemented method" end

# generate diff from mentioned revision number
# or current revision number
# required parameters - *base project path* and other options.
def diff(p_path, p_options = {}); raise "not implemented method" end

# find repository information
def info(p_path, p_options = {}); raise "not implemented method" end
# check out content from version control server
def checkout(p_source, p_path, p_options = {}); raise "not implemented method" end

end

# factory class for supporting different version control implementation
class ServiceFactory; end
end

and the implementation is written in this way -

class ServiceFactory

class < < self
# add subversion factory method implementation
@@subversion_instance = SubversionService.new
def subversion
return @@subversion_instance
end
end
end

so every implementation also push it’s factory method to “VersionControl::ServiceFactory” class. so this way i have implemented dynamic factory method on ruby.

GITIdea intelliJ idea plugin for GIT

i just started using GIT, i say i love it.

since i have been using IntellIJ idea for a long while, in GIT world i felt like alone though i was using idea but not getting any idea how to live without some IDE’s version control support.

i want to compare my changes before i commit, i want to see more user friendly diff merging tool which idea is providing with other version control support.

so i found GITIDEA over here http://github.com/Fudge/gitidea/tree/master, after cloning and building this project.

i found it wasn’t working. so looking into code i found, it is executing “git” command. since my “git” executable is not under /usr/s|bin, i had to create a new symblink to make it work.

now it is working great, i can see where i have changed just by looking at the code (which bhaves like the usual svn support).

thanks fudge

problem with lucene number range filter

today i was almost stucked with lucene RangeFilter. here is description -

i have used this query – “price:[0 TO 2000]” to find all items which are bellow 2000.
but my search result was also including 12000 any clue ?

good news is i got my solution though it took bit longer to find my actual reason.

the quick answer is i have to use NumberTools.longToString(…) to index my number fields and to perform range search i have to convert my range value using NumberTools.

you can see my code here -
source view

just digging through lucene source code here is what i found -
to perform range query (which is later rewrite to range filter), it tooks lower token for example price: [0 TO 1000] here ’0′ is the lower token. it asks TermInfosReader to return a list of available terms for a specific field (such as price here).

RangeFilter iterate through each suggested term and seek for the token position in a random access segment file.

the problem is, the suggested terms are not sorted. also i didn’t find any way to sort them before filtering out.

though i index my content after sorting them properly. but it didn’t affect any result.

since lucene treats everything as string, and it uses String::compare to determine the range position, i think NumberTools, DateTools are the only way out there.

anyway, i am satisfy with what i got.
best wishes,
“…i do what i love to do…”