I was recently working on a simple application where the user will enter famous quotations.  Obviously we want to avoid duplicates so I needed a way to check for quotations that were substantially similar before a new quote was added to the database.

The idea was to show the top 5 most similar quotes before letting the user save the new quotation to the db. I used Lucene for this which allowed me to punt on the more difficult task of figuring out if two quotes were similar or not. I left that up to Lucene and only had to worry about how to get my information in and out of Lucene in a usable manner.

Below is the interesting method that uses Lucene to build an index of all the quotes in the system and then returns the five quotes that are most similar to the new quote text.  Obviously creating a new index each time a quote is added isn’t particularly efficient, but makes it easier to demonstrate how it works and processor efficiency isn’t much of an issue with this particular task.

    public List<Quote> getSimilarQuotes() throws CorruptIndexException, IOException {

        String quoteText = quote.getText();
        logger.info("creating RAMDirectory");
        RAMDirectory idx = new RAMDirectory();
        IndexWriterConfig indexWriterConfig = new IndexWriterConfig(Version.LUCENE_31, new StandardAnalyzer(Version.LUCENE_31));
        IndexWriter writer = new IndexWriter(idx, indexWriterConfig);

        List<Quote> quotes =  session.createCriteria(Quote.class).list();

        //Create a Lucene document for each quote and add them to the
        //RAMDirectory Index.  We include the db id so we can retrive the
        //similar quotes before returning them to the client.
        for (Quote quote : quotes) {
            Document doc = new Document();
            doc.add(new Field("contents", quote.getText(),Field.Store.YES, Field.Index.ANALYZED));
            doc.add(new Field("id", quote.getId().toString() ,Field.Store.YES, Field.Index.ANALYZED));
            writer.addDocument(doc);
        }

        //We are done writing documents to the index at this point
        writer.close();

        //Open the index
        IndexReader ir = IndexReader.open(idx);
        logger.info("ir has " + ir.numDocs() + " docs in it");
        IndexSearcher is = new IndexSearcher(idx, true);

        MoreLikeThis mlt = new MoreLikeThis(ir);

 		//lower some settings to MoreLikeThis will work with very short
        //quotations
        mlt.setMinTermFreq(1);
        mlt.setMinDocFreq(1);

		//We need a Reader to create the Query so we'll create one
        //using the string quoteText.
        Reader reader = new StringReader(quoteText);

        //Create the query that we can then use to search the index
        Query query = mlt.like( reader);

        //Search the index using the query and get the top 5 results
        TopDocs topDocs = is.search(query,5);
        logger.info("found " + topDocs.totalHits + " topDocs");

        //Create an array to hold the quotes we are going to
        //pass back to the client
        List<Quote> foundQuotes = new ArrayList<Quote>();
        for ( ScoreDoc scoreDoc : topDocs.scoreDocs ) {
            //This retrieves the actual Document from the index using
            //the document number. (scoreDoc.doc is an int that is the
            //doc's id
            Document doc = is.doc( scoreDoc.doc );

            //Get the id that we previously stored in the document from
            //hibernate and parse it back to a long.
            String idField =  doc.get("id");
            long id = Long.parseLong(idField);

            //retrieve the quote from Hibernate so we can pass
            //back an Array of actual Quote objects.
            Quote thisQuote = (Quote)session.get(Quote.class, id);

            //Add the quote to the array we'll pass back to the client
            foundQuotes.add(thisQuote);
        }

        return foundQuotes;
    }

People Found This When Looking For:

• lucene indexwriterconfig (168)
• lucene example (150)
• lucene morelikethis (109)
• IndexWriterConfig example (90)
• IndexWriterConfig (51)
• morelikethis lucene (38)
• lucene indexwriterconfig example (34)
• lucene more like this (33)
• lucene examples (31)
• lucene morelikethis example (30)

I was working on a component for credit card processing where I needed to get the months of the year. Trying to plan ahead, I decided to use the DateFormatSymbols so it could easily handle localization if it ever needed to be used in a different language.

It wasn’t doing what I expected and I finally found the problem with the following code:

DateFormatSymbols symbols = new DateFormatSymbols();
String[] months = symbols.getMonths();
int numOfMonths = months.length
System.out.println(numOfMonths);

The output is 13. If you list all of the strings in the months array, you’ll find the first 12 are what you’d expect, but there is a 13th blank month at the end.

It turns out that there are some lunar based calendars that have to add a “leap month” every so many years. This month is called Undecimber and comes after December. However, I’m not clear why Java is giving me a blank month. If the given locale only has 12 months, what is the value of giving back a 13 item array and just leaving one blank?

People Found This When Looking For:

• are there 13 months (5)
• java 13 month (3)
• java 13 months (3)
• java 13th month (3)
• there are 13 months (3)
• DateFormatSymbols 13 months (3)
• java month 13 (2)
• Is there 13 months (2)
• java months 13 (2)
• WAS THERE 13 MONTH (2)

If you are trying to use Rome and Rome Feed Fetcher, you may want to change the user agent.  However, the following will not change the default user agent:

FeedFetcher feedFetcher = new HttpURLFeedFetcher();
feedFetcher.setUserAgent("User Agent 007");
SyndFeed feed = null;
feedURL = new URL(rssUrl);
feed = feedFetcher.retrieveFeed(feedURL);
List entries = feed.getEntries();

To change the user agent you must use the InfoCache as shown:

FeedFetcherCache feedInfoCache = HashMapFeedInfoCache.getInstance();
FeedFetcher feedFetcher = new HttpURLFeedFetcher(feedInfoCache);
feedFetcher.setUserAgent("User Agent 007");
SyndFeed feed = null;
feedURL = new URL(rssUrl);
feed = feedFetcher.retrieveFeed(feedURL);
List entries = feed.getEntries();

Otherwise the User agent is set to “Java/1.5.0_04″. This is odd because the default client for Rome is “Rome Client (http://tinyurl.com/64t5n) Ver: 0.7″. It seems like an attempt to change the user agent without having a HashMapFeedInfoCache will result in changing the user agent, but somehow it reverts to “Java/1.5.0_04″ instead of whatever you set it to.

People Found This When Looking For:

• Rome client (34)
• rome user agent (7)
• rome user-agent (5)
• rome useragent (4)
• agent rome (2)
• selenium change user agent (2)
• user agent Rome Client (2)
• Rome Client user-agent (2)
• feedfetcher rome (2)
• rome syndfeed user agent (1)

Cobertura is a fork of jCoverage. It runs reports to let you see how much of your code is being tested by unit tests. This is incredibly useful to find areas of your code where a bug would go undetected.

It looks like there is a plugin for Maven already, so I’m going to have to give it a try sometime. link

Maven 1.0 has some problems with memory leaks. Most of the time these aren’t issues, but if you are trying to compile a multiproject you might run into problems. By default Maven tells java to let it have up to 256MB of ram. If you need to increase this you’ll need to open the maven.bat (windows) or maven.sh (unix) file and change the way that java is called. Somewhere around line 118 (on the .bat file)you should see the following:

if "%MAVEN_OPTS%"=="" SET MAVEN_OPTS="-Xmx256m"

Change the line so it reads:

if "%MAVEN_OPTS%"=="" SET MAVEN_OPTS="-Xmx1000m"

This will give Maven 1GB of memory to work with. While this doesn’t really solve the problem of the memory leak, it may give you enough space to keep the problem from crashing Maven.

You can also set MAVEN_OPTS by setting it up as a variable in your environment. If you do this the environment will override anything you set in the .bat or .sh file.

People Found This When Looking For:

• maven out of memory (137)
• maven increase memory (91)
• increase maven memory (58)
• maven outofmemory (36)
• maven_opts memory (35)
• maven_opts windows (35)
• maven memory (29)
• maven out of memory windows (23)
• increase memory maven (20)
• MAVEN_OPTS (18)

Why Java Won’t Get It Right is an interesting entry about some of the problems with Java technology. The best part is that it is written by someone who actually knows Java. A part that I particularly liked was:

They over-architect everything. I’ve actually used a Java framework (I’m not gonna say which) that had XML config files that configured more XML config files! That’s just silly.

The author makes comparisons to Ruby on Rails and talks about how he doesn’t think Java will ever have anything like Rails.

I’ve seen a few demos of Rails and it is impressive, but much of the functionality it gives you has been available in WebObjects for some time. In fact I’ve met several Ruby developers that started with Rails and switched to WebObjects as their application got bigger. (Update: It turns out I was mistaken. They switched from Ruby to Webobjects, but they were using a different web framework instead of Rails.)

There is an interesting comparison between a Ruby project and a Java project posted on the Ruby on Rails site. The code comparison is interesting because it shows how much Ruby does for you automatically if you know how to use it. A lot of what Ruby is doing is giving you automatic setters and getters.

It would be interesting to see a comparison between the amount of code necessary to write a Ruby application and the same app in WebObjects, but when it comes down to actual productivity the language being used is rarely the bottleneck. The skills of the programmer are by far the most important factor. The tools available in the language are second and the language ranks third or lower.

Good tools have a huge impact on productivity. Simple things like auto-complete and real time syntax checking cumulatively make a large difference in productivity. One of the areas where WebObjects really shines is in giving you the ability to graphically connect your data with the view. You can still do everything manually in code, but the graphical tools give you the ability to really think about the problem on a level that is much closer to the user experience.

• Some thoughts on how Ruby is Similiar to Webobjects
• A look at some of the frameworks available for Ruby

The following is a JUnit test that looks like it should always run without a problem. Mark the current time in a variable called start call Thread.sleep and tell it to sleep for x number of seconds, note the current time again in a variable called end and then assert that end - start is going to be less than or equal to x.

    public void testThreadSleep() throws Exception {
        long start = 0;
        long end = 0;
        long elapsedTime = 0;
        for(int i = 1000; i < 1500 ; i = i + 20){
            start = System.currentTimeMillis();
            Thread.sleep(i);
            end = System.currentTimeMillis();
            assertTrue(i <= end - start);
        }
    }

However in acutally running this code the assertion is not always true. It appears that when you try to call Thread.sleep(x) it may not sleep for the entire x milliseconds. Obviously it might take longer than x because a thread isn’t guaranteed to run. There might be another thread with a higher priority or the system might be doing garbage collection. However I would expect that it wouldn’t run less than the specified amount of time, but that is what appears to be happening.

I believe this has to do with the way that the JVM operates. Evidently it may wake up a thread a few milliseconds before the appointed time. It is possible that it may be anticipating garbage collection and waking threads up slightly early

People Found This When Looking For:

• thead sleep garbage collection (1)

According to some posts on Java Ranch, Sun is looking to create a Sun Certified Java Associates exam. The idea is to have an exam that companies can use to certify entry level programmers. I’m not sure why this is better than the current Sun Certified Java Programmer certification. It sounds like they want a certification that is easier to pass for someone with less experience. I didn’t find the SCJP exam all that difficult. In fact I didn’t even bother to put it on my resume, but after reading all the comments at Java Ranch about how difficult it is to pass maybe I should.

This seems like an odd move on Sun’s part. Generally most companies aren’t complaining that the programmers they hire are overqualified. I suppose it may help them find promising junior programmers who they can mentor. The exam could help separate people who have a good foundational understanding of the language from those that have only played around with a few features. I suppose it makes sense from a financial stand point. The tests aren’t cheap.

I know that many scripting language programmers are trying to learn Java or C# as their next career step. If you’ve only worked with a scripting language, the transition to object oriented anything is going to be a big step. Trying to understand threads, nested classes, etc. when you are still trying to transition away from thinking of everything as a web page with code snippets can be a pretty big challenge. The new exam could help give these people their first real “win” in the Java language. Sun may see this more as a marketing tactic to help bring people to Java that might be considering C#. Once someone passes the SCJA they are probably going to want to move up to the SCJP.

At least in my case the Sun Certified Java Developer didn’t hold a lot of appeal for me once I passed the programmer exam. I figured that I’d rather spend my time creating something that I want to use instead of designing an application that is only going to be used as part of the certification process. It is a lot easier to code something you actually find interesting than something you plan to throw away.

• Short Comment on new exam
• Java Ranch Discussion of Exam

People Found This When Looking For:

• resume scja (6)
• scja blogs (2)
• SCJA buy cheap (1)
• SCJA CERTIFIED PEOPLE RESUME (1)
• scja difficult (1)
• scja help (1)
• scja resume (1)
• SCJA resume logo (1)
• scja what do i bring (1)
• cheap java certification (1)

Agylen: Comparable vs equals has a nice discussion of how compareTo is used in Sets.

If you don’t understand how Java is going to use your compareTo and your equals methods you can run into a problem with Sets. Basically you shouldn’t have a compareTo() method that returns 0 unless equals() returns true.

Since a Set only allows once instance of each object, it will ignore the addition of any objects it already contains. If your object implements the Comparator interface the Set will check the compareTo method not the equals method.

It is common to write compareTo methods that only look at one field. For example, you might want to sort Person objects by their last name and then first name. If you only check the last and first name field, then comparing two John Smiths would return 0 even if they were different objects. This would cause the Set to evaluate them as the same object and prevent you from adding the second John Smith to the Set.

People Found This When Looking For:

• compareto vs equals (6)
• java compareTo vs equals (5)
• java equals vs compareto (4)
• java comparable vs equals (3)
• equals vs compareto in java (2)
• equals vs compare in java (1)
• java Comparable equals (1)
• comparable and equals (1)
• java comparator equels (1)
• java comparator vs comparable (1)

Using JUnit With Eclipse IDE is a great simple tutorial on how to use JUnit in Eclipse. It doesn’t assume any knowledge of JUnit, but is useful as a quick tutorial for people wanting to make a switch to the Eclipse development environment.

People Found This When Looking For:

• eclipse ide genetic algorithms (1)
• junit eclipse ide (1)