Tagged: java

Programming a State Machine in Java

Some programming problems center around managing different states and the transitions between the states. These problems are ideally suited to be solved by using a finite state machine. This post shows a way of programming a finite state machine in Java to solve a text parsing problem and to keep the readability of the code at a maximum at the same time.

The Problem: a Parser for Unified Diffs

Recently, I needed to parse a unified diff. I looked for a Java library that does this for me without luck, so I decided to roll my own since it is not that big of a problem. A unified diff is a text fragment that contains the differences in lines between two text files (or between two versions of the same text file). It looks something like this:

[gist file=unified-diff.txt]7512852[/gist]

This text is to be parsed to create a Java object that contains all information.

A State Machine to the Rescue

Luckily, a unified diff follows a pattern that can be parsed. There are only a couple of different line types, each line type containing some specific information. These are the possible line types:

  • HEADER: a header line that contains part of the header of the diff
  • FROM_FILE: a line that contains the name of the first file that is compared
  • TO_FILE: a line that contains the name of the second file that is compared
  • HUNK_START: a line that marks the start of a “hunk” of differences in both files, providing the line numbers of the start line and the end line of the hunk
  • NEUTRAL_LINE: a line within a hunk that is contained in both files
  • FROM_LINE: a line within a hunk that is contained in the “from file” only
  • TO_LINE: a line within a hunk that is contained in the “to file” only

Projecting these types to the diff example from above looks like this:

[gist file=states.txt]7512852[/gist]

Now, each line type can only be followed by a certain set of other line types. Modelling the line types as states we get a state machine like this:

states

Coding the States

So, what’s the best way to code such a state machine in Java? Te natural feature for representing states in Java code is an enum. The plain enum would look like this:

[gist file=enum.java]7512852[/gist]

Note that there are two additional states for the initial state and the end state.

Coding the Transitions

That was easy. The question how best to code the transitions between the states is a little harder. One way to do it is to simply let each enum constant implement a method called “nextState” that returns the next state depending on some context information.

[gist file=enum-with-transition.java]7512852[/gist]

Here, each enum constant has to implement the abstract method nextState that takes an object containing some context information about the line of the diff that is currently being parsed and returns another enum constant representing the next state. Usually an implementation of the nextState method should not be overly complex since it simply uses a couple if/else statements to return the next state based on the current state and the information in the context object (an object of type ParseWindow in this case).

If calculating the next state becomes more complex, you should code each transition in it’s own class and simply call that class from the nextState method.

Coding the “Engine”

Having coded the states and the transitions between those states, you need some kind of “engine” that drives the state machine from the initial state through a number of other states to the final end state. While the structure of the state enum is pretty generic, the engine is very specific to the problem you are trying to solve. In the case of a text parser the engine starts by calling the INITIAL state’s nextState method providing the first line of the file and some context information on the next couple lines. Since the first line must be a HEADER line, this will be the result.

Now that we know that the current line is a HEADER line, we know how to parse it. Having parsed the first line, the engine takes the second line and passes it along with some context information to the HEADER state’s nextState method. Lather, rinse, repeat until the END state has been reached.

Wrap-Up

State machines are a natural way to solve some programming problems, parsing problems probably being the most prominent among them. However, a state machine like this can also be a nice and readable way to solve business problems like the transitions between states of an insurance contract or states of a user going through a complex registration process.

A Live Example

The code examples above are excerpts of a live github project called diffparser. Feel free to have a look at the source.

Why and How to Migrate from Eclipse to IntelliJ IDEA

After years of programming Java with Eclipse, and multiple hints from other developers that IntelliJ IDEA is superior to Eclipse, I finally tried it out. Meanwhile, I’ve come to the conclusion that programming with IntelliJ is just so much more fun than with Eclipse. Here’s some Pros and Cons about IntelliJ compared to Eclipse and some tips on migrating from Eclipse to IntelliJ.

Pros

As I am writing this, I have been trying out Intellij for about 3 weeks after having used Eclipse for about 6 years. In those 3 weeks, I have come to love many small and big features of IntelliJ that Eclipse does not offer (at least not without searching for some plugin that does the job). Here’s the positive things I stumbled upon while trying out IntelliJ:

Responsiveness

My current (legacy) project at work consists of about 50 modules, one Eclipse project each. The number of projects combined with Dependency resolution through the IvyDE plugin, it appears that I am spending more time waiting that coding. After having migrated the project to IntelliJ, my workspace is much more responsive. Searching is blazingly fast. Everything just seems to work faster. IntelliJ is only doing the work it needs to do, instead of doing all the work in case that you might need the results in the future. Because of this, it also has a smaller memory footprint.

Flexible Project Structure

In Eclipse it has always been awkward to create projects with a nested directory structure. It works when first checking out the main directory from VCS as an Eclipse project and then importing all projects that are contained within the main directory. Then you have the code of all Eclipse projects twice in your workspace, though: once in the Eclipse project of the main directory and once in the actual Eclipse project… . The project structure in IntelliJ is completely up to you. You import a directory structure and define which sub directories your modules are in.

Smart Tooling

IntelliJ comes with many features helping you to navigate or refactor code. There’s a shortcut for selecting the current word, then the current statement, the current block, and so forth if used repeatedly. Also there’s a shortcut for surrounding the currently selected code with try-catch, if-else, for-loop or several other code snippets. Using both those shortcuts together is an enormous help when coding, if you ask me. Auto-completion is also available everywhere, even when you don’t expect it. IntelliJ auto-completes my Maven dependencies in the pom.xml by checking which artifacts are available in my local repository!

Seamless VCS integration

The integration to Version Control Systems like Subversion or Git feels right at home in IntelliJ. No more switching to the “SVN Synchronizing” Perspective in Eclipse. Just click update or commit and go through the VCS dialog, which shows you all information you need. You can move changed files into different change sets which you can commit separately. And you can perform certain actions like code analysis or reformatting on commit (code analysis is enabled by default, which has saved me from a couple certain bugs already!).

Interactive Editor

The code editor is very interactive and marks up warnings and errors prominently. You can recognize outdated javadoc immediately, since @param-descriptions of parameters that don’t exist are colored red. If you use String.format(), IntelliJ tells you when the placeholders in your format string don’t match the parameter list! If you compile a regular expression using Pattern.compile(), IntelliJ will tell you if the regex is valid and even offers to enter test strings to match against it! IntelliJ also marks unused variables, methods and classes – not just private ones, but also public ones.

Intelligent Usage of Screen Space

There are no “perspectives” as in Eclipse. If you start a debugging session, the debugging view will simply open at the bottom of the screen. If you start a unit test, the testing view will open. No more switching between perspectives.

Cooler looks

IntelliJ ships with a pretty cool theme called “Darcula”, which is made up of dark backgrounds and light font colors and is a lot of fun to work with. There’s a theme plugin for Eclipse, but it only allows theming of the editor window.

Cons

Of course, IntelliJ has some drawbacks, too. I have expected more than I found, though. Here’s the list of bad things I found while evaluating IntelliJ:

Buggy Dialogs

When changing settings for project modules in the Project Structure dialog, the changes are not always saved as I would expect, resulting in having to do it twice. May have to do with the plugins, I’m using, though (I’m suspecting IvyDEA).

Awkward Renaming and Adding of Packages

When you want to add a new Package to an Eclipse-Project, you are presented with a dialog in which you can edit the fully qualified package name. In IntelliJ you only get a single input field for the last element of the name of the package to add or rename. If you want to add a package at some level in between, you have to add a package to the source folder and type the fully qualified name by hand.

Awkward Renaming of Modules

Modules have a directory name in the file system and a name within the workspace. Each time you want to rename a module, you are asked whether to change the former or the latter – I have not found an option to change both at the same time…very frustrating.

Configuring Subversion Integration can be awkward

If you are working with Subversion, you currently have to check out the sources initially via IntelliJ. If you check out the sources with another tool (like TortoiseSVN) and then import it into IntelliJ, it won’t work because of working copy inconsistencies!

It’s not free

IntelliJ costs money. A commercial single developer license costs €450. Which is a lot (at least, when trying to persuade your boss). Licenses for Open Source work are free, though.

Migration tips

Here’s a couple tips that might be helpful when you come from Eclipse and are trying out IntelliJ:

Use the default Eclipse shortcuts

If you have worked for years with Eclipse it seems to much for me to re-learn all key bindings. IntelliJ provides a default key binding that is based on default Eclipse shortcuts. Simply choose it from the list of available Keymaps in the Keymaps settings.

Disable “Caret after end of line”

Coming from Eclipse, IntelliJ has the annoying default setting to allow navigating a line after the end of the line. So, when you navigate up or down the editor to a line that is shorter than the line before, the cursor will stay at the same column instead of jumping to the last character in the new line, as I have come to expect. You can change this in the  IDE Settings by disabling the option “Allow placement of caret after end of line”.

Use ALT+INSERT and ALT+ENTER!

Get acquainted with those shortcuts, they make working with code a breeze!

Show Javadoc on Mouse-Over

By default, IntelliJ doesn’t show the Javadoc of a class or method on mouse over. You can activate this by adding the following line to the file idea.properties in the installation folder:

auto.show.quick.doc=true

Add a Live Template for creating {@link}s in Javadoc

IntelliJ does support auto-completion of class names in Javadoc, but it doesn’t surround them with the {@link} annotation to create hyperlinks in the resulting Javadoc. To mitigate this, you can create a live template with the following text yourself:

{@link $LINK$}

Add a variable called LINK that expands to the expression “complete()” and give it the abbreviation “link” and set it to be only applicable in Javadoc. From then on, you can type “link” and then TAB to create a Javadoc link with auto-completion.

My Personal Conclusion

For myself, I have concluded to use IntelliJ from now on. It’s just so much more fun to work with! All I have to do is persuade my boss to spend some money on the licenses for my team and me… .

A brown-paper package.

How To Release Your Open Source Project to Maven Central

 Once you have created and fine-tuned your current programming project, you usually want to share it with all those developers out there who might have a need for your solution. For Java-based projects, Apache Maven provides a good distribution channel for this. With Maven, you can release your project to Maven Central, which is a global repository for open source Java projects that can also be accessed by build tools other than Maven itself. Since I have not found an easy-to-follow guide on how to do this in the web, I try to document the needed steps in this post.

Step #1: Sign Up at Sonatype

Sonatype is a company that provides all sorts of support and tools for “component lifecycle management” as they call it. Essential for us, they provide the access point to making a project available in the Maven Central Repository. You have to create an account at the following sites (for free, of course!):

Step #2: Create your project in JIRA

Log in to the JIRA system at https://issues.sonatype.org/ with the username and password you chose in step 1 and create an issue of type “New Project”. Most input fields are self-explanatory. In any case, here’s what you have to put into those fields:

  • Project: select “Community Support – Open Source”
  • Issue Type: select “New Project”
  • Summary: name of your project
  • Description: to make things easier on the Sonatype guys, drop a line about what your project is about
  • Attachment: don’t add an attachment
  • Group Id: your desired Maven group id (your namespace). This must be a backwards URL like org.wickedsource (which is the namespace I use for my projects). If you’re not familiar with Maven, read this article on how to choose your Maven coordinates.
  • Project URL: URL to your project homepage. You should have a homepage for your project, even if it is just the frontpage of your github or googlecode repository.
  • SCM URL: URL to your source code management system like github or googlecode
  • Username: your JIRA user name
  • Already Synced to Central: choose “No”
  • Epic Link: leave empty

Some of the fields may probably be left empty and they will still create a repository for you, but it makes it easier if you provide all information. The Sonatype guys will usually respond within a couple hours, but it may take longer – be patient.

Step #3: Create a Key Pair

It is required for any artifacts you release to be signed. The easiest way to do this is by using GNU Privacy Guard (I use the Windows version of GPG). If you don’t have a key pair to be used in signatures yet, follow this article on how to generate a key pair and distributing your public key. You will have to choose a passphrase when generating a key pair. This passphrase will be needed later. If you have more than one key pair, make sure you define one of them as the default key pair in GPG.

Step #4: Create and Sign your Artifacts automatically with Maven

Once you have installed GPG and created a key pair you can use the following maven command to sign all your artifacts automatically:
[gist file=mvn-install.sh]6653895[/gist]
Calling the javadoc and source plugins is necessary to create javadoc and source JAR files along with your actual JAR artifact. Providing javadoc and sources is a requirement for releasing open source projects to Maven Central. The gpg plugin takes care of signing all those JARs with your previously generated default GPG private key.

Caveat: the gpg Maven plugin has to be coaxed into actually signing your javadoc and source JAR files by adding the following configuration to your pom.xml:

[gist file=pom-plugins.xml]6653895[/gist]

Step #5: Deploy your Artifacts to your Staging Repository

If your request for a new Project in the Sonatype JIRA was successful, you will get an answer something like the following:

Configuration has been prepared, now you can:

please comment on this ticket when you promoted your first release, thanks

This basically means that they have provided a staging repository for you where you can upload your artifacts (automatically or manually). From this staging repository you can then manually trigger a release to Maven Central.

The important information is the URL to the staging repository. Add this URL to your pom.xml like this:
[gist file=pom-distributionManagement.xml]6653895[/gist]
Additionally, create a file called “settings.xml” with the following content:
[gist file=settings.xml]6653895[/gist]
Username and password should be the ones you chose when creating your account at https://oss.sonatype.org. Since the file settings.xml contains your credentials and thus sensitive data, it should only be kept locally and not be committed into any SCM system! The id (“sonatype” in the example above) is used by Maven to create the connection between your credentials and the repository id in your pom.xml.

Now you can call the following Maven command to create, sign and deploy your artifacts:
[gist file=mvn-deploy.sh]6653895[/gist]
At some point during the build, you should see that your artifacts are being uploaded to the Sonatype server. The upload may take some while even if your artifacts are rather small, probably because the upload to the Sonatype server is throttled.

Step #6: Promote your Repository

After the Maven build has finished successfully, log in to https://oss.sonatype.org and click the link “Staging Repositories” in the left-side navigation. In the opening tab, select your repository (you can use the “filter by profile” search box in the top right to find your group id). Click on the “close” button to close your repository. This triggers some validation checks on the files you uploaded. You can see if those validations were successful by clicking on the “Activity”-tab in the bottom part of the screen and selecting the “close” node. If there were any errors, fix the errors and deploy again. A documentation of the validations that will be performed when closing a repository can be found here.

Once all validations were successful, you can now promote the repository by clicking the “release” button. This means that you want the contents you uploaded into this repository to be released to Maven Central. After this has been done you should receive an email with the subject “Nexus: Promotion completed” and you should add a comment to the JIRA issue you created and wait for them to activate the sync to Maven Central. This may again take one or two working days. After this, your project has been successfully released to Maven Central and you can upload and then release repositories at any time without having to wait so long (repositories that have been successfully promoted once before will automatically be synced to Maven Central about every two hours from now on).

To check if your project is finally available at Maven Central, you can simply go to http://search.maven.org and search for your group id.

Caveat: I have noticed that sometimes the signature validation failed when I closed a repository on the Sonatype Nexus Server. After repeating the Maven deploy command from step #5 (sometimes a couple times) the signature was validated successfully. I have not yet found out the reason for this, since the uploaded signature files are valid if checked manually with GPG.

Resources

Much of the above information comes from this article in the Sonatype wiki. In the guide above, I added my own experience with releasing Maven artifacts so it should be easier for first-time users to follow.

Wicket 101: Making Use of the MDC Logging Context

Logging frameworks like log4j and logback usually offer a class called MDC (Mapped Diagnostic Context) to store contextual information of a user’s session to be used in log statements. As an example, here’s how to attach a session id to log statements using Wicket and log4j. Impatient? Skip right to the github example project

The Logging Configuration

Log4j is usually configured with a properties or xml file. If you want to output the session id with each log statement, you have to define the session with a variable like %X{sessionId} (see table “Conversion characters” in the javadoc of the PatternLayout class):

log4j.appender.Stdout.layout.conversionPattern=%-5p - %-26.26c{1} - %X{sessionId} - %m\n

This configuration reads the parameter “sessionId” out of the MDC context (if available) and prints it out with each log statement.

Filling the MDC context with Wicket

The session ID has to be stored in the MDC context, of course, before it can be printed out by the above logging configuration. Wicket offers the IRequestCycleListener interface for this purpose. You can extend AbstractRequestCycleListener if you don’t want to implement all methods of the interface:


public class AddSessionIdToMdcListener extends AbstractRequestCycleListener {
  @Override
  public void onBeginRequest(RequestCycle cycle) {
    super.onBeginRequest(cycle);
    WebSession session = WebSession.get();
    // the check on isTemporary() is necessary, since temporary sessions
    // have a NULL sessionId
    if (!session.isTemporary()) {
      MDC.put("sessionId", session.getId());
    }
  }
}

All we do is simply calling MDC.put() at the beginning of each Wicket request. By the way: you can put anything in the context, it doesn’t have to be a session id. The listener can be added to your Wicket application with the following line within the init() method of the Application class:

@Override
public void init() {
  super.init();
  getRequestCycleListeners().add(new AddSessionIdToMdcListener());
}

On Wicket’s Session ID

Note that Wicket only creates a session id for stateful pages, i.e. if you have at least one stateful component on a page or if you explicitly call setStatelessHint(false). This should be the case for most Wicket pages, though.

Resources