# Scala and Akka actors with Prime Sieves

I had the chance to develop the problem of prime sieves in Scala. I took two approaches; (1) Scala Actors and (2) Akka Actors in Scala. The code for both is quite similar.

#### Scala version

`import scala.actors.Actorcase class sieve(n: Int)case class finish(n: Int)class Sieve(val p: Int) extends Actor {  var next: Sieve = null  def _sieve(n: Int) = {    if (isPrime(n)) {      if (next != null) {        next ! sieve(n)      } else {        next = new Sieve(n)        next.start        //println(n + " is prime")      }    }  }  private def isPrime(n: Int): Boolean = {    val d = n / p    val r = n - d * p    if (r != 0) true else false  }  def act = {  	loop {  		react {			case sieve(n: Int) => _sieve(n)			case finish(n: Int) => {				if (next != null) {					next ! finish(n)				} else {					val time = System.currentTimeMillis - Main.start					println(n + "," + time)					// System.exit(0)				}			}  		}  	}  }}object Generator {  def generate(n: Int): Unit = {    val p2 = new Sieve(2)    p2.start    for (i <- 3 to n) {    	p2 ! sieve(i)    }    p2 ! finish(n)  }}object Main {  var start = System.currentTimeMillis  def main(args: Array[String]): Unit = {		for (i <- 1 to 10) {			start = System.currentTimeMillis			Generator.generate(i * 20000)		}  }}`

#### Akka version

`import akka.actor._import akka.actor.Actor._case class sieve(n: Int)case class finish(n: Int)class SieveAkka(val p: Int) extends Actor {  var next: ActorRef = null  def _sieve(n: Int) = {    if (isPrime(n)) {      if (next != null) {        next ! sieve(n)      } else {        next = actorOf(new SieveAkka(n))        next.start        // println(n + " is prime")      }    }  }  private def isPrime(n: Int): Boolean = {    val d = n / p    val r = n - d * p    if (r != 0) true else false  }  def receive = {    case sieve(n: Int) => _sieve(n)     case finish(n: Int) => {    	if (next != null) {    		next ! finish(n)    	} else {    		val time = System.currentTimeMillis - MainAkka.start    		println(n + "," + time)    		// System.exit(0)    	}    }      }}object GeneratorAkka {  def generate(n: Int): Unit = {    val p2 = actorOf(new SieveAkka(2))    p2.start    for (i <- 3 to n) {    	p2 ! sieve(i)    }    p2 ! finish(n)  }}object MainAkka {	var start = System.currentTimeMillis	def main(args: Array[String]): Unit = {		for (i <- 10 to 10) {			start = System.currentTimeMillis			GeneratorAkka.generate(i * 20000)		}	}}`

Then I performed executions of the applications for `n = 20`000-200`000`. I used a dual core machine with 2GB of main memory. The performance of Akka is quite interesting. I also monitored the executions with VisualVM and it turned out for the execution Akka uses at most 28 threads for all the actors.

It’s very interesting to see how this can be explained that the time for computations gradually converge. I think this is an observation on how Scala/Akka actors are scalable; i.e. when the number of actors increase, the cost of resources/computations are affected in the same order.

Update on 02/21/2012. I took another approach in running the examples. Instead of having a main loop to generate all the prime sieves, each time the program was passed an argument to generate the prime sieves for. It turned out that the results now make more sense, however, I have no idea on how this could change this behavior. A snapshot of the Akka performance is as follows which complies with being close to exponential growth.

# OSGi Runtime Comparison: Eclipse Virgo vs Apache Karaf

I’ve been working for a while on deploying web applications onto an OSGi Runtime. Initially, we had Eclipse Virgo and Apache Karaf as the options. It’s not yet final but I thought maybe it would be nice to post some general comparison in this regard. It also fascinates me that although there are several works on OSGi Runtime/Server, but also in a way or another quite all of them end up in an dependency on or using Apache Felix Framework.

An overview of different OSGi Container Features

Shell FrameworkLogging APIInstance SupportHot DeploymentDynamic ConfigurationProvisioningWeb ContainerAdmin ConsoleOSGi Framework SupportedOSGi Container Runtime
Apache Felix GoGoLogBack?Yes?YesJetty /// Apache Tomcat /// Web Integration Layer (Java EE)YesEclipse EquinoxEclipse Virgo
Apache Felix GoGoLog4J, SLF4J, JCL, etcYesYesYesYesJettyYesEclipse Equinox ///
Apache Felix
Apache Karaf

I shall update the post with more results especially on Eclipse Gyrex if I can grab the time.

# Maven repository to OSGi Bundle Repository

Recently, I’ve had the chance to start to work with OSGi. One of the first things that I faces was how I can prepare a local Maven repository as a local OSGi Bundle Repository (OBR). The central concept is to maintain a `repository.xml` as the OBR descriptor. There are several ways to do so including manually doing it. I just found out that Apache Felix has a Maven plugin called maven-bundle-plugin. In the latest version of this plugin, there is a goal bundle:index. On a project configured with you Maven repository, first configure this plugin and then run:

[plain]
mvn bundle:index
[/plain]

This will generate a `repository.xml` at the root of your Maven repository. Then you can use that to feed the OSGi container that you use for bundle provisioning.

# Safari internal error with Facelets

We had an experience with Safari and Facelets that I thought is nice to share. Using facelet templates, we usually use two tags to define and design the layout: ui:insert and ui:include. The point is that the first one includes the “rendered” output of the inserted template while the second one includes it before rendering. It is typical that a doctype declaration is present at the first line of each XHTML template in facelet. But, in the case ui:include there must no such doctype declaration since it is already defined in the parent template. Firefox does not complain about this but Safari does leading to an internal error message.

# Customized WAR/JAR with Maven Multiple Modules

Maven provides a clean way to create multi-module projects such that using the natural style of POM’s will mange the way the final artifacts are bundled and released. However, dealing with non-standard project structure, there are situations that we need to tweak how the project artifacts in a multi-module project are created. In this post, we go through how this is done using Maven Resources Plugin, Maven Jar Plugin, and Maven Dependency Plugin.

The project layout we use is as follows:

• parent: is the POM project that is the parent of all projects defining the common dependencies and properties
• foo: is a child project of parent that is for instance provides some architectural API and other resources for other web-based modules
• bar: is a child project of parent that is a web module dependent on foo and provides some business specific features

In this setup, the final purpose is to build a web archive (war) for bar that selectively uses some features from foo. On the other hand, let’s assume that foo provides three different set of features provided as artifacts:

• API: a set of classes that is used by other child projects of parent such as bar
• configs: a set of configuration files and other resources that is needed to deploy when some other child uses foo; e.g. some Spring or Hibernate configuration files in the final artifact
• web: a set of common web resources that is used by other child projects; e.g. table list templates or common images for web modules

The steps that are required to produce the final bar war artifact are:

1. We use maven-resources-plugin to copy the resources into the desired locations for being bundles as separate artifacts (foo)
2. We use maven-jar-plugin to create different artifacts (jar files) such that they can be used independently (foo)
3. We use maven-dependency-plugin to again include the different artifacts of foo in the desired location to be bundles as a whole war artifact

For the first step, we use maven-resources-plugin to copy for instance the desired resourced into the location that should be considered when packaging the artifacts for foo. We bind the configuration to “prepare-package” phase:

[xml]
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<version>2.5</version>
<executions>
<execution>
<id>copy-configs</id>
<phase>prepare-package</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>\${basedir}/target/classes/WEB-INF/configs</outputDirectory>
<resources>
<resource>
<directory>src/main/java</directory>
<filtering>true</filtering>
<includes>
<include>**/*.xml</include>
</includes>
</resource>
</resources>
</configuration>
</execution>
<execution>
<id>copy-web</id>
<phase>prepare-package</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>\${basedir}/target/classes/WEB-INF/views</outputDirectory>
<resources>
<resource>
<directory>src/main/webapp</directory>
<filtering>true</filtering>
<includes>
<include>**/*.*</include>
</includes>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
[/xml]

For the second step, we defined multiple artifact using the “classifier” option in the maven-jar-plugin. This will install different JAR files as the artifacts of foo to be used in dependent projects:

[xml]
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.3.1</version>
<executions>
<execution>
<id>api</id>
<goals>
<goal>jar</goal>
</goals>
<phase>package</phase>
<configuration>
<classifier>api</classifier>
<includes>
<include>**/*.class</include>
</includes>
</configuration>
</execution>
<execution>
<id>web</id>
<goals>
<goal>jar</goal>
</goals>
<phase>package</phase>
<configuration>
<classifier>web</classifier>
<includes>
<include>WEB-INF/views/**/*.*</include>
</includes>
</configuration>
</execution>
<execution>
<id>configs</id>
<goals>
<goal>jar</goal>
</goals>
<phase>package</phase>
<configuration>
<classifier>configs</classifier>
<includes>
<include>WEB-INF/configs/**/*.*</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
[/xml]

For the third step, we use maven-dependency-plugin to include and unpack the desired artifacts from foo in the bar packaging process such that the final artifact of bar includes the required resources from foo:

[xml]
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.2</version>
<executions>
<execution>
<id>unpack</id>
<phase>prepare-package</phase>
<goals>
<goal>unpack</goal>
</goals>
<configuration>
<artifactItems>
<artifactItem>
<groupId>nl.hmsolutions</groupId>
<artifactId>core</artifactId>
<version>1.0</version>
<classifier>web</classifier>
<type>jar</type>
<overWrite>true</overWrite>
<outputDirectory>\${project.build.directory}/\${project.artifactId}-\${project.version}</outputDirectory>
<includes>**/*.*</includes>
</artifactItem>
</artifactItems>
<includes>**/*.*</includes>
<outputDirectory>\${project.build.directory}/\${project.artifactId}-\${project.version}</outputDirectory>
<overWriteReleases>false</overWriteReleases>
</configuration>
</execution>
</executions>
</plugin>
[/xml]

Finally, in the “\${basedir}/target/bar-1.0”, we can find the final war archive including all the resources and classes for the project.

# Implemeting Active Object pattern with concurrent Java – 2

In the previous post, I discussed some steps towards implementing active object patter with concurrent Java.

Now, let’s turn to each “active object” (Server or Client). We expect from each object to hold a single thread of execution. To have this, it maintains an instance of an ExecutorService described above. Also, it maintains a queue of messages (ProcessStore). Through time, it scans through its ProcessStore to fetch the available MethodInvocations to be processed. It submits each MethodInvocation to its ExecutorService to be executed. To achieve this, we make each active object implement Runnable interface. Then, we introduce an init() method to be called to actually submit the instance of the active object (itself) to its own ExecutorService. Then through the run method it processes the messages out of the ProcessStore:

[java]
class Server implements Runnable, ProcessStoreAware {

private ProcessStore ps = new ProcessStore();
private ExecutorService es = new InterruptibleThreadPoolExecutor(
private MethodInvocation currentMI;

public void init() {
this.es.submit(this);
}

@Override
public void run() {
for (;;) {
try {
MethodInvocation mi = ps.take();
currentMI = mi;
this.es.submit(mi);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}

@Override
public ProcessStore getProcessStore() {
return ps;
}

}
[/java]

Having a Server, a Client may now queue a message in Server to be processed:

[java]
Callable command = new Callable() {
public Object call() {
myServer.request();
}
}
MethodInvocation mi = new MethodInvocation(command);
[/java]

Now, there are circumstances when a MethodInvocation is being executed, it required to await the current thread of execution for a specific condition to take effect; i.e. a MethodInvocation awaits on a condition. For instance, Server maintains a taken property of type boolean to denote whether it is already occupied with a Client or not. So, in such situation, if another Client also requests for the Server, then it should await on taken be available again. On the other hand, when taken is again available, all the waiting MethodInvocations should be notified. We need a event notification mechanism to release (signal) all the waiting ones. We introduce SingalAction to encapsulate such event of notification:

[java]
public class SignalAction implements Callable {

private MethodInvocation originalMI;
private ExpressionHolder expressionHolder;
private ProcessStoreAware executor;

public SignalAction(final MethodInvocation originalMI, final ExpressionHolder expressionHolder,
final ProcessStoreAware executor) {
this.originalMI = originalMI;
this.expressionHolder = expressionHolder;
this.executor = executor;
}

public Object call() throws Exception {
if (expressionHolder.getValue()) {
originalMI.signal();
} else {
executor.getProcessStore().offer(copy());
}
return null;
}

private MethodInvocation copy() {
MethodInvocation mi = new MethodInvocation(this, originalMI);
return mi;
}

}
[/java]

SignalAction uses an interface ExpressionHolder that presents a value to denote whether or not the releasing condition is satisfied. If satisfied, the original MethodInvocation is signalled; otherwise, it queues another MethodInvocation based on itself to check the condition some time later in future. This continues till the release conditions are satisfied and the original MethodInvocation is signalled. ExpressionHolder is a simple interface to present a boolean value. We cannot use the original fields or variables since they are changed through that their corresponding value will not be updated:

[java]
public interface ExpressionHolder {
Boolean getValue();
}
[/java]

Having SignalAction, we add a method to Server to simplify the job of queuing an instance of SignalAction:

[java]
private void addSignalActionNotifier(MethodInvocation mi, ExpressionHolder expressionHolder) {
SignalAction sa = new SignalAction(mi, expressionHolder, this);
MethodInvocation saMI = new MethodInvocation(sa, mi);
this.ps.offer(saMI);
}
[/java]

Now, we modify Server.request() method to observe such circumstances:

[java]
private Boolean taken = false;

public void request() {
while (!(!taken)) {
addSignalActionNotifier(currentMI, new ExpressionHolder() {
public Boolean getValue() {
return !taken;
}
});
((Interruptible) currentMI).await();
}
taken = true;
}

public void free() {
taken = false;
}
[/java]

A sample code to run for the whole scenario would be:

[java]
Server s = new Server();
Client c1 = new Client(s);
Client c2 = new Client(s);
s.init();
c1.init();
c2.init();
for (int i = 0; i < 20; ++i) {
c1.process();
Thread.sleep((long) (Math.random() * 1000));
c2.process();
Thread.sleep((long) (Math.random() * 1000));
c1.finish();
Thread.sleep((long) (Math.random() * 1000));
c2.finish();
}
[/java]

As a side note, I am aware that there is already languages or frameworks for such implementation such as Scala, Groovy GPars or Clojure. The discuss here is used to in cases that some modelling language will be transformed to Java code using a model transformer. Still, the target code does not have to be straight Java since there are other languages that finally run on JRE.

# Implementing Active Object pattern with concurrent Java – 1

I used java.util.concurrent facility to implement Active Object pattern in Java. The Active Object pattern is a pattern in implementation of asynchronous message passing. The flow of how the discussion is formed is as follows that spans through two posts (the other one):

1. We first look at the approaches that can be taken for this implementation. I discuss the reason I chose the method here.
2. We discuss the message encapsulation and features that we expect.
3. We discuss the co-relation between a message and a thread of execution so that messages can be interruptible.
4. We discuss how we extend Java facilities to provide each active object deployed in one single core.
5. In the next post, we show the active object is implemented.
6. Finally, we discuss how the feature of interrupting a message can be achieved.

The current implementation status does not use a proxy method or a scheduler as the original pattern says so since our requirements differ in a way or two. However, these features may be seamlessly added.

Let’s use a sample. A Server is a token object that provides an exclusive access to a token to a set of Clients. A Client asks for the token of the Server through Server.request() to gain access to the exclusive token. To publish this request for the token, the Client has two options:

1. Client calls Server.request() and the Server is responsible for queuing the message in the message store.
2. Client creates a message and directly puts the message in the Server‘s message store to be processed later.

The first approach has one trivial problem. To implement this pattern, each object (Client or Server), has a single thread of execution that is used to process the receiving  messages from the other objects. So, if we use the first option, then we need to queue the message in a “synchronous” fashion; i.e. if the Server is running some code while an arbitrary Client needs to queue a message, the Client should wait till the current execution thread in Server to provide access to the Client. We need to have an “asynchronous” communication among the objects at all times so we use the second approach. Each active object exposes its message store (ProcessStore) through a ProcessStoreAware interface.

[java]
public interface ProcessStoreAware {
ProcessStore getProcessStore();
}[/java]

ProcessStore is an encapsulation of LinkedBlockingQueue that is an implementation of a concurrent List API. So, a Client will put a message directly into its associated Server’s message store:

One of the features of asynchronous message passing is its “non-blocking” nature. When a message is published to a Server, the Client continues its job and its own process. However, the expected return value of the published message can encapsulated as a “future” value computation; i.e. some value that is determined some time in future but does not block the thread of execution in the caller. So, an asynchronous message encapsulation should provide two major functionalities: (1) a wrapper around a call to the required server method (2) a future value as the result. FutureTask in Java provides these.

We define MethodInvocation class to introduce this encapsulation that extends FutureTask and adds some other properties. In our setting, each message is assigned a thread of execution to have control for awaiting and releasing the processor based on the semantics of the requirements. So, every MethodInvocation is aware of its “owner thread” as a property to such circumstances. In this regard, a MethodInvocation implements Interruptible interface to expose the features of “await” and “signal”; however, it actually delegates them to its owner thread. When a MethodInvocation “awaits”, its state is suspended.

[java]
public interface Interruptible {
boolean await() throws RuntimeException;
boolean signal() throws RuntimeException;
}
[/java]

Using locks in mi. The source of MethodInvocation

On the other, as each MethodInvocation is in a one-to-one relationship with Thread, the owner thread should be able to “await” and “release” based on its MethodInvocation. We introduce InterrutptibleThread to expose these features. It uses an instance of a ReentrantLock and its Condition to “await” execution as far as the MethodInvocation is suspended.

[java]
public class InterruptibleThread extends Thread implements Interruptible {

private Lock awaitLock = new ReentrantLock();
private Condition blockedCondition = awaitLock.newCondition();
private MethodInvocation mi;

// *** ADD Constructors

@Override
public boolean await() throws RuntimeException {
try {
awaitLock.lock();
while (this.mi.isSuspended())
blockedCondition.await();
} catch (InterruptedException e) {
interrupt();
} finally {
awaitLock.unlock();
}
return false;
}

@Override
public boolean signal() throws RuntimeException {
try {
awaitLock.lock();
blockedCondition.signalAll();
} finally {
awaitLock.unlock();
}
return false;
}

public void setMethodInvocation(MethodInvocation mi) {
this.mi = mi;
}

}
[/java]

So far, we have the required encapsulation for the “messages” that are stored in each active object. Apart form a ProcessStore, each active object composes an instance of an ExecutorService; a scheduler that directs JVM how to distribute jobs among the processors. We desire to have each active object deployed on one processor. So, we use an extension of ThreadPoolExecutor. It uses a pool of threads to manage jobs on a processor. It tries to re-use the threads that are idle to minimize the resource consumption through JVM. We extend this class for two reasons. First, to override the behavior of creating a new FutureTask out of a MethodInvocation since MethodInvocation is actually an instance of FutureTask:

[java]
@Override
protected RunnableFuture newTaskFor(Callable callable) {
if (callable instanceof MethodInvocation) {
}
}

protected RunnableFuture newTaskFor(Runnable runnable, T value) {
if (runnable instanceof MethodInvocation) {
}
}
[/java]

Next, we override beforeExecute method to dually assign the current thread of execution and MethodInvocation to each other so that they can control the flow of execution if necessary. This thread may await based on the underlying calls so other threads will be created for other messages:

[java]
protected void beforeExecute(Thread t, Runnable r) {
super.beforeExecute(t, r);
if (r instanceof MethodInvocation) {
it.setMethodInvocation((MethodInvocation) r);
}
}
[/java]

Each instance of ExecutorService is dependent on an instance of ThreadFactory. We provide an implementation since the default implementation uses only one thread group (i.e. “main”) to pool the threads while we need to maintain a pool of threads for each active object:

[java]

private final AtomicInteger counter = new AtomicInteger(1);

SecurityManager sm = System.getSecurityManager();
group = new ThreadGroup(parent, this.hashCode() + "");
}

@Override
InterruptibleThread it = new InterruptibleThread(group, r, "pool-" + group.getName()
+ "-thread-" + counter.getAndIncrement(), 0);
it.setDaemon(false);
return it;
}
}
[/java]

I will continue the discussion in the next post.

# Multicore Programming and Object-Oriented Languages

I had a literature study on how different object-oriented languages provide programming techniques for multicore programming. The outcome was presented as a research report and a presentation talk.

مطالعه‌ای بر روی برنامه‌نویسی چند‌هسته‌ای در زبان‌های شی‌گرا انجام دادم که نتیجه‌‌ش به صورت یک گزارش و یک ارائه در این زمینه در‌آمد.

# سامد

بالاخره افتتاح شد و امیدوارم که خوب استفاده بشه. مبارک باشه . اگر در وطن می‌بودم، شاید من هم نقشی می‌داشتم

News

# How Groovy compiles

In a previous post, I briefly wrote how Groovy compiler takes advantage of ANTLR [1] in the compilation process. In this post, I would like to elaborate on that. Simply put, Groovy goes through the following general phases:

### Parsing

Groovy, in past, has had an internal parser. When ANTLR was added to Groovy, seemingly, they still somehow first use the old one to read the source, called CST, then use the ANTLR parser plugin to read and adapt to the newer version. After parsing the source code, there is some facility that converts CST to AST [2] of ANTLR.

Groovy introduces its own complete data structure set to represent a compilation unit which is usually a script file containing at least one class or some statements of Groovy language. The CST is build to represent the whole structure of the script and then it is converted to the AST. More clarification on this is required.

### Byte Code Generation

When you first get introduced to ANTLR, the recommended approach to use ANTLR in programming language development is [3]:

1. Develop a parser grammar with which ANTLR generates the lexer and the parser for the grammar of the language. It is recommended to use the output format of AST in ANTLR that will give out the parse tree according to the language.
2. Develop a tree grammar according to the parser grammar so that when a sample program is input, you have the ability to traverse the structure of the program and inject the required actions on different nodes of the program.
3. Develop actions or take advantage of string templates for some form of output such as translation to a lower level opcodes.

Well, Groovy does not take the recommended approaches. Instead, and for the CST reason, they heavily take advantage of Visitor [4] pattern. Groovy introduces a GroovyCodeVisitor containing all the methods required for every possible construct in the Groovy language required in the compile process. One implementation of this visitor is AsmClassGenerator. As its name says, it uses ASM to generate byte code while it is based on visitor patter. Specifically, when the byte code generation begins, AsmClassGenerator receives an instance of ClassNode which is the root for the whole source unit parsed and converted to AST. It starts to traverse the children of the root node and visits every node in the tree. In every node, Groovy actually takes advantage of ASM’s facility called ClassVisitor. The ASM’s ClassVisitor is also based on the visitor pattern. So, for instance, when in a level of a statement or a class declaration, the visitor pattern in Groovy class generator takes advantage of the ASM’s ClassVisitor’s different visit methods to actually generate the byte codes for the current node in the AST structure. The concrete instance of the ClassVisitor that Groovy uses is ClassWriter; it has methods to generate operation byte code for different constructs according to JVM byte code specification.

So, at the end, when the starting class node is completely visited, on the other side of the story, the ASM’s class visitor has actually all the byte code for the whole class.