Strolch:Overview

From gsiwiki
Jump to: navigation, search

This page describes the Strolch software agent and the motivation behind its development.

Overview

Strolch is an open source component based software agent written in Java and can be compared, in a light sense, with the Java EE stack: Strolch takes care of persistence, implements Services for use cases, Commands as re-usable algorithms and has a parameterized data model.

Strolch has an intrinsic understanding for mandates, which are called realms so that a single agent can be used to implement applications with multiple users/customers for instance in SaaS environments.

The parameterized data model consists of two top level objects, Resources and Orders. These two objects can have any number of ParameterBags which in turn can have any number of Parameters on them. This allows for a very dynamic modelling of data structures including modification at run time. Multiple ready to use Parameter types are already implemented which handle the primitive types in Java including ListParameters for collections of these primitive types.

One of the main features of the Strolch agent, is that persistence is handled transparently and the user must not be worried about databases and the likes. Currently there are two implementations for persisting the Strolch model, a PostgreSQL and an XML file persistence. Currently both persistence layers persist the data by converting to XML and storing it into the database. The XML file persistence stores each object in its own file.

The agent itself has a small memory footprint and requires very few components to start. For the agent to be useful it needs additional functionality which is implemented in StrolchComponents. Each component is registered via its Java interface on the agent and is bound to the life cycle of the agent. When the agent is started, these components can be retrieved and used to perform any number of functionalities. This is the preferred way to extend the Strolch agent. There are a number of components already implemented, e.g. the ServiceHandler which executes Services in a controlled fashion and can validate authorized access to these services.

No software product is complete without a system for authentication and authorization. Strolch implements this by using the Privilege framework which has been written by Robert von Burg. The standard ServiceHandler detects the existence of the PrivilegeHandler and then validates that the user has authorization to perform the service. This framework is implemented as its own Strolch component, thus can be retrieved at any time during execution to perform fine grained and special authorization validation.

Motivation

A question often asked is why create Strolch. What are its benefits in contrast to using Java SE with an OR-Mapper like Hibernate, or using Java EE on JBoss or Glassfish? Especially since many of the features existing in those stacks needed to be re-created in Strolch.

The first answer to this question is that those systems are often overly complicated and bloated. Java SE with Hibernate certainly is a viable option when it comes to being light-weightier but Hibernate, even though it is supposed to, often fails to truly help remove the need to really understand an RDBMS. Often enough Hibernate will just get in the way of the most important part of development: writing the business code. Being an OR-Mapper which is supposed to implement all the nitty-gritty details of an RDBMS system, Hibernate, and JPA for that matter, still often has the developer go back to understanding these details.

Strolch tries a different approach to persistence. Instead of writing pojos/entities, Strolch's model has the concept that each element's attributes are part of a composition pattern: each attribute is its own object and thus can be dynamically changed at runtime, but also makes persistence of such an element generic. Instead of having fixed attributes for a concrete class, these parameters are stored in a map and are accessed through the parameter's ID.

Assigning an ID to an attribute for accessing of course brings its own downsides, i.e. the parameter might simply not be there, when being accessed. This is certainly an issue that the developer must handle, when implementing a project using Strolch, but allows the developer to not need to worry about persistence, as this is generically handled.

Since the persistence is generically handled, and Strolch stays lightweight on its requirements at runtime, the developer can quickly get down to what is important for business value: Writing the business logic and the presentation layer. Here too Strolch tries to help the developer by bringing in concepts which are easy to follow: Use cases are implemented as Services, and re-usable business logic is put into Commands.

There will be reasons against using Strolch, as there will be against using the Java EE stack, or an OR-Mapper or even the Java ecosystem for that fact. Important is to note, that the concepts behind Strolch are nothing new, but have been implemented in at least two previous proprietary products. Since those products are not accessible to the public, it was decided that a re-implementation might be of use to the programming community at large.

Currently there is at least one company using Strolch in a commercial project which helps drive Strolch's development and further motivates its existence.

Strolch is an open source project and licensed under the Apache License 2.0.

Technology

Strolch is written in Java and is programmed against the JDK 7. Strolch runs on any JRE 7 compliant environment. Strolch is tested on the Oracle JRE 7.

Dependencies

Strolch strives to use as few external dependencies as possible, so that the Strolch runtime is not bloated unnecessarily. The following list of Strolch dependencies is a summary and was created using mvn dependency:tree on the li.strolch.dev project on the 2014-03-08.

Basic runtime dependencies

  • ch.eitchnet:ch.eitchnet.privilege:jar:0.2.0-SNAPSHOT:compile
implements the authorization and authentication in Strolch
  • ch.eitchnet:ch.eitchnet.utils:jar:0.2.0-SNAPSHOT:compile
Consists of utility classes for recurring problems
  • ch.eitchnet:ch.eitchnet.xmlpers:jar:0.3.0-SNAPSHOT:compile
Implements the XML Persistence layer used by li.strolch.persistence.xml
  • org.postgresql:postgresql:jar:9.3-1100-jdbc41:compile
Implements the PostgreSQL Persistence layer used by li.strolch.persistence.postgresql
  • commons-cli:commons-cli:jar:1.2:compile
Implements the command line parameter parsing when starting from a main class
  • junit:junit:jar:4.11:compile
Testing facilities
  • org.slf4j:slf4j-api:jar:1.7.2:compile
Logging facilities

Web Restful dependencies

Sadly Restful services development requires quite a few extra dependencies:

  • com.google.guava:guava:jar:14.0.1:compile
  • javax.annotation:javax.annotation-api:jar:1.2:compile
  • javax.servlet:javax.servlet-api:jar:3.0.1:provided
  • javax.validation:validation-api:jar:1.1.0.Final:compile
  • javax.ws.rs:javax.ws.rs-api:jar:2.0:compile
  • log4j:log4j:jar:1.2.17:runtime
  • org.eclipse.persistence:org.eclipse.persistence.antlr:jar:2.5.0:compile
  • org.eclipse.persistence:org.eclipse.persistence.asm:jar:2.5.0:compile
  • org.eclipse.persistence:org.eclipse.persistence.core:jar:2.5.0:compile
  • org.eclipse.persistence:org.eclipse.persistence.moxy:jar:2.5.0:compile
  • org.glassfish.hk2.external:asm-all-repackaged:jar:2.2.0-b21:compile
  • org.glassfish.hk2.external:cglib:jar:2.2.0-b21:compile
  • org.glassfish.hk2.external:javax.inject:jar:2.2.0-b21:compile
  • org.glassfish.hk2:hk2-api:jar:2.2.0-b21:compile
  • org.glassfish.hk2:hk2-locator:jar:2.2.0-b21:compile
  • org.glassfish.hk2:hk2-utils:jar:2.2.0-b21:compile
  • org.glassfish.hk2:osgi-resource-locator:jar:1.0.1:compile
  • org.glassfish.jersey.containers:jersey-container-servlet-core:jar:2.5.1:compile
  • org.glassfish.jersey.containers:jersey-container-servlet:jar:2.5.1:compile
  • org.glassfish.jersey.core:jersey-client:jar:2.5.1:compile
  • org.glassfish.jersey.core:jersey-common:jar:2.5.1:compile
  • org.glassfish.jersey.core:jersey-server:jar:2.5.1:compile
  • org.glassfish.jersey.ext:jersey-entity-filtering:jar:2.5.1:compile
  • org.glassfish.jersey.media:jersey-media-moxy:jar:2.5.1:compile

Model

The Strolch model is implemented in the project li.strolch.model.

The Strolch model consists of two root level elements: Resource and Order. Each element has at least the following attributes:

  • Id - the element's id
  • Name - the element's name
  • Type - the element's type

Each root element can have any number of ParameterBag instances on it, which in turn can have any number of Parameters on it. Accessing these objects is always done by their IDs. Strolch root elements are always stored in the respective ElementMaps in their Strolch realm. Thus accessing a certain parameter from a Resource would look like this:

try (StrolchTransaction tx = realm.openTx()) {
  Resource resource = tx.getResourceMap().getBy(tx, "TestType", "MyTestResource");
  DateParameter dateP = resource.getParameter("@bag01", "@param6");
  Date date = dateP.getValue();
  logger.info("@param6 date is " + date);
}


XML Presentation of Strolch's top level elements:

<!-- Resource instance -->
<Resource Id="MyTestResource" Name="Test Name" Type="TestType">
  <ParameterBag Id="@bag01" Name="Test Bag" Type="TestBag">
    <Parameter Id="@param7" Name="StringList Param" Type="StringList" Value="Hello;World" />
    <Parameter Id="@param6" Name="Date Param" Type="Date" Value="2012-11-30T18:12:05.628+01:00" />
    <Parameter Id="@param5" Name="String Param" Type="String" Value="Strolch" />
  </ParameterBag>
  <ParameterBag Id="@bag02" Name="Test Bag" Type="TestBag">
    <Parameter Id="@param4" Name="Long Param" Type="Long" Value="4453234566" />
    <Parameter Id="@param3" Name="Integer Param" Type="Integer" Value="77" />
    <Parameter Id="@param2" Name="Float Param" Type="Float" Value="44.3" />
    <Parameter Id="@param1" Name="Boolean Param" Type="Boolean" Value="true" />
  </ParameterBag>
</Resource>

<!-- Order instance -->
<Order Id="MyTestOrder" Name="Test Name" Type="TestType" Date="2013-11-20T07:42:57.699+01:00" State="CREATED">
  <ParameterBag Id="@bag01" Name="Test Bag" Type="TestBag">
    <Parameter Id="@param7" Name="StringList Param" Type="StringList" Value="Hello;World" />
    <Parameter Id="@param6" Name="Date Param" Type="Date" Value="2012-11-30T18:12:05.628+01:00" />
    <Parameter Id="@param5" Name="String Param" Type="String" Value="Strolch" />
  </ParameterBag>
  <ParameterBag Id="@bag02" Name="Test Bag" Type="TestBag">
    <Parameter Id="@param4" Name="Long Param" Type="Long" Value="4453234566" />
    <Parameter Id="@param3" Name="Integer Param" Type="Integer" Value="77" />
    <Parameter Id="@param2" Name="Float Param" Type="Float" Value="44.3" />
    <Parameter Id="@param1" Name="Boolean Param" Type="Boolean" Value="true" />
  </ParameterBag>
</Order>

Realms

Strolch realms implement the multi-client capability which is thus baked right into the Strolch runtime. When configuring a Strolch runtime, realms are configured and for each realm the data store mode is set. Each realm has its own persistence configuration and can thus run in one of the 4 modes that the Strolch agent implements:

  • EMPTY
This is a transient data store mode, where no model changes are persisted, but they are only kept in memory. When the Strolch agent is started, this realm stays empty as no data is loaded.
  • TRANSIENT
This is the same as EMPTY, but with the difference that when the Strolch agent is started, an XML file is parsed and the in memory realm is populated with the elements parsed from that XML file.
  • CACHED
In this mode, all data is stored in memory, and any changes made are written back to the persistence layer. This allows for fast in-memory quries, but makes sure no data is lost when the agent is restarted.
  • TRANSACTIONAL
In this mode no data is kept in memory and every query, and root element retrieval is passed to the persistence layer to be retrieved from the underlying database. This is what comes closest to a typical Java+Hibernate implementation.

Strolch Realms are also responsible for opening Transactions, as these are bound to the persistence layer configured for this realm. At runtime, a realm is then accessed from the ComponentContainer:

ComponentContainer container = getAgent().getContainer();
StrolchRealm realm = container.getRealm(StrolchConstants.DEFAULT_REALM);
try(StrolchTransaction tx = realm.openTx()) {
  Resource resource = tx.getResourceMap().getBy(tx, "TestType", "MyTestResource");
  ...
}

In a Service implementation there is a convenience method, so that this is as simple as calling openTx(String).

Services and Commands

In the motivation section, it was discusses that writing business logic is what developing is about and a reason why Strolch is a different approach to the Java EE ecosystem. So this is where Services and Commands come into play, and tries to make writing business logic a first class citizen.

Services are to be used once for each use case. Services are not re-used or called by other services. Services open transactions are implement the calling of the re-usable commands. Thus when writing projects using Strolch, the first thing to do after configuring the runtime environmet for your situation, Services will be implemented.

Commands on the other hand are re-usable and should be implemented in such a way, that the encapsulate the use case's different actions. Commands are then passed to a transaction for execution and, when the transaction is comitted, will be executed. Commands also implement undoing its operation in the case of exceptions. Strolch transactions handle the life-cycle of a command. A further function of Commands is to lock the relevant Strolch elements before execution.

A typical Service and Command implementation would look as follows:

public class SetParameterService extends AbstractService<SetParameterArg, ServiceResult> {

  public static final long serialVersionUID = 1L;

  @Override
  protected ServiceResult internalDoService(SetParameterArg arg) {

    // open a new transaction
    try (StrolchTransaction tx = openTx(arg.realm)) {

      // find parameter to modify
      Parameter<?> parameter = tx.findElement(arg.locator);

      // instantiate the command
      SetParameterCommand command = new SetParameterCommand(getContainer(), tx);
      
      // set the arguments
      command.setParameter(parameter);
      command.setName(arg.name);
      command.setInterpretation(arg.interpretation);
      command.setUom(arg.uom);
      command.setHidden(arg.hidden);
      command.setIndex(arg.index);
      command.setValueAsString(arg.valueAsString);

      // add the command to the transaction
      tx.addCommand(command);
    }

    // return the execution result of the service
    return ServiceResult.success();
  }

  /**
   * The argument class for this service
   */
  public static class SetParameterArg extends ServiceArgument {
    public static final long serialVersionUID = 1L;
    public Locator locator;

    public String name;
    public String interpretation;
    public String uom;
    public Boolean hidden;
    public Integer index;

    public String valueAsString;
  }

  @Override
  protected ServiceResult getResultInstance() {
    return new ServiceResult();
  }
}
public class SetParameterCommand extends Command {

  // input fields
  private Parameter<?> parameter;
  private String valueAsString;

  // undo fields
  private String oldValueAsString;

  private StrolchRootElement replacedElement;

  /**
   * @param container
   * @param tx
   */
  public SetParameterCommand(ComponentContainer container, StrolchTransaction tx) {
    super(container, tx);
  }

  // setters for input ...
  // getters for output ...

  @Override
  public void validate() {
    DBC.PRE.assertNotNull("Parameter may not be null!", this.parameter);
  }

  @Override
  public void doCommand() {

    // lock the element to be modified
    StrolchRootElement rootElement = this.parameter.getRootElement();
    tx().lock(rootElement);

    // perform changes
    if (this.valueAsString != null) {
      this.oldValueAsString = this.parameter.getValueAsString();
      SetParameterValueVisitor visitor = new SetParameterValueVisitor();
      visitor.setValue(this.parameter, this.valueAsString);
    }

    // update root element
    if (hasChanges()) {
      replacedElement = new UpdateElementVisitor(tx()).update(rootElement);
    }
  }

  private boolean hasChanges() {
    return this.oldValueAsString != null || this.oldName != null || this.oldInterpretation != null
        || this.oldUom != null || this.oldHidden != null || this.oldIndex != null;
  }

  @Override
  public void undo() {

      // undo changes
      if (this.oldValueAsString != null) {
        SetParameterValueVisitor visitor = new SetParameterValueVisitor();
        visitor.setValue(this.parameter, this.oldValueAsString);
      }

      // update root element
      if (hasChanges() && this.replacedElement != null && this.replacedElement != this.parameter.getRootElement()) {
        new UpdateElementVisitor(tx()).update(replacedElement);
      }
    }
  }
}

Code

The Strolch code can be retrieved from GitHub, where the code is hosted. Each commit triggers a continuous integration build, so that SNAPSHOT builds can be quickly integrated in projects if needed.

Strolch is divided up into different projects on GitHub so that these projects can be developed, or bugfixed independently and not all parts are required in every context:

All Strolch projects on GitHub

Main Strolch components

This project implements the Strolch model. This is where you will find the different elements that can store data at runtime. Currently there are two root elements: Resource and Order
The agent is the Strolch runtime and is the component which implements the core Agent functionality. That is:
→ Provide the Agent instance which loads the configuration and is the entry point to the runtime
→ Provide the ComponentContainer instance from which the registered components can be accessed
→ Configure and maintain the realms, which implement the multi-client capability
→ Provide a default ServiceHandler to perform Services at runtime
→ Implements the realms which each can operate in different modes data store modes: TRANSACTIONAL, CACHED, TRANSIENT
Implements the basic Services and the re-usable Commands:
→ CRUD Services and Commands to modify the model
→ Commands to import and export the model to XML
→ Further services and commands...

Additional components

Implements a PostgreSQL persistence layer so that the Strolch model can be persisted to a PostgreSQL RDBMS when the realm is configured to have a data store mode of either CACHED or TRANSACTIONAL
Implements an XML persistence layer so that the Strolch model can be persisted to XML files when the realm is configured to have a data store mode of either CACHED or TRANSACTIONAL. This implementation uses the xmlpers project.
Implements a Restful API to communicate with the Strolch runtime from clients and external systems.

Meta projects

To quickly get started developing Strolch, this projects provides scripts to checkout all the relevant projects and implements a Maven module so that building this projects builds all Strolch projects.
A Maven parent project for the Strolch projects to synchronize the maven project structure
This bill of material is a Maven project which, when imported in one's own Strolch project, pulls in all required dependencies needed to set up a minimal working Strolch environment.
Implements a test base so that writing tests for Strolch is easy. It provides a RuntimeMock, which handles setting up and tearing down Strolch runtimes during tests.

Example projects

A tutorial application which showcases how to setup Strolch as a standalone Java SE project and starts the Strolch runtime by means of a main-class.
A tutorial application which showcases how to setup Strolch as a standalone Java Webapp which can be deployed to a servlet container e.g. Apache Tomcat 7.

Development

So, if this overview has gotten you eager to start developing Strolch, then let this little howto get you started:

mkdir strolch
cd strolch
git clone https://github.com/eitch/li.strolch.dev.git
./bootstrap_https.sh
mvn clean package -DskipTests=true

Note: If you want to run the tests, then either install a PostgreSQL DB and configure the users as is required by the project, or remove the project from the li.strolch.dev module build, otherwise the build will fail =))

After running the Maven build, you will have a full build of all Strolch projects. Now you can start modifying the projects, and add your own features, or, far more interesting, start developing your projects using the Strolch agent.

To use Strolch in your own projects, look at the two tutorial apps on how they are configured. You can also simply copy the projects, modify the Maven POMs and remove what ever you do not need.

Happy coding =))