Joomla! Open Source Content Management

Q Kinds of applications: What kind of applications can be developed using SOLoist?

A: SOLoist is a framework for building information systems and business applications. SOLoist is not limited for any particular application domain, such as ERP or DMS or CRM or similar. In fact, SOLoist is most suitable for building complex information systems and applications that have the following common characteristics:

• the problem domain implies a complex conceptual model, encompassing many abstractions (concepts, entities), their properties, and relationships;
• the system requires massive instantiation of the concepts that have to be persistently stored in a database;
• the system requires intensive interaction with users through complex, Web-based user interfaces.

The application domains include, but are not limited to: Enterprise Resource Planning (ERP), Customer Relationship Management (CRM), Document Management Systems (DMS), Human Resource Management (HRM), financial and banking, engineering systems, and all other kinds of domain-specific, custom applications.

In fact, we have built many different systems and applications using SOLoist, íncluding different kinds of system supporting engineering and design (by coupling with different external specialized systems), heterogeneous systems coupled with Geographic Information Systems (GIS) for national real-estate cadastre, specialized CRM and HRM systems, and various kinds of domain-specific and custom business applications.

Q Web sites: Is SOLoist suitable for building public Web sites? Is it a kind of a Content Management System (CRM)?

A: Although it can be used for building public Web sites, SOLoist is not primarily aimed for that purpose. This is because SOLoist does not (yet) have built-in and efficient support for search-engine optimization and for inter-page communication via the same mechanisms of pins and bindings (the classical argument passing via URLs is still possible, though). We plan to add this in the future, although this can be achieved by manual interventions even now.

SOLoist is not a CRM either.

SOLoist is best suited for internal, in-house business applications.

However, SOLoist can be efficiently used as a back-end application server for separate portals or public Web sites. A classical Web-based, page-oriented UI can be built with other technologies to work with a SOLoist application server. In such architecture, a SOLoist application works as a UI-less server providing services to other applications built in other technologies. The interoperability can be achieved through Web services or through servlets, for example.

Q Flexibility: It seems that the development speed is gained the most if the requirements can be satisfied by the built-in components that are already provided by and fit within the model and concept of the SOLoist framework. Does it mean that development speed may be severely impacted if the requirements step outside of what is provided by the framework, especially in the area of UI?

A: Indeed, the speed is gained the most in the described case, but it can be gained a lot in many other cases as well. Actually, there are lots of circumstances where the system under consideration can be shaped in the form most suitable for SOLoist, including the shape of the UI, and then the system's architecture becomes really neat and clear, as described in Dragan Milicev's book. SOLoist is a framework that assumes exploitation of the entire methodology of OOIS UML as described in the book, providing a specific perspective to building systems and a new way of thinking about it, leading to systems with simple, clear, and comprehensible architecture, without accidental complexity, where every detail fits in its right place. Besides, SOLoist allows creating arbitrary custom UI widgets, and we have done a plenty of them for commercial applications, some of them connecting with other systems such as GIS, Google Maps, and different custom Web widgets.

Essentially, SOLoist is based on a fundamental principle of not preventing flexibility and invention, but of significantly reducing arbitrariness: it is good if a requirement or an idea can be implemented in some way, but it is not good if the same thing can be implemented in many different ways, because different developers will do it differently and arbitrarily, increasing the system’s entropy and reducing its readability. In SOLoist, many things can be done, but each can be done in one or a very few predictable ways, as every piece has its right place.

Finally, since SOLoist is underpinned by standard and open technologies (Java, servlets, Google Web Toolkit, Relational DBMS/SQL), it is open and flexible for arbitrary extensions and variations. In fact, virtually every requirement can be fulfilled with SOLoist

• either using SOLoist’s features and concepts, in which case the efficiency is dramatically improved as compared when traditional techniques are used, or
• using traditional techniques, if SOLoist has not contributed to that particular technique or aspect with its original concepts and features.

• You can always write your piece of Java code or use a Java library to do a particular work on the server or to talk with an external system.
• If really necessary, you can always write and invoke an SQL query or a stored procedure to do some database-critical work.
• You can always develop your custom Web UI component that does a specific work and communicates with its environment or an external system.
• You can always invoke an external Web service or provide your own one to communicate with an external system,

Q Legacy systems: It seems that SOLoist can be recommended for ground-up projects, but is it suitable for maintaining or extending legacy software developed with other technologies?

A: Indeed, SOLoist is most suitable for ground-up projects. It is not suited for maintenance of legacy systems built with other technologies, at least not for the time being. This is primarily because SOLoist does not have customizable UML model-to-relational database schema mapping and cannot map its object space to an arbitrary database schema. (We may add this feature in the future.) Of course, this can be done by customizing SOLoist’s object-to-relational mapper and providing different adapters (we did this in the past for some projects), but this is not ready out of the box.

Of course, a system built with SOLoist can be connected to a legacy system as to any other external system via some integration bus, such as a Web service or other.

Q Interoperability: Can a system built with SOLoist interoperate with other external systems and how?

A: Yes, it can interoperate with other systems on different architectural levels:

• Via relational database, through triggers and stored procedures. Of course, this mechanism is completely independent of SOLoist and invisible to it. Although possible, this is not so common approach.
• Via relational database as a shared data space, by issuing SQL queries to (an external) database from Java code of methods of domain classes that execute on the server.
• Via calls of methods from Java libraries that execute within the same address space on the server and perform some specific or external task; for example, send an email.
• Via message passing or remote procedure calls issued from Java code of methods of domain classes that execute on the server; for example, sending a message via Java Message Service or calling a remote Web service. A SOLoist application can also provide Web services to external systems.
• Via custom UI widgets that execute on the client from Web browser and communicate to other servers; for example, via widgets built in JavaScript or GWT.

Q Reverse and round-trip engineering: Does SOLoist support reverse engineering and round-trip engineering? That is, if I need to change my design decision, can I change the code generated from the model and then update the model by reverse-engineering the modified code?

A: No, it doesn't.

We do not believe in round-trip engineering as a proper technique for model-driven development (exploiting reverse engineering for revealing hidden design information from legacy systems is another thing). We believe in model-driven development based on formal and executable modeling languages, as described in Dragan Milicev's book.

Some other methodologies use models (especially UML models) just as blueprints or sketches of design, leaving their semantic interpretation to the concrete implementation programming language, framework, technology, conventions, or even individual developer. In those methodologies, UML models are just used to “draw the code” without breathing life into models by providing semantics. Those technologies may need round-trip engineering, because their central artifact is code, as it dictates the semantics of the system. It may make sense there to modify the code to change design decisions and then to update the model as its documentation.

On the other hand, in our methodology, models are formal and authoritative specifications of software that carry semantics. Therefore, we do not encourage changing the code that is generated from the model. That code is just “machine-language code” obtained by “compiling” the model. Just as you are not interested in binary code obtained by compiling your source code, but you maintain your source code instead, the same way you should not touch the code that is automatically generated from the model – you should maintain the model if you need a change.

In addition, in SOLoist, every piece of system’s specification has its right and single place: the structure (classes and their properties and relationships) are modeled in UML, methods (bodies of operations) are coded in Java, UI is specified in Java modules for UI (for the time being). We strongly believe that every piece of system specification should be maintained in its source place, instead of changing its indirect manifestation obtained by code generation, what round-trip engineering encourages. Therefore, if you need a change in the structure – modify the model; if you need a modification in a method – modify its Java body. In our opinion, it is completely wrong to change the Java declaration of a class’ attribute that is generated from the model and then reverse-engineer the code to update the model. One reason for this is that this approach subliminally suggests that the semantics of the specification is that from the Java language and that the Java code is the “master” specification, which is not true for many reasons (for example, because properties are persistent and may be ends of associations, etc.). Another reason is a matter of discipline: the development procedure should be as much disciplined as possible.

As a conclusion, we do not believe in round-trip engineering during development and thus we do not need reverse-engineering in SOLoist.

Q Model-code synchronization: Since the code and database schema are tied to UML models, how does SOLoist keep the model and code in sync? For example, if I specify something in my code manually, such as a method body, and then change something in the model and regenerate the code, what happens with my hand-written code?

A: SOLoist properly keeps the code in sync with the model, while preserving the manually written code, provided that the code is placed properly in the sections aimed for manual coding. Namely, in every file generated from the UML model, SOLoist code generator leaves sections enclosed in special comments like this:

// ---------<SOL id="6de99478-2c5d-45a9-9196-754e901690b5:___body___" />

// ---------<LOS id="6de99478-2c5d-45a9-9196-754e901690b5:___body___" />

Within such sections, between “SOL” and “LOS” comments, you can freely write your code. SOLoist code generator will preserve such code when it is regenerated from the model. By using internally generated GUIDs, SOLoist relates these code sections with model elements (e.g., with operations or classes) to which they belong, in order to preserve them on next code generation.

SOLoist code generator provides such preserved code sections at many places, primarily within method bodies, but also within class declarations. You can use these sections to write methods or add other implementation details that are not part of the model. (This feature should not be misused, of course.) You should not write code outside these sections. You should also not change any other code outside these sections, especially the one generated from the model. All other code that is written outside such preserved sections will be lost on next code generation.

Apart from the files generated from the model (“managed” files, these are class files for the domain classes), other “unmanaged” free-style files with arbitrary Java code can also be part of the same application. SOLoist does not see those files and the maintenance of those files is completely up to the developer. For example, UI is written in such unmanaged files.

Database schema generated from the model is not aimed for any manual intervention, so the entire database schema is completely regenerated from the model every time the model is changed in a way that affects the database schema (for example, modifications of operations do not affect the database schema).

Q Configuration management and version control: How are SOLoist artifacts managed in terms of version control? For example, how can one compare and merge (and resolve potential conflicts between) two versions of the model from different development branches? How does this relate to the code generated from the model?

A: In general, version control is out of scope of SOLoist and completely transparent to it: SOLoist does not deal with version control at all. This is left to other tools. Usual version control tools can be (and should be) used in practice. Of course, this is an important practical issue, and we will briefly comment on how it affects development with SOLoist.

Files with Java code are managed by version control systems as usual, independently from SOLoist.

Managing different versions of model is left to the responsibility of the UML modeling tool (external to SOLoist) and version control system. Most UML modeling tools, and StarUML as well, store UML models in textual files. Therefore, standard version control systems can be used to manage versions. However, comparing and merging different versions of model, with resolving potential conflicts, although possible, is totally impractical if done by comparing textual forms (e.g., XML). It is the responsibility of the UML modeling tool to provide a better way for visually comparing and resolving conflicts between different versions of model. We are planning to support UML modeling tools other than StarUML with SOLoist code generator, and possibly to provide a separate tool for model diffing.

Of course, model files should be considered as integral parts of the system’s artifacts and should be stored in conjunction with other files in the version control system.

Q Database schema evolution: How is the schema evolution problem addressed? Namely, if I change the model, do I have to regenerate the entire database schema from scratch, or I can get an update script that does only the alterations in order to upgrade the database to the new model? What if I have a production system with lots of user’s data that have to be preserved and I still have to change the model?

A: The database schema is obtained automatically from the model by SOLoist schema generator. During development, when model changes are frequent and often substantial, and when there are only small amounts of test data present in the development database, while these test data are typically created by a program code that populates the database, it is most practical and completely convenient to regenerate the entire database schema from scratch and rerun the database population programs.

However, as soon as the system is put to production and populated with user’s production data that have to be preserved, another approach has to be taken. For these purposes, SOLoist offers two approaches and tools that support them:

1. You can export the entire SOLoist object space (the database contents maintained by SOLoist) from the existing (old) database to a set of XML files, then regenerate the entire database schema from the new model, and then import the object space from the XML files. You have to write the export/import utility yourself, but we plan to provide such a generic utility in the future. This approach is convenient for smaller applications and object spaces.

2. You can use a special SOLoist utility that does the following. It compares two models, the one corresponding to the existing (old) production database schema, and the new one to which the schema should be upgraded. The utility finds all differences between the models that affect the database schema and, with consulting the modeler in order to resolve potential ambiguities and avoid mistakes, resolves the differences and finds the modifications that have to be done to upgrade the database. It produces an SQL script consisting of ALTER statements that upgrade the database with preserving the existing data. It does not delete anything from the database, but simply renames the tables and columns in the database if they are deleted from the model, so they are not visible to SOLoist runtime any more.

Q Team work: How is team work organized with SOLoist?

A: Basically, SOLoist does not affect the organization of team work at all.

In particular, dealing with Java code and other artifacts in teams is the same as in traditional development.

As for modeling, most UML modeling tools (and StarUML as well) allow for dividing the model into separate units that are stored as separate textual files. These files can be then put into a central version control repository and dealt with in the usual manner, as all other artifacts. In particular, when a modeler wants to change a separate modeling unit, she:

• checks out that unit (as a file) from the central repository to her own private working space; exclusive locking of that unit is always recommended to avoid later conflicts; in SVN, for example, this means a simple update of the local copy from the central repository and a recommended exclusive lock;
• imports the checked-out unit into her own private copy of the entire model; this is actually just a conceptual operation, as it actually does not require any user’s action, because the UML modeling tool simply reads the part of the model from the corresponding file;
• changes (only) the part of the model that corresponds to the checked-out unit (this is always a UML package);
• when finished, checks in the unit back to the central repository; in SVN, for example, this is the “commit” operation; if the unit was exclusively locked, no conflicts will occur.

For smaller teams of up to 10 or 20 developers, it is sufficient and simpler to keep the entire model as a single unit. Typically, only one or a few persons do modeling and model changes then, so the entire model can be checked out and locked for modifications, while the lock can be passed from a person to a person as a token.

Q Other UML modeling tools: Does or will SOLoist support other UML modeling tools?

A: For the time being, SOLoists supports only StarUML. We are planning to support other UML modeling tools in the future. Interested UML modeling vendors are welcome to contact us.

In fact, the only dependences of SOLoist on StarUML are:

• SOLoist’s Java code generator, which is a plug-in for StarUML;
• SOLoist’s runtime reads XMI 1.3 exported from StarUML. Although XMI should be standard, we cannot guarantee full compatibility with XMI exported from other UML tools. In addition, we do not support XMI 2.

Q Model size and scalability: What is the size of UML models that SOLoist can handle? How does it scale in terms of model size?

A: SOLoist does not have limits in terms of model size. We have done systems with models with hundreds of classes and associations, and thousands of attributes. (Note that these are only domain (persistent) classes from the conceptual models, not any of UI-related ones. SOLoist models typically have much fewer classes than it is the case with systems developed in other technologies.)

Q SOLoist vs Hibernate, Java Persistence API, and others: SOLoist provides its own object-to-relational mapper (ORM) and runtime persistence layer. How does it compare with similar object persistence frameworks such as Hibernate, Java Persistence API (JPA), and similar?

A: Let’s start with similarities, of which there are only a few.

Of course, the main (and almost the sole) similarity is that SOLoist provides a classical ORM that maps classes and their relationships into a relational database schema, and a runtime object persistence layer that maps objects from Java address space into records of a relational database. Of course, they ensure full application independence from a specific DBMS. In addition, SOLoist applies optimistic concurrency control by using versions of objects and detecting conflicts of versions upon transaction commit. Finally, SOLoist also has its Object Query Language (OQL) with a SQL-like syntax, but different semantics (of OOIS UML).

The differences between SOLoist and other persistence frameworks are numerous, and we will mention just a few most important ones. They can be classified into several categories.

Semantics and usability. Other Java-based persistence frameworks are founded on the Java object semantics, while SOLoist is founded on the OOIS UML semantics of objects and their relationships. This has some important implications. For example, Java (just as all other traditional OO programming languages) recognizes only unidirectional references between objects. Therefore, in order to associate, for example, an employee (an object of class Employee) with a department (an object of class Department), you must write and execute both these two lines of code in Java to keep both sides in sync:

anEmployee.dept = aDepartment;
aDepartment.employees.add(anEmployee);

On the other hand, when you use SOLoist, you have to write and execute only one of the following two Java lines, either

anEmployee.dept.set(aDepartment);

or

aDepartment.employees.add(anEmployee);

Other persistence frameworks are heavily influenced by the underlying relational database technology and its semantics. They do not succeed in hiding all details and restrictions of the underlying implementation from the developer. As a consequence, a heavy burden falls on developers to take care about both the relational database and the Java object space, which are two semantically very different spaces. The application is then full of accidental complexity that is caused by the influence of the underlying implementation and its restriction. Let’s mention just a few examples of concerns that lead to accidental complexity in other persistence frameworks and do not exist in SOLoist at all.

a) Typically, every entity class must have a primary key (simple, composite or auto-generated). Therefore, developers must take care of providing a primary key in the database schema, and defining the details of the mapping to its field in Java, as well as of writing the boilerplate code like this:

@TableGenerator(
	name="employeeGen", table="EJB_ORDER_SEQUENCE_GENERATOR",
	pkColumnName="GEN_KEY", valueColumnName="GEN_VALUE",
	pkColumnValue="EMPLOYEE_ID", allocationSize=10
)
@Id
@GeneratedValue(strategy=GenerationType.TABLE, generator="employeeGen")
public Long getEmployeeID() { return employeeID; }

All this does not exist at all in SOLoist. According to the OOIS UML semantics, every object has its inherent identity that is completely independent of its attribute values and relationships, and is implied by its sole existence in time and space. SOLoist ensures a transparent, technical primary key (an ID of the database record) that it takes care of. Developers do not need to know that such a thing even exists. They access the object simply over Java references or over OQL queries, as they are used to. The ID is a completely hidden, implementation thing. (It still can be obtained through our API, if really necessary, but it is not generally.)

b) There are very often statements and rules like this:

All this is yet another occurrence of unnecessary burden and accidental complexity influenced by the underlying implementation technology (relational database). Again, the developer is required to be aware of all nuts and bolts of ORM. In SOLoist, the multiplicity is simply specified for an association in the UML model and SOLoist takes care of the rest.

Finally, as a consequence, other persistence frameworks inevitably put the developer in a position of having to be aware of the fact that a Java object and its database representation are separate items that have to be kept in sync by saving the Java object to the database or refreshing it from the database by loading it. For example, there is always boilerplate code like this (underlined):

@PersistenceContext
EntityManager em;
...
public Employee createEmployee(Department d) {
	Employee empl = new Employee();
	d.getEmployees().add(empl);
	em.persist(empl);
	return empl;
}

This is yet another instance of accidental complexity caused by the influence of the underlying implementation. In SOLoist, all these save() or persist(), load(), or entity managers are completely avoided. In SOLoist, you write what you really mean, something as simple as this:

Employee emp = new Employee();	// emp is already persistent.
				// No need for “persist” or “save”.
...
emp.dept.set(dept); // dept.members immediately includes emp.
...
emp.destroy(); // emp is removed from the object space (database).

As a conclusion, in SOLoist, an object is a unique, abstract entity that has its lifetime from its creation (by new) until its destruction (by destroy()); there is no any conceptual separation between the Java object (stored in a volatile memory) and its persistence counterpart (the record in a relational database), and there is no need for their synchronization. SOLoist provides a coherent semantic object space with OOIS UML semantics. Consequently, there is significant reduction in accidental complexity of applications and in bolierplate code. It is thus much easier to understand the semantics and write code, because developers do not have to think about the two spaces and their synchronization. Of course, this leads to less errors and easier maintenance. Finally, SOLoist's Object Query Language (OQL) is slightly different from its counterparts in other frameworks.

Implementation. Since SOLoist does not map the semantics of Java objects to a relational database, it has much more freedom in implementing the persistence layer. In particular, SOLoist objects in Java are not POJOs, and their fields do not carry real values of properties that are directly manipulated by the actions issued from the application’s code. Instead, they are just proxies. Fields of Java objects simply provide interfaces for OOIS UML actions on object’s properties, while the true values of properties and the implementation of actions upon them are moved to (and isolated in) the implementation of the persistence layer. This gives the implementation a flexibility to organize the data structures and operations in completely different and independent ways than it is the case in other technologies, where the true values are stored in Java domain object fields that are directly accessed from the application. Of course, this implies data structures that are better optimized for object-relational mapping and for performance of operations, as well as for concurrency control.

Performance and scalability. Our ORM and OQL compiler have some features and optimizations incorporated that lead to much better performance and more optimized compiled SQL queries in case of very large databases (with hundreds of millions of objects/records). Namely, it is well known that ORM and object queries may lead to very complex SQL queries with poor performance, especially in case of deep inheritance hierarchies of classes and multiple hops over relationships that produce many joins of database tables. SOLoist ORM incorporates several unique features to handle such cases that are not incorporated in other technologies. SQL queries generated by our OQL compiler outperform their counterparts in other technologies in terms of number of joins. This may lead to situations when, for huge data volumes and very complex queries, SOLoist OQL queries become executable in reasonable or very short time, while the equivalent queries from other technologies become intractable. We will publish more details on these features with concrete benchmarking results in a separate post.

Q Data volume: What is the volume of data SOLoist can handle?

A: SOLoist does not impose any limitation to the size of its object space (data volume). It leaves data handling to the relational database, trying to exploit its capabilities and specialization in data handling. We have experience with systems that successfully handled databases with 150-200 millions of objects, even of one single (base) class.

Q Custom UI components: The UI components provided with SOLoist cover some basic display, edit, and command UI elements. How can this be customized to enable complex web applications? Can I create my own custom UI component?

A: SOLoist’s library of built-in UI components is powerful enough to cover most of the functionality of complex Web applications. However, if you need your own custom UI component, you can easily create one by following instructions that we provide in the documentation. The custom component can do arbitrary work on the client and/or on the server side of your application. For example, it can embody an external widget (e.g., Google Maps or any other), or render its piece of HTML on the client on its own. What is most important is that such custom component stays in line with the overall paradigm and communicates with other components in a controlled and uniform manner (via pins and wires). To create your custom widget, you can use (inherit from) an existing one, or create your own from scratch. In the latter case, you basically have to do the following:

1. Create the UI component class that resides on the server side. This class has the usual SOLoist OOIS UML semantics and is instantiated on the server as any other UI component class. This class is responsible for maintaining the configuration parameters of the component in its attributes, as well as for serializing its configuration parameters into a so-called info structure that is transferred to the client.
2. Create the corresponding info structure that carries all necessary configuration data from the component to its representation on the client. This structure is a plain Java serializable class that is serialized by GWT.
3. Create the client-side controller class that is responsible to maintain the representation (through a GWT widget), handling events from the widget, and communicating with the component on the server, if necessary.

Q Wizard-like UI: Can I create a wizard-like UI, where the user is guided step by step through a set of screens to perform a certain business transaction that can be either cancelled or confirmed in the end, and how can I do that?

A: Yes, of course. We do that very often in our applications. The user’s experience is as usual, only the developer’s viewpoint is slightly different. Every screen of the wizard is simply a separate panel, and all steps are panels within the same parent deck panel. The usual “next” and “previous” buttons navigate through these panels in the deck by showing the corresponding step (the built-in deck panel component has the support for popping up the next or previous child panel in a sequence). If needed, you bind your input controls on the panels to the properties of SOLoist domain objects that store the values entered by the user. Upon user’s confirmation of the business transaction at the end of the wizard, you perform a command tied to a command button (“submit”). The command can then perform all the required business logic.

Q Localization: How does SOLoist support localization? In particular, in the tutorial, the labels are hardcoded into the components configuration code, but how can they be translated to other languages easily?

A: Yes, although labels are hardcoded, they still can be translated to other languages easily. SOLoist uses the following approach for localization. In order to make a string translatable, you simply hardcode your constant string (e.g., a label) in your string literal, but you should remember to add a dollar sign in it, as shown in the examples. This indicates that the SOLoist UI runtime engine should push that string through the translation engine before sending it from the server to the client. The translation engine is configured with a simple translation file that specifies the translation of each string to as many languages as you may need. Therefore, your programming is easier – you write your string literals directly in your code, in a “default language” that you use, and in an explicit and easy-to-read WYSIWYG form (instead of using identifiers of string constants that are less readable), and then you provide the translation mapping in a separate file. At runtime, the translation engine is simply configured with the given language, for which it selects the translation from the file. It is interesting to note that this mechanism works the same for string literals defined in the code, as well as for dynamic strings (e.g., stored in attributes of objects) with no difference – as soon as the translation engine encounters the dollar sign in any string that has to be passed from the server to the client, it makes the translation. (Of course, that string has to be found in the translation file, otherwise it remains intact.) Also note that the translation file can be provided or extended at any time later, and is not needed during the development: the code will execute perfectly well even with an incomplete translation file or even without it, as all strings are coded as simple Java literals.