In memory of Eliyahu Goldratt

Today searching web for goal programming sources I suddenly learned that last year Eliyahu Goldratt died. He was 64. I remember how impressed I was with his books “The Goal” and “Theory of Constraints” that helped me with an implementation of a custom supply chain system at the end of 90′s. I remember reading “The Goal” aloud to my wife – it was more than another technical book.  Not being a native English speaker, he used plain English to explain complex issues in a very unconventional way.

In the recent book “The Choice” he shares with his  daughter these thoughts: ”I smile and start to count on my fingers: One, people are good. Two, every conflict can be removed. Three, every situation, no matter how complex it initially looks, is exceedingly simple. Four, every situation can be substantially improved; even the sky is not the limit. Five, every person can reach a full life. Six, there is always a win-win solution. Shall I continue to count?“

I never met him in person. Once I wanted to attend his lecture but because of a conflict in my schedule I decided: “Another time.” Sometimes we think that smart people to learn from will always be around… At least, I frequently recommended his books to my colleagues and today I ordered his last book. Thank you, Eliyahu Goldratt.

Different Decision Tables for a Very Simple Use Case

The responses to the RFP of OMG  DMN (Decision Model and Notation) standard are scheduled to arrive in May of 2012. Trying to keep this process as close as possible to the everyday decision management reality, I posted several use cases at the DMN Discussion Group and tried to formulate down-to-earth questions that DMN will hopefully address. Here I want to take almost trivial rules and discuss different implementation options using decision tables.

We have already started to discuss this example in general rules management context with our friends from Sparkling  Logic.  So, this “trivial” problem can be described in plain English as follows:

“If a variable x is equal to 1 make it 2; and if it is equal to 2 make it 1”

Before you read further, I challenge you to represent this problem in your favorite programming language or in your favorite BR system with decision tables and without them.

Actually, many years ago this simple problem was given at an international student programming Olympiad. Here are some possible answers:

   (1)  x = 3 – x

The comment at the mentioned Olympiad was:  ”You are a mathematician but not a programmer“.

This “solution” had this comment: “You are not a programmer but probably a programming instructor“. I guess because people who do not program themselves frequently teach others how to do it.

The comment was something like “a normal programmer”.

From a rules management perspective, solution (1) is the worse one because looking at it you cannot recognize the initial “business” problem not mentioning that it does not allow any changes in the problem definition. A “good” solution (3) reflects thinking of a programmer who would not forget to put “else” before the second “if” like in the “bad” solution (2).

However, a business user (who is supposed to write and maintain business rules) in the most cases does not think as a programmer.  I believe it is more natural for a business person not to notice a programming error in the “bad” solution (2).  Carole-Ann offered this intuitive decision table as a possible solution:

However, she correctly noticed that everything depends on the execution logic of this simple decision table, e.g. ”the execution of this decision table in Rete mode will turn into an endless loop.”

Using OpenRules you may represent this table as

or

Any of these equivalent decision tables produce correct results because OpenRules table of the type “DecisionTable” is a single-hit table. It means OpenRulesEngine will execute rules from this table in the top-down order and it will stop its execution after hitting the very first true rule. So, if X is 1 then X will become 2 using the first rule and the second rule will be ignored.

OpenRules also supports two other types of decision tables “DecisionTable1” and “DecisionTable2”. The table of the type “DecisionTable1” is a multi-hit table that allows rules overrides. It means that before executing any rule inside such table, OpenRules will verify all conditions and only then for all rules with true-conditions will execute their actions. In our case if we replace a previous table with this one

It still will produce correct results. Why?  Even if X is 1 then X will become 2 using the first rule. However, the second rules will not be executed as its condition was validated as false before any rules execution.

And finally the table of the type “DecisionTable2” is a multi-hit table that executes rules in the top-down order one-by-one without any preliminary validation. It is very much similar to the traditional programming languages logic. So, if we replace a previous decision table with this one

it will produce invalid results. For example, if X is 1 then X will become 2 using the first rule. Then the second rule will be validated as true because a new value of X is already 2, and the second rule will make X to be equal to 1 again.

Along with predefined decision tables of types DecisionTable, DecisionTable1, and DecisionTable2, OpenRules allows you to design your own decision tables by defining your own execution logic in Java snippets. Here is an example of such table:

that will also produce the correct results. And finally, instead of any decision table you may simply write a snippet of Java code that will actually represent the solution (3) above. Here is an example of the proper table of the type “Method”:

CONCLUSION.The described representations of a very simple use case only prove the necessity of standardization of decision tables, and hopefully OMG  DMN will provide such a standard.

Note. Similarly to Rete-based rule engines, the initial decision table (offered by Carole-Ann) would not work with a constraint-based rule engine either – it will quickly diagnose an attempt to assign a different value to already instantiated constrained variable.

Dynamic Decision Glossaries

OpenRules decisions use business glossaries that are usually presented in Excel tables that may look like this table “glossary”:

The column “Variable Name” refers to decision variables mentioned in all rules (decision tables) under these names (any combination of words in plain English). The column “Business Concepts” refers to business concepts, to which the variables belong. Business users do not worry about how these concepts are defined – the same decision should continue to work in the same way independently whether these concepts are associated with Java objects or with Excel Data tables (used for testing). The column “Attributes” specify technical names of the attributes that represent variable names (no spaces allowed). For example, attributes may correspond to members of Java classes associated with business concepts.

The objective of this post is to explain the use of the OpenRules Glossary API for more complex practical situations. In large, real-world projects the actual content of business concepts such as the above “Customer” can be defined in external applications using Java-based Business Object Models or they may come from XML files, a database table, etc. The list of attributes inside business objects can be very large and/or to be defined dynamically.

In such cases, you do not want to repeat all attributes in your Excel-based glossary and then worry about keeping the glossary synchronized with an IT implementation. However, it is possible to programmatically define/extend the definition of the Glossary. For example, we may leave in the Excel’s glossary only statically defined business concepts and their decision variables, e.g. in the above table we may keep only the variables of the concept “Response” and remove all rows related to the concept “Customer”. Then in the Java module that creates an object “decision” of the predefined type Decision we may add the following code:

Decision decision = new Decision(fileName);
String[] attributes = getCustomerAttributes();
String businessConcept = “Customer”;
for (int i = 0; i < attributes.length; i++) {
      String varName = attributes[i].getName();
      decision.getGlossary().put(varName,businessConcept,varName);
}
…
decision.put(“customer”, customer);
decision.execute();

Here we assume that the method getCustomerAttributes() returns the names of attributes defined in the class Customer. The variable name and the attribute name are the same for simplicity – of course you may defined them differently.

You may add multiple concepts to the Glossary in a similar way. In  all cases keep in mind that the table “Glossary glossary” always has to be present in your Excel repository even when it contain no rows. You also may find that the same method “put(varName, businessConcept, attributeName)” of the class Glossary is used in the Glossary Template definition in the standard file “DecisionTemplates.xls”.

Representing and Solving Rule-Based Decision Models with Constraint Solvers

The latest rules conferences RulesFest-2011, BBC-2011, and RuleML-2011 were really great events in general and for OpenRules in particular. We announced a new constraint-based Rule Engine that is the first alternative to Rete-based implementations of a real inferencial rule engine.

The integrated Business Rules (BR) + Constraint Programming (CP) approach is supported by OpenRules Rule Solver for a while.  However, a new Rule Solver can directly execute decision models and can automatically support the inference principles. It means the following:

1) The order of rules inside decision tables does not matter. You may insert new rules anywhere inside a decision table and the execution results will be the same (even if your decision table does not cover all possible combinations, i.e. rules are not necessarily mutually exclusive)

2) The order of decision tables does not matter. Rule Solver automatically resolves inferential dependencies between decision tables.

Rule Solver considers a decision model as a direct input for a constraint solver (without any preliminary conversion to a rule language or to a constraint engine). Additionally, Rule Solver can effectively validate decision models for conflicts and completeness (across ALL decision tables). Rule Solver reports all errors in business terms of the original decision model. It also goes beyond regular decision models allowing a user not to describe all possible combinations of fact types to come up with a decision: some fact types may remain not instantiated and Rule Solver will automatically find a feasible or an optimal decision.

You may get more details from this, just published paper: “Representing and Solving Rule-Based Decision Models with Constraint Solvers”, F.Olken et al. (Eds.): RuleML 2011 – America, LNCS 7018, pp. 208-221, 2011, Springer-Verlag Berlin Heidelberg 2011. It is downloadable from http://openrules.com/pdf/RuleML2011.JacobFeldman.pdf.

The latest Rule Solver is based on the JSR-331 standard and can utilize different open source constraint solvers. It is currently in a test mode, and will become publicly available by the end of this year.

Rules vs. Decisions

There is a question posted yesterday at the OpenRules Discussion Group: “Is there a reason why you would steer actively away from Rules and into Decision?”

A simple answer is in another question: “I’m working on a project in which the business person wishes to have a strong control on his business rules and it may not be feasible to let the technical staff edit the sheets in order to code rules.  So, I ask you if the same example would be possible to be written without “hidden rows”.  Is there any situation in which the technical specialist doesn’t need to take part in “translating” the rules to programming language code?” posted by Monique Louise just a day earlier. Monique initially looked only at our Rules tables with separated technical and business views but probably not at Decision Tables with no coding.

I believe the question “Rules vs. Decisions” deserves more explanations. When we announced OpenRules-6 with Executable Decision Models, I posted my thoughts about “business rules” history: “When expert systems came back in 1996-97 under a new name “Business Rules”, the main promise that gave this movement a new life was: “Return Control over Business Logic back to Business People!”

In OpenRules, our initial intention was to accommodate cooperative work of subject matter experts and software developers while defining and managing business decision logic.  From the very beginning, we decided not to develop (again!) a proprietary, “Excel-like” rule editing GUIs but rather let users to directly use Excel with Java snippets. Thus, our, now highly popular, “Rules” tables with business and technical views came to life. Inside a hidden technical view, developers may use Java directly (without any artificial rule language) to describe semantics of rule conditions and actions of any complexity.  In 2007 we introduced Rule Templates which allowed our users to move technical views to a limited number of templates while hundreds and even thousands of rules were created based on these templates.  It nicely separated rules management responsibilities:  IT specialists help to create templates and business specialists create and maintain rules based on them.

We did not “steer away” from this approach and continue to actively support Rules and Templates.  Many of our customers really like this approach with a nice cooperation between business and IT departments. However, it still keeps IT involved into business logic definition and maintenance.

With OpenRules-6 our customers got an opportunity to use new tables to represent  Decisions, a business Glossary, and different types of Decision Tables with zero coding. “No hidden code” also means that business people can define and maintain their business logic by themselves with no IT involvement. Now, IT participation may be limited to an integration of decision models, created and tested by business analysts, into a specific IT environment. Developers do not even have to look at Decision Tables. We certainly promote this approach as it brings all of us closer to the initial promise of the Business Rules movement.

Now, OpenRules customers have a choice between Rules and Decisions (with Glossary and Decision Tables). They even may mix both approaches: we already see the situations when customers start with Decisions but add their own Rules whenever necessary (taking an advantage of the fact that they always have an access to objects defined in the Glossary). They also may customize the decision templates (available in Excel) to add an additional functionality. And when customers have a choice, they will always find the best alternative for their particular technical and organizational environment.

Fall-2011 Business Rules Conferences

In Oct-Nov 2011 OpenRules will participate in the following major Business Rules events:

1. Rules Fest 2011, October 24-27, 2011, Burlingame, CA. OpenRules is among organizers of this conference that was created “By Developers for Developers”.  I will present the latest OpenRules advances for ”Using Constraint Solvers as Inference Engines to Validate and Execute Rules-based Decision Models“. OpenRules customers can use the Discount code !OpenRules! during the registration and 10% discount will be applied at checkout.

2. Business Rules Forum 2011, October 30 – November 3, 2011, Fort Lauderdale, FL. As for many year, OpenRules, Inc. will again be a sponsor of this event. So, all OpenRules customers are eligible to a special 15% registration discount: click here and use the discount code OPENBBC15. If you attend, stop by at OpenRules exhibition booth T7 and listen our presentation ”How Business Analysts Create, Validate, Test, and Execute Tabular Decision Models without IT Involvement” on Nov 2 at 4:30 pm.

3. RuleML 2011, The 5th International Symposium on Rules, November 3-5, 2011, Fort Lauderdale, FL. Attend our presentation “Representing and Solving Rules-based Decision Models with Constraint Solvers“

OpenRules 6.1.1 is available for downloads

A new release 6.1.1 further advances the Executable Decision Model introduced by the OpenRules-6 in March 2011. It provides new features based on the real-world feedback. In particular, the release 6.1.1 includes more advanced templates for for Decisions, Decision Tables, and Glossaries. It also allows a user to go beyond the limitations of the Decision Model by adding different kinds of decision tables that support rules overrides.

The release 6.1.1 includes a very detailed User Manual (~150 pages) and several tutorials that hopefully will be appreciated by our customers. It provides an ability to compare values within decision tables with allowable values described in the glossaries as possible domains. These domains will be more effectively used in the oncoming new version of the Rule Solver that will be tightly integrated with decision models. As usual, the new release fixes some bugs – see Release Notes. You may download a new release from here. Post your comments and questions at the newly created Google OpenRules Group.

Forrester about OpenRules

On July 5, 2011 Forrester Research published a report “Market Overview: Business Rules Platforms 2011″. Here is what it says about OpenRules:

“OpenRules have the most-aggressive approaches to business-expert authoring and typically requires less developer support than IBM ILOG, FICO Blaze Advisor, and JBoss BRMS.“

Considering that we never asked for such an endorsement, it is really nice to get it from one of the most reputable market analysis firms.

James Taylor blogs about OpenRules

James Taylor posted several articles devoted to OpenRules BDMS:
- General Overview
- Rule Solver
- Rule Learner. 

I just want to add a news which we have not shared with James yet:  our latest release of Rule Learner can also present automatically discovered rules in the PMML format.

About OpenRules Scalability

Being in real-world production environment for many years, OpenRules Engine has a proven record of high efficiency and scalability. Several years ago some of our customers (a major European bank and a large government agency) assigned teams of people to do stress-testing of our product before they decided to use it instead of commercial counter-parts. The results were really good. For example, one report stated that OpenRules scaled to more than 300 concurrent users connected to one server providing us with the following results:

Actually we were not surprised, because by design one instance of the OpenRulesEngine can serve any number of concurrent user sessions. In 2006 an independent Japanese company reported efficiency comparison results of OpenRules with major commercial engines and they were in our favor even when other engines used a sequential mode. Especially good were OpenRules’ minimal memory requirements on the server.  The explanation was simple: with OpenRules you do not need a pool of rule engines and as a result, one instance of the OpenRulesEngine uses only minimally necessary memory to keep only single representation of all used rules.

In 2007 OpenRules introduced parameterized rules repositories that allowed a user to keep in memory only those versions of rules that expected to be actually used.  It further minimized memory requirements and improved OpenRules scalability.

In 2009 we made an additional improvement of OpenRulesEngine’s efficiency in multi-threaded environments supporting true parallelism.  Since then OpenRulesEngine constantly shows great efficiency and scalability in real-world multi-transactional environments.

However, last week our long-time customer noticed a growth of the server memory as they started to add more and more concurrent users.  The problem was not with the core OpenRulesEngine but rather with OpenRules Forms that became a foundation for their quite complex web-based GUI. OpenRules Forms uses the JSP technology incorporated into OpenRules BDMS. A user, who is not necessarily familiar with HTML or web design techniques, can use Excel to define layouts of all web pages in simple Layout tables along with validation and navigation logic presented in Rules tables.  By default, simple examples of web applications included in the OpenRules’ installation usually start a new engine for each new user. Our support team quickly switched to a mode that was supposed to use only one instance of the underlying OpenRulesEngine – it required changes only in the starting “index.jsp” file: we just have to be careful following the double-checked locking design pattern. However, we quickly found that the way we used our internal class Dialog (that maintains the state of every user session) did not work properly with a single instance of the OpenRulesEngine that serves concurrent sessions. It took a few nervous days but the problem was fixed, and now we again have a happy customer who is reporting no degradation in memory and well synchronized concurrent sessions.  If other customers experience any similar problems with OpenRules Forms or ORD, please contact support@openrules.com to receive this fix. The correct implementation will be included in the next release.