I watched on the sidelines, not long ago, as a team of architects carefully inspected and examined different technologies for managing a rules engine. I found it interesting, but not terribly pertinent, because…well… to be honest… rules engines tend to create more problems than they solve.
So let’s look at the problem we are trying to solve with a rules engine.
When an application is written, it will encapsulate some business rules related to the problems it is trying to solve. Assuming it is valuable, continued investment will occur. As this happens, more business rules will be placed in the application. Unfortunately, these rules can be applied at different points in the system’s architecture (user interface, composition layer, service layer, data layer, and in data migration processes).
Some rules apply in the user interface. We may decide that a field should be implemented as a drop-down list. Why? Because the data entered in that field must be “known” already to the application, either for integration purposes or just to reduce data entry error. That is a business rule. We could allow free form input in a user interface field… and then apply editing rules (like date formatting, or putting the spaces and dashes in phone numbers). That is a rule as well, especially if the user interface is data driven.
We may decide to validate data at the middle tier or services layer. For example, in some situations, I can only submit an order for products if I provide the agreement number that I have signed with vendor. That agreement specifies lots of things, like terms of payment and perhaps pricing and rebate rules. So what if I enter an order but provide an invalid or expired agreement number? There would be rules for this as well.
There are a couple of problems that rules engines are designed to solve:
- If a business rule needs to change, and it is implemented in many places, potentially in many applications, then it is tedious and expensive to change it. This slows down the business and increases the cost of agility.
- If an existing application will take on new needs, then new business rules may need to be added to it. This can increase the complexity of the application substantially.
- Business rules often drive tight coupling between systems, especially in an integrated environment. If I am going to pass data from system FOO to system BAR, and I want to make sure that the data will be acceptable to system BAR, I may be tempted (or required) to validate the data in FOO using the business rules from BAR. The person who write the code for those rules in BAR is long gone from the department. The expense of making sure those rules are in sync, and kept in sync, can be high.
These are very valid problems, and a rules engines propose to solve it by providing interesting mechanisms. They include the ability to pass values to the engine and have it calculate a result that can be understood by the caller. This allows isolation, to a point, because the calculation itself can be changed. You can also place process rules into a rules engine, so that you pass in the state machine information and one or more inputs or events, and the state machine reacts by sending out events and changing state. This is the core concept of workflow components.
That said, I think there are very narrow uses where rules engines are actually a good idea. Many folks argue that workflow engines are essentially a subclass of rules engines, and Workflow is a good idea to isolate. Why? Because writing parallel workflow capabilities into your code, unless you are an expert in Petri-Nets, is HARD. What you are really encapsulating is not the data, or even the process, but the capability of executing the process properly. I’m not even sure if I consider workflow engines to be a subclass of rules engines, given this fact, and the remainder of this blog post specifically excludes workflow engines or any other ‘rules engine’ where “how” you execute is more difficult to manage than “what” you execute.
The generic rules engine, on the other hand, is not so specific. More often than not, rules engine proponents say “use the engine for encapsulating the rules, and allow them to be executed here.” Nice idea. Too bad it doesn’t work.
The problem with making it work is, as always, in the details. In order to delegate the execution of rules, they have to be rules that are efficient to delegate (there goes user interface editing rules), not specifically associated with data coupling (there goes the problem of passing in domain data for a drop-down box), and describable from the standpoint of an algorithm or formula (there goes error handling rules). In addition, the algorithm sometimes has to be encoded in a programming language that is executed as script. This is slow and inefficient.
A much better approach is to create a set of strategy patterns to be used rules validation, write code that implements those patterns, and inject that code, at run-time, into the executing environment.
Write the rules as small bits of code, carefully controlled, adopting an interface that is called in a standard manner. Your data drives your system to use the code module. Inside the module, you have a good bit of freedom to figure out what you want to accomplish. You can even share ‘global’ values across code modules if the framework is put together well.
Note: this is not a rules engine. It is a rules framework. Rules engines execute the rules. A framework merely gives you the ability to control their instances. Your app directly executes them.
This is the basic idea behind event driven programming! Nothing new there. I’m just suggesting that you use a framework to do it, so that systems can change at run time by changing configuration files.
For those folks who don’t know what I mean by ‘inject,’ that means that you set up configuration in text files (presumably XML) that declares what code module contains your rules classes, all of which implement the proper interfaces. Then, your system uses that configuration data to load those modules at the right time and keep them around for rules validation.
This is far better than using a rules engine. What’s odd is that I haven’t seen many comparisons of the two, yet dependency injection has clearly won the battle. Over the years, a lot more code has been written to be injected than to call out to external systems for execution.
So why bother to revisit this topic? To declary victory for dependency injection and kill the generic ‘rules engine’ concept completely. Dependency injection won. Let’s not waste any more time discussing generic rules engines.
I agree that a rules engine is a bad idea most of the time but what about the scenario where you have a lot of complex business descisions that frequently must be modified by "business users" i.e the rules need a tight edit/test/deployment-cycle separated from your supporting IT-systems? (eg. rules for credit assessment, incurance policies etc.)
I guess those cases are rare but I can see some use of a BRMS in the above examples.
Hi Andreas,
Even then, consider this. If I have an insurance policy system, I want to be able to configure some fairly complex rules. With some Commonality – Variability Analysis (CVA), you can usually distill out a set of numeric values or a table that drives the logic. You write the component that consumes the data, and you put the data in a database.
You still want to have your app use configuration data to select in the component, but for over 80% of the changes, data changes are all you actually need. If your logic changes enough to need to change the code, then you WANT a longer test cycle.
As I said in the post itself, there are probably narrow cases for a component that is more general. However, for over 95% of the applications, a rules engine is just not an effective way to spend your money.
I agree with what you’re saying, as I have recently had that issue with integrating rules in a frontend (driving user control validation) and backend (validating data before saving).
Current rule engine generators do not address this, but I think with the enterprise library attacking business rules, hopefully we might see more integration of the tiers and common rules.
We ended up making our own rule validation engine where we create one ruleset and it can be used in the backend, or front end. If in the frontend, then any controls bound to data which is associated with rules are automatically affected and display validation of one or multiple rules.
And we can create a ruleset with rules that a ui wants, and have a backend use those rules and create additional ones that are only relevant for the backend.
So I do not necessarily think rule engines are bad – just that they are currently not thought out very well for different tiers.
Still real tired from my Oklahoma trip , partying with Raymond sure is exhausting-). Agile/Development
It appears that Nick Malik is out to motivate that rules engines are not as useful as dependency injection.
Couldn’t this idea, though it doesn’t conform to traditional "rules engine" ideas, just be called a new rules engine? 😛
How about a hybrid? I see a UI where business users edit business concepts, ie. change values etc., these are persisted (wherever – xml / db / etc) and your strategy-classes can be hydrated from this persisted data and passed on to the execution engine.
Nick Malik published a thought-provoking article on rules engines recently. You can read…
Nick, thanks for a really interesting article. However, I strongly disagree. In reality, many modern rules engines offer an excellent platform for implementing rules using dependency injection patterns. There is no ‘battle’, therefore, between the two approaches. There is, however, a marked degree of ignorance about what rules engines offer. I’ve written up my thoughts and posted them at http://geekswithblogs.net/cyoung/articles/108411.aspx.
Hi Charles,
I read your blog reply and attempted to add the following text as a comment, but unfortunately, the blog software gave me an exception message. So I’m posting it here. I hope you see it.
Wow. I take it as a compliment that you took the time to offer such a lucid and detailed counterpoint to my post. I respect your opinion and I’m glad you shared it. (as you probably guessed, I’m not swayed by your arguments, but I am impressed with amount of detail and considerable knowledge of your field you offer.)
Thanks,
— Nick
Interesting post though I think you miss the point on rules. It is not (mostly) about executing the rules but about managing and evolving them. That’s where business rules management systems (which include a rules engine) shine and outperform any pattern that involves writing a bunch of code 😉
JT
sorry to interrupt.very curious topic! I’m an application developer and confused a little. Nick is talking about just Rule Engine,right? jamet123 is talking about BRMS,isn’t it? I just have thought that dependency injection and Rule Engine help each other on the application code, by the way.
Hi JT,
I believe that my post mentioned the "point" of isolating the rules so that they can be managed. I believe I also made the "point" that only a small subset of the actual rules CAN effectively be encoded in a rules engine. That means that some rules are still in the "bunch of code," reducing the effectiveness of the whole rules-engine concept.
Of the 20% of the rules that an engine can handle, then a BRMS is useful. For the other 80%, it’s useless.
However, when you decide to embed a rules engine in an application, you increase the cost to the application, and decrease the performance, efficiency and maintainability dramatically. Those are prices that you pay to do what… maintain a subset of the rules?
The price is high and the benefit is low. QED
To Shige and Ernst,
Thank you for your thoughtful responses. Yes, injection is often used inside rules engines, and they are not incompatible.
The point I was trying to make is that any situation that forces you to go out of process to execute the rules is counterproductive. A framework, on the other hand, that runs in process with the app, that allows injection of modules that implement rules, that is fine. A framework inside the rules engine, even if it allows injection of an execution module, is still outside the process boundaries.
Frameworks are fine. It is cool if some of the "out of the box" functionality in that framework is the ability to execute scripted rules or generic rules, because (a) the value of the engine is not in the generic execution but in the injection framework, and (b) you can execute the rules in the location in the code where it is needed without having to package up the current system state and pass it out of process for evaluation.
I’ve seen rules engines work… when the application is written in the rules engine environment itself. In that case, you have the rules running in process with the app. It is the parallel to my concept. If you can’t take the mountain to the man, take the man to the mountain.
Still real tired from my Oklahoma trip , partying with Raymond sure is exhausting-). Agile/Development Tools On my short list for some time now, is to switch from NUnit to the definitely superior MbUnit. My friend Andrew has done some great work with