The other trend we are seeing more of in this decade is the use of more events, and less tasks. A task that says that something happened is not a task at all; it’s an event.   To be a task it has to be something that is performed by a person, system, or process.  One could argue that everything is performed somehow, so everything is a task.  So let me ask you this:  Is it a task for the weather when it rains?  Is it a task for the highway that traffic is backed up?   Is it a task for the stock market when the NASDAQ drops by 100 points?    If I can’t put a performer to the task, it can’t be a task.



Events can basically do two things: start activity or interrupt activity.  The weather changed, so what are you going to do about it?  The traffic is bad, so maybe try another route?   The stock market is down, so maybe you should buy stocks (unlike the herds of people who sell every time there is a jitter in the market).

Condition or event?

You could also argue that the bad weather is a condition, not an event, and you might be right. But what caused the weather to be bad?   It was likely an event.  And when the condition exists, is that not an event?  The BPMN specification says that a condition is a category of event.   There are two types of condition events; start and intermediate.   As stated above, an event can either initiate activity, or interrupt activity.  Also in the BPMN 2.0 specification we have the new non-interrupting start and intermediate variations of the condition event.  These shapes still serve the same purpose, but also add context to where they can be used.   For example, an intermediate non-interrupting shape is essentially a means to start an activity or flow that is relevant to the subprocess the event is attached to.

In the world of event processing you have essentially two things; events and conditions.  Conditions describe a combination of one or more events.  For example, it’s raining outside and the traffic is terrible.  This is a condition that possibly the weather caused the terrible traffic.  Although, one could argue that all of the cars are causing the weather to get worse (global warming). But that’s another topic entirely.   The two conditions combined of traffic and weather cause an event of something I was looking for, and the conditions are correlated as an event in time.  Weather can be bad on its own, and so can traffic.  The two are not necessarily connected until a stream of events (which we could also call facts) are correlated. For example, a weather report in combination with an emergency dispatch of a car hydroplaning due to excessive water on the road.   With respect to both of these events occurring within a window of time, a condition is born.

Conditions act as a filter of events.  Billions of trillions of events occur every millisecond.  But obviously we are not interested in all events; only the ones that are relevant to our business process activity.  When I figure out a way to filter the events down to something interesting then I have a “condition event”.  In other words, the billions of events have been aggregated down to just one single event that is important to my process.

BPM: meet CEP.  CEP: meet BPM.

Now that introductions are made, let’s talk about what they are and why they are both BPM and CEP are very important and relevant in this decade.


BPM was not designed to handle billions of events.  The BPMN notation is far too simple to handle the sophistication of monitoring millions of stock ticker streams, or monitoring millions of cars per day travelling on a highway.  The individual events are so insignificant that they go largely unnoticed by the larger business processes that everyone is familiar with.  Up until last decade we’ve simply called this an application and didn’t bother modeling it.  But now that Complex Event Processing (CEP) is coming of age, there is a new approach emerging in process modeling that efficiently handles complex events.
Up until recently I honestly couldn’t figure out how to use the condition event in a real-world process model.  The difference is that lately I’ve been experimenting with complex event processing concepts.  Suddenly I realized that a condition is the result of a complex event, then it was easy to put conditional events everywhere.

The job of the complex event system is filter through millions or even billions of events per second and find something interesting that I might want to act upon.  Once captured, this event causes a so-called complex event.  So is it an event or a condition?  Why not a condition event?    This led to my new nickname for the CEP acronym.  Instead of calling it “complex” I call it “conditional event processing”.  I wonder if this will catch on?   Probably not, but at least this might help you make sense of all this.


When CEP generates an event, BPM decides what to do with it.  There are two basic use cases here; you can send the event to a process participant, or the BPM system can further aggregate the condition into a decision of whether or not to take action.  For example, a condition event is detected, which causes a flow into a rule, which determines that either no action is required, or an activity should be routed to a participant.   However, if either one of these paths are taken too often, this is yet again another condition that could be used.  For example, too many condition events are causing too many people tasks, and the organization is overloaded with activity.  This overload condition can cause a feedback to the CEP system to relax the thresholds in which it triggers its complex events.   This is an environment where BPM and CEP help each other do what they do best.

Even Olympic size pools only have a limited number of lanes.

I know that if I suggest that swimlanes will be a thing of the past I’ll never hear the end of it.  So go ahead and start your comments now because yes, I’m about to go there.  I’m not saying we’ll see our friend the swimlane disappear anytime soon.  But I am saying that swimlanes are becoming less relevant.  In an event based world, we don’t necessarily have a performer of a task until runtime.  I don’t know who is going to do what.  So why would I model my process in a way that assumes a particular person is doing something?


Swimlanes have their place.  I stand by my advice that if you have more than five lanes in a pool you seriously need to stop and think about what you are doing.  Also, if you have more than five pools in a diagram, a problem should leap off the page and smack you in the face.  Organizations are not flat like this, and in practice, more than 5 roles would be unmanageable.  Instead, a hierarchy exists to manage the complexity.  So what I’m suggesting here is to select process participants that are more in line with how the organization works.


If you have too much activity happening in one place (a diagram) it could be best described as chaos.   Chaos is complex, and there are likely to be way too many events to process with BPM, or draw with BPMN.  You cannot possibly draw all of the events and conditions that might occur in a diagram that has more than 5 or 6 participants.  Likewise, for a single participant, if you are subdividing the role into many lanes there are likely way too many events to handle (too much participation for one participant).  So why not try the event approach instead?

Understanding complexity, events, conditions, and process activity

For better understanding, let’s take a look at a complexity analogy.  Imagine a large, crowded room full of people having a formal dinner evening.   The attendees (participants) walk around the room introducing themselves to others, and conversations begin. As the conversations increase, the noise increases, and you can no longer hear a conversation more than a meter or two from where you are standing.  A simple task such as getting to the dinner buffet is interrupted by hundreds of events; people bumping into you and trying to cut line ahead of you.  As a participant in this dinner process, if you happen to notice it’s getting quiet in the room, it might be a good idea to stop shouting at the top of your voice because someone probably has an announcement to make.  Or, if you happen to notice everyone running to the exit, maybe you should go too.  There might be a fire.

The point to this analogy is that most of the dinner evening was not planned.  Instead, it was a series of events triggering micro processes.  The overall objective of socializing, rubbing elbows with the important people in the room, and having a good time was achieved.  Everything else was random occurrence.  But even in the randomness there is order and process. For example, everyone got in line when it was time for the dinner buffet, and the other agenda items occurred according to schedule.

The basics of management theory state that the more people are involved in a meeting or gathering, the less productive the outcome.   The same can be said for processes.  By keeping the number of swimlanes down to just the important participants, you can actually show more relevant detail.  The other participants are involved, but there is no point in showing detail of what they do; it’s out of scope.  In the dinner evening process above, it only makes sense to make a process of the over-all agenda.  There might be hundreds of participants in the ceremony, but only a few are important.

At the same time, it’s important that all of the guests are having a good time.  For example, the host detects a high percentage of people complaining about the food or getting sick.  This might be a problem that could be fixed before the party is a disaster.  But the only way to know about this condition is to mingle with the crowd and ask everyone if they are having a good time.  Too many people not having a good time (individual events) is a condition that can be brought to the attention of the organization (the process context).

Event driven business processes

The same can be said for many processes in the business world today.  There is the general high-level process and all of the subprocesses that support the main objective.  In addition, there should be some sort of feedback mechanism to govern the process flow.  Otherwise, the highest level objectives will likely fail.   For example, if I have a manufacturing business and I don’t watch the market for signs of growth or slowing, how do I know how many units to make?  And when I slow down production, do I simply cut back on my labor force, or will that negatively affect my ability to operate?   These are questions that cannot be solved in the process modeling realm; it’s an event problem.  However, these events, filtered into conditions, are something that can be modeled.  I can set up market indicator monitors that create condition events, which tell me when to speed up or slow down production.  I can also set up a means to collect information on bottlenecks in my production line, and correlate this data with employee morale data from the HR department.   Too few people on staff can cost me just as much as too many (overtime, employee retention and training, product quality issues, etc).

The old approach to process modeling was to go about business  as usual and hope that everything works out.  The new approach with the event-enabled process is to provide a way to enable the process to self-optimize.  This is not to say that everything can be automated.  But the information needs to get back to the people who make the decisions, at the right time, and filtered to what is relevant.  As organizations become larger, the events become more important.  This is because the lines of communication are long, and much information is lost in the chain of command.  We cannot model the entire process in one big picture, but we can model the events and conditions that affect related processes.

-    Rick Geneva

Continuing coverage of the BPMN 2.0 specification, this post will focus on the new event types.   Prior to 2.0, BPMN had several problems when it came to escalating events and dealing with events in parallel.  Often I would end up using a pattern with multiple loops inside of loops to accomplish seemingly simple activities. Furthermore, it was difficult to distinguish between human centric and system centric activity for a mixed-mode diagram that includes both.

One goal of BPMN is to bring the SOA camp, the business analysts,  and the process modeling communities closer together.  A side effect of this has been that BPMN is very messaging intensive. For complex interactions, multiple pools are used, which requires lots of messaging lines to keep activity in sync between participants .  This is sometimes a problem for people who come from a flowchart/workflow backround using tools such as Visio, which essentially allows you to create a really bad BPMN diagram due to the lack of diagram validation.  Often I see messaging lines within a pool going between lanes, when what was really intended is to do a simple escalation.  Antoher common problem prior to BPMN 2.0 is showing enough detail without having to show the intricate patterns of looping and dealing with multiple events in parallel.


We are starting to see a shift from multiple pools and lanes to more of a style based on a single pool with no lanes, which means less explicit messaging notation. On the other hand, we are starting to see more capability in BPMN to document highly technical processes for the SOA community.


New Event Types

In this post I will cover the new BPMN escalation event and parallel multiple event.

Escalation event

The escalation event is new in the BPMN 2.0 specification.  The shape looks like the Star Trek symbol.  I’m not sure what you call it, but it somewhat resembles an upward facing arrowhead.  The full pallet of escalation events are available.  However, it’s important to note that both start (interrupting/non-interrupting) and both intermediate types can only be used with subprocesses. Only the throwing shapes can be used within normal sequence flow.   This implies that escalation can only be thrown from within a subprocess that has a catching shape.  The intermediates are used on the subprocess border, whereas the start types are used within the subprocess.  Note that you should review non-interrupting events in my previous post for more details on how non-interrupting events work.

BPMN 2.0 Escalation Events

Prior to the introduction of the escalation event, the only way to effectively accomplish the equivalent behavior was to throw an exception, or use a timer on a subprocess border. Both of these alternate patterns introduce potentially unwanted side effects.   If the exception event is used, there is no non-interrupting catching shape. Therefore the parent scope would always be cancelled, and only exceptional flow would be executed. But in the case of escalation, most often this is not the case.  For example, a participant is working on a problem which requires escalation to a manager. The original participant will continue the work, although the manager is now involved.  This is a collaborative escalation mode which requires a dynamic parallel flow, generated by the non-interrupting escalation event.  This would not be possible with an exception event.  Furthermore, a timer would not be appropriate in this case either because the event is not based on time. Instead, it’s based on a condition generated during the execution of a task, probably by the participant performing the task.

Escalation Event Example (non-interrupting)

The example above shows the non-interrupting start event catching the intermediate throw escalation event.  Again, note that if the start escalation event were interrupting (solid), the step “do more work” would not occur.   The start and intermediate catching events can only be used within or attached to a subprocess border.  To further clarify this behavior, study the following example:

Intermediate Escalation example

In this example the handler is on the subprocess border.  Technically this process will execute identically to the one above, with one easily overlooked difference.  The participant handling the escalation in example 1 can directly collaborate with the other active tasks, because they are both in the same scope.  Example 2 has an event handler that is external to the subprocess, which means that the handler does not have access to the same data and event scope as the other active tasks.  All communication between tasks would have to be handled in some other way.  Also note that external synchronization back into the main flow is required, because two parallel paths were created. Failure to synchronize would have the same consequences as explicitly using a parallel split with no merge. The inclusive merge is used because at design time we won’t know if the escalation path will occur or not.  Got it?  If not, bear with me for a few more posts and I’ll get back to these patterns.  For now I”m only covering the spec.

Parallel Multiple Event (inclusive behavior)

As of BPMN 1.2 the multiple event had only exclusive behavior, meaning that a single qualifying event would trigger the shape.  The inner shape of the event is a pentagon.

Multiple Events: Exclusive Behavior

Adding to this shape, you can see the new non-interrupting variety of both the start and intermediate shapes.   Refer to non-interrupting events in my previous post for details on how non-interrupting events work.


BPMN 2.0 introduces the parallel multiple event, which has inclusive behavior, meaning that all related events are required in order to trigger the shape.   There is no throwing type for this shape, because a parallel throwing multiple event would probably raise more questions than it solves.  Anyway, there are interrupting and non-interrupting varieties of the start and intermediate types.

Parallel Multiple Event shapes

Officially the specification calls this a parallel event, because it must receive all events to trigger the shape.  To help eliminate confusion I’ve also been referring to the behavior as inclusive, meaning that all events are included.  But actually, the parallel behavior is much closer to the truth of the use case.  The term ‘parallel’ is somewhat ambiguous because it might imply that all events must be received at the same time.  In reality, parallelism in BPMN does not mean that activities execute at the same time. Instead it means that multiple activities or events are enabled in parallel, and have the potential to occur at the same time.   Just because a task or event is enabled doesn’t necessarily mean that it will execute.  A parallel multiple event requires that all events in the correlation set to be triggered before the condition is satisfied.  Only then will the entire event trigger be fulfilled.

The epitome of complexity

The usefulness of this shape is debatable.  Most BPM vendors have not added support for many of the BPMN shapes because they just don’t make sense for an executable system.  In fact, about half of the BPMS vendors on the market today are using only a subset of BPMN 1.0 or 1.1.   To some, this might seem like a negative point towards the vendors.  But the reality is that most people have a hard enough time understanding the basic subset of BPMN, let alone something as complex as a “non-interrupting parallel multiple start event”.  For those of you just tuning in, this would mean that I have a generic shape acting as a filter for a bunch of specific events, and if this condition were to occur, launch an exceptional flow, in parallel, to deal with the problem.  Otherwise, keep working as normal.   Now see, that wasn’t so bad, was it?


For this shape to actually be useful in a diagram, you are doing one of two things; being purposefully vague (shorthand notation), or you are implying some sort of CEP (complex event processing) mechanism.   So, as I mentioned earlier, BPMN is getting simultaneously easier to use, and more complex at the same time.  Something as simple as a rope can be quite useful, but people somehow manage to hang themselves with it every day.

- Rick Geneva

BPMN 2.0 adds a lot of new concepts.  Many of these are long overdue to be added to the spec.  Others are a totally new concept.  In this post we are going to take a look at one of the long-overdue fixes to the BPMN specification; the intermediate events on the subprocess border.

In previous versions of BPMN, placing an event on the subprocess border meant that when the event was triggered, the exceptional flow would become active.  Also, this means that normal flow stops.  In the diagram below, subprocess A ceases when the timer event is triggered.  Instead, the “handle timeout” subprocess is active at that point.  In other words, subprocess A has been interrupted.

Intermediate Interrupting Event

There is another use case that BPMN 1.2 did not cover.  What if the timer event isn’t supposed to interrupt subprocess A?   Just to clarify, in the BPMN 2.0 specification, Interrupt means that the parent subprocess will end.  This is similar to a cancel, but cancellation is another concept and another shape entirely.  So it’s called interruption.  There is another use case for non-interrupting events that has long been a challenge prior to 2.0.

Interrupting and Non-Interrupting Events

Introduced in BPMN 2.0 is the concept of non-interrupting events.  They are illustrated with a dashed line instead of a solid line.

Intermediate Events (catching)

The difference is that when the event is triggered, the exceptional flow occurs in parallel to the main flow.  This means that you need to be careful how you use it.  Special attention is needed to how the two paths merge.  I recommend treating it just like any other parallel situation, using a gateway to merge.  But which gateway?  Parallel would be the obvious choice.  But remember that a parallel considers all paths to be active on the merge. This would cause a process deadlock if the event was never triggered.   Instead, the recommended merge shape is the inclusive gateway.

Intermediate Non-Interrupting Event

The gateway is used for flow control.  Without it, a potential exists that someone could misinterpret the diagram. I never recommend using implicit merge behavior because the risk is high that it might be misinterpreted.  This is especially true in this case because a small change such as a dashed line now means that we have a potential parallel situation. Most people new to BPMN won’t know this, and people not yet familiar with BPMN 2.0 might not spot this subtle, but critical difference.

Non-interrupting start event

This is where it gets complicated.  I had to read this part of the specification several times to figure out how a start event could be non-interrupting.  There are some small examples in the specification, but to fully understand what’s happening here we’ll have to talk about scopes of subprocess and many other concepts.  For now I’ll just introduce the idea and if you have any questions, please feel free to comment and I will reply.

First, let’s talk about scope.  When something occurs inside of a subprocess, the details are not available externally.   For example, if I were to assign an activity to you that requires several steps, I would use a subprocess.  Within that subprocess I don’t have access to the details of what you are currently doing, and I don’t have access to any data artifacts being created.   The subprocess is essentially functioning independently.  However, the subprocess has access to any information defined in the higher-level scope, such as a pool, or parent subprocess when subprocesses are nested.

Because of the limitations of subprocess scope, the event handler for a subprocess doesn’t actually have access to the data and activities contained within the subprocess.   Because of this problem, BPMN 2.0 introduces the non-interrupting start event.

Non-Interrupting Start Event

When a subprocess starts, non-interrupting start events inside the subprocess are activated.   When the subprocess completes, the evens are deactivated.  So there is a scope of when they are available to be triggered that is the same as the scope of the subprocess.

Also because of the subprocess scope, there is a change in accessibility of the events and the related exceptional flow.  Any activity or event on the exceptional flow can access anything from the parent scope.  In this case, everything is in the same subprocess scope, so both A and B can access the timeout handler, and likewise, the timeout handler can access both A and B.

This sort of behavior is significant because of the parallel behavior. In the example above, when the timer is triggered, a parallel flow is introduced.  The parallel flow is outside the scope of the original activity, which might cause synchronization issues. So you might to use additional events such as signals to keep things happening in the correct order.

Because the non-interrupting flow is in the same subprocess, a merging gateway is not needed. A subprocess has an implicit merge capability.  But you still have to pay attention to details.  If the timeout handler is not completed until after subprocess B, the entire subprocess will still be held back until timeout handler is completed.  This is the nearly identical behavior to the interrupting pattern shown above with the merging gateway.  The main difference is in the scope and visibility of data.

I asked the question why they didn’t just reuse the intermediate instead of the start.  I think it would have been easier to learn. It seems that many folks at OMG are very strict about using start and end events inside every subprocess.  However, the spec doesn’t explicitly say that intermediates are disallowed.  I suppose that using an intermediate as a start of a flow is just as confusing.   Prior to BPMN 2.0 I used to draw the entire pattern in yet another parent subprocess, and create any data artifacts in the higher-level parent scope so that both normal and exceptional flow would share a common scope.    So I can see where this new feature of BPMN 2.0 will be useful.  However, I recommend using text annotations so that people who read your diagrams are clear on the exact behavior you are trying to achieve.

BPMN getting complicated?

I have already received feedback based on my last few posts that BPMN is getting way too complicated.  In some ways I agree, but we also have to consider every use case and have a clear way of illustrating it.    Up until quite recently I was very hard-core on teaching every BPMN shape.  I’m now taking a different approach to process modeling.  Sometimes you need all of the shapes, and sometimes you don’t.  It also very much depends on who the audience is. For the engineering crowd, use every feature of BPMN to the maximum extent.  For the strictly business people, I suggest looking at the basic subset of BPMN and letting others worry about the complexity.  This is yet more evidence that we need a multi-tier process design.  You cannot shove everything into one process model and expect everyone to read and understand it.

I’m not making any promises of timeline yet, but I have a new book in the works that will help clarify the spec and make recommendations on which shapes to use in which situations.

Comments and questions welcome.

- Rick Geneva

This post is a continuation of the Highlights from BPMN 2.0 series.

At Last!  The long awaited changes to the numerous problems with the event based gateway.   First off, there was only one type of event based gateway in BPMN 1.0 – 1.2, and it is exclusive behavior.  Exclusive event behavior means that only one event can trigger the gateway.  Also, there was only one shape that served a dual role of start and intermediate event, yet the shape did not have any variations as the event shapes do (see Demystifying the Event Based Gateway from my previous post).

First, let’s take another look at the original Event Based Gateway from BPMN 1.0 – 1.2

Event Based Exclusive Gateway

Event Based Gateway (Exclusive, Intermediate)

The inner pentagon shape signifies multiple events.  The double circle signifies intermediate usage. Lastly, the outer diamond shape signifies a gateway.  When you combine these attributes you get a gateway that handles multiple event types.  Also note that the inner pentagon shape signifies that only one of the potentially numerous events are needed to trigger the gateway.   In many use cases of process modeling, this is less than adequate for the job.  In many cases I’ve had to create some very elaborate patterns to work around the shortcomings for this shape.  The previous BPMN 1.2 specification allowed for this shape to be used as a start or intermediate event.  With all this ambiguity you might now understand why I wrote a post called “Demystifying” the Event Based Gateway.  Still, there were many use cases that were just too complicated to understand.  So I’m quite pleased to see that BPMN 2.0 finally fixes these problems.

Event Based Start Gateway

Event Based Start (exclusive)

Adding only a slight variation of the original shape, the Event Based Exclusive Start Gateway (that’s a mouthful of words) uses the start event (single circle) instead of the intermediate (double circle).   It’s used as the start event, with intermediate events to the right.  Hopefully by the time the spec is released they will allow for start events to be connected to the right of this shape.   But either way it works for me.  This saves me at least 10 minutes of explanation every time this topic comes up in one of my training classes.

Event Based Inclusive

Event Based (Inclusive, Start)

Event Based (Inclusive, intermediate)

This is where the BPMN 2.0 spec gets interesting.  There has long been a use case where you need to receive multiple events to trigger a process to start or to continue. Previously the only option was to put the event based gateway in a loop until all events have been received.  But then this caused another problem where the same event could potentially be received more than once, which is often undesirable.  In BPMN 2.0 you the new inclusive multiple event shape is introduced (discussed in more detail in a later post).  The shape is an empty plus (+) shape (white filled).  This type of event signifies that all associated events must be received before the event is triggered.  So when you put this event type inside the diamond, you then have a gateway that is only triggered when all events are received.  Note that there are both start and intermediate variations of this shape.

Next post: Event shapes in BPMN 2.0

Comments and questions much appreciated and encouraged.

- Rick Geneva

Data Artifacts

Data Artifact

The data artifact shape was introduced in BPMN 1.0 and it is used to illustrate that some type of data is associated with shapes on the diagram. The problem with this shape is that it’s too generic to be widely used.  Also, it relies on the fact that everyone can understand the abstract concept of what a data artifact actually is.  Most people I talk to see this shape as a document, but don’t initially realize that it could in fact be representing a database record.  And then there was the problem of showing if it’s just one item or many.  In the case where it is actually many items, my diagram would be cluttered by a bunch of these.

Multiplicity

Data Artifact Collection

BPMN 2.0 introduces the Data Artifact Collection object.  In general, the three bars ||| indicate multiplicity.  They are used on several shapes.  In my previous post I showed the new activity shapes with the three bars as being multiple instance.  The meaning is similar with the new data artifact collection shape.  The additional annotation on the shape indicates that there are multiple items in the collection of data artifacts.  An example could be a pile of documents.  Or it could mean that multiple database records are involved.  In the IT architecture world, when dealing with data, this could symbolize an array of data.

Input and Output

Data Artifact Input

Data Artifact Output

The arrow annotation on the data artifact shape indicates that the data is either inbound or outbound, depending on the color of the arrow.  White indicates that the data is being received.  Black indicates that the data is being sent.  Sending and receiving is always relative to the participant pool where the shape is located.  For example, sending on one pool usually means that data is receiving in the opposite pool.  But we are also seeing a departure from the notion that swimlanes are required in all diagrams.  Many diagrams are abstracted, and show the perspective of only one participant.  The sending and receiving data artifact can help to clarify diagrams drawn in this style.

BPMN 2.0 is becoming more friendly to the technical crowd.  Early in 2009 I did a presentation at AJUG (Atlanta Java Users Group).  This group is primarily for Java software programmers and enterprise data architects.  They were very interested in SOA (Service Oriented Architecture) but there was some scepticism in the crowd as to whether or not BPMN would affect the world of programming. I showed a few diagrams that actually generate executable software code, but the story was not yet convincing to some.

My opinion on this topic is that BPMN will not replace traditional programming technique, because we still have a need for doing things “the old way”.  But reinventing the wheel in programming every day is very inefficient.   Already we have good IDEs (Integrated Development Environments) that generate code based on patterns. The next logical step in evolution of the IDE is to represent something technical via a graphical icon.  UML made an attempt at doing this, but UML was not very friendly to the business community.   So now BPMN is adding some of the same concepts as in programming, but abstracted in a way that both business and IT can understand it.

The main idea behind BPM is that business and IT architecture collaborates in a common environment, using a common modeling notation, with the goal of automating and improving processes.  This often means that a business analyst will define the need for a technology to handle data.  Data systems understand input and output.  So now we have shapes that can help to create requirements for IT without having to go into diagrams of explicit messaging.   For example, we could say that a task has an output of a customer record.  Drawing the UML class diagram of a customer record is pointless at this level of abstraction because we are focusing on process flow, not data definition.  Later, IT architects can design the customer object to whatever schema is required.  The point is that we know where in the process flow that the customer object is being used.  This not only helps for gathering requirements today, but also later on when we want to change processes.

Note that both the input and outputs can have the multiplicity (three bars ||| ) notation, indicating that a collection of data is used for the input or output.

Data Source Artifact

Data Source Artifact Shape

The data artifact shape is abstract, meaning that it could represent many types of data, or even a part in a manufacturing assembly line process.  Some modeling tools have extensions with extra icon annotations that are non-standard.  I don’t suspect BPMN is going to add shapes for everything, because it would be even more difficult to get everyone to agree (this would be a spec killer).   However, there has long been a need for clarifications on diagrams concerning data that is detached (in transit) or if it’s associated with a permanent data storage location.

The Data Source Artifact shape should be familiar to most people, because it’s similar to the database shape from flowcharts.  However, there is a distinct difference in BPMN from the flowchart shape.  A data source is not only a database, but it could also represent an abstract data source, such as a hard drive where documents are stored.  In the modern information age we have many types of data stores.  Often, data stores are virtualized, existing somewhere in the could on the Internet.  But for all practical purposes the general concept is similar a database.  Just be aware that this shape doesn’t necessarily mean “database”.

The new specification is over 500 pages long, which is much more than most of us have time to digest.  Furthermore, the specification is heavily laden with XML and references to mapping BPMN to the BPEL runtime. This is quite useful for standardizing BPM systems but might be more technical than the average process modeler wants to hear about.  Over the next few posts on this blog I will be highlighting some of the most important changes, and what it means to you as a process modeler.

In this post I will describe the new BPMN 2.0 task and activity types.

Activity Category

Often in my BPMN class I have a student who asks “how can I tell if the activity is a person or a system task?”.  Prior to BPMN 2.0 the only way to tell was to look at in the lanes in the pool, if you are using this style. Otherwise you would have to use a text annotation to clarify the intent of the digram.  BPMN 2.0 adds new icon annotations to the tasks. These icons are optional, but I encourage you to use them because it brings a lot of clarity to a diagram.

BPMN 2.0 Task Shapes

For example, there is now a manual task and a human task.  The difference is that the human task is meant for a BPM system or an application such as CRM where a task can be assigned to a person.  A manual task could mean many things, all of which are outside the scope of a typical system interaction diagram.  For example, a postal worker who sorts letters, a factory worker who assembles product, or a bicycle courier who delivers a package.  Manual tasks are sometimes just as important to an organization as tasks in a BPM system.  Therefore you should model these activities, and show a clear differentiation between the two types of human activity.

Messaging

The message task is something new, and solves a problem from the previous BPMN 1.2 specification.   Often, communications are not necessarily a one-way message.    The spec allows for either a task for a message event to send and/or receive messages. Students in my classes often ask me when should a task be used, and when should it be an event.  This usually results in a half hour lesson on the semantics and style of using task vs. event.  Thankfully, the 2.0 spec helps solve this problem.  The main intent here is to use a task for blocking communication, also known as synchronous.  In Synchronous communication, the initiator must wait for a reply before continuing with the next activity in a process.  This means that the initiator has a very active role in the communication, and a task should be used instead of an event.   In BPMN 1.2 I used the intermediate message event with a two-way message.  This was somewhat against the spec, but in order to illustrate the nature of the process, it was necessary.  So now we have a shape to solve this problem.

System Activity

System tasks were always a bit of a mystery in BPMN.  Often you would see a task that simply stated it was performed by the system, or an application.  The problem is that there was no way to tell what the system is actually doing.  The usage of rules, services, and scripts is very common in a BPM system.  But without a shape to differentiate which category of activity, the reader of the BPMN diagram was left to guess.  Again, the only way around this problem was to use a text annotation.  BPMN 2.0 adds three new icon annotations to activities, which helps cut down on text, and speeds the creation and interpretation of diagrams.

The service activity is a much welcome addition because now the business analyst who creates a process diagram can easily show where services are being consumed.  IT architects can now more easily justify the usage of BPMN instead of UML for process oriented projects.  This is a tiny change, but I think it will make a giant leap forward in terms of business and IT collaboration.

Scrips or Services

Not every system task in a BPM system is a service.  In an SOA system (service oriented architecture) this might be the case, but again, not always true.  Many tasks are simple scripts, which could be JavaScript, XPath, or other languages.  Often these scripts are proprietary to the BPM system that implements them.  But to communicate in a vendor neutral way, we must use shapes that are generic.  Often a higher level diagram will specify “a system task” but gives no reference on how it should be implemented.  So now with BPMN 2.0 you have a clear differentiation of service task verses script task.

Business Rules

Rules and process work together closely in a BPM system.  Prior to BPMN 2.0 there was no clear way to illustrate a rule activity.  The condition event didn’t quite work, because a rule is not actually an event.  Rules more closely resemble services, but a rule is a very specific type of service, often maintained by business people instead of IT.  Most BPM systems have their own BRE (business rules engine) or decision table system.  The rule shape represents the execution of a business rule.  This is quite useful in process modeling when you are working in a higher level multi-tier diagram structure and you are not yet ready to show the full details of a business rule.

Multi-Instance Activities

In a previous post Ins and Outs of Process Loops I pointed out a problem in the BPMN 1.2 specification related to multi-instance process loops.  The annotations state that the activity occurs multiple times.  However the spec says that the execution could be either in parallel, or in series.  But there was no way to tell which was which unless you looked at the “properties” which is unique to the BPMN system or the modeling tool.

BPMN 2.0 Multi-Instance Activity

My advice at the time was to use a text annotation to clarify the diagram.  This has now been fixed in BPMN 2.0.  The shape on the left is the parallel execution, which means all instances will execute simultaneously.  The subprocess on the right is the serial execution, meaning that all instances will occur one following the other.    Both of these shapes describe a “for-each” behavior because there is a set number of iterations.  In contrast, “do-while” and “do-until” loop types do not have a defined limit before the activity starts.   But a for-each normally would have a set number of iterations (for x = 1 to 5…).

Note that the looping subprocess and task shapes are still valid (no changes).

Next Post:  Artifact Shapes in BPMN 2.0

- Rick Geneva

The event driven gateway is one of the most useful symbols that I know of in BPMN.  Yet it’s often overlooked as a solution to common problems.  It is a compound symbol, inheriting attributes of other BPMN shapes.  First of all, it’s a gateway that is used to split sequence flow paths.  The gateway is the diamond symbol.  Inside this diamond shape you will see the intermediate event shape, which is the double thin line circle.  Inside the intermediate event shape there is the pentagon, which is a symbol for multiple events.  So when you put all of this together we have a gateway that deals with multiple intermediate events.

BPMN version 1.1 and higher

In all of my classes I tend to get more questions about the event based gateway than any other shape in BPMN.  This is probably because there are so many use cases for it, and its pattern to many people appears to be drawn backwards.  Most beginners in BPMN tend to draw the pattern with the intermediate event shapes to the left of the gateway.  Let’s start by looking at the basic pattern and then discuss some of the potential business scenarios where you could use this notation.

The basic pattern that applies to the event based exclusive gateway

The main purpose of the event gateway is to illustrate that you must choose between one, and only one event attached to the right.  Any other event would not correlate to the same process instance.  For example, I send a message to a coworker, and he replies either by email or by phone.  The message type is different for the reply, and requires a different action for me to process it.  I don’t know in advance which message I’m going to receive. Therefore it is an event that I must react to and process accordingly.

The exclusive behavior of the event based gateway is due to the fact that only one path can be chosen.  This is similar to the exclusive data based gateway, except that event-based means that we are reacting to events instead of data conditions.  With data-based gateways, I already have the condition, and I must evaluate it at the relative point in time that it appears in the diagram.   The event based behavior is expressing that we don’t yet have the condition at the specific point in time, and we must wait for an event to make the choice of which path should proceed.

There is a unique capability for this gateway to be used at the beginning of a process.  It actually takes the place of the start event in this case.  Often I encounter diagrams where there is additionally a start event to the immediate left of the event based gateway.  This might actually be incorrect, depending on what you are trying to express.  The literal interpretation when you use the event based gateway as a start event reads like this:  My process starts when I receive one of the following events.  The BPMN specification is very specific about using only intermediate events to the right of the event driven gateway. But if it makes more sense to you, think of the intermediate events acting as start events, with the gateway acting as the place where the choice of event is made.

The event based gateway acting as a start event expresses that multiple events can start a process.

When used in place of a start event, the event driven gateway is expressing that there are many events that can cause the process to begin.  When used as an intermediate gateway, it expresses that the process is in a wait-state, and will proceed only after one of multiple events have occurred.

In either case of starting a process or continuing a process, the event based gateway merges with the exclusive data-based gateway.  This might sound strange, but merging on gateways is found in a diagram when we want to express that a synchronizing flow exists.  It’s similar to a traffic symbol.  Exclusive merge means that only one path will ever be permitted through this point in the diagram.  In contrast, parallel merge indicates that all paths are required in order to move forward, and will create a wait-state until all paths have reached the point of the gateway.

Using the exclusive data-based gateway for merging

Because per BPMN specification, the event based gateway is only permitted to have events to the right, we cannot use it for merging.  But since we only need to express the merging behavior, the data-based exclusive gateway is a suitable substitute.   Often a process modeler will omit the merging gateway.  I am a proponent of always using a gateway for merging because it shows an explicit behavior, rather than leaving the diagram open to interpretation errors.  For example, if I write a quote and don’t use the close quote symbol, doesn’t this leave you hanging a bit?   So it’s better style to always use a merging gateway.

A best practice of using event based gateways is to use multiple event types whenever practical. If you have investigated the merging behavior of gateways, you might already know that it is possible to cause a deadlock or a replicated instance with improper usage of gateways.

Expecting the Unexpected Condition

A replication of sequence flow occurs when a parallel split occurs and there is no checkpoint or gateway to ensure that synchronization or merge of the parallel paths occur.  For example, I split with a parallel gateway and I forget to merge with it.  Somewhere downstream I could potentially end up with replication of effort on the same process instance.  Suppose I send a new hire document to IT, human resources, and accounting departments in parallel.  After these departments are complete with the document it is passed to another department who does a background check.  The background check is expensive, and it is outsourced to an external agency.  The external agency will not check to see if the same person is being processed more than once. There must be a point in the process that prevents the background check from occurring three times on the same person.  Potentially every subsequent step downstream in the process might also occur three times until someone finally realizes the error and merges the files.  Before the error is caught, duplication or triplication of effort occurs.  This would be expensive and wasteful.

Without merging, a potential for replicating the downstream activities exists

Likewise with exclusive behavior it’s possible have sequencing and synchronization errors without properly handling the merging behavior.  A deadlock condition occurs when a splitting gateway doesn’t match up with a corresponding merging gateway.  Sometimes this can accidentally occur implicitly whenever the merging gateway is omitted from the diagram.  For example, I send a document to one of three departments.  A fourth department is expecting to receive documents from all three proceeding steps in parallel, but it only ever receives one.  This can be a deadlock situation, meaning because of a merging flow error, the process will never proceed past the checkpoint.

Improper usage of gateways can lead to process deadlock

Deadlocks can occur in event based gateways as well.  But in this case, it’s not because of using the improper merging shape, or from omitting the merging behavior.  The deadlock is event based.  Event based deadlocks occur when the primary means receiving the expected events has failed.  For example, I send an email to my friend to have lunch with me.  He might reply via email or via calling my mobile phone.  But I don’t receive a reply so I go ahead and wait at the restaurant anyway.  I assume he must have received the message, because email is so reliable, right?   Meanwhile I’m waiting at the restaurant for him.  I wait all day.  I wait all week.  Eventually, after about three months, the manager at the restaurant feels inclined to call the police to physically remove me from the restaurant.  My friend actually did receive the message and left a note on my desk that he could not join me.  Because the event (note on my desk) was outside of my processing capacity, I wasted three weeks of my life waiting.  This was a  terribly inefficient process, and could have benefitted from planning for the events based deadlock.

Obviously this is a fictional story.  As human beings we have enough common sense to avoid this situation. But the elaborate computer systems we build don’t have this kind of common sense.  A stuck automated process will wait forever until an administrator comes along and terminates the process or reboots the computer.   Meanwhile, money was spent, customers were lost, paperwork piles up on someone’s desk.  Eventually someone will realize the problem, but not until after the customer gets angry.  Or maybe the error is realized after the board of directors becomes angry because your company keeps losing money.

The deadlock of events is usually caused by the assumption that things will always go as planned.  The reality is that in business things go as planned only about 35% of the time.  The other 65% is either the more expensive way to make money, or the ways that money is lost in a business transaction.

In the scenario of me waiting for my friend at the restaurant, my common sense tells me that I should only wait about an hour before I make the decision that he is not coming to meet me.  For other friends this might be two hours because they might be an hour late on a regular basis. But the point is, I would not wait indefinitely.  So the addition of an intermediate timer event to my process flow can save the day when I have an event based deadlock.

If you wish to be even more careful about event based deadlocks, try to consider more reasons why the process could fail.  For example, as I’m waiting for my friend I hear the tsunami warning siren.  The restaurant is by the beach, so I don’t think my friend will meet me.  In fact, its time I consider evacuating to higher ground.  In this case I would add a signal event to my process that listens to all “broadcasts” that might be interesting to me.

Full example of the event based gateway in action, using multiple events to avoid process deadlock.

A more realistic example

Now let’s apply the same pattern to a common business problem. Suppose that I send an order to my supplier.  Do I wait indefinitely, assuming the supplier received the message?  After all, they have been fairly reliable in the past, right?  So what happens when I don’t receive the order?  Should I continue waiting months later when I have no products to sell because I have no supplies to make anything?   Maybe you assumed that someone else would realize this problem and call the manufacturer.  But when will they make the call?  Most likely it will be after supplies are low and we are at risk of shutting down manufacturing.  In other words, it might be too late before we spot the problem.  So instead, we model our process to include for provisions such as a timer that monitors when the product is received.  Also we allow for the vendor to report an error in processing the order, which should prompt someone in my organization to reorder.  And while we are at it, why not ask the supplier to broadcast system outage messages so that we know when to resubmit orders that might have been lost in transmission.

Ordering supplies with an error handling event and a timeout event

Hopefully this clarifies the usage of the event-based exclusive gateway.  I hope you can now see that although the event based gateway uses a pattern difficult to understand, it’s well worth while learning. It’s one of the most powerful shapes in BPMN.  Processes are not linear and don’t always function like clockwork.  Modeling in the flowchart style with swimlanes and everything being perfectly sequential doesn’t depict the true nature of business processes.  In the real world, processes are event based, and 65% of the time things don’t follow the optimal plan.   By using an event based process model you are more likely to uncover the subtle details and pockets of inefficiency.

The simple answer to which loop method to use is to express yourself.   The specification leaves room for subtle differences, that when combined with some experience in process modeling and some modeling style, BPMN can be very expressive.   In some cases there is definitely an incorrect choice for the process at hand, but somehow there never seems to be an absolutely correct one.   So here is my style guide on how to pick the best expression for activities that loop.

Style 1: Upstream Loop-back Flow

Example:

Upstream Looping Process Flow

Style expression: Use when you want to do something over again (redo), but not when you want to repeat activities and preserve the previous results.  If you want to preserve the previous results I would recommend using the looping subprocess instead.

Upstream looping flow with timer

Upstream flow is also very useful in high-level diagrams where you don’t wish to show all of the intricate details of process flow.  I’ve found that if you are using a high-level view, it’s best to stick with subprocess shapes rather than using tasks.   A task implies an atomic single activity where something is done.  The subprocess expresses that there are several steps to complete a task, which can include the points of communication (message events, signals, or anything else required to complete the activity).   I’m not saying that the subprocess is required here.  But it might be a better option because it’s more likely to accurately depict the true nature of the process.

Upstream looping flow across participant lanes

Advantages:

• Simple to model and most people with no BPMN training can easily understand it.
• Great for showing high-level sequence flows across multiple participants (roles, systems, etc).
• Easily shows the loopback condition as part of the gateway, without using annotations.
• Allows you you to do a loop across multiple swimlanes, but be cautioned that this is not always a good idea (see my other post about swimlanes).

Disadvantages:

• Doesn’t always capture the detail of a process flow (see my other post about swimlanes) and limits ability to show explicit message interaction between participants.
• Not very easy to introduce exception conditions into the flow without adding a lot of extra shapes.
• Implicit way of showing multiple iterations of an activity might lead to inaccurate interpretation.

Style 2: Looping subprocess

Looping Subprocess

Style Expression: Use when you need to repeat one or more activities while preserving the data/results of the activities for future reference.  Also very useful when you need to deal with multiple exceptional conditions that might interrupt the loop iterations.

In the following example I am using the simple “check mail” process.  Every hour I check the mail, and if there is something in the mailbox I reply to the mail.  When I’m done replying to the new mail the cycle repeats.   Immediately after replying to mail I go back to check for new mail.   This starts to optimize the process because if new mail arrives while I’m replying to other mail, as soon as I’m finished replying there is no wait state.

Looping Subprocess with Timer

Dealing with exceptional conditions in a subprocess is much easier to do than with straight-through linear flow.  This is because you can place one or more exception handlers on the subprocess border.  For example, I can catch an error condition if it occurs, but I’m not really expecting this to happen all the time.  Or I can cause an alternate flow to occur when the activities don’t complete within a specified time period.  Or, I can watch for all of the above conditions and deal with them accordingly.   In contrast, the linear flowchart style diagram gets extremely complicated and difficult to read without the capability to “dynamically” create exception flow.

In a linear flowchart style, many conditional gateways or flows are used to check for current state.  In the BPMN style, intermediate handler events are used on the subprocess border instead.  In the following diagram I add a few layers that handle the exceptional conditions that could potentially occur.  I’m not going to explicitly check for the errors all the time. It’s more of a passive monitoring for things that might occur.

Complex Nested Looping Subprocess with Error Handling

First I can receive the intermediate message labeled “cancel”.   This is here because instead of watching for mail all day someone might give me a more meaningful way to spend my time.   So maybe my boss sends me a message to go out for coffee or something.  When this occurs I terminate this process and go do something else.

Next there is an intermediate event that catches any errors that might occur in my process.  This error handler will catch anything that might occur on the check mail or reply to mail.  Note that this is not necessarily a technical step geared toward the geek crowd.  This gives me a “catch all” way of dealing with exceptional conditions, regardless of what the condition is.  Most likely I’ll have to interrupt the normal pattern and do some improvising to get the process back on track.  For example, one day someone plays mailbox baseball.   For those of you who don’t know what this means, it’s when someone takes a baseball bat and smashed the box where I receive my mail.  So I can’t receive mail.  Sounds like an exceptional condition, right?  I’m not planning for this specific event to happen, but anything can happen.  So I don’t want to stop my process dead in its tracks just because I don’t have a way to deal with anything that might come up.  The error handler is most likely a generic manual step.

Note the outer looping subprocess.  This ensures that if an error condition occurs that I can easily jump back directly into a new instance of my inner loop, which is the main normal flow.

An finally, at 5 PM I’m going home, so I terminate the process.   Because I’m a “clock puncher” type of person (How I wish this were true instead of me writing this post past midnight for three days now) it doesn’t matter what I’m doing.  When the whistle blows at 5 PM (the timer event) I’m going home.  Forget about replying, forget about checking for new mail.

So that’s the process.   Now what would this same process look like if I didn’t have a subprocess and used upstream flow instead?  Here is the equivalent diagram without using subprocesses, or should I say as close as you can get to the equivalent behavior.

Complex Upstream Flow with Error Handling

To achieve 100% of the same process flow behavior using upstream flow it’s incredibly difficult. In fact, this small example took several tries for me to get it right, and I’m still not totally convinced its the same.  Feel free to comment and tell me how I did.   The point is, when you start handling errors (as you should be doing in all processes anyway) in the upstream flow style, the diagram gets very complex to create, and even more difficult to read.  Now just imagine this upstream flow example using swimlanes across multiple participants.  You will probably end up with lines going all over the place and it’s a real eye sore to look at.

Just to put some context around this, all we are doing here is checking for mail and it’s already getting hard to read.  Imagine if we were to model a real process!   So for detailed process design including proper exception handling, the upstream flow doesn’t work so well.

Looping Subprocess Advantages:

• Explicit loop control
• Focus on a single process participant, which frequently yields a more accurate/detailed process diagram (see my other posting about swimlanes).
• Ability to easily deal with multiple exceptional conditions without complicating the normal flow.

Looping Subprocess Disadvantages:

• Slightly more difficult to understand for people that have not been trained on BPMN syntax.  Note however that this can easily be explained to the BPMN newcomer in about two minutes, or you can just as easily add a text annotation to explain the loop.
• No way to determine what the loop condition is without annoations.

Style 3: Multiple Instance

Multiple Instance Subprocess

The multiple instance has two use cases.  This sometimes happens in the BPMN specification.  It’s not always clear in the diagram which use case is actually occuring.  I suspect this will be cleared up in the 2.0 or later specification (or at least I hope so).  Because of this current issue in the BPMN specification I recommend using a text annotation externally explain the sequence behavior.    Note that the external text annotation is also a good practice for the looping subprocess as well.

Use case 1:   Serial Execution

Use when you want to repeat some activities for a fixed number of times. There is no condition on breaking the loop.  For example, “do this five times”.    Each instance of the activity is performed in an iteration, meaning that each after the first iteration the subsequent iterations will not occur until the previous has completed.   Essentially this is the same as the looping subprocess, except that there is no need to specify a condition that will cause the loop to break it’s loop.  Only a number of iterations is provided.

Multiple Instance Serial Execution Mode

The same diagram can be expressed using only task shapes, but requires many more shapes to accomplish the same expression.

Multi Instance Serial Task Equivalent

Style expression for Serial Execution:  Use in a “for each” loop when the number of  iterations is known prior to execution.  Not applicable to a “do while” or “do until”.    Don’t use in any loop where a condition or normal flow might exist that would cause the loop to break before the number of iterations is complete.   The usage of exception handlers is acceptable (and most often encouraged).

An important consideration for using this expression is whether or not you know in advance the number of iterations you will perform (If I am repeating myself here it must be because this is important).  Otherwise you should use the looping subprocess instead.   Conditions such as “do until the color is sufficient”  or “keep checking the temperature every 5 minutes until the inner temperature is 200 degrees” is not a candidate for a multiple instance loop.   Instead, a good example for a multiple instance loop is “Get 100 people to sign the petition”.   Another example could be “take exactly 10 steps forward before turning left” if each step is an iteration of the looping activity.   But then again, if you want to get very technical here one could argue that each step has a dependency on the other leg (left leg/right leg) to perform its duty, so this would disqualify the multiple instance for each step.  Instead I could either hop 10 times or take 5 pairs of steps forward.

Use case 2:  Parallel Execution

The basic use case is the similar to the serial execution use case, except that instead of iterations, X number of instances will instantly be launched, all running in parallel.   Like the serial execution, there is no need to break the loop.  In fact, it’s not really a loop at all.   You are simply showing a different form of parallel flow in which you are launching X number of identical activities simultaneously.

Multiple Instance Parallel Mode

The same diagram using parallel gateway looks like this

Multiple instance parallel gateway equivalent

Style expression for parallel Execution:  Another way to word this notation is “do something X number of times, right now, and don’t wait for anyone else to complete”.  Use when you want to perform the same activity a fixed number of times, and each activity can be performed independent of the other with no dependencies of any other instances of the same activity to complete.   Never use the parallel execution in situations where one activity might depend on another completing.  Never use this expression when the activities in each instance might contain a different sub-sequence.

In the example above referring to “Get 100 people to sign the petition” we could apply the multiple instance.   However, there is a subtle difference that occurs.  We would have to get 100 people in action simultaneously.   So in the serial execution example we are likely a single person going door to door for signatures.   In the case of the parallel, it’s more likely that we blasted 100 emails out to people and we are awaiting 100 responses.   So it might not be possible to actually receive 100 positive responses out of 100 emails sent unless the response is unanimous.  Instead you might want to express that out of 1000 sent emails we want to count at least 100 positive responses, which causes a break of some sort.   Normally I would suggest that you use the looping subprocess for this example.   But in this case you have a unique problem that is very difficult to resolve with a normal loop; you have to correlate each response with the preceding request that is unique for each instance.

Looping Subprocess Emulating Parallel Multi-instance

It’s important to note that the serial execution and parallel execution appear identically in BPMN.    According to the specification the execution mode is determined by the “attributes”.   Now you see, this is what happens when a perfectly good modeling notation meddles in process automation and execution.  I mentioned earlier that I hope this will be fixed in an upcoming version of BPMN.  For example, vertical lines for parallel execution and horizontal lines for serial execution would have been nice.  But the specification simply stops short of offering a graphical solution and suggests that execution engines use the “attributes” to specify behavior.  This does nothing for creating a diagram.  So I ask the folks at OMG to please fix this so that people who want to just create a diagram can have an easy way of doing so without having to read “attributes”.   Now that I’ve vented my frustration on this subject, might I suggest for the rest of us to use a text annotation external to the subprocess to resolve the ambiguity (as in the “multiple instance parallel mode” example).

Multiple Instance (parallel) – Advantages:

• No need to specify a loop condition based on data.  You can clearly tell that the activities will occur a fixed number of times, and this number of times is known before the loop begins.
• Has the ability to dynamically generate a bunch of parallel flows without having to draw all of them explicitly (parallel execution).

Multiple Instance (parallel) – Disadvantages:

• You cannot tell if this is a parallel or serial execution without any sort of annotations externally.
• You cannot tell exactly how many parallel instance or serial iterations are actually being performed.  Again, to resolve this problem you should use external annotations.
• Hardest to understand, least documentation and examples in the BPMN 1.2 specification.

UML Specification:

Swimlane: a way to group activities performed by the same actor on an activity diagram or to group activities in a single thread.

The term swimlane appears for the first time in formal process modeling in the UML specification.  First there was the activity diagram.  Then there was a need to separate the activities of individual actors.   The swimlane was born.    So why are we using BPMN these days instead of UML?    Well the answer is quite complicated.  The UML specificaiton is full of other types of diagrams including class diagrams, state diagrams, interaction diagrams, and more.    Most of the UML specification is very technical, therefore making it less than idea for most business uses.

Many people today believe that the term swimlane is part of BPMN.  In a way, this is partially true because the BPMN specification says that both a pool and a lane are techinically a swimlane.

BPMN Specification :

UML was created in 1996.  Remember 1996?  Internet… What’s that?   1998 -  Internet is pretty cool, and by 1999 my mother sent her first email.   I recall in 2001 I heard someone say the word “blog” for the first time.   In 2002 BPMN was born.  And now what we have the history lesson out of the way…

• Pool: Defines activities designated for a single participant.    Pay special attention to the words single participant.
• Lane: “A further subdivision of a pool”.    Quoting from the BPMN 1.2 specification: “Lanes are used to organize and categorize activities”.

The BPMN specification says specifically that both pools and lanes are considered to be a swimlane.  So this is where some of the confusion comes from.  Technically it’s correct to call a BPMN pool or lane a swimlane, but the swimlane is not defined as part of the BPMN specification.

Here is an example of a UML style diagram using swimlanes:

UML Swimlane Style

This is a small process where a manager wants to hire someone.   The manager submits a job requisition to human resources.  HR returns a list of candidates and the manager interviews them. This cycle continues until a suitable candidate has been found.   After a candidate has been found, accounting is notified to set up the payroll for the new employee.  On the first day of work the manager conducts the initial briefing and gives the new employee a tour of the facilities.

The example diagram used the BPMN shapes but is actually a UML style diagram in terms of swimlanes.  Often I see this style of diagram in BPM tools (not calling out any vendor specifically here).   This diagram is actually incorrect according to both the UML and BPMN specifications.  In UML the shapes and usage is slightly different than they are in BPMN.  In BPMN you cannot have a lane without it being inside of a pool.  I suppose one could argue that the entire diagram is the pool, in which case the only thing that is missing is the label for the pool.  So we would have to assume that the pool has something to do with people.

Now let’s take a look at the corrected version of the diagram with a BPMN pool correctly displaying the lanes.

BPMN Pool with Lanes, single participant

In this example the pool is called People because that is the only logical assumption we can make regarding who the participants are.  In fact, if you take the specification literally, there is only one person performing all of the activities.   Remember that a pool is a container for activities associated to a single participant.   In this respect it is not possible to transition from one person to another within a pool.  Each additional participant requires another pool. The lane could be used to indicate that the activities are associated with a particular category of work.  For example, the subprocess for finding candidates is actually a performed by the same person that creates the job requisition, but the category of activity is human resources.   So I suppose we could say this participant role wears a lot of hats in their organization.

Often I see this style of diagram and I realize that the intent was to show that multiple people are involved in the process.  However, this is incorrect in BPMN.  If the intent is to show the multiple participants’ activities, you must draw them in multiple pools.   But this creates another problem; I cannot draw a transition line (solid line) between pools.  I must use a message.  Why?  First we have to understand what transition means.

• Transition: One activity from a participant has completed and another has begun.  Transitions are notated with a single solid line with an arrow head pointing in the direction of the flow (sequence flow line).   Sequence flow only occurs within the scope of a single participant.
• Interaction: A process participant wishes to involve other participants through messaging.  Interaction is a form of communication that involves the sending of a message.  Interactions are shown in BPMN with a single dashed line with an open arrow head indicating the direction of message flow.
• Message: A means to communication between process participants.   A message has a well defined recipient and can have a message body containing process data or other data artifacts such as document attachments.

You can see from the definition that a transition is not allowed between participants.  This is because it’s simply not possible.  In any process, no matter if it’s human centric or system centric, any interaction between participants requires some sort of communication.   Messaging is the the means in BPMN to have direct communication between participants.  It is also possible to coordinate activities with things like signaling, but this is not actually an interaction.  Signals (see my other post for details on signals) are used to coordinate events, but are not used to communicate.

So if you wanted to create a diagram that showed correct BPMN formatting, it would look like this:

BPMN Diagram with Pools and communication between participants

The pools for the Accounting and Human Resources participants are collapsed for simplicity.  This is a common practice when you wish to show in your diagram the focus on one single participant, but you still wish to show the points in which this participant interacts with others.  Most likely I don’t care to show the details of the accounting department because it’s outside my juristiction, and I have no visibility on what they actually do.  Also, it’s better to let accounting manage their own processes.

The other concept of style that I’m including in this example is the looping subprocess instead of the “upstream flow” lines.  Upstream flow is when you sequence an activity that has already occurred.  This is almost like going back in time.   The problem with going back in time is that I’m not actually going back in time here. Something is different.  I have a new iteration of an activity, but I’m not redoing the exact same instance.   In this example I am interviewing a new job candidate.   I’m not interviewing the same job candidate again.  So in some cases the upstream flow is acceptable, but only if you want to redo something.  Otherwise the proper BPMN notation is to use a looping subprocess, which shows that a new iteration is required, and probably a new set of data input is also required.    With this in mind you can see that it is impossible to have upstream flow between participants.  Any time messaging is involved I highly recommend using a looping subprocess instead of upstream flow.  Otherwise your process participants could be out of sync and you might never know it.  This is especially important if you intend to make this process automated and executable.

Another important concept to note from this example is that I actually created a parallel flow without actually using a parallel gateway.  This done through messaging.  The first interaction with the Human Resources participant creates a blocking asynchronous message.  This means that until I receive a response from HR I cannot do anything else in respect to this process.  A message must be recieve before I can continue at the step “Receive HR Response”.   However the second point of interaction is quite different.  When I interact with the Accounting participant I don’t wait for a response.  I am not showing in this diagram how long it takes for accounting to set up a payroll account. In fact, it might take longer than what it does for the new hire to start working.  In the first two examples above the process showed that I actually cannot have the person start work until payroll is set up.   This behavior occurs when two processes intersect.

Process intersection is a concept that is hard to understand so it deserves some attention here.  In this small process example we have encountered a problem in just a few small activities.  Imagine how complicated this would get if we actually draw the entire set of activies for the other participants.   There is a way to avoid this overhead and still accurately depict your process diagram.  But you must understand process intersection to determine where to stop drawing one diagram and start on another.

The answer to this problem is that we have two different processes.  One is the process of hiring someone.  Another process is the payroll setup, in which the hiring manager is the requestor, but the hirning manager is in no way in control of this process.  Therefore when it comes to syncrhonizing activity between the manager and accounting it gets very difficult to draw from the manager perspective.   It’s better to attack this problem from the accounting participant’s perspective, and show the manager as a secondary participant.   You can additionally show escalations and notifications from the accounting perspective back to the manager, which resolves the problem of accountability for getting the job done.  But what does this have to do with hiring someone?   This is why I say it’s a process intersection, not an explicit interaction within the same process.   Attempting to mix the two very different business objectives in the same diagram will lead to a very complicated flow that takes a BPMN expert to understand.

Now back to the 3rd diagram example.  Because I’ve seperated out what accounting does by simply showing my interaction point, I can easily go forth with the current process and not have to worry about whether or not accounting has done their job or not.    We are trying to show the steps a manager takes to hire someone.  If accounting doesn’t do their job, the other diagram from the accounting perspective should notifiy the submitter (the manager) that the work has not been done by the due date.  But this is out of scope here.  Let’s assume it’s been done.   Next all we have to do is wait until the day the new hire is supposed to be at work and conduct the first day activities.   Plain, simple, understandable, and and from the perspective of the primary participant – the hiring manager.

Signal Event:

• Used for sending and receiving  signals
• A signal is a generic, simple form of communication
  • Within pools (same participant)
  • Across pools (different participants)
  • Across Diagrams
• Has no specific recipient, meaning it could be received by any participant that is listening.   (note that a participant could be a person, a system, or another process).
• The signal has a name
• Has attributes that further describe the signal
• Does not have a message body

Use case 1:  Signal an unknown participant

Perhaps the most useful feature of the signal event is signaling an unknown participant.   Often when I am in a store I hear an announcement “There is a car in the parking lot with its lights on”.  Or sometimes I get an email asking “Who is working on the xxxx account?”.    These are requests to get someone’s attention, but we don’t know specifically who the person it.

In the shopping mall example, we have a car with the lights on.  The participants are the store(s) and the shoppers.   We don’t know who’s car it is, so the announcement describes the car.   So to model this in swimlanes, you would have these two participants.   There are however a some problems with the swimlane appoach.  First, you don’t know which of the shoppers the car belongs to.    You must announce to every shopper.   Second, you don’t even know if this is in fact a shopper that owns the car.   The car could be owned by an employee, or it could be a shopper in another store.   So the actual participant in this case is anyone that might potentially park a car in the store parking lot (potential car parkers).

An announcement for a car in the parking lot with its lights on

The signal event is also limited by the attributes “Black”, “BMW”, “Plate Number x12345″.   So if I don’t have a black car I immediately ignore the rest of the message.  Or if I have a black car but it’s not a BMW I can ignore this message.  But if you have a black BMW and you don’t know what your plate number is, I’m assuming that you probably have some kind of subprocess for this.  The point is, it is the responsibility of the potential signal receiver to identify its self and act accrdingly.  Nobody is going to make you go turn off your lights.  From the perspective of the store, they have fulfilled their obligation by making the announcement.

Note the usage of the group artifact shape (dash-dot-dash box around the signal events).  This is used solely for style purposes and is not required according to the BPMN specification.  Personally I have found it to be helpful when reading diagrams, similar to how punctuation makes it easier to read text.   Case in point, if you notice the example below, it does not have the group artifiact shape around the signals.  Your eyes must wander the diagram to find the matching signal receiver.

In the email example where someone is asking who is working on a particular account, again the signal is intended for anyone who might respond.   We have potentially an account manager participant, or maybe it’s a customer service participant who gets a call from a customer but doesn’t know who the account manger is.   In this case there is a message event (the email) but not necessarily.  If the address is sent to a disribution list such as “all-account-managers” then this is more like a signal event than a message event.   Yes, it is in fact an email message.  It might even have a body attached.    But who really reads these broadcast emails anyway?   If you are like me you get 200 emails a day (on a slow day).  I read the subject line and if it’s not interesting it goes straight in the trash.   If someone is not smart enough to put the account number in the subject line which makes me actually read the message it’s not as efficient.   So in the following example diagram we cover both cases, where the subject contains the signal attributes and the other case where I have to actually read the message.

Signal sent to find out who is responsible for an account

In theory you could use either a message event or a signal event for this example.  The differenciation in this case is the fact that the sender participant might receive a response, but might not.  The message body is probably empty in this case and the subject line might read “Who owns account number A12345″.    If this isn’t my account I’ll delete the email it without reading it.   Othewise I might want to do something before the customer becomes upset with me.     The signal event simplifies this diagram because if I were to use a message event instead, I would also need several shapes to “filter” the message data to determine if it’s mine or not.    Hence, the “is it for me?” gateway, which covers the case where someone didn’t bother to but the account number in the subject line and forced me to read the message.   But if I know for a fact that all signals of this type will have the account number in the subject line, this is a true signal event and I don’t needt o evaluate if this is for me or not.  This is because a signal event has an implicit filtering built-in.   The inteneded process participant(s) must know their responsibility for receiving the message.

For persons not familiar with the notation, I chose to use an event driven gateway (the pentagon symbol insde the diamond) with two potential solutions to the problem of who owns the acocunt.  In one case, someone will respond within a reasonable amount of time.  In the other case, after a period of time has elapsed, I’m not going to wait any longer and take care of the custmomer myself.

Use case 2: Subscription

One good example I can think of for subscription is Twitter http://www.twitter.com .   Email distribution lists are on the border line between what is a message and what is a signal.   To qualify a form of communication to be a signal, it must be simple.   Emails can contain large amounts of text, so they are not simple.  Twitter is a site that allows you to broadcast some text (up to 140 characters maximum) to the world.    Being that the messages are limited in size, they must be simple.  To receive “tweets” (the broadcasts) on this site, you must subscribe to it.   With no subscriptions, the “tweet” goes nowhere.    Often the “tweet” goes out to everyone even though the message might be intended for just a few people.  So again, the participant(s) must identify themselves and determine if the information is useful and relavant to their current process instances.    In contrast, if I were to send you an email instead of broadcasting a tweet, this would indicate that I want specifically you to read it.

Another example of subscription is a stock symbol monitor.  Stock prices are streamed over the internet.  You can subscribe to specific events.  For example I want to know ABC corp’s price.  If I want to start receiving stock quotes, I simple “turn on” that stock symbol and the events will continue to stream until I turn it off.   Whenever the stock market is open, the stock streamer simply sends the quotes, and doesn’t care if anyone is receiving them or not.  Every stock symbol that has a price change will cause an signal event.

Signal event name: Stock Quote

Attributes: Up or Down, Price

Use case 3:  Synchronization

In a large process with many participants, some processes require that some activities run in parallel, but must also be synchronized.  For example, a procurement process where someone asks for some money but simultaneously starts to seek out a vendor.  The vendor can be selected but the order cannot be placed until the funds have been acquired and transferred into the appropriate account.  Let’s say that the vendor selection is handled by a special group of people.   Being that it’s a group of people the originator might not know who exactly is working with the vendor(s).  The order will not be processed until the signal to proceed has been sent.   I might want to know about the status of the process as it progresses, but I don’t really need to know the full details.

I chose to use a signal event in this example for one specific reason: There are persons involved that were not introduced and potentially have no knowledge of each other.  For example, the funds manager processes the request to transfer the money.  Because the order manager is interested in this process instance, they can now proceed with the order placement.  There is nothing in this process that says that the funds manager and the order manager are specifically aware of each other.  They are in different departments and potentially these are different processes entirely. The only thing tying the process together is a common communication channel of some kind that has the ability to broadcast messages.

Synchronization accross multiple participants

In this example the loading dock receives a package.   For the sake of argument, let’s say that the loading dock maintains a simple website that shows a tracking number and a date when it was received.   If the Originator participant does not read this website regularly (the subscribe) then the package will remain in the receiving department indefinitely.    It is in the interest of the originator participant to regularly monitor the status page to receive timely updates.   Eventually the loading dock will delete the old messages.    So if I don’t actively monitor the status page, I might not ever know the package was received.

Use case 4: Interruption

Interruption involves stopping multiple participants from performing their current activities, most likely because a major event has occurred that requires the process flow to change significantly.   A simple example of this is when a customer is requesting a home mortgage loan.  Mortgages are a complex process that requires many items on on checklist to be completed by multiple participants.   Let’s say that the customer changes their mind and decides that now is not a good time to buy a home.  Suddenly we have 10 people doing things for a process instance that no longer exists.   We don’t want anyone to continue working on this process instance because it would be a waste of time.  So a signal is sent, which would cause all interested participants to stop working on this loan.

Signal Event Name: Stop Processing

Attributes: Loan Number, Reason

Hopefully this post will help reveal the difference between a signal and a message.

Questions?  Feel free to contact me.