Why You Should Never Run BPM5 and 6 on the Same Database

Surely, this is a sick joke.

'An error has occurred. Please check your constraints and try again.'

I DID ALREADY! 5 TIMES!

We had run into a nasty bug while testing an application for our client. So now, it was important that we correctly set expectations for how long it would take.

Us: "Please understand this could take awhile to solve"

Global Bank: "But..our testing deadline is in two days. Can you fix it by then?"

And no matter what, we wouldn't overcommit

Us: Well..

I SAID, we wouldn't overcommit!!

Us: ...sure thing!

Oh, come on!

A Little About Global Bank

Global Bank* is a financial corporation that constantly examines individuals and businesses for credit accounts. And to manage these complex processes, they use RedHat JBoss BPMS.

Late last year, we helped them upgrade to BPMS 6, which makes it even easier to manage processes at scale.

We thought everything was fine...until we started testing

* name changed for security reasons

Testing, Testing

This is going to be easier than I thought.

One main change from v5 to 6 is the addition of high level services. These make it easier to interact with any process or task in the system.

So to test their application, we just needed to verify that we could perform basic operations on credit application tasks or processes (start, claim, abort).

"Hey guys, everything is working well so far!"

We planned to finish the testing tomorrow morning.

And who knows, maybe we'd even get some of that "spare time" people talk about.

Good Morning

As a sanity check, we ran the exact same test that succeeded on the previous day. But this time..

[ORA-00001: unique constraint (GBNK_GBL_JBPM.SYS_C0089525) violated]

Hmm...

As part of the upgrade, we had sent the database team a script to update the sequences on a few tables.

But OUR script wasn't wrong, was it?

Ah, I got it!

"It's the database team! They didn't run the script correctly!"

Phew.

And after we contacted them to run it again, our tests worked fine.

...until next morning.

[Constraint violated. Please try again]

What IS GOING ON!?

The clock was ticking. Maybe it was time for us to take responsibility. Maybe our scripts were broken.

OR....

"Setup a meeting with the database team! I bet someone was playing around with the database and changed the sequences."

...right?

Eventually, we did verify our scripts too.

But we couldn't figure out why these tests only worked sometimes.

Is this someone's idea of a SICK JOKE!?

"Hey guys, is there anyone else using this database that is having problems?"

"No, other teams are testing older versions of the app just fine"

And that's when it hit us.

"Older versions of GlobalBank's application used jBPM 5."

A Tiny Change From BPM 5 to 6

In BPM5, all entities use the default hibernate sequence (HIBERNATE_SEQUENCE). But in BPM6, entities now use their own sequence (task_id_seq, etc).

Why did they change that? Just to mess with us, of course!

Anyway, when other teams created records with BPM5, they were inserted using the hibernate sequence. But the individual sequences used in jBPM6 aren't updated.

So when we insert, we get constraint violations because those primary keys have already been used.

Good News, Bad News

A simple query confirmed that the hibernate sequence had a higher value than the individual table sequences.

That was the good news: we found the problem.

The bad news? We'd need to request a separate database to test the new version. And that would take much longer.

But...you can't have it all.

Just Remember: When you upgrade from jBPM5 to 6, DO NOT RUN THEM IN THE SAME DATABASE.

Happy Coding,

-T.O.