Back in 2012 data modelers were fighting back an invasion of JBOTs, and I was reporting back from the front lines at various conferences. JBOTs were rapidly taking over our data warehouses, destroying them to the point where they had to be rebuilt from scratch. The average lifetime of a data warehouse was becoming shorter according to scientific studies. In 2012 you could expect a complete rebuild after just over four years, which given the cost of doing so yielded quite poor return of investment. Even so, the necessity of having a data warehouse still saw almost everyone building them and keep struggling.
We had gotten ourselves into this situation by not adhering to methodologies. Data warehouses either started out with no real enforcement of a methodology or they degraded over time from having more and more deviations. This is understandable. With long lived technological artefacts you are bound to see different people working on them over time. With different people come different ideas, and creativity that extends beyond the set out methodology is rarely a good thing. Soon enough you will have parts that share no resemblance whatsoever and coming from working on one part will in no way help you when you are asked to tend to another part. The JBOT has invaded the data warehouse and transformed it into Just a Bunch Of Tables.
In the coming years we largely won the war. The fear of the JBOT became real and widespread. Rebellious new technologies with stronger enforcement of fewer principles became more widely known. Thought leaders were sharing their wisdom and people actually listened. Metadata-driven frameworks helped drive the JBOTs away. Guardians, the data governors, were put in place to stem any uproar before they grew dangerous. We were victorious. But, alas, this victory was not going to be long lasted.
In the wake of the war, while we were working on refining the methodologies, the JBOTs soon found a new home in the data lake. An idea specifically formed to host and grow JBOTs. While this idea sounded laughable to many of us, it still managed to gain traction. As it turned out, we had gravely underestimated a dark force within businesses – following the path of least resistance. Creating a lake was simple. It requires almost no thought up front. And you have a problem with the lake? Go fish!
As we mobilised to fight the lakes with more or less the same arsenal as before, the JBOTs weaponised themselves with MDS tools. It did not matter that the lakes were swamped by JBOTs and were easy targets to shoot down with proper argumentation. Thanks to the MDS the skill barrier to manufacture JBOTs had suddenly been lowered so much that they started to pop up everywhere. You’ve got data? The JBOT is only a few mouse clicks away.
It’s 2022. The world is being overrun by JBOTs and I have to return to the front lines. The war is about to start anew and I brace myself for what is to come. How will we win this time? I honestly don’t know. Perhaps we can catch them in a mesh of federated, well governed and methodologically sound data products? Perhaps we can put the model at the core of the business to preclude the JBOTs in the first place? Perhaps this will bring about new discoveries rendering the JBOTs obsolete?
Anyway, just waiting for them to eventually kill themselves and hope that we won’t go down with them is not a part of my plan.
The JBOT has returned. Join the fight!
