Anchor Modeling

At the Next Generation DWH Modeling event tomorrow, we are proud to announce concurrent-temporal Anchor Modeling, which adds support for multi-temporal modeling, information reliability, and enhanced time traveling. The presentation, with the subtitle “What if you could travel through time and become someone else?” is available by clicking this link: Back to the Moment.

We finished the research during the autumn of last year and have since been working on producing a scientific paper and implementing the code generation in our modeling tool. Neither is yet finished, but we have enough material, and a couple of actual (hand-made) implementations, to go public. We expect to have code generation finished after the summer and the paper to follow after that. There is an example showing inflation estimates using concurrent-temporal modeling available by clicking here.

The Simple Bitemporal Test made using concurrent-temporal Anchor Modeling can be found by clicking here. Please note that this was an early experiment, and that we have changed the way the restatement checking works, as well as added insert, update, and delete triggers since. It does, however, convey the general idea of concurrency and reliability.

We have made a logotype and some smaller icons that you can use to show that you support or use Anchor Modeling. Below are low-res PNG and vector SVG formats of the different logotypes and icons:

Feel free to use them. The color is #c80514 or RGB(200, 5, 20), and the two squares symbolise anchors. The font in the trademark symbol is Arial with reduced character spacing, and ANCHOR is written using Magmawave by Yanko Tsvetkov available under a Creative Commons License:

We are proud to be presenting Anchor Modeling at the Data Modeling Zone in Europe (Hannover, Germany) on September 23rd-24th and in America (Baltimore, Maryland) on October 8th-10th. Even better is that if you use the promotional code: LarsDMZPromo when you register for America or: LarsDMZEuropePromo for Europe you will recieve a $100 discount of the registration price! We hope to see you there.

We’ve just pushed version 0.96 from test to release, making it available by clicking here or on the “Online Modeler” menu item. Please see the three previous posts for details of what has changed since 0.95 (which is still available online here). As well as the listed changes, the following has been done:

• Fixed a bug in the column name for attributes in the insert trigger.
• Fixed bug in update trigger for knotted historized attributes.
• Fixed bug where the navigation miniature was updated at every frame of a dragging operation.
• Fixed a bug where the modeler could crash after a delete operation, trying to write out a non existent node name.
• Fixed a bug in the update trigger for the ChangedAt column.
• Fixed a bug where the miniature could get stuck in a resize loop causing the browser to hang.
• Changed the edge stiffness algorithm (back) to a more physically accurate one.

This also means we are beginning development of 0.97, of which a test version soon will be made available. Trust me that we have some very interesting features coming in the next version!

We have added space partitioning to the graph layout engine in the modeling tool (rev. 411 now live in test). This will improve performance for very large graphs. The idea is to first divide the space containing the graph into a number of sections called partitions, then calculate the “center of mass” for each partition with respect to the contained nodes, and then calculate the collective repelling force in each partition by the other partitions. Rather than having to do an exponentially growing iteration over nodes and other nodes, we can now iterate only over nodes in the same partition, calculate their respective forces with an addition of the already calculated force in the partition, and move to the next partition.

Looking at recent research in graph layout, an octree is used for the partitioning. However, constructing an octree was not fast enough, so we decided upon another method. The partitioning is made by dividing the node coordinates by the partition size and converting the result to an integer. A hashtable is used to contain the partitions with a pointer to an array of the contained nodes.

This works well, except that since we are continuously moving nodes around, we may get into a situation where nodes are “jumping” between two different partitions, and the layout never reaches equilibrium. We were able to remove such jittering by making the partitions fuzzy, with respect to their size. Every iteration of the layout, the size of a partition will be somewhat different, resulting in the nodes always settling down in a single partition.

In the screenshot above you can see the different partitions in grey (switch on debugging in the test version to see for yourself), and a grey small square in each partition indicating its center of mass.

We have updated the test version of the modeling tool to version 0.96 revision 407:

https://www.anchormodeling.com/modeler/test

Updates in revision 407 (and earlier):

• Fixed bug with restatement control rendered useless in bitemporal models.
• Added restatement control for ties.
• Fixed bug where setInterval was being called too many times in older browsers.
• Replaced setInterval code with setTimeout in older browsers.
• Sped up rendering even further by pushing run once-if-statements to init-functions.
• Added insert triggers on the latest views for ties, with idempotency control.
• Fixed all insert triggers such that when knots are involved, either the ID or the value column from the knot can be used, whereas previously the value column had to be specified.
• Added delete keyboard shortcut ‘d’.
• Added a (long overdue) consistency check for ties, such that roles cannot become “deep-equal”, according to the XML Schema for anchor models.
• Roles in a tie can now have the same name, as long as they do not refer to the same anchor.
• Removed triggers from anchors that have no attributes (to fix broken code generation).
• Temporalization metadata is now recognized when models are loaded.
• Fixed bug that made input fields repeat more than once when selecting bitemporal output.
• Improvements of how knots are handled in the triggers.
• The naming convention now allows upper and lower case Latin-1 characters.
• The naming convention is changed in order to support multiple attributes on the same anchor referencing the same knot. In order to be consistent, the name of the identity in the attribute was changed accordingly.
  This change is NOT backwards compatible!
• Edit (revision 409 released on Oct 30):
  Added an option in the “Defaults” menu to change back to the “Original” naming convention. This quickly became a public request!

Use this with some caution as some changes are not backwards compatible. Should you decide to use this version you cannot go back to a previous version. You cannot mix the output from this version with an existing implementation either, since the naming convention will differ. However, it should be quite easy to migrate to a new version if you have a new and an old implementation side by side.

We have set up a development snapshot of the modeler (v0.96) in order to make it available for testing. It can be reached by clicking the link below:

https://www.anchormodeling.com/modeler/test

This version currently includes improvements to the graphical user interface, an abundance of descriptive tooltips, new and more perfomant rendering in the very latest browsers, and idempotency control.

Idempotency controls whether the insert and update triggers should ignore and discard any attempts to insert or update a value that is identical to the currently stored value (in changing time). Say for example that the attribute “Surname” has the value “Rönnbäck” from “2004-06-19” and that it has been marked as idempotent. Then any attempt to insert the value “Rönnbäck” with a later changing time will be silently ignored.

As such, idempotency can be of great use in systems that receive many repeated values where it also can be ensured that information arrive in changing time order. In a situation where information may arrive out of changing time order, idempotency should be avoided, since there is no way of knowing what values may come between the one you are trying to insert and the currently stored one.

The online modeler has been updated to version 0.95, which includes the following features and fixes:

NEW FEATURES

• Full support for bitemporal modelling.
• Latest views behave like tables thanks to insert, update, and delete triggers.
• Restatement control for historized attributes.
• Searching in models.
• Optionally hidden mnemonics for less cluttering.
• Optionally hidden attributes using implode/explode on models or parts of a model.
• Metadata is now optional.
• Improved layout for smaller screens.
• Keyboard shortcuts.
• Editing window can be closed by pressing ‘Return’.
• First field in the editing window automatically gets focus.
• Public models can be shared using a simple URL.
• Twitter integration for public models.
• Models can now be deleted from the cloud.
• Models can now have a description.
• Improved model browser for the cloud.
• Straight edges can be set to improve performance on older hardware.
• Capsules (schema owner) can be specified.

BUG FIXES

• Added checks for mnemonic uniqueness.
• Fixed the issue where historized ties could get an uneditable time range.
• Fixed keyboard shortcuts from being active in all editable fields.
• Fixed CR+LF problem in the generated SQL.
• Ties will now always have an identifier.
• Reduced the number of string objects created by the modeller.
• Fixed bug where cardinalities were not shown in the SVG output.
• Fixed missing items in the generated HTML documentation.
• Changed the z-index of menus so they overlap gracefully.
• Refactored global variables that were just global because of laziness.
• Rewrote the delete operation, so that it will no longer go berserk.
• Fixed mouse wheel zooming on Macs.
• Fixed bug in imploded edges.
• Fixed the issue where only 20 items from the cloud would be returned.
• Removed recording time from knots.
• Improved security in the cloud connection.
• Fixed the isEmpty function for boolean values.
• Fixed bug with empty WHERE clause for 1-1 ties.
• Fixed select all (Ctrl+A) in the generated code window.
• Fixed bug where default capsule would not affect new items.
• Fixed bug where default identity would not affect new anchors.

IMPORTANT: The XML may be updated such that it is no longer backwards compatible. Please save and keep a copy of your model XML before migrating to a newer version, in case you need to go back. The old versions can be found by clicking here.

In order to ensure temporal entity integrity in a bitemporal model a few constraints have to be in place:

• Primary key constraint over (ID, ChangedAt, RecordedAt, ErasedAt) that ensures that no duplicate rows can exist.
• Unique constraint over (ErasedAt, ID, ChangedAt) that ensures that no temporal duplicates can co-exist for the same entity and version.
• Check constraint that ensures that RecordedAt < ErasedAt.
• Check constraint that ensures no overlaps in the recording timeline.

These have now been added to the modeling tool in the test version, and can optionally be switched off from the “Defaults” menu. Below is a script that tests the functionality of these constraints.

Borrowing concepts from Bitemporal Anchor Modeling we have created a suggestion for how to temporalize Dimensional Modeling. It makes use of the distinction between immutable and mutable information to introduce a dimensional anchor for every dimension. It uses the rewind and point-in-time parametrized views. It used the proper primary keys together with foreign keys to achieve table elimination (immutable data is not touched during execution of a query). It uses constraints to ensure temporal entity integrity.

Note that this does in no way imply that we favor Dimensional Modeling over Anchor Modeling. It just shows that the techniques developed in Anchor Modeling can be widely applied in other modeling techniques.