The new Scribe Insight 7.9 release offers multi-threaded integration processes for all four of the integration process types available within the Scribe Console, but, let’s clarify.

As you may (or may not) know, Scribe Insight is made up of two user interfaces; The Workbench, which is used to build the translation mapping from a data source to one or several targets, and the Scribe Console where you set up the management parameters to automate the integration processes. Previously, Scribe had only offered multi-threaded processing using a MSMQ based integration process. This is true multi-threaded translation of data, with each record of the source record set having its own thread for translation, allowing for simultaneous translation of data from the source to target. The following diagram represents a typical Scribe queue based integration process.


Previous to the Scribe 7.9 release, only one thread was reserved for processing all File, Time and Query based integration processes. That meant that if you had a File based integration process and a Query based integration process, that was triggered to run at the same time (by coincidence or otherwise), the processes would be queued and first come first serve would be run in a serial fashion; one process waiting until the one in front of it had finished.

Now with release 7.9 you can configure your integration processes, regardless of type, to run simultaneously. This means, using the above example, that the File based integration process and the Query based integration process can run at the same time.

This is where a slight bit of clarification comes in. With a Queue based integration process, the source record set is split up into individual XML documents for each record in the record set, and then processed individually. With the new 7.9 release, the File, Time and Query processes are multi-thread, but NOT the record set translation. With the File, Time and Query based processes, the source data is still cashed as a record set and the records processed one at a time. So, the integration processes are multi-thread, but not the data translations themselves.

With this in mind, you need to take a look at available resources on the processing machine. Queue based integration processes are CPU heavy and File, Time and Query processes are RAM intensive. Adding the multi-thread capability to your integrations will increase the CPU. Just something to keep in mind.

All in all, the Scribe Insight 7.9 release is a great improvement, giving you the ability to manage your integration processes to another degree higher than previous versions.

For more information about C5 Insight or this blog entry, please Contact Us.