1.5.1.2 Databases or Flat file Data stores:

The data collector needs to temporary stores his extracted data, to provide them to the SNA component. The data collector can accomplish that in two ways, store them in a database or in separate flat file formats, like xml, CSV or an own simple data structure.

The data store used for the extracted and transformed data from the source systems will be stored in a database system with several database schemas per tool type (Source system type). The elaboration and explained conclusion with the decision is stated in the appendix under section: Appendix Database Flat Files

The database will have several database schemas for every used tool type, which represents a source system. This has an impact on the architecture, because so every data gatherer, which is used to gather the data from one type of source system, has its own database parser with own connection and user credentials. An more explained and elaboration and explained conclusion to that decision is stated in the appendix under section: Appendix Database Flat

1.5.1.3Data gathering Scenario

To imagine a simple Data collection scenario, here the data gathering process from a SVN repository, we provide one Scenario to collect data from a Data Source. This helps to imagine what an adaptor needs for components to perform its tasks.

In the scenario above you can see, what for possibilities we have to extract an SVN Log for a Repository. Several Clients updates a repository, as you can see in the customer description, it can happen that several teams work in one repository or a team works on an isolated repository for itself. Every Repository has an associated Log in the SVN server. So there came two main options to get the data from the Log. These options are also applicable on the other systems where data needs to be gathered from. These two options are based on the push & pull conclusion.

Resulting types of gathering scenario are identified:

The two gathering options:

The two options have in several ways an impact of the architecture, one is that option one gives only hardcoded access to the source systems, where option two gives with the use of standardised and well proved libraries access to the databases or source systems to gather the information. A set of connectors can be configured and provided to separate this part and bring also more extensibility and flexibility, with a low price. So the data gatherer gets its own connection library which defines the access to the system, for example a simulated SVN client or a direct connection to the source database. These methods are lightweight and use given infrastructures. For further details about this decision, please have a look in the appendix under section: Data gathering options