[Metafacture] Processing context information with Metafacture

[Thomas Gängler]

Hi guys,

we reached a point in our project, where we need to be able to process 
context information via our ETL engine (which is Metafacture atm) to be 
able to have the necessary information at the end of the pipeline to 
insert the processed data at the right places in our datastore.
So we need to figure out a rather easy way to this, and our question 
right now is:

Is this really possible with the current Metafacture architecture?

We think or have the impression that Metafacture is optimized for/ 
designed for String-based processing, i.e., we can't really see where to 
put metadata for records, entities and key/values pairs. However, right 
now we need to be able to attach metadata á la node ids for entities to 
be able to know which entity was modified at the end. Otherwise, we'll 
get a somehow fuzzy resultset where we are not able to process this 
changeset back to the datastore.
Please let me know, what you think about this issue and whether you can 
imagine if it is rather easy to implement (i.e. without re-implementing 
every part of Metafacture). Right now I can just imagine a way where I 
need to change the processing component from being able to process 
Strings into being able to process Objects, e.g., the record is an 
object, the entity is an object etc. Unfortunately, this would result in 
a complete re-implementation of Metafacture (which also gives me the 
"opportunity" to start from scratch (where we don't have the capacities 
to do so ...)).

Thanks a lot in advance.

Cheers,


Thomas