Loops in Informatica
The simplest way to implement Loops in Informatica is explained below.
Source Name: Product_Details
Sample Input:
Pcode | Pname | Pdetails |
Water12 | Himalaya Water | Price,1.20;Qty,250ml;Brand,Himalaya;Class,Liquid |
Snack23 | Mad Pringles | Price,0.65;Qty,165 g;Brand,Mad Ltd;Class,Snacks;Batch,12312334;Exp,12/Feb/2012 |
note: in the pdetails column each product attribute separated with semi colon “;”
Targets: Product_Dim, Product_Details
Expected Output:
Product_Dim
Pskey | Pcode | Pname |
P1 | Water12 | Himalaya Water |
P2 | Snack23 | Mad Pringles |
Product_Details
Pskey | Sno | Type | Value |
P1 | 1 | Price | 1.20 |
P1 | 2 | Qty | 250ml |
P1 | 3 | Brand | Himalaya |
P1 | 4 | Class | Liquid |
P2 | 1 | Price | 0.65 |
P2 | 2 | Qty | 165 g |
P2 | 3 | Brand | Mad Ltd |
P2 | 4 | Class | Snacks |
P2 | 5 | Batch | 12312334 |
P2 | 6 | Exp | 12/feb/2012 |
If you closely observe the above Source and Target sample data, we need to have some kind of loop logic to extract the “Product_Details” from Pdetails column. Assume that each product has a different set of attribute values which can’t be predicted I.e. the number of attributes for each product varies.
Informatica Mapping:
Step1: Create a new table or a Flat file with the below structure and values
Tablename : Loop_Tab
| You can create this table using a simple PL/SQL script or VB script, in my example I created a table using below script. CREATE TABLE LOOP_TAB ( NO_LOOPS NUMBER(3), LOOP_SEQ NUMBER(3) ); BEGIN FOR I IN 1..100 LOOP FOR J IN 1 .. I LOOP INSERT INTO LOOP_TAB VALUES (I,J); END LOOP; END LOOP; COMMIT; END; |
Step2: Create a mapping as shown in below picture
- Pull the “Product_Details, LOOP_TAB” sources to mapping designer.
- Create a “Sequence Generator” transformation as show in picture.
- Create a Expression Transformation and pull all ports from SQ_ Product_Details and also pull the NEXTVAL from SEQTRANS.
Now add a new output port in Expression and name it as “out_NO_LOOPS” also change the data type to Integer, Now write the below expression in that output port “LENGTH(PDETAILS)-LENGTH(REPLACECHR(1,PDETAILS,';','')) “.
Basically from the above expression we will get to know how many attributes exists from a product.
- Now take a Joiner Transformation and Pull “LOOP_TAB” ports as Master Table and All the ports from “Exp_Find_No_loops” as Detail ports and apply Normal Join on NO_LOOPS= out_NO_LOOPS .
- Now Pull the targets “PRODUCT_DIM, PRODUCT_DETAILS” to mapping designer and connect them as per above mapping picture.
- You are done , All the best … Srikanth Mullamuri
srikanth..
ReplyDeleteNice implementation of the logic.
Can you please tell in detail what logic you have used in substr functions in expression transformation.
thanks
shankar
Hi Shankar,
ReplyDeleteIn the expression take 2 variable ports and name as v_AttrStratPos and v_AttrEndPos the data type for this local variables as integer.
Now write the below expression
v_AttrStratPos=IIF (LOOPSEQ=1, 1, INSTR (PDETAILS, ’;’, 1, LOOPSEQ-1)+1)
v_AttrEndPos = INSTR (PDETAILS, ’;’, 1, LOOPSEQ)-1
By doing this you can extract the attribute portion from start to end based on loop sequence
For ex: “Price,1.20;Qty,250ml;Brand,Himalaya;Class,Liquid”
In the above sample record the loop sequence two the attribute the start and end positions
v_AttrStratPos =12
v_AttrEndPos=20
If you get these start and end position it a very simple to extract the strings
O_TYPE = SUBSTR (PDETAILS, v_AttrStratPos, INSTR(PDETAILS, ’,’, v_AttrStratPos)-1)
O_VALUE= SUBSTR (PDETAILS, INSTR (PDETAILS, ’,’, v_AttrStratPos)+1, v_AttrEndPos)
Thanks
Srikanth Mullamuri
Hi Kavi,
ReplyDeleteHope you keeping well.
1) In my particular example if you closely observe the target table out puts for Product_Details and Product_Dim, The PSKEY is populated from Sequence Generator and SNO was populated from Loop_Tab.Loop_Seq column.
Note: The primary key in Product_Dim Table is PSKEY
And for Product_Details Table the primary key is PSKEY and SNO
2) Yes may be you can.
But here I am explains how we can implement loops in informatica using some example. Also I personally not prefer stored proc in informatica because of performance.
Cheers,
Srikanth Mullamuri
This is best one to see with good information about the etl tools. Thanks for sharing the best information.
ReplyDeleteDatastage Online Training
I just thought that i'd drop you a line and tell you that I think you're doing a fantastic job.
ReplyDeleteThanks! if you want, you can check out my blog at
Informatica Interview Questions and Answers
Thank you so much for sharing this. Can you also show how you would implement this if the source was from a relational table where the details appear in multiple rows instead of from a single column value?
ReplyDeleteI have a master and details from the source that I need to load to the target, also in a master and details structure much like your target.
Thank you in advance for your help in this.
Bonnie
Hi Bonnie,
DeleteCould you please give some example exactly what you looking for, it's appear fairly simple if every thing in relational table and you don't need any loop logic.
Thanks,
Srikanth Mullamuri
You can use the new "Loop Transformation in PowerCenter"
ReplyDeletesee https://community.informatica.com/solutions/loop_transformation_in_powercenter
Thank you so much for throwing light on such a important aspect of Informatica. Beginners and even professionals can actually benefit from this.
ReplyDeleteInformatica Read JSON