Tuesday, 15 March 2011


Loops in Informatica


The simplest way to implement Loops in Informatica is explained below.

Source Name: Product_Details

Sample Input:

Pcode
Pname
Pdetails
Water12
Himalaya Water
Price,1.20;Qty,250ml;Brand,Himalaya;Class,Liquid
Snack23
Mad Pringles
Price,0.65;Qty,165 g;Brand,Mad Ltd;Class,Snacks;Batch,12312334;Exp,12/Feb/2012


note: in the pdetails column each product attribute separated with semi colon “;”

Targets: Product_Dim, Product_Details

Expected Output:

Product_Dim
Pskey
Pcode
Pname
P1
Water12
Himalaya Water
P2
Snack23
Mad Pringles

Product_Details
Pskey
Sno
Type
Value
P1
1
Price
1.20
P1
2
Qty
250ml
P1
3
Brand
Himalaya
P1
4
Class
Liquid
P2
1
Price
0.65
P2
2
Qty
165 g
P2
3
Brand
Mad Ltd
P2
4
Class
Snacks
P2
5
Batch
12312334
P2
6
Exp
12/feb/2012


If you closely observe the above Source and Target sample data, we need to have some kind of loop logic to extract the “Product_Details” from Pdetails column. Assume that each product has a different set of attribute values which can’t be predicted I.e. the number of attributes for each product varies.



                                                                                                                                                    
Informatica Mapping:

Step1: Create a new table or a Flat file with the below structure and values


Tablename : Loop_Tab
No_Loops
Loop_Seq
1
1
2
1
2
2
3
1
3
2
3
3
4
1
4
2
4
3
4
4
.
.
.
.
etc
etc

You can create this table using a simple PL/SQL script or VB script, in my example I created a table using below script.

CREATE TABLE LOOP_TAB (
 NO_LOOPS NUMBER(3),
LOOP_SEQ NUMBER(3)
);

BEGIN
FOR I IN 1..100 LOOP
            FOR J IN 1 .. I LOOP
                       
                         INSERT INTO LOOP_TAB VALUES (I,J);

            END LOOP;
END LOOP;

COMMIT;
END;




Step2: Create a mapping as shown in below picture




  1. Pull the “Product_Details, LOOP_TAB” sources to mapping designer.
  2. Create a “Sequence Generator”   transformation as show in picture.
  3. Create a Expression Transformation and pull all ports from SQ_ Product_Details and also pull the NEXTVAL from SEQTRANS.
Now add a new output port in Expression and name it as “out_NO_LOOPS” also change the data type to Integer, Now write the below expression in that output port LENGTH(PDETAILS)-LENGTH(REPLACECHR(1,PDETAILS,';','')) “.
Basically from the above expression we will get to know how many attributes exists from a product.
  1. Now take a Joiner Transformation and Pull “LOOP_TAB” ports as Master Table and All the ports from “Exp_Find_No_loops” as Detail ports and apply Normal Join on NO_LOOPS= out_NO_LOOPS .
  2. Now Pull the targets “PRODUCT_DIM, PRODUCT_DETAILS” to mapping designer and connect them as per above mapping picture.
  3. You are done , All the best … Srikanth Mullamuri  
  4.  

    9 comments:

    1. srikanth..
      Nice implementation of the logic.
      Can you please tell in detail what logic you have used in substr functions in expression transformation.

      thanks
      shankar

      ReplyDelete
    2. Hi Shankar,

      In the expression take 2 variable ports and name as v_AttrStratPos and v_AttrEndPos the data type for this local variables as integer.

      Now write the below expression

      v_AttrStratPos=IIF (LOOPSEQ=1, 1, INSTR (PDETAILS, ’;’, 1, LOOPSEQ-1)+1)


      v_AttrEndPos = INSTR (PDETAILS, ’;’, 1, LOOPSEQ)-1

      By doing this you can extract the attribute portion from start to end based on loop sequence

      For ex: “Price,1.20;Qty,250ml;Brand,Himalaya;Class,Liquid”

      In the above sample record the loop sequence two the attribute the start and end positions


      v_AttrStratPos =12
      v_AttrEndPos=20

      If you get these start and end position it a very simple to extract the strings

      O_TYPE = SUBSTR (PDETAILS, v_AttrStratPos, INSTR(PDETAILS, ’,’, v_AttrStratPos)-1)

      O_VALUE= SUBSTR (PDETAILS, INSTR (PDETAILS, ’,’, v_AttrStratPos)+1, v_AttrEndPos)

      Thanks
      Srikanth Mullamuri

      ReplyDelete
    3. Hi Kavi,

      Hope you keeping well.

      1) In my particular example if you closely observe the target table out puts for Product_Details and Product_Dim, The PSKEY is populated from Sequence Generator and SNO was populated from Loop_Tab.Loop_Seq column.

      Note: The primary key in Product_Dim Table is PSKEY
      And for Product_Details Table the primary key is PSKEY and SNO

      2) Yes may be you can.
      But here I am explains how we can implement loops in informatica using some example. Also I personally not prefer stored proc in informatica because of performance.

      Cheers,
      Srikanth Mullamuri

      ReplyDelete
    4. This is best one to see with good information about the etl tools. Thanks for sharing the best information.
      Datastage Online Training

      ReplyDelete
    5. I just thought that i'd drop you a line and tell you that I think you're doing a fantastic job.
      Thanks! if you want, you can check out my blog at

      Informatica Interview Questions and Answers

      ReplyDelete
    6. Thank you so much for sharing this. Can you also show how you would implement this if the source was from a relational table where the details appear in multiple rows instead of from a single column value?

      I have a master and details from the source that I need to load to the target, also in a master and details structure much like your target.

      Thank you in advance for your help in this.

      Bonnie

      ReplyDelete
      Replies
      1. Hi Bonnie,

        Could you please give some example exactly what you looking for, it's appear fairly simple if every thing in relational table and you don't need any loop logic.

        Thanks,
        Srikanth Mullamuri

        Delete
    7. You can use the new "Loop Transformation in PowerCenter"

      see https://community.informatica.com/solutions/loop_transformation_in_powercenter

      ReplyDelete
    8. Thank you so much for throwing light on such a important aspect of Informatica. Beginners and even professionals can actually benefit from this.

      Informatica Read JSON

      ReplyDelete

    Total Pageviews

    14,403