Showing posts with label columns. Show all posts
Showing posts with label columns. Show all posts

Friday, March 30, 2012

Merge join not matching "correctly"

And by correctly, I mean the way it *should* match, of course.

I've got 2 data sources, using a left outer join, matching 2 columns. Whatever is the right side of my table is never matching and returning data. Here's my basic setup:

OLE DB Source 1 (table1)
Sel Name Order JoinKey
no UniqueID 1 yes
yes Column1 0 no
...

OLE DB Source 2 (table2)
Sel Name Order JoinKey
yes ID 1 yes
yes Columns 0 no
...

There is a link (arrow) between UniqueID and ID, and the join type is "Left Outer Join". When I execute the statement "SELECT * FROM Table1 LEFT JOIN TABLE2 ON TABLE1.UniqueID = Table2.ID", the data returns correctly to me. What am I missing with the properties I've set above with the merge join?


some more information... my two queries are from variables...

"select * from Table1 ORDER BY UniqueID ASC"
and
"select * from Table2 ORDER BY ID ASC"

|||

Are the join columns the same data type?

I set up a similar scenario, and it's working fine. It looks like you've set your SortKeys, etc, so it should work.

Have you added data viewers before the merge join to verify the data is what you expect?

|||Just a general observation: it seems very odd to only be selecting the key from the right side in a left join. Maybe you have a reason for doing it that way, but as a troubleshooting step I'd swap it to the left side to see if that changed anything.

I don't know what data types your ID columns are or if this even applies, but the SSIS components perform case-sensitive matches while SQL Server's default collation is case-insensitive. This allows things that match in SQL not to match in SSIS.
|||

papalarge wrote:

And by correctly, I mean the way it *should* match, of course.

I've got 2 data sources, using a left outer join, matching 2 columns. Whatever is the right side of my table is never matching and returning data. Here's my basic setup:

OLE DB Source 1 (table1)
Sel Name Order JoinKey
no UniqueID 1 yes
yes Column1 0 no
...

OLE DB Source 2 (table2)
Sel Name Order JoinKey
yes ID 1 yes
yes Columns 0 no
...

There is a link (arrow) between UniqueID and ID, and the join type is "Left Outer Join". When I execute the statement "SELECT * FROM Table1 LEFT JOIN TABLE2 ON TABLE1.UniqueID = Table2.ID", the data returns correctly to me. What am I missing with the properties I've set above with the merge join?


Papa,

In the top of the sugestions above:

Are the OLE DB source components joined directly to the Merge join or there are other transfoms in between? The second could alter the original order of the rows.

Have you tried placing a sort transfom before each merge join input? I know you are using a order by in the queries; but just to test it.

BTW,

Why are you using Merge Join?

I noticed in your original post that you can run a query where you actually join the 2 tables in a single query. Did you know you could place the same query in a single OLE DB Source component? That would give you better performance and less trouble. If a single query is not an option; what about having a 1 table OleDB source component that goes to a Lookup transform to add the columns from the 2nd table.

|||Can't believe I didn't think of that. Yeah, just ended up using a merge in sql, and not through the IDE. Thanks for the help all... still don't know exactly why the merge results weren't connecting. I guess the next step was going to be getting a data view control working so I could see exactly what's returning.

Thanks again...
sql

Merge Join

All,

I need to use Merge Join transformation to join two sources. One is from a PIVOT transformation and one of the output columns is ISSORTED, the other is from an OLE DB Source using a query. The Merge Join transformation requires both input source have to be sorted. I cannot find the ISSORTED property on the OLE DB Source!!

I tried to use Derived/ copy transformations but cannot find the property also. How can set the OLE query sorted in order to use the MergeJoin?

Thanks a lot

Jessie,

You have 2 options.

1) Sort the data in your query in the OLE DB Source using an ORDER BY clause. If you do this you will have to manually set IsSorted=TRUE - you do this in the Advanced Editor.

2) Use a SORT component to sort the data.

Note that setting IsSorted=TRUE does NOT sort the data for you. It only tells SSIS that the data is sorted. If you set IsSorted on a data-path that is not sorted then you will get the wrong results coming out of the MERGE JOIN.

-Jamie

|||Thank you so much Jamie|||

Hi, jamie,

This might be a dummy question.

I got the same problem as Jessie's, but when I went into the Advanced Editor of OLE DB Source, I could not find where to set up the Issorted=True. Could someone help me to find out?

TIA

|||

john zhang wrote:

Hi, jamie,

This might be a dummy question.

I got the same problem as Jessie's, but when I went into the Advanced Editor of OLE DB Source, I could not find where to set up the Issorted=True. Could someone help me to find out?

TIA

It is a property of the output. OLE DB Source only has one output (plus the error output) so its not too difficult to find.

-Jamie

Merge Join

I am trying to understand mege joins. In BOL it reads "The merge join
requires that both inputs be sorted on the merge columns, which are defined
by the equality (WHERE) clauses of the join predicate."
What does it mean "both inputs must be sorted"?
Message posted via droptable.com
http://www.droptable.com/Uwe/Forums...erver/200603/1
> What does it mean "both inputs must be sorted"?
That you on each table either have a clustered index on the columns over which you do the join, or a
non-clustered index that covers the data that is needed from the table.
Tibor Karaszi, SQL Server MVP
http://www.karaszi.com/sqlserver/default.asp
http://www.solidqualitylearning.com/
Blog: http://solidqualitylearning.com/blogs/tibor/
"cbrichards via droptable.com" <u3288@.uwe> wrote in message news:5d9898960f213@.uwe...
>I am trying to understand mege joins. In BOL it reads "The merge join
> requires that both inputs be sorted on the merge columns, which are defined
> by the equality (WHERE) clauses of the join predicate."
> What does it mean "both inputs must be sorted"?
> --
> Message posted via droptable.com
> http://www.droptable.com/Uwe/Forums...erver/200603/1

Merge Join

I am trying to understand mege joins. In BOL it reads "The merge join
requires that both inputs be sorted on the merge columns, which are defined
by the equality (WHERE) clauses of the join predicate."
What does it mean "both inputs must be sorted"?
Message posted via droptable.com
http://www.droptable.com/Uwe/Forum...server/200603/1> What does it mean "both inputs must be sorted"?
That you on each table either have a clustered index on the columns over whi
ch you do the join, or a
non-clustered index that covers the data that is needed from the table.
Tibor Karaszi, SQL Server MVP
http://www.karaszi.com/sqlserver/default.asp
http://www.solidqualitylearning.com/
Blog: http://solidqualitylearning.com/blogs/tibor/
"cbrichards via droptable.com" <u3288@.uwe> wrote in message news:5d9898960f213@.uwe...[vbcol
=seagreen]
>I am trying to understand mege joins. In BOL it reads "The merge join
> requires that both inputs be sorted on the merge columns, which are define
d
> by the equality (WHERE) clauses of the join predicate."
> What does it mean "both inputs must be sorted"?
> --
> Message posted via droptable.com
> http://www.droptable.com/Uwe/Forum...server/200603/1[/vbcol]sql

Merge Join

All,

I need to use Merge Join transformation to join two sources. One is from a PIVOT transformation and one of the output columns is ISSORTED, the other is from an OLE DB Source using a query. The Merge Join transformation requires both input source have to be sorted. I cannot find the ISSORTED property on the OLE DB Source!!

I tried to use Derived/ copy transformations but cannot find the property also. How can set the OLE query sorted in order to use the MergeJoin?

Thanks a lot

Jessie,

You have 2 options.

1) Sort the data in your query in the OLE DB Source using an ORDER BY clause. If you do this you will have to manually set IsSorted=TRUE - you do this in the Advanced Editor.

2) Use a SORT component to sort the data.

Note that setting IsSorted=TRUE does NOT sort the data for you. It only tells SSIS that the data is sorted. If you set IsSorted on a data-path that is not sorted then you will get the wrong results coming out of the MERGE JOIN.

-Jamie

|||Thank you so much Jamie|||

Hi, jamie,

This might be a dummy question.

I got the same problem as Jessie's, but when I went into the Advanced Editor of OLE DB Source, I could not find where to set up the Issorted=True. Could someone help me to find out?

TIA

|||

john zhang wrote:

Hi, jamie,

This might be a dummy question.

I got the same problem as Jessie's, but when I went into the Advanced Editor of OLE DB Source, I could not find where to set up the Issorted=True. Could someone help me to find out?

TIA

It is a property of the output. OLE DB Source only has one output (plus the error output) so its not too difficult to find.

-Jamie

Merge Join

I am trying to understand mege joins. In BOL it reads "The merge join
requires that both inputs be sorted on the merge columns, which are defined
by the equality (WHERE) clauses of the join predicate."
What does it mean "both inputs must be sorted"?
--
Message posted via SQLMonster.com
http://www.sqlmonster.com/Uwe/Forums.aspx/sql-server/200603/1> What does it mean "both inputs must be sorted"?
That you on each table either have a clustered index on the columns over which you do the join, or a
non-clustered index that covers the data that is needed from the table.
--
Tibor Karaszi, SQL Server MVP
http://www.karaszi.com/sqlserver/default.asp
http://www.solidqualitylearning.com/
Blog: http://solidqualitylearning.com/blogs/tibor/
"cbrichards via SQLMonster.com" <u3288@.uwe> wrote in message news:5d9898960f213@.uwe...
>I am trying to understand mege joins. In BOL it reads "The merge join
> requires that both inputs be sorted on the merge columns, which are defined
> by the equality (WHERE) clauses of the join predicate."
> What does it mean "both inputs must be sorted"?
> --
> Message posted via SQLMonster.com
> http://www.sqlmonster.com/Uwe/Forums.aspx/sql-server/200603/1

Friday, March 23, 2012

merge 2 columns to one?!

Hello everybody!
I′m trying to put together two columns of integer into one. They are coming from the same source table, so the "merge"-Task doesn′t work. This is what they look like:
column1: <123>
column2: <0123456>
The result should be:
merged_column: <1230123456>
Is there a task for that kind of operation or do I have to write a script? I would like do do that within the data flow.

thanks in advance! you can use SQL to concatenate these columns

select
column1
, column2
, cast(column1 as nvarchar) + cast(column2 as nvarchar) as column3
from
yourtable

how you use this depends on what you are trying to do|||

So basically you want to concatenate 2 integer columns right? You can try with the derivate column transform for that. Notice that you might need to cast them as string before concatenating them

Rafael Salas

|||Thats quite simple - I just forgot that this is possible in SQL :D
Thanks for your help!sql

merge 2 columns to one?!

Hello everybody!
I′m trying to put together two columns of integer into one. They are coming from the same source table, so the "merge"-Task doesn′t work. This is what they look like:
column1: <123>
column2: <0123456>
The result should be:
merged_column: <1230123456>
Is there a task for that kind of operation or do I have to write a script? I would like do do that within the data flow.

thanks in advance! you can use SQL to concatenate these columns

select
column1
, column2
, cast(column1 as nvarchar) + cast(column2 as nvarchar) as column3
from
yourtable

how you use this depends on what you are trying to do|||

So basically you want to concatenate 2 integer columns right? You can try with the derivate column transform for that. Notice that you might need to cast them as string before concatenating them

Rafael Salas

|||Thats quite simple - I just forgot that this is possible in SQL :D
Thanks for your help!

Merge 2 columns from the same table

Hi *,

I have a table with 2 columns A1 and Firm. A1 holds "dirty data" and there are no NULL values. Firm holds "clean data" for some of the rows in A1 but not for all. So there are quite a few NULL values in this column. I want to replace the value in A1 if there is any data in Firm (value <> NULL).

Rigth now, I solve this issue with a simple VB.NET script:

Public Class ScriptMain
Inherits UserComponent

Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
'
If Row.Firm_IsNull Then
Row.Standard = Row.A1
Else
Row.Standard = Row.Firm

End If
'
End Sub

End Class

Any ideas how to solve this issue without custome code?

Thanks!

It sounds to me like you can use Derived Column transformation:

Repalce A1:

With an expression like

ISNULL(Firm) ? A1 : Firm

Rafael Salas