Skip to content
Advertisement

Hide duplicate column cells in view

First of all, this here doesn’t solve my problem at all. I tried this too.

I want generate random unique fake data (first name and address)

I used the following SQL request:

The result apparently looked like this (the addresses repeat themselves for the next name too): what is wrong

How do I make the first names AND addresses unique?

Please note this is in CSV format:

Expected result:

Sample Data:

Advertisement

Answer

A CROSS JOIN is the wrong approach to begin with. Multiple FROM items separated by comma (,) are cross-joined. See:

10 names in anon.first_name and 10 addresses in anon.address form a Cartesian product of 100 rows, with each name and address multiplied by the cardinality of the cross-joined table. Exactly what you don’t want. All the confusion with duplicates was created by that. Applying DISTINCT after the cross join is hugely inefficient.

To use each name and address only once, attach a random (or arbitrary) number to each side and join on that.

Assuming (for lack of declaration) that each source table has distinct entries. (Else you need subqueries with DISTINCT to collapse dupes first – window functions like row_number() are applied before DISTINCT on the same query level.)

Combinations are arbitrary, rather than random. See:

The result has the cardinality of the smaller table. Excess rows from the bigger table are skipped.

To allow some duplicates, you just introduce some duplicate numbers (arbitrary_nr) on one (or both) side(s), either with the same row multiple times, or different rows with the same arbitrary number. For your example:

There could be like 2 persons living at the same address sometimes

This adds ~ 10 % of all names a second time. (So some names get two addresses.) This time, names are ordered alphabetically. The random sample starts with 0 (- 1) and numbers can can only increase slower, so the same name can never get the same arbitrary_nr, and the same address is never combined with the same name twice.

The result is still arbitrary (or random) as long as at least one side gets arbitrary (or random) numbers.

There are many ways. Much depends on your exact input, and exact requirements for the result.

User contributions licensed under: CC BY-SA
9 People found this is helpful
Advertisement