Duplicate Strings – How to Find in C#

algorithmsccode-reviews

I allow my user to create profiles, meaning they potentially could create some with duplicate names. This can be problematic so I want to prevent them from doing so. I coded up this first pass recursive-ish algorithm for detecting what number copy I should assign to the Name, however I'm almost certain there is a better way to write it, but my writers block has dropped the hammer down on me–in fact, I wasn't even sure how to word the question or Google something that might already exist. Here is the code:

//Check to see if there is another profile with the same name
var exiProf = profiles.FirstOrDefault(p => p.Name == newProf.Name);
int counter = 1;
while (exiProf != null)
{
    // Only match with the last number surrounded with parens
    Regex exp = new Regex(@"\((\d)\)$");
    Match ma = exp.Match(exiProf.Name);
    if(ma.Success)
    {
        num = int.Parse(ma.Groups[1].Value);
        var numChar = (counter).ToString();
        var nextNumChar = ( ++counter).ToString();
        newProf.Name = newProf.Name.Replace(numChar, nextNumChar);

        //Loop to make sure new name doesn't already exist
        exiProf = profiles.FirstOrDefault(p => p.Name == newProf.Name);
    }
    else
    {
        newProf.Name += " (" + num + ")";

        //Loop to make sure new name doesn't already exist
        exiProf = profiles.FirstOrDefault(p => p.Name == newProf.Name);
    }
}

profiles.Add(newProf);

Meaning, if I try to add "New", "New", "New", "New" in that order, "New (1)", "New (2)", "New (3)", "New (4)" is what shows up in my list.

Note: I have tagged algorithms in here because I am capable of turning pseudocode into C#.

Best Answer

I would think this will do it:

if (profiles.Any(p => p.Name == newProf.Name))
{
    string template = newProf.Name + " ({0})";
    int counter = 1;
    string candidate = String.Format(template, counter);

    while (profiles.Any(p => p.Name == candidate))
    {
        counter++;
        candidate = String.Format(template, counter);
    }

    newProf.Name = candidate;
}

Related Solutions

Algorithms – Detecting Duplicates in a Subset of Elements

Take a subset, add each element into a hash table. If the hash table already contains a value - print out saying there is a duplicate.

A hash table will only allocate 1 memory block per entry. Hence its easy to find if the number already exists. Since these are just regular entries your table will be something like this:

struct Hashtable{
   int number;
};

static struct Hashtable hashTable[10];

int getHash(int x){ return x; }

void addHash(int number)
{
    if(hashTable[number].number == -1)
    { /*OK*/
        hashTable[number].number = number;
    }else{
        printf("DUPLICATE FOUND! BAILING OUT!");
    }
}

void initTable(){ for(int i = 0; i < 10; i++){hashTable[i].number = -1;}}

How to best detect duplicate data in a large dataset

SQL solution

You could load all the demographic data into an SQL database:

CREATE TABLE PERSON(Id integer PRIMARY KEY, zip text, birth date, gender char /*... */);
...

Unfortunately the file importing statement is not SQL standard (e.g. BULK INSERT for SQLServer, LOAD DATA INFILE for mysql, or use SQL*Loader for Oracle).

The easiest and most efficient way would then be to use aggregate functions with a GROUP BY clause to count number of persons sharing the same values for the grouping columns, and keeping only those with duplicates, using a HAVING clause:

SELECT zip, birth, gender, count(*) FROM PERSON 
   GROUP BY zip, birth, gender
   HAVING count(*)>1;

Online demo

Sorted file solution

You could als get your census file sorted by zip, birth and gender. Then you could read the data, compare each record read to the previous one, and if the same, and count until these value change for a record.

Pseudocode:

lastrecord = {  };
counter = 1; 
while there's a record to read {
    read record 
    if (record.zip == lastrecord.zip 
          and record.birth==lastreacord.birth 
          and record.gender == lastrecord.gender) {
       counter = counter +1; 
    } 
    else {
         if (counter>1)  {    // output the count of duplicates
               write lastrecord.zip, lastrecord.birth, lastrecord.gender, counter
         }  
         counter =1; 
    }      
    lastrecord = record; 
}
if (counter>1)  {    // output the count of duplicates
     write lastrecord.zip, lastrecord.birth, lastrecord.gender, 
}

Associative map

A last way, here would be to read each record as it comes, and store the 3 tuple values in a map:

store 1 if the tuple was not yet loaded
increment existing tuple value if it already exists

In the end, iterate trough the map and process the elements having a count greater than 1. Ok, this one will cost you some memory ;-)

Best Answer

Related Solutions

Algorithms – Detecting Duplicates in a Subset of Elements

How to best detect duplicate data in a large dataset

Related Topic