Javascript – Get all non-unique values (i.e.: duplicate/more than one occurrence) in an array

arraysjavascript

I need to check a JavaScript array to see if there are any duplicate values. What's the easiest way to do this? I just need to find what the duplicated values are – I don't actually need their indexes or how many times they are duplicated.

I know I can loop through the array and check all the other values for a match, but it seems like there should be an easier way.

Similar question:

Get all unique values in a JavaScript array (remove duplicates)

Best Answer

You could sort the array and then run through it and then see if the next (or previous) index is the same as the current. Assuming your sort algorithm is good, this should be less than O(n²):

const findDuplicates = (arr) => {
  let sorted_arr = arr.slice().sort(); // You can define the comparing function here. 
  // JS by default uses a crappy string compare.
  // (we use slice to clone the array so the
  // original array won't be modified)
  let results = [];
  for (let i = 0; i < sorted_arr.length - 1; i++) {
    if (sorted_arr[i + 1] == sorted_arr[i]) {
      results.push(sorted_arr[i]);
    }
  }
  return results;
}

let duplicatedArray = [9, 9, 111, 2, 3, 4, 4, 5, 7];
console.log(`The duplicates in ${duplicatedArray} are ${findDuplicates(duplicatedArray)}`);

In case, if you are to return as a function for duplicates. This is for similar type of case.

Reference: https://stackoverflow.com/a/57532964/8119511

Simple ways:

var arr = [1,2,,3,,-3,null,,0,,undefined,4,,4,,5,,6,,,,];


arr.filter(n => n)
// [1, 2, 3, -3, 4, 4, 5, 6]

arr.filter(Number) 
// [1, 2, 3, -3, 4, 4, 5, 6]

arr.filter(Boolean) 
// [1, 2, 3, -3, 4, 4, 5, 6]

or - (only for single array items of type "text")

['','1','2',3,,'4',,undefined,,,'5'].join('').split(''); 
// output:  ["1","2","3","4","5"]

or - Classic way: simple iteration

var arr = [1,2,null, undefined,3,,3,,,0,,,[],,{},,5,,6,,,,],
    len = arr.length, i;

for(i = 0; i < len; i++ )
    arr[i] && arr.push(arr[i]);  // copy non-empty values to the end of the array

arr.splice(0 , len);  // cut the array and leave only the non-empty values

arr // [1,2,3,3,[],Object{},5,6]

via jQuery:

var arr = [1,2,,3,,3,,,0,,,4,,4,,5,,6,,,,];

arr = $.grep(arr,function(n){ return n == 0 || n });

arr // [1, 2, 3, 3, 0, 4, 4, 5, 6]

UPDATE - just another fast, cool way (using ES6):

var arr = [1,2,null, undefined,3,,3,,,0,,,4,,4,,5,,6,,,,], 
    temp = [];

for(let i of arr)
    i && temp.push(i); // copy each non-empty value to the 'temp' array

arr = temp;

arr // [1, 2, 3, 3, 4, 4, 5, 6]

Remove empty values

['foo', '',,,'',,null, ' ', 3, true, [], [1], {}, undefined, ()=>{}].filter(String)

// ["foo", null, " ", 3, true, [1], Object {}, undefined, ()=>{}]

Javascript – Get all unique values in a JavaScript array (remove duplicates)

With JavaScript 1.6 / ECMAScript 5 you can use the native filter method of an Array in the following way to get an array with unique values:

function onlyUnique(value, index, self) {
  return self.indexOf(value) === index;
}

// usage example:
var a = ['a', 1, 'a', 2, '1'];
var unique = a.filter(onlyUnique);

console.log(unique); // ['a', 1, 2, '1']

The native method filter will loop through the array and leave only those entries that pass the given callback function onlyUnique.

onlyUnique checks, if the given value is the first occurring. If not, it must be a duplicate and will not be copied.

This solution works without any extra library like jQuery or prototype.js.

It works for arrays with mixed value types too.

For old Browsers (<ie9), that do not support the native methods filter and indexOf you can find work arounds in the MDN documentation for filter and indexOf.

If you want to keep the last occurrence of a value, simple replace indexOf by lastIndexOf.

With ES6 it could be shorten to this:

// usage example:
var myArray = ['a', 1, 'a', 2, '1'];
var unique = myArray.filter((v, i, a) => a.indexOf(v) === i);

console.log(unique); // unique is ['a', 1, 2, '1']

Thanks to Camilo Martin for hint in comment.

ES6 has a native object Set to store unique values. To get an array with unique values you could now do this:

var myArray = ['a', 1, 'a', 2, '1'];

let unique = [...new Set(myArray)];

console.log(unique); // unique is ['a', 1, 2, '1']

The constructor of Set takes an iterable object, like Array, and the spread operator ... transform the set back into an Array. Thanks to Lukas Liese for hint in comment.

multi array

var a = [[2,4],[2,3],[1,3],[1,3],[1,8]]

var unique = (value, index, self) =>{
  var findIndex = (element) => element[0] == value[0];
  return self.findIndex(findIndex) === index;
}

console.log(a.filter(unique))