Alternative Deduper

cropped-apex-logo-copy1.pngThis deduper is a slightly more advanced version of the previous one I posted. There are big differences in how this one works. The lead SOQL is sorted by street which is what I am matching on. This makes it easy to look for duplicates. The first instance of a street is a unique record and each subsequent matching record is a duplicate. Boolean logic is used to store the first instance of a unique record which changes when a non-matching record is found.

The visual force page makes use of the panelGrid tag which allows the duplicate and unique tables to be placed side by side. I was unable to get the nested table code to work. Each row of the table is a duplicate and for each duplicate there is a unique record which is displayed in the second table. Where there are multiple duplicate records the unique records will always be the same. You can click the ID links to verify the leads.



Create the TestController as follows:

public class TestController {

public List<Lead> leads { get; set; }
public List<Lead> firstLead { get; set; }
public List<Lead> duplicateLeads { get; set; }
public List<Lead> uniqueLeads { get; set; }

public string str;
public string temp;
public string firstInstance;
public integer i;
public boolean flgFirstInstance = false;

public TestController() {

leads = new List<Lead>();
firstLead = new List<Lead>();
duplicateLeads = new List<Lead>();
uniqueLeads = new List<Lead>();

//Order leads by Company name

leads = [SELECT Company FROM Lead WHERE Company != null ORDER BY Company ASC limit 30001];

//Loop to go through leads list

for (lead singleLead : leads) {

//Extract Company name

string str = JSON.serialize(singleLead); //convert the first list element to a string

List<String> split = str.split(‘,’); //split the string using comma as a delimiter

i = split[2].Length();  // this gets the length of the string and reduces it by 1 eliminating the last quote


split[2] = split[2].mid(11,i-11); //starts the string from the first character after the closing quote’

If (flgFirstInstance == true) {

If (firstInstance == split[2]){ //found duplicate?

duplicateLeads.add (singleLead);

uniqueLeads.add (firstLead[0]);


flgFirstInstance = false;



If (flgFirstInstance == false) {

flgFirstInstance = true;

firstInstance = split[2];

firstLead.clear() ;

firstLead.add (singleLead);



system.debug(‘Total count: ‘ + leads.size());
system.debug(‘Duplicate count: ‘ + duplicateLeads.size());
system.debug(‘Unique count: ‘ + (leads.size() – duplicateLeads.size()));


public List<Lead> duplicateLeads() {return duplicateLeads;}

public List<Lead> uniqueLeads() {return uniqueLeads;}

public Integer getTotalLead() {

return Leads.size();

public Integer getTotalDup() {

return duplicateLeads.size();

public Integer getTotalUnq() {

return (leads.size() – duplicateLeads.size());



Create the Visual Force page Dups as follows:

<apex:page controller=”TestController” tabStyle=”Lead” readOnly=”true”  >

<apex:variable var=”sr” value=”{!0}”/>

<P>Total Leads: {!totalLead} &nbsp;&nbsp; Unique Records: {!totalUnq} &nbsp;&nbsp;  Duplicates: {!totalDup}  </P>

<apex:panelGrid columns=”2″>

<apex:pageblock title=”Duplicate Records” id=”blockA”>

<apex:pageblocktable value=”{!duplicateLeads}” var=”dupLead”>

<apex:column headervalue=”Number”>

<apex:variable var=”sr” value=”{!sr + 1}”/>


<apex:column headervalue=”Duplicate ID”>

<apex:outputLink value=”/{!dupLead.Id}”>{!dupLead.Id}</apex:outputLink>


<apex:column headervalue=”Duplicate Company”>

<apex:outputField value=”{!dupLead.Company}”></apex:outputField>




<apex:pageblock title=”Unique Records” id=”blockB”>

<apex:pageblocktable value=”{!uniqueLeads}” var=”uniLead”>

<apex:column headervalue=”Unique ID”>

<apex:outputLink value=”/{!uniLead.Id}”>{!uniLead.Id}</apex:outputLink>


<apex:column headervalue=”Unique Company”>

<apex:outputField value=”{!uniLead.Company}”></apex:outputField>











One thought on “Alternative Deduper

  1. Pingback: Advanced Deduper — | SutoCom Solutions

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s