In these yet-to-be-determined-frequency installments, I’ll look at examples of Org troubles that either I’ve experienced or stories from the community. For our first installment we’ll look at an Org that has gone over their allotted data storage limit.
Storage Overview
There are a few different types of storage limits. The first is File Storage, this includes the storage for files, documents, and attachments. The other is data storage which is contains all your actual Salesforce data. Every record in your org represents approximately 2kb of data storage.
Salesforce provides every org a certain about of data per Salesforce License (Not including Chatter Free). This is your total amount storage for your org. You can see how you’re using your storage from Setup -> Data Management -> Storage Usage (Or Use the quick find). It’ll provide you with your limit, the amount used, and the percent used. It’ll also show you how your Storage Usage is being used across your objects.
You can go over your limit. Salesforce frowns on this. In my experience, Salesforce.com will work with you and cut you some slack on being over, but if you cannot address the issue they will suggest you purchase additional storage space. This can get pricey quickly and may be just a short term fix depending upon the nature of your storage problem.
The Problem
In my org we track doctors. We also track the various addresses where they practice, their specialties, and some other salient information. We have an external database that has all this information. We were pushing the data into our org – effectively duplicating the database – every night.
This resulted in millions of doctor related records. What makes this interesting is we use Person Accounts. A Person Account is both a Contact and an Account. So 1 person account counts TWICE in your storage limit once for an Account, once for a Contact. 2 million Person Accounts multiplied by 4kb (2kb for the Account, 2kb for the Contact) and you have a nice amount of storage.
Then add all the related data against those person accounts. You add millions of more records. The data usage compounds.
It wasn’t long until we were using 114% of our data storage. The great majority all comings from our Person Accounts, Addresses, and Specialties. Here’s a snapshot of what my storage usage looked like.
Analyzing the Problem
It’s quite obvious that the biggest data usage was coming from our physician related data. So what were our options:
- Delete some of our Data
- Pay for additional storage
- Increase our Edition from Enterprise to Unlimited
Options two and three cost money. It can be also be quite costly and potentially only temporary depending the root cause of the problem. So we started by looking at our doctor data.
The primary reason that data was in Salesforce was for support. We needed to be able to attached doctors to cases when they contact support with a problem. After waiting for reports to come back (Running reports across 2 million records takes time), it turns out we only had cases on 22,000 or so physicians. Less than 1% of the actual data causing our storage problems were being used!
I also discovered that some data was being duplicated between the external database. Some providers had too many records for addresses. We didn’t need all of those! More data that wasn’t needed.
The Solution
Obviously we had more data than we needed. We could delete all the records that we didn’t need, but how will support create the physician when a case was submitted? How would we get the information on those doctors we did need once the doctor was created.
The solution we choose was to create a webservice between Salesforce and the external database. This was a visualforce page were the customer support rep could search for the Doctor. The page would search both Salesforce and the external database. If the doctor already existed – the page only showed the Salesforce result. Otherwise, it showed a result from the external database. The Customer Service rep can then select the record and the page will create the physician and attach the case to the new record.
This allowed our customer support folks add physicians as needed. We also changed our nightly update. The update only updated records that existed in Salesforce. For the related data we completely delete and re-add to prevent un-needed duplicates.
All that remained was to delete all the extra data. I’m very thankful to have CRM Fusions’ DemandTools. I was able to create a field and use Demandtools to do an ad-hoc roll-up summary formula behavior. I can then query any Person Account with 0 cases and delete it with all it’s related data. We went from over 2 million Person Accounts to under 100,000. That’s a hue data change!
Finally, we’re below our storage limits and we’re able to still keep our important business processes.
Leave a Reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
This was a great learning experience shown to me by Brian. Being able to go through this effort with him, not only showed me how important it was to troubleshoot the issue, but how to correct it with a scaleable solution that will prevent it from happening in the future. Hats off to you Brian for your continued knowledge and training!
As admins we all know how data issues can create havoc in our Salesforce environment!