It’s Spring! Spring is a sign of renewal, growth, and change, and time to clean up data! In our homes we tend to do some sprucing up of the inside and outside. Why don’t we do the same with our business?
It’s the perfect time to clean up your data to prepare for growth in the spring season, for the pace of the busy summer season, and for new strategies in the fall season.
If we can set aside time to strategically review data every year and prepare for future development cycles, we may save enough money to impact the bottom line and open budgets that were once closed.
What Data Do You Archive?
When you prepare to clean up date, first decide what data do you keep in your vault for long-term storage?
The first thing you need to do is review your entire data structure. What exactly are you keeping on your company computers, individual computers, databases, file shares, and document storage facilities.
Then look at how they’re accessed. Who accesses them and how often?
Regulations and service level agreements could determine some of the decisions as to what data you should keep in long-term storage.
After that has been determined, you might want to look at your own company’s users for answers. I bet your sales and marketing team (or researchers) are going to want access to more data than your accountants or inventory control managers.
This is where you have to look through the weeds. How much of the data is valuable as time passes?
For example, will real-time aspects of who recorded the purchase of 20 widgets and how the sales were processed be useful later? Or is just the type, color, version of the widget, who sold and purchased it, and when the transactions took place the most important aspects?
If so, you can strip out the unnecessary data and only keep the data that’s vital to statistical research you may need for insight into your market.
It’s not always necessary to put all data and databases into a vault. Analyzing what should go into the vault can save you in hardware and maintenance costs.
There’s a lot of metadata (data about the data) shared and stored about the hard data you’ll want to store. You may want to review that metadata to see if it’s necessary to keep in your data warehouse or long term vault.
If you have data about who bought what product from you on a certain date, that all seems to be good data to store.
But do you need to know who keyed in the data, what IP address they used, and other user-related code?
This may be something you want to investigate and search later, but it also could basically be garbage data that can be kept from the warehouse.
How Long Do You Need to Keep Old Data?
In addition to the regulations you must adhere to, you should take a few moments to discuss with your team on what data you need to keep and what data (even metadata) you can get rid of after a period of time. You can't just jump in a clean up data, its imperative to pay attention to records retention laws.
Some data is necessary to keep in order to collect historical points, if only to compare your business’ performance decade over decade.
Other data, like who entered an order for 4,000 widgets and when, may not be important so you may be able to cleanse some of the related metadata but keep specific data to run reports year over year, saving time and space.
Sit down with your users, CFO, CIO, and compliance officer to discuss what data you need to save and create a strategy to purge old unused data.
Create and carry out a strategy with these team members, with rules. If you’re not sure, back up or archive some of the data in a way it can be brought back into the database.
Often we have clients who hire us because they just don’t have time for all the grunt work. Sometimes the database and applications have been stored for so long no one is sure what’s valuable and what isn’t.
Once we’ve been hired, we’ve been able to come through with monitoring and monthly analysis that helps determine living active data from old inactive data.
We’ve then worked with clients to map unusable to usable data. We could remove or archive old inactive environments off production environments, saving our clients millions in maintenance and storage costs.
The work was tedious and took a team effort, but in the end it was a profitable process for our clients.
To summarize, it’s important to review what you want to keep for long-term storage. You don’t want to spend time and resources managing this data for decades.
On the other hand, you don’t want to throw anything away that could be of value later. You may want to ask impossible questions to see what you get from your data. There may be surprising results as to how you can realign the information and find a market for it or create algorithms for insights about clients, products, and markets you never imagined you could get from data.
We suggest once a year you should review the database and entire data environment schema and create a wish list to see what you might like to ask your systems. The answers may surprise you.
Also, there are tools that can be brought to up to speed with the investigation phase of this exercise.
After talking with users, the CIO, CFO, and regulatory members, present your project with rules and suggestions to your IT staff for a plan to start cleaning out data.
At this time, you may want to ask the IT staff to create a schema for each physical server on the databases, applications, and software is housed there.
There may be old third-party software on servers you have never used, or no longer use, stored on your devices. This may be a good time to weed out these items and gain needed space. Also, you may have some in-house applications you built a decade ago that has been replaced with new systems. You can identify and either remove them from production environments and store in cheaper devices or delete altogether.
The reason I bring all this up is to help you consider approaching data from this perspective. How much of the data is your company emotionally tied too? What data can you live without? We have a couple of large retail clients. They have old databases and data they aren’t sure which, if any, applications touch them. They aren’t sure if the data is used. But they hold onto these databases like each one has value. (Tip: If you’re unsure whether people are accessing a database, try cautiously disconnecting it for a while.)
Part of the spring cleaning plan should be to review a couple of these databases for a year or a quarter. There are several tips we have to do this.
Look to see if there are new rows of data. Look at the database metadata to see when the last time was that a user accessed the data. Then determine if the database should be kept or if you can at least archive the databases onto lower cost storage instead of keeping them on active devices.
At this point, look at the used applications, databases, and software to make sure all are pointing to correct devices, databases, and software versions. You may discover unused databases at this point and then decide to archive those databases, delete them, or keep them on your production servers.
If your company is a legacy company or if you have bought and sold parts of your business, this may be a challenge.
Schedule a regular cleanout session quarterly or twice a year to help manage the workload without interfering with your day-to-day business development needs.
There are organizations with software that can help you map out what’s being stored on your hardware and provide you with reports to help clean up data.
Cost Savings
When we’ve reviewed environments after being brought in to design and develop maintenance strategies, we’ve found some of the databases empty and unused. This can help cut down on software licensing costs from database server licenses to all types of management and security licensing that’s used to manage old or empty databases.
Once you map databases, data and applications, and you know those you use and those you don’t, you can decide which are needed to keep in production environments, which you want to archive, and which you want to remove.
You will not only have more space on your servers and storage devices, but you’ll also be able to remove these databases from your backups, and from maintenance and audit processes.
This will save money on monitoring software for your backups. Remember backups can be staged so you may get a lot of space, software, and personnel time freed up by just not having to manage the backups.
If you have auditing requirements, you’ll save on possible software, personnel hours, and reporting time for auditing management of unused data.
Add maintenance management, such as integrity and job failure checks, space and performance analysis, and you saved personnel time.
Of course, each time you clean up your environment, your savings continue to grow and the cleanup and discovery processes become a lighter load.
The trick is to plan and begin this process. Just like spring cleaning at home, you eventually get through it and life is more organized.
By looking at your IT budget you can do the math to determine whether you should start to clean up data. If your hardware costs are $2 million, and, let’s say, technical personnel costs are $500,000, software is $1 million and once you clean up environments you release 10 percent of the workload, then your savings could be 10 percent of each cost structure (between hardware, software and personnel time costs) saving $200,000-$300,000 for your overall budget.
This can affect your hiring and growth of your business.
If you’re unsure whether this is worth the effort, try reviewing one physical server. Don’t do the work, just do the analysis and review what data, applications, and software are being stored, and determine what’s unused.
Contact Soaring Eagle Consulting for Consultation
Fill out a form or give us a call and we will do a free 30-minute database evaluation to identify the root causes of your issues and decide the best way to resolve your problems.
If you see value in the review exercise, start planning to do the cleanup. If possible, schedule it during a slow business season.
There are software and service providers who will do this investigation for you if you don’t have the staff to manage it. You may want to contract out the service.
Contact us today to help clean up your data.
