March 7, 2018

Here at RMMI we take a lot of pride in the level of service we provide, and we’re constantly looking to our clients to tell us how we can improve.  One theme that seemed to be reoccurring across many of our clients was the need for a solution that was easy to implement and offered top-level support, reliability, and security without adding management headaches for already overloaded IT departments.  Many of our clients don’t have dedicated IT staff, or share technicians with other departments.  Finding a way to deliver our scanning and document management solutions without adding any load to internal or contracted IT personnel moved to the top of our list.  To accomplish this goal, early in 2016 we began the steps to implement a hosted PaperVision Enterprise and Capture Scan solution.  After many months of planning, touring multiple colocation datacenters, purchasing and configuring hardware, and what seemed like endless testing we went live with our first client in the fall of 2016.

We knew we had several clients who would benefit from the hosted offerings, but we honestly had no idea just how popular the solution would become.  We are now hosting over 80 million digitized documents for more than 40 different clients and there are more than 500 end-users who access our systems each week.

In order to keep up with the increased demand, we’ve applied our continuous-improvement model to our hosted solution and have been constantly working on making the system better since our initial roll-out.  We’re certainly not done improving, but we wanted to highlight some of the major changes and how they impact you and your team.

The foundation of any electronic document management system is storage.  Performance and ease of access mean nothing if your data isn’t protected from deletion (intentional or accidental), loss due to hardware failure, or corruption from malicious attacks.  To ensure that we’re meeting these goals, we have implemented a multi-tiered storage and backup system that uses several techniques to make sure your data is secure:

• RMMI’s hosted solution is housed within the top datacenter in Colorado and the only Colocation Datacenter in the world to carry a Tier 3 Gold certification. The datacenter recently celebrated 15 years of uptime without a single outage to any of their major systems (Power, Cooling, Fire-Protection, and Service Connection).  Building our hosted solution on top of this excellent foundation allows us to provide the highest level of service.
• At the base level, all of your data rests on high-quality storage arrays that utilize a technology called erasure-coding to spread your data across multiple physical hard-drives – these storage arrays are designed to remain fully functional even in the event that 33% of the disks in the array fail simultaneously and allow our staff to replace potentially failing drives without disrupting service.
• We have a sophisticated monitoring system in place that checks over 8,000 different ‘trigger points’ every minute – this system alerts our technical staff to signs of disk failure far before that disk becomes unusable. The reality is hard-drives fail at an industry average rate of 2.5 – 3% per year and our system utilizes more than 100 drives so a couple drive replacements per year is inevitable.  Our monitoring system has given us the tools to make sure that we’ve caught every pending drive-failure in the ‘pre-fail’ warning stage and replaced them before they’ve gone down.
• To prevent against data deletion or corruption we use another technology called snapshotting. Snapshotting allow us to take a virtual ‘snapshot’ of your data at a certain place in time.  In the event a document is inadvertently removed or altered in an unwanted way, we can ‘roll-back’ to a previous version of that file.  We take snapshots of all of our clients’ PaperVision data every 4 hours and keep those snapshots for 90 days.  We can roll back an entire PaperVision entity or a specific file to any point in the last 3 months.
• Another added benefit of utilizing snapshots is it allows to do what’s called ‘snapshot-mirroring’. This means that every time we take a snapshot (every 4 hours) we replicate that data to a second storage array.  These backup arrays carry the exact same specifications as the primary arrays and can completely replace them in the event we suffer a hardware failure that is deemed ‘non-recoverable’.  All 90 days’ worth of snapshot data is contained on both the primary and secondary arrays.
• To prevent against a total system loss due to fire or natural disaster, we send nightly snapshots to a second datacenter in Palo-Alto California. While this site isn’t equipped to provide hosting services, it means that even in the event of a catastrophic local disaster, your data is secure and we’ll be able to provide a full restore once the local infrastructure is back up and running.
• Our off-site backup system also allows us to write a backup copy of your data to your own contracted storage provider (Google Cloud, Microsoft Azure, Amazon S3, or Backblaze B2). These complete backups are ‘owned’ by you and you can elect to keep as many versions as you would like.  If you would like a quote on this service please contact your sales representative.

After reliable storage, uptime and accessibility are our next goals.  RMMI’s entire hosted solution is set up for maximum reliability with no single point of failure.

• All of the components of the system from network switches to web-servers are configured in groups. Single failures in any system trigger an automated re-balancing spreading the workload over the remaining operational devices.  This redundant configuration has the added benefit of allowing us to perform most maintenance tasks without taking the system offline – meaning we rarely have to file for any ‘maintenance windows’.
• The system is connected to the rest of the internet by a ‘blended-mesh’ of 4 different internet service providers. As long as at least one of these service providers is operational the system is accessible.
• All services and components are actively monitored by our IT staff. Most issues are quickly assessed and resolved before they can cause any noticeable impact to our end-users.

Security is the final part of our configuration.  When you trust us with your documents, we want you to rest soundly knowing that not only is your data safe from loss, it’s also safe from unauthorized access.  To ensure that all of our client’s security concerns are met, we have utilized the following physical, procedural, and electronic security measures:

• The equipment for the hosted PaperVision system is located at a highly-secure datacenter. Physical access to the system is tightly controlled by two layers of key-card and biometric security, and the server cabinets themselves are protected by combination locks.  The entire facility is patrolled 24×7 and is under constant video surveillance.
• Electronic access to the system is limited to RMMI IT administrators only – all IT administrators have completed extensive background checks and have been trained in applicable security practices.
• The entire hosted system resides behind multiple firewall layers and potential cyber-threats and unauthorized access attempts are actively monitored and tracked.
• Each PaperVision entity we host is separated from our other clients by utilizing its own Sql Database and segregated storage shares. Backend data is protected using encryption keys unique to each entity.  This ensure that absolutely no data can be inadvertently shared across entities.
• Any and all requests for new user accounts or new access permissions are verified with account administrators prior to completion.

In addition to the changes to the main PaperVision infrastructure, we’ve also been working hard to improve our support system.  Live chat support is now embedded directly into the PaperVision system, and we  have implemented a complete ticketing and tracking system to make sure that all of your issues are resolved quickly.  If you haven’t already had a chance to read my previous post about the changes we made to our support terms and the updated SLA please click here.

We know that there you have a lot of options for document management in today’s market, and we’re truly grateful for the opportunity to serve you.  If you have any ideas on how we can improve our products please don’t hesitate to drop us a line at support@rmm-i.com.  As always, if you have any questions about any of the changes we’ve made and how they directly affect you, please contact you sales representative.

Rick Harper – Director of Technology