Many companies have a legacy of microfiche that encompasses millions of images stored in temperature controlled records management departments. Companies use the microfiche for records management purposes so they can comply with regulatory bodies. The process of creating microfiche is cumbersome, costly and in some cases a health risk due to the photographic equipment and chemicals – not to mention the real-estate required for such a service. This blog will cover the process of creating and retrieving records using Microfiche and an Electronic Document and Records Management (EDRM) system so that you will understand the complexities and benefits of each and be able to articulate best practices for EDRM projects. Also, I’ve worked on several EDRM projects over the years and this article summarizes the most interesting of the them.

Microfiche and Diazo Overview

Microfiche is a widely adopted standard for records management due to its ability to image record[1] and store them in an economical manner. Simply, Microfiche as it implies is a micro version of the original record in paper version (e.g, 8.5×11 letter). As a result of using Microfiche, several benefits are realized:

  • The records life is increased since paper becomes yellow and fades
  • The size of the record is reduced and therefore the real-estate required for storage is greatly reduced
  • The Microfiche is catalogued in such a way that it’s easy to find.
  • The Microfiche viewers allow for quick location of the records

Once Microfiche is created, it’s placed in a jacket that contains the microfiche’d record. The following diagram depicts a sample Jacket:

diazo

To further classify Microfiche, they are placed in jackets and then in a filing system such as a carousel[2]. In cases where records must be referenced, copies of the microfiche are required since actual records must remain in the records management office for safe keeping. When copies are required, a Diazo[3] is created as a copy of the record for temporary use.

Creating and Retrieving Records using Microfiche and Diazo

Today, the Corporation conducts filming of records using filming equipment and facilities in-house. Filming requires the use of photographic equipment and chemicals (e.g., ammonia for Diazo) that require special handling and storage to meet health safety requirements.

filmingprocess

The following is a brief description of the process followed by the Records Management Department (RM) when filming[4] and filing a record (Microfiche) as depicted in the Filming and Filing Process diagram:

  • A record is identified and must be filmed.
  • The record is sent to RM for filming and filing.
  • RM obtains access to filming equipment.
  • The record is filmed with the photographic equipment using procedures to ensure quality.
  • The film is processed using procedures for checking quality.
  • The filmed record (Microfilm) is cut.
  • The pocket is retrieved from the file cabinet based on the corporation’s classification scheme.
  • The jacket is opened and the new film record is added.
  • The jacket and pocket are placed back in the carousel.

The following is a brief description of the process followed by the various Business Units when requesting a Record (Microfiche):

  • A Corporation staff member has a need to view a record.
  • A request is placed with the RM with appropriate identification information (example, Last name, account #).
  • The request form is physically delivered to the records office and placed in a bin.
  • RM obtains access to the file cabinet.
  • An RM staff member retrieves[5] the record by walking to the appropriate Microfiche library.
  • The appropriate pocket is removed from the library.
  • The pocket is browsed by identification information.
  • The Microfiche is copied using Diazo and returned to the file cabinet.
  • The Diazo is given to staff member at the counter or delivered at the next microfiche run to the designated drop off areas.
  • If an amendment or add (example, legal correspondence letter) is required, the form is sent to RM, filmed and added to the jacket.
  • Once finished with the Diazo it is destroyed.

Electronic Records Management Overview

EDRM provides a technological solution to the problems associated with legacy Microfiche systems. Specifically, it provides a more cost effective means of creating, classifying, storing, retrieval and disposition of records. The specific benefits are as follows:

  • Creating – records are created using scanners and saved in popular formats such as PDF, JPG or TIFF; which is much simpler and doesn’t require nasty chemicals and permits.
  • Classifying – classification schemes are used based on lingo specific to business units and business rules that enforce the scheme to greatly reduce errors.
  • Storing – images are stored on Storage Area Networks (SANs) which consume less real-estate.
  • Retrieval – the EDRM client provides the user with an interface that enables them to query the EDRM based on a combination of classification criteria (example, Last name, account #).
  • Disposition of records – the EDRM contains a business rule system that disposes of records based on predetermined business/legal rules (example, emails are 1 year, pension records are death of recipient plus 10 years)

An EDRM solution consists of several core systems such as scanners, image processing software, EDRM software for classification, storage, retrieval and disposition and, storage.

Sample ERM Solution

The following are the specific components:

  • Scanners – image capture devices available in standard flatbed variations or specialized for Microfiche.
  • Kofax Image Scanning and QA Software – control image format and quality.
  • Meridio Records and Document Management software – the ERM system that provides storage and retrieval.
  • Storage Area Network – storage for images and index.
  • SharePoint Portal – optional Portal for collaboration on documents.

The following diagram depicts the image production lifecycle stages:

sampleedrmprocess

The following diagram depicts a typical EDRM system:

sampleedrmsystem

Sample ERM Interface

The EDRM interface provides the functionality for standard users and administrative functions. Most EDRM provide a Web and Windows interfaces for user and administrative functions. The following diagram is the EDRM interface for a web client:

sampleedrminterface

Sample FilePlan

The FilePlan consists of a hierarchical means of classifying records based on containers, folders and parts. Classification schemes consist most of primary, secondary and tertiary levels of classification. Primaries typically consist of department names (example, procurement), secondaries are breakdowns of the departments information (example, purchasing agreements, guidelines) and tertiaries which are further breakdowns of information (example, year – 2005).

The following are some best practices for designing a FilePlan:

  • Adopt a standard classification scheme that meets the requirements of the Corporation – investigate state/provincial records authorities since they have typically developed standard schemes.
  • Adopt retention and disposal schedule that complies with regulatory body requirements – investigate state/provincial records authorities since they have typically developed standard schemes.
  • Don’t place more than 100 items in same location (e.g. Class or Folder) within the FilePlan since this slows browsing and paging.
  • Assign Access Control Lists and disposal schedules as close to the root of the FilePlan as possible, these will be inherited by child nodes – it is better that exceptions are treated by overriding the inherited disposal schedule. The use of default access on categories to assist in defining security model.

The following is a sample of a simple FilePlan:

Function
1. Human Resources Managing the employees of the department, through all activities and tasks associated with completing their job duties
Activities Classifications Additional Details
1.1 Recruitment:Ensure adequate pools and select the best candidates for open positions 1.1.1   Advertisement[Scope note if necessary] Samples: Job Description, Selection Criteria, AdvertisementSeries: Job Description FilesRetention: 1 year then destroy
1.1.2 Receive Applications[Scope note if necessary] Samples: Application Forms, Cover LettersSeries:Retention:
1.1.3 Conduct Interviews[Scope note if necessary] Samples: Correspondence, Interview NotesSeries:Retention:
1.2 TrainingProvide opportunities for growth and staff development 1.2.1 Attendance[Scope note if necessary] Samples:Series:Retention:
1.3 Review and PromotionConduct annual reviews and use those reviews to retain and promote the best employees 1.3.1 Annual Review Process[Scope note if necessary] Samples:Series:Retention:

 

Sample Retention Schedules

Retention schedules are required to lock record retention for a duration specified by regulatory (example, SEC) and legal bodies for compliance purposes. This is typically a function of the ERM which has configurable retention schedules that can be applied to the FilePlan by Records Officers.

The following table depicts a small sampling of a retention (disposition) schedule:

Record Retention Schedule Final Disposition Off Site Storage Facility
Electronic Member Information Dead, Dead + 10 Years Archive Archive
Original Microfiche I Year after scanning Archive Offsite records office
Original Microfiche in ARC Store Dead, Dead + 10 Years Archive Archive
Email 1 year Destroy None
Project Documents 5 years Destroy None

Sample ERM Project Team

An ERM project team consists of a diverse set of skill sets that represent the business units and IT. The following table depicts the team member and their implied skill sets:

Company Member Notes
Client Project Manager
CIO Steering committee
CEO Steering committee
IT Manager
Knowledge Management Officer
Lawyer
IT Developer
Business Unit Representative One for each business unit, region etc.
Senior Records Management Officer
Records Manager
IT infrastructure
Services Firm Project Manager
Regional Manager Steering committee
Local Manager Steering committee
Solution Architect
Developer
Imaging Tech
Storage Tech
Server Tech
Third Party ERM Tech Advice and support
Imaging Advice and support

Lessons Learned

The following are some key lessons learned from working in the space the past 10 years:

  • Organization must be ready for change
  • Executive management must establish communications and measurement process
  • Legal counsel must be available to advise on FOIP and regulatory compliance
  • The client must have a dedicated team to
    • Deal with additional business process work load
    • Allow staff to participate in project effectively
    • Facilitate adoption of new role(s)
  • The team roles must include records mgmt, legal, knowledge mgmt, communications, project mgmt and technology staff
  • Migration of existing unstructured data requires an approach that distributes responsibilities to business units, provides a structured process and staff training and measures progress regularly.
  • Don’t under estimate organization push back and resistance to change
  • Deploy a pilot for the purposes of testing systems, staff support services and business process changes
  • Microfiche image quality requires business decisions regarding the amount of cleanup vs. liability for damaged and/or aging records
  • Conduct image scanning tests from various years (i.e. 1970s, 80s, 90s) to determine required quality – test compression versus usability
  • Don’t undersize the technology platform since future retrofitting introduces risk
  • Leverage provincial record and information mgmt guidelines to save time (ARDA)
  • Plan to run parallel business processes and staff accordingly to handle increased work load
  • Couple training close to deployment

Best Practices for an ERM project

The following are best practices for EDRM and document management projects:

  • Organization must be ready for change.
  • Executive management must establish communications and measurement process.
  • Legal counsel must be available to advise on FOIP and regulatory compliance.
  • The client must have a dedicated team to:
    • Deal with additional business process work load
    • Allow staff to participate in project effectively
    • Facilitate adoption of new role(s)
  • The team roles must include records management, legal, knowledge management, communications, project management and technology staff.
  • Migration of existing unstructured data requires an approach that distributes responsibilities to business units, provides a structured process and staff training and measures progress regularly.
  • Don’t under estimate organization push back and resistance to change.
  • Deploy a pilot for the purposes of testing systems, staff support services and business process changes.
  • Microfiche image quality requires business decisions regarding the amount of cleanup vs. liability for damaged and/or aging records.
  • Conduct image scanning tests from various years (i.e. 1970s, 80s, 90s) to determine required quality – test compression versus usability.
  • Don’t undersize the technology platform since future retrofitting introduces risk.
  • Leverage provincial record and information management guidelines to save time (ARDA)
  • Plan to run parallel business processes and staff accordingly to handle increased work load.
  • Couple training close to deployment

Summary

ERM projects require a dedicated team of individuals and C level representation within the organization in order to have a chance at being successful. The impact to the organization and the change imposed on its staff is substantial. C level representation will be key in the communications and steering aspect of the project. Expect routine escalations due to pushback within the organization related to making decisions and delegating work. Leverage third parties for support where possible specific to imaging and product expertise.

[1] Record defined: anything (such as a document or a phonograph record or a photograph) providing permanent evidence of or information about past events; “the film provided a valuable record of stage techniques”.

[2] Carousel defined: a record storage and retrieval device that stores jackets based on a classification scheme. The carousel revolves to enable ease of locating and retrieval.

[3] Diazo defined: Photocopying method, which uses a coating of a diazo compound on the paper that makes it sensitive to ultraviolet light, so that when exposed to light, it decomposes.

[5] Note this process occurs within 2 hour period with 24hrs maximum in accordance with service levels.

Advertisements