The application software selected should be the best fit for the busi- ness operation. As a first step, business requirements must be thoroughly documented.
Special Report
Selecting Regulated Application Software Rodeina Davis, Jerry Holcombe (BloodCenter of Wisconsin, Milwaukee, WI) DOI: 10.1309/1956K8CNQ8L5K0LL
Functional Business Requirements The approach is to define detailed requirements for each key business function to be addressed by the software. Detailed requirement definition is pivotal to the selection process and must address critical control points, compliance issues, good manufacturing practices, and essential business functions. Each requirement is weighted according to its importance to the organization (3 – Must Have; 2 – Need to Have; 1 – Nice to Have). Requirement weighting is used to score returned RFPs and should not be conveyed to prospective vendors. Each software vendor receiving the RFP should be asked to rank its software product against each requirement as illustrated in Table 3. It is generally advisable to document detailed requirements in a database or spreadsheet that will automate mathematical calculations and accept data input as vendors respond to the RFP. This tool will automatically multiply the internal weighting for each requirement by the vendor’s ranking to calculate a score. All requirement scores are then totaled by category as described later on.
Request for Proposal The request for proposal (RFP) should include: • Cover letter explaining the purpose of the request, standards for responding to the RFP, point of contact, and timeline. • Minimum system requirements (if the vendor does not meet minimum system requirements, there is no need to respond to the RFP). Such requirements can include 510(k) labmedicine.com
Table 1_General Requirements Sample Function: Backup and Recovery Rank each requirement (3-Meets or Exceeds Requirement; 2-Partially Meets Requirement; 1-Does Not Meet Requirement) and provide comments as appropriate. The organization needs to be highly confident that the data which ensures the safety, potency, and purity of the blood supply is accurate and available. In the event of system failure, a backup process for all data in the system is required as well as the capability for a quick restore of the most current data. Requirement Description Permit a backup of the entire system to be scheduled for a specific time interval (daily, etc). Accurately identify the impact of a system backup on the performance of the system (amount of time required if system is unavailable; % performance is reduced if system is not required to be unavailable). Identify the type of system activity (updating, queries, etc) that can be performed if the system can be operational during a system backup. Identify the impact to data that is file read or being entered manually if a system failure occurs. Identify the amount of data that can be stored on the system and still ensure that the product runs effectively.
Table 2_Key Business Function Sample Function: Infectious Disease Testing: Receive Sample Rank each requirement (3-Meets or Exceeds Requirement; 2-Partially Meets Requirement; 1-Does Not Meet Requirement) and provide comments as appropriate. Requirement Description Identify samples as donor or patient samples within the testing process. Provide the ability to identify donor samples by originating blood center. Provide the capability to upload test orders from other blood centers in automated fashion (ie, blood IDs from outside centers should not need to be manually entered by technologists). Provide method for indicating or “ordering” appropriate set of tests for a given sample (eg, patient vs donor). Allow for multiple tests of a single type to be ordered and/or performed (eg, duplicate repeats, erred tests, 2 ABO tests for first-time donors).
Table 3_Vendor Software Ranking Score
Definition
3 2 1
Meets or exceeds requirement Partially meets requirement Does not meet requirement
October 2006 䊏 Volume 37 Number 10 䊏 LABMEDICINE
593
Downloaded from http://labmed.oxfordjournals.org/ by guest on January 6, 2016
A thorough assessment of software products on the market can be a daunting task, particularly when the software must be compliant with FDA, HIPAA, CLIA, or any other agency’s regulations. The process for narrowing the field of software products and vendors includes defining requirements, publishing a request for proposal (RFP), evaluating vendor responses, checking vendor references, and onsite product demonstrations by the most promising candidates. A predefined methodology is used to make the final selection. The criteria and methodology include evaluation of software and vendor, strategic positioning, and cost. The market generally has a number of software packages designed to meet the needs of a health care organization. The application software selected should be the best fit for the business operation. As a first step, business requirements must be thoroughly documented. Two types of requirements are defined: general and key business functions. General requirements (system security, tabledriven policy, backup and recovery, etc) apply to the use of the application across the organization. Key business functions are specific to operational areas within the organization. Detailed requirements are defined for both. Examples are illustrated in Tables 1 and 2.
Special Report Table 4_Category Weighting
1 2 3 4 5 6 7
Category
Relative Weight
Software Features Performance/Compatibility With Operations Training/Documentation/Vendor Support Implementation/Conversion/Data Scrubbing Vendor Qualification System Architecture Hardware Total
40 5 15 20 10 5 5 100
Evaluation of Software and Vendor Each software package will be evaluated in terms of its ability to satisfy requirements stipulated in the RFP. Seven categories and their relative weights, as illustrated in Table 4, summarize RFP requirements. Software Features – evaluates the key functional and system features provided by a software package; it identifies the ability of the software package to meet the organization’s key business requirements. It also addresses regulatory requirements, good manufacturing practices, and interface compatibility with existing systems and instruments. Performance/Compatibility With Operations - evaluates the capabilities of the application software to align with operational
Strategic Positioning A strategic positioning analysis is important to selection of any key system and includes an assessment of the software’s ability to achieve both the organization’s current and future strategic initiatives. Evaluation in this area should consider today’s and future software offerings, interface engine, product roadmap,
Table 5_Software/Vendor Evaluation Scoring Matrix Category
Weight
Vendor 1
Vendor 2
Software Features Performance/Compatibility With Operations Training/Documentation/Vendor Support Implementation/Conversion/Data Scrubbing Vendor Qualification System Architecture Hardware Total
40 5 15 20 10 5 5 100
40 x __ = 5 x __ = 15 x __ = 20 x __ = 10 x __ = 5 x __ = 5 x __ =
40 x __ = 5 x __ = 15 x __ = 20 x __ = 10 x __ = 5 x __ = 5 x __ =
Category
Weight
Vendor 1
Vendor 2
Current Strategic Business Initiatives Future Strategic Business Initiatives Total Rating
60 40 100
60 x __ = 40 x __ =
60 x __ = 40 x __ =
Table 6_Strategic Positioning Matrix
594
LABMEDICINE 䊏 Volume 37 Number 10 䊏 October 2006
labmedicine.com
Downloaded from http://labmed.oxfordjournals.org/ by guest on January 6, 2016
clearance, relevant and accepted standards, required symbologies, and the key business functions that must be addressed by the software. • Request for vendor to provide the standard software agreement, project plan template, minimum hardware and network configuration, and available documentation such as user guide, installation guide, administration guide, training material, etc. • Supplier questionnaire to be used during vendor qualification (regulated software only). • Functional business requirements.
processes and organizational structure. It specifies, for example, whether a current back office function will be done online by the users, or whether the application will require additional resources for the same activities handled by existing software. The scoring here is the cumulative score derived by the requirements database tool. Training/Documentation/Vendor Support - evaluates vendors on the scope and quality of hard copy and online documentation, screen helps and hints, availability of training programs, the range and frequency of support activities, and reference checks for vendor services and management. Implementation/Conversion/Data Scrubbing - evaluates the vendor’s technical expertise and implementation process to ensure a smooth and organized conversion. Vendor Qualification - evaluates the vendor in terms of commitment to the industry, regulatory standing, financial strength, user appraisal, and responsiveness to requests during the RFP process. An assessment of vendor support during implementation should consider conversion, archiving, and data scrubbing. System Architecture - evaluates the software based on the technical integration of system components, database functions, report generator, and backup capabilities. Hardware - evaluates the hardware considerations in 3 areas: purchase of a new hardware platform, ease of operation, and vendor support. It also evaluates the vendor-recommended hardware configuration based on acquisition of additional skill sets or staff. A ranking system is used to gauge each vendor’s performance against these seven categories. The findings of the evaluation are tabulated as illustrated in Table 5.
Special Report strength of the user group, and the vendor’s commitment to technology upgrades. Further, it might be advantageous that system architecture be open to accommodate peripheral systems and to support product and service enhancement. The strategic analysis can be weighted and scored as illustrated in Table 6.
Cost Which of the competing software products is most favorable in terms of cost? The financial analysis must take into account both one-time implementation and ongoing support costs. Additional elements to consider include any transactional costs, user fees and licenses, additional staffing costs, return on investment specified in the decision to acquire new software, etc. A cost comparison matrix is illustrated in Table 7.
Conclusion The weight of each category in software and vendor evaluation and strategic positioning, along with each section in the final matrix, drives the selection process. It is important that the organization give thoughtful consideration to each weighting element. This involves collaborative decisionmaking by a multi-functional project team formed specifically to evaluate the new application software. The team will be representative of the various business and support areas that will be impacted by the software. Weighting decisions need to be reviewed and approved by a steering committee chartered to guide the project team. LM Acknowledgements: The author gratefully acknowledges the technical assistance and collaboration of two colleagues at BloodCenter of Wisconsin, Patrick O’Donnell and Jerry Holcombe.
labmedicine.com
One-Time Cost Expense Area
Vendor 1
Vendor 2
Application Software License Third-Party Software License Customization Implementation Hardware/System Software/Peripherals Other Consulting Travel/Lodging Contingency Total Rating
$ $ $ $ $ $ $ $ $
$ $ $ $ $ $ $ $ $
Expense Area
Vendor 1
Vendor 2
Application Support Hardware Support Third Party Software Support Total Rating
$ $ $ $
$ $ $ $
Annual Cost
Downloaded from http://labmed.oxfordjournals.org/ by guest on January 6, 2016
Final Evaluation A final evaluation and analysis, based on relative weights assigned to each of the 3 major sections multiplied by the score attained by each software package, is illustrated in Table 8. The highest score reveals the application software that best fits the business operation.
Table 7_Cost Comparison Matrix
Table 8_Final Scoring Matrix Section
Weight
Vendor 1
Vendor 2
Software/Vendor Evaluation Strategic Positioning Cost Total
70 15 15 100
70 x __ = 15 x __ = 15 x __ =
70 x __ = 15 x __ = 15 x __ =
October 2006 䊏 Volume 37 Number 10 䊏 LABMEDICINE
595
