The University of Texas at Austin

Current ITS Projects

Product Assessment and Selection Approach

Product assessment and selection are performed during the Solution Analysis phase. The ultimate purpose of the Solution Analysis Phase is to provide the university with a solution recommendation that will address the current and future needs of students, faculty, and staff. A solution can be a software application, software as a service (SaaS), a custom-built solution, consulting services, hardware components, or a combination of these. The approach outlined in this document provides an overview of the steps and procedures used to determine which solution set will best meet the needs of the university and will comply with state procurement guidelines. The outcome of this process documents a solution recommendation. This methodology is not intended to provide detailed guidance for each step; it is intended to give the user an overview of the process and its steps. The methodology is accompanied by templates that teams can tailor to document their findings and criteria.

Assessment Approach

The product assessment approach involves the following steps:

  1. Identify Business Need: This step documents the purpose and goals for the assessment, often to replace the current solution or to create a new solution. The Assessment team defines the business need by gathering information from customers, staff, governance groups, and other stakeholders. The business need can be documented in the Project Charter and is used to describe the overall purpose and goals of the project.
  2. Determine Requirements: Document requirements for the solution and consider functional and technical requirements, application security and identity management integration, accessibility policies, implementation, applications and systems integration, support models, and other needs as defined by the business need.
    Requirements should be prioritized and categorized and identify the “show-stoppers”, which are requirements that must be met for the solution to be successfully implemented; see the Requirements and Traceability template. Once requirements have been solicited, defined, and documented, they need to be approved by the appropriate authority, such as a customer steering committee or project sponsor.
  3. Establish Purchasing Communication Channel: In order to facilitate the overall product assessment and selection, the project team should meet with Purchasing to review status of the product assessment, agree upon next steps, and schedule periodic status updates.
  4. Identify Solution Options: Complete benchmarking with other universities (including Educause data, surveys and/or interviews), industry analysis (for example, Forrester, Gartner, IDC), solutions in use on campus, and other research as needed. Using the research, identify potential solutions that may meet the requirements. Identify any additional assessment criteria, such as vendor business models, licensing, vendor SLAs, maintenance contracts, and timeline for solution implementation. In addition, consider other campus services that need to be involved in assessing the solution, such as the Information Security Office (ISO) for security and ITS for accessibility, identity and access management, infrastructure, and user support. In situations with a wide range or large number of potential solutions, use show-stoppers and high-priority requirements to narrow the options. At times, multiple products may need to be integrated for the overall solution. Assess the solutions and determine which are potential matches using functional, technical, and business model information. This information can be documented in the Research Summary.
    Discussions and/or demonstrations with vendors to learn and understand their offerings and business model/cost structures may be necessary. While general estimates of cost scales are acceptable, specific price quotes cannot be requested from vendors.
  5. Conduct a Purchasing Assessment Review Meeting (PARM): If your solution set involves a vendor, conduct a Purchasing Assessment Review meeting. In preparation for this meeting, determine a preferred selection approach based on the results of the assessment. Approaches may include: exclusive acquisition (sole source, best value, emergency, professional services), DIR/DBITS, RFI/RFP/RFQ, or other purchasing vehicles that may be available.

    The purpose of the PARM is to:
    • Review the documentation of which products are being considered and why;
    • Review the documentation of which products were eliminated from selection and why;
    • Identify potential suppliers;
    • Identify and verify any applicable contracts (DIR, Alliance, other) and procurement vehicles; and/or
    • Decide most appropriate procurement process as determined by the Purchasing Department.

    Based on the outcome of the PARM, proceed to the Selection approach steps in close collaboration with the Purchasing Department. Please see the Purchasing Office home page for more information.

Selection Approach

  1. Create Procurement and Evaluation Documentation: Based on the approved procurement process, work with Purchasing to develop procurement documentation. Purchasing should be consulted throughout the development of the request to ensure compliance with university policies. In the case of a competitive bid via DIR/DBITS, RFI, RFP, RFQ, the team should also define the evaluation process, including a core evaluation team, mandatory requirements, and scoring criteria. Scoring criteria should consider quality of response, cost, references, support options, and other components that distinguish alternatives and solutions.
  2. Perform Evaluation of Solutions: Perform an evaluation of the solutions considering the following:
    Use the Solution Evaluation Summary to document the results. Review the recommendation and obtain required approvals.
  3. Perform Cost Evaluation of Solutions: Perform an evaluation of the solutions considering the following: Solution Evaluation Summary.
    1. Technical Evaluation:To evaluate options, use product demonstrations, vendor responses, solicited user input, product documentation, product trial versions, and industry reviews. Analyze solutions against requirements in conjunction with purchasing procedures to determine which meet the requirements. Requirements evaluation may include consultation with other campus groups that can provide expertise in specific areas, such as security, accessibility, authentication, infrastructure, and user support.
    2. Cost Evaluation:To evaluate costs, develop cost estimates for implementation, on-going maintenance, and sustainment, or use the estimates from the vendor responses, working in conjunction with the Purchasing department.

      The estimates should include:
      • Application costs such as build/configuration, customization, conversion, interface implementation, and on-going maintenance. This should include an evaluation of the skills required for these efforts and the associated training or hiring costs for staff;
      • Integration costs for changes and/or upgrades that would be required to on-campus systems that need to integrate with the proposed solution;
      • Licensing, including initial and on-going maintenance / subscription / support costs;
      • Hardware and operating system or hosting costs;
      • User support, help desk, sustainment, and documentation; and/or
      • Training estimates for the product users, help desk, and support staff.

      Consider the overall cost. Some products may have a low upfront cost but high maintenance or user training costs, while others may have a higher initial cost but lower maintenance and user support costs.
  4. Make Selection: Using the evaluation criteria and working in conjunction with Purchasing, determine the best solution for the university. Describe the technical and cost evaluation of the solutions, and also document the final recommendation in the Solution Evaluation Summary. Review recommendation, and obtain approval by the appropriate authority, such as a customer steering committee or project sponsor.
  5. Award Work with the Purchasing and Software Contracts (if a software purchase is involved) teams to notify the chosen vendor that they have been selected and negotiate the contract. Initiate the definition of the vendor relationship management process that the team will have with the chosen vendor.

Need more information? Contact the Methodology team at in ITS Applications.