OIT 2014-15 Activity Summary

Version 1.1: 06/15/2015
Prepared by Kian Colestock

Introduction

The Office of Information Technology (OIT) at UC Irvine is responsible for providing support and technical partnership across a broad range of IT domains. The following report covers the period of the fiscal year 2014/15, highlighting a subset of initiatives OIT is involved in, working with partners on and off campus. A significant change has occurred during the reporting period, both internal and external to OIT. At a high level, activity domains include but are not limited to:

Infrastructure

• Telecommunications & Campus Networking
• Data Center Services Support
• Enterprise Systems Administration
• Commodity Platform as a Service

Education

• Classroom & Lab Technology Support
• Teaching & Learning Management Systems Support
• Academic Senate & Academic Initiatives Support

Research

• High-Performance Research Computing Infrastructure & Support
• Commodity Research Storage as a Service
• Collaborative Research Networking Infrastructure

Enterprise Applications

• Enterprise Financial & Administrative Systems
• Human Resources Systems
• Enrollment Services Systems
• Sponsored Research Systems

Unit Application Support

• Full campus unit IT support for Facilities, Parking & Distribution, Student Housing, Bookstore & Student Center, Human Resources, Student Affairs (Central & Auxiliary), Office of Institutional Research, Budget Office, Graduate Division, Department of Undergraduate Education, Alumni, Office of Research, Chancellor’s Office, Athletics, UCI Police, Environmental Health & Safety.

Central IT Services

• Campus Data Privacy & IT Security
• Enterprise Data Warehouse & Business Intelligence
• IT Architecture, Middleware, and Identity Access Management
• Data and Electronic Document Management
• Operations Support & Production Control
• Project Management Office (Quality Assurance and Project Management)

Client Support Services

• Campus Communications
• Web Content Management
• IT Service Desk & Student Support Desk
• Standardized & Specialized Desktop Support
• Enterprise Licensing and Procurements

Strategic IT Realignment

Over the last 5 years, UCI has undergone a campus-wide consolidation of IT services while simultaneously grappling with unprecedented service demands. Many major new initiatives were launched over the same time period that required OIT’s attention, including the replacement of legacy financial systems with the Kuali Financial System (KFS), Kuali Coeus Research Administration, UCPath, and Student Information System replacement. As a result, organizational changes required to optimize OIT’s ability to support the campus had been delayed. We are pleased to report these have now moved forward.

In determining a new structure for OIT the following was taken into account:

• Definitions of core functions of IT service operations are outlined in ITIL (Information Technology Infrastructure Library).
• Research originating from Gartner, detailing best practices in IT for aligning around service delivery and operational service management.
• The research was done through an engagement with Trullium Consulting to review common central IT constructs used at similar universities.

The result was the creation of five over-arching OIT divisions. Each division represents a balanced approach between simplified customer relationship management and the efficiencies gained from the unification of similar IT services.

Student & Academic Services

Provide services for campus faculty and student communities.  Act as a fast-moving, nimble technology partner in conjunction with evolving educational and research needs.  Support educational technology, research computing, and broad student services.

Enterprise Applications

Manage campus-wide and unit applications through their lifecycle. Help design, test, and improve applications. Identify manageability and functionality requirements for application software. Provide application support and improvements.

Central Services

Provide “internal” OIT services that are universal constructs for other services provided externally.  Major examples are data management, security, project management, quality assurance, and enterprise architecture.

Enterprise Infrastructure

Provide data center, server, network, telephone, and related campus infrastructure. Plan, implement and maintain stable IT infrastructure which is well designed, resilient, and cost-effective.

Client Services

Serve as the central point of contact between the campus user and IT service management. Provide help-desk and desktop support; handle incidents, requests, and support of process activities. Coordinate overall broad campus communication.

Key Initiatives

OIT has identified five, over-arching, strategic priorities as follows:

1. Build out and maintain scalable IT infrastructure and services.
2. Support academic goals through educational and research computing initiatives and services.
3. Partner with functional units to implement enterprise and unit-based applications.
4. Evolve as an enterprise IT organization through continual improvement.
5. Foster technology innovation and collaboration on campus and externally.

The remainder of this report summarizes activities and progress in each of these areas.

1. Build out and maintain scalable IT infrastructure and services.

• Disaster Recovery Planning – By July 2015 we will complete Phase I (comprehensive data redundancy) of a three-phased risk mitigation strategy. Phase II will involve an offsite data center in a seismically different region of the US for critical IT services. Phase III will endeavor to establish true 24/7 service high availability with distributed, redundant assets.
• Campus Data Security Program
  • Development of the Security Risk Assessment – Not having a good way to systematically assess the risk level of our most critical data assets was a significant challenge. In order to properly assess the risks associated with our information technology assets across campus, a “Security Risk Assessment Questionnaire” (SRAQ) tool was developed to provide repeatable and systematic risk measurements.
  • Campus-wide risk assessments in progress – currently, there are nearly 50 centrally supported IT systems/assets undergoing security risk/threat assessments as part of campus-wide efforts to understand and elevate the visibility of critical vulnerabilities to their owners. As an outcome of these assessments, threats will be prioritized and mitigation efforts identified accordingly.
  • Inventory of assets – OIT maintains an interactive inventory of electronic information resources (EIR) which contains all high-risk data stores throughout the campus. Information Security Coordinators (ISCs) in each area of the campus update the EIR database every year to assist in managing high-risk data assets.
  • Policy and Procedures – Much of IT security is focused on infrastructure and escalating technological requirements. As a complementary initiative to technology-based defenses, campus policy and procedures must be vigorously reviewed on a continual basis. Recently we have reviewed and revised technical management policy in our high-risk areas of campus (Student Health, Central Infrastructure Administration, Central Roles/Access Management).
  • Hardening of key infrastructure – key infrastructure components on campus require constant review in order to prevent or detect escalating threats. This year major strides have been made to upgrade campus firewalls and migrate older legacy assets under their protection. Additionally, UCI recently kicked off an initiative to install a high-grade intrusion prevention system which should be in place in early 2016.

• Consolidation of distributed data center assets and standardization of protocols – Major progress has been made to consolidate, retire and centralize data center equipment into UCI’s main data center which provides uninterrupted power, better environmental controls, and physical security.
• Virtualization of critical infrastructure – as a complementary initiative with data center consolidation, UCI has been virtualizing servers that formerly resided on physical hardware. This has allowed us to leverage the scale of a centralized infrastructure along with eliminating the maintenance risk of aging hardware.
• Phone system replacement – Replacement of our 30+-year-old legacy telephone system is nearly complete. The new system is more resilient to catastrophic failure due to a distributed architecture with additional built-in redundancy. It will also provide additional new features including Voice Over IP, mobile device integration, and voicemail-to-email.
• Cloud-based email – the standard email for UCI’s student population is nearly 100% cloud-based (Gmail) and the faculty/staff email managed by OIT (Office 365 or Gmail) will be approaching the same metric by the end of 2015. This will allow OIT to wind down its local email support services and retire/repurpose infrastructure over time.
• Network Upgrade – UCI started a major network upgrade initiative in 2014 to replace equipment going off of vendor support, and to enhance redundancy and connectivity within the campus and externally. This included adding a second 10 Gpbs off-campus link, replacing 4 building routers, adding 10 Gbps ports to backbone routers, and upgrading our intrusion detection system to 10 Gbps. We also replaced central wireless networking (WiFi) controllers and added 300 new WiFi access points to facilitate the increasing demand. UCI is also working with UCLA to construct a secondary external fiber path for the campus through Los Angeles. This will eventually replace our secondary internet connection that is currently provided through expensive leased circuits at slower speeds.
• Utilization of the Public/Private Clouds – we are in the process of moving over 50 campus websites and their content to the Amazon Web Service (AWS) cloud platform. On the web application side, the UC Recruit application (a system-wide shared service) will be our first major application migrated over to the AWS platform – this will be completed by July 2015. As a result of the lessons learned from this migration, UCI will be producing a cloud “toolkit” for developers and system administrators that can be re-used within the campus and other UCs. Lastly, as a third cloud initiative, OIT will be migrating all off-site physical tape backups to Amazon’s Glacier cloud service to provide a more cost-effective and efficient method for backup and recovery.
• Mainframe System Retirement (FS) – In 2014, UCI went live with its new enterprise financial system (Kuali Financials) replacing the 30-year-old FS system running off of an IBM Mainframe. Efforts to archive the historical data are well underway and the last remnants of the old system will be retired and archived by the Fall of 2015.
• IT Accessibility Program – Compliance with the UC IT accessibility policy is challenging given the distributed nature of information technology at each campus and the lack of dedicated resources. UCI is working on an IT accessibility approach that includes a hybrid of representatives from OIT in coordination with campus stakeholders. The intent is to provide a sustainable IT accessibility program for UCI that directly addresses policy requirements, along with communication and outreach, monitoring, and centralized vendor management (captioning products and services).
• Identity Management Upgrade – July 2015 will see the final steps in a 5-year project to replace the Enterprise Identity Management Database used at UCI since 1990. The new IDM System, built at UCI, has integrated data validation, and better data isolation for security and data integrity. It is also more suited to real-time provisioning with platforms like UCPath and UCI’s new SIS.   The new product was built while running in parallel with the existing IDM system, minimizing change for data consumers and IDM-enabled applications for a nearly seamless transition with this critical service.

 2. Support academic goals through educational and research computing.

• UCI LightPath — Thanks to a $500k NSF CC-NIE grant, we have established “UCI LightPath,” a separate “ScienceDMZ” network dedicated to research. The two major campus research compute clusters have been connected to LightPath, along with labs in 7 additional buildings
• HPC Buildout and recharge model – UCI continues to expand the high-performance research computing cluster – HPC – with 43 Faculty Partners contributing servers and storage nodes. A larger than PetaByte high-performance parallel distributed file system has been deployed with the use of the Fraunhofer/FeeGHS file system. A more efficient funding model for HPC is also being reviewed with cluster partners with the goal of helping to fund the purchase of shared hardware and software and with needed staffing increases.
• UCLA collaboration on storage – A major need of UCI researchers is to increase the safety and long-term curation of research data. OIT is testing the use of UCLA’s ‘CASS’ research storage system as a viable shared service. Results of the test are promising and we are starting a pilot test by the faculty of the service while also trying to improve UCI-to-UCLA networking with CENIC. Our hope is to have UCI purchase a large, multi-PetaByte share of the CASS service that would be recharged to UCI users as redundant, off-site storage of research data. Comparisons with commercial storage services show that CASS service would be much faster and significantly cheaper. Data management and curation services supported by UCI Libraries could then be layered upon the service as well.
• eTech – In 2011, UCI instituted the Educational Technology Initiative (eTech) and a new undergraduate lecture course fee to sustain UCI instructional technology. The use of funds from the eTech fee is guided by the Educational Technology Initiative Advisory Committee (ETIAC). In the past year, 20-30 smart classrooms and lecture halls in Engineering, Computer Sciences, and Physical Sciences had technology upgrades and wireless coverage improved or added. In addition, a subset of central computing lab equipment was refreshed, a new virtual computing lab was made available, and numerous educational technology upgrades in schools were funded. 
• Canvas – UCI’s learning management system (EEE) has been one of the “crown jewels” of our learning technology services since its inception in 1996. Since then, commercial products have caught up to EEE and provide better plug-and-play integration with rapidly evolving external learning tools. In 2015, UCI began a pilot of Instructure Canvas with a planned hybrid strategy that will augment our in-house EEE offerings to create a best-of-breed feature set for campus instructors.
• Virtual Computing Lab rollout – In an effort to provide more options for instructional software access, OIT has been piloting a virtual lab (VCL) service with good adoption and usage success. The service, which is provided through California State University Fullerton, allows authenticated UCI students to access computing resources and selected software titles from off and on-campus sites using their own personal computers. In this way, homework and other ‘incidental’ uses of often expensive and difficult-to-install software can be used by students for needs specific to their classes. This coming quarter, UCI plans to end the pilot phase and start marketing the service to the full campus student population.
• Student Success Collaborative – UCI has embarked on a pilot program to analyze sets of enrollment data to understand and predict student success factors. These factors are then integrated into academic advising programs in each school. The pilot uses the EAB (Educational Advisory Board) Student Success Collaborative and is a collaboration among the Office of Institutional Research, Academic Planning, campus leadership, and the schools.
• Library Survey Project – In 2014 UCI began capturing metadata on library remote access/usage, as well as including the use of research services through UC tools. The data collected will help guide future decisions on library services as well as other broader initiatives.
• Digital Media Strategy – The ability to record, provide and archive digital video media is becoming more important with the introduction of online classes, in-course recordings, and the use of internet-based content for large populations of students and campus constituents. OIT is currently reviewing a number of lower-cost cloud-based options for active content hosting and archive support.
• O365 Pro Plus for UCI students – As of spring 2015, all students on campus now have full access to the entire suite of MS Office products in a cloud-based, “take it anywhere and access anywhere” offering. The work platform is available free of charge, as long as they remain a student at UCI.

 3. Partner with functional units to implement enterprise and unit-based applications.

• Kuali Financial System – In July 2014, UCI went live with its new enterprise financial system (KFS); 10 different KFS modules were simultaneously implemented – ranging from travel to e-procurement to the general ledger. KFS replaces an aging and unstable 30-year-old legacy system well past its useful life. In 2015 and beyond, the platform will continue to be enhanced to refine its overall usability and provide a continuing robust set of features that are being driven by the functional offices.
• Student Information System – The implementation phase of a major project to replace UCI’s legacy student information systems with the Ellucian Banner platform has started in January 2015. The multi-year project will retire a loose collection of aging legacy applications and unify the campus business processes around student lifecycle management. In the last year, UCI, with assistance from Deloitte Consulting, went through a rigorous requirements gathering process collecting 3,638 individual requirements, leading to an exhaustive RFP and evaluation process.
• Kuali Coeus (KC) – KC provides an integrated research administration system that supports the management of research projects from “cradle to grave” across all funding sources; from the time a proposal is initially conceived until final reports are submitted and approved by sponsors. KC will make research administration easier and more efficient through an integrated set of application modules. As of 2014, UCI has implemented the electronic grant proposal submission, routing, approvals, and negotiation functionality. In 2015, conflict of interest, award, and sub-award functionality will begin implementation.
• Data Warehouse build-out – The power of data to inform key decisions, is undeniable in today’s large organizations. As our technology becomes more connected and enterprise-wide, the need to capture and apply business intelligence to our institutional datasets is paramount. In 2015 we continue to aggressively build out our data analysis capabilities in the domains of financial, HR and staffing, and student/academic as well as institutional and academic research. We are also working with the Office of Institutional Research, the Budget Office, Chancellor’s office, and other units to create targeted, meaningful BI outcomes. In order to scale out BI for the entire campus to leverage, UCI will be rolling out new self-service student data BI capabilities in 2015.
• Electronic Document Management – In the summer of 2015, UCI will have implemented the campus’s first major commercial product (IBM FileNet) to handle enterprise-wide electronic document management. The initial use will be for HR document management but will expand to other uses later this year – including sensitive data management and special document handling.
• Facilities Work Management – A major upgrade of the campus facilities management service platform (IBM Tririga) is slated to complete in July 2015. This project unveils more software automation and work-management efficiencies within the common system, as well as business process re-engineering that has taken place in the campus facilities management departments. New capabilities include the ability to integrate better with a mobile workforce and better data/reporting on operational service performance.
• UC Path – UCI has been participating in the initiative led by the office of the president to centralize the support of HR/Payroll functions within the UC system. From the technology standpoint, we will be retiring a 35-year-old legacy payroll/HR system (PPS) and adopting the Peoplesoft platform as a target in 2017-2018.
• UC Recruit – UCI hosts the UC Recruit system, which provides an elegant and intuitive platform to manage the faculty recruitment process; it has now been deployed for all 10 campuses. UC Recruit exemplifies the attributes of a successful UC shared service. The platform continues to evolve its functionality and general capability set in 2014.
• Sensitive Data Management – a number of campus units either work with or have exposure to sensitive data of varying degrees throughout the business of day-to-day campus operations. UCI is assessing a combination of products and solutions to address the unmet need for secure file transfer, sensitive data protection in transit, along with tools required to communicate with external entities in a secure fashion.
• Campus unified storefront – at UCI there are currently 44 merchant gateways on campus that accept credit cards for a variety of goods and services. In 2015, the campus will be implementing a unified storefront from Touchnet to accept credit cards in a single location, under a centrally managed model. This will reduce the combined cost and risk of maintaining distributed credit card infrastructure and PCI compliance activities going forward. Additionally, the storefront brings new opportunities around event management and ticketing.
• Enterprise Access Management – In 2014 UCI implemented an enterprise roles & access management system (KSAMS) which contains a fully electronic workflow with approvals. To date, the system is managing the enterprise roles for the financial system, ServiceNow, and the campus access to data warehouse segments. Planned for 2015 and beyond are more integration with enterprise tools such as the Student Information System and Research Administration.

4. Evolve as an enterprise IT organization through continual improvement.

• IT Service Management Platform – OIT began a multi-year service management platform implementation using the industry-leading product ServiceNow. The initiative endeavors to unify IT service delivery onto a single platform, replacing half a dozen non-integrated tools. By July 2015, campus clients will be able to use a self-service portal to submit and track the progress of their requests or simply call the help desk which now has electronic workflow routing with the rest of the 59 OIT service teams.
• Service consolidation and portfolio management – as part of our department re-alignment, our IT portfolio has been under examination looking for opportunities to refine, retire or invest in our services. One outcome has been further efforts to retire some of our “on-premise” commodity services in favor of cloud-provided options. We are also working to consolidate some overlapping roles, responsibilities, and similar service offerings (server administration, identity management, and software development).
• Benchmarking Information Technology – OIT engaged a consultant from Trullium Inc. to do a benchmarking study to explore common IT organizational constructs in higher education, along with comparing and contrasting key metrics with similar campuses. The outcome of this study informed our realignment strategy and also identified key differences in technology investment across similar service offerings.
• Development of Organizational Performance KPIs – the benchmarking study mentioned above provided some metrics around staffing, funding, and technology investment. These include IT spend – per student served, FTE supported, and by IT domain. Other measurements included the ratio of central IT investment to distributed IT investment, plus staffing, outsourcing, and investment trends comparing UCI to similar peer institutions. On the operational side, OIT has begun deeper adoption of ITIL concepts in ServiceNow and we are starting to gather new performance analytic metrics on our workforce (work queue depth and trending, work queue average per FTE) service delivery (SLA compliance per service/team/individual, average fulfillment times), costs (infrastructure and labor spend breakdowns), and risk (process breakdown metrics and risk rating for work across the service portfolio).
• Strategic Cost Management – cost control and leveraged economies of scale are part of our push in 2014 to slow the growth of central IT maintenance and support. The multi-faceted strategy includes re-negotiating and consolidating pre-existing vendor contracts, centrally managing commonly budgeted expenses, reviewing cost/benefit through an acquisition review team, re-calibrating cost recovery charges to closely match expenses and the use of selective outsourcing for specific short-term needs.
• New Quality Assurance Program – in support of our new financial system, and in conjunction with IT realignment activities, a new quality assurance program was initiated and a team was formed to manage it. A major component of the QA program is the integration of a fully automated software testing platform (Cucumber), including a framework for standardized distributed use and management. To date, UCI’s QA automation platform is testing hundreds of pieces of functionality every day, across several different software products – with more planned in 2015.
• Formal IT Governance – A climate of finite IT resources, coupled with multiple priorities and competing interests, makes focusing on the right work at the right time a complicated question to answer. In 2014, UCI made considerable strides in maturing its governance mechanisms to better focus on the “right work at the right time”. Using many of the pre-existing unit-based working groups as a foundation, executive-level steering committees were formed, as well as supporting structures for proper architecture and acquisition review.

Maximizing our IT talent and technical capabilities is an important component of evolving as an IT organization and continual improvement:

• Operational Risk assessments – on a number of fronts, OIT is assessing the level of risk across a broad spectrum of services and support. In 2014 as a parallel initiative with the department realignment, operational support risks were identified, classified and remediation plans created. These plans involved staff cross-training, infrastructure replacement/upgrades, and information documentation and dissemination.
• Organizational Performance Management Program
  • Cascading goals –to align OIT around clear, central themes of purpose and focus, the concept of cascading goals was introduced. The five strategic goals laid out in this document represent the template for what future work should be aligned with and measured against. The goals go 3 levels deep into the management structure and are eventually tied to an individual manager’s performance.
  • Strategic Objectives – the collection and illustration of UCI’s IT 3-year strategic objectives were laid out last year and are updated on an annual basis in conjunction with the annual budget process. The objectives fulfill strategic needs in the dimensions of improvement in operations, customer focus, financial stewardship, and employee workforce management, and campus strategic initiative partnership.
  • Performance Management – performance management consists of standardization of our rewards and recognition programs, performance evaluation criteria, and ratings, plus the integration of goal accomplishment with all of the above.
  • Succession Planning – to mitigate risk from the potential loss of key individuals, and to provide organizational flexibility, OIT has made efforts to establish more formal succession plans.
  • Job Standardization – to the extent practical, we continue activities to normalize IT jobs against market benchmarks and expectations of job duties.

• Strategic Talent Acquisition and Retention – the IT market is increasingly competitive and OIT has faced significant challenges in recruiting staff. In 2014 OIT instituted a new “contract to hire” policy for recruitments, allowing hiring managers and candidates an evaluation period to assess fit for the role, while providing the flexibility of hiring more aggressively for emerging needs. Additionally, a dedicated IT recruiter position was created in HR to shorten the length of open recruitments and deepen the applicant pools. Lastly, agreements have been signed with IT staffing vendors for talent acquisition and placement. The results of these actions have started to bear fruit, with the average time to fill vacant positions starting to decline.
• Targeted Training (SANS, PM, Data Privacy, Cognos) – the OIT security team has been rolling out and sponsoring various security training sessions with audiences ranging from extremely technical to casual end users. Additionally, our data warehouse group will be rolling out similar educational sessions with the objective of scaling BI capability through education.

5. Foster technology innovation and collaboration with campus and externally.

• System-wide IT shared services – As a member of the UC Information Technology Leadership Council (ITLC), UCI’s CIO helped author/edit the “Pillars of Collaboration” whitepaper. The piece lays out the foundational aspects for cross-campus collaboration in information technology and has materialized in several actionable initiatives that are laid out below. The CIO and Assistant CIO are also involved in a number of other UC “shared service” development activities, including the recruitment of an “IT Planning Lead” to coordinate efforts, and the development of a “Shared Services Framework” to operationalize fundamental questions like funding and governance, while creating incentives for adoption and minimum standards for engagements. OIT staff are also participating in an effort to establish interoperability standards to make it easier to share applications across campuses.
• Kuali Foundation contributing partnership – As a contributing partner to the Kuali Foundation, UCI has agreed to contribute KFS code extensions to be part of the product’s base code in 2015. The functionality under discussion spans across financial processing, disbursements, and security improvements.
• UC Data Center collaboration – UCI is in need of an alternate redundant data center site to provide business continuity and rapid recovery in the event of a disaster affecting critical infrastructure. UCI will join several other UC campuses and medical centers in a joint RFP to locate and share a viable secondary site. The outcome from this collaboration is promising in terms of overall risk and cost reduction over the long term – as opposed to UCI as an individual actor.
• CSU Fullerton VCL collaboration – The Virtual Computing Lab (VCL) capabilities OIT is providing UCI students mentioned earlier uses California State Fullerton’s (CSUF) Virtual Computing Lab services. UCI and CSUF have partnered to expand the use of VCL while funding CSUF to further develop the VCL service.
• Instructional Technology Workshop offerings for instructors – OIT is working with campus partners to support faculty-focused workshops on instructional technology, pedagogical approaches, and third-party tools for instructors. Collaborative projects include instructional tool workshops, Faculty Institute for Hybrid Learning (FIHL), the Introduction to Hybrid Learning, Flipped Classroom workshop, and the New Faculty Orientation.
• Time & Attendance (Multi-Campus Pilot) – The time and attendance system written and used by the UCI campus (TRS) has been architected in a way that it can be adopted by other UC campuses. The system is also used by UCLA and UC Davis. Starting in spring 2015, UCI and UC Hastings will be working on a collaboration project to have UCI implement and host the system for UCH. Using the new shared services framework as referenced above, the collaboration will serve as a pilot for other similar shared service initiatives.