The University of Texas at Austin

ITS Services Status

Current Service Interruptions

5/6/2015
11:14 PM
RESOLVED: Central ITS Oracle RAC
Service Restored

5/6/2015 7:44 PM
The Central ITS Oracle RAC service has been restored as of approximately 7:20PM.

The following environments have been verified to be working:
* IDPhotos applications (Student Photo Roster, Virtual ID Cards, and ID Photo Gateway)
* PyPE applications using Oracle
* EID Admin application
* EID Self-help application

The root cause of this service interruption was storage maintenance (vendor assisted) that did not go as planned.

This is the final planned update for this incident. 

An Incident Report shall be prepared and posted by the Storage team per ITS Incident Communication Policy.


5/6/2015 7:36 PM
The Central ITS Oracle RAC service has been restored as of approximately 7:20PM.

The following environments have been verified to be working:
* IDPhotos applications (Student Photo Roster, Virtual ID Cards, and ID Photo Gateway)
* PyPE applications using Oracle

The following applications are available, but EID updates are being delayed to downstream systems:
* EID Admin application
* EID Self-help application

The root cause of this service interruption was storage maintenance (vendor assisted) that did not go as planned.

An Incident Report shall be prepared and posted by the Storage team per ITS Incident Communication Policy.

An update will follow at 8:00, or earlier as appropriate.


5/6/2015 7:02 PM
The ITS Central RAC Database Server appears to be down.  The Housing and Food Public website is currently not operational as well as several other services that depend on the Oracle Database residing on The ITS Central RAC Database Server.


5/6/2015 7:07 PM
As a result of the loss of connectivity to NFS storage during storage maintenance, Oracle services are not in a desired functional state, and are being restarted in a controlled manner to ensure consistency of service availability. 

The Oracle service disruption is causing downstream outages of services dependent upon the Central ITS Oracle RAC environment, including:

* TIM
* EID Admin application
* EID Self-help application
* IDPhotos applications (Student Photo Roster, Virtual ID Cards, and ID Photo Gateway)
* PyPE applications using Oracle
* Housing and Public Foods website

An update will follow at 7:30 or earlier as warranted.

5/6/2015
11:13 PM
RESOLVED: EID Admin Tool Inaccessible
Service Restored

5/6/2015 7:55 PM
Resolved at 7:44 after the the Central ITS Oracle RAC outage was resolved.


5/6/2015 6:19pm
The EID Admin Tool became completely inaccessible around 6:05pm. All consultants currently working at the Help Desk were able to replicate the issue.

ITS applications staff are investigating.

5/6/2015
6:55 PM
RESOLVED: UT Direct website is unresponsive
Service Restored

5/6/2015 6:51 PM
During a planned maintenance event there was a very brief loss of connectivity to NFS storage.

UT DIRECT and ITS Oracle services have been verified to be functional. Service owners dependent NFS storage should verify the health of there services.


5/6/2015 6:35 PM
UT Direct is responding normally.  ITS Systems staff is investigating the root cause.


5/6/2015 5:56PM
At 5:50 pm today we were made aware that UT Direct (utdirect.utexas.edu) was unresponsive. It appears all users are impacted.

5/6/2015
11:32 AM
Windows Web Hosting: Service Outage
Under Investigation

05/06/15 11:25
At this time, it appears this issue has been resolved but we will leave it ‘under investigation’ for now as we continue to monitor for any further evidence of a problem. 


05/06/15 10:44
ITS has discovered that this issue has not been completely resolved.  The issue remains under investigation.


05/06/15 10:18am
The DNS issue with Windows Web Hosting has resolved and all sites appear to be responding normally again.


05/06/15 09:05
ITS Systems has engaged ITS Networking to assist with the investigation into the DNS resolution issue. There is no ETA on return to service yet. Additional updates will be provided when more information is known.


05/06/15 08:09
Sites hosted on Windows Web Hosting that have their DNS hosted at UT are currently not resolving properly and thus are offline. Sites whose DNS is hosted by a 3rd party provider appear to be unaffected. ITS Systems staff are working to resolve the issue and will update when more is known.

Affected sites include:

Tools.crwr.utexas.edu
His.crwr.utexas.edu
Efis.crwr.utexas.edu
Data.crwr.utexas.edu
Fsel.engr.utexas.edu
www.crwr.utexas.edu
lifelong.engr.utexas.edu
systems.webhost.utexas.edu/sysmod/systems/
https://systems.webhost.utexas.edu/inventory/vsphere/

5/6/2015
8:35 AM
RESTORED:Technology Resources JIRA Application Maintenance
Service Restored

05/06/15 08:31
This maintenance event is complete.


05/06/15 7:12 AM

Event Title: TRecs JIRA Application Maintenance

Description: The Technology Resources instance (issues.ecs.utexas.edu) of JIRA will upgrade to the latest stable version. TRecs JIRA will not be available during this maintenance event.

Start Date: Wednesday, May 6, 2015

End Date: Wednesday, May 6, 2015

Start Time: 7:00 AM

End Time: 8:30 AM

Past Service Interruptions

Upcoming Maintenance

<<< 06/30/2013 - 07/20/2013 >>>
12:00 AM - 6:00 AM
Blackboard Weekly Maintenance
Service: Blackboard
Description: Weekly maintenance event for Blackboard (courses.utexas.edu).

The service may be degraded or down during this time.
12:00 AM - 6:00 AM
Austin Disk Services Planned Maintenance Window
Service: ITS Storage Overview
Description: Customers utilizing the following dependent services may be affected:

* Austin Disk shares (user home directories and departmental shares)
* Stat Apps Server (user profiles, redirected documents ... (more)
12:00 AM - 6:00 AM
Austin Exchange Message Service (AEMS) Planned Maintenance
Service: Austin Exchange Messaging Service
Description: This maintenance includes any or all of the following maintenance tasks:

* Installation of security updates and hotfixes.

* Moving of user mailboxes to different mailbox databases to better ... (more)
10:00 AM - 12:00 PM
Migration of dpdev1.dp.utexas.edu to new servers
Service: UT Direct development environment
Description: We will be migrating the dpdev1.dp.utexas.edu servers to new servers. No user impact expected

Maintenance has been postponed to finish conversion and testing of special functions specific to dpdev1
12:00 PM - 1:00 PM
Windows Deployment Service Server Upgrade
Service: Windows Deployment Services
Description: On July 1, 2013, existing Windows Deployment Services servers will be swapped out with new servers running Windows Server 2012. There will be a brief outage while IP addresses and names are swapped.
7:30 AM - 8:30 AM
Virtual ID Cards Planned Maintenance
Service: Virtual ID Cards
Description: On Tuesday, July 2nd from 7:30-8:30 am the Virtual ID Cards will undergo routine scheduled maintenance. Users should experience no interruption in service.
7:30 AM - 8:30 AM
ID Photo Gateway Planned Maintenance
Service: ID Photo Gateway
Description: On Tuesday, July 2nd from 7:30-8:30 am the ID Photo Gateway will undergo routine scheduled maintenance. Users should experience no interruption in service.
7:30 AM - 8:30 AM
Student Photo Roster Planned Maintenance
Service: Student Photo Roster
Description: On Tuesday, July 2nd from 7:30-8:30 am the Student Photo Roster will undergo routine scheduled maintenance. Users should experience no interruption in service.
9:30 AM - 10:30 AM
Javamail, FTPMail Qual and Test host patching
Service: Mainframe Email
Description: Javamail and FTPMail QUAL and TEST hosts (jmail-t01 and jmail-t02.its.utexas.edu) will be patched serially, with accumulated patches from Red Hat.

There will be no service interruption, as at ... (more)
5:00 PM - 5:15 PM
Wireless Network Maintenance
Service: Campus Network
Description: On Tuesday, July 2, from 5pm to 5:15pm, Networking will be conducting routine maintenance on some parts of the campus wireless network. Clients on the wireless network may see a brief 5-10 second ... (more)
9:00 AM - 11:00 AM
University Directory ( White Pages ) Maintenance
Service: University Directory (White Pages)
Description: Routine software updates for the University Directory (White Pages) will be implemented. No interruption of service is expected.
5:00 PM - 10:00 PM
Blog Service Weekly Maintenance
Service: University Blog Service
Description: Scheduled maintenance may occur weekly on Wednesdays between 5 p.m. and 10 p.m. To the maximum extent possible, installation of service, application, and security updates will be performed during ... (more)
10:00 PM - 11:00 PM
UT Lists planned maintenance
Service: UT Lists
Description: The UT Lists mailing list service at utlists.utexas.edu will undergo routine service maintenance. List mail delivery and access to the Web GUI at https://utlists.utexas.edu may be unavailable during ... (more)
7:00 AM - 10:00 AM
SecureDoc service and server maintenance
Service: Enterprise Whole Disk Encryption
Description: Recurring maintenance window for SecureDoc services.
12:00 AM - 1:00 AM
Mainframe maintenance window
Service: Mainframe
Description: The UT Austin mainframe will undergo planned maintenance early Sunday morning beginning shortly after 12:00 AM midnight and lasting approximately 1 hour into Sunday morning. UT Direct, applytexas.org,... (more)
12:00 AM - 6:00 AM
Austin Disk Services Planned Maintenance Window
Service: ITS Storage Overview
Description: Customers utilizing the following dependent services may be affected:

* Austin Disk shares (user home directories and departmental shares)
* Stat Apps Server (user profiles, redirected documents ... (more)
12:00 AM - 6:00 AM
Austin Exchange Message Service (AEMS) Planned Maintenance
Service: Austin Exchange Messaging Service
Description: This maintenance includes any or all of the following maintenance tasks:

* Installation of security updates and hotfixes.

* Moving of user mailboxes to different mailbox databases to better ... (more)
8:00 AM - 5:00 PM
CX4 Storage Patching
Description: Vendor (Dell) onsite to perform patching of our CX4 storage array.
10:00 AM - 11:00 AM
Migration of utdirect.utexas.edu to new servers
Service: UT Direct
Description: We will be migrating the utdirect.utexas.edu servers to new servers. No user impact expected
6:30 AM - 7:00 AM
PROD uTexas Identity Manager (TIM) Routine Maintenance
Service: uTexas Identity Manager (TIM)
Description: We will be conducting routine maintenance on the TIM Production system. Users of the EID Administrative application and the EID Self Service application may experience connection resets during the ... (more)
4:00 PM - 5:00 PM
ERP Web Infrastructure Maintenance Window
Service: ERP UI Web Infrastructure
Description: ERP UI, ERP Application Registry, ERP UI Site generator application upgrades occur during this time. Minimal impact on users should be expected.
8:00 AM - 5:00 PM
Upgrade hypervisor from ESXi ver 5.0u2 to ESXi 5.1.
Service: Virtual Servers
Description: Upgrade hypervisor from ESXi ver 5.0u2 to ESXi 5.1. This will complete UT-V's upgrade to VCenter 5.1.
9:00 AM - 12:00 PM
uTexas Enterprise Directory (TED) routine maintenance
Service: uTexas Enterprise Directory (TED)
Description: Routine maintenance on TED (The uTexas Enterprise Directory) will be done at this time. No interruption of service is anticipated. Updates to TED might experience a small delay.
9:30 AM - 10:30 AM
Javamail, FTPMail PROD host patching
Service: Mainframe Email
Description: Quarterly OS patching of the jmail-p01, -p02 and -p03 production hosts.

There should be no interruption of outbound email from Natural modules, since the hosts will be patched serially.
5:00 PM - 10:00 PM
Blog Service Weekly Maintenance
Service: University Blog Service
Description: Scheduled maintenance may occur weekly on Wednesdays between 5 p.m. and 10 p.m. To the maximum extent possible, installation of service, application, and security updates will be performed during ... (more)
6:00 PM - 11:00 PM
Session Border Controller Software Upgrade
Service: Voice Services (VoIP)
Description: The vendor will be upgrading the software on the VoIP System Session Border Controllers. Rescheduled from 7/8.
9:00 PM - 11:00 PM
OS Patching of Document Repository production servers
Service: Document Repository
Description: Linux operating system on Document Repository virtual machines will be patched. There will be two brief interruptions of 2 to 3 minutes each during this maintenance window.
7:15 AM - 7:30 AM
MSSQL Maintenance for SQL01
Service: Microsoft SQL Server
Description: in order to accommodate an infrastructure upgrade, the Microsoft SQL Server cluster SQL01.austin.utexas.edu will require a failover. During this maintenance client connections may experience an ... (more)
8:00 AM - 5:00 PM
Upgrade hypervisor from ESXi ver 5.0u2 to ESXi 5.1.
Service: Virtual Servers
Description: Upgrade hypervisor from ESXi ver 5.0u2 to ESXi 5.1. This will complete UT-V's upgrade to VCenter 5.1.
8:00 AM - 5:00 PM
Upgrade hypervisor from ESXi ver 5.0u2 to ESXi 5.1.
Service: Virtual Servers
Description: Upgrade hypervisor from ESXi ver 5.0u2 to ESXi 5.1. This will complete UT-V's upgrade to VCenter 5.1.
7:00 AM - 10:00 AM
SecureDoc service and server maintenance
Service: Enterprise Whole Disk Encryption
Description: Recurring maintenance window for SecureDoc services.
12:00 AM - 6:00 AM
Austin Disk Services Planned Maintenance Window
Service: ITS Storage Overview
Description: Customers utilizing the following dependent services may be affected:

* Austin Disk shares (user home directories and departmental shares)
* Stat Apps Server (user profiles, redirected documents ... (more)
12:00 AM - 6:00 AM
Austin Exchange Message Service (AEMS) Planned Maintenance
Service: Austin Exchange Messaging Service
Description: This maintenance includes any or all of the following maintenance tasks:

* Installation of security updates and hotfixes.

* Moving of user mailboxes to different mailbox databases to better ... (more)
9:30 PM - 10:30 PM
XML Gateway host patching
Service: XMLGateway
Description: XML Gateway service will be interrupted for upto an hour, in order to apply accumulated patches to the host Operating System (Red Hat Enterprise Linux) and software.
10:00 PM - 12:00 AM
Monthly Group E-Mail Maintenance
Service: Group E-Mail
Description: Monthly maintenance window for the Group E-mail server.
7:00 AM - 7:30 AM
Planned Maintenance for Central Web Authentication
Service: Central Web Authentication (CWA)
Description: Central Web Authentication will undergo routine scheduled maintenance. Users should experience no interruption of service.
8:00 AM - 9:00 AM
Urgent Email Planned/Expedited Maintenance
Service: Urgent Email System
Description: The Urgent E-Mail system will undergo maintenance. No user impact is anticipated.
9:00 AM - 10:00 AM
Natclipse server maintenance
Description: The natclipse servers will undergo regular operating system (OS) maintenance on the third Tuesday of every third month. No user disruption is expected due to service architecture.
9:00 AM - 11:00 AM
TEST uTexas Identity Manager (TIM) Routine Maintenance
Service: uTexas Identity Manager (TIM)
Description: We will be conducting routine TIM TEST maintenance. There will be two 10 minute outages during the maintenance period.
6:00 PM - 10:00 PM
Wiki Service Monthly Maintenance
Description: Scheduled maintenance for the University Wiki Service occurs the third Tuesday of the month from 6 p.m. - 10 p.m. To the maximum extent possible, installation of service, application and security ... (more)
9:00 AM - 9:30 AM
QUAL uTexas Identity Manager (TIM) Routine Maintenance
Service: uTexas Identity Manager (TIM)
Description: We will be conducting routine maintenance of the TIM QUAL system. During this maintenance period, there will be one 10 minute outage.
5:00 PM - 6:00 PM
Wireless Network Maintenance
Service: Campus Network
Description: Networking will be replacing a faulty hardware module on one of the campus wireless routers. Users on the affected router will experience a brief (~10 minute) interruption in Wi-Fi connectivity.
5:00 PM - 10:00 PM
Blog Service Weekly Maintenance
Service: University Blog Service
Description: Scheduled maintenance may occur weekly on Wednesdays between 5 p.m. and 10 p.m. To the maximum extent possible, installation of service, application, and security updates will be performed during ... (more)
10:00 PM - 11:00 PM
UT Lists planned maintenance
Service: UT Lists
Description: The UT Lists mailing list service at utlists.utexas.edu will undergo routine service maintenance. List mail delivery and access to the Web GUI at https://utlists.utexas.edu may be unavailable during ... (more)
1:00 PM - 5:00 PM
Urchin Planned Maintenance Window
Service: Urchin
Description: Planned monthly maintenance window for Urchin.
9:00 PM - 11:30 PM
UT-V Virtual Server Consolidation
Service: Virtual Servers
Description: Hardware resources in the UT-V environment will be consolidated in order to maximize efficiency and utilization. No service interruption is expected.
7:00 AM - 10:00 AM
SecureDoc service and server maintenance
Service: Enterprise Whole Disk Encryption
Description: Recurring maintenance window for SecureDoc services.
7:00 AM - 9:00 AM
WebSpace planned database maintenance
Service: WebSpace
Description: The database server for WebSpace will undergo routine software maintenance. WebSpace will be unavailable while this maintenance is being performed. We expect this maintenance window to be exercised ... (more)

We Can Help

Get help from an expert:

* ITS Help and Service Desk

* Call us at 512-475-9400

* Submit a help request online

We also have a walk-in service in the first floor lobby of the Flawn Academic Center (FAC). Stop by and let us help you!