Change management at the CME

At the Chicago Mercantile Exchange, the onus of determining what changes to make, and when to make them, falls on an approvals board, which meets each Tuesday morning for a risk assessment.

The directors of enterprise technology services, technical support and development make the change decisions. But anyone with a change request must come to the meeting for a quick run-through on what they need to make happen.

“We rely on anyone else in the room to speak up if they think the change will impact them,” says Joe Panfil, director of enterprise technology services.

Changes and testing generally take place when the market is down, from 4 p.m. Friday until Sunday at noon. The application team does its updates first, then the network team works on larger changes that sometimes require that all processing be complete, Panfil says.

With all the changes installed comes rigorous systems testing.

“Somewhere in the 4 to 5:30 a.m. time frame on Saturday, we start the servers for the markets that would come up at the beginning of the workweek and start testing,” he says.

If problems arise, the team rolls back the change, retests the original configuration and makes sure all systems are ready to go when live trading begins at 5 p.m. Sunday. The team will try to implement the change on another weekend.

To accommodate the change-management cycle and to ensure continuous coverage, the 49-member operations team works in three shifts from Sunday 10 a.m. until Saturday 5 p.m.

The 47 software and hardware systems administrators who provide second-level support work in shifts from Sunday 2 p.m. until about Friday 10 p.m.

“Both operations and the software and hardware support teams are here Friday evening supporting system changes, then on a rotational basis here on Saturday to support testing of the changes implemented,” Panfil says.

He notes an automated change-management tool makes the workweek far more predictable for the IT teams.