STAKEHOLDER MIDSTREAM CRUDE OIL PIPELINE, LLC

Incident Narrative

PLAINS LOST POWER AT THE WASSON FACILITY ON 3/24 AROUND 14:15. WHEN POWER WAS LOST, IT REMOVED THE PERMISSIVE FROM STAKEHOLDER'S LACT UNIT, WHICH SIMULTANEOUSLY SHUTS BOTH SALES CONTROL VALVES ON THE SKID, SHUTS DOWN THE UPSTREAM PUMPS AT STAKEHOLDER'S SAN ANDRES CRUDE TERMINAL, AND TRIGGERED A CALL TO THE CONTROL ROOM MANAGER. DURING THE INITIAL REVIEW, THERE WERE NO INDICATIONS OF ANY HIGH-PRESSURE EVENTS AND THE SCADA PROGRAM APPEARED TO HAVE WORKED AS EXPECTED. THE NEXT MORNING (3/25), PLAINS NOTIFIED STAKEHOLDER OF A LEAK ON THE LACT SKID. THE LEAK WAS SHUT IN, AND SCADA DATA HISTORY WAS INVESTIGATED. IT WAS FOUND THAT WHILE THE SHUTDOWN COMMAND FOR THE TERMINAL PUMPS WAS TRIGGERED AS DESIGNED, THE TIMING OF THE DATA POLLING CYCLE DELAYED THE COMMAND MAKING IT TO THE TERMINAL. BECAUSE THE TERMINAL AND SALES SKID ARE NOT AT THE SAME LOCATION, COMMUNICATION BETWEEN THE TWO SITES DEPENDS ON DATA TRANSFER OVER SCADA. THIS DELAY ALLOWED THE SALES SKID CONTROL VALVES TO CLOSE BEFORE THE PUMPS FULLY SHUT DOWN AT THE TERMINAL, CAUSING A FLUID HAMMER EFFECT ON THE SYSTEM. THIS FLUID HAMMER CAUSED A PRESSURE SPIKE THAT SUBSIDED SO QUICK THAT THE CONTROL ROOM WAS UNABLE TO CATCH IT. THE LEAK ON THE SKID WAS FOUND TO BE DUE TO A SOLENOID BEING PACKED WITH PARAFFIN MATERIAL, WHICH HINDERED ITS OPERATION AND, IN TURN, OVERRAN THE SAMPLE POT ON THE LACT SKID. THE INVESTIGATION DETERMINED THAT THE BLOCKAGE WAS ATTRIBUTED TO THE PREVIOUS DAY'S EVENT. THE SOLENOID WAS THOROUGHLY CLEANED, INSPECTED, AND RETURNED TO OPERATION WITH NO DEFECTS OR DAMAGE OBSERVED. IN ORDER TO ADDRESS THE FLUID HAMMER EFFECT THAT CAUSED THE LEAK, STAKEHOLDER HAS MODIFIED THE SHUT-DOWN PROGRAM TO INCLUDE A 3-MINUTE DELAY BEFORE SHUTTING THE ON-SKID CONTROL VALVES ONCE PERMISSIVE IS REMOVED. THIS WILL ENSURE THE PUMPS RECEIVE THE COMMAND AND FULLY SHUT DOWN PRIOR TO THE CONTROL VALVES CLOSING. IN ORDER TO ENSURE THAT FUTURE PRESSURE SPIKES ARE ALWAYS CAUGHT BY THE CONTROL ROOM, ANY PRESSURE READING EXCEEDING MOP WILL RESULT IN AUTOMATIC CONTROL ROOM NOTIFICATION AND A CALLOUT FROM THE CONTROL ROOM TO STAKEHOLDER ON-CALL PERSONNEL TO INVESTIGATE THE LOCATION IN WHICH THE PRESSURE READING EXCEEDED MOP (EVEN IN AN INSTANCE WHERE PERMISSIVE HAS BEEN REMOVED FROM A SYSTEM AND THE SYSTEM HAS BEEN SUCCESSFULLY SHUT IN WITH ACCEPTABLE STATIC PRESSURE READINGS). THESE MODIFICATIONS ARE IMPLEMENTED AT ALL STAKEHOLDER CRUDE SALES POINTS TO ENSURE THE EVENT IS NOT REPEATED AT ANOTHER LOCATION.