AIRBUS A330-243

Full Narrative

SUMMARY

On November 7, 2017 at approximately 9:00 PM Pacific Daylight Time a Hawaiian Airlines Airbus A330-243 airplane, Registration Number N375HA, flight 8075, sustained a control issue of the left-hand (LH) Rolls Royce (R-R) Trent 700 turbofan engine resulting in pulses of flame from the aft of the engine just after landing on runway16L at Seattle-Tacoma International Airport (SEA). After touchdown, the engine emitted sufficient liquid fuel and flames from the exhaust to cause thermal damage to the nacelle, pylon, wing, and flaps. The repositioning flight originated at Paine Field (PAE), Washington and was on a ferry flight after having interior upgrades installed, a 10-day job, and was enroute to Seattle, Washington, to begin regular service. No engine work was carried out during this period.

There were two crew and no passengers on board. It was reported that the pilot was unaware of the fire and was informed of the condition by the control tower. The first officer shut down the left engine using the engine fire switch and discharged one fire bottle. Seattle aircraft rescue and Firefighting (ARFF) responded; however, the fire was extinguished before they arrived.

During an initial inspection, the maintenance staff discovered fire distress on the engine common nozzle assembly, underside of the wing, pylon, flap track fairings, spoilers, and flaps. The initial examination of the incident airplane and engine occurred between November 9 to 12, 2017 at the Seattle-Tacoma Airport.

The engine was shipped to a Rolls-Royce Trent overhaul facility, N3 Engine Overhaul Services (N3EOS) GmbH in Arnstadt, Germany where the team met between December 17 and 19, 2017 to remove specific external fuel related components for detailed teardown.

DETAILS OF THE INVESTIGATION

On-Scene Examination

Engine Data Review

The engine health monitoring (EHM), the aircraft communication addressing, and reporting system (ACARS) and the aircraft condition monitoring system (ACMS) data was reviewed, and the following observations were revealed.

ESN 42543 exceedance messages:
- 05:00:17 (UTC) N2 Redline Exceedance for 3 seconds
- 05:00:32 (UTC) N2 Redline Exceedance for 7 seconds
- 05:01:15 (UTC) N2 Over Limit
- 05:02:46 (UTC) N2 Redline Exceedance for 8 seconds
- 05:03:26 (UTC) Turbine Gas Temperature (TGT) Redline Exceedance

Each exceedance was approximately 104 percent (%) N2 speed.

The following observations were made from the findings:
A comparison of the variable stator vane (VSV) positions revealed that there was a large behavior difference between the LH and RH engines.
The P30 pressures of LH and RH engines were both equal and stable; however, while the VSV positions on the RH engine corresponded to the VSV demand, the LH engine exhibited large variations and did not correspond to the VSV demand.
VSV variations directly impact N2 speed and the high angle of the LH engine VSVs directly increased the N2 to overspeed.
The fuel flow (FF) did not correspond to the engine speed increase during the overspeed events, indicating that the electronic engine control (EEC) was not commanding the overspeed.
The variation in engine pressure ratio (EPR) did not correspond to the FF variation, indicating that the FF was not the significant cause of the EPR variation.
The EEC did not stop the overspeed occurrences because its logic only intervenes above 114%.

Because of these findings, the fuel metering unit (FMU), EEC and VSV control system were examined in more detail.

Event Timeline Data Review

During descent VSV control was slow to respond and engine became increasingly unstable.
The electronic centralized aircraft monitor (ECAM) message ENG1 CTL SYS FAULT “avoid rapid thrust change” message inhibited as aircraft on final approach permitting full reverse thrust application.
Aircraft landed at - 05:01:48.
Full thrust reverse was selected at - 05:01:51 – It is noted that during reverse thrust operation, the EEC logic controls in N1 mode. The VSV system did not respond to engine power selection, and FF increased to maximum output at about 31,600 pph to achieve the demanded N1 speed. This would indicate that flames from the engine tailpipe had not occurred until after the aircraft had landed.
Little or no response from VSV system to commanded thrust resulted in restriction to core airflow and suppression of N1 speed.
Simultaneously the resultant N3/P30 mismatch triggered P30 pipe failure detection, which inhibited the engine surge detection function.
The EEC commanded an increased FF to maximum of about 32K pph in an attempt to achieve the demanded N1, incorrect VSV position resulted in engine surge - 05:01:56.
Unburnt fuel continued to ignite at and behind the engine tail pipe.
Thrust reverse cancelled and EEC logic changed engine governance from N1 to N3 control - 05:02:14.
At this point N3 speed, which was within the specified EEC synthesized levels, stagnated at about 67%, and the control system logic maintained the high FF delivery.
The engine continued to surge, and the airport closed circuit TV (CCTV) footage indicated that unburnt fuel continued to ignite to the rear of the engine tail pipe.
TGT exceeded 900°C and EEC reduced the FF at 05:02:43.
FF demand was only reduced when the EEC detected an engine TGT exceedance, and shortly afterwards the flight crew shut the engine down.
Pilot made aware of tail pipe fire by ATC and engine was shut down using the fire handle (aircraft low pressure spar valve closes) - 05:02:46.
Engine master lever selected “off” and engine shutdown - 05:02:47.

Analysis of the VSV positional data taken from the N2 exceedance reports observed a disagreement between the demanded and actual position of the VSVs, to a point where control was lost. Further assessment noted the VSV response time had become increasingly sluggish for each N2 exceedance.

General Airplane Examination

Initial examination of the incident airplane and engine occurred from November 9 - 12, 2017 at the Delta Airlines maintenance facilities in Seattle, Washington. Investigation team members including the NTSB, FAA, Hawaiian Airlines, UK AAIB, R-R, and ALPA were in attendance. The left-wing external composite panels on the common nozzle assembly, lower panels of the flaps, and flap track covers had evidence of burn patterns and blistered paint consistent with unburnt fuel vapors igniting towards the back of the engine

The event engine was detached, lowered from the airplane, placed on an engine stand, and moved to a secure area. The engine was externally intact and undamaged. The fan could be turned with normal effort and when turned, no grinding or other abnormal sounds could be heard emanating from the engine core. The engine pylon mount hardware was intact and undamaged. There were no leaks in any of the oil or fuel lines.

The engine was externally clean. There were no signs of mechanical or thermal distress. The fan cowls and thruster reverser cowls were undamaged and clean. The front spinner cone was undamaged and exhibited only operational erosion of the paint. The last stage of the LP turbine was undamaged. There was no obvious unusual discoloration on the LP turbine blades. The fan blades were undamaged. The fan case track liner was undamaged and showed no evidence of scoring. The fan was not further disassembled or examined.

The scavenge oil filter and LP fuel filter were removed, examined, and found to be in an unremarkable, nominal clean condition.

A borescope inspection of the entire rotating group was performed and included the LP turbine, high-pressure nozzle guide vanes, HP turbine, combustion section, HP compressor, IP compressor, and IP turbine. All rotating group components appeared to be intact and undamaged.

An external visual inspection of the VSV system found no obvious damage or distress. The VSV rams were disconnected from the unison rings to enable the independent movement of the vanes. The movement of the assembly was noted to be consistently smooth throughout the range with minimal input load.

The common nozzle assembly was intact, however; there was evidence of oily soot at several locations. There was a dislocated panel at the 2 o’clock location that displayed heat distress. The external surface showed evidence of heat distress at the 9 o’clock position consisting of light blistering and discoloration of the paint surface.

Engine Externals Examination and Findings

The engine was shipped to a Rolls-Royce Trent engine overhaul facility, N3 Engine Overhaul Services GmbH where the team met to remove external components below that were germane to the faults observed:

- Fuel metering unit (FMU)
- EEC and power control unit (PCU).
- VSV controller and the RH and LH VSV actuators
- FOHE and LP fuel filter
- Fuel Pump (an assembly, consisting of the HP and LP pumps)
- High pressure (HP) filter – 70 micron (µm)

Variable Inlet Guide Vane (VIGV) & VSV Control System Description

The variable inlet guide vanes direct air into the intermediate pressure compressor at the correct angle-of-attack to avoid compressor surge and stall while maintaining optimum engine efficiency. A single stage variable inlet guide vanes are located immediately behind the engine section stators. A further two stages of variable stator vanes are located after the first and second stages of the intermediate compressor. Two identical VSV actuators provide the power to move the VSV mechanism to the required position. Each actuator is connected to the unison rings via an adjustable bellcrank linkage. The unison rings then connect to the individual VSV airfoils via a lever arm. The actuators are powered by high-pressure (HP) fuel from the VSV actuator control valve and there are separate fuel lines to the ‘extend’ and ‘retract’ sides of the actuator. A variable stator vane control system operates the variable inlet guide vane system by receiving an electrical signal from the EEC that sets positional demand to match the demanded engine power condition. The torque motor responds by directing servo fuel to either side of the control servo valve to extend or retract the VSV actuators to the required position.


Fuel Metering Unit (FMU)

The purpose of the FMU is to control the flow of fuel to the fuel spray nozzles and combustion chamber from electrical inputs from various control units.

The FMU was scanned using computerized tomography (CT) at Rolls-Royce, Bristol facilities and no evidence of internal damage or anomalous features was found. Electrical testing of the unit confirmed the main metering valve, shut-off valve, turbine overspeed, and linear variable differential transducer functions all met the component maintenance manual (CMM) test requirements. No further testing was done.

Engine Electronic Controller (EEC)

The EEC is a dual channel digital unit located on the LP fan case of the engine. Movement of the aircraft throttle levers generates a command signal for the EEC. The EEC converts this signal to an Engine Pressure Ratio (EPR) value or N1 value (when in N1 control mode). VSV scheduling is a function of compressor airflow. This is calculated by the EEC, using measurements of rotor speed (N2) and air pressure (P30). The EEC performs a VSV sweep check on engine starting and engine shutdown and tests the speed of operation of the VSV actuators and alerts maintenance of an impending failure if the time is longer than specified causing action to service the VSV system. The EEC did not issue any warnings of an impending blockage of the supply port fuel filters within the TM, considering the 95% blockage of the filter.

A review of the fault store found the initial EEC fault set during the last flight was for a slow VSV torque motor response, which coincided with the momentary ENG 1 CTL SYS FAULT warning in the cockpit. It was noted that the cockpit ECAM warning “ENG1 CTL SYS FAULT” with the associated message “avoid rapid thrust changes” was observed by the flight crew during approach. However, the aircraft system subsequently suppressed the electronic centralized aircraft monitor (ECAM) fault warning to reduce the flight crew workload during demanding phases of flight. United Technologies Aerospace Systems (UTAS) stated that these faults normally relate to filter blockage, or slow response of the VSV controller, constant pressure valve (CPV), pressure drop regulator (PDR) or the actuator control valve (ACV) itself due to debris or surface lacquering.

EEC NVM data confirmed the faults associated with reduced / loss of VSV control, including -
a) N2 redline limit exceedance.
b) P30 pressure tube leakage / blockage – N3/P30 mismatch (actual reading and not a tube fault).
c) EPR shortfall – Engine control limiting power due to maximum limit.

An ambient, thermo-cycle and vibration test was done with no faults found. No further testing of the EEC was done.

Power Control Unit (PCU)

The PCU is located adjacent to the EEC and converts 115-volt (V) alternating current (AC) aircraft electrical power supply and engine dedicated generator output to 22V direct current (DC) for use by the engine EEC.

The power control unit was tested with no faults found and was returned to service.

VSV Controller Unit

The VSV controller unit consists of the control servo valve (CSV), the constant pressure valve (CPR), the pressure drop regulator (PDR), the main inlet filter (MIF), the torque motor (TM) and the extend & retract filters. The original unit fitted to the engine on entry-into-service with no removals for repair recorded. The CT scan of the VSV controller and actuators revealed no evidence of internal damage or anomalous features, so they were forwarded to the manufacturer for a teardown and examination without functional testing.

The Control Servo Valve (CSV)

There was no binding or resistance to movement of the CSV. The CSV contained very fine black debris particles and displayed no significant level of lacquering. The valve was not examined further.

The Constant Pressure Valve (CPV) and Pressure Drop Regulator (PDR)

The CPV and the PDR contained very fine black debris particles and dark staining was observed around the end of valves. There was no significant level of lacquering. The valves were not examined further.

The Extend and Retract Filters

The extend and retract filters were clear of contamination and not further examined.

The LP Return Check Valve

The LP return check valve was clear of contamination. It was not further examined.

The Main Inlet Filter (MIF) (45µ)

The MIF filter (45µ) showed no visible distortion or breaching of the filter element and was remarkably clean, considering it was just stream of the contaminated torque motor supply port filter. The MIF from the sister engine was removed and a backflush of both MIFs revealed a marked difference of captured debris. Significantly more debris was recovered from the sister MIF compared to the event engine MIF. The captured debris consisted primarily of carbon (C), oxygen (O), sodium (Na), and sulfur (S). Other elements present included iron (Fe), nickel (Ni), zinc (Z), aluminum (Al), silicon (S), magnesium (Mg), and copper (Cu), all consistent with component wear.

Compared to field experience of other engines this clean MIF was inconsistent with its time in service. Evidence of contamination of aluminum sulfate (also known as alum) was found in the VSV actuators and the LP fuel filter. The presence of alum provided evidence of free water within the system, with the water providing the main driver for the potential “cleaning” effect of the MIF, sending a cloud of higher concentration of debris to the downstream TM supply filter causing a sudden increase in blockage instead of predictable gradual operational change.

R-R controls specialists stated that VSV faults were almost always caught by the EEC checks on the ground during start or shutdown. In comparison, the thrust instability on the event engine occurred in cruise/landing phases of flight and different to “normal” experience with this type of fault, indicating a sudden anomaly rather than normal gradual behavior.

The Torque Motor (TM)

The TM operates via an EEC input signal, which energizes the coil in the unit which moves the flapper valve assembly between two nozzles. As the flapper moves away from one of the control nozzles the fuel pressure on one side of the CSV reduces, causing the control piston to extend or retract the VSV actuator rams. The gap between the contacting faces is very small. The event engine’s TM was the original unit fitted to the VSV controller and had been in service for 11,879 hours TSN and 1,945 CSN.

The TM has four filters - Supply Port (SP), Return Port, C1 Control and C2 Control. Examination of the TM revealed significant contamination of only the supply port filter (70 - 80µ) outer element. A backlight assessment of the filters visually confirmed both the supply and return port filters were 95% blocked with contamination. It was noted that the debris accumulation was adhered to the supporting mesh and element rather than being settled on the upstream side. A scanning electron microscope (SEM) energy dispersive x-ray (EDX) material analysis found its composition to include metallic elements (Cu, Fe, Ni and Cr) combined with fuel breakdown products (C, O, S and N).

The other three filters in the TM were blocked between 0% and 5%.


Image of the contaminated TM filter

General Findings:

The VSVC TM supply port filter contained significant contamination and was 95% blocked, restricting the fuel flow thereby slowing the VSV actuators down.

It was noted that despite the finer LP filter (40µ) and a HP fuel filter (45µ) being upstream, the comparatively coarse (70 - 80µ) TM SP filter contained particles of debris that were larger than 45µ that were adhered on the filter element.

A review of the upstream components that had wear type operation with similar elemental composition, identified the HP pump gear (Fe - 82%, V - 10%, Cr - 6% and Mo - 1.3%), HP pump bearing (Cu - 90%, Sn - 5%, Zn - 3% and Pb - 2%) and the FOHE by-pass valve (Fe - 71%, Cr - 18%, Ni - 9%, Mn - 2%) as possible sources of the particles. The particle sizes were smaller than the 10µm, which is below the upstream filter capability of 45µm for the HP fuel filter and the 70µm HP wash flow filter and therefore could enter the VSV TM filters and flapper area; however, if they remained as individual particles they would normally pass through the area. As agglomerations with adhesive qualities, they became attached to the VSV TM SP filter and likely were the reason for blockage.

The adhesive behavior of the particles as observed on the filter could not be positively determined; however, recent industry research of fuel quality have concluded that jet fuel available on the United States west coast contains up to seven times higher levels of sulfates than east coast fuel. It has been noted that sulfate loading on fuel screens is routinely observed both on fuel component screens and on engine fuel filters of other engine installations.
Left and Right VSV Actuators

The two VSV actuators are fueldraulic actuator units were the original units fitted to the event engine since new. A CT scan was performed on both units with no anomalous features found on either unit. Both actuators were disassembled, and an examination revealed no evidence of damage or binding to any of the individual components. The internal fuel-washed surfaces were found contaminated with translucent globular deposits (about 200µm in diameter) and fine black particulates were found on LVDT housing inner diameter, piston stop and jack piston chamber. This finding is inconsistent with super absorbent polymer (SAP) contamination which is typically smaller in diameter at approximately 50µm. The globules turned white when dried, transforming into hollow white friable shells. A lab analysis confirmed the chemical composition to be aluminum sulphate (Al2(SO4)3), also known as alum, a chemical that is not used in the aviation industry and is not a component in fuel. Aluminum sulphate is not soluble in fuel. The presence of the aluminum sulphate in the VSVA's is evidence that free water was present. The source of the alum in the fuel system could not be discovered. A closer examination of the surfaces of the jack piston revealed a mottling of the anodized surfaces. It was observed in the locations of the white deposits, that once they collapsed, some of the alum remain bound to the underlying mottled surface damage, implying a connection between the alum contamination and the corrosion observed on the anodized surfaces.

FOHE & LP Fuel Filter Assembly

The FOHE housing also incorporates the LP fuel filter housing and a spring actuated bypass valve, which is intended to open in the case of a blocked fuel filter. Two pressure differential transducers on the housing, sense and alert the pilot of the impending blockage of the LP filter.

LP Fuel Filter

The filter element was examined by the Rolls-Royce laboratory and an SEM analysis of the filter element found a very small and fine spherical contamination, like that observed in the VSVAs. Chemical analysis revealed the composition to be copper, aluminum, sulfur, and oxygen consistent with alum. R-R stated that the levels of fine debris observed were consistent with the service life of the elements of 4491 hours, however the presence of alum was not. The source of the alum could not be determined. The filter element of the sister engine was also removed and examined, revealing evidence of alum contamination.

FOHE

The event engine FOHE was the original unit fitted at entry-into-service. The unit passed a pressure drop test and fluid leakage check; however, it did not pass a functional check of the fuel filter bypass valve. The measured bypass valve ‘crack open’ pressure was approximately 8 pounds per square inch (psi) compared to the required CMM target of 24-26 psi. Disassembly of the bypass valve found the valve conical sealing face to have minor contact wear which the vendor considered the to be normal for a unit with approximately 12,000 hours of service.


Main (LP & HP) Fuel Pump Assembly

The main fuel pump on the Trent 700 series of engine is a combined LP and HP pump from a single drive off the rear of the engine’s external gearbox. The LP pump has a single stage centrifugal impeller that receives fuel from the aircraft wing tanks and delivers fuel to the FOHE and LP filter. The HP pump is a positive displacement spur gear type pump fitted with a fuel flow relief valve to prevent over pressurizing the pump casing. The HP pump receives LP fuel from the LP filter and delivers it to the FMU and VSV system and is mounted to the pump casing. The HP pump consists of driving and driven spur gears, which rotate within two leaded bronze fixed bearing sets.

The fuel pump assembly was the original unit fitted to the event engine. It was disassembled at the manufacturer.

LP Stage

The lube flow screen was partially clogged with debris that, scoring was observed on the low-pressure pump stage inlet housing and minor impact marks to the impeller were observed; however, all the marks were within the Eaton’s experience of overhauling typical service run fuel pumps.

HP Stage

Inspection of the fixed bearing set noted cavitation damage to the discharge side of all four bearing faces and bearing dams, which, according to the manufacturer, was abnormal. The fuel discharge windows of the gear housing bores exhibited abnormal cavitation erosion wear.

Examination of the HP pump drive and driven gears found evidence of deep cavitation erosion to the tooth roots and drive flanks; however, the pitting was within the allowable CMM limitations and within normal operational experience for approximately 12,000-hour service life.

An SEM EDX analysis of the gear material found the composition comprised of Fe 82%, V 10%, Cr 6%, and Mo 1.3% which was consistent with the specified material. The composition of the gear and bearings closely matched the debris analyzed in the downstream VSV filters. The extent of the cavitation damage would indicate the gears were the probable source for material released into the fuel system.

The cavitation wear in the fuel pump is another unusual finding, which could be a result of low inlet fuel flow caused by upstream blockage (possible FOHE) or fuel contamination. SEM analysis of the debris found in the VSVC TM filter showed agglomerations of very fine debris particulates blocking the filter, and consistent with pump wear and fuel breakdown products. Ordinarily this fine material should pass easily through the filters (MIF measuring 45µ and the Torque Motor supply filters measuring 80µ). However, there appears to be a process in which the material clumps together, possibly with sulfates as a binder.


Fuel

Fueling History of the Event Airplane

According to Hawaiian Airlines (HAL), no service activity related to the engine fuel system of the airplane was done during the 10 days it was in PAE undergoing cabin upgrades.

The fuel quantity of the A330 is 139,090 liters (36,744 US gallons) volume, equivalent to 109,185 kilograms (240,712 pounds) weight.

HAL stated that they do not provide their own ground service at PAE and that the interior upgrade and ground service tasks were subcontracted to Delta Airlines and the Hawaiian and Delta task management systems were not easily compared. Very limited fuel management information was available while the airplane was at PAE, so the fuel management records were therefore not considered very reliable.

Of the records available, during the time at PAE the outboard tanks were left empty which limited proper sumping. On November 3, 2017, 4 days before the event flight, records indicate that sumping of the tanks was scheduled; however, it was not accomplished because the fuel temperature was below 4°C, too low according to the Airbus manual. The low ambient temperatures precluded sumping on at least two occasions. HAL was not able to confirm that that water drains were cleared on the event aircraft prior to their departures from PAE.

According to HAL, the Task Card allows for a minimum fuel of 15K lbs. if there is not sufficient time to preload the aircraft with 60K lbs. and allow 1 hour for settling. 60K lbs. is standard to ensure all tanks have a minimum of 10% of its capacity for gravity sumping. If 15K lbs. is used, the outer wing tanks may have fuel/water below the drain that is only sumped by the suction method.

After the event, HAL initiated a review and revision of their fuel preload procedures.

Fuel Quality

A sample of Jet A were taken from Castle and Cook Aviation, truck 7, which was the truck that last fueled the event airplane, on 2017-11-17. The particulate test determined 0.55 mg/l, within the fuel specification limit. It also passed a thermal stability test at 260°C.

HAL took fuel samples from several wing tanks on February 9, 2018 and sent for microbial contamination evaluation.

The findings from the lab revealed that the:
The "particulate contamination" was in the normal range for ppm.
The fungi quantity in the LH #3 inner tank was 10,000 per milli-liter – considered heavy
The fungi quantity in the #6 RH surge tank was 1000 per milli-liter - considered moderate

Results from this analysis were inconclusive. Although some fungus contamination was found in the fuel, no fungal contamination was found in the fuel system components that were examined.

The possibility of microbial contamination was considered but ultimately discounted because (a) microbial contamination typically shows itself as biofilms and deposits in other areas of the fuel system and blocks filters - this condition was not found. (b) the FOHE end face looked clean from a microbial perspective (c) the SEM EDX analysis of the TM indicated that deposits were from fuel breakdown components, and (d) ay review and comparison of detailed images from experience of biological contamination from another airline operator concluded that the deposits on fuel wetted surfaces of the event system did not look similar regarding the water-soluble microbial deposits seen in their fuel systems.

It is unlikely that the fuel component had a large influence on the clogging of the filters.

Evaluation of Water-Soluble Components in Fuel

The presence of aluminum sulfate (alum) throughout the LH engine fuel system is unique to this event. Alum is used in the water treatment industry to coagulate contaminants and thus aid their removal. It is also used in the paper manufacturing industry. There is evidence that alum dissolved in water can produce sulfuric acid that may chemically attack components and may strip the lacquer off fuel pipes and valves.

The NTSB has found some aviation industry reports that have observed dissolved sulphate in water can act as a binding agent leaving water soluble residues in the locations of screens and filters resulting in a binding attraction from an ionic behavior. The adhered agglomerations observed in the VSV TM SP filter was a possible result of sulfate action.

R-R Test of Hypothesis of Water-Soluble Scrub of Contaminants in MIF

The VSV TM can operate normally with a 95% blockage of the TM filters and the EEC will pick up a gradually slowing behavior of VSV TM and trigger a maintenance action at about 97-98% blockage, so the sudden blockage of the filter with a 95% blockage cannot be explained. Another abnormal finding was the cleanliness of the MIF which was inconsistent with its time in service.

To reconcile these two abnormal observations, R-R developed a hypothesis that the MIF had accumulated water-soluble residues that were suddenly released when an amount of water containing alum was passed through the filter, quickly cleaning it, and causing a cloud of debris to flow to the VSV TM SP filter, clogging it.


Findings of the tests:
Notably more deposits were evident from the post (alum) solution extractions.
This mechanism is consistent with the event MIF releasing debris due to water exposure.
Debris extraction tests indicated this water-soluble binder released significantly more material after the filter had been exposed to water.
The test indicated exposure to both water and the alum solution resulted in some degree of cleaning, with the presence of alum having marginally more impact, likely because of the higher acidity.
Laboratory findings confirmed that filter debris comprised of two distinct elements which were (a) metallic particulates (predominately wear products) and (b) fuel-based products in the form of particulates in a water-soluble binder.
Comparative findings were demonstrated during assessment of the LP filters on the event aircraft.

Subsequent testing with various service-run filters has shown the same results indicating the debris binder is water soluble and is not affected by solvent cleaning.


Airplane Fuel Tank Management

On the Airbus A330-200, fuel is supplied to the engines from the aircraft inner tanks via two main fuel pumps in the left wing and two main fuel pumps in the right wing. The pumps are located within a collector cell to ensure their immersion in fuel during normal operation.

Aviation fuel is hygroscopic and will therefore absorb water from the air. Fuel uplifted in warmer, more humid airports will have a greater amount of dissolved water, despite efforts by the fuel suppliers to remove free water from the supply. The other source of water occurs when the aircraft descends through humid warm air, which enters the tank through the vents. As warmer humid air hits the cold aircraft structure, water condenses on the surfaces.

The HAL A330 fleet incorporate an additional water management system in the form of jet pumps. During fuel pump operation, the jet pumps draw fuel from the low points in the tank and continuously mix any residual water into the fuel to disperse it. This results in the removal of a large proportion of settled water.

If an aircraft is not operated for several days, water begins to settle out and descends to the low points in the tank. To prevent water levels in the tanks increasing to a point affecting aircraft operation, regularly water draining or “sumping” must be undertaken in accordance with Maintenance planning document (MPD) Task 281100-08.

Review of the water management history for aircraft N375HA indicated that no water sumping had been performed following the modification work at PAE. HAL maintenance records indicated that the last aircraft sumping had been completed on October 22, 2017, at which time the fuel tanks were found to be free of water.

The event aircraft had been on the ground for 10 days undergoing cabin modification prior to the incident. Fuel supplier Castle and Cook Aviation uplifted 4300 lbs. of fuel approximately 1 hour prior to engine start, with departure fuel level reading 20500 lbs. HAL/Delta Airlines’ records from the October 27, 2017, indicate tank sumping could not be completed due to low fuel temperature (HAL task card prevent sumping below <5°C). The AMM does provide an alternative sumping method if fuel temperatures are below 5?C; however, this was not applied. The AMM allows for additional fuel to be added, helping to increase the fuel temperature within the tanks’, provided fuel is allowed to settle for 1 hour before sampling.

Examination of the fuel components found significant evidence of water content in the fuel, supported by the presence of alum contamination.

Post event sampling records from the November 9, 2017 recorded 1.0 and 0.5 liters of water recovered from the left and right #3 tanks respectively. A review by HAL operations safety personnel identified several inconsistencies in the water management records, with many not containing information of fuel on board (FOB), temperature and method used.

West Coast Fuel Quality

The NTSB has become aware of other engine shutdowns, where further examination of the fuel related components found ammonium sulfate rich deposits as well as other water-soluble constituents. These airplanes have been operated predominantly in the United States west coast.

Another current study is looking into the fracking, and the methods used to chemically recover the oil. It is thought that many of these chemicals are water soluble and remain in the fuel even after the refining process. These contaminants are subsequently transferred through the fuel system and collect on the surfaces of the filter element. No clear results have yet been published.

HAL gets its fuel overwhelmingly from United States west coast. Nine of the events airplanes last 10 fueling locations were in Seattle-Tacoma International Airport (SEA), Paine Field Airport (PAE), Daniel K. Inouye International Airport (HNL) (5 times), Los Angeles International Airport (LAX), Harry Reid International Airport (LAS), Oakland International Airport (OAK), all in the west coast. The one exception was Incheon International Airport (ICN), Korea.

Other Investigations

During the investigation period, it was noted that other agencies were undertaking research into water-soluble deposits in engine components. The Coordinating Research Council (CRC), a non-profit organization that directs engineering and environmental studies, submitted a proposal on April 9, 2018 to undertake a new research project reference No. AV-25-16. The background to the research noted that several airlines had suffered disruptive incidents (such as aborted take-off, technical delays from engine start faults, etc.) over the preceding years. The incidents primarily occurred in North America and involved several airframe and engine types. The CRC research program sampled fuel from various airports around the USA over a 12-month period and the study is still ongoing at the time of this report issue.

One of the significant findings from the study so far is the discovery of water-soluble deposits on engine hardware that did not appear related to by-products of fuel thermal oxidation, such as water-insoluble fuel lacquering.