Lesson Learned Briefing


No.LL19-0001

TitleDeeper Dive Detects Suspect Hardware

EventOther Facility Event

Event Date11/29/2018

CategoryOCA - Property, Procurement, Suspect/ Counterfeit Items

Summary
Statement: An item arrived without the required documentation, and the engineers deeper dive revealed a suspect part. The principles the engineer applied to this procurement apply to every procurement.

Discussion: A refurbished electrical automatic transfer switch (ATS) from a third-party supplier arrived without the wiring diagram or owner/maintenance manual. The engineer involved in this procurement communicated directly with the Original Equipment Manufacturer (OEM). This resulted in the hardware being identified as a suspect/counterfeit item before it could be placed into LLNL inventory.

Analysis: The OEM noted that the label was not theirs - it was a color they had never used, it was not their official logo (font was incorrect), and the serial number field was blank and serial number title was missing (the untitled empty box in Photograph 2).

Further investigation revealed two attributes that would have made this unit dangerously unsuitable (potential personnel injury and property damage) for LLNLs needs (outdoor use and 600-amp capacity):
- A rating of NEMA 1 (for indoor use) and not the NEMA 3R rating (for outdoor use) specified by LLNL.

- The serial number on the items sub panel (27656-3) was a valid Russelectric serial number, but it was assigned to a 400-amp system (Russelectric model RMTD-4004CE). Note that the label attached to this unit (Photograph 2) shows a model number for a 600-amp unit (RMTD-6004CE) with a 600-amp rating. Additional checks of the unit resulted in finding a label that confirmed the 400-amp rating (Photograph 4).


Recommended Actions: Hardware procurements have three key components - the specification for the item ordered, the item received, and the documentation for the item received. All three must be present and match for the item to be accepted. If any discrepancies are found, they must be resolved to a level of certainty that allows the item to be deemed authentic and safe to use for the intended purpose.

In this case, there were multiple discrepancies:

Discrepancy: Required documentation not received.

Resolution Action: Documentation(wiring diagram owners/maintenance manual) requested from OEM. The OEM requested the serial number from the label on the item and LLNL provided the OEM a photograph of the label found (Photograph 1).

Discrepancy: The OEM responded that the label in the picture provided by LLNL (Photograph 1) was not an authentic OEM label.

Resolution Action: Additional examination located a serial number on the unit.

Discrepancy: OEM information for the serial number indicated that the unit does not meet LLNLs specification:
Ordered: 600-amp unit rated for outdoor use.
Received: 400-amp unit rated for indoor use that was labeled as a 600-amp unit.

Resolution Action: RESOLUTION NOT POSSIBLE. Item set aside as a potential suspect/counterfeit item. Additional inspection found a label identifying the item as a 400-amp unit. Supplier notified for appropriate follow-up with respect to suppliers quality control and certification program.

"Pulling the string" resulted in a determination that resolution of the discrepant conditions cannot be achieved for this procurement. The item was set aside as a potential suspect/counterfeit item and appropriate follow-up communications and action were initiated.


Lessons Learned are part of the ISM Core Function 5, Feedback and Improvement. Applicable Lessons Learned are to be considered during working planning activities and incorporated in work processes, prior to performing work.
 
Please contact the following subject matter experts if you have any questions regarding this briefing.

Uploaded documents/attachments:
Deeper_Dive_Detects_Suspect_Hardware_PDF.pdf

Give feedback for this briefing


For other lessons learned and best practices, go to Lessons Learned and Best Practices Library