The Swing-Arm Controversy

The most difficult of all launch mechanisms to describe verbally is the mobile launcher, at times called the launch umbilical tower. It consisted of three main features: a two-story platform 49 meters long by 40 meters wide, on which the launch vehicle stood both on the crawler-transporter during its journey from the VAB to the pad, and on the pad itself, held erect by four hold-down arms; a tower that resembled the Apollo-Saturn in shape and size, and stood beside it surmounted by a hammerhead crane; and, attached to the tower, nine swing arms of various sizes that carried electric, propellant, and pneumatic lines to the space vehicle. These swing arms would automatically move away from the vehicle between the time of ignition and liftoff. The two-story launch platform housed computers that were connected to the launch control center. The platform also had a 14-meter square opening in the center for the rocket exhaust. Two high-speed elevators were centrally located in the tower. Besides their ordinary function of bringing personnel and equipment to various levels, they formed part of an emergency egress system.18

The three mobile launchers under construction, October 1964.

The mobile launcher: the platform andbase of the tower. Note the four hold-down arms around the square opening for the rocket exhaust. The crawler-transporter is moving beneath the mobile launcher, May 1965.

Mobile launcher on a crawler-transporter in front of the assembly building, July 1965.

The Jacksonville-based firm of Reynolds, Smith, and Hills designed the tall mobile launcher that replaced the umbilical towers previously used at the Cape. Ingalls Iron Works of Birmingham, the prime contractor for steel erection, began work on the first launcher in December 1963. Nine months later workers hoisted the last major piece of steel, a 19-ton crane boom, into place on the first mobile launcher. The crews proceeded to outfit the finished tower with ground support equipment and electrical apparatus. They expected to have the giant completed in another 12 months.19 In February 1965, Ingalls topped out the second tower, and on the afternoon of 1 March of the same year topped out the third (and last) with the hoisting of its huge hammerhead crane to the top of the 136-meter structure.

One of the nine swing (service) arms; when installed, this one would connect to the second stage 43 meters above the base of the rocket.

In planning and building the mobile launcher, the most difficult features were the nine swing arms, or service arms, as they were also called. The Brown Engineering Company of Huntsville, Alabama, designed the service arms in conjunction with Theodor A. Poppel's Launch Support Engineering Division. Brown faced unusual problems: the equipment was novel - no one had built such large access and umbilical devices in combination before; the vehicle for which the swing arms were being designed was developing so fast that the criteria changed continually, even after NASA had let the contract for construction; and the service arms were to be amazingly complex pieces of equipment. By way of example, as many as 24 electric cables, each 50 millimeters in diameter, and about 44 fluid service lines, ranging from 12 to 25 millimeters thick, went into a single umbilical carrier. Each arm would be wide enough for a jeep to drive across - though none ever was to do so. Their length varied with the configuration of the vehicle; they would average over 22 metric tons in weight.20

Employing 250 people on the project, Brown Engineering made 5,000 drawings and 11,000 pages of specifications, but NASA designers found many unsatisfactory features. The company admitted errors in the drawings - but not in numbers or significance out of proportion to the average error rate for such a complicated enterprise. In retrospect, Cliff Boylston, the design project engineer for Brown at the time, was to agree with individuals at KSC that "one typical arm should have been totally tested before going into production." Boylston concluded that the "design was started before the criteria were set.... The developmental effort was not complete before the production started.... [In spite of this] we gave the customer the best effort that he could have gotten anywhere in the time, and within the limitations we had on us....21

Boylston was correct in saying that NASA had not developed a prototype of an entire service arm. As early as 30 July 1963, however, William T. Clearman, head of the Apollo-Saturn V Systems Office, had authorized a prototype within the allowable funds and schedules.22 Before the end of June 1964, NASA had built and tested a partial prototype of arm 6 - a typical one that included all critical aspects. Contemporary photographs of the prototype compare favorably with the final version of service arm 6.23

When NASA opened bids on the service arms on 31 July 1964, the low bidder was Hayes International Corporation of Birmingham, Alabama. A pre-award service team made an on-site inspection of Hayes. The following day Raymond Clark, in charge of the team, reported: "Past experience with sub-standard quality from Hayes, under previous contracts, along with the results of this survey, may dictate an evaluation of Martin-Baltimore." Debus penned a note on the bottom of this statement: "This is in conflict with what you told me."24 The service team concluded that Hayes had the personnel to do the job, but needed additional facilities and tools and would have to incorporate into their plans further recommendations of the survey team. It seemed that Hayes had built up a good team in earlier years, but had lost many of its better men during a time it had fewer contracts. On 25 August the Launch Support Equipment Engineering Division expressed serious reservations about Hayes's technical capacity to perform the task. Yet during the previous week, the Division had changed several hundred drawings, which would have strained the capacities of any bidder.25

In spite of these misgivings, the contract went to Hayes International on 10 September 1964, with a fixed price of $11,480,113 and a completion date of 26 April 1966.26 At this time, incidentally, KSC was under pressure from the Office of Manned Space Flight to contract every possible item by fixed price competitive bidding. The fixed price contract, however, soon proved untenable. The details of the service arms were in a state of constant change, and a fixed price contract is valid only for a fixed design. The arms were a new design, more complex and mechanically larger than those used on the Saturn I.27

Shortly after Hayes International started work, it "uncovered innumerable discrepancies in the design and bills of material." Hayes notified the KSC Procurement Division that although it had believed it had a complete document package, many drawings were missing or tentative. On 27 October, Procurement delivered to Hayes less than 100 missing drawings. Ultimately, however, drawing changes for all reasons, including research and development, went above 4,000.28

To alleviate these problems, in late November 1964 the Apollo-Saturn V Test and Systems Engineering Office of KSC's Apollo Program Management Office concurred with the recommendation of Procurement Division's Launch Support Equipment Section to change the contract so as to incorporate revised lists of drawings. These lists would supersede all documentation previously incorporated, including documents attached to the original invitation for bids and subsequent change orders that revised, added, or deleted drawings.29 In an effort to maintain control over the many changes and revisions, a change review board made up of representatives of the Apollo Program Management Office, Procurement, Quality Assurance, and Launch Support Equipment Engineering began to meet in late September. By early November, the board had promulgated a formal procedure for handling engineering changes in the Hayes contract. Petrone, KSC Apollo Program Manager, approved this procedure on 20 November 1964.30

By the time the review board approved complete drawing documentation, another problem surfaced. On 2 December Hayes could not buy 190 items specified in the contract for sole-source procurement. It seems that, in designing the swing arms, Brown engineers had changed specifications on components without the knowledge or approval of the manufacturers of these components. Further, Brown engineers had accepted sales representatives' promises that their respective companies could meet specifications or proposed changes. Some companies, however, did not back their salesmen's promises and refused to deliver. Hayes then took the position that since the items were listed as sole source, the government was required to specify alternate sources. Hayes would do no engineering or expend any effort to supply items from sources other than those specified in the contract. Neither would Hayes rework substitute items to make them meet specifications unless NASA furnished detailed rework designs, nor would it provide the necessary engineering design to facilitate rework without a contract modification.31

The review board evaluated all design changes for their impact on contract time, costs, and delivery schedules. Hayes and KSC revised the contract whenever necessary. In addition KSC in mid-1965 established a Resident Apollo Program Office, headed by Willard L. Halcomb, at the Hayes plant in Birmingham to reduce the time involved in approving decisions. KSC also set up a so-called tiger team, an ad hoc team that went to Birmingham every Monday to review and identify problems in design and production and returned to KSC to report on progress each Friday. In addition Hayes employed the consultant firm of Booz, Allen, and Hamilton to recommend managerial improvements.32 By late summer of 1965, however, the situation had reached the point that Hayes International management felt it necessary to approach the KSC Procurement Division with a formal recommendation to change the contract from fixed-cost to some form of cost-reimbursable method.

Debus, wishing to have time to assess contract progress objectively following the establishment of the resident program office, waited until early November before replying personally to Hayes's management. In his reply, Debus said:

There is no doubt in my mind that we both entered into the fixed price contract (NAS10-1751) in good faith. . . . It is indeed unfortunate that it was necessary for a considerable number of design changes to be introduced subsequent to the award of the contract. . . . I am fully aware of your recommendation that the method of contracting should be converted from fixed price to a cost reimbursable type. . . . You have been briefed in detail on the reasons why a conversion is neither feasible nor satisfactory to us since it would, in all probability, generate more complications than it would solve. I do, however, have the utmost confidence that so long as a proper spirit is evidenced by our respective representatives at all levels, then we each will be able to achieve our joint objectives - delivery of the highest quality arms, in an acceptable time frame, at a fair and reasonable price. I trust that you too are now encouraged that continuation on a fixed price basis does not present an unworkable contract relationship.33

In spite of Debus's hopes, a subsequent reappraisal converted the contract into a cost-plus arrangement.34

It had originally been planned to transport completed service arms from the Hayes plant in Birmingham to the Marshall Space Flight Center in Huntsville for testing, calibration, and acceptance. When it became clear that such testing would take a great deal of time and the deliveries were already late, NASA decided to have Hayes International deliver untested arms directly to the Kennedy Space Center, for installation on the first mobile launcher. After validation of the overall complex, KSC could then remove the arms and transport them to Marshall for a thorough testing of the service arms themselves. The arms that were not needed at KSC for validating the complex went directly to Marshall.

Eventually the total cost of the contract tripled from the initial $11.5 million. Major mixups had occurred, but none of them was deliberate and, given the press of time, none may have been avoidable. In the end these unprecedented devices performed with astounding reliability and majestic smoothness.35