Bisbee-Baldwin Corporation v. Laurie E. Tomlinson, District Director of Internal Revenue for the District of Florida

320 F.2d 929, 12 A.F.T.R.2d (RIA) 5025, 1963 U.S. App. LEXIS 4859

• Court: Court of Appeals for the Fifth Circuit
• Decided: June 25, 1963
• Docket: 19567
• Status: Published

Opinions

WISDOM, Circuit Judge.

The question for decision is whether payments the taxpayer received for the termination of mortgage servicing contracts are taxable as ordinary income or as capital gains.¹

This Court has recently decided three other cases concerning the tax effects of the termination of similar contracts: Nelson Weaver Realty Co. v. Commissioner, 5 Cir.1962, 307 F.2d 897; United States v. Eidson, 5 Cir.1962, 310 F.2d 111; Commissioner v. Maurice L. Kil-lian, 5 Cir.1963, 314 F.2d 852. In Weaver, Judge Cameron, for the majority of the Court, held: “It cannot be doubted that the sum total of the ingredients of [the mortgage company’s] longstanding relationship with * * * [its] clientele constitutes a property right which is the equivalent of goodwill * * * a capital asset * * * entitled to capital gain treatment” on its sale. Judge Rives dissented: The “part alloca[ble] to the right to receive service fees should be taxable as an ordinary income.”* In Eid-son Judge Tuttle, for the Court, held that it “makes no difference whether the assignment [of rights under the contract] * * * be denominated a sale”; the taxpayers assigned the right to receive “net profit of operations during the remainder of the life of the contract”, which must be treated as ordinary income. Killian followed Weaver “[t]o the extent that the facts * * * established a sale of good will”, but the property sold consisted of “expirations” not a bundle of rights as in Weaver, Eidson, and the case now before the Court.²

* Footnote 6 of the majority opinion in Weaver points out that the question of allocation according to fragments was not presented or argued before the Tax Court or this Court.

/ A majority of this Court conclude that they cannot bridge the gulf between Weaver and Eidson. Since we must choose between the two, we choose to follow Eidson on principle and the solution to the problem suggested by the dissent in Weaver.

I.

The facts are not in dispute.

Bisbee-Baldwin, the taxpayer, is in the mortgage banking business. Most of its loans are secured by mortgages on residential property in Jacksonville, Florida. After making a loan, the company invariably assigns the mortgage to an institutional investor. The essential profit-making element is the investor’s agreement to employ the mortgage company as its agent to service the mortgages. The company receives no profit on the assignment of a mortgage but earns an annual commission of one-half of one per cent of the principal outstanding balance of the mortgages serviced. The servicing activities generate other business. For example, the company often writes fire insurance on the property mortgaged, acts as real estate broker when the property is sold, and serves as property manager when a mortgage is foreclosed. Escrow deposits by the mortgagors enhance its credit standing, a substantial benefit since the company must borrow large sums from the banks in the operation of its affairs. Thus the success of the mortgage servicing business depends upon the amount of mortgage indebtedness it services.

The taxpayer had no right to assign the servicing agreement and could not demand any payment from a successor servicing agent if the investor transferred its business to another company. The taxpayer was not the exclusive agent for any investor, even in the Jacksonville area serviced by it. Each investor had the right to enter into similar agreements with other servicing agents, and the taxpayer had the right to assign and service mortgages for other investors.

During the fiscal year ending April 30, 1957, various investors cancelled servicing agreements with the taxpayer, and gave the business to other agents. When an investor cancels such an agreement without cause, it is customary for the investor to pay a termination fee equal to one per cent of the principal balance of the mortgages then being serviced by the mortgage company. In this case, several of the taxpayer’s agreements with investors expressly provided for such a termination fee. The taxpayer received net termination fees of $206,454.63. The investors paid this sum to Bisbee-Bald-win, but were reimbursed by the new servicing agents for the amount of the termination fees paid to the taxpayer. In substance, therefore, the mortgage servicing was transferred from Bisbee-Baldwin to other agents for, as the district court found, the cancellations would not have taken place had the successor mortgage servicing agents not agreed to reimburse the investors in the amount of the termination fees.

The district court found that:

“Plaintiff kept extensive files con-taiping information concerning the mortgages which it serviced. When an investor canceled an agreement the files containing information concerning the mortgages assigned to that investor were turned over either to the investor or to the new servicing agent designated by the investor. Those files were of value to the Plaintiff in obtaining the [indirect] advantages [of servicing mortgages].”

Bisbee-Baldwin reported the termination fees as a net long-term capital gain of $206,454.43. The Commissioner ruled that the sum was ordinary income and assessed a deficiency of $45,705.35 against Bisbee-Baldwin. The taxpayer paid the deficiency and then brought suit for refund. The district court, citing the “Weaver case [as] a situation very similar * * * agree [d] with the conclusion of the Tax Court that a mortgaging servicing contract is not a capital asset”. The district court also held that the “termination of the mortgage servicing contracts by the various investors did not constitute sales or exchanges by the plaintiff.” Bisbee-Baldwin appeals from the judgment of the district court.

II.

The fact that contractual rights in a mortgage servicing agency constitute a species of “property” under . state law affords no assistance in determining whether such rights are capital assets.³ As the Supreme Court stated in Commissioner v. Gillette Motor Transport Co., 1960, 364 U.S. 130, 134-135, 80 S.Ct. 1497, 1500, 1501, 4 L.Ed.2d 1617:

“While a capital asset is defined in § 117(a) (1) as ‘property held by the taxpayer,’ it is evident that not everything which can be called property in the ordinary sense and which is outside the statutory exclusions qualifies as a capital asset. This Court has long held that the term ‘capital asset’ is to be construed narrowly in accordance with the purpose of Congress to afford capital-gains treatment only in situations typically involving the realization of appreciation in value over a substantial period of time, and thus to ameliorate the hardship of taxation of the entire gain in one year. * * * Thus the Court has held that an unexpired lease, * * * corn futures, * * * and oil payment rights, * * * are not capital assets even though they are con-cededly ‘property’ interests in the ordinary sense.”

In Hort v. Commissioner, 1941, 313 U.S. 28, 61 S.Ct. 757, 85 L.Ed. 1168, the Court held that amounts received in cancellation of a lease were taxable as ordinary income to a lessor. The Court said:

“Simply because the lease was ‘property’ the amount received for its cancellation was not a return of capital, quite apart from the fact that ‘property’ and ‘capital’ are not necessarily synonymous in the Revenue Act of 1932 or in common usage. Where, as in this case, the disputed amount was essentially a substitute for rental payments which § 22(a) expressly characterizes as gross income, it must be regarded as ordinary income, and it is immaterial that for some purposes the contract creating the right to such payments may be treated as ‘property’ or ‘capital.’ ” (313 U.S. 31, 61 S.Ct. 758)

This statement agrees with comparable statements in other cases that the capital gains section is an exception from the usual requirements and must be narrowly construed. See Corn Products Co. v. Commissioner, 1955, 350 U.S. 46, 76 S.Ct. 20, 100 L.Ed. 29; C. I. R. v. P. G. Lake, 1958, 356 U.S. 260, 78 S.Ct. 691, 2 L.Ed.2d 743; Burnet v. Harmel, 1932, 287 U.S. 103, 53 S.Ct. 74, 77 L.Ed. 199.

The question is, what do the mortgage servicing rights under the contracts represent. If they represent the right to earn future income in the form of commissions for services rendered, then the sum received for the cancellation of the contracts and the transfer of rights is ordinary income.

In Roscoe v. Commissioner, 5 Cir.1954, 215 F.2d 478, a sum received by taxpayers for their real-estate corporation stock in excess of the amount received by other shareholders was held to be ordinary income representing the commission which the taxpayers would have been entitled to had the land represented by the stock been' sold directly. The Tenth Circuit recently reached a similar result in Wiseman v. Halliburton Oil Well Cementing Co., 10 Cir.1962, 301 . There the taxpayer had agreed to the termination of an exclusive license to use and to grant sub-licenses for a patented process in exchange for a nonexclusive license and one-third of the royalties received from third party licenses. The Court held that in substance the taxpayer had received the present value of income which it would otherwise have received in the future through the sublicenses. In so holding the Tenth Circuit expressly disapproved and in effect overruled Jones v. Corbyn, 10 Cir. 1950, 186 F.2d 450, on which the taxpayer relies. In that case, the Court gave a broad interpretation to the terms “property” and “capital assets”, and held that money received from the sale of an insurance agency represented capital gains. Indeed, that case has long been recognized as being “in large measure a departure * * * from the general principle governing sale of the proceeds from future services.” 3B Mertens, Law of Federal Income Taxation § 22.34 (Zimet ed.). See 30 Texas L.Rev. 374 (1952).

The line between contractual rights representing capital assets and those representing the right to receive future income is far from clear. Judge Friendly, for the Second Circuit, after an extremely able, thorough survey of all the relevant cases, reached the following conclusion :

“One common characteristic of the group held to come within the capital gain provision is that the taxpayer had either what might be called an ‘estate’ in (Golonsky, McCue, Metropolitan), or an ‘encumbrance’ on (Ray), or an option to acquire an interest in (Dorman), property which, if itself held, would be a capital asset. In all these cases the taxpayer had something more than an opportunity, afforded by contract, to obtain periodic receipts of income, by dealing with another (Starr, Leh, General Artists, Pittston), or by vendering services (Holt), or by virtue of ownership of a larger ‘estate’ (Hort, P. G. Lake).” Commissioner v. Ferrer, 2nd Cir.1962, 304 F.2d 125, 130-131.

In Judge Friendly’s analysis, as in Judge Rives’s analysis in Weaver, some components of the “bundle” of contractual rights held by a taxpayer are capital assets while others represent a substitute for future income. Thus in Ferrer the taxpayer’s “lease” of a play and his power, incident to the lease, to prevent a disposition of motion picture, radio, and television rights until after a certain date were capital assets. Cf. Commissioner v. Ray, 5 Cir.1954, 210 F.2d 390, cert. den., 348 U.S. 829, 75 S.Ct. 53, 99 L.Ed. 654. However, that part of the taxpayer’s compensation for his contractual rights representing his right to forty per cent of the proceeds from the motion picture was taxable as ordinary income. We agree with this analysis. As Judge Rives said in Weaver:

“It seems to me, however, that a substantial part of the $121,841.11 paid to the Mortgage Company was for the right to receive service fees for the remaining years of the outstanding loans. To that extent the Mortgage Company was simply converting future income into present income. Commissioner v. P. G. Lake, Inc., 1958, 356 U.S. 260, 267, 78 S.Ct. 691, 2 L.Ed.2d 743.

“The sale of ‘all of the rights, title, obligations and benefits pertaining to the servicing contract,’ like the sale of any going business, should, I think, be comminuted into its fragments and the ‘purchase price’ should be allocated among the various assets sold. Williams v. McGowan, 2d Cir., 1945, 152 F.2d 570, 572; C.I.R. v. Chatsworth Stations, Inc., 2d Cir., 1960, 282 F.2d 132, 135. I would agree that a part of the $121,841.11 represented the purchase price of capital assets.”

Applying these principles to the factual situation before us, we find that the basic rights Bisbee-Baldwin sold were the annual servicing commissions on the principal balance outstanding on the mortgages. Indeed, the termination fee of one per cent of the mortgages serviced by the taxpayer was equivalent to two years gross income in commissions and was, to our minds, a substitute for the income which would have been earned by Bisbee-Baldwin had the contracts not been transferred.

This holding is, of course, in direct conflict with Nelson Weaver Realty Co. v. Commissioner, 5 Cir.1962, 307 F.2d 897. In an almost identical situation Weaver held that compensation for the assignment of a mortgage-servicing agency should be treated as capital gain. In reaching its decision, the Court relied heavily upon the fact that the amount paid the taxpayer “bore no convincing-resemblance to the amount the purchaser would actually collect out of the service contract sold since the purchaser had to pay the cost of collection.” It is equally true in the present case that the net income from commissions over a two year period would be much smaller than the sum received by Bisbee-Baldwin. But, as this Court recently stated in Eidson, “while, of course, the sum of $170,000 which [the taxpayers] received in the taxable year at issue was doubtless much larger than the income they received in any year for a current year’s operation, this clearly resulted from the fact that they were assigning, * * * not a capital asset whose value had enhanced or accrued over several years, but the right to receive this net profit of operations during the remainder of the life of the contract.” Since, according to undisputed testimony, the primary value of these mortgage correspondent relationships lay in the right to receive annual percentage commissions, there can be little doubt that the bulk of the sum paid to Bisbee-Baldwin was compensation for the loss of such commissions. In such a situation, the statement by the Supreme Court in Commissioner v. P. G. Lake, Inc., 1958, 356 U.S. 260, 78 S.Ct. 691, 2 L.Ed.2d 1143, is applicable.

“The substance of what was assigned was the right to receive future income. The substance of what was received was the present value of income which the recipient would otherwise obtain in the future. In short, consideration was paid for the right to receive future income, not for an increase in the value of the income-producing property.”

The taxpayer seeks to distinguish between the right to receive future income and the right to earn future income. This distinction finds no support in the decisions. Indeed, in Roscoe v. Commissioner, 5 Cir.1954, 215 F.2d 478, the very fact that the taxpayer’s ten per cent commissions on the sale of real estate had to be earned by the taxpayer’s efforts caused the compensation for this right to be classified as ordinary income.

In Gordon v. Commissioner, 5 Cir.1958, 262 F.2d 413 this Court held that a lump-sum settlement of payments to be earned under an employment contract were taxable as ordinary income. In General Artists Corp. v. Commissioner, 2 Cir.1953, 205 F.2d 360, cert. den’d 346 U.S. 866, 74 S.Ct. 105, 98 L.Ed. 376, the commissions were to be earned. In Commissioner v. Pittston, 2 Cir.1958, 252 F.2d 344, cert. den’d 357 U.S. 919, 78 S.Ct. 1360, 2 L.Ed.2d 1364, in Leh v. Commissioner, 9 Cir.1958, 260 F.2d 489, and in Mansfield Journal Co. v. Commissioner, 6 Cir.1960, 274 F.2d 284 the profits taxed as ordinary income were future net profits. In Ferrer the royalties were to be earned by production of a play.

Still, some parts of the “bundle” of contractual rights transferred by Bisbee-Baldwin were capital assets. The mortgage correspondent relationships have value in addition to the rights to servicing commissions. It acts as a “feeder” for related businesses, such as insurance and real estate, frequently engaged in by mortgage bankers. The monthly escrow deposits made by the mortgagors considerably enhance the servicing agent’s credit standing. Moreover, as the dissenting opinion in Nelson Weaver recognized, there is a sale of “good will”. The mortgage portfolio of the mortgage banker tends to increase each year as both the mortgagors and the investors look to the mortgaging servicing agent for further funds and further outlets for investment. The taxpayer’s extensive files and equipment are in the nature of capital assets. (Most of these were retained by the taxpayer.) These items are closely related to the everyday business operations of the taxpayer. They are not so integrally related, however, as to be insusceptible of separate valuation.

We summarize. Essentially, the contract was a management contract for the employment of personal services. The consideration received for the right to earn future servicing commissions must be regarded as a substitute for such future ordinary income. This important part of the bundle of rights sold or exchanged can be separated from the other parts and should be taxed for what it is —not for what it is not. As in Ferrer: a transaction calls for

“In such instances, where part of a transaction calls for one tax treatment and another for a different kind, allocation is demanded, Helvering v. Taylor, 293 U.S. 507, 55 S.Ct. 287, 79 L.Ed. 623 (1935); Ditmars v. C. I. R., 302 F.2d 481 (2 Cir. 1962). If it be said that to remand for this purpose is asking the Tax Court to separate the inseparable, we answer that no one expects scientific exactness; that however roughly hewn the decision may be, the result is certain to be fairer than either extreme; and that similar tasks must be performed by the Tax. Court in other areas, see Webster Investors, Inc. v. C. I. R., 291 F.2d 192 (2 Cir.1961); Meister v. C. I. R., 302 F.2d 54 (2 Cir.1962) [determination of portion of purchase price attributable to good-will].” C. I. R. v. Ferrer, 2 Cir.1962, 304 F.2d 125, 135.

“The contract provided for termination without cause by either party. Plaintiff could not have sold its rights under the contract unless the investor gave approval. Plaintiff did not assign its rights. Its contracts were terminated and the investors then entered into new contracts with other agents. This is not equivalent to an assignment of sale from plaintiff to the new agent. General Artists Corp. v. Comm., 205 F.2d 360 (2nd Circ.1953). Plaintiff had no choice in selecting the new agent. The sums received by plaintiff were determined by the termination clauses in its contracts with the investors or by trade custom. There was no bargaining as to price between plaintiff and the new agents. The elements which normally are involved in a sale or exchange are lacking in this situation.”

The case should be remanded to allow the district “Court to allocate the amount [of the purchase price] according to the realities of the transaction.” Nelson Weaver Realty Co. v. Commissioner (dissent).

III.

Looking only at the form of the transaction, it might be said that the payments fail to qualify for capital gains treatment because there was no “sale or exchange” as required in Section 1221. Thus, the Commissioner argues that on termination of the contracts, Bisbee-Baldwin’s rights vanished; nothing survived to be transferred. Commissioner v. Starr Bros., 2 Cir.1953, 204 F.2d 673, 674, held that cancellation of an exclusive agency did not result in capital gain. Such release not only ended the promisor’s previously existing duty but also destroyed the promisee’s rights. They were not transferred to the promisor; they merely came to an end and vanished.

In General Artists Corp. v. Commissioner, 2 Cir.1953, 205 F.2d 360, 361, this rule was applied, though here the “sale” was to a third person. There the court said: “But we think the correct view is that there was a release to the obligor of a negative covenant in order to allow a new covenant to be made with the third party.” See also Roscoe v. Commissioner, 5 Cir.1954, 215 F.2d 478, 482, cited and followed those cases. See, also, Leh v. Commissioner, 9 Cir.1958, 260 F.2d 489, 493-495. The district court followed these cases in holding that the “elements which normally are involved in a sale or exchange are lacking in this situation.” ⁴

In the fact pattern this case presents, where the “realities of the transaction” undeniably show a market value for the bundle of rights said to have vanished, we feel compelled to look through form to substance. In substance the transaction was a two-party transfer. The investors were conduits: Bisbee-Baldwin received the payments; the transferees paid through the investors. Something was transferred. What Bisbee-Baldwin transferred was a bundle of rights under its contracts to service certain mortgages. For a price, the transferees stepped into Bisbee-Baldwin’s shoes. It .is irrelevant that the investors’ approval was required, that the consideration was fixed in the contract or by trade custom, and that Instead of assignment the transfer was effected by termination of the old contracts and execution of new contracts. These circumstances do not extinguish the fact that Bisbee-Baldwin gave up mortgage servicing commissions and other profits for a consideration. See Commissioner v. Ray, 5 Cir.1954, 210 F.2d 390, cert. den’d 348 U.S. 829, 75 S.Ct. 53, 99 L.Ed. 654; Commissioner v. Goff, 3 Cir.1954, 212 F.2d 875, cert. den’d 348 U.S. 829, 75 S.Ct. 52, 99 L.Ed. 654; Commissioner v. Golonsky, 3 Cir. 1952, 200 F.2d 72, cert. den’d 345 U.S. '939, 73 S.Ct. 830, 97 L.Ed. 1366; Commissioner v. McCue Bros. & Drummond, Inc., 2 Cir.1954, 210 F.2d 752; and Ayrton Metal Co. v. Commissioner, 2 Cir. 1962, 299 F.2d 741.

We hold therefore that property rights passed by sale or exchange from the taxpayer to the successor agents. This holding is circumscribed by our decision ■ that not all of the sticks making up the bundle of rights constituted capital assets. As in Ferrer, although the whole bundle was sold or exchanged, the parties set no separate price upon each stick.

We therefore affirm the judgment below in part, reverse it in part, and remand the case for the district court to determine (1) what part of Bisbee-Bald-win’s compensation was a substitute for future earnings, and therefore taxable as ordinary income and (2) what part or parts, if any, were capital assets, sold or exchanged, entitled to capital gains treatment.

1 . The tax treatment of assignment of income is examined intensively in Lyon & Eustice, Assignment of Income: Fruit and Tree as Irrigated by the P. G. Lake case, 17 Tax L.Rev. 295 (1962). See especially pages 346-353, 393-397. See also Surrey, Definitional Problems in Capital Gains Taxation, 69 Harv.L.Rev. 985 (1956); Comment, The P. G. Lake Guides to Ordinary Income: An Appraisal in Light of Capital Gains Policies, 14 Stan.L.Rev. 551 (1962).

2 . In Killian the Court said:

“To the extent that the facts in the case here under review clearly established a sale of good will attendant to business, the holding of the Tax Court was in accord with this Court’s opinion in Nelson Weaver Realty Co. v. Commissioner of Internal Revenue, supra, and to this extent only is this case comparable to Nelson Weaver. We do not have in the instant case a bundle of property rights involved in a sale as did the Nelson Weaver case. There is nothing what-sover here to be ‘comminuted into fragments,’ with the purchase price then to be allocated among the various assets, as suggested by Judge Rives in his special concurring and dissenting opinion in Nelson Weaver. And by the same token, there are no components of sale requiring separate scrutiny as discussed in Commissioner of Internal Revenue v. Ferrer, 304 F.2d 125 (CA 2, 1962). There was no bundle sold by Killian for the $12,500 payment under review here. The property sold had one single practical and legal attribute: good will.

* s¡« * * *

“We note particularly also that Killian had already received all of the commissions due him under the policies covered by the informational date, and that he had reported them all in his tax returns and received none thereafter. This is quite distinct from the sale of anticipated renewals from, for example, life insurance policies, which pay commissions after the first year without being rewritten in a new policy. There was no service fee or other definitive accrual right here. The Tax Court was correct in its determination that the payment to Killian under these facts was not a lump sum consideration essentially a substitute for what would otherwise be regarded at a future time as ordinary income. See Commissioner of Internal Revenue, et al. v. P. G. Lake, Inc., et al., 356 U.S. 260, 78 S.Ct. 691, 2 L.Ed. 743. There was nothing here which was automatic, perfunctory or legally enforceable coming to Ryan and Fry which could be equated with Killian’s future income. Neither they nor Killian could do more than project, prospectively, the value of Kil-lian’s relationship with the policy holders identified on the informational data.”

3 . Internal Revenue Code of 1954:

Ҥ 61. Gross Income Defined.

“(a) General Definition. Except as otherwise provided in this subtitle, gross income means all income from whatever source derived, including (but not limited to) the following items:

“ (1) Compensation for services, including fees, commissions, and similar items;

“(2) Gross income derived from business ;

“(3) Gains derived from dealings in. property;”

(26 U.S.G.1958 ed., Sec. 61.)

Ҥ 1221. Capital Asset Defined.

“For purposes of this subtitle, the term ‘capital asset’ means property held by the taxpayer (whether or not connected with his trade or business), but does not include * * *.”

(26 U.S.C.1958 ed., Sec. 1221).

4 . The district court held: