How Warren Buffett’s uphill battle against the S&P 500 is changing

Warren Buffett’s Berkshire Hathaway made some big purchases in 2020, but the biggest Buffett buy speaks to the big challenge the company will continue to face: there was nothing it could find at a better price than its own shares.

Berkshire bought back more than $18 billion in its own shares last year, through October, almost twice as much as Buffett spent on his next biggest acquisition, the $10 billion natural gas assets transaction with Dominion Energy, which including the debt acquired, was the largest acquisition since 2016. Taken at his word — which he has earned — that he won’t do a share repurchase just to prop up the stock, the big buyback was a signal from Buffett that he thinks the stock is undervalued.

“If he had acquired a portfolio company for $18 billion or purchased $18 billion in the stock of a public company, we would call it significant,” said James Shanahan, Edward Jones analyst, who estimated the buybacks through Oct. at $18.4 billion and maintains a positive outlook on Berkshire shares.

“Few better places for Berkshire to allocate its capital than to investing in itself with share buybacks,” says Lawrence Cunningham, author of several books about Warren Buffett and Berkshire, and a professor at George Washington University. “Price was right. Disclosure was clear. Pure investment rationality.”

Even if Buffett is making the right call on the buybacks, the size of it being bigger than any M&A acquisition since 2016’s deal for Precision Castparts highlights a reason Berkshire has struggled to keep up with the broader market in recent years: investors don’t see Buffett being able to make the bold bets that made his name and so many Berkshire shareholders’ fortunes.

Over the past two decades, Buffett has done reasonably well against the index, actually beating the S&P 500 in 12 calendar years between 1999 and 2020. Though years in which Berkshire has trailed have included some big margins and resulted in the company lagging the index over the past 5-, 10- and 15-year periods, according to data from S&P Global.

In 2020, Berkshire Hathaway shares were up, but not by much (2%), against an S&P 500 that gained over 18%, with dividends reinvested, according to S&P Global. Taken together, the two-year stretch of 2019 and 2020 marked one of the biggest gaps between Berkshire and the broader U.S. stock market in recent history, with the Buffett trailing the index return by a combined 37%.

At least in the short-term, investor sentiment is against the company because market forces are working against him. The market Buffett faces is defined by record low interest rates — which the Fed has signalled will remain the norm for some time — making more cheap debt available to fund acquisitions from competitors, such as private equity, that don’t now need the cash reserves of a Berkshire. As a result, asset prices across the market are elevated.

“No one is contesting his multi-decade track record, the intelligent use of cheap flows to build tremendous value over time. He has done that repeatedly, but now capital is cheap for everyone,” says Meyer Shields, a KBW analyst. “It diminishes his advantage. It was an element of the secret sauce, but maybe the best of bad choices is to not buy anything.”

Shields, who has covered Berkshire Hathaway for a long time, often critically, and has had the distinction of being excluded from analyst invitations to the company’s annual meetings, says Buffett can be right about not overpaying and still be punished by investors for that decision. “It mostly comes down to valuation for acquisition opportunities and barring some crash, I don’t see tremendous opportunities for acquisitions going forward. We are stuck with the idea that they have a ton of capital, but so does everyone.”

Between the pandemic onset and through the May virtual annual meeting — where Charlie Munger was notably replaced by Berkshire Hathaway Energy head Greg Abel as Buffett’s counterpart, which will be the case again this year as speculation centers on Abel as the first post-Buffett CEO — frustration with the pace of investment activity grew. Buffett was uncharacteristically fearful when others were fearful, too.

“It was fair for them to be concerned about the implications of the pandemic. But people were nervous because they didn’t see opportunities … and they didn’t see attractive stocks to buy, and weren’t even buying back their own stock, just hoarding cash and people worried about what he would do with it,” Shanahan said.

In March, no one had a handle on what mortality rates from the pandemic would be, or how the government would act, and Berkshire’s long-term history shows that Buffett holds cash during periods of risk. “There might have been some disappointment, but he was being consistent,” Shields said, even when it came to not rushing to buy back Berkshire shares.

The distressed investments Buffett was able to make in the last financial crisis of 2008, meanwhile, were not repeatable during the pandemic, as the government acted rapidly to stimulate the economy and support devastated industries.

“This time around government support was swift, rolled out so much faster than in 2008,” Shanahan said.

The one investment that ultimately helped to boost investors confidence starting in the summer was the increased buyback activity being disclosed. “They got aggressive with the buybacks from June through October. There is not as much competition to buy back his own stock,” Shanahan said.

Cunningham expects more of the same: “Added bonus: get rid of the short-term Berkshire shareholders, such as Bill Ackman, in favor of the true believers. … expect continued repetition.”

The $145 billion last reported in cash by Berkshire does need to move to keep the stock moving higher.

“The worst thing that can happen is they aren’t deploying capital. I would like to see much more cash put to work. It is a huge driver of earnings and market cap,” said Shanahan. “But there are very few acquisitions over $20 billion. … Capital deployment in excess of cash generation will drive the stock and the easiest way to accomplish that is buybacks.”

Precision Castparts, in 2016, was a $37 billion deal.

In recent years Buffett has made clear that assets are so pricey, his preference is to focus on buying more stocks of public companies. But that had the unintended consequence of making some of his smaller missteps with those public stock holdings more notable in 2020.

Berkshire sold the big stake it had recently built in airline stocks including American, Delta, United and Southwest, early in Q2 2020, possibly near trough valuations. And through the end of the second quarter, Berkshire was a net seller of stocks. Its reduced holdings of financial stocks, including investments in Goldman Sachs, Wells Fargo, JP Morgan, PNC, M&T Bank and Bank of NY Mellon — which in the broader scheme of things should attract more investors as the public stock portfolio tilts more to tech and consumer trends — did include some questionable timing decisions.

“The airline stocks was messy,. He certainly sold at a bad time and was liquidating bank stocks at cheap valuations. JP Morgan has rallied a lot,” said Shanahan. “Those were timing mistakes. With the airlines it was the right thing to do, but the wrong time to do it.”

Buffett has been loading up on Bank of America as his No. 1 bank stock for the future. “Bank of America has been a homerun,” Shanahan said of the investment agreement first reached in 2011 when Berkshire bought $5 billion in bank preferred shares and the right to 700 million in common shares over a decade at a price of roughly $7 per share, which currently trades above $30.

“Getting rid of some of the financials was my biggest frustration,” said Greg Womack, president of Womack Investment Advisers, which has held Berkshire as a core portfolio stock across many client accounts. “The timing hasn’t always been the best … but I don’t think they look so much at the technicals.”

With more of the investment decision-making being in the hands of hedge fund managers brought in-house years ago,Ted Weschler and Todd Combs, the timing decisions could change more in the future, analysts and investors say. The recent past has seen a significant shift in the stock portfolio managed by Berkshire.

At the end of 2018, the portfolio was still “handcuffed by financials,” Shanahan said.

Even with Apple as its largest stock holding — which has proved to be a great investment in tech by Buffett after his IBM stumble and has grown so much Berkshire was led to trim the position last year — Berkshire’s internal managers who have discretion to buy in the billions of dollars have been underweight technology, and that is one reason it has trailed the S&P 500.

“These portfolio managers charged with trying to outperform the market didn’t own stocks in the sector that was the strongest performing,” Shanahan said. “Financials like Wells Fargo and U.S. Bank sort of set them up to underperform into 2019 and 2020. It is a hard thing to do, hard to beat the market, when you have large percentages in value stocks. They took action.”

Since the end of 2018, Berkshire’s stock portfolio exposure to financials has fallen from 49% to 28%, while tech has swelled, primarily due to Apple. Technology and consumer stocks are now close to 50% of the public company holdings. Stocks that have appeared in recent years include Amazon, T-Mobile and, in the most recent quarter, a handful of health-care stocks.

“Tech, communication services, health care. It is starting to have more of a new economy feel and look more like the market,” Shanahan said. In the Sept. 2020 quarter alone, Berkshire disclosed $7 billion in new positions in tech, communications and health care sectors. Health-care stock Davita is a major Berkshire holding, but the company has not historically invested much in health care sector relative to other sectors.

The most surprising new portfolio direction was Berkshire’s first-ever U.S. IPO investment, Snowflake, which turned out to be the biggest tech software IPO in history.

“Tech investments Buffett didn’t used to think he understood. The industry is easier to understand now, and unavoidable,” Cunningham said. “But those companies [Apple, Snowflake] are still very much Berkshire kinds of companies, led by long-term strategic thinkers who want to have high quality shareholders.”

Snowflake may have been in investment led by Todd Combs, who Shanahan noted has signed the paperwork on the deal and was also behind Berkshire Hathaway’s investment in Brazilian credit-card-processing company Stoneco, which was made just days after the IPO in late 2018.

A very Buffett-like deal in support of Scripps’ acquisition of ION Media, was led by the other manager Ted Weschler, through which Berkshire invested $600 million in preferred stock (8% dividend if paid in cash, 9% if deferred).

With prices high in the U.S. market, Berkshire bought slightly more than 5% stakes in Japanese conglomerates Itochu, Mitsubishi, Mitsui, and Sumitomo & Marubeni, driven by frustration with the lack of opportunities in the U.S., according to Shanahan. And Berkshire issued yen-denominated bonds at an all-in cost of about 1% to finance the transaction. “Highly unusual, though. Berkshire hasn’t made a lot of large investments in international stocks in the past,” he said, adding that the transactions can be increased to near-10%.

Buffett also ended his remaining ownership within the newspaper business in 2020 and while not surprising, it was notable in light of Buffett’s longer history of never selling companies he acquires, and could also be a signal to the next era of Berkshire management post-Buffett that some philosophies are allowed to change.

“They are doing IPOs and selling operating companies, which they wouldn’t do in the past, and making investments in international stocks, and willing to buy back shares at an accelerated pace. Quite a bit has changed,” Shanahan said.

For first time, Berkshire also invested in gold — a commodity Buffett has long criticized — though it bought gold mining stock Barrick Gold rather than the metal directly. Nevertheless, the market saw the move as a significant reversal as gold miners are levered to the underlying price of the metal.

“It looks like they are building a little bit of a hedge there,” says Womack. “I wouldn’t be surprised if they add to that position.”

The big changes to the stock portfolio may hold one more big surprise in 2021. Berkshire recently disclosed that confidential information had been omitted from its 13F report, which details the latest stock buying and selling. In the past, this has meant that Berkshire was building a large position in a public company and wanted regulatory permission to complete their purchases prior to making the information public.

Shanahan noted that the value of the equity portfolio was $245 billion as of Sept. 30, 2020, excluding Kraft Heinz, and he could only account for roughly $220 billion. Some of that difference could be related to more shares in the Japanese trading firms being purchased, but this is a strategy Berkshire has used in the past when building a big position in a company, such as IBM, and there are other options even in a pricey U.S. market, such as in the depressed energy sector, though Berkshire has been a seller of some energy stocks in recent history.

Berkshire’s existing operating companies have potential in the near-term without a major new position. Berkshire Energy, in particular, should have faster growth than many other subsidiaries as electricity demand grows, and many of the other wholly owned businesses negatively influenced by Covid, from reduced shipping to higher insurance claims and restaurant shutdowns and retail store closures, hurting Brooks and other retail brands under the Berkshire umbrella, could see improved earnings in 2021, even though Shanahan said he thought insurer Geico and railroad Burlington Northern held up well in 2020.

Buffett may yet get his opportunity to make some major moves with his company’s cash. “We will still see a lot of companies in financial trouble over the next few years when they go to refinance debt and they are already over-leveraged. Cash is still king and he knows it,” Womack said. ” We will see many more companies becoming even more attractive and I think that’s why Buffett is biding his time. The smart money is allowed to be cash heavy.”

Timing, of another kind, is an issue with investors today, especially newer ones, Womack said, and that has repercussions for the idea of an investment in Berkshire. “There are so many options for investors today for growth and tactical management and staying exposed to the upside …. and we are in tactical environment. Investors want to be tactical rather than more passive, so maybe a new investor won’t sit on a BRK like they used to.”

In 1999, the performance gap between Berkshire and the S&P 500 reached close to 40%, but in hindsight, that was a buy signal. After 1999, Berkshire had one of its best stretches in recent history against the S&P 500, gaining approximately 30% during a period of years when the index was down by close to 40%.

“Back to 2000, it beats the S&P by a lot,” Womack said. “If the S&P goes down substantially, Berkshire could do better.”

Barron’s noted in a recent investment call on Berkshire, “It has paid to buy Berkshire after prior periods of underperformance.”

That was a bet at least one big investor made in 2020: Warren Buffett.