The USDA just released its first look at the 2019/2020 balance sheets of grains and oilseeds in the latest WASDE report. The WASDE is probably the most widely followed report in the industry. The USDA has tremendous resources and puts a lot of effort into presenting a complete global supply and demand balance for 63 of the most important agricultural commodities. Although markets trust the USDA for US crop data, many other sources of data can add further value. Even the USDA relies on multiple foreign governments and leading industry groups.
Analysts experience great difficulty accurately measuring each of the balance sheet line items even well after harvest. Ideally, multiple sources for each commodity or country would be available for comparison. For example, in Brazil, crop production history is available from CONAB, a respected national supply company that is linked with the Ministry of Agriculture, and the Brazilian Institute of Geography and Statistics (IBGE). It’s often interesting, and illuminating, to cross reference a national institute with the USDA to see if the USDA is adopting that source’s data. Comparing data can bring attention to significant differences between the sources, which can be used to better understand difficult metrics to forecast. For instance, analysts have trouble accurately defining consumption and therefore it might be more appropriately viewed as a rough estimate. Many sources lump in a ‘residual’ with the livestock feed component to act as a balancing item so that supply and demand add up to the change in ending stocks. In two recent instances of this, Russian wheat and Brazilian soybeans this season, exports exceeded expectations by so much that the feed and residual numbers from previous seasons had to be retroactively reduced to account for the additional availability. Revisions well after the fact highlight which data points lacked certainty earlier.
This chart shows monthly revisions to the USDA’s forecast of Brazil soybean exports and feed use. Exports were revised significantly higher as each new month of data came in above expectations. Feed use was revised lower, first in the current and previous season, and then over several prior seasons, in order to make the balance sheet work.
It is also very important to know which sources other people in the market are looking at. Building your own balance can be informative, but understanding the broad perception of the balance is essential for context in a dynamic market. Further, analysts should understand the timeliness and frequency of each of the sources as well as how often and when they revise previous data points. For example, Russia’s United Interdepartmental Information and Statistical System (EMISS), which is part of ROSSTAT, releases its official crop production results in late December, well after harvest is complete, altering many balance sheets including the USDA’s. Geographical granularity is another consideration. In China, national crop production data is forecasted and updated in the CASDE reports, but only the National Bureau of Statistics (NBS) provides province-level data and only after a long delay. Data sources can provide different benefits based on scope, timeliness, frequency, and geographic granularity and can combine in various ways depending on the analytical goal of the balance sheet.
Other techniques for tracking the accuracy of sources, making and modifying forecasts, and identifying unknowable parts of the balance will be covered in the examination of specific supply and demand line items.
Typically, traders focus on crop production in major exporting countries. It drives availability for domestic producers and importing countries. Ramifications of droughts or bumper crops can carry over for several seasons as stocks expand and contract across the supply chain.
A production forecast starts with a determination of area planted. Some countries, like the US and South Africa, issue reports on planting intentions based on farmer surveys. But, a number of factors like relative crop prices or spring weather can alter the actual area planted from intentions. South Africa suffered significant droughts in December and January that prevented corn planting. Lower planted area was updated in monthly revisions from South Africa’s Crop Estimates Committee. Unprecedented flooding currently menaces the 2019 US planting season, and the USDA will revise US area numbers in its June 30 crop acreage report.
Analysts have several techniques to model planted or harvested area. The soybean to corn price ratio is a common factor used in trying to predict farmer intentions in the US, because it can factor into the decision to rotate acreage between the two crops. In other areas with an absence of reliable reported data a simple linear regression modified with domestic prices could be a good starting point.
Gro Intelligence has developed an approach to create crop masks in areas without reliable acreage data. By combining satellite images, typical crop growing cycles, analogs to areas with ground truth data, and other published journal articles, Gro analysts have built successful models of planted area and used them to develop yield models for Argentina soybeans and India, Russia, and Ukraine wheat. Map Spatial Production Allocation Model (MapSPAM) data was a useful reference. The source compiles harvested area data globally at the district level once every five years across over 40 commodities.
Initial production forecasts will be based on projections of yield and planted acreage from historical data. Extrapolation of the positive yield trend observed in most crops is meant to approximate growth due to ongoing technological improvements in seeds, inputs, or farming techniques. But a linear trend projection on reliable data like US yield can be very sensitive to the length of the look-back period. For instance, the calculation of trend in US corn yields back to 1987, a record yield, compared to the same calculation performed on yields starting in 1988, a drought stricken year, would result in different slopes. Further, some analysts choose to remove major drought years under the premise that the extreme downside has an outsized effect on the projection of “normal” conditions. These and other differences in analytic technique mean that production projections vary widely before the growing season has begun. The markets see them as loose estimates.
Once the crops are planted and growing season has started, weather becomes the focus. During critical growing periods each new weather forecast model run can lead to an immediate price reaction in the physical and futures markets. Gro has developed several yield models and provided background and data for users to develop their own versions. But any yield model will have an associated error term due to unpredictable weather (and other) factors. Using several production or yield forecasts can help by defining a range of estimates. Sometimes the path of forecasts over the season provides insight that the current number alone might not. Brazil’s soybean crop in 2017 was revised higher by 10 million tonnes from February to June resulting in a 13% decline in futures prices. This information is frequently lost when looking back over the years at a historical series of production and yield (or any other line item for that matter). A source like the USDA revises its yield estimate several times each season, but the WASDE report only prints this month and last month’s numbers side by side.
The chart on the left shows historical revisions to the USDA PS&D’s forecast of Brazil soybean production. 2017 saw a drastic rise in production including revisions months after key growing weather was known, which pressured prices. The chart on the right shows Gro’s US corn yield model which is updated daily and shows the forecasting history.
Harvest time typically offers the first concrete data. Production forecast accuracy should increase. But in some crops and countries, the current forecast can jump around until the very end of farmer deliveries. The past two sugarcane crops in India are examples of late season crush statistics taking the market by surprise and resulting in sizable revisions to production forecasts. Similarly, cocoa arrivals in Côte d'Ivoire in 2017 resulted in upward revisions of more than 10% as beans surprisingly kept showing up at the ports and processors. In both cases weather based yield modelling did not detect the coming production increase. In India, a new variety of sugarcane greatly increased yield, representing a discontinuous change in trend. For Côte d’Ivoire, critical planted areas in conflict zones remained unsurveyed as fearful crop scouts steered clear of the violence. These are just a few illustrations of the inherent volatility of forecasts even with known growing weather. Analysts must maintain flexibility in their balance sheet assumptions by understanding when and how incoming data can lead to revisions.
Imports are the other key component of supply, particularly for those countries with minimal domestic production. An initial forecast of imports should be based on projections of domestic production based on harvested area and yield forecasts compared to local consumption needs. A number of countries have detailed customs-based trade databases organized by Harmonized Commodity Description and Coding System (HS code) on a monthly frequency, including virtually all of the South-East Asian nations. For countries with no readily available customs data, exports from the major origins can be used as an approximation. Further, export statistics from major origins preview imports that will arrive the following month. They can then be revised when the official import numbers are released. If exports from major origins are the only option, it is useful to cross reference the annual total combined exports, on a one month lag, to a data source such as the USDA PS&D that provides an annual import figure historically. If there is a significant gap between shipments from exporters like the US, Argentina, and Brazil and the annual total USDA-reported imports for the same destination, then data from another exporter may be needed. If none exists, then an average of the difference could be used as an approximation for the current season.
Given a history of monthly imports, a seasonal pattern can be established. For example, a five or ten year average of each month as a percentage of the annual total import for the marketing year. Monthly progress can be monitored against this projection once the marketing year has begun. Another way to monitor progress is the simple cumulative import comparison year-over-year added to the previous year’s total import figure.
In general, the consumption side of the balance is more opaque. Far less data exists measuring each component directly. Frequently some or all of the demand side will need to be calculated as an implied demand based on known production, trade, and stocks. Even in the case of the US, a residual line item is included to account for unknown loss.
Broadly speaking, consumption of commodities should be correlated to demographic factors like GDP or urban population. Price can be a significant factor, particularly if substitutes exist like in feed rations for livestock. However, local price data might be difficult to obtain and given the myriad of intertwined trade deals, import duties, tariffs, and freight rates, futures prices may not be an accurate approximation. Forecasting domestic consumption of an individual country with a simple linear trend or linear regression against GDP may be the best starting point. The IMF’s World Economic Outlook Database provides forecasts for GDP at the country level five years into the future. Doing some research on each country in question can help refine the forecast. USDA’s FAS issues extensive GAIN reports across many commodities and countries. These reports can identify trade protections or limitations of domestic processing capacity. For example, Japan has two soybean crushing plants that have been running close to capacity for a few years. In order to meet additional feed demand Japan needs to import soybean meal.
Many sources will combine line items on the demand side of the balance. The USDA’s PS&D combines food, seed, and industrial use as well as feed and residual. It is useful to try and break these up. Human consumption of various commodities tends to have a more stable growth rate year to year and a linear trend projection of staple foods should be a good approximate. Non-staple products may be more sensitive to GDP, for example, Brazil’s sugar consumption was reduced during the significant recession a few years ago.
The chart on the left shows total world domestic consumption of cereals such as corn, wheat, and rice. Consumption grows at a rather steady rate despite some fluctuations in GDP growth. The chart on the left shows how GDP is more closely correlated to sugar consumption in Brazil.
Feed demand, on the other hand, can be volatile year to year for a specific commodity. Taking an aggregate of grains and oilseed meal used for feed can smooth out some of the volatility and correlate better with demographic factors. Forecasts for feed use of each commodity can then be refined based on relative prices within the context of total feed demand growth. It should be noted that feed use is very difficult to measure. Frequently a residual is added to the feed line item from most sources, which acts as a catch-all for anything from loss in handling, spoilage, or unknown disappearance. US soybeans have a specific residual line in the WASDE, which is negative in some years, demonstrating the unpredictable nature of the item.
Major commodity producers will typically have additional categories of domestic consumption like processing or industrial use, exports, and seed use. If the commodity in question has a strong processing industry there will often be good data for that component of demand. Soybean crushing data is available on a monthly basis in the US, Argentina, Brazil, Canada, and China. For cocoa, regional reports on grinding are issued on a quarterly basis. In the case of oilseeds, there is minimal direct consumption, therefore it is essential to build the balance sheets of the products as well, namely vegetable oils and oilseed cake and meal. Over half of Brazil’s sugarcane is directed to ethanol production thus necessitating an ethanol balance to get a complete picture of the sugar industry.
Exports are a key component for understanding any commodity market. As discussed in the imports section above, there are customs databases from most major exporting countries which breakdown exports by destination on a monthly basis. This data can be used to monitor progress during the marketing year. Higher frequency weekly export sales from the USDA can be used in a pace model. Gro’s model uses a regression of total commitments versus final exports historically for a given week of the season. A similar technique can be used to forecast corn used for ethanol production based on weekly EIA statistics.
The chart on the left shows monthly corn exports from major origins provided by several different sources. The seasonal shift between northern and southern hemisphere exporters is evident. The chart on the right is the progression of total commitments for US corn sales historically, which are a good indicator of final exports.
Pre-season forecasts for most exporting countries can be approximated as a simple percentage of the production forecast. Looking at historical data, domestic consumption, whether it be processing in the form of soybean crush or direct consumption like corn, as well as exports tend to be a fairly stable percentage of production. The exception is when the country in question is suffering from a poor crop, but forecasts always assume normal weather. Deciding whether to use, for example, a five or ten year average of exports or consumption as a percent of production depends on the country in question. A shorter average may be more a better starting point, particularly if production has made significant increases recently and the country has become a bigger exporter.
The US has shifted into the role of marginal global exporter of corn, wheat, and soybeans, due to its extensive on-farm and commercial storage network. When importing countries exhaust supplies from their preferred origins, typically those closer geographically, then they will turn to the US. Forecasting the exports required from the US necessitates a global analysis based on the supply and demand projections outlined above. Each of the net importing countries are aggregated based on projections of demand that are not covered by domestic production. Then, based on the production forecasts of the net exporting countries outside the US, an estimate of total available supply can be calculated. The deficit remaining is the forecast for US exports required globally for the upcoming season. Southern hemisphere countries, due to their opposite seasons, have different marketing year definitions for agricultural products. Furthermore, Southern Hemisphere agriculture has boomed in the past forty years. As a result, it’s necessary to break the annual balance sheets into monthly or quarterly balances if at least some of the data is available at that frequency. Combining major exporters into a single sub-annual balance sheet gives a more complete picture of the supply and demand balance at any given point of the year and avoids increasingly damaging errors from adding up out-of-sync agricultural calendars.
When each of the supply and demand line items are considered and added up, the remainder is the ending stocks line item. Most simple price models rely on the ratio of ending stocks to annual demand (“stocks-to-use”) as a predictor variable. However, forecasts of ending stocks suffer from compounded volatility in both supply and demand. Further, accurate stock data may not be available for most countries or commodities. Commercial enterprises only provide partial stock data. In sugar, for example, various consultants have such disparate measures of stocks globally that most analysts simply leave out the absolute number in favor of a surplus or deficit for the season. Public and private estimates for China’s corn stocks range anywhere from 75 to 200 million tonnes, hardly reliable information. The examination of historical domestic prices to identify particularly high priced environments can provide some insight. Tight stocks can be assumed at that point and thus we can establish a starting point. From there, we calculate the change in stocks in the succeeding years based on more transparent production and trade figures along with estimated demand.
This table shows the balance sheet component on the Gro web app. Historical data is provided by USDA PS&D along with the current Gro Yield Model forecast in red. The far right column allows users an interactive option to modify the numbers as desired.
Balance sheets that are well constructed are available publicly, but they all have assumptions and analytical decisions built in that aren’t necessarily clear. Identifying appropriate sources, formulating forecasts, and maintaining supply and demand balances can greatly enhance perspective and insight into commodity markets. Therefore, serious analysts should construct their own.
Unfortunately, getting the data and performing the analysis presents great difficulty due to the fragmentation and wildly inconsistent formats of the various sources. Gro has gathered and organized an enormous array of the necessary data from all over the world. We’ve finished and openly explained models built from data on the Gro platform. Gro’s data and analyses empower small teams of analysts to rapidly construct high quality, bespoke models on subjects of interest for all sectors of global agribusiness.