Over the past decade, macroeconomists have begun to pay more attention to the concept of economic or business uncertainty. The very concept of “uncertainty” began to be widely used in the economic context after the publication of F. Knight’s book “Risk, uncertainty, and profit” (Knight, 1921). Even though risk and uncertainty are often examined without clear distinctions, it can be important to recognize the conceptual differences (Cascaldi-Garcia et al., 2020). Risk, often interchangeably associated with volatility, is a statistical gauge of the variability in observed results. Uncertainty, on the other hand, is defined by both unknown outcomes and probability distributions.

Despite the difficulty of measuring uncertainty due to the lack of objective empirical measures, in modern macroeconomic literature one can increasingly find empirical studies of uncertainty using various proxy indicators. Popular methods for their construction at the moment include, but are not limited to:

One of the most popular sources of data for calculating uncertainty indicators is business tendency surveys. The key features of the information obtained in the course of business tendency surveys include the fact that it describes the perception of company managers and consumers of observed economic trends, its non-quantitative nature, representativeness and relatively large size of samples used, as well as the relative promptness of obtaining information compared with official quantitative statistics. For all questions concerning assessments of the dynamics (trends) of indicators, a three-category graduation is used: respondents choose between growth or improvement (+), no change (=) and decline or deterioration (–) (Lola and Bakeev, 2019).

Indicators based on data on entrepreneurial expectations obtained as part of business tendency monitoring have advantages over alternative measures. They are based on the knowledge of economic agents directly operating in the market and provide detailed information on many economic variables without analogues in quantitative statistics (Lola et al., 2024b). In ­addition, the international harmonization of business tendency monitoring makes it possible to compare survey results in different countries and use them in cross-country business cycle studies (Claveria, 2021a).

A technique for using business tendency survey data to indirectly measure uncertainty was first proposed by Bachmann et al. (2013); various modifications have subsequently been developed (see, e.g., Claveria, 2021b, 2021c; Girardi and Reuter, 2017). This approach has now become one of the leading ones in the empirical analysis of uncertainty (see, for example, Binding and Dibiasi, 2017; Girardi and Reuter, 2017; Meinen and Röhe, 2017; Mokinski et al., 2015). The methodology proposed by Bachmann et al. (2013) consists of defining uncertainty as the dispersion of forecasts made by economic agents regarding future events. Accordingly, during periods of high uncertainty there will be greater dispersion of forecasts about the future, whereas during periods of low uncertainty respondents will be more likely to agree in their assessments. We call indicators constructed using this methodology ex‑ante business uncertainty indices.

Subsequently, European Commission researchers developed alternative­ measures of uncertainty based on the same empirical basis (Girardi and Reuter, 2017). One is based on a measurement of respondents’ forecast errors by adjusting expected trends recorded in the previous month, quarter or year, taking­ into account current trends recorded in the current month, quarter or year. Uncertainty in this case is understood as a measure of the unpredictability of future economic trends. We call such indicators ex‑post business uncertainty indices. Their ­construction becomes possible only when we can assess the accuracy of reported expectations.

The purpose of this article is to test these approaches to measuring business uncertainty in the context of forecasting the dynamics of economic activity in Russia. For the Russian economy, business uncertainty is especially important due to the country’s dependence on the export of raw materials and geopolitical turbulence. In this sense, it is the current period, accompanied by unprecedented industry deformations, that represents a suitable information environment for the development of new methodological tools to measuring and analyzing business uncertainty in Russian statistical practice.

The goal of the study is achieved within the framework of the following tasks:

Indicators based on data from business tendency surveys were tested by the ­authors in previous studies, where a leading relationship was demonstrated between time series of the survey-based indicators and quantitative time series (Lola, Manukov, 2021; Lola et al., 2020; Lola, 2019). The methodology for ­constructing business uncertainty indices on Russian data has been already ­described in detail in the authors’ previous studies (Lola, 2023b; Lola et al., 2024a).

The paper uses observational data from the Russian Federal State Statistics Service (Rosstat) to measure business activity in mining and quarrying, manufacturing, retail trade, wholesale trade, construction, and services. The time series cover the period from 2009Q1 to 2024Q2 (mining and quarrying, manufacturing, retail trade, wholesale trade, construction) and from 2012Q1 to 2024Q2 (­services). The respondents are representatives of enterprises (executives or senior managers) who have the necessary level of competence on the questions asked in the questionnaire.

When filling out the questionnaires, respondents are asked, among other things, to assess trends in the development of such indicators of business activity as demand for manufactured products, volume of sales or services provided, sales prices of goods or services, the number of employees at the enterprise, economic situation, etc. Questions in the questionnaires may concern events over the past three months or year (current trends), as well as expectations for the next three months or years (expected trends). The main way to quantify such non-parametric data is the balance method (OECD, 2003). Balances are calculated as the share of respondents who gave a positive answer minus the share of those who answered negatively.

The paper describes the following indicators — ex‑ante and ex‑post business uncertainty indices: at the national level, in mining and quarrying, in manufacturing, in construction, in retail trade, in wholesale trade and in the service sector (Lola, 2023b). The methods used are primarily based on the experience of the European Commission in the field of uncertainty measurement (Girardi and Reuter, 2017).

First, let us consider the methodology for calculating ex‑ante indicators that operationalize uncertainty as a measure of the dispersion of respondents’ expectations regarding the future business environment.

1. The calculation of the cross-sectional deviations of positive and negative responses for each survey question considered in each quarter using the following formula:

σ=P+N−(P−N)2 (1)

where P is the proportion of positive answers to survey questions; N is the proportion of negative answers to survey questions.

2. The standardization and normalization of the constructed time series in order to avoid the predominance of questions with particularly pronounced dispersion in answers and/or high absolute average values when constructing an overall index accumulating the results for individual questions.

3. The construction of time series aggregated at the industry level as the average value for series of indicators related to individual questions.

4. The calculation of the national ex‑ante index based on a weighted aggregation of standardized values obtained within individual industries. When aggregating industry values, the percentage shares of industry subsamples in terms of the number of enterprises in the total population were used as weights.

5. To simplify interpretation, scaling indices so that the mean is 100 and the standard deviation is 10. Thus, values above 110 or below 90 should indicate an extremely high or low level of uncertainty respectively.

Accordingly, the methodology for constructing ex‑post indices was as follows.

1. The calculation of the cross-sectional deviations of positive and negative responses for each survey question considered in each quarter using formula (1).

2. Scaling the dispersion of expected trends recorded in the previous period using the dispersion of estimates of actual trends recorded in the current period, which makes it possible to highlight the dispersion of expectations provoked by uncertainty. For each indicator within each industry, logic ratios of the standard deviations of positive and negative expected assessments of entrepreneurs in period t – 1 to current assessments in period t were calculated. The following formula was used:

Ut=ln⁡(σt−1eσtc) (2)

where e and c indicate whether the estimate is an expected or current trend, and t indicates the point in time (quarter) in which the study was conducted.

3. The construction of time series based on indicators calculated using formula (2) related to individual questions in the questionnaire.

4. The standardization and normalization of the constructed time series in order to avoid the predominance of questions with particularly pronounced dispersion in answers and/or high absolute average values when constructing an overall index accumulating the results for individual questions.

5. The construction of time series aggregated at the industry level as the average value for series of indicators related to individual questions (when aggregating industry values, the percentage shares of industry subsamples in terms of the number of enterprises in the total population were used as weights).

This section presents the main methodological principles used to build the models, as well as the results of intermediate tests. Calculations were carried out using the EViews statistical software package.

Below is the applied algorithm for modeling the relationship between the effects of uncertainty on economic activity:

1. Testing time series for stationarity (Augmented Dickey–Fuller test).

2. Determination of the optimal order of lags based on the Akaike information criterion (AIC).

3. Test for lack of cointegration (Johansen test).

4. Granger causality analysis.

5. Pseudo-out-of-sample analysis.

The final stage of the study involved pseudo-out-of-sample analysis, which simulates out-of-sample analysis by artificially dividing the existing total sample into two subsamples (Inoue and Kilian, 2005). In this case, the first subsample is used to build a forecast, and the second is used to evaluate it by comparing the model results with real data.

As part of the pseudo-out-of-sample analysis, different models were built at the national level and within each industry under consideration — based on business uncertainty indices (1) ex‑ante and (2) ex‑post (Lola, 2023a).

For each of these econometric frameworks, fourteen models were built examining the following relationships between the variables. The selection of reference statistical series for the indicators was carried out in accordance with the recommendations of the European Commission (2023).

At the first stage, using the Augmented Dickey–Fuller test (ADF-test), the time series under study were tested for stationarity, which is an important condition for constructing adequate VAR models. The test results presented in Table 1 indicate that all the time series under study are stationary in first differences, so we can proceed to modeling.

The optimal numbers of model lags were selected based on AIC. The value “4” was taken as the maximum number of lags, in accordance with the standard practice of determining this parameter in existing empirical studies based on quarterly data (Lola et al., 2020). Models with the optimal number of lags are characterized by the lowest AIC value. Thus, in all the cases under consideration, models with 4 lags (p = 4) are optimal, with the exception of mining — in this industry, model with 1 lag turned out to be optimal.

The time series under study were also successfully tested for the absence of cointegration using the Johansen test.

To confirm the hypotheses about the presence of predictive power of business uncertainty indices in the dynamics of economic activity indicators, Granger causality tests were carried out. Almost in all cases, p-values were less than 0.1 (except of pair “Ex-post index in mining industry — industry’s GVA”), therefore, hypotheses about the presence of Granger cause of economic indicators from the constructed business uncertainty indices are accepted at the 10% significance level.

The following periods were specified for model evaluation (depending on the availability of quantitative data):

Forecasts were calculated for the period from 2024Q3 to 2025Q2 (four quarters).

To assess the effectiveness of the constructed models, the root mean squared forecast errors (RMSFEs) were calculated as a loss function for forecasting economic activity indicators based on business uncertainty indices. We also calculated RMSFE of reference autoregressive (AR) models based on quantitative referents, within which the forecast values of economic activity indicators are calculated based on their previous dynamics, without taking into account business uncertainty indices. Next, the RMSFE ratios between, on the one hand, VAR models based on business uncertainty indices and, on the other hand, reference­ AR models (rRMSFE) were calculated (Lola, Manukov, 2021):

rRMSFE=RMSFEVARRMSFEAR. (3)

Thus, in cases where these ratios take values less than one, models based on business uncertainty indices have higher predictive abilities of economic activity indicators than autoregressive models.

Table 2 shows the calculated rRMSFE ratios between VAR and AR models at the national level and for the six industries considered, representing the results of the pseudo-out-of-sample analysis. The results obtained indicate differences in the effectiveness of forecasting based on the constructed specifications of VAR models among the industries under consideration, depending on the methodology used for calculating business uncertainty indices:

Thus, within the framework of most of the industries under consideration (except for construction), as well as at the national level, VAR model specifications that were effective for forecasting real indicators of economic activity were built, characterized by lower forecast errors compared to standard autoregressive models (rRMSFE < 1). This interpretation was also confirmed by tests for the difference between forecast errors within each two separately compared models and is consistent with theoretical studies.

The following is a visualization of forecast estimates obtained on the basis of VAR and AR models constructed within the framework of pseudo-out-of-sample analysis.

The graph of the dynamics of Russia’s actual GDP (log) and its forecasts (Fig. 1), obtained using the constructed VAR models based on the national ex‑ante and ex‑post indices, as well as an autoregressive model, demonstrates the maximum accuracy of forecasting the dynamics of GDP precisely on the basis of the national ex‑post index, while forecasts constructed using the AR model and VAR model based on ex‑ante index are less accurate and generally have similar dynamics.

In turn, in manufacturing there is a similar picture with the dynamics at the national level — the most accurate forecast of the industry’s GVA, built using a VAR model based on ex‑post index, despite the fact that its advantage is not as pronounced as at the national level (Fig. 2).

In retail trade, despite the initial visual impression of the relative accuracy of industry turnover forecasts built on the basis of three models, the forecast obtained using a VAR model based on ex‑ante index should be considered the most accurate by a slight margin (Fig. 3).

In the construction industry, we can note that there is an insufficient effectiveness of industry GVA forecasts built on the basis of VAR models based on ex‑ante and ex‑post indices — the forecast obtained using an autoregressive model is more accurate to the real dynamics of the quantitative referent (Fig. 4).

Finally, in the service sector (Fig. 5), we want to highlight the most accurate forecast values of the industry’s GVA, built on the basis of a VAR model based on ex‑post index. Also, it is worth mentioning that all 3 models repeat the dynamics of the quantitative referent very well.

In addition, on the basis of the VAR model, which is based on the business uncertainty index, which has the best predictive abilities (ex‑post) in terms of forecasting the quantitative referent (GDP), forecasts for Q3–Q4 2024 and Q1–Q2 2025 were calculated at the national level. A graph with the obtained forecast values is presented in Fig. 6.

Assessing the predictive power of uncertainty indicators is important in determining their usefulness in informing economic policy decisions. Uncertainty can stem from various sources and affect the economy through various channels­. For example, an increase in the uncertainty perceived by economic agents can change their decisions regarding consumption, investment, hiring, etc. (Lautenbacher, 2021). Businesses must consider uncertainty when making­ decisions to invest in new technologies, as external factors beyond­ their control­ can fundamentally ­affect future sales and profits (Avalos et al., 2022). To ­account for this uncertainty, entrepreneurs carefully evaluate­ scenarios in which results could be worse or better­ than expected before making final decisions and take into account ­potential risks associated with these forecasts. In situations of high uncertainty, they may prefer to wait and see how the situation ­develops to avoid costly mistakes (Bernanke, 1983). However, such behavior can have negative consequences for the national economy in the form of ­reduced productivity, as waiting leads to delays and sometimes complete cancellation of projects. This ­issue is particularly relevant in developing­ countries, where insufficient investment and delays in technolo­gy adoption often hamper productivity growth.

In addition to the fact that the sentiments of real economic agents are recorded, there are several reasons to believe that business survey data in certain segments of the economy are best suited for measuring the level of business uncertainty. The so-called “wait-and-see” effect of uncertainty, resulting from adjustments to perceived risks in the use of capital or labor, operates primarily in the short term, making high-frequency data a better candidate for capturing these dynamics. At the same time, the use of an industry cross-section is based on the assumption that respondents themselves generate idiosyncratic shocks through their intentions and actions, so that aggregate fluctuations in opinion dispersion also include an element of fluctuation in perceived uncertainty, and not just objective compositional changes in the cross-section. Surveys with broader coverage allow for more precise testing of these compositional effects.

According to the results obtained, we can highlight that, firstly, indexes reflecting the degree of uncertainty among economic agencies can be used to determine the dynamics of business activity in Russia successfully and, secondly, at the national level, when forecasting GDP, clear preference should be given to the ex‑post business uncertainty indicator. It proved to be effective within the frameworks of VAR, and better at improving forecasts compared to the national ex‑ante business uncertainty indicator. It is worth noting that ex‑ante indices also have a conceptual problem in that when relying only on the spread of expectations, we can deal with heterogeneous, but quite confident expectations due to objective differences in the economic situation among enterprises. In the case of expectations, variance can be influenced by both uncertainty and heterogeneity of respondents and the information available to them.

The paper examined ex‑ante measures of business uncertainty, using estimates of the business community regarding future business trends to determine uncertainty as a measure of the spread between expressed opinions, and ex‑post measures, using assessments of both future and current trends, defining business uncertainty as the degree of deviation of entrepreneurial expectations from the real picture. Calculated business uncertainty indicators have great opportunities to describe the dynamics of economic activity in Russia. According to the empirical results obtained and due to the theoretical advantages of the approach, the ex‑post approach should be considered preferable for modeling based on business survey data. In terms of forecasting at the industry level, business uncertainty indices were particularly effective in construction, manufacturing and services, while for other industries the results were negative or weakly robust.

First of all, the proposed measures seem suitable within the framework of nowcasting forecasting, understood as prediction of the present, near future and recent past (Banbura et al., 2013). Nowcasting methods are relevant for those key macroeconomic aggregates that are published with a relatively low frequency (usually on a quarterly basis) and with a significant lag. Early estimates of these key economic indicators use relevant information, collected more frequently and published more quickly.