(AGENPARL) - Roma, 30 Ottobre 2024 - (AGENPARL) – mer 30 ottobre 2024 Central Bank Business Surveys: Version 2.0
Speech by Chiara Scotti
Deputy Governor of Banca d’Italia
14th Annual Conference on ‘Central Bank Business Survey and Liaison Programs’
Centro Convegni Carlo Azeglio Ciampi
Rome, 30 October 2024
Ladies and gentlemen, good afternoon.
The annual conference on ‘Central Bank Business Survey and Liaison Programs’ is now
coming to a close. Let me warmly thank all the participants and speakers who have
contributed to the success of this initiative, jointly organized by Banca d’Italia1 and
the European Central Bank. The large number of central banks on both sides of the
Atlantic participating in this conference reveals the importance of business surveys for
policymakers. The usefulness of firm-level surveys has become so clear that an increasing
number of central banks have added them to their analytical toolkits in recent years.
Indeed, we have many more central banks attending the CBBS conference today than a
few years ago, partly thanks to our colleagues at the Atlanta Fed who have pushed for
such a larger group.2 This year’s programme brought together insightful methodological
and applied papers on various aspects of business surveys.
In the remainder of my intervention, I would like to briefly discuss the advantages and
limitations of traditional business surveys, and then take advantage of having such an
incredible audience in front of me to nudge researchers, both from academic institutions
and from central banks, to explore new directions to develop what I defined in the title
as business surveys 2.0, to make sure we seize the opportunities presented by new
technologies and digitalization.
Some advantages and challenges of traditional surveys
Sample surveys have been a fundamental tool for capturing policy-relevant
heterogeneities among agents, especially firms. Banca d’Italia has a long tradition
With thanks to Marco Bottone, Elena Mattevi, Andrea Neri and Concetta Rondinelli for their valuable
contributions.
Central Bank Business Survey and Liaison Program Group – Federal Reserve Bank of Atlanta.
in this field. In the early 1960s, we launched our Survey on Household Income and
Wealth,3 and soon after, in 1972, our annual Survey of Industrial and Service Firms
(INVIND).4 Over time, we have continued to expand our toolbox: in 1999, we launched
the quarterly Survey on Inflation and Growth Expectations,5 which systematically
collects firms’ expectations of consumer price inflation at different time horizons, as
well as quantitative information on past and expected changes in their own selling
prices. More recently, in 2022, we started the biannual Short-Term Survey of Italian
Households,6 which tracks household economic conditions over the business cycle.
Business surveys remain crucial to this day for providing timely insights into key
analytical issues. As shown in many academic papers, they have allowed us to achieve
a deeper understanding of expectations7 and their relation with firm-level dynamics,8
of uncertainty9 and other outcomes10 that would have been otherwise unattainable.
Because they can provide timely, specific, and flexible information, business surveys
have proved to be particularly useful in recent years, which have been marked by
large unexpected shocks. During the COVID-19 pandemic, for example, our surveys
were crucial to understand the underlying heterogeneity that drove the surge in
bank borrowing and deposits.11 More recently, in a world of heightened geopolitical
fragmentation, they have been useful to understand the potential impact of supply
disruptions from high-risk countries on European regions, sectors, and firms, and, thanks
to the coordinated efforts of several central banks, they have allowed researchers to
conduct harmonized cross-country analyses.12
‘Survey on Household Income and Wealth’, Banca d’Italia, Statistics Series.
‘Survey of Industrial and Service Firms’, Banca d’Italia, Statistics Series.
‘Survey on Inflation and Growth Expectations’, Banca d’Italia, Statistics Series.
‘Survey of Italian Households. Short-term Outlook’ (only in Italian), Banca d’Italia, Statistics Series.
For example, C.F. Manski, ‘Measuring expectations’, Econometrica, 72, 5, pp. 1329-1376, 2004;
L. Bartiloro, M. Bottone and A. Rosolia, ‘The heterogeneity of the inflation expectations of Italian firms
along the business cycle’, International Journal of Central Banking, 15, 5, pp. 175-205, 2019.
For example, O. Coibion, Y. Gorodnichenko and T. Ropele, ‘Inflation expectations and firm decisions:
New causal evidence’, The Quarterly Journal of Economics, 135, 1, pp. 165-219, 2020; A. Rosolia, ‘Do
firms act on their inflation expectations? Another look at Italian firms’, Journal of Political Economy
Macroeconomics, 2, 4, 2024.
For instance, L. Guiso and G. Parigi, ‘Investment and demand uncertainty’, The Quarterly Journal of
Economics, 114, 1, pp. 185-227, 2019; D. Altig, J.M. Barrero, N. Bloom, S. Davis, B. Meyer and N. Parker,
‘Surveying business uncertainty’, Journal of Econometrics, 231, 1, pp. 282-303, 2022.
For instance, the interpretation of firms about the ECB’s inflation target, see M. Bottone, A. Tagliabracci
and G. Zevi, ‘Inflation expectations and the ECB’s perceived inflation objective: Novel evidence from
firm-level data’, Journal of Monetary Economics, 129, S15-S34, 2022.
M. Bottone, E. Mattevi, L. Modugno, M. Mongardini and A. Neri, ‘Indebitamento e liquidità delle
imprese nel 2020: evidenze su microdati di impresa’, (only in Italian), Banca d’Italia, Covid-19 Notes,
21 December 2021.
I. Balteanu, M. Bottone, A. Fernandez-Cerezo, D. Ioannou, A. Kutten, M. Mancini and R. Morris, ‘European
firms facing geopolitical risk: Evidence from recent Eurosystem surveys’, VoxEU, 18 May 2024; L. Panon,
L. Lebastard, M. Mancini, A. Borin, P. Caka, G. Cariola, D. Essers, E. Gentili, A. Linarello, T. Padellini,
F. Requena and J. Timini, ‘Inputs in Distress: Geoeconomic Fragmentation and Firms’ Sourcing’, Banca
d’Italia, Questioni di Economia e Finanza (Occasional Papers), 861, 2024.
However, conducting traditional business surveys based on sound methodologies such
as probability sampling is becoming increasingly challenging and costly for central
banks. Examples of key challenges include the rise in both the item non-response rate
? which has tripled to 15 per cent for investment plans since the inception of our
INVIND survey ? and the unit non-response rate, especially among firms that perceive
a high response burden.13 Moreover, the growing awareness and concerns about data
privacy and time constraints14 not only complicate the conduct of surveys, but also
pose serious risks to data quality and inevitably raise questions about how to use and
potentially adapt sample surveys in the future.
Where could we go next?
The digitalization of information systems, the increasing availability of administrative
data, big data, and non-probabilistic and online surveys15 (which may seem like quick,
easy, and inexpensive ways to collect data), as well as the rise of artificial intelligence
(AI), bring potential opportunities for the development of surveys that must be carefully
assessed. I see some promising directions. For example, we should take full advantage of
the growing availability of administrative and other unstructured data. Combining these
data with methodologically sound and carefully designed sample surveys can greatly
enhance the informative value of both. This integration can occur at various levels, such
as sample selection, weighting, and streamlined questionnaires. In addition, we should
harness the rapid advances in AI and machine learning techniques, which have potential
benefits at various stages of the survey process. Let me briefly highlight some avenues
for further exploration and research in the latter area.
Survey research focuses primarily on the wording of questions and answers in an attempt
to gather soft information that it is not readily available in more traditional data, a.k.a.
hard information. So it seems natural, at least to me, that a first way to improve traditional
surveys is to use large language models (LLMs). In the preparation phase of surveys, for
example, LLMs could help create questions, identify inconsistencies and suggest effective
response options. In data cleaning and management, LLMs could help detect inconsistent
responses and prevent low-quality entries. In the final stage of reporting results, LLMs
could help ensure accessible formats, whether in the form of summaries, visualizations,
presentations, or written text.
M. Bottone, L. Modugno and A. Neri, ‘Response burden and data quality in business’, Journal of Official
Statistics, 37, 4, pp. 811-836, 2022.
For example, M. Galesic and M. Bosnjak, ‘Effects of questionnaire length on participation and indicators
of response quality in a web survey’, The Public Opinion Quarterly, 73, 2, pp. 349-360, 2009.
R. Gambacorta, M. Lo Conte, M. Murgia, A. Neri, R. Rizzi and F. Zanichelli, ‘Mind the mode: lessons
from a web survey on household finances’, Banca d’Italia, Questioni di Economia e Finanza (Occasional
Papers), 437, 2018; A. Neri and F. Zanichelli, ‘Principali risultati dell’Indagine Straordinaria sulle Famiglie
italiane nel 2020’ (only in Italian), Banca d’Italia, Covid-19 Notes, 26 June 2020; R. Cummings and
E. Tedeschi, ‘The effect of online interviews on the University of Michigan Survey of Consumer
Sentiment’, Briefing Books, October, 2024.
A second way to improve traditional surveys is to use natural language processing and
text analysis to quickly extract signals from text. These methods are already widely used
in the literature to extract signals on economic variables, such as industrial production
from text analysis of survey responses from manufacturing firms,16 or economic activity
from earnings calls,17 which are an important channel of communication between
market participants and the management of publicly traded companies. In light of
these experiences, we might further explore the possibility of extracting information
from informal interviews with managers that are traditionally more difficult to interpret,
moving away from traditional surveys in which respondents read and select the most
appropriate answer.
Third, even in the conduct of more formal interviews, surveys could be facilitated by the
use of AI-assisted interviewing, which integrates advanced artificial intelligence into the
interviewing process, particularly as a tool to assist interviewers in improving data quality
(minimizing human error and bias). Such a development would enable more dynamic
and responsive interviews that adapt in real time to respondents’ answers, automating
routine tasks, providing a more engaging and personalized interaction, possibly leading
to higher response rates and greater satisfaction with participation.
Fourth, machine learning models could replace current methods for imputing missing
data in business surveys, as they are able to capture the complexity in data structures,
recognizing non-linear relationships and allowing a large number of predictors to be
included.18 Machine learning techniques could also help to detect outliers in surveys, with
algorithms that compare responses within the same questionnaire and across waves.
Last but not least, LLMs could revolutionize traditional data collection methods. For
example, instead of sending out questionnaires, we could provide a programme that
firms could adapt to their own databases using LLMs, allowing them to extract the hard
information they need, leaving more time for qualitative or soft information, such as plans
and assessments. Clearly, the significant benefits of reducing the burden on respondents
would have to be carefully weighed against the potential problems of data confidentiality
that might discourage firms from participating in the survey.
These are just some examples of the potential evolution of business surveys. Understanding
whether and how new technologies and tools could be integrated into survey production
processes will surely be a challenge, but one that I think is worth taking on.
T. Cajner, L.D. Crane, C. Kurz, N. Morin, P.E. Soto and B. Vrankovich, ‘Manufacturing sentiment:
Forecasting industrial production with text analysis’, FEDS Finance and Economics Discussion Series,
Working Paper No. 2024-26, May 2024.
M.A. Gosselin and T. Taskin, ‘What Can Earnings Calls Tell Us About the Output Gap and Inflation
in Canada?’, Bank of Canada, Staff Discussion Paper, 29 June 2023; T. Taskin and F.U. Ruch, ‘Global
Demand and Supply Sentiment: Evidence from Earnings Calls’ Bank of Canada, Staff Working Paper,
30 June 2023.
In this sense, random forests seem to offer some improvements over traditional logistic regression
for the complex response model. See T.D. Buskirk and S. Kolenikov, ‘Finding respondents in the forest:
A comparison of logistic regression and random forest models for response propensity weighting and
stratification’, Survey Methods: Insights from the Field, 2015.
Conclusions
This conference has provided an excellent opportunity to exchange experiences on
methodological and applied issues. It has fostered scientific debate on core topics for
monetary and economic policy, facilitated dialogue between institutions, and explored
future avenues for research and innovation. Going forward, it is essential to build on the
momentum generated here.
In an increasingly interconnected world, strengthening our networks and increasing our
cooperation could improve the quality and reliability of business surveys and our ability
to identify emerging phenomena. Given the busy agenda of this two-day meeting and
the growing number of participants, we might consider following up on this event with
additional online meetings throughout the year, focusing on specific areas of interest,
such as survey design, data collection and data analysis, perhaps with an effort to include
the suggestions we have just discussed.
Individual institutions often lack the resources to tackle these challenges alone. By working
together, we can share lessons learned, best practices, and innovative methodologies to
advance the development of business surveys and ensure that they remain a valuable
part of our economic toolkits.
Designed by the Printing and Publishing Division of the Bank of Italy