International guidance treats transparency as the first condition of legitimate lobbying. The OECD recommendation of 2010 asks countries to provide enough information about lobbying for the public to scrutinise it (OECD, 2010), and the organisation's later stocktake found that many members still lacked the means to do so (OECD, 2021). Comparative research has responded by scoring the stringency of lobbying laws country by country (Chari et al., 2010). Indices of that kind measure rules. They do not measure what lobbyists in fact disclose, and they are silent where no lobbying law exists.

Consultation records fill part of that gap. Notice-and-comment rulemaking in the United States, the European Commission's public consultations and similar procedures elsewhere generate large, public collections of submissions from firms, trade associations, professional bodies, civil society groups and individuals. Political scientists have used these texts to study influence, by comparing what submitters asked for with what the final rule contained (Yackee and Yackee, 2006; Klüver, 2013). They have been used far less to study disclosure. Yet a submission either says whom the author represents or it does not, and work on corporate funding of non-profit commenters shows that undeclared ties are of real consequence (Bertrand et al., 2021).

This article proposes a procedure for constructing a text-based index of lobbying transparency from consultation submissions. It is written for researchers and for integrity bodies that hold such records. The procedure is described and justified step by step; it is not applied, and the article reports no results.

Text analysis in the social sciences rests on a small number of principles that shape the design. Grimmer and Stewart (2013) state the most important: automated methods are models of language that are always wrong in some respect, they augment careful reading and do not replace it, and every application must be validated for its own purpose. The methods fall into two families. Dictionary methods count words or phrases chosen in advance; they are transparent, and they fail when terms are used in a domain other than the one the dictionary was built for, as Loughran and McDonald (2011) showed for financial text. Supervised methods learn from documents that people have coded and extend the coding to the rest (Gentzkow et al., 2019).

Two precedents show that an index can be built from text. Baker et al. (2016) constructed an index of economic policy uncertainty from newspaper articles and validated it with a large human audit. In interest group research, Klüver (2009) tested quantitative text methods on consultation submissions against hand coding. The literature on composite indicators adds a separate set of requirements: a stated theoretical framework, defensible normalisation and weighting, and analysis of how sensitive the result is to those choices (OECD and JRC, 2008).

The procedure has seven steps, shown in Figure 1. They are described in order, though in practice the fourth and seventh send the researcher back to the first.

The procedure can be scaled to the resources available. A single researcher studying one consultation with a few hundred submissions may code everything by hand and omit the fifth step. The automated stage earns its cost only when the corpus is too large to read, and it should not be adopted for its own sake. A team working across many consultations will need a second coder throughout and a modest computing budget for model training.

Jurisdictions differ in ways that affect comparability. Some portals oblige respondents to choose a category and to state a register number, which makes disclosure on those dimensions a matter of compliance and not of choice. Others accept free text with no structure. An index built in one setting will not transfer unchanged to another, and comparisons across jurisdictions should be confined to the dimensions on which the portals leave respondents equal freedom. Where submissions from private individuals are included, names should be removed from any released data.

The procedure is proposed and has not been tested. Whether coders can reach acceptable agreement on the two ordinal dimensions, whether a classifier can reproduce their judgements, and whether the index correlates with independent measures are open questions, and a negative answer to any of them would require the design to change.

Some limits are built in. The index measures disclosure and not truthfulness: a submission that names a funder falsely scores as well as one that names it honestly. It observes only formal, written lobbying through one channel, and organisations that lobby mainly through meetings will be invisible to it. Published consultations over-represent actors with the resources to respond, so averages describe those who took part and not the population of interests. Finally, once an index is known to be in use, submitters may adapt their wording to it, which is welcome if the disclosure is real and a problem if it is formulaic. A measure of this kind should be read alongside register data and case knowledge, never in place of them.

Consultation submissions are an abundant and underused record of lobbying, and the tools needed to read them at scale are now ordinary. The procedure proposed here ties those tools to a defined construct, a written codebook, measured coder agreement and explicit validation, so that the resulting number can be defended and reproduced. It would let researchers compare disclosure across types of organisation and over time in settings where no lobbying register exists. Whether it works as intended must be established by applying it, and the article is offered as an invitation to do so.

To be declared by the author(s) before publication.

To be declared by the author(s) before publication.