The simplest measure is the share of votes on which two states make the same choice. Lijphart's index refines it by counting a pairing of an abstention with a yes or a no as half an agreement (Lijphart, 1963). Signorino and Ritter (1999) showed that rank correlation, then common in the field, confuses similarity with association, and proposed the S score, which is based on the distance between two states' positions scaled by the maximum possible distance. Häge (2011) pointed out that S and raw agreement are inflated when most states vote the same way on most resolutions, as they do in the General Assembly, and recommended chance-corrected coefficients of the kind used to assess agreement between coders.

A different family of methods estimates each state's position on an underlying dimension. Spatial models of roll-call voting were developed for legislatures (Poole and Rosenthal, 1985). Bailey et al. (2017) adapted the approach to the General Assembly with a dynamic model that uses resolutions repeated in identical form across sessions to make positions comparable over time, which dyadic measures cannot guarantee when the agenda changes. An overview of the data and of its uses is given by Voeten (2013).

Grimmer and Stewart (2013) set out the principles that govern automated content analysis in political science: all quantitative models of language are wrong, some are useful, and none can be trusted without validation. Methods fall into those that assign documents to known categories, with dictionaries or supervised learning, and those that discover categories, such as topic models. Scaling methods place texts on a dimension using reference documents (Laver et al., 2003) or word frequencies alone (Slapin and Proksch, 2008). The corpus of General Debate statements assembled by Baturo et al. (2017) made such methods usable for the United Nations and offers a source of state positions independent of votes. Open software has matured alongside: the quanteda package handles corpus management and document features (Benoit et al., 2018), and the stm package fits structural topic models (Roberts et al., 2019).

The analyst states which body is studied, over which sessions, and what the unit of comparison is: a pair of states in a session, a state relative to a reference state, or a state relative to a group. The rules of the body should be recorded at this point because they determine what a vote means. In the General Assembly each member has one vote, decisions on important questions need a two-thirds majority of members present and voting, and a member in arrears may lose its vote under Article 19 of the Charter.

Two things are gathered for each recorded vote: the choice of every member, and the full text of the resolution with its symbol, title, date and agenda item. For the General Assembly both are published in the United Nations Digital Library, and the roll-call dataset maintained by Voeten and colleagues can serve as a check on the vote records (Bailey et al., 2017). The symbol is the key that joins votes to text. Votes on separate paragraphs or amendments should be kept and flagged, since they are often more divisive than the vote on the whole.

Four outcomes are possible for a member on a recorded vote: yes, no, abstain and absent. The coding rule must say whether abstention is a middle category, as in Lijphart's index and the S score, or a soft no; and whether absence is treated as missing, which is the usual choice, or as informative where a state is known to stay away deliberately. A further rule is needed for the many resolutions adopted without a vote. Leaving them out overstates division, because recorded votes are requested on the contentious items; the general problem that roll calls are a selected sample of decisions was set out for legislatures by Carrubba et al. (2006). Including them as unanimous agreement pushes every similarity score towards its ceiling. The procedure asks the analyst to compute measures both ways and report the difference.

Topic classification is where the text tools enter. The resolution texts are loaded into a corpus, tokenised and reduced to a document-feature matrix. Preprocessing choices such as stemming and the removal of rare terms should be made deliberately and reported, since they can change the results of unsupervised models (Denny and Spirling, 2018). Two routes are then open. Where the categories are known in advance, for example the Middle East, disarmament, human rights and development, a dictionary or a supervised classifier trained on a hand-coded sample assigns each resolution. Where they are not, a topic model proposes groupings that the analyst labels after reading the most representative documents (Roberts et al., 2019).

Either way, a random sample is coded by hand by at least two readers and compared with the machine output, with agreement reported using a chance-corrected coefficient (Krippendorff, 2018). Resolutions repeated from year to year with little change should be identified by text similarity at this stage, as they are needed for any dynamic model.

No single measure is correct for every purpose, and the procedure asks for at least two. Table 1 compares the main options. A reasonable default is a chance-corrected dyadic coefficient for questions about pairs of states and an ideal point estimate for questions about change over time, each computed for all votes and separately within the topics from Step 4.

Validation follows the advice of Grimmer and Stewart (2013) that every automated measure be checked against something outside itself. Three checks are proposed. Where a published benchmark exists, as it does for the General Assembly, the new measure is compared with it over the overlapping period and large discrepancies are traced to their source. The vote-based measure is compared with positions scaled from speeches in the same sessions; the two should be related without being identical, and persistent gaps between what a state says and how it votes are worth examining in their own right. Finally, the highest and lowest scoring pairs in each topic are inspected by a reader who knows the cases. If any check fails, the analyst returns to the coding rules.

The final step is to publish the vote file, the topic codes, the scripts and a codebook stating every rule adopted in Steps 1 to 5 and the software versions used. Since the inputs are public documents, there is rarely a legal obstacle to release. A measure that others can rebuild and alter is worth more to the field than a marginally better one that cannot be inspected.