3Experimental Design
The experiment has two arms. In the main arm I pose three financial questions to thirteen frontier LLMs for twenty-one countries. I vary the language of each question, the country it refers to, and the local financial context it provides. In the second arm, I vary the investor rather than the country. I refer to this arm as the stated-investor arm. Specifically, I use the portfolio question with amended statements on the investor’s risk tolerance, pension generosity, investment horizon, currency hedging and other balance-sheet items in an otherwise fixed English prompt. I describe this arm in more detail when I discuss the associated results in Section 6.
This section instead focuses on the first arm, and describes the three questions and the country selection, the prompt versions and the contrasts they identify, the models and API parameters, and the data collection. Appendix Table A1 lists the prompt language, the local-currency investment amount, and the full Version C vocabulary for every country, and Appendix A documents the technical implementation of the data collection.
3.1The Three Questions
Each LLM answers three financial questions: a primary portfolio-allocation question and two secondary questions on mortgage prepayment and inflation protection. Question 1 asks the LLM to split the local-currency equivalent of $50,000 between a broad stock-market index fund and government bonds over a five-year horizon. The outcome is the fraction allocated to the stock-market index fund. Question 2 asks whether to invest savings in a stock-market index fund or to use them to pay down an existing 4 percent mortgage with twenty years remaining. The outcome is the fraction allocated to the index fund. This question has a sharper normative benchmark than Question 1, because mortgage prepayment is a risk-free investment with a return equal to the contract rate. Using a 4 percent rate also allows me to test whether the models condition on a reasonable rate for each country. I use a pre-specified rule that assigns any country whose prevailing mortgage rate exceeds twice the prompted 4 percent to a premise-acceptance group. In these countries the question also tests whether the models flag the 4 percent rate as unusual for the local environment or simply condition on it. In the end, all models conditioned on it.
Question 3 asks how to protect the real value of $50,000 in a savings account that earns a nominal 1 percent interest rate when annual inflation is 3 percent. Similar to Question 2, the explicit rate in the question allows me to understand whether models push back against an unrealistic rate. The outcome has two parts: a numeric fraction to invest and a categorical primary instrument (equities, inflation-linked bonds, real estate, gold, or other). Because the stated 3 percent inflation rate is far below prevailing inflation in some countries, Question 3 was ex ante the question most likely to lead the models to reject the premise. I use a pre-specified contingency that promotes the refusal indicator to the primary Question 3 outcome if the refusal rate exceeds 10 percent in any country–model cell. However, the refusal rate was zero across 37,021 model calls in 273 country \(\times\) model cells and the contingency never applied. While several models noted that the inflation rate was unrealistically low for the country, most often for Turkey, none refused to answer.
3.2Country Selection
I chose the country set to span meaningful cross-country variation in pension generosity, equity-market development, inflation history, and measured preferences, while keeping translation risk low. The preference regressions of Section 5.5 require each country’s measured risk preferences, so the Global Preferences Survey (Falk et al. 2018) must cover every country. This requirement excludes otherwise natural candidates such as Denmark.
The United States serves as the baseline for the country contrasts. I prompt five further countries in English (Australia, Canada, India, South Africa, and the United Kingdom). They add observations at essentially no translation risk while spanning very different pension institutions, from compulsory superannuation to a replacement rate among the lowest in the panel.
The fifteen non-English countries include high-literacy European systems with different occupational pension schemes (Sweden, Germany, the Netherlands, Switzerland) and further European variation (France, Spain, Italy, Poland, Finland). Outside Europe they cover the Asian markets (China, Japan, South Korea), several high-inflation economies (Brazil, Turkey), and Mexico.
3.3Prompt Structure and Version Design
Each prompt has two blocks. An Instructions block, written in English in every cell, asks for a response that contains the recommendation and a word-limited explanation. Keeping the instructions in English holds response-format compliance constant across languages. A Question block describes the investor and poses the financial question. The main treatments change only the Question block, but a robustness check in Section 5.6 that re-collects the primary cell with the instructions translated as well shows that this has only a limited impact on the results.
The Question block varies along three dimensions. The first dimension is the country and its prompt language. I prompt six countries in English and the fifteen others in their local language, with Switzerland in German and Mexico in Spanish (Appendix Table A1 lists each pairing). The second dimension of the prompt is the investor’s stated age (30, 40, or 55), which varies for the primary equity-share question and is fixed at 40 for the two secondary questions. The third dimension is the prompt version. I use three versions of the prompt, described in detail below: the first version uses the local language with little or no local context, the second version uses English but states the investor location, and the third includes local context in the form of pension and fund names. I use these different prompts to decompose the country effect into language, country-naming, and local-framing components.3 The version lettering follows the pre-registration. Version A denoted a translated prompt that named no country. I dropped it before collection because it could not separate the effect of the question’s language from the style of any particular translation, and because the comparison of Versions B and B-cross identifies the language effect more cleanly by holding the country reference fixed. I keep the labels of the surviving versions unchanged so that the paper matches the registered design.
More specifically, Version B states “I live in [country]” but uses a generic instrument vocabulary (“a broad stock market index fund,” “government bonds,” “a standard pension plan”). In the fifteen non-English countries it appears in the local language. Its investment amount is the round local-currency equivalent of $50,000 (e.g. 500,000 kronor in Sweden, 45,000 euros in Germany, 40,000 pounds in the United Kingdom), written in the native local number format (“500 000 kr,” “45.000 Euro”). Version B-cross retains the country reference (“I live in [country]”) and the local-currency investment amount of Version B but reverts the question to English. It is collected only for the fifteen non-English countries. Note that the baseline prompts do not adjust for PPP or income. In a robustness check I also adjust the stake by PPP, which does not alter the result.
Version C keeps the language, country reference, and investment amount of Version B but replaces the generic financial vocabulary with country-specific equivalents. Specifically, I use local index funds (e.g. an index fund via Avanza or Nordnet in Sweden, eMAXIS Slim All-Country Equity in Japan), local government bonds (Svenska statsobligationer, JGBs, Tesouro Direto), and local pension references (tjänstepension, bAV, PER, iDeCo).
The different versions allow me to identify different components of the country effect. The cross-country comparison of Version B identifies the joint country-and-language effect relative to the U.S. baseline. The within-country comparison of Versions B-cross and B holds the country reference fixed and changes only the language of the question, which identifies the language effect. The within-country comparison of Versions C and B holds both country and language fixed and changes only whether the prompt names local instruments and pensions, which identifies the local-framing effect.
The translations were first done by an automated pipeline that translates only the Question block, followed by a native-speaker review of each version. Before I filed the pre-registration, every assembled prompt passed a consistency audit with zero failures, an adversarial LLM-based review covered all ninety unique translated blocks, and I read every block in full myself with the help of Google Translate. I then froze the translations for collection. Appendix A.4 documents the pipeline mechanics, the automated checks, and the review-and-correction protocol.
3.4Models and Data Collection
The experiment queries thirteen frontier LLMs from six vendors through the OpenRouter unified API. Four vendors are U.S.-headquartered: Anthropic (Claude Haiku 4.5, Claude Sonnet 4.6, Claude Opus 4.6, Claude Opus 4.8), OpenAI (GPT-5-mini, GPT-5.5), Google (Gemini 3 Flash, Gemini 3.5 Flash, Gemini 3.1 Pro), and xAI (Grok 4.3). To make sure that the panel does not consist solely of U.S.-headquartered firms, I add DeepSeek (DeepSeek V4 Flash, DeepSeek V4 Pro), a non-U.S. vendor whose models are primarily trained on Chinese-language data, and Mistral (Mistral Medium 3.5), a European (French) vendor. These six firms account for roughly 98 percent of web visits among the leading standalone chatbots (Einberger 2026), and about 85 to 94 percent of AI-assistant app users (Sensor Tower 2026). OpenAI, Google, and Anthropic account for 56.6, 24.7, and 11.7 percent of web visits, respectively.4 The numbers in Einberger (2026) are based on Similarweb estimates of desktop and mobile web visits. Shares are of combined visits to seven standalone assistants (ChatGPT, Gemini, Claude, DeepSeek, Grok, Perplexity, Microsoft Copilot). Country shares refer to September 2026, worldwide and U.S. shares to October 2026. The combined six-vendor shares are my calculations. Mistral is not included in the source data. Similarweb figures are model-based estimates. See Jansen et al. (2022) on the accuracy of Similarweb relative to site-side analytics. Note that the denominator does not include embedded assistants such as Meta AI or Chinese domestic platforms.
A cell is one combination of country, version, question, age, and model. I query each cell fifty times at temperature 0.7, send no system prompt, and cap the response at 4,000 tokens. Three models, GPT-5-mini, GPT-5.5 and Claude Opus 4.8, do not accept a temperature setting, and the interface drops the parameter without an error, so these models ran at their providers’ default sampling. The setting is the same for every query to a given model, so the model fixed effects absorb any difference it makes to the level of advice.5 The pre-registration provides for a model that rejects a custom temperature, provided the substituted setting is documented and held constant across all queries to that model. I identify the three models from OpenRouter’s listing of the parameters each model endpoint accepts, and the temperature check of Section 5.6 confirms it, since the dispersion of their answers does not rise between temperatures of 0 and 1.0 (Appendix Table A15). Fifty repetitions are enough because the unit of inference is the model. Additional repetitions would reduce only the within-cell sampling term, which is already small relative to between-model heterogeneity. In total the design comprises 267,150 queries, of which 185,250 form the country/language arm and 81,900 the stated-investor arm. Appendix Table A2 gives the accounting by arm, version, and question.
I ran an exploratory pilot in late June 2026 and filed the pre-registration at the Open Science Framework (OSF) on July 2, 2026, with the prompts frozen. The main collection ran on July 2–4, beginning a few hours after the filing. Two pre-registered robustness cells, temperature bookends at 0 and 1.0 and a Version B variant that states the investment amount in U.S. dollars, added 8,450 queries on July 6. Every cell reached its fifty repetitions, with one observation per query. The parser recovered a numeric recommendation from 99.9 percent of responses, and no model refused to answer. I exclude 237 unparsed responses and, under the pre-registered truncation rule, a further 7 that exhausted the 4,000-token budget, which leaves 266,906 responses. Appendix A.5 documents the parser, the refusal rule, and the model-identity checks. Appendix Table A3 lists the dated version of each model, and Appendix A.1 reproduces the prompts.
After the main collection I added three rounds of exploratory collection, none of which is part of the pre-registration. In August 2026 I collected the balance-sheet probes of the stated-investor arm (Section 6.3). In September I ran a model-tracking wave with a panel of twenty-one models. In October I re-collected the primary cell three times, with the Instructions block translated, with the model allowed to ask questions before advising, and with the stake restated at purchasing-power parity. Section 5.6 reports the tracking wave and the three re-collections.