Methods

How the numbers are made

Anyone who needs a defensible number can use this app — a reporter on deadline, a student writing a paper, a researcher checking a hypothesis, or a policy analyst briefing an audience. Every estimate has to survive scrutiny, whatever the setting. Here is exactly what happens between your question and the figure on screen.

From your question to the answer

Ask an ordinary question in plain language. You don't need to name a dataset, a variable or a statistical method — IDK handles that for you:

  1. Identifies the relevant dataset and the exact survey questions and answer categories.
  2. Searches scholarly sources such as OpenAlex for methods suited to that kind of question.
  3. Selects a compatible statistical approach from an approved list and checks it fits the data, weights and survey design.
  4. Runs the analysis in R on the source data.
  5. Returns the answer with short notes on the method, weighting, sources and uncertainty.

Finding a scholarly source supports the choice of method but does not, by itself, prove the method is statistically valid. Each method's validation stage is shown with the result.

Every number comes from the data, not the AI

The AI in this app does one job: it reads your question and routes it to the right survey variable. It never produces a number itself. Every estimate you see is calculated by an R server using the actual microdata from the survey's publisher.

Survey weights

Estimates use the weights the survey publishers provide so results represent the U.S. population, not just the people who happened to answer. For the General Social Survey that means the NORC weight (wtssnr for 2004 onward, wtssall for earlier years). For the Current Population Survey ASEC, the person-level weight from Census.gov public-use files. The one exception is the NLSY97 pilot, which is explicitly unweighted and labeled as such.

Limited pilots stay limited

The NLSY97 pilot covers the 1997 youth cohort and runs a single fixed, unweighted descriptive comparison of job-income reporting by sex. It returns aggregate percentages, 95% confidence intervals, and valid sample sizes only — never microdata — and it is not a general-purpose or nationally representative analysis tool.

Uncertainty intervals

Each estimate ships with a 95% confidence interval computed with the survey package in R. If a subgroup is too small to support a reliable estimate, the server refuses to produce one and tells you why — a narrow answer is better than a wrong one.

Full source details

Every result includes the exact question wording, the variable name, the years covered, the weight used, the unweighted sample size, the dataset release, a codebook citation, and the reproducible R code that produced the numbers. A fact-checker can rerun every figure independently.

Refusals are honest

If a question maps to no known variable, a variable wasn't asked in the years you picked, or the sample is too small, the app says so plainly and asks you to revise. It will never fill the gap with an invented number.

One engine for people and developers

The Ask page and the developer API preview use the same controlled research engine. Language models help interpret questions and explain results, but numerical findings come only from approved statistical computation on the source data — never from generated text.

Scholarly sources

OpenAlex

OpenAlex is an open catalogue of scholarly publications, authors and research institutions. IDK uses OpenAlex to discover scholarly publications and connect statistical methods to their sources.

Results include source provenance and retrieval time. Discovery is not validation: an OpenAlex match does not by itself validate a statistical method, confirm a paper’s claims, or change a method’s review status. Statistical findings still come from the existing R analysis service.

About OpenAlex

Checking scholarly discovery availability…

Ask a question and see the source details for yourself.