Scudder, Circuit Judge.
*963When a person applies for disability benefits, the Social Security Administration evaluates that person's capacity to work and, at the fifth and final step of the analysis, assesses whether significant numbers of jobs exist that someone with those abilities and limitations could perform. This determination is consequential: answering no means the claimant is disabled and entitled to supplemental income, whereas a yes answer results in a denial of benefits. At this final step, the agency bears the burden of showing that suitable jobs exist in significant numbers. The vocational expert enlisted by the agency to estimate the number of jobs suitable for Kelly Chavez offered two vastly different projections-testifying that for one particular job there were either 800 or 108,000 existing positions. The vocational expert preferred the larger estimate, and the administrative law judge who presided over Chavez's hearing agreed with that choice. In the end, the ALJ denied Chavez's claim for benefits, and the district court affirmed.
We vacate the ALJ's decision at step five. The decision was not supported by substantial evidence because the ALJ failed to ensure that the vocational expert's job estimates were reliable. To the contrary, the vocational expert offered no affirmative explanation for why his estimates (or the method that produced them) were reliable and instead reached that conclusion through a process of elimination-by determining that the estimates yielded by an alternative method seemed too low. By affording such broad deference to the vocational expert's chosen estimates, the ALJ relieved the agency of its evidentiary burden at the final step of the disability analysis and impermissibly shifted the burden to Chavez.
I
Kelly Chavez has severe impairments. In 2007, at the age of 21, she was diagnosed with a brain tumor and underwent five surgeries. Around this time, Chavez began feeling depressed and anxious. She struggled to maintain enough concentration to complete simple household tasks like loading a dishwasher. Chavez also suffered from migraine headaches, back pain (caused by degenerative disc disease ), and numbness in her feet and hands. Perhaps owing to becoming sick at such a young age, Chavez has no prior work experience.
In 2010 Chavez applied for supplemental security income, alleging that she had been disabled since 2007. Eventually she received a hearing before an ALJ. At step one of the five-step disability analysis delineated in 20 C.F.R. § 416.905(a), the ALJ found that Chavez had not worked since applying for benefits. At steps two and three, the ALJ concluded that Chavez had several severe impairments, but found that *964none matched or equaled the impairments listed in 20 C.F.R. Part 404, Subpart P, Appendix 1, that presumptively establish that a claimant is disabled. The ALJ then assessed Chavez's residual functional capacity or RFC-her ability to work on a sustained basis despite the limitations caused by her impairments, as required by 20 C.F.R. § 416.920(e) -and found that Chavez was quite limited. She could perform only simple, routine tasks with significant restrictions imposed on how much she could lift and carry. The ALJ further specified that Chavez could work only in an unchanging environment that neither proceeded at a fast pace nor required more than brief interactions with colleagues or the public. At step four, therefore, the ALJ determined that Chavez could perform a limited range of light work. None of these findings is at issue in this appeal.
The ALJ then proceeded to step five. Because Chavez had no past work experience, the question became whether she was able to do any work in light of her RFC, age, and education. See 20 C.F.R. § 416.960(c)(1). At this step, the agency bore the burden of demonstrating the existence of significant numbers of jobs in the national economy that Chavez could perform. See id. § 416.960(c)(2) ; McKinnie v. Barnhart , 368 F.3d 907, 911 (7th Cir. 2004) ("It is the Commissioner's burden at Step 5 to establish the existence of a significant number of jobs that the claimant can perform.").
Understanding how the agency generally approaches its burden at step five provides essential context for this appeal. The agency does not tally the number of job openings at a given time, but rather approximates the number of positions that exist, whether vacant or filled, and without regard to the location of the work and a claimant's likelihood of being hired. See 42 U.S.C. § 423(d)(2)(A) ; 20 C.F.R. § 416.966(a). In the same vein, the regulations direct that other factors of clear import to anyone pursuing employment, such as economic conditions or an employer's hiring practices, are not to affect step-five estimates of job numbers. See id. § 416.966(c). The design of these limitations is clear: they establish a framework for approximating the availability of suitable alternative work that the agency can apply across massive volumes of applications for disability benefits.
To obtain a job-number estimate at Chavez's hearing, the ALJ followed the common path of seeking the assistance of a vocational expert. See 20 C.F.R. § 416.966(e) (authorizing use of a VE and other specialists to aid with step five assessments). VEs tend to have master's degrees in vocational rehabilitation or psychology and often work in the field of job placement. The agency expects VEs to testify objectively and impartially about the exertional requirements of various jobs and their frequency in the national economy. See SOC. SEC. ADMIN., VOCATIONAL EXPERT HANDBOOK , 9-10 (Aug. 2017). Doing so requires a VE to be familiar with and draw from various sources of occupational information produced by the Department of Labor, Social Security Administration, Census Bureau, and state employment studies. Amy E. Vercelli, Consultation in Social Security Disability Law , in FOUNDATIONS OF FORENSIC VOCATIONAL REHABILITATION 311, 318-21 (Rick H. Robinson ed., 2014). The VE selected to assist the ALJ at Chavez's hearing had three decades of experience as a vocational consultant. The parties stipulated to the VE's qualifications as an expert, and those qualifications are not at issue here.
The VE testified that someone with Chavez's abilities, limitations, and impairments could perform "unskilled work" at a *965"light level," including, for example, working as a "bench assembler," "domestic laundry worker," or "hand packager." If these job titles sound obscure, that is a fair reaction, as they come from a 1977 publication of the Department of Labor known as the Dictionary of Occupational Titles , regularly abbreviated as the DOT. The Social Security Administration's regulations authorize the agency to "take administrative notice of reliable job information" from the DOT, among other publications. 20 C.F.R. § 416.966(d)(1). As a result, in cases like these, the DOT is a source that VEs regularly canvass to identify job titles suitable for a claimant.
The DOT divides jobs into groups and then lists and describes particular job titles within each group. The title "domestic laundry worker" (DOT 302.685-010), for example, is situated in group 302 ("Private Family Launderers"), which contains one other job title, "ironer" (DOT 302.687-010). U.S. DEPARTMENT OF LABOR, I DICTIONARY OF OCCUPATIONAL TITLES 302 (4th ed. 1991). Other DOT groups are much larger. For instance, a "bench assembler" (DOT 706.684-022), another job that the VE identified as suitable for Chavez, is listed as one of 59 job titles in the group "Metal Unit Assemblers and Adjusters, Not Elsewhere Classified." U.S. DEPARTMENT OF LABOR, II DICTIONARY OF OCCUPATIONAL TITLES 706 (4th ed. 1991). The third job the VE identified, DOT 920.687-122, which he called a "hand packager," is actually entitled "machine-pack assembler" and exists only in the artillery industry. Id. at 937. That job is one of 109 titles in group 920, "Packaging Occupations." Id. at 931-38.
The Department of Labor last revised the DOT in 1991. Recognizing the outdated nature of many of the DOT's job descriptions and titles, the Social Security Administration has been working (since 2008) on a new resource that better reflects the jobs that exist in today's economy. The agency has announced that it anticipates replacing the DOT with the Occupational Information System in 2020. SOC. SEC. ADMIN., OCCUPATIONAL INFORMATION SYSTEM PROJECT , www.ssa.gov/disabilityresearch/occupational_info_systems.html (last visited July 18, 2018). Courts including our own have invited this development. See, e.g., Dimmett v. Colvin , 816 F.3d 486, 489 (7th Cir. 2016) (encouraging agency to complete its efforts to replace DOT in light of its "obsolescence"); Browning v. Colvin , 766 F.3d 702, 709 (7th Cir. 2014) ("No doubt many of the jobs [in the DOT] have changed and some have disappeared. We have no idea how vocational experts and administrative law judges deal with this problem."); Purdy v. Berryhill , 887 F.3d 7, 14 n.10 (1st Cir. 2018) (recognizing the criticism leveled at the DOT and acknowledging that the agency plans to implement a replacement in 2020).
Beyond being outdated, the DOT's other significant limitation is that it describes only job duties and requirements, without also reporting an estimate of how many of those positions exist in the national economy. To determine the number of jobs, a VE must consult another resource. One commonly used is the Department of Labor's compilation of Occupational Employment Statistics. That publication does not use the DOT job grouping system, but instead relies upon another classification system, the Standard Occupational Classification (SOC).
The use of one system to supply the job titles and another to provide the number of jobs creates a matching problem: a one-to-one correlation does not exist. When a VE identifies an SOC code and the number of jobs in that code, that number approximates (at best) the number of positions within a DOT job group-not the specific DOT job title that the VE identified as *966suitable for a particular claimant. Vocational counselors have recognized that this crude data matching is highly inaccurate and thus are advised not to perform this analysis in other areas of their practice (when they are not testifying in a disability hearing). See Mary Barros-Bailey and Sylvia Karman, Occupational and Labor Market Information , in FOUNDATIONS OF FORENSIC VOCATIONAL REHABILITATION 203, 221-25, 232-33 (Rick H. Robinson ed., 2014).
Here the VE testified that the three jobs he had identified as suitable for Chavez existed in meaningful numbers. He estimated that across the country there were 108,000 bench assembler jobs, 306,000 domestic laundry worker jobs, and 57,000 hand packager jobs. These estimates drew a quick and repeated objection from Chavez's counsel, who asked the VE how he had arrived at the job figures. The VE stated that he applied what is known as the equal distribution method.
This question before us centers on the reliability of the equal distribution method. The method operates on the illogical assumption that all job titles within a particular DOT job group exist in equal numbers in the national economy. A few examples illustrate the point and expose the method's distorting effects. Consider a VE who determines that a particular claimant is able to perform one of the 24 jobs listed within the DOT under group 313, entitled "Chefs and Cooks, Hotels and Restaurants." U.S. DEPARTMENT OF LABOR, I DICTIONARY OF OCCUPATIONAL TITLES 313 (4th ed. 1991). Under the equal distribution method, the VE would assume that each DOT job title encompassed by the corresponding SOC code exists in equal numbers. But it does not take much knowledge of job markets to know that, while certain jobs may exist in large numbers (for example, a "pizza baker," DOT 313.381-014, who "prepares and bakes pizza pies"), others clearly do not (such as a "chef de froid," DOT 313.281-010, who designs "artistic food arrangements for buffets in formal restaurants" including "mold[ing] butter into artistic forms"). Or, by way of a second example, take "Cashiers and Tellers" (DOT group 211). Of the 28 positions included in that group, six exist only in the racing industry, with five of those six existing only at horse-racing tracks. Id. at 182-84. It seems unlikely that over 20% of all cashier and teller jobs in today's economy are at racetracks.
At the hearing, Chavez objected to the VE's job-number estimates. The ALJ then asked the VE to explain how he arrived at his projections. The VE responded by stating that he applied the equal distribution method because he prefers it over the occupational density method, which approximates job numbers utilizing a software program known as JobBrowser Pro. The VE then observed that the two methods can produce substantially different estimates, and he used the bench assembler position as an illustration. Under the occupational density method, the VE explained, the JobBrowser Pro software estimated that 800 bench assembler positions existed in the national economy, while the equal distribution method put the number at 108,000. When asked by the ALJ why he adopted the higher estimate, the VE answered only by offering that, "I find the information on occupation density information produce[d] in JobBrowser Pro, a SkillTRAN product, to the estimated numbers of jobs significant[ly] lower than I believe would be the numbers in the national economy." (A.R. at 688.)
Recognizing that the VE's response did not answer the question, the ALJ tried again, asking the VE to explain his decision to adopt an estimate of 108,000 bench assembler jobs. The VE then replied:
*967Oh, the example I gave for the unskilled, light production worker was [for the position of] bench assembler. I find that JobBrowser Pro's estimate, and it is an estimate, for that group of occupations [for] bench assembler is [a] little less than 800 in the national economy. I just think it's almost logical that there are more bench assemblers in the national economy than 800.
(A.R. at 689.)
The ALJ did not stop there, trying yet a third time to understand the basis for the VE's decision to choose the estimate from the equal distribution method over that from the JobBrowser Pro software:
Q: And you said that-or that you assume that there would be more than 800 of that particular kind of job in the national economy, and other than just a guess is there some other basis in addition to your expectation that informs that opinion? Have you done labor market surveys or other things that would indicate that that's-that those kind of numbers are too low or what is it that's telling you those-
A: I would have to say no that I have not done labor market surveys in order to get number counts.
(A.R. at 689-90.) When asked the same question a fourth time, the VE did not elaborate much:
I'm not sure that either one of the methods give a very accurate count on numbers. Numbers are always ranges and estimates no matter how they're counted. I consistently find that the numbers that I read in the Job Browser Pro estimate at the DOT level are much lower than what I normally find. And virtually when I look at down to the local level in a very small region, a metropolitan statistical area[,] I don't have the confidence with those [JobBrowser Pro] numbers are high enough to be a more accurate reflection of the numbers of jobs than that group of numbers that I have given [from the equal distribution method].
(A.R. at 690-91.) The colloquy concluded with the VE stating that his confidence in estimates generated from the equal distribution method was not rooted in surveys or job data, but rather more generally "based on [his] experience as well as consultation with other experts throughout the country." (A.R. at 691.)
The ALJ adopted the VE's testimony over Chavez's objection. In her written opinion, the ALJ explained that "[t]he vocational expert's opinion is accepted as it is not contradicted and in light of [the VE's] professional qualifications and familiarity with the rules governing the vocational aspects of the Social Security disability evaluation." (A.R. 641.)
Chavez appealed to the district court, contending that the ALJ erred in accepting the VE's arbitrary and unreliable opinion. The district court affirmed the ALJ's decision, concluding that although this court has criticized the equal distribution method in prior opinions, we have not prohibited its use or deemed its application reversible error. Chavez then sought our review.
II
A
On appeal we ask whether substantial evidence supported the ALJ's conclusion that there are significant numbers of jobs in the economy suitable for Kelly Chavez to perform. See 42 U.S.C. § 405(g) (requiring Commissioner's findings to be sustained if supported by substantial evidence); Johansen v. Barnhart , 314 F.3d 283, 287 (7th Cir. 2002). The Supreme Court has explained that substantial evidence *968requires more than "a mere scintilla" of proof and instead requires "such relevant evidence as a reasonable mind might accept as adequate to support a conclusion." Richardson v. Perales, 402 U.S. 389, 401, 91 S.Ct. 1420, 28 L.Ed.2d 842 (1971). Information "without a basis in evidence having rational probative force" cannot satisfy this standard. Id. at 407, 91 S.Ct. 1420 (internal citation omitted). When assessing the administrative record, our role is not to reweigh the evidence or substitute our judgment for that of the agency. Elder v. Astrue , 529 F.3d 408, 413 (7th Cir. 2008).
In the context of job-number estimates, we have observed that the substantial evidence standard requires the ALJ to ensure that the approximation is the product of a reliable method. See Donahue v. Barnhart , 279 F.3d 441, 446 (7th Cir. 2002) ; see also McKinnie , 368 F.3d at 910 (relying upon Donahue and emphasizing that "an ALJ may depend on expert testimony only if the testimony is reliable"). To put the point most succinctly, "[a] finding based on unreliable VE testimony is equivalent to a finding that is not supported by substantial evidence and must be vacated." Britton v. Astrue , 521 F.3d 799, 803 (7th Cir. 2008).
Our prior cases have not required us to explain the full contours of what it means for a VE's step five testimony to be reliable. What we have said, though, goes most of the way to resolving this appeal, if not resolving it entirely. In Donahue , we observed that the substantial evidence requirement brings with it an obligation for experts to use reliable methods. 279 F.3d at 446. We further underscored-and reiterate here-that the measuring stick for assessing the reliability of expert testimony in this administrative setting is not Federal Rule of Evidence 702. Id. (pointing to Richardson and underscoring that " Rule 702 does not apply to disability adjudications, a hybrid between the adversarial and inquisitorial models"). We added that, at the very least, "[e]vidence is not 'substantial' if vital testimony has been conjured out of whole cloth." Id.
Establishing the reliability of a job-number estimate does not require meeting an overly exacting standard. Many variables combine to create uncertainty in a VE's job-number estimate. Consider what is being estimated. It is not general employment levels. Nor is it the number of people working within the same general job category in a particular industry. Rather, the agency is required to focus on what jobs a particular individual can perform (in light of the person's age, education, work experience, and residual functional capacity) and then to estimate the number of those jobs that exist across the nation (or, at a minimum, in a region). See 20 C.F.R. § 416.960(c)(1) ("Any other work (jobs) that you can adjust to must exist in significant numbers in the national economy (either in the region where you live or in several regions in the country).").
A VE's estimate will be just that-an estimate. VEs are neither required nor expected to administer their own surveys of employers to obtain a precise count of the number of positions that exist at a moment in time for a specific job. Think of the difficulty, if not impossibility, of acquiring the data necessary to tally how many residential laundry worker jobs exist throughout the United States or even in the Midwest. The VE necessarily must approximate, and there is no way to avoid uncertainty in doing so.
The law recognizes and respects these realities and limitations. The agency's regulations do not mandate a precise count of job numbers. See 20 C.F.R. § 416.960(c)(2). And, while Congress has required the agency to support its findings *969with substantial evidence, see 42 U.S.C. § 405(g), this standard affords the agency sufficient flexibility in approaching the approximation required at step five. But any method that the agency uses to estimate job numbers must be supported with evidence sufficient to provide some modicum of confidence in its reliability.
B
This is not our first encounter with the equal distribution method. We have seen the method applied in other Social Security cases and-for the exact reason its application here troubles us-have questioned its use in at least four opinions. See, e.g., Alaura v. Colvin , 797 F.3d 503, 507-08 (7th Cir. 2015) ; Voigt v. Colvin , 781 F.3d 871, 879 (7th Cir. 2015) ; Browning , 766 F.3d at 709 ; Herrmann v. Colvin , 772 F.3d 1110, 1112-14 (7th Cir. 2014). Other courts have sounded similar concern, if not specifically about the equal distribution method, then about the many layers of uncertainty in the available data on job numbers. See, e.g., Brault v. Soc. Sec. Admin., Comm'r , 683 F.3d 443, 447 n.4 (2d Cir. 2012) (noting the "information loss" that results from the "many-to-one mapping," between the DOT titles and SOC codes, which results in an estimate of existing jobs that "may deviate significantly from the actual number of existing positions"). What most concerns us is that the method rests on an assumption about the relative distribution of jobs within a broader grouping that lacks any empirical footing. To return to a previous example, we seriously doubt the reliability of a method that assumes that the totality of all cashier and teller positions in today's economy exist in equal numbers at racetracks as they do in banks and retail stores.
All of this brings us to the evidentiary record before the ALJ at Chavez's hearing. And all the record shows is that the VE preferred the job-number estimates produced by the equal distribution method over those from the occupational density method. What is entirely lacking is any testimony from the VE explaining why he had a reasonable degree of confidence in his estimates. The VE, for example, could have drawn on his past experience with the equal distribution method, knowledge of national or local job markets, or practical learning from assisting people with locating jobs throughout the region, to offer an informed view on the reasonableness of his estimates. The absence of any such testimony left the ALJ without any reasoned and principled basis for accepting the job-number estimates.
The ALJ sensed the evidentiary void and rightly reacted by charting the course we suggested in Donahue -by asking the VE to explain why it was reasonable to believe that 108,000 bench assembler jobs was a better estimate than 800. See 279 F.3d at 446 ("If the basis of the vocational expert's conclusions is questioned at the hearing, however, then the ALJ should make an inquiry (similar though not necessarily identical to that of Rule 702 ) to find out whether the purported expert's conclusions are reliable.") (emphasis omitted).
When pressed four times to explain why he chose the estimates produced by the equal distribution method over those from the JobBrowser Pro software, the VE focused entirely on the latter method's shortcomings, testifying that "it's almost logical" that there exist more than 800 bench assembler jobs in the national economy. Perhaps so. But concluding that one approach is flawed does not demonstrate that another method is reliable. Our confidence in the VE's methodology is further diminished by his later concession that he was "not sure that either one of the methods give a very accurate count on numbers." An affirmative explanation for the *970estimates he produced was required, for without one there was no evidentiary foundation on which the ALJ could rest a finding of reliability. See McKinnie , 368 F.3d at 911 ("Without first inquiring into the reliability of [VE] Bose's opinions, the ALJ should not have so unquestioningly accepted her testimony that a significant number of jobs were available to McKinnie."). The transcript leaves us with the conviction that the VE mechanically relied on outdated sources to estimate job numbers, without bringing any aspect of his extensive experience to bear on the reality of those numbers. We are left with the possibility that the job-number estimates were "conjured out of whole cloth." Donahue , 279 F.3d at 446.
The Commissioner argues that the ALJ fulfilled her duty by asking questions about the VE's method. But accepting that argument renders meaningless the substantial evidence standard. The ALJ needed to do more than just ask questions; she needed to hold the VE to account for the reliability of his job-number estimates. See Richardson , 402 U.S. at 407, 91 S.Ct. 1420 (reinforcing the necessity of evidentiary reliability in administrative proceedings). By accepting the VE's estimates at step five because they were "not contradicted," the ALJ effectively and impermissibly shifted the burden to Chavez.
Before accepting a VE's job-number estimate, the ALJ, when confronted by a claimant's challenge, must require the VE to offer a reasoned and principled explanation. At or after a hearing, the VE could support the approximation by, for example, drawing on knowledge of labor market conditions and occupational trends, gleaned from reviewing relevant data sources or from placing workers in jobs. This is particularly true where, as here, there is a world of difference between two estimates offered by a VE-800 versus 108,000 bench assembler jobs. This approach not only properly leaves the burden on the agency at step five, it also aligns fully with the expectations the agency itself has articulated for VEs. See SOC. SEC. ADMIN., VOCATIONAL EXPERT HANDBOOK , 8, 38 (Aug. 2017) ("You should be prepared to explain why your sources are reliable.").
With this opinion we intend no new obligations. Substantial evidence remains the governing standard. We also recognize and underscore that VEs cannot be expected to formulate opinions with more confidence than imperfect data allows. Nor is it our place to enjoin use of the equal distribution method. What we do require, though, is more than what supported the ALJ's decision here.
We recognize that the VE identified three suitable jobs for Chavez and then estimated that, in total, nearly 500,000 of those jobs existed in today's economy. The observation leads nowhere, however, as each of the VE's job estimates was the product of the equal distribution method, and nothing in the administrative record allows us to conclude with any reliability that the estimates reasonably approximate the number of suitable jobs that exist for Chavez. The substantial evidence standard does not permit the shortcut, and too much is at stake for Chavez for us to take it. Her case must go back.
III
Whether Kelly Chavez is disabled and entitled to supplemental income remains for the Social Security Administration to decide. At a new step-five hearing, it may be that the evidentiary gap is filled through expanded testimony from the VE about his estimates or through some other showing that there are a significant number of jobs in the economy Chavez can perform given her limitations. Chavez will *971have the opportunity to challenge any such showing by the agency.
We VACATE and REMAND for further proceedings consistent with this opinion.