Opinion by Judge SCHROEDER; Concurrence by Judge N.R. SMITH.
ORDER AND AMENDED OPINION
SCHROEDER, Circuit Judge:
ORDER
Respondents’ petition for panel rehearing is GRANTED.
The Opinion filed September 10, 2015 is amended as follows:
On opinion page 522, replace the first sentence with:
The Federal Insecticide, Fungicide, and Rodenticide Act, known as FIFRA, prohibits the sale of pesticides that lack approval and registration by the Environmental Protection Agency (“EPA”). 7 U.S.C. § 136a(a).
On opinion page 522, replace the first sentence in the Background section with:
FIFRA prohibits companies from selling any pesticide that the EPA has not approved • and registered. 7 U.S.C. § 136a(a).
Petitioners’ motion for issuance of mandate is GRANTED. The mandate shall issue forthwith.
OPINION
The Federal Insecticide, Fungicide, and Rodenticide Act, known as FIFRA, prohibits the sale of pesticides that lack approval and registration by the Environmental Protection Agency (“EPA”). 7 U.S.C. § 136a(a). The EPA may deny an application for registration when “necessary to prevent unreasonable adverse effects on the environment.” Id. This case is a challenge to the EPA’s approval of insecticides containing sulfoxaflor, which initial studies showed were highly toxic to honey bees. Bees are essential to pollinate important crops and in recent years have been dying at alarming rates. Petitioners are commercial bee keepers and bee keeping organizations.
The EPA initially proposed to conditionally register sulfoxaflor and requested additional studies to address gaps in the data regarding the pesticide’s effects on bees. A few months later, however, the EPA unconditionally registered the insecticides with certain mitigation measures and a lowering of the maximum application rate. It did so without obtaining any further studies. Because the EPA’s decision to unconditionally register sulfoxaflor was based on flawed and limited data, we conclude that the unconditional approval was not supported by substantial evidence. We therefore vacate the EPA’s registration of sulfoxaflor and remand.
BACKGROUND
FIFRA prohibits companies from selling any pesticide that the EPA has not approved and registered. 7 U.S.C. § 136a(a). FIFRA uses a “cost-benefit analysis to ensure that there is no unreasonable risk created for people or the envi[523]*523ronment from a pesticide.” Washington Toxics Coal. v. EPA, 413 F.3d 1024, 1032 (9th Cir.2005). Specifically, FIFRA allows the EPA to deny an application for registration of a pesticide to prevent “unreasonable adverse effects.” 7 U.S.C. § 136a(a). “Unreasonable adverse effects” is defined as “any unreasonable risk to man or the environment, taking into account the economic, social, and environmental costs and benefits of the use of [the] pesticide.” 7 U.S.C. § 136(bb).
In order to register a new pesticide, a manufacturer must submit an application for registration, describing how the pesticide will be used, the claims made of its benefits, the ingredients, and a description of all tests and studies done and the results thereof, concerning the product’s health, safety, and environmental effects. 7 U.S.C. § 136a(c); Thomas v. Union Carbide Agr. Products Co., 473 U.S. 568, 571, 105 S.Ct. 3325, 87 L.Ed.2d 409 (1985). The EPA may either “unconditionally” register a pesticide, 7 U.S.C. § 136a(c)(5), or “conditionally” register it, id. § 136a(c)(7)(C). The EPA conditionally registers a pesticide when there is insufficient data to evaluate the environmental effects of a new pesticide, permitting the pesticide to be used “for a period reasonably sufficient for the generation and submission of required data.” 7 U.S.C. § 136a(7)(C). Unconditional registration necessarily requires sufficient data to evaluate the environmental risks.
In 2010, Respondent-Intervenor Dow Agrosciences LLC applied for approval of sulfoxaflor under 7 U.S.C. § 136a(c)(5). That provision of FIFRA states that the administrator shall register a new pesticide if:
(A)its composition is such as to warrant the proposed claims for it;
(B) its labeling and other materials required to be submitted comply with the requirement of this subchapter;
(C) it will perform its intended function without unreasonable adverse effects on the environment; and
(D) when used in accordance with widespread and commonly recognized practices it will not generally cause unreasonable adverse effects on the environment.
7 U.S.C. § 136a(c)(5).
Dow applied to register three different products, each containing sulfoxaflor as the main/active ingredient. Sulfoxaflor is a new insecticide that targets a range of insects. It acts on the same receptor in insects as does the class of insecticides referred to as neonicotinoids, but its mechanism is distinct from other neonicotinoids, so it is currently the only member of a subclass of neonicotinoids called sulfoxi-mines. Some insects that are resistant to other neonicotinoids are not resistant to sulfoxaflor because of the unique mechanism sulfoxaflor uses. All neonicotinoids kill insects by interfering with their central nervous system, causing tremors, paralysis, and death. Neonicotinoids, including sulfoxaflor, are “systemic” insecticides, which means that they are sprayed onto plants, which then absorb the chemicals and distribute them throughout the plant, into the tissues, pollen, and nectar. Sul-foxaflor and other systemic insecticides therefore kill insects in two different ways: insects die when they come into contact with the pesticide, as when they are sprayed with it, and also when they ingest the plant which has absorbed the pesticide.
Dow asked the EPA to approve sulfoxaf-lor for use on a variety of different crops, including citrus* cotton, cucurbits, fruiting vegetables, canola, strawberries, soybeans, wheat, and many others. The maximum rate of application that Dow proposed var[524]*524ied depending on the crop, with the highest rate being 0.133 pounds of active ingredient per acre (“lb a.i./A”) per application. The minimum number of days between applications also varied by crop, ranging from 5 to 14 days. The maximum yearly application rate of 0.266 pounds of active ingredient per acre was consistent across all crops.
As part of its registration application, Dow made a number of claims regarding the benefits to using sulfoxaflor over other comparable pesticides. These claims and the support for them are not in the public record. The public record does indicate that a number of commenters to the EPA’s proposed conditional registration decision made similar claims regarding the need for and benefits of using sulfoxaflor. Dow also submitted studies and data about the effects of sulfoxaflor on various species, including bees.
The EPA analyzed the studies submitted by Dow using a new framework it had ■recently developed to better analyze the risks to bees, in light of growing concerns about the rapid decline in bee populations.
I. The Pollinator Risk Assessment Framework
The new framework, called the Pollinator Risk Assessment Framework, was developed through consultations between the EPA, Canada’s Pest Management Regulatory Agency, and California’s Department of Pesticide Regulation. It was presented to the FIFRA Scientific Advocacy Panel in 2012 in the form of a document called a “White Paper.” The framework explains it is intended to be iterative and to rely on multiple lines of evidence to further refine and characterize potential risk. It therefore establishes several tiers of evaluation.
The first “preliminary or screening-level” tier, Tier 1, is intended to identify whether potential risks to bees exist. If a risk to bees is identified in Tier 1 then the next tiers, Tier 2 and Tier 3, are intended to define when and where the risks exist and their magnitude. Tiers 2 and 3 “attempt to refine and/or characterize risk estimates to determine the conditions of risk occurrence and, when relevant, to identify spatially-and temporally-specific risks.”
A. Tier 1
At Tier 1, the EPA compares the extent to which bees would be exposed to a pesticide in the environment with the doses at which that pesticide is toxic to bees. Specifically, the EPA reviews studies to determine the “acute median lethal dose” (or “LD50”) of a pesticide, meaning the dose at which half of the individual bees tested die. See 40 C.F.R. § 152.3. The EPA looks at the acute median lethal dose for both contact doses (i.e., bees sprayed directly with the chemical) as well as oral doses (bees consuming nectar or pollen contaminated with the chemical).
Based on the studies Dow submitted as part of its registration application, the EPA determined that the acute median lethal contact dose of sulfoxaflor for bees was 0.13 micrograms of active ingredient per bee. The EPA also determined that the acute median lethal oral dose of sulfox-aflor for bees was 0.052 micrograms of active ingredient per bee. These levels of acute median lethal dose mean that sulfox-aflor is classified as “extremely toxic” to honey bees.
The framework then calls for the EPA to compare these median lethal doses with the concentration of the pesticide that would be expected in the environment, based on characteristics of that pesticide as well as the proposed application rate. For sulfoxaflor applied at the maximum application rate Dow proposed, the expect[525]*525ed environmental concentration for oral consumption by bees was determined to be 4.3 mierograms of active ingredient per bee. The expected environmental concentration for contact exposure for bees was determined to be 0.72 mierograms of active ingredient per bee.
The next step is for the EPA to divide the expected environmental concentrations with the acute median lethal doses to arrive at a figure it calls the “risk quotient,” or “RQ.” The EPA has determined that risk quotients over 0.4 for bees (called the “level of concern” or “LOC”) trigger a need for further study, as a risk quotient of 0.4 represents a scenario where 10% or more of bees in an environment would be killed.
The risk quotient for oral exposure to sulfoxaflor for bees was measured at 83, while the risk quotient for contact exposure to sulfoxaflor was measured at 2.8. Thus, both risk quotients clearly exceeded 0.4, the level at which the EPA expresses concerns serious enough to require further testing.
Before proceeding to Tier 2, however, the EPA also analyzed additional studies Dow had provided regarding residue, or the amount of sulfoxaflor that manifests itself in the pollen and nectar of sprayed crops, in order to refine the risk analysis for' oral exposure. Using those residue levels, the EPA also separated out different risk quotients for different types of bees (e.g., adult and larvae, and different castes within adult bees such as worker, drone, and queen) depending on their different consumption patterns.
The new risk quotients for different types of bees ranged from less than 0.8 to 5.7, all of which continued to exceed the LOC of 0.4. However, the EPA identified several shortcomings in the residue studies, that made it difficult to determine whether the residue results were accurate and suggested the results actually understated the risk. For example, some data was collected several days after pesticide application and at a rate lower than proposed.
B. Tier 2
Because the risk quotients, even with refinement, continued to exceed the level of concern, the EPA proceeded to Tier 2 of the risk assessment framework. Whereas the framework’s Tier 1 analysis is based on studies and statistical modeling of bees in the laboratory, Tier 2 and Tier 3 studies are structured to evaluate the effects of an insecticide on bees in the environment. And whereas the Tier 1 analysis focuses on the effects of the insecticide on individual bees, Tier 2 and Tier 3 analyses attempt to measure the effect on the colony as a whole.
Tier 2 studies are generally referred to as “semi-field studies”; they consist of bees placed in a tunnel enclosure and forced to feed on pesticide-treated crops. Because bees interact with each other, feed each other, and transport and consume food differently depending on their role within the hive, semi-field studies attempt to better capture the effect that a pesticide would have on the functioning of the entire colony.
Semi-field studies, nevertheless, have a number of limitations. The bees are stressed by being put into a tunnel; thus even in the control tunnels, where the bees are not fed pesticide-treated food, they die at much higher rates than in their normal environment. Because bees in the environment feed on a variety of crops, some of which may be treated and some of which may not, tunnel studies may overstate the effects on nearby hives of spraying one crop. In addition, the studies cannot last more than seven to ten days [526]*526due to the added stress bees incur in the tunnels. This means the studies may not capture sublethal effects of insecticides that take longer to manifest (i.e., effects that do not immediately kill bees but adversely affect their ability to function in myriad ways, leading to a long term decline in bee population).
For its Tier 2 analysis Dow submitted six tunnel semi-field studies. These studies were done at different times over the course of several years, and varied widely in terms of application rate, number of tunnels, times the study was replicated, the timing of pesticide application, duration of observation period, and the time of year at which the study was conducted. Perhaps most significantly, all but one of the studies used application rates substantially below the maximum proposed by Dow, 0.133 pounds of active ingredient per acre. The application rates used in five of the six studies ranged from 0.006 to 0.088 pounds of active ingredient per acre.
The sixth study, “Ythier 2012,” had additional limitations. Only two of the seven applications used the maximum application rate of 0.134 pounds of active ingredient per acre. The other five applications tested rates less than the maximum proposed application rate. The Ythier 2012 study used cotton as its test crop, which may have skewed its results, as cotton is a suboptimal source of pollen for honey bees and the bees were therefore not able to maintain sufficient stores of pollen during the study. Additionally, the Ythier 2012 study was designed for quantifying the residue of sulfoxaflor in the plant’s nectar and pollen, not for studying the biological effects of sulfoxaflor on the bees; thus, as the EPA’s own assessment explained, it provided only “limited biological effects information.”
After its Tier 2 analysis, the EPA concluded that additional data was needed. The EPA found that the results from the Tier 2 studies showed that, at the rates used (generally 3-67% of the maximum application rate proposed), the effects on adult bees were relatively short-lived, lasting 3 days or less. But the EPA also concluded that, because the majority of studies used a lower application rate than the one Dow proposed, “[t]he direct effect of sulfoxaflor on [measures of adult forager bee mortality, flight activity, and behavioral abnormalities] at the maximum application rate in the U.S. is presently unknown.” Furthermore, due to various limitations in all of the studies, the effect of sulfoxaflor, when applied at the maximum proposed rate, on both brood development and long-term colony health was “inconclusive.”
The EPA’s environmental risk assessment concluded that additional studies were required, including “one or more Tier 2 semi-field tunnel studies conducted according to OECD 75 guidance.” “OECD” refers to the Organization for Economic Coordination and Development, an international organization that has developed protocols for honey bee semi-field tests. In its regulations, the EPA has viewed the OECD protocols favorably, explaining that they “can be used to develop data necessary to meet the requirements [of FI-FRA].” 40 C.F.R. § 158.70(d)(2). The record does not suggest that those studies have ever been conducted.
II. The Initial, Proposed Conditional Registration
Because of the gaps in data, the EPA declined to give unconditional approval to sulfoxaflor. It proposed instead to conditionally register sulfoxaflor while it collected additional data. As part of its proposed conditional registration, announced in January 2013, the EPA decided to lower the maximum single application rate of sulfox-[527]*527aflor from 0.133 pounds of active ingredient per acre to 0.09 pounds of active ingredient per acre. The EPA also proposed some other mitigation measures, including crop-specific restrictions on spraying before or during bloom and suggested guidelines on some labels regarding the best time to spray.
To address the insufficiency of data, the EPA’s proposed conditional registration required Dow to conduct and submit the results of additional tests, specifically “a Tier 2 semi-field study for assessing impacts on honey bee colony strength and brood development in accordance with OECD-established test guidelines” and “an additional residue study to address the nature and magnitude of sulfoxaflor residues on a pollinator-attractive crop (e.g., canola).” The EPA said that these additional studies would resolve “any residual uncertainty” regarding the effects of sul-foxaflor, and would help it determine whether Dow’s requested maximum application rate of 0.133 pounds per acre could be used in the future.
The EPA was concerned about the risk to bees during the time it would take to do more studies. Nevertheless, in addressing the risk, the EPA determined that sulfox-aflor could be used while these studies were being performed, because “sulfoxaf-lor applications will not result in a catastrophic loss to brood during the time period required for the conditional studies to be performed and assessed.” (emphasis added).
III. The Final Unconditional Registration
Although the EPA announced its decision to propose conditional registration of sulfoxaflor in January 2013, pending receipt of additional data, less than seven months later, on May 6, the EPA decided, to “unconditionally” register sulfoxaflor. It did so even though the record reveals that Dow never completed the requested additional studies. The EPA acknowledged the insufficiency of the data to support unconditional registration at the maximum rate Dow had sought, but gave its approval to usage under modified circumstances.
The EPA justified its new unconditional registration decision by the addition of various mitigation measures: lower maximum application rate of 0.09 pounds of active ingredient per acre, longer minimum intervals between applications, and certain crop-specific restrictions on spraying before or during bloom. The record does not indicate the EPA had ever received any additional data on the effect of such measures. Moreover, the restrictions on spraying during bloom could not apply to continuously blooming crops such as cotton and citrus. For these crops, the EPA required added warning language on the label. For example, the label for cu-curbits, strawberries and citrus was mandated to contain the following statement:
Advisory Pollinator Statement: Notifying known beekeepers within 1 mile of the treatment area 48 hours before the product is applied will allow them to take additional steps to protect their bees. Also, limiting application to times when managed bees and native pollinators are least active, e.g., before 7 am or after 7 pm local time or when the temperature is below 55' F at the site of application, will minimize risk to bees.
The EPA acknowledged that it had classified sulfoxaflor as “very highly toxic” to bees. And it acknowledged that the existing studies were inconclusive as to risks to brood development and colony strength. Although the EPA argued that all of the studies provided relevant data, as noted, only two of the six semi-field studies studied the effects of sulfoxaflor at 0.09 pounds [528]*528of active ingredient per acre (the application rate that the EPA ultimately unconditionally approved). The EPA nevertheless concluded that while there exists “potential hazard to bees from exposure to sulfoxaf-lor,” that hazard will be appropriately mitigated by “reduced application rates, increased minimum application intervals, and the pollinator-related labeling' mitigation.”
The EPA could not give unconditional registration if usage would result in unreasonable adverse environmental effects. The EPA concluded that applying sulfoxaf-lor according to the label would not cause “unreasonable adverse effects”' on bees, and that “the benefits of [sulfoxaflor] compared to the registered alternatives, as well as [sulfoxaflor’s] ability to control problematic target pests” outweighed the costs and therefore justified registration. Overall, despite its earlier refusal to grant unconditional registration pending additional studies, and despite the lack of any meaningful study of the effects of the mitigation measures, the EPA concluded that no additional data on sulfoxaflor was required.
After the EPA announced its final decision to unconditionally register sulfoxaflor, petitioners filed a petition for review with this court, claiming that the EPA’s decision was not supported by substantial evidence in the record as a whole. Dow and the EPA defend the registration decision by arguing that, while the studies submitted by Dow had limitations, the EPA was nonetheless able to glean sufficient useful data from the studies to support its registration decision, and that the EPA retains the flexibility to determine the type of data needed to support registration in each case. The EPA thus maintains that the record contains sufficient evidence to support its unconditional registration at the lower maximum rate.
DISCUSSION
Under FIFRA, a court reviewing the EPA’s decision to register a new insecticide shall uphold the EPA’s decision “if it is supported by substantial evidence when considered on the record as a whole.” 7 U.S.C. § 136n(b).
We have jurisdiction under 7 U.S.C. § 136n(b). All the parties agree that the studies underlying the EPA’s Tier 2 analyses had serious limitations. No additional studies were submitted to evaluate the mitigation measures the EPA added in its unconditional registration. We cannot conclude that the unconditional registration is supported by the record as a whole.
With respect to remedy, we conclude we must vacate the registration and remand to the agency.
I. Limitations in Tier 2 Studies
The EPA’s regulations require the EPA to “Review[ ] all relevant data in [its] possession ” and to “determine[ ] that no additional data are necessary ” to make determinations of no unreasonable adverse effects. 40 C.F.R. § 152.112(b)-(c). The regulations also require “field testing for pollinators” to be submitted as part of an application for registration if data from other sources indicates a risk to honey bees. 40 C.F.R. § 158.630(d) and (e) n.25. All parties agree that, because some data indicated a potential risk of adverse effects on honey bee colonies, Dow was required to submit pollinator field testing. In the White Paper addressing the new tiered risk assessment framework for pollinators, the framework drafters noted that while the requirement of “field testing for pollinators” in 40 C.F.R. § 158.630 is general, semi-field studies would satisfy that requirement.
[529]*529What data Dow submitted did not support approval of sulfoxaflor at either the proposed maximum rate of 0.133 pounds of active ingredient per acre or the reduced maximum rate of 0.09 pounds of active ingredient. In its original conditional registration decision, the EPA had required additional testing per “OECD guidance.” The record reveals a number of deficiencies in Dow’s submitted semi-field studies, most of which did not conform to OECD guidelines. Although the EPA’s regulations do not mandate that studies be performed in accordance with OECD guidelines, the regulations state that studies done in accordance with OECD protocol will suffice to meet the data requirements ofFIFRA. 40 C.F.R. § 158.70(d)(2).
Some of the deficiencies in Dow’s submitted studies were unrelated to the fact that these studies did not comply with OECD guidelines. Such deficiencies included limitations inherent in the tunnel study design, such as the fact that bees undergo significant stress when placed in a tunnel, and some limitations related to the fact that the application rates used were generally much lower than the maximum proposed application rate. Some of the limitations would have been ameliorated, however, had the studies conformed with OECD guidance. For example, proper controls could have been used, the studies could have been replicated more times, and the bees could have been observed for a longer period of time after being removed from the tunnels.
One of the problems the EPA originally identified with the Tier .2 studies was that only one of the studies used the proposed maximum application rate, 0.133 pounds per acre, and that study, Ythier 2012, was designed for quantifying residue, not studying biological effects. Yet, at the lower rate that the EPA approved, 0.09 pounds per acre, the studies provided only slightly more data about the effects of application. Portions of only two different studies provided data regarding the effects of sulfoxaflor at the maximum application rate of 0.09 pounds per acre: Ythier 2012 used an application rate at or above 0.089 pounds per acre for four of its seven applications. Hecht-Rost 2009 used an application rate of 0.088 pounds per acre as one of its five applications.
'The Ythier 2012 study, because it was designed to quantify plant residues, not biological effects, had other limitations. It did not assess forager mortality, flight intensity, forager behavior, or reference toxicant effects (meaning the effect of sul-foxaflor as compared to known toxic chemicals on control tunnels), and it did not include any concurrent controls for interpreting biological effects or brood development. These were noted in the EPA’s environmental assessment.
Hecht-Rost 2009 also had a number of limitations: there was an infestation of a pest, Varroa mite, which can kill honey bee colonies; there was a long pre-exposure period in the tunnels of 11 days, which, given that bees are stressed by being in tunnels, compromises the data; the colonies varied widely between the different tunnels; there was a short observation period of only 7 days; ■ and there were few larvae in the tunnels. Furthermore, Hecht-Rost 2009 included no observation after the hives were removed from the tunnels. And only one of the five applications provided data about an application rate around 0.09 pounds per acre.
Moreover, all of the studies, regardless of application rate, suffered from an additional significant flaw: they provided inconclusive' or insufficient data on the effects of sulfoxaflor on brood development and long-term colony health. Because the honey bee colony is an interdependent “su-perorganism,” the effect of an insecticide [530]*530on one type of bee can ripple through the hive. The two studies that provided the best data for evaluating the effects on brood development, Schmitzer 2011a and 2011b, used application rates less than half of the EPA’s new maximum rate of 0.09 pounds per acre. In these two studies, the measurements for brood termination rate (the number of brood that fail to develop through to emergence) in the control tunnels was quite high, with 56% and 65% of the brood failing to survive. By comparison, other studies done according to OECD Guideline 75 reported brood termination control rates of 8^13% in five control studies, with three of the five controls reporting high brood termination rates only because of poor weather.
Because the brood failed to emerge at high rates even in the control tunnels in the Schmitzer studies, the controls were not suitable for comparison with the brood termination rates of the tunnels treated with sulfoxaflor; thus, there was inconclusive data as to the effect of sulfoxaflor on brood termination rate. Accordingly, even these two studies reveal little about the effect of sulfoxaflor on brood development, both due to the low application rates used and due to problems with the control tunnels that caused high brood losses.
In addition, all of the semi-field studies provided limited information about longer term effects on colony strength. The three studies with valid controls that studied colony strength all used application rates of about half the new maximum application rate. The studies that did not have valid controls measured colony strength before and after application and did not discern a measurable decline in colony strength; however, these studies measured colony health over a relatively short time period, between 7 and 17 days after sulfoxaflor treatment. The EPA assessment acknowledged that different negative effects could appear on the hive-level over a longer time period: for example, the death of foraging bees that occurs immediately after sulfoxaflor is sprayed might lead to premature recruitment of hive bees into the forager work force.
It was on the basis of all of these deficiencies and others it noted, that the EPA in January 2013 concluded that it needed additional studies regarding the effect on brood development and long-term colony strength. That was when the EPA decided to grant sulfoxaflor only conditional registration.
Though the EPA specified in January that the additional studies were necessary to evaluate whether sulfoxaflor could ever be used at the maximum proposed application rate of 0.133 pounds of active ingredient per acre, and unconditional approval in May 2013 was for a lower application rate, the limitations of the data regarding brood development and long-term colony strength exist even with the lowered application rate of 0.09 pounds per acre. By the EPA’s own reckoning, the data was insufficient to evaluate the effect of sulfox-aflor on brood development and long-term colony strength. The decisions are not consistent. The later, unconditional approval lacks support.
In addition to needing studies on brood development and long-term colony strength, it is clear that the EPA was lacking sufficient data on the impact of sulfoxaflor generally even at the reduced application rate of 0.09 pounds of active ingredient per acre. The EPA had only portions of two studies evaluating the effects of sulfoxaflor at that rate. And the EPA also recognized in its proposed conditional registration that it needed more studies regarding the residue of sulfoxaflor that appears in nectar and pollen in. a crop that is pollinator-attractive, such as canola. On the basis of the studies submitted, the [531]*531EPA lacked substantial evidence to support its conclusions that application of sul-foxaflor at a rate of 0.09 pounds per acre would not have an unreasonable adverse effect on the environment.
The EPA and Dow argue that since the studies are inconclusive as to the risks of sulfoxaflor for bees, the studies affirmatively prove that sulfoxaflor does not cause unreasonable adverse effects on bees. Neither logic nor precedent can sustain this position. We have previously held that an agency cannot rely on ambiguous studies as evidence of a conclusion that the studies do not support. See Tucson Herpetological Soc. v. Salazar, 566 F.3d 870, 879 (9th Cir.2009) (finding that the Secretary of the Interior erred when he affirmatively relied on ambiguous and inconclusive studies to support a conclusion). The limitations of the underlying data in this case mean that no such conclusion can be reached.
The EPA also argues that Tier 2 studies are only required when analysis at the Tier 1 phase shows measurements above the level of concern and, at the lower application rate of 0.09 pounds of active ingredient per acre, few of the residue measurements were high enough to trigger the acute level of concern. Thus, the EPA asks us to conclude the showing was “close enough” so as not to trigger the requirement for Tier 2 studies. The EPA therefore contends that it is irrelevant that the Tier 2 studies are severely flawed.
This argument just does not fly. It is trae that, at the lower rate of 0.09 pounds per acre, only 2 of the 66 nectar measurements and 1 of the 66 pollen measurements in the Ythier 2012 study were above the level of concern. And only 1 of the nectar measurements in the other residue studies (MRIDs 48476601, 48445806, 48755601) was above the level of concern. It may well be true, as the EPA argues, that its definition of the “level of concern” is inherently conservative. But at least some of the measurements do exceed the level of concern, and where data indicates a potential risk to pollinators, the EPA’s regulations mandate pollinator field testing. 40 C.F.R. § 158.630(d) and (e) n.25 (table showing data requirements for non-target terrestrial and aquatic organisms, including honey bees). Moreover, the EPA’s assessment noted significant shortcomings in the residue studies which created uncertainty in the extrapolation of the results and which may have understated the risk. The EPA acted in accordance with its regulations and common sense in proceeding to Tier 2.
We have previously held that we cannot allow the EPA to avoid its own regulations when actual measurements trigger risk concerns, even where the measurements were “in the neighborhood” of measurements that would not trigger such concern. We have said such an argument may be “well taken as a practical matter” but is “irrelevant as a legal matter.” See Natural Res. Def. Council v. EPA, 735 F.3d 873, 883-84 (9th Cir.2013) (holding that, where risk concern was triggered by a measurement less than or equal to 1,000, and one of four treatments measured exactly 1,000, the EPA could not argue that the risk concern was not triggered). The EPA had similarly argued in that case that its calculations were based on very conservative assumptions, so a risk estimate that was “close” to the threshold was close enough. Id. We rejected that argument, holding that we cannot “revise EPA’s assumptions, alter its rule of decision, or perform our own risk assessment.” Id.
The same is true here. The EPA chose to set its level of concern at a measurement it now feels is overly conservative, but a court cannot alter the agency’s own rule. And because the EPA set its level of [532]*532concern accordingly, and because at least some of the residue measurements with the lower application rate of 0.09 pounds per acre still triggered the acute level of concern, pollinator field testing- was required. That requirement cannot be met by portions of two studies which are limited in a variety of ways, particularly as regards brood development and long-term colony strength&emdash;and particularly given that the EPA itself recognized those limitations.
“It is well-established that an agency’s action must be upheld, if at all, on the basis articulated by the agency itself.” Motor Vehicle Mfrs. Ass’n of the U.S., Inc. v. State Farm Mut. Auto. Ins. Co., 463. U.S. 29, 50, 103 S.Ct. 2856, 77 L.Ed.2d 443 (1983) (internal citation omitted). The EPA’s basis for unconditionally registering sulfoxaflor in the absence of sufficient data documenting the risk to bees does not hold up under its own rationale. Without sufficient data, the EPA has no real idea whether sulfoxaflor will cause unreasonable adverse effects on bees, as prohibited by FIFRA. Accordingly, the EPA’s decision to register sulfoxaflor was not supported by substantial evidence.
The matter must be remanded to the agency. We need not reach the other claims of error raised by petitioners.
II. The Remedy
The remaining issue is whether to vacate the registration or remand while leaving the registration in place. We order remand without vacatur only in “limited circumstances.” Cal. Cmties. Against Toxics v. EPA 688 F.3d 989, 994 (9th Cir.2012). We leave an invalid rule in place only “when equity demands” that we do so. Idaho Farm Bureau Fed’n v. Babbitt, 58 F.3d 1392, 1405 (9th Cir.1995). When determining whether to leave an agency action in place on remand, we weigh the seriousness of the agency’s errors against “the disruptive consequences of an interim change that may itself be changed.” Cal. Cmties. Against Toxics, 688 F.3d at 992 (quoting Allied-Signal, Inc. v. U.S. Nuclear Regulatory Comm’n, 988 F.2d 146, 150-51 (D.C.Cir.1993)).
When deciding whether to vacate rulings by the EPA, we consider whether vacating a faulty rule could result in possible environmental harm, and we have chosen to leave a rule in place when vacating would risk such harm. See, e.g., Idaho Farm Bureau Fed’n, 58 F.3d at 1405-06 (choosing not to vacate because setting aside listing of snail species as endangered would risk potential extinction of that species); Cal. Cmties. Against Toxics, 688 F.3d at 994 (remanding without vacating because vacating could lead to air pollution,' undermining the goals of the Clean Air Act). We have also looked at whether the agency would likely be able to offer better reasoning or whether by complying with procedural rules, it could adopt the same rale on remand, or whether such fundamental flaws in the agency’s decision make it unlikely that the same rule would be adopted on remand. Compare Allied-Signal, 988 F.2d at 151 (declining to vacate because there was “at least a serious possibility that the [agency would] be able to substantiate its decision on remand”), with North Carolina v. EPA 531 F.3d 896, 900 (9th Cir.2008) (concluding that the EPA’s rule “must” be vacated because “fundamental flaws” prevented the EPA from promulgating the same rule on remand).
In this case, given the precariousness of bee populations, leaving the EPA’s registration of sulfoxaflor in place risks more potential environmental harm than vacating it. Moreover, on remand, a different result may be reached. Once the EPA obtains adequate Tier 2 studies, it may conclude that a lower maximum application [533]*533rate of sulfoxaflor is warranted, or that sulfoxaflor cannot be registered at all because of its effects on brood development and long-term colony strength.
We therefore vacate the EPA’s unconditional registration of sulfoxaflor and remand for the EPA to obtain further studies and data regarding the effects of sulfoxaflor on bees, as required by EPA regulations.
VACATED and REMANDED.
Costs are awarded to the Petitioners.
Concurrence by Judge N.R. SMITH.