TROTT, Circuit Judge:
I
OVERVIEW
Michael Prime (“Prime”) was charged with, and convicted of, one count of conspiracy to commit wire fraud, in violation of 18 U.S.C. § 371; one count of conspiracy to manufacture counterfeit securities, in violation of 18 U.S.C. § 371; and three counts of possessing, manufacturing, and uttering counterfeit securities, in violation of 18 U.S.C. § 513(a). Prime raises four issues on appeal: 1) whether the district court properly denied his motion for a
Franks
hearing;
2) whether the court abused its discretion in allowing the testimony of an expert handwriting analyst; 3) whether the court abused its discretion in not allowing Prime to substitute counsel; and 4) whether the jury’s potential exposure to extrinsic evidence was grounds for a new trial. We have jurisdiction under 28 U.S.C. § 1291, and we affirm all of the district court’s orders and decisions.
II
BACKGROUND
Between April and June 2001, Prime, along with three co-conspirators, David Hiestand (“Hiestand”), Juan Ore-Lovera, and Jeffrey Hardy, sold non-existent items on eBay, purchased items using counterfeit money orders created by the group, sold pirated computer software, and stole credit card numbers from software purchasers. To facilitate this operation, Prime and his cohorts used a credit card encoder to input the stolen data on their own credit cards, set up post office boxes under false names, manufactured false identifications, and used a filter bank account to hide proceeds of the crimes.
At trial, numerous victims testified as to the details surrounding how they had been defrauded by Prime’s various scams. In addition, co-conspirators Hiestand and Hardy both extensively testified as to the details of the conspiracy, implicating Prime in all of the crimes charged. The prosecution also elicited the expert opinion of Kathleen Storer (“Storer”), a forensic document examiner with the Secret Service.
She testified that Prime was the author of as many as thirty-eight incriminating exhibits, including envelopes, postal forms, money orders, Post-it notes, express mail labels and postal box applications. Prime took the stand in his own defense and claimed that despite all of the evidence linking him to the various scams, including admissions that his fingerprints were on several items linked to the crimes, he was simply attempting to engage in legal entrepreneurial ventures. Prime also confirmed that he had previously been convicted of first and second degree theft, two counts of possession of stolen property in the second degree, and forgery. The jury found Prime guilty on all counts.
Prime moved for a new trial based on the improper submission of extrinsic evidence to the jury. The district court denied the motion, and this appeal follows.
Ill
ADMISSIBILITY OF EXPERT TESTIMONY
Prime moved in limine to éxclude Storer’s expert testimony. The court held a
Daubert
hearing where both sides were allowed to offer voluminous materials and expert testimony regarding the reliability of the proposed testimony.
Daubert v. Merrell Dow Pharms., Inc.,
509 U.S. 579, 113 S.Ct. 2786, 125 L.Ed.2d 469 (1993). After careful consideration, the court denied the motion,
see United States v.
Prime, 220 F.Supp.2d 1203 (W.D.Wash.2002), and Storer testified that, in her opinion, Prime’s hand-writing appeared on counterfeit money orders and other incriminating documents. On appeal, Prime contends that the admission of expert testimony regarding handwriting analysis was unreliable under
Daubert,
and thus the court abused its discretion by allowing Storer to testify.
Handwriting Analysis
In
Daubert,
the Supreme Court set forth the guiding principle that “under [Federal Rule of Evidence 702]
the trial judge must ensure that any and all scientific testimony or evidence admitted is not only relevant, but rehable.” 509 U.S. at 589, 113 S.Ct. 2786. In order to assist the trial courts with this task, the Court suggested a flexible, factor-based approach to analyzing the reliability of expert testimony.
Id.
at 593-95, 113 S.Ct. 2786. Although not an exclusive list, these factors include: 1) whether a method can or has been tested; 2) the known or potential rate of error; 3) whether the methods have been subjected to peer review; 4) whether there are standards controlling the technique’s operation; and 5) the general acceptance of the method within the relevant community.
Id.
at 593-94, 113 S.Ct. 2786.
Kumho Tire Co. v. Carmichael
resolved any post
-Daubert
uncertainty that the trial judge’s responsibility to keep unreliable expert testimony from the jury applies not only to “scientific” testimony, but to all expert testimony. 526 U.S. 137, 148, 119 S.Ct. 1167, 143 L.Ed.2d 238 (1999). As a result, this “basic gatekeeping obligation” applies with equal force in cases, such as this one, where “non-scientific” experts wish to relate specialized observations derived from knowledge and experience that is foreign to most jurors.
Id. Kumho Tire
also makes it clear that “the trial judge must have considerable leeway in deciding in a particular case how to go about determining whether particular expert testimony is reliable,” as well as
the ultimate determination of whether the proposed expert testimony is reliable.
Id.
at 152, 119 S.Ct. 1167. Accordingly, we review the district court’s decision to admit or deny expert testimony for abuse of discretion.
Id.
In accordance with
Kumho Tire,
the broad discretion and flexibility given to trial judges to determine how and to what degree these factors should be used to evaluate the reliability of expert testimony dictate a case-by-ease review rather than a general pronouncement that in this Circuit handwriting analysis is reliable. As the Supreme Court concluded,
we can neither rule out, nor rule in, for all cases and for all time the applicability of the factors mentioned in
Daubert,
nor can we now do so for subsets of cases categorized by category of expert or by kind of evidence. Too much depends upon the particular circumstances of the particular case at issue.
Id.
at 150, 119 S.Ct. 1167;
see also United States v. Hankey,
203 F.3d 1160, 1168 (9th Cir.2000) (quoting
Skidmore v. Precision Printing and Packaging, Inc.,
188 F.3d 606, 618 (5th Cir.1999) (“Whether
Daubert’s
suggested indicia of reliability apply to any given testimony depends on the nature of the issue at hand, the witness’s particular expertise, and the subject of the testimony. It is a fact-specific inquiry.”) (internal citations omitted)).
In this case, Storer was given 112 pages of writing known to be Prime’s, 114 pages of Hiestand’s, and 14 pages of Hardy’s. She was then asked whether the handwriting on 76 documents associated with the alleged conspiracy, such as envelopes, postal forms, money orders, Post-it notes, express mail labels and postal box applications, belonged to any of the co-conspirators.
Storer “identified” Prime’s handwriting on 45 of the documents.
Following the
Daubert
hearing, the district court issued a brief order concluding that the proposed forensic document examination testimony was reliable. After the conclusion of the trial, the district court issued a more detailed Order Regarding Defendant’s Motion in Limine, which thoroughly and specifically analyzed the reliability of Storer’s testimony with respect to each of the
Daubert
factors.
See Prime,
220 F.Supp.2d 1203.
1. Whether the theory or technique can be or has been tested
Handwriting analysis is performed by comparing a known sample of handwriting to the document in question to determine if they were written by the same person. The government and Storer provided the court with ample support for the proposition that an individual’s handwriting is so rarely identical that expert handwriting analysis can reliably gauge the likelihood that the same individual wrote two samples. The most significant support came from Professor Sargur N. Srihari of the Center of Excellence for Document Analysis and Recognition at the State University of New York at Buffalo, who testified that the result of his published research was that “handwriting is individualistic.” With respect to this case in particular, the court noted that Storer’s training credentials in the Secret Service as well as her certification by the American Board of Forensic Document Examiners were “impeccable.” The court also believed that Storer’s analysis in this case was reliable given the “extensive” 112 pages containing Prime’s known handwriting.
2. Whether the technique has been subject to peer review and publication
The court cited to numerous journals where articles in this area subject handwriting analysis to peer review by not only handwriting experts, but others in the forensic science community. Additionally, the Kam study,
see infra,
which evaluated the reliability of the technique employed by Storer of using known writing samples to determine who drafted a document of unknown authorship, was both published and subjected to peer review. The court also noted that the Secret Service has instituted a system of internal peer review whereby each document reviewed is subject to a second, independent examination.
3. The known or potential rate of error
In concluding that the type of handwriting analysis Storer was asked to perform had an acceptable rate of error, the court relied on studies conducted by Professor Moshe Kam of the Electrical and Computer Engineering Department at Drexel University. Professor Kam’s studies demonstrated that expert handwriting analysts tend to be quite accurate at the specific task Storer was asked to perform — determining whether the author of a known writing sample is also the author of a questioned writing sample. When the two samples were in fact written by the same person, professional handwriting analysts correctly arrived at that conclusion 87% of the time. On the other hand when the samples were written by different people, handwriting analysts erroneously associated them no more than 6.5% of the time. While Kam’s study demonstrates some degree of error, handwriting analysis need not be flawless in order to be admissible. Rather, the Court had in mind a flexible inquiry focused “solely on principles and methodology, not on the conclusions that they generate.”
Daubert,
509 U.S. at 595, 113 S.Ct. 2786. As long as the process is generally reliable, any potential error can be brought to the attention of the jury through cross-examination and the testimony of other experts.
A
The existence and maintenance of standards controlling the technique’s operation
The court recognized that although this area has not been completely standardized, it is moving in the right direction. The Secret Service laboratory where Storer works has maintained its accreditation with the American Society of Crime Laboratory Directors since 1998, based on an external proficiency test. Furthermore, the standard nine-point scale used to express the degree to which the examiner believes the hand-writing samples match was established under the auspices of the American Society for Testing and Materials (“ASTM”). The court reasonably concluded that any lack of standardization is not in and of itself a bar to admissibility in court.
5. General acceptance
The court recognized the broad acceptance of handwriting analysis and specifically its use by such law enforcement agencies as the CIA, FBI, and the United States Postal Inspection Service.
Given the comprehensive inquiry into Storer’s proffered testimony, we cannot say that the district court abused its discretion in admitting the expert handwriting analysis testimony. The district court’s thorough and careful application of the
Daubert
factors was consistent with all six circuits that have addressed the admissibility of handwriting expert testimony, and determined that it can satisfy the reliability threshold.
See United States v. Crisp,
324 F.3d 261, 269-70 (4th Cir.2003);
United States v. Mooney,
315 F.3d 54, 63 (1st Cir.2002);
United States v. Jolivet,
224 F.3d 902, 906 (8th Cir.2000);
United
States v. Paul,
175 F.3d 906, 911 (11th Cir.1999);
United States v. Jones,
107 F.3d 1147, 1161 (6th Cir.1997);
United States v. Velasquez,
64 F.3d 844, 850-52 (3d Cir.1995).
IV
SUBSTITUTION OF COUNSEL
On November 29, 2001, four days before trial was set to begin, Prime filed a motion to substitute counsel, which the district court granted. The trial was continued to accommodate the newly-appointed counsel, Lee Covell (“Covell”), and after an additional stipulated continuance, was set for May 20, 2002. On May 9, 2002, at Prime’s request, Covell filed an ex parte motion to withdraw and substitute counsel. The following day the court held a closed-court inquiry without the prosecution to address this request. After hearing from both Prime and Covell, the court denied the motion.
Four days before trial, Prime filed yet another motion for substitution of counsel. On the morning of trial, just before the proceedings were set to begin, John Rosel-lini (“Rosellini”), Prime’s privately retained attorney, appeared before the court requesting, pursuant to this motion, that he be substituted as counsel on the condition that a 120-day continuance be granted. The court denied this motion as well. Prime appeals the denial of both motions to substitute counsel.
A. Standard of Review
A district court’s refusal to substitute counsel is reviewed for abuse of discretion.
United States v. Castro,
972 F.2d 1107, 1109 (9th Cir.1992). The district court’s ruling on a motion for a continuance is also reviewed for abuse of discretion.
United States v. Garrett,
179 F.3d 1143, 1144-45 (9th Cir.1999) (en banc).
B. Attempt to Remove Covell
We must examine three elements when reviewing a district court’s denial of a substitution motion: 1) the timeliness of the motion; 2) the adequacy of the district court’s inquiry into the defendant’s complaint; and 3) whether the asserted conflict was so great as to result in a complete breakdown in communication and a consequent inability to present a defense.
Castro,
972 F.2d at 1109. Given the judge’s recognition and proper assessment of each of these factors, we conclude that he did not abuse his discretion in denying the motion to remove Covell and substitute new counsel.
1. Timeliness
In
United States v. Garcia,
we held that a motion made six days before the trial was scheduled to begin was not timely because the quantity and complexity of the discovery materials would have required a continuance. 924 F.2d 925, 926 (9th Cir.1991). In this case, the substitution motion was made ten days before trial, which given the quantity and complexity of the evidence and issues is not significantly different from the situation in
Garcia.
As the district judge noted, “it would be extremely unlikely that any new counsel could be appointed and be in a position to be prepared to go to trial in a mere 10 days from now.” We are not suggesting that any particular time period prior to trial is dispositive regarding this factor. Rather, timeliness may depend on the reason for substitution, and its strength. If, for example, counsel was indeed unprepared, the defendant might not have cause to raise unpreparedness until shortly before trial, when preparedness would be expected.
2. Adequacy of the Inquiry
Prime was given a full and fair opportunity to explain why he felt substitution was
necessary. After the court allowed Prime an opportunity to voice his concerns, the court responded “[s]o it’s basically Mr. Covell met with your parents, they told you that they didn’t feel that he was prepared, that he was not — didn’t have a defense plan, and you’re going with their advice?” Prime agreed with the court’s summary of his position. The court then asked Prime “Is there anything else you want to bring to my attention?” At this point, Prime expressed his concern that Covell had given up, and was working on sentencing issues rather than his defense. Covell then testified that he was well prepared for the trial and he had no difficulties communicating with Prime. Because Prime was given the opportunity to express whatever concerns he had, and the court inquired as to Covell’s commitment to the ease and his perspective on the degree of communication, we find that the hearing was adequate.
S. Degree of communication breakdown
Based on Covell’s representation that he had no difficulties communicating with Prime and that he and Prime enjoyed a good rapport and working relationship, in addition to the lack of any indication by Prime that communication was a problem, the court properly determined that Prime failed to demonstrate any breakdown in the attorney-client relationship.
In light
of
the district court’s reasoned determination with regard to each of the three factors, the court did not abuse its discretion in denying Prime’s motion to
remove his
appointed attorney days before trial.
C. Attempt to Substitute Rossellini
The district court did not abuse its discretion in denying this motion.
As the district court expressed “Mr. Prime has already gone through two attorneys at public expense and did not choose to try to retain counsel until the very, very eve of trial.” In addition, the court noted that the government witnesses had already been brought from great distances at a considerable expense. The court also reminded counsel that the trial had been set for this time because, due to the court’s busy schedule, this was the only time available to try the case in a timely manner. Finally, as the court suggested, a strong inference could be drawn that this motion was brought for purposes of delay, as it was the second such eve-of-trial motions, accompanied, as before, by a request for a continuance. The district court’s decision was not, therefore, an abuse of discretion.
V
JURY EXPOSURE TO EXTRINSIC EVIDENCE
As jury deliberations commenced, a problem arose when the jury was mistakenly provided access to 24 exhibits that had not been admitted into evidence. The extrinsic evidence included money orders and e-mail correspondence with aliases used to conduct fraudulent transactions, written reports by both the fingerprint and handwriting expert, and certified copies of
prior convictions for both Prime and his friend Shawn Cahill.
The court became aware of this mistake when the jury made a request to see Storer’s handwriting report, and shortly thereafter informed the court that they had found it. At this point, the court recognized that the jury had been given exhibits that had not been admitted into evidence and that it had to make a decision as to the impact of the evidence. After a brief review, the court concluded that Storer’s written report did not include anything that had not been testified to at trial, and that there was no harm given the brief period it was available to the jury.
During this time, the prosecution also brought to the court’s attention that there may be other exhibits in the jury room that had not been admitted into evidence. The court then called the jury into the courtroom and informed them that “the report from Kathleen Storer ... was never offered into evidence, and was never admitted into evidence. It should not have gone to the jury room. We have withdrawn the report and you should only consider the testimony of Kathleen Storer as you remember it at trial.” The court also requested that the jury refrain from reviewing any exhibits that were not on the master exhibit list, and inform the court if they came across such exhibits. The judge asked the jury foreperson if she had “come across any other exhibits so far that were not identified on the master list,” to which she responded “no.” The judge then stated “I’m going to ask this question of the entire jury, and if in [sic] anybody says, yes, please raise your hand.” Thé judge asked “[h]as anyone else come across an exhibit that was not on the master exhibit list.” The court noted that there was no response. All extrinsic evidence was then pulled from the exhibit boxes before the exhibits admitted into evidence were returned to the jury. Once more, the judge called the jury into the courtroom and admonished them that “[y]ou should not hold this mistake against Mr. Prime at all. Neither he nor Mr. Covell had anything to do with this, but it is so important that you decide this case strictly on those exhibits that have been admitted into evidence.... So, if you have any questions or doubts about anything, and you want to look back and make sure that it is an exhibit that has been admitted, I would urge you to be very, very careful in that regard.”
Based on the availability of this extrinsic evidence to the jury, Prime filed a motion for mistrial, which was denied.
Ordinarily, we review the denial of a motion for mistrial for abuse of discretion.
United States v. Mills,
280 F.3d 915, 921 (9th Cir.2002). Where jurors are exposed to extrinsic evidence, however, we are to engage in an independent review of the entire record.
United States v. Keating,
147 F.3d 895, 899 (9th Cir.1998).
B. Improperly Admitted Exhibits
“A defendant is entitled to a new trial when the jury obtains or uses evidence that has not been introduced during trial if there is ‘a reasonable possibility that the extrinsic material
could
have affected the verdict.’ ”
Dickson v. Sullivan,
849 F.2d 403, 405 (9th Cir.1988) (quoting
United States v. Vasquez,
597 F.2d 192, 193 (9th Cir.1979)). The prosecution bears the burden of proving beyond a reasonable doubt that extrinsic evidence did not contribute to the verdict.
Id.
at 405-06.
In
Dickson,
we developed a five factor approach to determine whether the prosecution met this burden. Those factors are:
1) whether the material was actually received, and if so, how; 2) the length of
time it was available to the jury; 3) the extent to which the jury discussed and considered it; 4) whether the material was introduced before a verdict was reached, and if so at what point in the deliberations; and 5) any other matters which may bear on the issue of the reasonable possibility of whether the extrinsic material affected the verdict.
Id.
at 406. The fifth factor includes consideration of the nature of the extrinsic evidence.
Keating,
147 F.3d at 902.
In
Jeffries v. Wood,
we expanded upon the
Dickson
factors, and introduced several other factors that should impact our consideration of the extrinsic evidence in this case, including:
whether the extraneous information was otherwise admissible or merely cumulative of other evidence adduced at trial; whether a curative instruction was given or some other step taken to ameliorate the prejudice; the trial context [including consideration of the
Dickson
factors]; and whether the statement was insufficiently prejudicial given the issues and evidence in the case.
Jeffries v. Wood,
114 F.3d 1484, 1491-92 (9th Cir.1997).
In this case, application of the
Dickson
and
Jeffries
factors suggests that the extrinsic evidence did not affect the verdict. Although the jury had access to the evidence for approximately three hours, jury review of the Storer report was not prejudicial, as it did not include anything that had not already been testified to at trial. The fingerprint exhibits were also cumulative of what had been testified to and admitted at trial. Likewise, the money order, checks, and e-mail correspondence were cumulative of evidence introduced at trial, and would have been admissible had the prosecution chosen to lay the proper foundation. As the judge stated during his attempt to resolve this problem, “if [money orders, or other items] had been incriminating, I’m sure the Government would have offered it....”
Prime’s main concern relates to his and Cahill’s prior conviction reports. The court, however, after specifically inquiring of the jury, found that the jury had not reviewed the certified copies of convictions of either Cahill or Prime. Moreover, the court determined that even if the jury had seen the reports, they would not have affected the verdict. The only evidence in addition to the five felonies Prime admitted to during his testimony was a conviction for possession of an incendiary device. If the jury
had
discovered this evidence, it would not have affected the verdict because evidence introduced at trial already established that Prime had in the past armed himself with weapons and had obtained stun guns. With regard to Cahill’s prior convictions, there is no possibility that that information would have affected the verdict because as the judge commented, “I’m not sure the jury would be surprised to find that Mr. Cahill had some prior convictions, since everyone else in the apartment seemed to.... ” In addition to the lack of prejudice, the judge also issued two separate curative instructions, which under
Jeffries,
weighs in favor of finding that the government established, beyond a reasonable doubt, that the extrinsic evidence did not affect the verdict.
Jeffries,
114 F.3d at 1491.
The extrinsic evidence given to the jury was cumulative and non-prejudicial, and the court gave proper curative instructions. Therefore, in light of the entire record, we conclude that the extrinsic evidence had no impact on the verdict. We affirm the denial of the motion for mistrial.
AFFIRMED.