An recent article in the journal Ophthalmology reported the results of a clinical trial that evaluated acupuncture as an adjunctive treatment for anisometropic amblyopia. In the abstract, the authors conclude:
Acupuncture is a potentially useful complementary treatment modality that may provide sustainable adjunctive effect to refractive correction for anisometropic amblyopia in young children. Further large-scale studies seem warranted.
A little background information is in order.
In a healthy visual system the various structures in the eye focus light ambient light to form a sharp image on the retina. The retina functions like the CCD in a digital camera. Light energy is transduced to electrical signals, which are transmitted through the optic nerve. The signals from the optic nerve are relayed to the occipital cortex (at the very back of the brain) where they are interpreted into the experience of vision.
Amblyopia is often called “lazy eye”. If during childhood the brain is deprived of quality visual information it does not mature properly and loses the potential for good vision. Any opacity within the eye which significantly degrades the quality of the image forming on the retina (such as congenital cataract) can cause amblyopia. Strabismus (misalignment of the 2 eyes) can also cause amblyopia. If the eyes are not aligned, the brain has difficulty reconciling the disparate images from the 2 eyes. In order to avoid double vision the brain may “ignore” the input from one eye, and corresponding part of the visual cortex will not develop properly. Extreme, uncorrected, refractive errors (nearsightedness, farsightedness or astigmatism) can cause amblyopia. Patients with different refractive errors in their 2 eyes, a condition known as anisometropia, can also develop amblyopia in one eye.
Diagnosing and treating amblyopia during childhood is important. If the cause of the amblyopia is corrected during childhood, the brain can “learn” to see properly. As the child grows older, treatment becomes less effective, and amblyopia can become permanent. Often, intermittent patching of the unaffected is also used in the treatment of amblyopia.
The study under discussion involves the treatment of anisometropic amblyopia and was performed at the Joint Shantou International Eye Hospital in Shantou China. All children were treated with appropriate glasses to correct the refractive error. Patients were randomized into 2 treatment groups. For the first 15 weeks group 1 was treated with glasses alone; group 2 was treated with glasses plus acupuncture. After the initial 15 weeks, there was a crossover, and Group 1 was treated with glasses + acupuncture and Group 2 was treated with glasses alone for an additional 15 weeks. Both groups were then treated for 15 more weeks with glasses alone. Vision was measured every 5 weeks. The total duration of the study was 45 weeks, but the primary endpoint was considered to be at 15 weeks.
While assigned to acupuncture, patients received treatment 5 days per week for 15 weeks. Each treatment consisted of needling of 5 acupuncture points. They are described in greater detail in the paper, but in layman’s terms, needle placements were Baihui (in the scalp), Taiyang (beside the eye), Cuanazhu (in the eyebrow), Hegu (on the back of the hand), and Fuyang (in the lower leg). There was no sham acupuncture, or any attempt at masking patients to treatment assignment.
Imagine yourself as a patient with an eye problem reading a consent form describing a “double- blind study.” Because of the possible misinterpretation, in ophthalmology we avoid the terms “blind” or “blinding”, as in “double-blind study” to indicate measures taken to hide treatment assignment. We usually substitute the terms “masked” or “masking”.
At the primary (15 week) endpoint of the study both groups had improvement in vision, but Group 2 had improved 0.77 logMAR lines more than Group 1, a difference reported as statistically significant. LogMAR is an abbreviation for the logarithm of the minimal angle of resolution. I won’t go in to the arcane and tedious explanation of what 0.77 logMAR lines means, but it is quite a small difference. If you are familiar with audiology, one logMAR line is analogous to 1 decibel. On a relative scale a .77 logMAR line difference is roughly proportional to the difference between 14 and 12 point type. Within the context of clinical trials, differences of this magnitude are usually considered clinically insignificant.
At 30 and 45 weeks (after the crossover phase), the vision for the 2 groups was essentially equal.
First, the authors should be complimented for many things they did right. The designed a prospective, randomized trial. They used standardized methodologies. They reported well-characterized study groups. The sample size was fairly high for a pilot study, and followup was quite good. The paper was clearly written. However, all these virtues are outweighed by the omission of a sham acupuncture treatment group.
Measuring visual acuity is a deceptively complicated task. Ideally, it is an objective measure of the resolving power of the entire visual system. In practice, measurement of vision is never completely objective. It depends on patients reporting of their observations. In general, patients are asked to read progressively smaller letters on a chart until they can no longer reliably identify the letters. Surprisingly little is known about placebo effects on visual acuity. It is well known, however, that visual acuity measurement is highly dependent on the motivation, attention, and cooperation of the patient. This is especially true in children.
In clinical trials, vision is usually measured under highly controlled conditions, with standard lighting, a standard eyechart, etc. Because motivation and attention are so important in vision testing, the technician measuring the vision can have a significant influence on a patient’s performance. Call it the “cheerleader effect”. A highly engaging, encouraging tester, can generally squeeze a few additional letters from a patient compared to tester who is apathetic. The expectations of the tester can influence the way they collect and report the visual acuity measurements. In order to minimize potential bias, visual acuity examiners should be unaware of treatment assignment of the patients they are testing.
The paper states that vision was measured by an examiner who was masked to patient treatment assignment. Can an examiner can be truly masked to treatment when one group is having needles placed in their hand and face 5 days a week, 15 weeks at a time? By the time the patient reaches the primary 15 week endpoint, he or she has had needles placed at the same 5 spots 75 times, each. There were likely to be visible clues at some visits in some patients. Bruising was reported as an adverse event in 7.2% of participants. More subtle marks seem inevitable in a higher proportion of patients. If acupuncture treatments and visual acuity measurements were performed in the same building, visual acuity examiners could be unmasked simply by seeing the acupuncture kids in the building 25 times more frequently than the glasses-only kids. Given the very small differences in vision between the 2 groups, the possibility that examiners were unmasked in a small subset of patients could be significant.
Another important bias in this study is the very different experiences of the patients during different phases of the study. During the glasses-only intervals, patients came in every 5 weeks and had their vision measured. During the acupuncture intervals patients were brought to the office 5 days a week for 15 weeks and had needles pierce the skin at 5 points on their body. If the placebo effect influences vision measurement, this would seem to be a very provocative protocol.
Remember, also, that in this study acupuncture was an adjunctive treatment. The primary treatment, administered to every patient, was refractive correction (glasses). In order to get the benefit of the primary treatment, the children had to actually wear their glasses. Compliance is a known issue in amblyopia therapy, and getting 3 to 7 year old children to wear glasses is not as trivial as it may sound. Children with amblyopia in one eye generally do not perceive that they have any problem with their vision, and often find glasses more an annoyance than a help.
By receiving acupuncture treatment 75 times over 15 weeks, the children and their parents received constant reminders of their condition, and to the fact that they were in a clinical trial. During these visits they are under the real or perceived scrutiny of eyecare professionals. Sometimes know as the Hawthorne effect, these factors are strong motivators for compliance with any medical treatment. In comparison, patients in the glasses only group came to the hospital once every 5 weeks to have their vision measured.
The authors of the paper did attempt to address the question of compliance with glasses. Parents were asked to keep calendars documenting the use glasses over the first 30 weeks of the study. The parents reported that every child wore their glasses >90% of their waking hours. The authors concluded that differential compliance in use of glasses between the treatment groups did not likely explain the difference in visual acuity. Perhaps parents and children in this study were especially compliant, but universal >90% compliance challenges credibility. Patient (or parent) reported compliance is a flawed metric. Can any parent know what their 3-7 year-old is doing 90% of the time? I could find no studies looking at the reliability of parent-reported diaries of glasses use, but a found a quite interesting paper looking at use of orthodontia headgear. The authors devised an ingenious way of objectively monitoring the actual hours of use. Parents estimated that their children used their headgear about twice as long as documented by the objective measuring tool. I suspect this study has some relevance to parent reported use of eyeglasses. The difference in vision between the 2 groups was so small, that even a small difference in hours of glasses use could be significant.
In the discussion the authors did acknowledge that omission of sham treatment was a weakness of the study, and concede the possibility that placebo effects may have contributed to their observations.
Finally, we did not include a sham acupuncture group, which is considered necessary for evaluation of the effectiveness of acupuncture for such conditions as pain control. Although VA testing is relatively objective and may thus be less influenced by psychological effects, we cannot totally rule out the possibility that part of the observed treatment effects were attributed to a placebo effect from acupuncture.
They go on to say that further studies using sham controls are warranted. It remains to be seen whether of not such trials will be done. Given the very small treatment effect seen in this open-label study, is likely that any measurable effect will vanish with appropriate controls and masking. The authors appear to anticipate this result as well, and have given themselves a preemptive fail-safe explanation.
In other words, acupuncture works, it just doesn’t require actual acupuncture. This gambit has been used before to explain the inconvenient truth of data showing that acupuncture works no better than sham acupuncture. This fantasy has also been deconstructed in previous SBM posts.
In conclusion, the authors of this study reported a statistically significant, but clinically insignificant benefit associated with the use of acupuncture as an adjunctive treatment for anisometropic amblyopia. Given the implausibility of the treatment modality, and the lack of masking, it is likely that the very small difference between groups is due to one or more sources of bias inherent in the study design.
by David Weinberg For Science Based Medicine