Q&A With Lars Bertram. Questions by Pat McCaffrey.
SRF: Could you give us an overview of the new and improved SZGene? What’s changed from the previous version?
LB: We’ve actually done a few things. First, we completely updated the database, adding nearly 200 studies that were published between August 2009 and today. As of 15 September 2009, there were approximately 1,400 studies included, 790 genes, and something like 7,000 polymorphisms. Now we’re talking 1,592 studies, 935 genes, and over 1,000 new polymorphisms. We did some routine data maintenance, going over the entire database to locate errors and fix entries that were wrong.
Part of this endeavor is to take all the published data and run it through a meta-analysis routine. Before the update we had somewhere in the order of 150 meta-analyses that we could do with the available data, and now, after adding these new studies, we’re looking at 250 meta-analyses that are available online. Before the update, we had 30 genes that contain at least one genetic marker that was significantly associated, and now we have 43 genes that fulfill this criterion in our Top Results list.
More importantly, we have implemented a way of grading the top results. Using criteria that were developed by colleagues at the Human Genome Epidemiology Network—the so-called Venice criteria—we take every significant result and assess it for the degree of credibility. We categorize them into three groups, with an A, B, or C grade to indicate findings that have strong, moderate, or weak credibility. Before, the top hits list was essentially a list of odds ratios in descending order from 1 to 30. Now, from those 43 genes with significant associations, we have 14 genes that, according to these criteria, show a strong epidemiologic credibility. Clearly not all of them will pan out to be true schizophrenia genes, but getting that A grade takes something for any genetic finding, and at least 14 of those fulfill this criterion and so have that “special something.” In essence, it’s a more sophisticated ranking of the top results.
SRF: Can you explain the criteria that you are now using for ranking?
LB: They are based on three different questions that are being asked. First, is the sample size that went into the meta-analysis large enough to produce a strongly credible result? This is the power of the studies.
Second, meta-analysis is all about aggregating data from independent samples or independent groups or both, so the second criterion is about assessing how homogeneous, how similar the estimates are that came out of these association studies. Do we have one study that says this is a risk and another study that says the exact same allele is actually protective? That would be considered a very heterogeneous finding. It would be like one study saying smoking caused lung cancer, and then another study saying smoking protects from lung cancer. That, too, would be considered heterogeneous. We apply a statistic across all the studies that are included in the 250 or so meta-analyses and assess the degree of what we call heterogeneity, but really what we test is how homogeneous the results are from these studies.
SRF: So you are asking if they are all going the same way.
LB: Exactly. The third criterion is how well the meta-analyses are protected from bias, meaning artificial influences that may have systematically made them look very strongly associated, but in fact they are not. Publication bias would be a typical sort of bias that most people know about. That’s the bias that positive studies, meaning ones with significant findings, get published more easily and quickly than negative findings, which may end up in the drawer of a researcher. Among other things, this is what the third criterion assesses: the possibility of, or the protection from bias.
There are three grades—A, B, C—as in school, and the lowest grade across these three criteria determines the overall grade. So, for a finding to be an A grade, it means it would have to get the best grade across all three of these criteria—it would have to have enough power, it would have to be homogeneous enough, and it would have to be well protected from bias in order to get an A grade. Fourteen genes of the 43 currently fulfill these criteria.
Most of the top results are actually C, which means there is weak evidence, or weak credibility, but still they are significant. I think adding this label of epidemiologic credibility also helps researchers from all kinds of fields to make a better judgment call on how much credence they should put into any of these findings.
Now, some of the C’s are clearly false positives. Likely so are some of the B’s and A’s, but the idea behind this is that the A classification should contain fewer false positive results than the C. That doesn't mean that there aren’t any real schizophrenia genes in the C category.
SRF: Looking at the current Top Results ranking 1 to 43, can you give us a feeling how exactly a gene achieves an A rank, and how, for example, did PGBD1 become number 1 among the A’s?
LB: It is number 1 because first it has an A grade within the three categories. After that, ranking is by P value. If P values are exactly the same, genes are ranked by effect size. PGBD1 was actually the finding with the smallest P value, meaning best statistical support of all the A grades. The difference between PGBD1 and the second one, neurogranin (NRGN), isn’t very large in terms of the P value, but you have to apply some criterion, and PGBD1 just happens to rank on top.
SRF: Previously, the genes were ranked solely by effect size, right?
LB: Exactly. There was no grading in terms of the Venice criteria for credibility or in terms of P value. Now we’re taking all these things into account.
SRF: Lately, there has been a lot of emphasis on copy number variants, and people are starting to look at rare polymorphisms. Can you say exactly what kind of variations you are cataloging and analyzing here?
LB: That’s a very good question. We’re still working out a protocol on how to include those copy number variants and other potential rare variants. This is a genetic association database, and you can’t do proper association analysis with very rare variants. Most of these CNVs that were published were just found in a handful of subjects. At least some of them seem to be private, meaning they just occur in individual families or cases, and while some may be in the same region on the same chromosome, it doesn’t seem to be that it is the same exact genetic defect. They may have the same effect, but they are not the same defect.
One of the things that we’re doing in SZGene is trying to group together in our meta-analysis studies that are looking at the same defect, that is, the same allele, at the same exact base pair in the genome. This is very hard to do with CNVs in general, which differ in size and sometimes location, and because they are rare. We feel these studies are important, and we think they should be very prominently displayed in SZGene, and we are working on a procedure by which we can include them, for instance, on a summary level. We won’t necessarily use the data because we can’t meta-analyze that, but we would like to display them so that they make sense and fit into our overall approach. That is still pending and is probably going to come up in our next update.
SRF: Will they eventually affect rankings or weigh into your analysis in some way?
LB: Probably not. You cannot do a proper association analysis on most of these, not of the sort that you do with SNPs, for example, or other common polymorphisms. These are rare polymorphisms, and they don’t lend themselves easily to a quantitative approach. For quantitation, you need numbers, and if you have one or two or three people who carry a certain CNV or a similar CNV, you can’t do that type of analysis. It is a gray area that is difficult to include in this context, and not in a quantitative way with the data that is available.
SRF: What is the minimum frequency of the alleles that you consider?
LB: Our official threshold is 1 percent. These estimates are always difficult to obtain, so if a variant is present at .99 percent, then we would probably still include it, but the rule of thumb here is any marker that has a frequency of greater than 1 percent in the general population, we would include in our database for sure and also for the meta-analysis if applicable. If they’re below that, and all the current CNVs are way below the 1 percent mark, we currently do not include them.
In addition to being 1 percent, in order to be included in the meta-analysis, there have to be at least four independent samples with available genotype data to have studied that same polymorphism.
SRF: Can you talk about the meta-analysis?
LB: We have completely changed our style of presenting the meta-analyses. In addition to updating the ranking and using a more sophisticated way of ranking our top results, we have changed the layout of the meta-analyses, and added some new functions to it. If you look at the meta plots as compared to ones we had posted previously, these contain a lot more information.
One thing we changed is that we used to run the meta-analyses just on the Caucasian ancestry subgroups. Now we’ve extended this to all kinds of ethnic subgroups, so if there is a polymorphism in samples of just Asian or African descent, and with sufficient genotype data, we run a meta-analysis and display it. If the result of this meta-analysis is more compelling than on any other descent group or in the analyses irrespective of ancestry, it would also go into the top results. This increases power and explains a few of the new top hits, which are most prominently seen, for example, in the Southeast Asian population.
SRF: Can you give an example?
D-amino acid oxidase activator (DAOA) is a good example. If you analyze all studies, or you just use Caucasian studies—and this is what we did exclusively before—there does not seem to be a significant effect. But when you just look at the Southeast Asian studies, you see that the summary odds ratio is significantly greater than 1, and actually is suggesting a relatively strong risk effect with an odds ratio of 1.18. So this would be something that we would have missed before.
Another thing that we’ve added to make it easier for users, because there is so much information on these graphs already, is the ethnicity of each study sample. In addition, an important criterion in assessing all of these meta-analyses is the heterogeneity or homogeneity, so we’ve added that I2 statistic, which is a relatively simple and easy-to-understand number, which we didn’t have on the graph before. Whenever that number is above 50, that means there’s a lot of going back and forth. Whenever it’s below 25, (you can find these cut-offs in the Methods), as is the case for the Asian studies of DAOA, where it’s 18, that means it’s a relatively homogeneous finding in these studies.
Besides making all these graphs look a lot nicer, we’ve also added cumulative meta-analyses. TPH1 is a good one to look at, which is number 6 in the top results. If we look under the Meta-Analysis, then you see the “new” flag. When you click on Cumulative Meta-analysis, the graph looks very similar to the meta-analysis, but the studies are presented in chronological order, with the older studies on the bottom, newer studies on top, with the results of the cumulative analysis as you go up. For example, if you look at the first study, you see estimated an odds ratio just around 1. So that was insignificant, and then the second study came along, and combining those two actually gave an odds ratio of 1.21. After adding the third study, the finding became statistically significant. Adding the next didn’t change the odds ratio too much. What you see on these graphs is that they wiggle around the final odds ratio, but eventually, as more and more data are added, the squares [indicating study size] will get bigger and confidence interval will get tighter, and you will see that these settle on a specific odds ratio which here seems to be on the order of 1.2. By cumulative, we mean we recalculate the odds ratio for each stage through the process.
There are more striking ones. If you go to MTHFR, which is also on the top list, and go to Cumulative Meta-analysis, you see now here we have a typical example where the first study (Arinami et al., 1997) is significant, and you have a relatively strong effect, but when you add more and more studies, you see that the cumulative summary odds ratio tends towards the null, towards an odds ratio of 1. In most cases, the odds ratio just stays on 1, and that means the initial finding was false, or there’s no effect. In this case, it actually stays clear of 1 all this time. It has been significant ever since the first study and the odds ratio has leveled into what it is now, 1.17. At the time of the Philibert study in 2006, the odds ratio was already at 1.18, so it hasn’t changed with the six studies since, but the confidence intervals have gotten tighter, suggesting that the odds ratio that we’re looking at for this particular factor is on the order of 1.17, and all that’s happening now is it’s getting more significant over time.
SRF: What about the data coming out of the Psychiatric GWAS Consortium?
LB: What they’re doing is fabulous, because they’re taking all the GWAS data and meta-analyzing them. There’s so much data that it deserves the term “mega-analysis.” But so far, there are only a handful of GWAS studies. The GWAS overview page shows there are currently 10 different GWAS studies, so meta-analyzing these will give you essentially the data of 10 studies, if the data from all were available to the GWAS Consortium. There are lots of markers, but it’s essentially just 10 datasets. And for many of these small effects, you might need way more samples to have sufficient power in your analyses. However, some of these, when you look at the sample size, are very small, with the exception of the O’Donovan and Stefansson and ISC studies, which are also the newest ones.
I think this is one of the strengths of our approach that we not only consider GWAS, but we also consider all other types of studies. In particular, for example, the Riley study of the ZNF804A polymorphism: that was the first independent follow-up of the GWAS finding, and it was actually confirmatory. Doing GWAS meta- and mega-analysis is extremely valuable, but then people will say, I want to see if this effect is also true in my sample that was not included in the GWAS, and they will add more valuable data. Maybe they’ll perform some fine mapping, identify the functional variant that was actually carrying the biochemical effect. And this is something that just wouldn’t be covered by the GWAS meta-analyses of the Consortium. GWAS is a great tool to pick out the most promising new locations, but then follow-up studies are needed, and they, by design, don’t go into those GWAS meta-analyses but certainly will be included in SZGene.
Another example concerning GWAS is APOE. It is the weakest of the A findings, and the most debatable. But, according to our criteria, it is an A finding. One of the major weaknesses, at least of the current GWAS arrays, is that they don’t have this particular allele on them. This marker is just not included in the GWAS. There are surrogate markers, but they’re not perfect and they have their own issues. That’s another reason why I think these are two complementary approaches—using GWAS data and also using data that were generated independently and on very specific and likely functional polymorphisms across many different samples. Once the Psychiatric GWAS Consortium publishes their results in a peer-reviewed journal, they will immediately be included in SZGene. Hopefully, the Consortium will publish the genotypes that went into their analyses, too, so that we can add them to our top results list as well.
SRF: The new graphs are visually striking and give a real appreciation for the numbers.
LB: Yes, it was one goal that we wanted to fulfill, not just post the top hits list. There are A findings that, in terms of the Venice criteria, are A grade findings, but they just don’t look like A’s. They just happened to make it. APOE may be one of those examples because it is relatively weak and just about making the A threshold. Looking at the cumulative meta-analysis, though, I would think it may be real, but more data are clearly needed to make that call. And then there are also C findings that have all the ingredients, visually at least, to believe this is more than just a C. For example, ZNF804A. I honestly believe that more than APOE, but it’s a different category. There are not enough data out there yet.
SRF: It seems to be a way to quickly put new data in context and evaluate if they bolster candidates or not.
LB: Yes, exactly. Things have already changed since we got the database back online, so the top results have changed, and the ranking of top results. There are another 10 studies or so that have been added since last week. That shows why it’s such a challenge to keep up-to-date. We have a team of four people, though not all working full-time on this, and if we have trouble keeping up-to-date, then individual labs have probably even more trouble. Hopefully, what we do continues to be useful to the community.
SRF: Is there anything else new that you wanted to highlight?
LB: We’ve added mitochondrial studies. Mitochondrial DNA is a completely different story, and in some ways similar to CNVs. You can’t really easily include them into meta-analyses, so we’re just displaying qualitative overviews. There won’t be any meta-analysis graph, but still, we have included this to give a quick overview of the studies that have been published on mitochondrial DNA, and their relationship to schizophrenia.
We have also updated the linkage studies. Carolyn Lewis’s group and colleagues published a very nice and very comprehensive review of the literature (Ng et al., 2008), which we have now summarized on our linkage page. If you go to the chromosome graphs, the little red line on the left indicates those linkage regions (see CHR1, for example). The idea behind that is if you have a gene that’s both associated and maps to a linkage region, this looks more promising and makes more sense for a geneticist than a gene that is way clear of any of these regions. While the argument that I’m more interested in the gene that maps to a linkage region is true, I wouldn't necessarily discard a gene that’s not mapping to a linkage region, because linkage also has its limitations. Anyway, we upgraded the linkage regions completely, adopting the new findings from the Ng paper.
To summarize, we did an all-around update on SZGene: more studies, significantly more meta-analyses, and a more refined way of parsing through and categorizing our top results. We have an updated, more informative, and more visually pleasing look to our meta-analyses. We got rid of the funnel plots we used to have in that spot for the cumulative meta-analysis and replaced them with what I believe is a more informative measure of meta-analysis results over a time, which is cumulative meta-analysis.
SRF: Any parting thoughts?
LB: First, I want to give credit to my group, Christina Lill, Brit-Maren Schjeide, Ute Zauft, and Johannes Roehr, who have worked very hard on this update and continue to work very hard on keeping SZGene up to date. Second, I want to emphasize that with SZGene, we are just collecting and analyzing the data that are produced by hundreds of labs around the world. We’re trying to put these data into a “big-picture” context, but the data obviously come from everybody in the field who has published. In a sense, we're something like a mirror of the genetics findings in this field. This also means that the data and results on SZGene can only be as good as that original data, minus the mistakes that we make. Through this update process, we fixed a number of mistakes, but we’re just humans, and with 1,600 studies, and over 8,000 polymorphisms, it would be ridiculous to say that it’s all error-free. Actually, there are probably a lot of errors. So here's a request to all SZGene users: if you see an error, please let us know, and we’ll fix it as soon as possible. If there are things we’ve overlooked, for example, studies that we’ve missed, by all means contact us! A lot of people have done that, and we’re very grateful to them. The whole project becomes more and more valuable with this type of input.
SRF: Thanks so much, and good luck.
LB: Thank you.