Moderna follows Pfizer with interesting vaccine news: how to read these dramatic developments

It is very exciting to hear another positive story about the results of vaccine studies: a good vaccine is the most likely way to end the pandemic.

Last week, Pfizer’s interim results suggested that its vaccine reduces COVID-19 cases with 90% effectiveness. Now Moderna fared better, with interim results showing nearly 95% effectiveness for its vaccine, with hints that it may protect against serious disease. Neither reported serious safety concerns and tested their vaccines on tens of thousands of participants.

With so many COVID-19 vaccines in development, more results are likely to follow in the coming months. Headline figures can, like these, be very impressive, but you need to dig deeper to find out exactly what the new results mean.

With that in mind, here are six questions to ask about any new vaccine trial findings.

1. Does this mean that the vaccine is safe?

Almost certainly yes, if it has successfully passed a Phase 3 trial with thousands of participants. Vaccines don’t get to this point if there are big concerns about safety.

Historically, pharmaceutical companies have been able to suppress negative results, but it is now legally required that all studies publish their results for other scientists to review. As a result, the industry is generally much more reliable than it was in the past, although we should still be cautious if only provisional results are reported.

Some people fear that COVID-19 vaccines have been produced with unprecedented speed; however, the vast majority rely on platform technologies with excellent security profiles. Some newer technologies are in use, but the clinical trial and regulatory process is extremely rigorous and will pick up most of the potential complications quite early in development.

Of course, it is still difficult to know the long-term side effects, but these are rare for vaccines and any risk is normally significantly less than that from vaccinating the disease.

2. Do the title figures reflect what the study was designed to measure?

Studies often measure many things, but there is always a single primary research question or goal that a study is designed to answer.

The tests will also include several secondary research questions, but answering these is not considered a sign of success. If you try out enough different goals, some will always be achieved due to blind chance. Misrepresenting evidence in this way is a form of research misconduct called p-hacking. You can find out the primary and secondary objectives of any trial by checking a clinical trial registry.

Again, it is important to consider whether these are interim results. While such results may be promising, as Pfizer and Moderna have shown, there is no guarantee that they will be the end result.

3. Did the process measure the right thing?

Determining what counts as a “working” medicine or drug can be quite complicated for many diseases. But for vaccines, the question is quite simple: Did people who had the active vaccine contract the disease? Any measure more complicated than this (often referred to as a surrogate outcome) should be treated with caution.

4. Who has the vaccine been tested on?

Are the results of a trial transferable to the real world? Here it is important to understand the difference between a population (in this case anyone can catch COVID-19) and the sample of that population that took part in the trial.

In many cases, tests use two carefully matched (and therefore comparable) samples under carefully controlled conditions. One is given the vaccine and the other a placebo (such as saline injection or a vaccine already developed for another disease) to control the effect of participants who think they have been vaccinated, which has an effect .

In Phase 1 trials, safety concerns indicate that samples are generally young, fit people with few health problems, who are likely not representative of the general population. However, as the evidence progresses into the later stages and gets larger, researchers try to ensure a more representative sample of the population.

This is why final phase (phase 3) trials are so important, as the sample is chosen to represent the population that the vaccine is targeted at. Formal publications of study results usually provide a table describing who was in the sample and often the effectiveness rates for different groups (broken down by gender, age, and so on). Unfortunately, the data on the effectiveness of the title (e.g. 95%) may not be applied uniformly to the entire population.

This is very important for COVID-19, as we know older people are much more vulnerable. Therefore we shouldn’t read too much into the results until we can see an age breakdown for effectiveness.

5. Will the vaccine be usable?

Before we get too excited, we need to ask some practical questions. How much will the vaccine cost? Can it be produced in bulk? Is it easy to carry and store? And how many boosters will be needed? These logistical problems (for example, the need to be stored and transported at very low temperatures) can easily prevent a new vaccine from entering the clinic.

6. Can we trust what is reported?

It is an increasingly important skill to identify between reliable and unreliable sources. Social media is often superficial and prone to spreading disinformation. On the other hand, journal articles and clinical trial records can be difficult for anyone but specialists to interpret.

Trust journalism is the answer. Look for publications with editorial supervision and a proven track record of trustworthy scientific and medical reporting. Reading more than one interpretation can help you get a balanced view.

It is also important to ask where a reporter found the information he is reporting on. Referring to the results published in peer-reviewed journals is a good sign: it shows that a rigorous fact check has been performed. Be careful if the main sources of an article appear to be preprints (documents not yet peer-reviewed) or other so-called gray literature, such as press releases or company reports.

Likewise, be careful if the main source seems to be interviews or quotes from people with PhDs or impressive sounding job titles. A quote from a scientist in an interview is not the same as a quote from the same scientist in a peer-reviewed academic article.

This article was republished by The Conversation under a Creative Commons license. Read the original article.