Friday, 15 July 2016

Finish line (nearly)

We are very close to the finish line $-$ that's being able to finally submit the BCEA book to the editor (Springer).

This has been a rather long journey, but I think the current version (I dread using the word "final" just yet...) is very good, I think. We've managed to respond to all the reviewers' comments, which to be fair were rather helpful and so this should have improved the book. 

Anna and Andrea have done very good work and I didn't even have to play the bad, control freak guy to have them prepare their bits quickly $-$ in fact, I think at several points, I've been late in doing mine... 


Here's the (somewhat simplified to only sections and sub-sections) table of content:
  1. Bayesian analysis in health economics
    1. Introduction
    2. Bayesian inference and computation
    3. Basics of health economic evaluation
    4. Doing Bayesian analysis and health economic evaluation in R
  2. Case studies
    1. Introduction
    2. Preliminiaries: computer configuration
    3. Vaccine
    4. Smoking cessation
  3. BCEA - a R package for Bayesian Cost-Effectiveness Analysis
    1. Introduction
    2. Economic analysis: the bcea function
    3. Basic health economic evaluation: the summary command
    4. Cost-effectiveness plane
    5. Expected Incremental Benefit
    6. Contour plots
    7. Health economic evaluation for multiple comparators and the efficiency frontier
  4. Probabilistic Sensitivity Analysis using BCEA
    1. Introduction
    2. Probabilistic sensitivity analysis for parameter uncertainty
    3. Value of information analysis
    4. PSA applied to model assumptions and structural uncertainty
  5. BCEAweb: a user-friendly web-app to use BCEA
    1. Introduction
    2. BCEAweb: a user-friendly web-app to use BCEA
Throughout the book we use a couple of examples of full Bayesian modelling and the relevant R code to run the analysis and then use BCEA to do the "final part" of the cost-effectiveness analysis.

We've tried to avoid unnecessary complications in terms of maths, but we do include explanations and formulae when necessary. It was difficult to strike a balance for the audience $-$ especially as it was complicated to define what the audience would be... I think we're aiming for statisticians who want to get to work in health economic evaluations and health economists who need to use more principled statistical methods and software (I couldn't resist in several points moving my tanks to invade Excelland and replace the local government with R officials...).

The final chapter also present and discuss the use of graphical front-ends to R-based models (eg as in SAVI) $-$ we have a BCEA front-end too. I think these may be helpful, but they can't replace making people in the "industry" more familiar with full modelling and away from spreadsheets and stuff (these work when the models are simple. But the models that are required are not very often that simple...).

We also present lots of work on value of information (including our recent work), which is also linked to our short course. May be it's time to link BMHE and this to do a long course... (there's more on this to come!)

Wednesday, 6 July 2016

Bad medical journal

This is an interesting story, I think and I have to say I'm sort of being inspired in the title of the post from a talk that Stephen Senn gave a while back at UCL. He in turn had been referring to Ben Goldacre's book Bad Pharma $-$ the book argues that the pharmaceutical industry is often guilty of cherry picking the evidence they use to substantiate claims of clinical benefits for their product, while Stephen counters that this is rather a two-way street and often respectable medical journals are just as bad.

So: a while back we have done some work on a paper analysing an intervention to facilitate early diagnose of dementia and then submitted it to one of the leading medical journals. Unfortunately, the intervention didn't turn out to produce a massive difference, but we thought it would be interesting anyway. The paper went out for review $-$ in fact it was assessed by 3 reviewers. As per the journal policy, the reviewers have to state their name explicitly, so that the whole process becomes more transparent. 

Now, interestingly, the 3 reviewers' comments were (I am copying their comment verbatim, but add my own text in italics below):
  1. The authors report a well-designed and well-conducted cluster RCT which addresses a subject of high clinical importance.  The findings are conclusively negative in terms of their simple informational intervention effecting earlier diagnosis. This is however an important and clinically useful negative finding, in that it demonstrates that a very simple intervention is not effective in enhancing timely diagnosis. This means that a more multifaceted approach may be needed to address the clinical and policy priority for earlier ab d better diagnosis of dementia. The change in ascertainment of primary outcome to include an imputed MMSE score from ACE data was necessary in the light of unpredicted change in clinical practice due to the prospect of incurring a charge from the copyright holders.  The method used is scientifically reasonable and will not have obscured the results of the intervention. This is a good strong paper with an important negative finding that warrants publication.
  2. THE MANUSCRIPT: [title of the paper] is very important and of high standard.I cecommend [sic] the publication in [name of the journal] without restriction. 
  3. This is a cluster randomized controlled trial of a simple intervention meant to empower patients and their families to seek early assessment for memory difficulties from their primary care physicians. The intervention which included an information leaflet and personalized physician letter was laudably developed with input from patients and caregivers. The experimental design was quite rigorous and utilized an appropriate sample size calculation. One might argue about the choice of the primary outcome measure and whether the intervention as designed could reasonably have been hypothesized to have families bring their relative to EARLIER attention or not (as opposed to bringing them at all), but all outcomes were clearly described and reported. Unfortunately, the study was largely negative. Besides the potential reasons suggested by the authors, not enough attention is paid to primary care physician attitudes to the value, or lack thereof, of an early diagnosis of a memory disorder, when there is a widely perceived lack of effective therapies. Until effective therapies are available, or until physicians can be convinced of the reasons why an earlier diagnosis is advantageous, earlier GP referral to a memory service is unlikely to occur even in the context of a highly empowered patient population.

I think one could argue that reviewer number 3 (remember, in the spirit of the open and transparent policy of the journal, the reviewer is named) is perhaps less enthusiastic than the other two. But I think I would be very happy to receive these sort of review for any paper I ever submit to a journal. And that you'd expect a request for changes to the manuscript (to be fair, reviewer 1 had some minor comments requesting clarification in a couple of parts in the text) but full consideration for publication, at this stage.



Well, that's not quite what happened, because the research editors overruled the reviewers and rejected the paper straightaway. Now, obviously, it is perfectly possible that the editors find flaws in the reviews (although I think you'd question the choice of reviewers, if all 3 turned out to be not up for the job, in the view of the editors who have appointed them...). But I think that their comments were rather out of place $-$ for example, they mention a "8 year difference between Intervention and Control groups", while actually the difference in age was less than a year (and they got confused with an 8 point difference in % of males).


I don't want to sound petty and, as I said, the journal have all the rights to reject the paper $-$ after all, we did acknowledge some of the limitations (e.g. we had important issue with missing data and we dealt with it using statistical methods $-$ interestingly, that wasn't a problem for the editors per se, although they said they thought we had "changed our primary outcome", while we simply imputed missing values for the primary outcome using, among other variables, a similar outcome variable, as mentioned by reviewer number 1). So I think it would have been OK to be told that, because of the issue with missing data, the results did not look strong enough. In fact, we did acknowledge the uncertainty in our estimation, but even after modelling missingness the results were negative, so I think that should have been less of an issue. 

Anyway, we did appeal (more to make a point than in hope of any real change) and yet again I was on the losing side $-$ I'm starting to think I should probably start supporting all the causes that I really do not believe in, so that I'll either win some of the future referenda to create as many city-states are there are in the world, or at least will see them fail and take some credit for my personal jinx...

Tuesday, 5 July 2016

Fire and mouse alarm

Yesterday we finally had our workshop on infectious disease modelling in health economic evaluation. I think the day went very well and we had very interesting talks (as soon as I can, I will upload the slides on the website above, for all the speakers who can grant us permission to do so).

The morning was the most technical part (as intended), but I think everybody was able to follow the talks (which at times had lots of statistical details), because they all had some practical results and nice graphical displays. 

The "industry" session was also very interesting and quite varied with different perspectives and problems being highlighted. 

Then, just after the start of the final session (which had speakers from NICE and the JCVI, to present the reimbursement agencies perspective), all hell broke lose: first, the fire alarm went off, for what turned out to be basically nothing $-$ I was trying to listen to the conversation between the UCL people and Fire Marshals trying to determine whether there really was a risk and I think I understood that they were getting really annoyed at the alarm going off for what was clearly nothing). 

Then, as we resumed the session, we had another interesting surprise. Shortly after the beginning of the second talk, a little mouse (probably disturbed by all the fuss caused by the fire alarm) decided to start roaming through the lecture theatre. I thought I caught a glimpse of something moving suspiciously while I was listening to the talk, but I made nothing of it $-$ until I saw other people looking away from the lectern and increasingly slightly disgusted... Eventually, the mouse got bored (and possibly scared) of the people and disappeared somewhere in one of the air conditioning holes. But that's Central London for you...

Thursday, 30 June 2016

Silver lining

Fivethirtyeight has just published their first prediction for the next US presidential election, stating that Clinton has around 80% chance of winning to Trumps' 20%. This has been also reported in the general media (for example here).

I think the tone of the Guardian's article is kind of interesting $-$ basically if first praises Nate Silver's ability but also points out a series of "high-profile misses that could lead some observers to discount their predictions this year". Author Tom McCarthy goes on to report on very wrong predictions for example on Trump's chance of securing the Republican nomination.

I guess this is such a fluid and dynamic situation that perhaps it's a bit too early to call a definitive outcome. But I'm sure we'll be bombarded with predictions in the next few months...

Friday, 17 June 2016

Workshop on Infectious Disease Modelling in Public Health Policy: Current status and challenges (yet, again)

I've written about this a couple of times already (here and here). We've finalised the speakers line up (see here for the latest information) and I think this is a very exciting programme! (I know you may think "well, he would say that, wouldn't he?" $-$ and you're probably right... But I do think that the line up is really interesting!).

Anyway, although the registration list on Eventbrite is formally close, I think we do have some places available $-$ so if you would like to go, send me an email (details on the website)!

My week at ISBA (2)

I should add to my previous post that while there have been many very good talks, I thought two were incredibly good: David Spiegelhalter's Foundational lecture on Monday and Adrian Raftery's talk in a session on Bayesian Demography that was very interesting overall. I'm not sure whether slides will be made available, but if they are, you should definitely check them out!

In particular, Adrian's talk was about the application of Bayesian hierarchical methods to formally account for and quantify uncertainty in population projections (I think this is the relevant paper). He started his talk by showing a screenshot from the BBC website reporting on Boris Johnson's claim that if the UK stays within the EU, its population will "increase to 80 millions" (from the current level of around 65 millions). 

As Adrian pointed out, it wasn't clear what time frame was Johnson referring to. However, I'd say unsurprisingly, his model showed that this event had virtually no chance of happening within the next 5 years and at most around 40% chance of happening in a period of 25 years [I am citing the numbers by memory, so the details may be slightly different, although they are most definitely in the right ball-park!].

My week at ISBA

I've spent the last few days in beautiful Sardinia for the ISBA world conference. The place is outstanding, really beautiful, although it's kind of weird that there is no real town along the cost for miles and miles. Leaving Cagliari and driving for over 50km, you only come across a massive oil refinery and the town of Pula. There are many resorts (some super high-end, like the one where the conference is, some more like very nice camping sites with tents or bungalows replaced by nice apartments), but no real town. I felt that was a bit weird as I wasn't used to things like that, in Italy $-$ but it may well be that I am just ignorant about my own country and there's tons of places like that...

As for the conference, my last ISBA was 10 years ago $-$ that was the last one of the original "Valencia meetings". Even then, it felt like a big conference, but that was nowhere near the level it has reached now (I think there are over 700 people here!). This means there are several parallel sessions and lots of heterogeneity in the topics. Also, the schedule is quite packed with talks from 9am to 7.30pm (with some breaks throughout) and then poster sessions after dinner. I was impressed by how well the organisation has worked: I've not seen a single session running late! 

My talk is later today; I'll be talking about our work on the RDD $-$ I've planned a rather high-level talk, showing some of the general themes we've developed and giving broad examples, rather than going to the details. Luckily, we do have a few papers on these so hopefully I'll be able to point people to the relevant references. 

More importantly, I'll have to rush off just after my talk (which isn't great) so that I won't miss my flight home $-$ I've tried to plan everything ahead, so I've put petrol in the car, checked out the hotel, etc. I'll give the talk with my fingers crossed, just in case...

Friday, 3 June 2016

Large enough probability?

I've deliberately stayed away from the Brexit referendum (that is in terms of spending time doing some modelling to see what the predictions may be) $-$ mostly because I was otherwise busy and didn't really have much time. 

However, I was talking earlier to my "polls guy" Roberto (yes: I have a polls guy) and he's suggested I took a look at Stephen Fisher's blog, who is predicting a 72% chance that the outcome of the referendum will be "remain" (in the EU). I think I'll take that as a good sign.  

In the past couple of days, people have put forward the argument that the UK should take their economy back and "do like New Zealand in the 1980s $-$ that was very good for their economy, so surely it'll work for us too". [start rather annoyed irony]Surely the global economy and trading system were exactly what they are today, back in the 1980s, so surely this is super relevant evidence and something we should be basing our judgement on, today...[end rather annoyed irony]