L'esprit de l'escalier

Actually, I don't think I disgraced myself too badly, but reading back over the transcription of today's disagreement with a professor from an ivy league school, I feel like there were some points that I should have elucidated more clearly, if only I'd known I'd want to raise them... I'm going to repeat most of the exchange here verbatim, but I reserve the right to edit minimally (i.e. no ums, ahs, or repeated you knows - his, not mine!, and no-one but me will be identified).  This took place about two and a half hours into a four hour session on 'Cost Effectiveness and Decision Analysis', so it will also give you a flavour of the hell I have been enduring for the past three weeks of sitting in on similar lectures and being told the same sort of thing.  It's a wonder I haven't broken before now!

Me: Can I ask, do you see these decision trees as being some sort of post hoc rationalisation, or would...

Him: Oh God, no! Heavens, no! No.

Me: So, you would expect somebody to sit down and...

Him: Yes

Me: ... work out all these sums...

Him: Yes

Me: ... and not be able to intuitively make a judgement on all these factors?

Him: Well, on every patient, one at a time? No.  Although there are people [that have] this notion that there will be a day, there might be a day when just as you can go to your electronic health record and get information about patients like this, you can tap into a program that will use a pre-constructed decision tree and give you a result, not that that's what you have to do, but it will give you the information in real-time.

Me: OK.  But, so what is the value then of having a profession that have learnt a practice, rather than going to a computer and putting in your own symptoms and then having a print-out saying 'this is the test you should take' or 'this is the treatment you should have'?

Him: [...] you know the mechanisms.  Why do certain tests work?  What do they do?  What are they measuring?  What's the '-iology'?  What's going on at a cellular level?  You want to know what's going on inside that black box, it's important.  That's one answer.  Another answer is that, and I said this earlier but I'm gonna say it again, the main advantage of doing decision analysis is not to get the number.  The main advantage is to sharpen your thinking.  So, yes...

Me: OK, but there's other ways to sharpen your thinking than being entirely 'systematic' and positivistic about your approach to problem solving, I would have thought.

Him: Say that again?

Me: There's other ways to sharpen your thinking and consider alternative options than taking one particular track.

Him: Well, I mean, if you want to think systematically and clearly about the circumstances under which getting the test, ordering the test is a good thing to do...

Me: But, systematic thinking does not necessarily mean that you have to be probabilistic in your approach.

Him: I suppose not, but it's one way to be systematic...

Me: So, it's one alternative.

Him: You could make decisions based on other criteria than expected outcomes, but eh, this is one way of doing it...

This brought the group off on a bit of a discussion for a few minutes, but about five minutes later, he starts lecturing again and pointedly comes back to me.  I apologise in advance for what you're about to have to skim over, but I had to listen to it several times (once live and then a few times for transcribing) and I have no sympathy for you and make no apologies for what I am about to subject you to, even if it is a bit like water boarding...

Him: (to me) But I think your question was 'why do you need to know what goes on inside the black box?', and the answer is probably, maybe you don't.  But, you know, it's usually a good idea to have some vague idea if you order a test why you're ordering that test, what a positive result means [...] I don't expect you to go home and do this; I expect you to be able to use the software and have an increased appreciation of what the software is doing, which may make you a more informed user of the software [...] If you just pick up the software package and plug in numbers, you might not know what some of those options mean.  You know, these days,  people buy software and they don't know what half the options do.  They might tinker with it but I'm trying to give you a sense of what you can do with the software.  There's a whole branch, sort of an extension, of decision analysis that I didn't get into called Markoff models, which are sort of decision trees that sort of recycle themselves.  That, you sort of have an event and then you go on, you make another decision and then you have another possibility of an event and things go on and on and on.  m.  There are other extensions that involve simulating individual patients, simulating the probability, basically rolling dice for each patient in a thousand patients to get a sense of what would happen in a thousand patients when you're developing strategies, em.  These are things that the software can do and you don't really need to know how it does it, you just need to know that it can do it and rolls up those kinds of analyses

Me: But then there's a whole other bunch of black boxing going on there about how the software is written and developed and how that program, what biases are built into the program.  So, there's black boxing no matter what you're doing...

Him: I don't want to be critical of your point, but there are people who do statistics and don't know what the hell they're doing...

I'll leave it at that.  As I re-wrote this, you have no idea how much I wish I could annotate and comment on several different things within his (interrupted) monologue.  I'll make do with a comment on how this ended.  He 'didn't want to be critical' of my point, the dirty liar!  He not only completely dodged a valid criticism (open the black box on medical decision making by piling it all into a black box of software-driven analysis tools?) by refusing to engage with the question, instead going off on a rant about how people misuse statistics (using a t-test when they should've used a chi-squared test, fr'instance) to redirect (misdirect?) attention from the point I raised.  If I were a stronger person, maybe I'd have pursued it, USPOS-style, but I felt that I had taken my share of class time and I was not feeling a ground-swell of support.  Tough room.

I did at least feel like I had stood my ground, somewhat admirably.  Next time, I won't be so nervous and the arguments will be better honed from the experience of sharpening them against such opposition.  There will be a next time.