Measuring in Echocardiography is hard!

The paper ‘Automated echocardiographic left ventricular dimension assessment in dogs using artificial intelligence‘ was published last month. On the face of it, it seems like it’s a paper all about artificial intelligence, but I would urge all users of echocardiography to dig deeper.

What did this study do?

In this study, a team of veterinarians and echocardiographers collaborated to label hundreds of canine right parasternal long axis images, in order to train artificial intelligence to perform these measurements itself.

Then, a group of expert veterinary cardiologists labelled a different, smaller set of images, which served as the ‘gold standard’ to which any artificial intelligence must perform. The performance of the trained neural network (the AI) was then compared with that of experts.

I don’t use AI for echo; why should I care about this?

Apart from the main conclusion that artificial intelligence can be trained to perform as well as experts, the other important take-away from this paper is just how much spread there is in expert opinion. This is so important for beginner and intermediate level users of echo to appreciate. As someone who spends a lot of time teaching echocardiography, I get very frustrated at the number of papers that stress how reproducible various measurements are, when they must surely know that the research environment and the operators they choose does not translate into clinical practice. These statements are misleading, and can make beginners think a measurement is “easy” and “trustworthy,” when very few measurements we take in echo are either of these things.

This paper is transparent in its demonstration that even among the world’s top veterinary cardiologists, there are differences in opinion and methodology, even for this “simple” linear dimension measurement that everyone performed on the exact same images. Undoubtedly, all of these vets would be highly reproducible with their own colleagues in their own institutions, which for their patients is all that matters; but for the first opinion vet struggling to find consistency, this hopefully provides reassurance that – particularly given the current lack of published guidelines – these measurements aren’t straightforward, intuitive or common sense; if you’ve been feeling uneasy about certain measurements you are performing, you are right to; and sometimes, the best use of echocardiography in challenging patients is as a subjective tool to rule severe pathology in or out. You do not always need to perform a measurement, and it isn’t a failure if you decide not to. It’s responsible veterinary medicine.

Steps for checking your own reproducibility

If you really want to know whether or not you can ‘trust’ the reproducibility of a measurement, you can check yourself. It’s a good idea to do this periodically. Here’s how:

• Take 3 different captures, removing your probe between each one;
• Then select the right frame and perform the same measurement on each one (you can do this when the patient is gone).
• How different are your answers, and what does that mean for that particular measurement? 3% variation in ejection fraction measurements are unlikely to have massive clinical significance, but even a 1mm change each time you measure wall thickness in a cat could make a big difference to your diagnosis. By simply making yourself aware of this, you will no longer want to rely on a single measurement (if you currently do!). For measurements in which you notice a clinically significant difference, consider making the practice of taking multiple captures and multiple measurements standard practice, every time you echo.

If you have colleagues also performing echocardiography within your practice who may very well follow up one of your patients at a later date, or vice versa, you can do the same with interoperator reproducibility: comparing your measurements with your colleagues’ measurements.

Having everyone who is scanning the same patients singing off the same hymn sheet is more important than worrying about international differences in measurement methodology, because it is reproducibility within your own practice that is going to allow you to follow up patients over time and know that changes are real and meaningful.

Sounds great – but I’m already short on time!

Repeating measurements is time-consuming, but there are (currently free) tools which can help. Use the contact form here to request more information on reporting software that can speed up your measuring and reporting, and improve confidence.

Want to learn echocardiography?

If you are learning echocardiography right now and would like to accelerate your progress with learning modules dedicated purely to echocardiography, and expert support whenever you need it, you can book a call with us so that we can see if our programme is a good fit for you. Even if it’s not ideal for you right now, we will still give you guidance on steps you can take for whatever challenges you are facing right now with cardiac ultrasound.