I am an emergency physician without any formal software training and for the last three months I’ve been trying to build a program that segments the wall of the heart from an ultrasound video and then identifies regions that aren’t moving (an early sign of heart attack).
There are many similarities between this man’s project and mine. And if I had his knowledge I may have cracked my problem by now and would have new way to detect heart attacks early.
Hi! You should consider contacting Stephen Aylward at Kitware. His corner of the company specializes in connecting clinicians with expertise in point of care ultrasound with machine learning programmers paid through Kitware, funded by NIH grants. I worked for him for the last three years, focused on assessing intracranial pressure with ultrasound and assessing pneumothorax with ultrasound.
For intracranial pressure did you look at ocular ultrasound? How did that work out? Did you have ICPs from actual bolted patients for the gold standard / ground truth? That would be incredibly useful especially in patients on a ventilator who can’t provide a neuro exam.
Pneumothorax seems tricky since really it requires a lung point for diagnosis, and those can be hard to find. Did you simply look for lung sliding? That’s all a physician really needs to make an informed diagnosis.
I love hearing about this. Thank you for the work you do
Do you realize that there are thousands of scientific publications in medical image processing that address exactly the same concrete problem that you have?
Yes, however much of that research is for formal ultrasound obtained by a professional sonographer with an expensive machine. I'm interested in bedside ultrasound performed by an emergency physician with a mediocre machine (eg butterfly).
It seems the primary way to detect regional wall motion abnormalities is with speckle tracking, which requires way too much post-processing for a clinician.
A system that segments the left ventricle and finds akinetic regions in realtime from a parasternal long axis view or an apical four chamber view would be pretty nifty.
If you know of a paper or system that does this now then please let me know. I would love for someone else to have solved this, haha.
I love this comment deeply, because it is a view into someone else's world, someone who is chipping away at real problems and making "the future" happen. A future where more people are saved from death through incredible-yet-easy-to-use technology.
This paper might be useful [0]. It measures the thickness of a tissue. If you do that over time you can detect the parts that aren't moving. I studied under Yezzi for a while. Cool dude.
Awesome, thank you for the reference. That's the essence of finding regional wall motion abornomalities. When looking at the ultrasound I look for parts of the myocardium that don't change in size relative to their neighbors during the cardiac cycle.
If you have any examples of what you're talking about, can you send them to me? That sounds like something I'd love to poke at. My email is my username at gmail.
Correct me if I'm wrong (I haven't worked with health data science like this) ... wouldn't an end-to-end approach work better from the get-go?
What I mean is, rather than developing a segmentation algorithm and then a motion detection algorithm, why not just feed a bunch of frames into a CNN and have it directly predict "heart attack risk"?
Or is the segment-then-motion-detect approach necessary because of its better explainability?
I guess I view the end-to-end approach as being less fiddly than the more traditional computational imaging approach. And it has a bonus. If data is available, you could feed it historical ultrasound data from patients that later had heart attacks. With that, it's possible it will learn other features of an ultrasound that predict future heart attack.
That would require a dataset of ultrasounds from people having active myocardial infarctions, which we don’t have, and would take at least a year of academic coordination to assemble.
The current datasets are just labeled anatomy at end systole and diastole.
Hmm, curious. I suppose I would wonder, then, if there's no dataset how any algorithm could be developed. What I mean is ... once you've developed something, how do you test how well it performs? How do you analyze the effectiveness of the algorithm, the false positive rate, etc.
I could assemble a small dataset of less than a hundred ultrasounds to test the algorithm. A big dataset that could train an AI would require quite the effort.
Great questions, and you highlight the need for shared ultrasound data.
Fast.Ai is incredible. Love it. I've bumbled my way through it and was able to make a unet learner trained from this dataset -- https://www.creatis.insa-lyon.fr/Challenge/camus/ -- in order to segment the left ventricle from an apical four chamber view. It's a start, but I still have a long way to go.
I grew up making PB&B by mashing the banana and peanut butter together in a bowl, tuning the ratio to taste at that time, and then applying the mixture to the bread. I'm starting to think maybe this was not the usual way?
I also mash the banana with peanut butter, but instead of putting it on bread, I put it on steel cut oats. Then add a small amount of milk and mix. Finally, top with honey.
Wow this guy can really see something through to the end. I have had some good ideas that I give up on once things become complex but he carefully saw this “meaningless” but super difficult project through from beginning to end. That alone is a sort of sacred skill, regardless of the task.
This is something I struggle the opposite of. When I dive into something, I get super focused, to the point of it severely disrupting my life. I had a few projects in grad school that kept me up for like 2 days straight even though they weren't due for a few weeks. I think the author has that same level of manic focus. It's honestly why I can't be a programmer, it's not that I can't write good code, it's my head space. I can't achieve a level of Zen that all the great programmers seem to have, I get super super focused, and then too emotional.
As someone with ADHD, this sounds like an ADHD symptom called hyperfocus. Note I am not trying to e-diagnose you with something, I'm just comparing the similarity of your description to the description of an ADHD symptom, not saying I think you have ADHD.
I make that comparison because if you do a search for hyperfocus, you may be able to find strategies to help you break out.
For me, hyperfocusing can be so strong that hours will pass with me barely noticing. I often say that the house could burn down around me and if I am hyperfocusing, I'd be toast. I don't get it for multiple days as you describe, but it's not unheard of for others. My dad is a metallurgical engineer, and described something basically exactly the same to your experience with programming. He had some lab course involving programming freshman year and stayed in the lab for 24 hours straight obsessed with a project. When he broke out of the 'trance,' he realized that he could never be a programmer full time because it would completely take over his life.
For others who maybe aren't familiar with this feeling, it is most definitely NOT a pleasant feeling. I always come out of that state feeling almost sick to my stomach, headachy, dissociated. Not something I enjoy, and not really something I can direct to be more productive.
----
(Note that it may sound odd to describe hyperfocusing as an ADHD symptom -- isn't ADHD about a deficit of attention? But actually ADHD is kind of a misnomer; a better name might be "Executive Functioning Disorder." ADHD is more about being unable to direct or control your focus, and this can work either way -- to make you appear less focused, or severely over-focused. In particular this symptom is not a side effect of medication, and in fact medication can help to break out or prevent you from being sucked in in the first place.)
That sounds like a gift haha. It would definitely not work well for most programming gigs (sprint style, kanban board, small estimated tasks with regularly reported progress), but in the right niche you could do stuff that most of us can only dream of.
You could take the chef approach by picking straight banana's and squaring them off first. The sizing would be pretty consistent, the leftovers can go in a shake, or frozen for a smoothie later and you would have a higher percentage of perfect coverage :)
+1 for freezing leftover bananas. Freezing ripe bananas just before they over-ripen is the greatest life hack I ever learned. I can't believe it took me so long to stumble on it.
Slice ripe bananas into a reusable sandwich bag, pretty much in a layer (not a frozen clump) so you can break off a part of the layer, and freeze. Later semi-defrost and throw in a smoothie or protein shake. Never throw away an over-ripe banana again.
My optimal is a tortilla, banana, peanut butter and honey tube wrapped in parchment paper or aluminum foil.
it's one of the few sandwich formats that tastes better squished, and it stores very well, so it's kind of the ideal food for stuffing into your bike jersey or ski jacket and eating on the go.
Excellent project. Very well written and executed. Love it.
I've found that I can get excellent consistency in coverage by slicing the banana vertically to get long rectangular slices and then laying the slices side-by-side - generally about 4-5 across.
Over many years, I have highly optimized my PB&B production process:
1. Apply peanut butter to bread. (Copious amounts. I prefer Adam's.)
1. Peel banana and cut length-wise. Place one half across top of bread where the loaf curves. Use curve of banana to match curve of bread. Cut off part of banana that hangs off.
2. Invert other half of banana and place it parallel to first piece, slightly tucked under. Cut overhang.
3. Take the two overhanging pieces, and place them next to each other on the remaining third of the bread.
With a properly sized banana and loaf, this provides almost perfect uniform banana coverage, requires only three cuts, and dirties only one knife.
If you were a machine learning model (or my wife), then you would tell me to just cut long rectangular strips along the long axis of the banana, but I’m not a sociopath.
I initially assumed the challenge was to fit the entire banana on the bread in a single layer; I am disappointed that problem was not solved, but it seems like it should be a relatively trivial extension of the package provided (produce a packing for [N, M] slices of banana, pick the number of slices which covers the bread without any left over pieces).
(Of course, as someone who does not eat banana sandwiches, I have no idea of such an end is even desirable.)
If you want to make a PB&B that uses the entire banana, simply peel the banana, coat the flesh in PB, tear up your bread slices and adhere them to the banana.
This method is not guaranteed to use all the bread.
Smear peanut butter on the bread, NOW tear up the bread, and apply the peanut-butter-smeared shreds to the (peeled) banana.
Still the same primary advantage as pliny's method, same primary drawback (may not exhaust the bread), but we've solved the structure problem that you raise.
I feel the trend-line here is looking good; with a few more iterations, we may reach the true ideal PB&B sandwich.
Yeah, my grandmother would mash the PB and banana together into a delicious, homogenous mush and then spread that one the bread. Thanks to the magic of imprinting, I believe this is the Correct way to make a PBB, and if you disagree, I'll cut you.
Despite its obvious optimality, the Homogenous Mush approach creates extra dishes. So, as a lazy undergrad, I used the circles method for a long time. Sometime in grad school I had the long-cut epiphany, and tend to do that most of the time now.
(The other downside of the circular method is that they're more likely to try to jump out of the sandwich, since they have less contact area.)
Oh, and one other solution, which is great for long bike rides: Switch out the bread for a flour tortilla, and throw some honey in the mix as well. Fits in an undersaddle bag and beats the hell out of a clif bar.
Speaking of Grandmothers-- my Grandmother didn't bother with the peanut butter. Instead we had banana and mayonnaise sandwiches. I remember loving them as a kid, but I haven't had the guts to try them again as an adult.
I think the urban legend version of this is "just wear gloves"[1].
I do not understand the whole bread and bananas thing. Do as the wife says and make a slice down the middle. add peanut butter and put the two halves back together, and slice in segments. No bread is best.
Would make a better sandwich too as there would be fewer gaps. Could also stack more readily using the entire banana if one wishes.
Honestly, I never considered slicing that way before so besides an interesting read I also learned a new technique for making these kinds of sandwiches!
I love a good PB&B, but I'm with the other commenters who just smash the banana onto one of the slices of bread.
Lately my go-to PB&* sandwich has been peanut butter and kale. Yes, PB&K. Toast the bread, spread peanut butter on each slice, pile up some baby kale in the middle and smash it together.
No machine learning needed. Sometimes brute force algorithms are best!
To make a really great PB&K or just about any sandwich, if you're in the Palo Alto area visit the downtown Palo Alto farmers' market on a Saturday morning and get either of:
1. The big fat sourdough English muffins from the bakers across Gilman from the produce vendors, or
2. Any loaf from the bakers on the Gilman sidewalk next to the produce vendors, or
Better yet, just get both!
Now a pro tip on peanut butter. We usually get the salted crunchy from Trader Joe's, but it is a pain to stir it up when you open it. The "microwave for 30-60 seconds before stirring" trick helps, but I found something even better.
This was in one of those "71 weird tricks" listicles, but you won't believe what happened next: it turned out to actually work!
Store the peanut butter jar upside down. Then for a few days before opening it, shake it up a few times each day and keep it upside down. After a few days of that, it will be easy to stir.
Another PB&K fan here - but I drink them instead of eat them. Throw kale, pb, some mineral salt and a sweetener of choice (I use erythritol currently) into the blender and blend until texture is smooth.
You can smash a pitcher full of kale leaves and balance it out with just a tablespoon of PB. Not that making everything taste divine is always the end goal, but it was surprisingly difficult to find an ingredient that balances out the taste of kale.
While this is technologically awesome, I want to propose an alternate method. What if one gets a sufficiently large/long enough banana that by cutting off the ends, it's essentially cylindrical? One could size the banana to the size of the bread, coat the bread in peanut butter, then wrap the bread around the banana burrito style.
Maybe the banana ratio is too high in that case. I'll have to try it.
Additionally, sometimes I'll take a slice, break it into smaller pieces, and put the smaller pieces in-between the gaps of the other slices. Or I'll be a monster and just layer the slices...
My wife just came up with the idea of slicing the bananas sides off to make the slices square. That way you'd also be able to pack in that banana far more efficiently.
Others also have mentioned mashing the banana and spreading it.
As expected scientists never talk about the actual taste. You probably want ripeness and the peanut butter/banana sugar ratio to make sure banana flavour doesn’t overwhelm the entire taste profile.
One technique I've seen for tomatoes is that if you cut two slices in half you can make a square with the rounded edges facing in towards each other: I'm sure you could do the same with bananas. You could possibly extend your algorigthm to match similarly sized slices to get the most optimal pairings.
For the ultimate PBB experience,after assembly into standard sandwich form,grill said sandwich like a grilled cheese. After removing from the pan, spoon spread a mixture of cinnamon and sugar (small amount is my preference)and consume with a fork. The PB and banana are slightly melted. You could become addicted!
I would suggest further optimization, in that one should choose from ALL slice segments for the optimal coverage. We all know if nobody eats the rest of the banana right away, it's just going to go bad. Sure you think you'll make banana bread later, but you won't.
Okay but if we're taking radial slices instead of parallel slices one side will be thicker than the other, an actual optimal sandwich would rotate the slices so that the sandwich was ~equal banana thickness for every bite.
An optimal pb + b sandwich must contain not just circle pieces, but pie slices of circular slices at well. All those gaps left by the circles could be filled with little fractions of circles.
My father solved this problem before I was born in 1969. Peel the top of the banana as if you are going to eat it. Put peanut butter on the knife and wipe it on the top of the banana. Take a bite. Continue until the banana is gone. Ignore the bread. It more keto that way anyway.
If you really want to use the bread, mash the banana and spread it on one slice, spread the peanut butter on the other and put them together.
Peanut butter is visible in the last picture, which is the only one where he has actually placed the banana slices. I am not sure whether it is best to apply peanut butter to both slices or just one; I would suspect both, but we may not be seeing a finished product.
Having a function that calculated a coverage score at the end would also make it easy to take a baseline measure of how effective people randomly placing the slices are. I think a comparison between those and the software solutions would add to the comedy of the project.
I would modify the fitness function to do something like subtract from the score the area of the largest circle which could be drawn over a bread-only section. This would deal with your having a very unsatisfying empty bottom right corner in your last example.
> I also take the slices in the image above to represent the major axis of the banana slice ellipse, and assume that the minor axis is 85% the size of the major axis.
I don’t think they’re really that elliptical in practice. Banana slices are nearly circular.
Totally unrelated but my current favorite breakfast is a peanut butter banana milkshake. < 2 mins prep, well balanced and doesn't ruin your tastebuds. Add some protein powder if needed. The bestest breakfast on earth.
Sauté the banana slices in caramel, serve inside Cinnamon and sugar coated, lightly toasted and buttered brioche, with Sugary peanut butter like jiffy/skippy
Dip in honey. Enjoy with a glass of milk or cup of coffee
I prefer jelly/jam for PB&J sandwiches on bread, but long-sliced bananas when I'm making rice-cake PB&banana snacks. Round rice cakes & curved bananas are a different issue.
Elvis Presley used to rouse his crew in Memphis, herd them onto his private jet, and fly to Denver to get what he called the greatest PB&B in the world.
> If you were a machine learning model (or my wife), then you would tell me to just cut long rectangular strips along the long axis of the banana, but I’m not a sociopath
> If you were a machine learning model (or my wife), then you would tell me to just cut long rectangular strips along the long axis of the banana, but I’m not a sociopath.
Ignoring that you called your wife a sociopath for the moment... I'm one for peanut butter and pickle sandwiches, personally. I've made some danged optimal ones, too, slicing them lengthwise. Pickles are interesting because you can bend them and make a curved cut with a straight knife, and the slice lays flat. And then your packing problem has an added constraint because the slices are beveled. You need a smaller number of slices, so picking the proper thickness is actually the hardest part. Of course, I eat my mistakes, so the reinforcement phase is rather busted.
Slice 1 in your final packing typifies my complaint with slicing on the round: it will fall out of the sandwich. This is especially true with pickles. Lengthwise slices result in a more structurally resilient sandwich.
OTOH bananas do have an easy mode: smash and spread. If your wife was actually a sociopath, that's probably what she'd do. But maybe just don't badmouth your wife; our industry has a misogyny problem and that's an ugly pattern.
There are many similarities between this man’s project and mine. And if I had his knowledge I may have cracked my problem by now and would have new way to detect heart attacks early.