Hacker News new | past | comments | ask | show | jobs | submit login
Raspberry Pi Pet Detector Camera Using Python, TensorFlow, and Twilio [video] (youtube.com)
277 points by viniciosbarros 7 months ago | hide | past | web | favorite | 79 comments

I have a capacitive touch button that my dogs press when they need/want to go outside that makes a noise and sends me an SMS. They used to have all hours access to it and when they expected me home at the end of the day, I would start receiving texts in rapid succession. Access was removed shortly thereafter. The novelty of getting drinks after work and receiving a "Where are you?" text from my dogs every three minutes wore off quickly.

I just imagined reading your comment from the perspective of a layman 20 or 30 years ago. It's amazing how far we've come as a species!

that's hopelessly optimistic. so dogs can send texts. so what? we still clamor for power and ignore the consequences of our actions at every level and in every facet of life. we are hopelessly animalistic and primal in our social and environmental dealings, which is not offset by technological advances which often amplify our shortcomings.

We can appreciate that one of us was patient enough to set those burning desires for power aside and allow his dog to communicate that he misses him, and he has to take a poop.

of course we can. i thought it was funny.

Mmm, I think you were at least kinda serious

i legitimately thought the dog sending too many texts was funny and the project was cool. i was serious in disagreeing with our species coming so far.

What if it scanned your local network for one of your MAC addresses or use something like avahi to find your devices to check whether you're home before sending a text or notification?

It actually does this using MAC addresses. :-) It says "Where are you?" if not I'm not home and "We need to go outside" if I am home.

How did your dogs figure out that a button gets your attention? Did you have to train them in some way?

Also, I am wondering how dogs know when to expect you back. Is their sense of time that accurate and they can easily realize that 9-ish hours passed?

It started with a small bell on a hook next to the door and I would take their paw and hit the bell before letting them outside. Eventually they caught on and I'd give them a lot of praise when they did it on their own. The next iteration was an analog setup and the latest uses a Raspberry Pi. I will say, I have three dogs and only one (the smartest) ever caught on with the bell and analog version. Two of them are able to use the newest version which is the easiest to trigger, since it has a touch surface that is 4x4 inches. The third dog has never caught on. But, he hates going outside, so it's likely he doesn't want to catch on.

I'm not exactly sure how they know when to expect me home, but they seem to have a decent sense of time or are sensitive enough to external triggers that condition them, like neighbors arriving home after work, rush hour traffic, etc.

They do exhibit certain behaviors when it's feeding time (hasn't been automated yet), which is the same every night and that is consistent through daylight savings changes and day lengths varying with seasons. So I assume they have some rudimentary sense of time.

Last Christmas, I used an RPi to build a dial-up WiFi modem for my Grandma, who lives in a rural area where dial-up is the only Internet option, but still wants to download books onto her Kindle (requires WiFi, but not fast speeds).

Was fairly straightforward--required just a USB dial-up modem, WiFi card, switch with some lights, a case, and a little Python programming. Oddly enough, the hardest part of the whole setup was "reverse engineering" what the proprietary Netscape dial-up app was doing with her password before sending it out--turned out it was lowercasing it! Facepalmed real hard there. After figuring that out, everything worked in Linux beautifully.

Very cool. The Raspberry Pi is turning out to be a nice platform for little AI driven projects like this.

It's not using machine learning, but I'm just now finishing up a Christmas present; a web accessible treat dispense with live video. I know those exist commercially, but it has been a fun project to build one anyway.

Terrible quality pictures:


Raspberry Pi Zero W for the brains; Pimoroni speaker hat; cheap stepper motor and bridge; IR emitter and transistor create an IR break beam to detect when treats are dispensed; Raspberry Pi camera for live video feed; all controlled by a custom AppEngine deployment which provides a nice web interface for dispensing treats, viewing the video, and even uploading their own sounds which get played before the treats come out.

The most difficult thing has been the mechanical engineering; something I have little to no experience with. The whole CAD process took me a few weeks of working on and off. Then another two weeks of printing and iterating the parts. The main body itself is a 14 hour print even at high speed. I ended up using FreeCAD for the whole design. Not sure if that was a mistake or not. I like supporting open source software, but ... I have this terrible notion that something like Fusion360 would have made the whole process 10x faster... But it's all printed and working!

Since this is going as a gift to someone else, I also had to take a lot of time to make it easy to use; it couldn't just be a fiddly hack. So, for example, I got a nice setup procedure built into it for setting up wifi. You just hold the button on the back to put it into setup mode, and then show it a QR code on your phone with your Wifi network's details. And since there's a speaker, I'm using some text-to-speech to walk them through the setup. It turned out nice.

And using the magic of ffmpeg, I was able to get the Raspberry's camera feed into an HLS stream, which gets securely uploaded to the AppEngine server. So it's a live video, rather than still pictures or delayed video uploads. Was really happy when that got working. (I use hls.js to handle desktop browser compatibility. Works a treat. Also, FFMpeg is supposed to be able to upload the HLS stream itself to a server, but that was broken or something; had to use some Python to pipe it manually.)

Just putting the finishing touches on the frontend now. 4 days to go... wish me luck!

Love it, and about to dive into the world of 3d printing IoT devices myself. Currently in the process of choosing my modeling software, so your comments on freecad hit home. Would you do it in freecad again? I too want to support open source software and will accept ~2x speed reduction to do so, so I hope 10x is hyperbole, otherwise I need to reconisder.

Unfortunately the community definitely seems more geared towards fusion360. My gut reaction is that in general they don't value the open source aspect of freecad, but maybe it is just subpar for now.

Did you buy a 3d printer for this project? Don't have any source code available?

I haven't tried Fusion 360 yet, so I can't compare. For lack of any other options I would definitely use FreeCAD again. It wasn't a horrible experience.

I love supporting open source. For example, the 3D printer I have is the Lulzbot Mini 2. The company behind it is ... aggressively open source, to put it mildly (_everything_ is open source. Even their manuals for factory assembly are online.). Which is why I went with FreeCAD initially. But I lost a lot of time to A) the awkward UI; and B) bugs. Having just now, the day before Christmas, finished wrapping the gift ... I wonder how much time I would have saved using a "professional" CAD solution. shrug

Words of advice if you do go down the FreeCAD route; learn about dataplanes and shapebinder (check YouTube for tutorial videos). They are somewhat recent features. If you just look up FreeCAD tutorials you'll find a lot that tell you to bind sketches to faces. That's the old way of doing it and it can lead to a lot of problems. Dataplanes are the new way of doing it.

And be careful in the sketcher. A number of times I've had it tell me a sketch is fully constrained, but I was scratching my head because I didn't think it was fully constrained. A day later I'll go back into that sketch and it'll have realized its mistake and moved the sketch around (had one failed print because of that bug).

> Don't have any source code available?

I'm thinking of writing a blog post about the whole project at some point, with all the documentation and such. Bit exhausted right now...

Best of luck with your 3D printing!

If it texted my wife I know in my heart of hearts the cats would quickly figure out how to abuse it for food. Neat video linked from the article https://www.youtube.com/watch?v=gGqVNuYol6o

That's the truth. I had a cat for a total of a week and she basically had me trained fully to do her bidding within that time.

A 'cat sms' would put humans into full servitude!

That video is great. It's short, well organized and after watching it, I feel like I could recreate the project without any further research.

Does it also know when the cat is just going to stand in the window frame for 2 minutes until it goes back inside?

I bet the cat soon learns that sitting in front of the door magically summons a human to let it out.

To be fair, I don't think the cat even knows that, considering their habit of scratching at the door, then staring outside for 30 seconds before deciding to stay in.

Clearly not, and that's a serious flaw.

Cool twist on an old problem.

Here are two others, using different tech to automatically open the door for the pet.

Facial recognition: https://www.hackster.io/windowsiot/cat-door-with-pet-recogni...

Pattern recognition: https://hackaday.com/2010/05/14/cat-door-unlocks-via-facial-...

The first link, "Flo Control" (named after Flo the cat) was built in the late 1990's, and this link on MetaFilter (https://www.metafilter.com/15802/) is from 2002, but the host link appears to be dead.

The OP is kinda swatting a fly with a sledgehammer, but blindfolded, because the system would probably break if the door changed or someone moved something into the detection window. Problem with CNNs is training them to know what is NOT a valid condition, and that is VERY VERY HARD.

This kind of "blind overdesign" makes me sad.

Here's an archive.org link for "Flo Control":


Thanks for finding that! I forgot about the wayback machine. I also forgot that is what websites looked like in the late 90's. /r/blunderyearsforwebsites

As human intervention is required anyway to open the door, why have inside box and outside box? There can be just one big box with pet detection and an SMS that says 'Pet at door'.

Nevertheless, this is a cool project, and the explanation in the YouTube video is even nicer, informative, and succinct.

One addition could be to have a motion sensor outside the door to turn on the light outside, so that this project can work even during late evenings/nights.

Perhaps he forgot if the cat is inside or out and he prefers to know directionality so that he can think "nah, it's about to rain so the cat shouldn't go out."

Great thought. Possibly can also use a weather API to monitor weather conditions to decide whether or not to send SMS.

The inside box is poorly located because they can't overlap in this model.

In this case it would be better to just have one detection box, and say 'pet is at door'

You're right. In this model it does not matter as in both cases the door gets opened. It would only make sense to make a difference if after a given time you may only want entries and not exits (like after some predetermined time).

I'm leaving for a backpacking trip soon and left my dad in charge of my dog. I've put up three cameras to watch her and noticed they can all be included in a grafana dashboard using a text field and markdown.

(One of the cameras is my old security camera but the two new ones are fish eye lense zeros close to the ground.)

Detector is really overkill for this problem. A classifier would do just fine. For a stationary camera you don't need to know where the cat is in the frame. But hey, still a cool project. Anything worth doing is worth overdoing.

What if you're not home? It is for such occasions that cat flaps were invented. However, today's cat flaps either open for any cat, or they're RFID-based, and for that to work, you implant a chip into your cat, which I find utterly barbaric.

Now what could be a great improvement is a cat flap identifying your cat(s) and letting them in based on computer vision, instead of relying on implants. I'd pay good money for it, provided that it worked in all weather conditions and all cat conditions (a wet and dirty cat needs to get in even more than a dry and clean one but I expect false negatives in these scenarios.)

Implanting RFIDs in pets is extremely common, as a method for identifying them to return them to their owners if they are lost or injured[1]. I agree it would a little silly to get an implant strictly to open a pet door, but given that they are already often present, leveraging that to open the door seems like a pragmatic choice.

1: https://en.wikipedia.org/wiki/Microchip_implant_(animal)#Use...

Could also have the tag on a necklace if the pet doesn't mind.

> which I find utterly barbaric.

Your threshold for “barbaric” sounds pretty low, but I guess it’s all relative.

I agree, not much "barbaric" about it... :


Another problem with cat flaps is that you can't always install one if you have a rental. I suggested a doggie door for my parent's house, they said they looked into it but you'd be amazed at just how much space they take up, and they couldn't find a good spot to put it.

If the rental has a sliding glass door one of the flaps at the linked site can be fitted but be prepared for sticker shock https://www.petdoors.com/pet-doors/cat-doors/sliding-glass-d...

Sorry if this sounds idiot, but can't you put the RFID chip into the leash or something like that? Why it needs to be "inside" the animal?

A few years ago our dog got out (Stupid power company went though our back yard and opened our gate and let the dogs out). We put up signs and were looking for him. Someone took him to the vet to get him checked out and told the vet he had the dog for years and was just able to afford to take him in to get checked out. The vet scanned his chip and our information came up. They called us. The guy left the vet and left our dog there. He is a purebred pekingeses.

That is why it needs to be inside the animal. And yes that is a true story. And yes he was also wearing a collar with a nameplate and address on him, the guy replaced it with a different collar.

Yes. The this is the alternate way of doing it with the RFID flaps if for some reason your pet doesn't have a chip or there are issues reading it. Cats can lose collars though, many have a release mechanism so that if the collar gets caught the cat doesn't get stuck forever too.


Collar, yeah. How bad is an RFID chip though?

If a collar sits comfortably on a cat's neck, the cat will remove the collar. If a collar sits very tightly on a cat's neck, it's somewhat uncomfortable.

Everything is awesome until you have to go through the installation for Tensorflow and OpenCV which is a real PITA. Are there any alternatives for this also would Tensorflow Lite work better for RPi?

cats are assholes, so they will find all the bugs and this will never work and probably crash your system. :P

I want a neural network to convince my cat otherwise.

I love my cat.

Url changed from https://hackaday.com/2018/12/21/neural-network-knows-when-ca... to the source it links to.

This is a good example of how titles dominate threads. Because of the title "Neural network knows when cat wants to go outside", we got cat quips. "Raspberry Pi Pet Detector Camera Using Python, TensorFlow, and Twilio" primes a different sort of discussion.

That said, this submission likely would not have received as many upvotes had it linked to the YouTube video originally, as YT videos rarely do well on HN.

sorry guys, that was not the intention. I just send the link for the hackaday news. next time I will try a better approach.

Don't worry about it! As minimaxir pointed out, the story probably wouldn't have gotten much attention otherwise, and readers are clearly interested.

Cat quips are a sign of a healthy community atmosphere.

Up to a point, yes. But alas that point is unstable.

My cat doesn't even know when he wants to go outside

Cat claws at door. Opens door. Cat sits there doing nothing.

I've noticed that a lot of cat behavior can be explained by realizing that they have an incredibly short attention span and any change in their sensory environment can distract them for the half second required to drop their entire mental state. So for the cat it's like "UGHHH I want in... Oh, there's the human! ... What's going on, why are you looking at me?"

I don't know if its so much that they have a short attention span.

My cat definitely, however, is unaware of the idea that the environment around him can be changed, and isn't static. He can figure out how to open doors, and that if something is playing dead it will run when he knocks it from a shelf, but those are the limitations of his ability to interact with the environment.

He's smart enough to know his treats are kept in a small bin on a shelf in my desk, that weighs under a pound, and is covered by a thick sheet of paper. He could knock the whole thing to the ground and get all the treats he wants. But because it looks too big to be an eatable animal playing dead, it simply doesn't occur to him he can push it open to get the treats inside.

That, I think, explains a lot of their behavior. The cat will scratch at the door to go outside. But when I open the door, he needs to take a moment or three to re-situate himself in his new environment, for him, his entire environment just changed unexpectedly. If I then poke my head through the door, he'll realize that its open now, and run through. But if I don't, he needs to re-assess his entire locale, realize that the open door means its passable, and ignore his confusion for how his environment changed so rapidly.

This is true with a lot of things. He'll complain that his litterbox is too dirty. I'll clean it. He'll leave the litterbox, and go do something else. Later, he'll come back, find that its clean, and go to the bathroom. Its not that he was being an asshole, he just knows enough to have figured out that meowing next to a dirty litterbox results in it eventually being clean, but doesn't seem to recognize that the action of me scooping in there is what is actively changing the environment, he needs to rediscover his litterbox as clean 1-5-10-15 minutes later.


You can use this to your advantage to play games with the cat, interestingly enough. I play with him with this thing: https://www.chewy.com/snugglycat-ripple-rug-cat-activity/dp/...

Some days, he gets bored with it. So I unvelcro it, change its shape, and start playing again, and suddenly, its an entirely new toy, and he gets excited again! lol

> That, I think, explains a lot of their behavior.

I honestly think the hardest part of explaining cat behavior is how chaotic they are as a species. You can explain A cat's behavior, but it's harder to explain all of them. My cats, for example, completely understand knocking over (even large) containers to get food, understand that turning door knobs will open doors (thankfully their lack of thumbs means they usually don't have the grip to pull it off), but fail to understand that sitting ON the closed food container means I can't use it to feed them.

Ive read that not only are cats more recently domesticated than dogs (like, by an order of magnitude of years, with dogs domesticated 50k-80k years ago) but that domestication didn't involve OUR deliberate breeding of them outside of recent centuries, so the traits selected for are more about compatibility (for their benefit) than for anything we explicitly want. My personal theory is that housecats are often from such small genepools (that haven't had a lot of recessives weeded out of them yet) that cats tend to be this mix of genius and idiot and weird, so it's very hard to extrapolate generalities from a few specimens. But that theory is likely worthless.

>> cats tend to be this mix of genius and idiot and weird

I applaud you for trying to explain this. You could be completely right, or wrong... We're talking about cats after all.

When my cat asks to go outside, I open the door, and he stands in the open door for a few moments as if he's frozen, then he runs outside. It's as if he is assessing the outdoor environment for threats.

But when he is outside asking to come in, as soon as I open the door, he runs right inside. Perhaps it is because the inside environment is very stable for him with no threats ever so he trusts it.

He does this freezing behavior when guests enter the house. My housemate's cousin brought a huge dog for an afternoon visit. My cat froze entering the living room, detecting an unknown threat, and turned around again and went back to my room.

Gwern published a nice article about cats recently: https://www.gwern.net/Book-reviews#cat-sense-bradshaw-2013-a...

At least that’s what he wants you to think ;)

the eventually clean litterbox algo :)

async/await cleanLitterbox

I let my cat in and out of the side door a few times per day. When it's near dusk and the light shines directly through the glass, he won't come in. Like he thinks it's going to take him to another dimension. Very strange, but also very typical cat behavior.

You just misunderstand. The cat doesn't want out, it wants YOU to leave their house.

Try closing the door very slowly. I've observed cats waiting at a door becoming more and more nervous until they rush outside just before the door closes.

Maybe cat wanted to trigger the door system to get some fresh air.

Your comment sounds like its a joke - but I suspect you are completely serious :) my cat comes down to the door as if it wants out, I open the door and it runs back inside. its about 50/50 chance it will actually go out.

Someone smarter than me should adapt this so it sends a push notification every time it detects the cat ralphing up another load of tinsel, so I don't have to step in it in the morning.

Same. And we also have some neighborhood cats that occasionally hang around. We'd need to train the network to recognize which cat is by the door...

Using fast.ai to retrain some layers of resnet, after collecting and organizing the data, might take you ~2 minutes to train. I just trained one to differentiate between Coke, Diet Coke, Dr. Pepper, and Sprite. From beginning to end (excluding the time to learn on fast.ai and get it set up) it took 2 hours. Collecting the photos from Google's image search took most of my time.

Project idea: Neural network instructs cat when to go outside.

My cat perpetually wants to go outside.

Never heard of Schrödinger’s cat? You can’t know whether it wants to go outside or stay inside until it makes a decision

I don't think you understood Schrodinger's cat. This is not relevant.

I don't think you understood the joke.

Well, maybe quantum computing could allow the machine to predict when the cat simultaneously wants to go in and out, until the system ruins it all by measuring which way it's actually going...

Registration is open for Startup School 2019. Classes start July 22nd.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact