Hacker News new | comments | show | ask | jobs | submit login

The code for the camera.c is here: http://git.marcansoft.com/?p=libfreenect.git;a=blob;f=lib/ca...

I've always wondered how people reverse engineer these things. Do they just guess what the interface might be based on the chips? Or are they able to probe it somehow through the port?

Here is a very nice metaphor by Andrew Tridgell (from Samba fame) on the subject:


"I call this method the "French Cafe technique". Imagine you wanted to learn French, and there were no books, courses etc available to teach you. You might decide to learn by flying to France and sitting in a French Cafe and just listening to the conversations around you. You take copious notes on what the customers say to the waiter and what food arrives. That way you eventually learn the words for "bread", "coffee" etc.

We use the same technique to learn about protocol additions that Microsoft makes. We use a network sniffer to listen in on conversations between Microsoft clients and servers and over time we learn the "words" for "file size", "datestamp" as we observe what is sent for each query.

Now one problem with the "French Cafe" technique is that you can only learn words that the customers use. What if you want to learn other words? Say for example you want to learn to swear in French? You would try ordering something at the cafe, then stepping on the waiters toe or poking him in the eye when he gives you your order. As you are being kicked out you take copious notes on the words he uses.

The equivalent of "swear words" in a network protocol are "error packets". When implementing Samba we need to know how to respond to error conditions. To work this out we write a program that deliberately accesses a file that doesn't exist, or uses a buffer that is too small or accesses a file we don't own. Then we watch what error code is returned for each condition, and take notes. "

Hector said in the video that he doesn't even have an Xbox. I guess that's like turning up to the cafe five minutes after opening time when it's just you and the waiter.

I don't think the xbox would be of much use in that case. Can he run a USB sniffer on an Xbox? Makes more sense to connect it to his PC and use one of the hundreds USB sniffers available.

The guy actually says in the video "this is not a sniffer, this is not a man in the middle", which is quite incredible. Just a laptop running Linux.

Look at his desk: a complete mess of wires and hardware and a single Rubiks cube. Total hacker :-)

I loved this book as a kid: http://www.amazon.com/Reversing-Secrets-Engineering-Eldad-Ei...

It's probably a bit out of date now, but my dog-eared copy is still a good read. Ah, nostalgia. There once were days when I dreamed that a CS degree would make me as a god; the silly thoughts of a child. Now I know that it is a _PhD_ which makes gods of men.

Now I know that it is a _PhD_ which makes gods of men.

I know you're joking, but...

When I was halfway through my Ph.D. I formulated a hypothesis: The proximate challenge that keeps you from graduating is that you have to write a thesis. But the ultimate challenge to getting your Ph.D. is this: You somehow have to learn to understand, deep down, that all your romantic notions about the Ph.D. are bunk, that you will be exactly the same person on the day after you get it that you were the day before, and that you need to stop waiting for the day when you feel like a god and just write something down and get on with life.

It may take you years to accept this, and it may drive you to drink, but after you get to that point you can graduate.

Only then will you be able to live with the fact that your thesis looks like crap to you. Your thesis will always look like crap to you. Either you will have figured out absolutely everything and your thesis will look incredibly boring to you, because you've moved on, or -- vastly more likely -- your thesis will look woefully incomplete because, geez, there is so much that you couldn't figure out, and you're just so stupid!

Or, most likely of all, you will think both of these things at the same time.

Similarly: Being the world's foremost expert on a particular scientific problem is a lot less exciting in real life than it seems in the movies. In fact, being on the frontier of science feels like being totally, hopelessly lost and confused. Why this came as a surprise to me I'll never know.

When I went to do mathematics at Chicago I figured I was the smartest person alive. There I was, facing the gargoyles of my dreams; a poor kid aspiring to a better life by shrugging off the accent I was born into and the mentality of defeat so common among the poor. But I had gone too far, became too confident and failed horribly. I was sure that the world had failed--I was too good--and that everything was bullshit. I left, walked away from a full scholarship because I had overcome the constrains of my life before and Mathematics and University were no different. I took a job at a small software shop in Portland, OR instead, enrolling part time at PSU doing computer science.

I failed at both, as you might expect. The world wasn't wrong, I was. While I could program, I had no discipline. While I had intellect, I had no ability to learn. The world was not wrong, I was. All of my anger and suffering and frustration were my fault. From the defeat of my new University and my new job I learned that my romantic notions of most things were not reality. Enrico Fermi, on whose stairway I bounded up, did not simply decide to conjure nuclear fission under what is now a library. He worked for years, a thing which I had never done.

The novice says to the master, "Coal is black." The master replies, "No, it is not."

The intermediate says to the master, "Coal is not black." The master replies, "Of course it is."

The masters say among themselves, "It is coal."

I hold no romantic notions as I held when I were a boy; I have not become a cynical man. Life is suffering and pain. Life is joy and love. I have built a business from nothing and sold it for a profit. I am now very poor. Life is life and that is beautiful. What we learn, what we truly learn, we so incorporate into our being that we cannot perceive it as unknown to all. We are the streams into which a man steps: never the same, yet always the same.

To gain mastery over the frontier of science is to gain mastery over nothing, over one's self. It is confusion and pain and truth and beauty.

You wrote all that as a 3rd level reply to an offhand comment in a random thread? Wow. This is the reason I keep coming back here.

Just yesterday I saw so much negativity and pettiness on another thread that I had pretty much written off HN as a lost cause.

Your post brought me back. Thanks!

It came as a surprise because up to that point, someone had the answer. Even if you had great teachers and even if you're a problem solver...at the end of the day, someone had done what you were doing.

I think that's why most people I know that are on "the cutting edge" are very humble: either they got "it" right and know 40 people just as smart that went in one of 40 equally promising directions and got it wrong. Or they're one of the 41 people still trying to figure out just where the heck they can go from this apparent dead-end.

Then along comes 42...

The hopelessness and confusion that comes at the frontier of science is precisely why I stopped studying biochemistry for my bachelors degree. By the first 300 level course, people begin to start asking relatively simple questions that are not yet known to mankind. It freaked me out. I couldn't imagine ever discovering new knowledge and subsequently dropped out and into computer science...

Ah, but the spice of life is staring into the Abyss of Unknowing and recognizing your very self in it! The most beautiful questions of mathematics and computer science so very often start out "Does there exist..." and we are left with no answer other than, "Who knows!" The world is wide and strange and we are very small indeed. That is beautiful to me.

hmmm... maybe thats the upside of a masters degree. You have that "come to jebus" moment the day after you graduate, but without all the nasty research and writing.

All of your hypothesis is in total agreement with this: http://matt.might.net/articles/phd-school-in-pictures/

Can I get a Reverse Engineering PhD?

You can study molecular biology and become an expert at reverse-engineering the most amazing machines in existence.

Though the Kinect is apparently a lot more tractable.

I've been writing some code for people working in bioinformatics recently. It's pretty similar.

I've looked into this a bit. There are three programs that I'm aware of:

The BitBlaze project at UC Berkeley. http://bitblaze.cs.berkeley.edu/

There's also http://www.cs.kent.ac.uk/people/staff/amk/ which offers a PhD studentship in "Reverse Engineering for Security."

CERIAS at Purdue will definitely have some RE related courses, e.g. http://www.cerias.purdue.edu/site/projects/detail/malware_re...

you can probe USB commands, but that takes a lot of time. much faster to connect a usb analyzer and then 'replay' to see what commands do what.

http://www.adafruit.com/blog/2010/11/09/kinect-hacking-video... (demo of analyzer)

http://github.com/adafruit/Kinect (USB log dump)

I've reverse engineered some protocols -- haven't done a USB one yet, but I'm sure the principals are similar. Grab some data, look at it (usually with a good hex editor -- last time I used ghex2), look for patterns. Usually there's some sort of packet structure, or maybe some data that looks like something in ascii -- ghex2 shows you what every 2 bytes or 4 bytes are if they were signed/unsigned ints, floats etc. Usually something will jump out at you.

It's a fun game usually.

And if you can actually input data using the protocol, you can take some standard packets and tweak a byte/short/long at a time and see what changes.

Reversing USB is the same as any protocol on top of TCP is the same as any other protocol, just with different tools.

I wish there were an open source hex editor like Hex Workshop for Windows - one of the features I loved was tagging a section of bytes with comments, and being able to use those same tags across multiple data dumps.

Particularly handy for USB on Linux is the 'usbmon' module. Which when used with a kernel that has debugfs support, you can mount debugfs and use a new enough Wireshark to monitor the USB traffic.


You could write your own by extending OSS like hexdump or hexcurse or any other hex editor. It shouldn't be to big a deal to add that functionality. It's just beautiful what you can do with open source code! Don't forget to share.

As others have pointed out, in this case he probably used a hardware USB sniffer (though he apparently doesn't own an XBox, so who knows). If your device has existing drivers that will run on an OS in a VM, you can use USB passthrough (most virtualisers including VirtualBox) to run it on top of Linux, which comes with a USB logging module called "usbmon" (http://www.mjmwired.net/kernel/Documentation/usb/usbmon.txt).

There are also software loggers for Windows such as "Snoopy Pro" but last time I tried that it dropped certain types of packets. No such issues with usbmon.

Wireshark has had support for usbmon for a few releases now too, which gives you a considerably more useful GUI for working with USB streams.

Sadly, there aren't many (any?) protocol dissectors yet, so comms with common chips like FTDI devices don't automatically translate into something human-readable like, say, HTTP conversations.

Really? That's sort of baffling. FTDI's USB chips are damn easy to work with, I'd expect dissectors for them to be out there already. I'll have to remedy that.

The README says he used the great big (~500mB) data dump of the USB data that Adafruit released the other day.

You use a USB sniffer to observe traffic between the host and the device and make guesses about how to interpret the data.

Exactly this.

I'm knee-deep in a personal project that involves reverse-engineering a USB device; since the only drivers for the device are for Windows, the solution was to virtualize Windows on a Linux host (presenting the USB device to the guest OS), fire up wireshark on the host OS (using the usbmon kernel module), interact with the Windows software and drivers as per usual, and capture anything the guest OS sends to the device for analysis.

Really simple stuff; much easier than following this stuff with scopes or logic analyzers.

Guessing the interface is like shooting in the dark. More likely is someone had access to a USB snooper like the USB Beagle - http://www.totalphase.com/products/beagle_usb480/

My last reverse engineering project consisted mostly of hours sitting at Chipotle with hex printouts and a highlighter. That was a file format though, not a protocol.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact