Hacker News new | comments | show | ask | jobs | submit login
MicroSD card FAQ (bunniestudios.com)
110 points by pmarin 1543 days ago | hide | past | web | 29 comments | favorite

Flash is frightening.

I wrote a transactional flash file system (well, object store) in the early 90s. It used NOR flash, which allows you slam individual bits from 1 to zero in any order. Basic strategy for doing a write: Write the payload, verify it, then clear some "commit" bits, and have enough infrastructure bits around that you can handle commit failure from stuck-at-1 errors. My failure model was probably pretty stupid, but it seemed to work well and I never heard of someone losing data. Of course, flash parts were a lot clunkier back then.

Fast-forward 20 years and they're playing terrifying games with those flash cells. I worry that the people writing the firmware are willing to trade off reliability for capacity or speed. Or they make crazy risks ("Hey, this window is only for ten milliseconds, what are the chances?"). I know too many lame firmware engineers to be really comfortable.

The failure modes are not block-by-block now, either. The whole device can go dead just because the firmware lost a lookup table to a bad bit. Something like this very clearly happened to Intel a couple years ago, and I know of other manufacturers who weren't very careful with their firmware quality (along the lines of "If we bump the version number on every fix, this will just make our customers nervous." Wow).

I'd love to see an eMMC spec that:

- Allows raw access to the underlying flash (useful for data recovery, in any event)

- Provides metadata describing the treatment of that flash (how many reads / erases, leveling requirements, geometry info, etc.)

- Provdes access to ECC engines, buffers and other support hardware (to reduce bus traffic and host computation)

... then we could push the data management into a FOSS layer in the host, where people can inspect it, fix it, and improve it.

The combination of a hot commodity, pressure to make it perfom, complex management algorithms and opacity of implementation do not make me feel very happy about flash.

[edit: typos]

> Provides metadata describing the treatment of that flash (how many reads / erases, leveling requirements, geometry info, etc.)

My Corsair SSD seems to return some SMART information regarding wear levels, though I'm not altogether sure how to interpret most of it.

#177 "Wear Level Count" seems to return the most relevant value, in my case 4% of the total rated write cycles. Not as bad as I expected for a MacBook with 4300 hours of use under it's belt.

Some great ideas there. Agree about complexity - its usually a sign of something dodgy being hidden.

Perhaps a flash raid 1 device would suffice?

The scary thing is that you lose the whole device, and flash seems more fragile this way that disks are.

Perhaps we just understand disk failure modes better, and are more comfortable with them.

Good point. Mechanical disks usually start exhibiting warning signs when failing (the disk in my T61 started clicking about a month ago is being replaced tomorrow). I'm not sure I'll get away with that in the Samsung 840 that is going in it instead.

(yes I do backups and rsync hourly before anyone slates me for working on a cranky machine).

> Perhaps we just understand disk failure modes better, and are more comfortable with them.

In some ways, predictability is an issue too.

SSD typically fail after a relatively set number of write cycles, which doesn't seem too much of an issue until you try to put them into a RAID5 array. I've heard stories of a couple of people trying this, and having multiple drives all fail within hours of each other (usually during the array rebuild).

It's something I'd never have thought about had I not been prompted.

This is the kind of thing that ISO900x systems could be quite good at dealing with.

Unfortunately, quality assurance systems are often just a paperwork exercise so a company can get a logo for their letter footings, so all those goods-in checks and route cards and batch numbering etc end up not being so much use.

It might feel like he's going to a lot of effort. He should be able to spec a part; get quotes; order; do minimal checking of the delivered product (does the pack say "4 GB MicroSD? does the device have printing that says "4 GB"?); and then use the part. The lengths he's going to are normally saved for when you're forced[1] to use obsolete stock from brokers - you don't have much idea about who has had those parts or how they've been stored[2] and so you want to put them through rigorous testing. (Especially because the situation means they're usually going into things that must not fail.)

[1] I was working on a device where the manf had specified a Bourns resister network at 1% tolerance. That design had been approved and certificated and was thus "locked in" - there was no possibility to change that device. 1% devices were not available. The manf offered a complicated alternative (testing a bunch of 5% devices) but PHBs declined to use that ("What? We buy stuff and we might not use it?") and thus we lost that bid.

[2] Certificates of conformity are easy, everyone offers them. But there's a further requirement for some manfs of "traceability" - each component must be traceable through your paperwork (and the paperwork of your suppliers) to the batch where it was made. In theory this is great. If you get a bunch of devices failing you can look at what device is failing, and see where else you've used that device, and predict other failures. In practice the paperwork is nonsense; or no-one is skilled enough to do that kind of analysis. (I guess the huge manfs might do it, but people building 30,000 off builds don't tend to have the time or software or expertise to do it.)

tl:dr QA for hardware is often a joke and there's plenty of room for disruption there.

Bunnie has a lot of experience dealing with Chinese manufacturing and he's blogged quite a bit about it. The impression one is left with is that the manufacturers are always playing the 'game' which is "Can I get away with shipping X?" Where the rules are to not explicitly break any rule in the contract. I've known a number of people who have experienced this game first hand.

From my reading of it there isn't any particular 'malice' in the game, its just a game they have played for decades as a way of getting business by offering a slightly lower price. Sometimes a middle-man (distributor) gets caught out, sometimes its the manufacturer. The great electrolytic capacitor purge was a good example of that game in action.

So in the rules of that game, if getting a certificate would make you the buyer look less closely at the product and thus take material you might not otherwise? Well they will go out and get one of those.

I got to experience it sort of in reverse, where a Japanese manufacturer was buying systems from my company and re-selling them. We spent 8 months on a contract that was over 200 pages because of the detail to which everything was specified. It was amazing to me and educational at the same time. I came to see lots of ways we could have made our boxes that would have made them cheaper to build but less reliable :-).

as described, the 'relatively common' providing under-specified cards with loopback tricks to make a user think it's full-sized - surely that's breaking an explicit part of the contract, not to mention wilful fraud?

Let me illustrate how the game is played. The contract says "Flash will be 128MB", card produced says "I'm 128MB"

Are the contract terms met? Even if the card only has 8 million unique locations? The contract doesn't say that the card has to hold 128M unique things all at the same time, it just says that you can address 128M things in it. So we take our clever software give you 128M addressable locations but if you try to put more than 8M things in the card you'll erase one of the other things you put in there.

You go through a process where you ask a number of vendors for a product that you've specified, perhaps you pick the cheapest one, and it turns out that they have an interpretation of the contract that isn't the same as you. (so they sold you an 8MB flash which pretends to be 128M flash). Your next step is to get them to 'fix' it, and they will, they will say "Oh we didn't realize you wanted to store 128M different things at once, we can do that too! That will by $Y" except that $Y is anywhere from 2% to 15% higher than the second or third lowest bid you got earlier.

So this manufacturer will tell you with a completely straight face about how sorry they were for mistaking what you wanted, and how its perfectly reasonable that some people really only need about 8MB of unique things, but since you have a relationship with these folks now you get to the 'real' product you wanted (128M flash) and you end up paying more for that than you would have with a different vendor at the start. They have 'won' this round, you have lost.

If you know this is how the game is played you write your contracts differently, you write : "128M Flash, which can hold 128M unique pieces of information at the same time that can be read back at a minimum of x bits per second and written at a minimum of Y bits per second, and can be re-written at least Z times at all temperatures between 0 and 50 degrees C and in the presense or absence of any magnetic field up to x Gauss, to be measured using the following tests on the following equipment that will be supplied by me and cannot be changed by the manufacturer, and will be inspected every 30 days by an agent I appoint who does not work for the vendor."

And then they know you've played the game before.

cute :)

I have my doubts that shipping 8MB cards in this way when contracted to deliver 128MB would be accepted by a chinese court - it just probably wouldn't reach court most of the time, especially when dealing with a westerner who would just learn some kind of lesson and move on.

This is pretty normal in the electronics industry. From major suppliers, in the 90s we had 74hc logic which was just relabelled 74ls and some east German clone z80s delivered as genuine zilog parts. You don't usually notice until something fails test or you get a high failure rate in the field.

Comedy moment was the 1M resistors which were actually zero ohm shorts.

If there is a dollar to be made...

I've always wondered if we paeons truly know what the heck is going on in our chips. I mean, the basis of trust is pretty large .. but there is really nothing we can do to ascertain whether there isn't a backdoor in every single chip package, and indeed this has been discovered to be a real situation by the Pentagon (Chinese backdoors in milspec chips), so .. I can hardly imagine that we can do much more than just plain trust, and get over it.

The truth is that you don't and this is fairly apparent when you find a buggy device and the vendor either shrugs it or threatens to sue you. (yes that does happen)

You can build out of discrete logic. The opportunity for something dodgy to be built in at that level is unrealistic. Some military equipment still works on that principle.

Consumer flash cards are an example of a market where the purchaser has very little information to go on. The raw user available storage number is pretty misleading as the block remapping system (or absence of one) is a key determinant of reliability.

The fact that SanDisk and other name-brand manufacturers make a practice of switching out the underlying parts without changing the part numbers make reviews pretty much useless.

I'm always amazed by the lengths forgers go in order to successfully deceive their victims. Specially when only takes the same amount of work, or a little, to achieve uniqueness. Poor bastards, they don't know what they're missing.

Actually, forgers don't need to do anything extraordinary to produce these fake cards. According to the article that OP links to at the beginning, they just push some buttons on the same machines that make genuine cards.

> Very low serial numbers, like very low MAC ID addresses, are a hallmark of the "ghost shift", i.e. the shift that happens very late at night when a rouge worker enters the factory and runs the production machine off the books. Significantly, ghost shifts are often run using marginal material that would normally be disposed of but were intercepted on the way to the grinder. As a result, the markings and characteristics of the material often look absolutely authentic, because the ghost material is a product of the same line as genuine material.

Actually, forgers don't need to do anything extraordinary to produce these fake cards. According to the article that OP links to at the beginning, they just push some buttons on the same machines that make genuine cards.

The same thing occurs in counterfeit DVDs. The tell-tale sign is when the factory-identifying marks have been ground off from the clear middle plastic part of the DVD.

I believe these are sometimes also called overruns, and it affects basically every industry out there, from electronics through to handbags.

This is something that I too find confusing.

I don't get what the added value is of a handbag that costs $2,000. There's good quality materials, high quality assurance, restricted numbers, sales from expensive shops. But still, that's no where near $2,000. So a forger can make a knock off and try to sell it for $1,800; or for $200; or a bad similar bag and sell it for $50. I don't know why they don't make nice bags for $200. This is true when you look at wristwatches. I want a nice movement in a nice case with a nice face for a reasonable price.

There's a lot of philosophy around it if you're interested in that kind of noodling.

"Bob has a science-fiction machine that can make an exact copy of a famous artwork. No one will be able to tell the difference between the two pieces. Bob borrows a famous artwork, but he then gets the copy and original mixed up. Should he tell the museum? Or should he just return one? No-one will eve know, but Bob." etc.

So a forger can make a knock off and try to sell it for $1,800; or for $200; or a bad similar bag and sell it for $50. I don't know why they don't make nice bags for $200.

Interesting enough, this happens; there are high quality fakes as well as poor quality. I've sat down with the head of brand enforcement for a major fashion house, held the different fake items, and seen what can be made.

There are different grades of counterfeit available, which basically boil down to three: the cheap and nasty; something nice (the $200 example); and something incredible.

The latter is particularly interesting; sometimes a counterfeiter will understand the brand so well, the cuts, styles and use of materials that define them, and come up with entirely new ranges that have never and will never be made by the authentic manufacturer.

I've had more than one brand manager tell me that in any other situation, the designer behind the items would be able to walk into a job with them due to their new designs being so good.

I've had more than one brand manager tell me that in any other situation, the designer behind the items would be able to walk into a job with them due to their new designs being so good.

Thank you for telling us your experience. That was exactly the point I was trying to get through. The market that buys cheap to reasonable copies would never be able to pay the premium price of the original merchandise, thus there's no loss there.

However, the role of the high quality counterfeiter and the market that they aim for should be better understood in order to be economically explored. Though not by litigation, but by normal market channels.

In the end, instead of paying lawyers to pursue the lost revenue, we could create new brands that would fit the expectations of these unharvested markets.

I don't know why they don't make nice bags for $200.

It's because of the demand side of the equation. There's a large enough market of aspirational consumers who really just want the brand name but can't afford to pay the mid-market price ($20 vs $200 vs $2000). The perceived quality, whether it's actually there or not, comes from their ability to wear and show the brand name. Globally, that's a large enough market to make it continually worthwhile for producers and resellers to continue the counterfeiting, especially at the very low end.

My wife loves expensive shoes and purses. Not the "aspirational" brands like Coach or even Louis Vuitton. She has graduated to brands like Chanel ($3000-5000) and Hermes ($15,000) and Louboutin shoes ($900).

I'll tell you exactly why women love to spend lots of money for purses and shoes. It's for the sheer exclusivity of it. It's not because of things like quality or functionality, so the argument that a $200 knockoff is just as functional or has just as good craftsmanship is useless.

I have argued that for something so expensive like $5000 handbag, you would expect an INCREASE in functionality or durability. I would expect things like waterproofedness, etc but no. They are even more fragile than you would expect. The price is simply a way to create a sense of exclusivity.

It similar to why a man buys a 600 hp Ferrari when most highways only allow speeds up to 60 mph. It's a show of power, prestige and wealth. Sure you could get a car that had the same acceleration with better fuel efficiency, but that's not why people buy ferraris.

You may find this an interesting read:


It goes into how the goods are nearly always made in low-cost countries like China & Philippines these days, and that the demand for them is purely status-driven.

"I want a nice movement in a nice case with a nice face for a reasonable price."

You might want to look at some higher-end Chinese watches. Some of the Parnis watches look very nice indeed.

Or just get a Casio f91w for $10. More accurate, has led to light it in the dark, waterproof, has a stopwatch, alarm and won't get nicked by anyone and doesn't carry a repair cost ever.

I sold my Omega watch and got one and have had it for 3 years. When it dies, I'll just buy another one.

Unfortunately it's so butt ugly that you'll likely get on a government watch list.

I really wish I made that last part up.


Yes well aware of that :)

Reminds me of a Stephen Wright joke (IIRC): Someone broke into my apartment last night and stole all of my furniture... and replaced it with exact duplicates.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact