
Apple, It’s Time To Block iOS 5 Beta Users From Reviewing Apps - davidedicillo
http://techcrunch.com/2011/08/17/apple-ios-5-block-app-store-reviews/
======
kogir
They should also ban reviews from jailbroken devices, where users may have
installed custom fonts or other "features" for which you can't possibly test
and account.

And yes -- any test Apple does for this _could_ be defeated, but I expect most
jailbreak developers would side with their app writing comrades and let it
stand.

~~~
Xuzz
The reason that Cydia doesn't have reviews (or even ratings) is that we
haven't found a way to make a working review system either, so we don't even
try. I don't think that any of us would argue or work against something like
that, but I also seriously doubt that Apple will do it.

(I'm a contributor to the Cydia open source project, but I don't speak for
saurik or Cydia, and that's probably not the official stance.)

~~~
saurik
So, I /am/ saurik, and I probably /would/ argue with and work around something
like that, as I think I should be allowed to review things in the App Store.

Really, the "official stance" I have is that developers who believe that
having their application fail horribly when the platform is even slightly
different is not their fault are deluded: Apple is going to change the
platform over time, and when your app fails for Apple they tend to get pissed
and just block your app from the store and may even de-prioritize your
updates.

Seriously: if your app fails because the font is different, there is something
epically wrong with your coding methodology; and yes: Apple actually did
change the font slightly for the iPhone 4 (on retina screens they switched to
Helvetica Neue from Helvetica).

Meanwhile, in desktop-land, everyone from large companies to small independent
developers deal with this supposedly horrible problem of random fonts,
different operating system versions, upgraded libraries, client-side
firewalls, or any other number of things that would make developers like kogir
freak out, and somehow the world doesn't end, and almost all of the software I
download works without issue (with the exception of Chrome on Tablet editions
of Windows, and that's squarely Google's fault). (Hell: even almost all of the
software on the App Store works under these conditions.

What should /really/ happen is that Apple should internally have a different
"test version" of iOS that developers are not allowed to see; one that has a
newer version of SQLite (which I state in specific as developers make all
kinds of incorrect assumptions about the SQLite ABI) and has every core
library compiled to the opposite instruction set (Thumb if normally ARM, and
vice versa), that has every font replaced with Comic Sans, and uses a
different pixel density, on a screen with a slightly different aspect ratio,
using a CPU that runs slightly faster than the shipping devices.

The reason this needs to be done is that an app that fails to run correctly in
that (hidden) configuration is pretty much guaranteed to be an app that is
going to stop working when any of the following occur: iOS 6, iPhone Nano,
iPad+Retina, or another "game changing" device (such as the iPad when it came
out). Rather than let these apps in the store just to fail later, they should
be weeded out before they even get in.

(And yes: Cydia itself strives not to make any assumptions about any of the
things listed, and in fact worked almost entirely correctly when loaded for
the first time on an iPad with the flag flipped for "don't show me in a 1x/2x
box"; certainly well enough that the reviewer in this scenario would have
given it a pass were they testing it in that environment. The one place it
tends to get burnt is with what it does to WebKit; and the current
implementation, which works fine on everything from 2.0 through 5.0, is likely
stable enough to "stand the test of time".)

(FWIW: I do believe that people using beta versions of iOS should not be able
to review applications. These people are using a version of the firmware that
is under NDA... they should not be tempted to be providing feedback that might
violate that agreement to applications that fail to operate correctly,
possibly due to no fault of the app. I have a good example of this related to
Cydia on 5.x, but I can't tell you what it is, for this very reason ;P.)

~~~
shimfish
Um.

One of the main reasons beta versions of an OS are made available is to find
bugs _in the OS_. Apps can crash because the OS isn't doing what it's supposed
to. Which is why we're up to iOS Beta 5. [edit: didn't see your FWIW - was
that an edit?]

I'm afraid this reads like a hacker rant who is angry that lesser mortals get
to write code too. I'm a bit disappointed this comes from someone as
influential as yourself.

[edit] It is a great luxury to have a standardised platform to write for and
it cuts development costs. Any Android devs want to comment on their
experience of writing code for diverse hardware and setups vs iOS? Or indeed
Android users wanting to comment on the quality of software being written?

~~~
saurik
A) Yes, my FWIW was an edit. B) I do not see how anything you have said here
responded to any of my arguments. (edit: in specific, I don't even know what
your one comment about finding bugs in the OS has anything to do with this
conversation) C) My friends who are developers on Android claim that
supporting multiple devices is not actually a serious issue.

[edit] D) Further, I believe that a response like mine is the only serious
response that can be given to someone (kogir) who actually just suggested,
"with a straight face", that a full 10% of the people using the App Store
should be /banned from leaving reviews/ because he doesn't want to deal with
/fonts being changed/. Are you telling me here that you actually agree with
him?

~~~
Troll_Whisperer
_"Further, I believe that a response like mine is the only serious response
that can be given to someone (kogir) who actually just suggested, "with a
straight face", that a full 10% of the people using the App Store should be
/banned from leaving reviews/ because he doesn't want to deal with /fonts
being changed/. Are you telling me here that you actually agree with him?"_

Considering that that "full 10%" of the people you speak of have broken their
terms of service and are running a non-sanctioned OS, why should Apple be
obligated to publish their reviews? I understand the anger at Apple bricking
jail-broken phones, but I really don't see how they should be obligated to
accommodate them.

------
chris24
Apple should remove the reviews section from the App Store for people running
pre-release versions of iOS and replace it with a form to send a bug report to
the app developer (and possibly even a place for a list of known issues that
the app has on iOS 5).

~~~
hellweaver666
Apple should just add a bug reports section to the app store period.

~~~
vaporstun
Potentially, but if so, it should be something which you'd have to opt into or
open only for fellow developers.

The last thing I want is to get spammed with "bug reports" from lay users with
trivial or non-existent issues.

If they implemented this but made it take even one extra step like flipping
some switch in the settings, it would filter out most lay users and garner
some more useful feedback.

------
benatkin
This reminds me of how PyPI ratings got disabled after a long discussion with
the community, started by jacobian:

<http://code.activestate.com/lists/python-catalog-sig/3482/>

I strongly agree that the PyPI ratings were mostly useless and that removing
the feature was a good decision.

iPhone app ratings can be much more useful for popular apps, but for the long
tail of apps, they suffer the same problems as PyPI's rating and review system
did.

Amazon and Yelp ratings, on the other hand, are quite useful. I think it's
partly because users have profiles and are better able to express themselves.

~~~
ojilles
I think the app ratings for the popular apps are all useless (they're well
know apps, other people rave about them, so no need to read all the reviews).
I agree on the long tail being useless as well.

Instead, I think it's most useful for the "middle tail": those apps that do
get a fair bit of usage, but are not considered popular: the developer may not
have build up a reputation yet, but the reviews help confirming the
application does what it is supposed to do (or not of course).

------
jhawk28
An alternative fix would be to have the iOS version used when submitting a
review. It doesn't limit "free speech", but it gives context on what the
problem actually was.

~~~
j79
I think this is a great idea. Apple could allow filtering to see _IF_ I really
want to upgrade to iOS 5 right away or wait until iOS 5 reviewers are happy.

------
spacemanaki
I pointed this out the last time this came up, but an additional measure would
be to also crack down on sites that sell access to iOS betas, if they aren't
doing that already. That would (likely) drastically reduce the number of non-
developers running beta OSes since isn't it $100/year for a developer account,
but you can install betas on many devices? There are sites selling access for
less than $10. If it's not against the TOS of the developer accounts, it
definitely should be.

~~~
zhoutong
Apparently Apple has noticed this issue and they have taken down a few sites.
([http://appadvice.com/appnn/2011/08/apple-takes-steps-
against...](http://appadvice.com/appnn/2011/08/apple-takes-steps-against-
developers-selling-ios-5-access-what-you-need-to-know)) In fact Apple can try
to increase the technical barrier to install iOS beta. (Must upgrade in Xcode
or must own at least one app in App Store to be eligible for beta access.)
This way can block the vast majority of non-developers while not affecting
real iOS developers.

~~~
silencio
There was speculation that that might have just been betas expiring, and not
developers selling access getting their access revoked
([http://www.macrumors.com/2011/08/05/apple-disabling-
ios-5-be...](http://www.macrumors.com/2011/08/05/apple-disabling-ios-5-beta-
installs-and-developer-accounts-over-sales-of-device-slots/)).

> must own at least one app in App Store to be eligible for beta access

That doesn't work for many reasons. App Review, as if they didn't have enough
of a hard time with a bad reputation, would just be flooded by trash from
first time developers or contract developers/companies that just created an
account. Apple could make it mandatory to install using Xcode, but in the
past, Xcode upgrades have bricked devices where iTunes doesn't. So I have a
feeling that's not really a priority and more of a burden. Plus, it's
braindead simple to install and use Xcode to install betas, and anyone
desperate enough to pay $10 but not $100 should easily be able to get around
that.

I really think all Apple needs to do is to block reviews from iOS 5 devices.
Developers running betas won't be reviewing apps in the store (that's
incredibly hypocritical and cruel if so) so nobody's going to be missing that.
This doesn't harm beta testers and devs that want to install iOS betas.

~~~
dunham
I'd also block reviews from beta versions of iTunes, since many reviews are
entered via iTunes.

------
msftguy
Maybe Apple doesn't block beta reviews because it incentivizes developers to
fix their apps before the release? Most of those problems should be trivial to
at least work around with version-dependent logic to minimizes regressions on
versions < 5.x.

Bad reviews don't exactly hurt Apple, and it's not like iOS developers are
scarce these days, so it might just be Apple having their priorities set
straight (users > devs) and not a coding blunder..or maybe it's just me
overthinking it.

~~~
X-Istence
Except that developers are not even given the chance to fix issues before
Johnny Public is allowed to start rating apps. The whole point of the beta is
to ALLOW the developer to test their apps and find the bugs, as well as to
iron out the bugs in the core OS. What if it is an issue with a core API that
is now returning something that it shouldn't and it wasn't caught by Apple in
testing, then the developer is getting blamed for something they can't even
fix.

You have to be a part of the developer program ($99) to get pre-release
software, and as part of that you go with the understanding that it isn't
meant for everyday use.

Johnny Public shouldn't be running the beta in the first place. If by the time
iOS 5 is out the developer still hasn't fixed the bugs in their app, THEN it
is perfectly fine to start leaving bad reviews as the developer has had plenty
of time to fix their app, but during the beta period, no.

~~~
sgarman
It is additionally time consuming building work arounds for issues that might
be altered by the time the OS makes it to gold master. Beta is beta, it might
mean the OS does not work with your app not your app does not work in the OS.

------
pat2man
This is just obvious. Isn't pointing out a bug thats due to an unreleased OS
breaking your NDA anyway?

~~~
stevenwei
Yeah, but non-developers who manage to install iOS 5 probably haven't actually
signed the NDA.

The funny thing is as a developer you're not even allowed to mention in your
description that something in your app is broken on iOS 5, since _that_ would
be breaking the NDA.

~~~
psychotik
Apple also doesn't like it when you fix an issue, release an update and
mention that you fixed a bug which affected iOS 5 compatibility. I needed to
change it say "compatibility fixes for future iOS versions".

------
desaiguddu
There are reasons to block the iOS5 users comment or review the applications.

1\. iOS5 is beta , so chances are there that inclusion of new APIs , pre
release iOS version can adversely affect the already approved application. But
as the problem is not with the Application but with the Operating System , the
review can be biased or completely irrelevant.

2\. As a Developer I will welcome the step by Apple that, this will allow
users to comment and review apps from the users who run most stable Operating
system, so any bug on my application is because of me, not because of the
iOS5.

------
RyanKearney
All the effort developers are putting into complaining about people on iOS 5
Beta leaving negative reviews could better be spent testing and fixing their
app for iOS 5.

Are you going to complain when iOS 5 goes live and your app still isn't ready
because you stayed up until 2am writing that blog post rant instead of ironing
out that last bug?

Even if Apple blocks iOS 5 devices from leaving reviews, you can still leave a
review through iTunes.

