

Understanding the Reaction to Amazon Prime Air - adamnemecek
http://nbviewer.ipython.org/github/ptwobrussell/Mining-the-Social-Web-2nd-Edition/blob/master/ipynb/__Understanding%20the%20Reaction%20to%20%22Amazon%20Prime%20Air%22.ipynb

======
MrUnknown
My Reaction to Amazon Prime Air is that I live around a mile from a Amazon
warehouse and I can not drive there to pick it up. Instead I need to use Free
Shipping, have them "package" my purchase randomly anywhere from 2 days to 5
days so I can't cancel it on them for taking too long, then ship the item
1,000 feet or so.

They could setup a location for Amazon Locker in the warehouse location for me
to pickup my items. I understand I will receive absolutely no on-site customer
service. They could even use the Locker for returns. I have suggested this to
their support.

Or, I could pay for Prime free shipping so they can mail it such a short
distance and actually receive the item the next day like I do with Newegg, who
is further away, which has been enjoying my business if they have the item.

------
Tepix
5kb per serialized tweet? That doesn't sound very efficient.

~~~
csmuk
Ouch.

They're probably encoding in XML and several nested layers of base64!!

(yes I have actually seen people doing this in a finance company's integration
platform).

~~~
jzwinck
They're storing all the metadata, and for better or worse they are storing
each string field as an "object" whereas they could have used a fixed-width
field. The former of course is something like a reference count and a pointer
to a string elsewhere; the latter would have some overhead when strings are of
varying lengths. If memory usage were really a concern, decent savings could
be realized by changing (most of?) the 13 float64 columns to float32, and some
of the object (string) columns to fixed-width. For example, "lang" is two
characters always, and usernames (which are stored in several fields) would be
more efficiently stored as char[15]. Easy to try, just not the default
behavior; I bet you could cut the total size by a third (too bad they didn't).

~~~
csmuk
Ick. I wonder if anyone has actually heard of ASN.1 BER before?

Seems like ridiculous encoding and structures are the in thing at the moment.

~~~
jzwinck
Have you used Numpy/Scipy/Pandas? It works a bit differently. It basically
creates a matrix as in C--the trick is choosing the right column types. I
don't see what relevance ASN.1 has here.

~~~
csmuk
Yes SciPy (although I usually end up with Mathematica).That's a generic
encoding. ASN.1 is a specific and compact encoding of the data.

