
How expensive is SF rent? Use Unix commands to find out - jph
How expensive is SF rent? Here&#x27;s a coding challenge that uses Unix command line tools to find out.<p>Hacker friends suggested I post the result and code here because I&#x27;m writing open source statistics software and this is a practical example of results and code.<p>The results for a one bedroom in SF: mean $3348, median $3475, interquartile range $1514, and standard deviation $1130.<p>How to use Unix commands to find out: use `curl` to fetch Craigslist apartment listings, then `sed`, `grep`, and `tr` to filter and parse the data, then `num` to calculate the statistics.<p><pre><code>    curl -s http:&#x2F;&#x2F;sfbay.craigslist.org&#x2F;search&#x2F;sfc&#x2F;apa\?min_price\=1\&amp;bedrooms\=1 |
    sed &#x27;s&#x2F;\(&lt;p class=&quot;row&quot;\)&#x2F;\n\1&#x2F;g&#x27; |
    grep &#x27; 1br &#x27; |
    tr &quot;&gt;&quot; &quot;\n&quot; |
    sed -n &#x27;s&#x2F;^\$\([0-9]\+\).*&#x2F;\1&#x2F;p&#x27; |
    num mean median interquartile-range standard-deviation OFMT=&quot;%.0d&quot;
</code></pre>
The software I&#x27;m personally writing is the `num` command to calculate the statistics. The software is free at http:&#x2F;&#x2F;www.numcommand.com
======
gesman
@echo "very expensive"

------
DrScump
What happens if the price overflows the limit of a 32-bit integer?

~~~
jph
If the price overflows 32-bit, then everyone in SF moves. :)

For this script, the fetch returns at most 100 rows, so won't overflow.

More broadly, the Num command runs using awk, and number handling depends on
your system's awk implementation and C libraries. Current awk implementations
use an internal representation of all numbers, including integers, that uses
double-precision floating-point numbers. I know GNU awk supports arbitrary
precision arithmetic, and also can use --bignum, GNU MPFR and GNU MP (GMP);
these will be great additions to the Num command for the next major version
upgrade.

------
atsaloli
I had a perl script to do mean, median and std dev; it'd be great to have this
functionality available through the OS vendor as a package.

~~~
jph
Yes. I'm writing the Num command specifically to become an OS vendor package.
(For example I'm writing Num using Awk, which is available on every current
POSIX system.)

~~~
atsaloli
Love it!

------
romanovcode
sed: RE error: illegal byte sequence gawk: cmd. line:1691: fatal: division by
zero attempted

~~~
jph
Can you contact me about this? I'll diagnose it. My direct email is
joel@joelparkerhenderson.com. I really appreciate it.

------
anon6_
FreeBSD 11-CURRENT

    
    
       num: command not found
       tr: Illegal byte sequence

~~~
jph
To install the num command, use
[http://www.numcommand.com](http://www.numcommand.com)

