Hacker News new | past | comments | ask | show | jobs | submit login
Installing and setting up OS X Mavericks for Python development (kristian.io)
52 points by alesdotio on Oct 25, 2013 | hide | past | favorite | 60 comments

This article is so opinionated. Just some counter-options:

Enabling apps from everywhere: Don't disable this. It's actually a good security feature. If you want to run an app that isn't signed, right-click and click "Open" and confirm you really want to run it and the OS will remember that choice for that app in future.

Terminal: The default Terminal is actually pretty solid. Much better than it used to be. Two common complaints (lack of tabs, lack of colours) are now moot.

Oh My ZSH: Or you could use fish – a new Mavericks compatible build (2.1) should be coming to the homebrew package manager literally any time now.

Also, for Python specifically (i.e. the title of this post), you can install a new version of Python via homebrew (and then place /usr/local/bin above /usr/bin in your PATH). This has the advantage of being able to run pip without superuser privileges. Though if you're using virtualenvs, it's probably not that big a deal anyway.

+1 for installing python from homebrew; letting the default OSX installation be used at all has caused me mad problems in the past.

Homebrew is definitely the way to go for installing python for development on Mac

A while ago I tried fish. Unfortunately I ran into quite a few issues because it isn't POSIX compliant. When I wasn't running into issues, it was great! I now use zsh.

Be wary of using case sensitive FS. Many apps simply don't work on case sensitive.

Maybe they've fixed it, but not long ago the whole Adobe suite just wouldn't work on case sensitive EXT FS.

You should use a convention for your filenames anyway. Eg in Python you always use lower case.

The major reason behind the switch is that I recently had to rename a file, that someone accidentally comitted with an uppercase letter. We had to do it on another machine. It's a valid concern though.

At my previous employer, where most developers use Linux, there was a README and a readme with different content in the same directory in a git repository. Loads of fun on case-insensitive OS X, since git always believed some data was removed ;).

The way you phrased it one could assume that it's somehow git's fault ;)

It doesn't completely ignore the case, it just doesn't allow you to have two different files with the same case.

We've had a couple hiccups going from Mac to Linux because of this (if you call a folder 'Assets' and there's already one called 'assets', you're going to get two distinct folders on your Linux server and scratch your head for a while until you figure out why something didn't change), but you mostly don't have to worry too much about it.

mv File _file; mv _file file

I wish it was always that simple.

I don't know about Mac, but once I had a problem that some of the game installers created folder " Games", although "Games" was already there.

Fun time trying to delete or rename to something same this " Games" folder using Windows Explorer, Total commander and command prompt. Each one of them got confused and tried to delete "Games" instead of " Games". At the end, I just fired up lcc, and made a simple C program calling `ulink(" Games");`. And it worked ;)

Well, you can just use a mounted disk image formatted with a case sensitive HFS as development env. You'll need to mount that every time you restart but that's a minor hassle compared to problems you might run into when using a case sensitive FS for your primary partition.

So far - for development - I haven't had any issues with the case sensitive file system. I guess it's because in the end of the day, it's all tools made to work on Linux I use.

If you're using OS X for an all-round machine, it might be the wrong choice.

Apple themselves used to discourage using a case sensitive partition for the OS. It's probably fine, just not the standard and widely tested configuration. The major hassle starts when one single program you need fails because of the setting - you'd just have to reinstall everything. So the safer way is use case insensitive for the primary and have a dedicated space for development with a case sensitive FS.

From the post: "Homebrew helps you compile different kinds of software on your mac, making it feel almost as easy as apt-get'ing a package on e.g. ubuntu." (emphasis added)

This is something I've noticed from using both OSX and Linux for the past few years — unless software is available as a drop-and-install package, it's universally _easier_ to install in Linux* than it is in OSX. Given the market for OSX and the amount of development that happens on it, I'm surprised the situation isn't better (for OSX), but perhaps that just speaks to the work people have done on package managers for the various linux distributions.

* - Holds true, in my experience, for Ubuntu and Arch. Likely the same for other distros..

Yes and no. Having done professional and personal development for OS X and two linux distributions (Fedora / Gentoo) it is most definately easier on Linux if you want to just hack together some software for your own use. As soon as you want to distribute that software however, the advantage often disappears - you can't use the package manager version of a lib because you need a feature in a later version, or because you need a different option activated during compile. Or you have a patch for said library that was refused upstream.

If anything like this happens in a dependency of your module, you're probably going to have to recompile all of the downstream dependencies by hand, or risk breaking your development system.

To avoid all of this you end up compiling static versions of the libs from your dependencies, and linking them statically to your module. This is particularly true if you don't want to be at the mercy of upgrades to those dependencies in your distribution that break your code.

Looking at things from the other side, there have been various package managers for OS X that give some of the ease-of-development available thanks to yum or apt-get. At the moment homebrew seems to be getting quite a bit of traction, and has addressed many of the pain points that predecessors such as MacPorts and fink have had, making OS X decidely more linux-like when you're writing the software without plans for distribution.

I'm a long-time macports user, and brew just blew me away with how much less work it made me do! (even just in terms of other source builds finding libraries by themselves - a real boon.)

I'd really like to use both macports and homebrew, and switch between them as necessary, but I don't know a good way of doing this. I know homebrew is relatively well-behaved and doesn't really put stuff outside /usr/local, but other things install into /usr/local too so I can't just remove that from my PATH when I want to use macports.

If anyone knows a reliable and simple way to do this I'd be very grateful.

I'm not sure to what extent this works with brew, but you could try using virtualenv to manage different environments?

I've never had to deal with multiple configurations on the same mac (my solution, off hand, would be to use debian VMs, possibly through SSH rather than a desktop GUI if I needed >1 at a time.)

Yeah, I'm thinking about going down the VM route with Ubuntu Server inside Parallels. Seems a bit of a waste of computing resources though.

virtualbox is free! And fine for the purposes of running scientific software.

EDIT: and in terms of resource usage, RAM is cheaper than headaches.

it's universally _easier_ to install in Linux than it is in OSX*

Unless the package is not in the repositories or you want to have an instance with some different flags. One of the really nice things of Homebrew is that besides being a package manager, it is a great /usr/local manager.

For instance, installing something that is not in Homebrew is as simple as setting the prefix to /usr/local/Cellar/whatever/version and then running brew link whatever and it puts symbolic links in /usr/local. Want to make it 'invisible'? Just do a brew unlink whatever

It's also much easier to maintain your own Homebrew repos (taps) than e.g. maintaining an Ubuntu PPA.

Or it simply has a name you don't expect. A lot of the problems I had in Linux, before I got used to it, was simply not knowing what I needed to install.

I'm used to being able to do, 'brew install foo' and not needing to worry if I needed libfoo, libfoo-dev, libfoo3-dev or just plain foo.

  apt-cache search foo
Done :). But I see your point. There's usually the library, the development headers plus static link library, and possibly some command line tools.

And usually the headers aren't that big (with the notable exception of template-heavy C++), so it's much simpler installing everything.

But such splitting primarily occurs in the Debian family.

I wouldn't do the "Enabling apps from everywhere" step. If you have an unsigned app, just right click on it and select open. OSX will ask you once and be fine with it from then on.

If you develop your own applications, that can get very annoying very fast...

It never asks for locally created applications. It's purely for things downloaded over the Internet. In essence, it's a more advanced version of the original "You just downloaded this, are you sure you want to run it" warnings from the file quarantine system introduced with....Leopard, I think?

Yep, its asks it for files with the 'com.apple.quarantine' flag, which is attached to downloaded files.

You can see its value with:

  xattr -p com.apple.quarantine <filename>
And remove it with:

  xattr -d com.apple.quarantine <filename>

Binaries/application bundles that roll out of your linker/xcodebuild do not have that attribute. Hence, no bugging :).

Always install an upstream Python in parallel to the OS X provided version if you're doing Python development. Apple custom compiles theirs, and it's always out of date. In fact, there are Mavericks specific bugs in both latest upstream and in the version included with Mavericks; look for a new Python release in the next few days.

See: http://code.activestate.com/lists/python-dev/125760/

And don't forget to `brew install python`!


When I was running the developer version of Mavericks, the site_packages weren't preserved, which meant pip & setuptools were broken, so I had to reinstall them like so:


then upgrading pip and virtualenvwrapper:

$ sudo easy_install pip

$ sudo pip install --upgrade virtualenvwrapper

If that doesn't do the trick, you might need to install the Xcode command line tools.

[update: oh well, it looks like the parent deleted the question, but for posterity, it dealt with problems with virtualenvwrapper and pip after upgrading to Mavericks]

Wouldn't it be better to not install eggs into the system path and instead into site-packages on a per-user basis? Or, quite frankly, not use the system python at all?

Sure ideally, I've played with pyenv a bit but for my main machine I've kept my systemwide installs down to just enough to run virtualenv and virtualenvwrapper (everything else lives inside virtualenvs).

For several situations though (like when Xcode command line tools stops working after a major OS update) it won't matter. You need a c compiler for several packages and having your stuff isolated per-user or using a non-system Python won't help that one bit.

Before Mavericks, I was stupid (and lazy) enough to use the system Python, so my site-packages folder was completely destroyed on OS upgrade and instead of restoring a backup I decided to finally switch to home-brewed 2.7.5.

Installing numpy/scipy/matplotlib can be pretty tricky because of non-standard homebrew library locations (for stuff like gtk+, freetype2 etc), so you're better off using this http://github.com/samueljohn/homebrew-python and http://github.com/Homebrew/homebrew-science for all your scientific needs. By the way, Qt, Numba and a couple of other libraries seemed to get broken (upstream) in Mavericks, but I guess that wouldn't take too long to fix.

Also, upgrading homebrew libraries on Mavericks requires reinstallation of the entire dependency tree (because newest clang uses the newer libc++ instead of libstdc++).

I dont code in Phyton, but i think that you should add xcode commands line tools in this tutorial. http://www.computersnyou.com/2025/2013/06/install-command-li...

I think Homebrew requires this? Or at least some of the basic packages I installed with it did....

Homebrew certainly requires the Xcode tools.

Instead of installing PostgreSQL with homebrew, I suggest http://postgresapp.com/, "the easiest way to run PostgreSQL on the Mac"

I've also decided to do a clean install of Mavericks on my mini. This time I'm going to try to limit the number of different ways I use to install software. Each standard way of installing seems to handle things differently. Over time the system becomes unknowable and a clean install is needed.

On a clean installation of Mavericks, you should install the latest version of setuptools before you install anything else, as setuptools 0.6 is very broken:

  wget https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py -O - | python

Homebrewed Python includes the latest versions of pip and setuptools, making the command you suggested only necessary if you plan on sticking with the bundled system Python (which isn't recommended for reasons discussed in other comments here).

Why are people still using powerline when there's a pure Vimscript implementation? https://github.com/bling/vim-airline

Or even better, lightline? https://github.com/itchyny/lightline.vim

Why is it even better?

  Why yet another clone of powerline?
    vim-airline is a nice plugin, but it uses too much functions of other plugins, which should be done by users in .vimrc.

So it isn't strictly better just more standalone. I happen to use the other plugins already so I don't mind the dependency.

But I will try airline out. :)

The result of my vexations with powerline -


just use a vm (eg vagrant) or a remote server and be platform independent!

Why use extra RAM on Vagrant when your Mac OS X is a full-featured Unix OS? Don't get me wrong, I know that the more complicated your setup gets, the more it would make sense to use Vagrant - but if you're just developing a CRUD Flask / Django app with PostgreSQL, you might as well just install Postgres.app and develop locally.

I had tons of issues trying to get Postgres to work on my Mac. Lots of other conflicts, too. I think the Mountain Lion OS upgrade fried my host dev environment, too. Eventually, I went the VM route....afterall, you're going to eventually deploy to a Linux box anyway, and there's enough differences that you'll feel the pain when you go to deploy.

And now, I don't have a polluted host OS with tons of versions of software. I can easily try out new technologies, kill the VM, and start over.

Keeping VMs for different projects in their on silo'd around is really convenient.

Switching OSes or upgrading, rebuilding a computer, etc. I don't loose these configurations.

The list goes on and one.

RAM is cheap. I think I paid $100 to upgrade my Mac Book Pro to 16 GB.

I can also switch back to a host Windows machine and take my VMs with me (should the need arise)

>> RAM is cheap. I think I paid $100 to upgrade my Mac Book Pro to 16 GB.

RAM was cheap. Prices have went up a lot. Most DDR3 prices have almost doubled since this past spring/winter. 16GB laptop kits are ~ $140 when they were $75 a few months back. Still relatively cheap, though.

Not to mention Apple only makes 1 laptop that you can add your own RAM to nowadays (13" MBP). It's $200 to go up to 16GB on a Retina.

I had tons of issues trying to get Postgres to work on my Mac

Did you try http://postgresapp.com ?

Yes, I actually have that installed. Back when I was originally trying to get it to work, that wasn't available.

I prefer vagrant for most development projects for the simple reason that I can automate the VM + development environment generation AND commit it to a revision control system (git/svn).

Once set, I don't have to worry about re-installing my development environment - and can just let anyone check out my sources, who then has to do a simple "vagrant up", wait a few minutes, and he's up & running. No worrying about which things he specifically has to install and configure, my vagrant config takes care of that.

Also - the memory requirements are very limited imho, I run most of my vagrant Ubuntu 12.04 server VM's with with 128 or 256mb...

Because even though OS X is a pretty full-featured unix not all software that installs easily on a linux installs easily on a mac os. Vagrant boxes also are platform independent if he had a vagrant development env, he wouldn't have to blog about how to install python. You also can get closer to the target env - i.e. use the python you'll have on the target server etc. You certainly can install all pythons that your projects use side-by-side, but this quickly turns into a mess when you have multiple projects that deploy on different target systems, maybe with different sets of libraries etc. You can test/develop deployment scripts locally. All in all: It's cleaner, less work and more flexible. In general the extra ram is not relevant now that the upgrade from 8G to 16G costs a mere 200 EUR.

> Because even though OS X is a pretty full-featured unix not all software that installs easily on a linux installs easily on a mac os

Having used Homebrew now for many years, this is really just lingering FUD from many years ago. I've spent at least as much time building custom RPM/DEB packages, finding PPAs, etc. for things which don't compile cleanly on a particular Linux distribution. That's just a cost of using bleeding-edge or obscure packages: the right lesson to draw is to avoid taking on the cost of using less stable packages unless you gain significant benefit.

> Vagrant boxes also are platform independent if he had a vagrant development env, he wouldn't have to blog about how to install python

He didn't — that's a local cost he chose to take on for perceived benefits but it's neither necessary not advisable if you don't need something other than the system installed Python (2.5-2.7). This is exactly the same situation as on a Linux distribution.

> You certainly can install all pythons that your projects use side-by-side, but this quickly turns into a mess when you have multiple projects that deploy on different target systems, maybe with different sets of libraries etc.

This is a platform-neutral problem which has been well solved for many years using Virtualenv:


> All in all: It's cleaner, less work and more flexible.

What you described is a LOT more work and adds significant performance and usability overhead to things you do many times a day, particularly if you need to work on more than one project and spend time spinning Vagrant instances up and down, installing updates, etc. rather than doing something productive.

It might still be worth it if you really need to test your deployment process all the time but otherwise that cost adds up. Personally, running locally and testing on EC2 has been much easier, faster, and makes my testing realistic by using the actual real production infrastructure rather than a facsimile crammed onto one laptop.

> Having used Homebrew now for many years, this is really just lingering FUD from many years ago.

Actually, I can cite an example. Try installing libmediainfo on your mac. It's a pita that requires hacking the makefiles in obscure ways. Even after hacking the makefile it installs in different locations than on linux, making all software that depends on it more difficult to compile. Oh, and yes, there's a gui package but it doesn't contain the required library files and headers.

Homebrew is fine for everything that homebrew packages exist for. Anything else you'd better hope that the authors provided a working makefile or be prepared to learn how to fix it.

> spend time spinning Vagrant instances up and down, installing updates, etc. rather than doing something productive.

I install updates when the target env installs updates. I share the puppet/chef files that are used to update the target env - so the maximum amount of time I spend doing unproductive things is typing "vagrant up" in the morning and "vagrant suspend" in the evening. Each one of those commands runs 30 seconds.

> http://www.virtualenv.org/

Since when does virtualenv install the actual pythons?

> Personally, running locally and testing on EC2 has been much easier

You're certainly aware that you can use packer and vagrant in combination to build the same box for virtualbox/vmware for local use and as an AMI for use in EC2, making your development env a clone of the production env? Just with faster connectivity and no need to be online for work? Vagrant can even spin that ec2 instance up and down for you.

yeah that works okay if you only need that, but then comes another project or new requirements and suddenly you run mongodb, redis and some background worker stuff as well and this slowly pollutes your system.

I did it that way for a long time too, but running vms became so convenient with vagrant and all the box configurations you can download that its kind of a no-brainer as long as you have 8Gigs of RAM or more.

Because your settings won't always be the same as your production machine if you run a Mac?

another neat trick: - install brew cask https://github.com/phinze/homebrew-cask to have homebrew manage other apps: chrome, alfred ..

Sadly it seems that many python packages have big issues with overloaded =, which means that brew can't compile many tools I need. so If you need anything relying on boost you might want to delay the update


Is the sarcasm necessary? The guy mentions Django a few times, in which case a web browser _would_ be necessary.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact