Hacker News new | past | comments | ask | show | jobs | submit login
Automate the Boring Stuff with Python (automatetheboringstuff.com)
282 points by adamnemecek on Apr 21, 2015 | hide | past | web | favorite | 51 comments

This is awesome...I've actually been writing a Python 3.x guide like this for my own class...the more I teach programming, the more I find it necessary to tell students that programming is about learning how to make computer do shit work so that you can do the "real" work (at least, the thinking/designing phase). And so when figuring out what to do with programming, focus your programs on doing things you've already done before...just like when you learn Mandarin Chinese, you do it to be able to express things you've already expressed before in English, but to a wider/different audience.

edit: I had pitched this topic for a conference, though I'm mostly expecting it to be rejected for, well, being too mundane :) http://srccon.org/sessions/proposals/#proposal-106215

> Programming creates so many technical and creative inventions that it's natural for aspiring programmers to dream of big projects in the cloud. But this ambition ignores the actual goal of programming, which is almost completely about making machines do mundane work. And it is counterproductive to learning how to program, which requires consistent practice as in every other form of literacy and art. So this session will be about mundane programming. Programming not to be the next Zuckerberg, or to get a better job 3 months from now, but to make today or just the next ten minutes more enjoyable. Instead of focusing specifically on how to code, we'll expand upon the reasons of why we code (though seeing is often believing when it comes to code, so feel free to bring both ideas and Gists). And we'll trim the personal prerequisites of programming, which don't include being an entrepreneur, having a profitable idea, building a website, contributing to open source, or changing the world or your career. Programming can be learned, and done, with a willingness to learn and a wide variety of small problems to practice upon.

If your talk gets accepted please post the video and/or transcript.

Yes, post the video!

It's a great title. "Automate the Boring Stuff" seems to be the direct replacement (more or less) of "programming" for me. Maybe people without programming experience don't usually realize that programming is mostly about "automating the boring stuff". This title appeals for this simple "use case" which could win against the "scariness" of programming.

Or maybe it's not that scary for people to learn programming if you don't mention the word "programming"? It doesn't seem to be the intention here, the first part's title is "The Basics of Python Programming". But it seems to be a good strategy to appeal with the title first than in the TOC face the reader with the truth: what you are going to read about is programming.

I know, I speculate too much. But I really like this title. I can imagine a businessman in a bookstore at the "computer" section looking for Excel or Word books looking into this book, while filtering out all "programming" books in his vision.

I agree the title is great, but unfortunately I fear that same business man will read the table of contents and go so what is the boring stuff that I can automate. I hope that the text brings to life some of the problems that the programming constructs can automate.

Author here. I'm hoping the back cover material will work, and the opening story in the first chapter is catchy enough:

“You’ve just done in two hours what it takes the three of us two days to do.” My college roommate was working at a retail electronics store in the early 2000s. Occasionally, the store would receive a spreadsheet of thousands of product prices from its competitor. A team of three employees would print the spreadsheet onto a thick stack of paper and split it among themselves. For each product price, they would look up their store’s price and note all the products that their competitors sold for less. It usually took a couple of days.

“You know, I could write a program to do that if you have the original file for the printouts,” my roommate told them, when he saw them sitting on the floor with papers scattered and stacked around them.

After a couple of hours, he had a short program that read a competitor’s price from a file, found the product in the store’s database, and noted whether the competitor was cheaper. He was still new to programming, and he spent most of his time looking up documentation in a programming book. The actual program took only a few seconds to run. My roommate and his co-workers took an extra-long lunch that day.

Hi Al, I looked for a way to send feedback on the site, but couldn't find anything. So, posting it here. On the menu of chapters (displayed when you click on the floating menu button on every page), the sorting is broken; lexical order instead of numerical order is being used.

Thanks for writing this book and making it available under the Creative Commons License.

I was thinking about similar concepts and problems then I decided to create a framework [1] that could possibly solve those problems.

I created a lightweight system, that serves web pages, added proper authentication, storage (mongodb), memory storage/cache (redis) , simple ORM (mongoengine), automatic backend, background processing/scheduling (celery), email integration, single command deployment (automation)

I have also demonstrated how to do scrapping/scheduling/email sending in this tutorial:


and this one explains how to automate the deployment of everything you create with the framework with a single command:


Hope someone will find this helpful.

[1]:My framework is called Enferno: http://enferno.io and runs on top of flask.

This is awesome! Thank you.

I'm very interested in this book. However, the author leaves no way for me to be updated when it's released. I have to bookmark the page and check later if I remember. For a book on automating, that's asking a lot of me. ;)

You can order at No Starch Press: http://www.nostarch.com/automatestuff

You can order it now from the No Starch Press site, using code PYBUTLER for a 30% discount: http://www.nostarch.com/automatestuff

You'll get the full PDF/epub/mobi version immediately, and the print book will be shipped soonish. (Or maybe immediately. They've come in from the printer and I have my author copies.)

I bought the PDF version of this a few weeks ago and it is pretty straightforward. Doesn't really teach you anything about classes/object-oriented programming. It really has a strong focus on getting sh*t done and is well-suited for it. Overall I would recommend this if you already have some programming experience.

By the way, if you have the book and are experiencing problems with `pip3 install python-docx` on OS X Yosemite, the solution is to install the Xcode command line tools first:

xcode-select --install

Then `pip3 install python-docx`

Yeah, I've always been of the mindset that until your programs get over 1000 lines of code, you probably don't need OOP.

Thanks for the pip3 tip, I'll post it on the website as I get content up.

Honestly, that's all that many people need.

Object orientation and concepts like those are useful for big systems, but simple automation doesn't always require it.

Hi, author here. The HTML version of the full book content will be up in a few days. (I'm formatting it right now.)

Meanwhile, you can use PYBUTLER for a 30% discount on the book/ebook, and I'm offering a 70% discount on my other books: http://inventwithpython.com/blog/2015/04/21/celebrating-the-...

Basically, it's a book for complete beginners who don't necessarily want to be software devs but do want to automate parts of their office job. Part 1 is a bare bones Python tutorial (I skip OOP entirely) and Part 2 covers practical stuff: moving/renaming/copying files, updating Excel spreadsheets, parsing PDFs and Word docs, web scrapping, GUI automation.

This is very cool! I think this'd be a great incentive for people to learn programming and a great way to get into it - I know I've gotten a few folks to look into it purely by showing them how much easier life can be when you automate things. And people are more motivated to learn when something is solving real problems for them.

Looks interesting. It sounds like it will attack a lot of the first things I did actual (non-homework, non-toy) programing for, which are also things I've tried (unsuccessfully) to convince others to learn just a bit of programming to be able to do. I hope this book accomplishes that.

Great title, great cover, nice job Al!


This is kind of how I got my girlfriend interested in programming. She was killing herself at work doing stuff manually with QGIS that could be automated with python.

Any recommendations for similar material but for Javascript?

Any recommendations for similar material but for Ruby?

Every Day Scripting with Ruby by Brian Marick, published by Pragmatic Programmers.

I'm not sure if this is a weekly release of a topic or a total release on April 25th. Wish there was a way to get in touch with the author!

Hi, author here. My email is al@inventwithpython.com and Twitter is @AlSweigart. Please do send me feedback/questions.

I'm currently working on the HTML version of the book to be posted online, which will be at http://automatetheboringstuff.com

This book title seems also a great proposition for a software company aimed at improving and automating enterprise office work...

Author here. I guess so, but there are plenty of software companies with products to "increase office productivity".

But at the same time, there are a lot of times where you need a very specific, one-off, 30 line script to do something for your particular job/company. This is where knowing a general (and easy to use) programming language like Python comes in handy.

Hence why I wanted to write a programming book for office workers who wanted to skip the computer sciencey and software engineering aspects. Who cares if a throwaway script is O(N^3) if N is going to be a few hundred? Most of the time these things don't matter, because it's still faster than doing this stuff by hand.

Looks great! Was pretty bummed when I saw the April 25th release.

E-books are available now at http://www.nostarch.com/automatestuff

Author here. Yes, I can confirm that if you order the ebook now you will get the full ebook. The physical books have also been printed, so they'll probably get shipped before the 25th (don't quote me on that).

but we cant download it unless we buy the physical copy !!

I'm not sure where you're seeing that. I see an option to buy the e-book only for $23.95 and the page has red text that reads "Full PDF, ePub, and mobi ebooks now available!". Am I missing something you see on the page?

Author here. The 30% discount code is PYBUTLER, which you use on the No Starch Press site: http://nostarch.com/automatestuff

> His books are freely available under a Creative Commons license...

So perhaps they will be available for free from April 25th?

He posted somewhere that the free html version is almost complete and will be released in a few days.

This is cool but why wouldn't you just do all this in bash?

Because I've noticed that my general happiness in life is inversely proportional to the time I spend writing bash.

Python works better on windows than bash does. This is for less technical people who may not want to install and maintain cygwin.

Are you really asking why one wouldn't "read PDF and word documents", "parse CSV and json files" or "manipulate images" in bash?

You... probably aren't the book's intended audience. :)

But seriously, Python makes easy things easy. In bash, just to extract a substring you need to do something like echo someletters_12345_moreleters.ext | cut -d'_' -f 2 That's not very beginner-friendly.

Bash is great for really simple things, like renaming a load of files in a loop but the bigger the scrip gets the more I would move towards Python. (i.e. more than one nested loop). Plus somethings are horrendous to remember the syntax for, like capturing a regular expression group.

(My personal choice would probably be Perl for many of the tasks, though I would point beginners to Python for its readability).

Bash has more "gotchas" to remember, and tends to be less readable.

Because if you're going to learn a programming language to automate the boring stuff, Python will be easier and take you farther.

This is cool. Python 2.x or 3.x?

It's python 3

thanks, and also - :(

Why would any new material, educational material especially, ever use Python 2?...

don't think there was room for a downvote really... I'm used to python 2 syntax and would argue more than 50% of python enthusiasts prefer it.

There is very little stuff that is different between the two languages that you are likely to encounter on a day to day bases.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact