Ask HN: How do I teach intermediate Python engineering skills?

[_9iuc]

I have several reports with backgrounds in science or ops who can write python to accomplish tasks, but are bad at engineering (i.e. the type of code you would write for a large project).

I feel that much of my own engineering skills were acquired through a time consuming combination of reading lots of well crafted code and getting feedback pointing out my mistakes from more experienced engineers. Are there resources that would be more time efficient to fill some of the gap for these folks?

What do folks on HN suggest?

[hackermailman]

Volume 2 here teaches Python optimization https://foundations-of-applied-mathematics.github.io/ or you could go through a copy of the Pragmatic Programmer with them https://www.cs.cornell.edu/courses/cs3110/2020sp/reflections...

All the "engineer"-ry things like constructing testing oracles or understanding code complexity this can all be found in Programming and Programming Languages https://papl.cs.brown.edu/2020/ that uses Pyret which has similar enough syntax to Python none of your bad engineers will be lost. There are lectures for it too: https://learnaifromscratch.github.io/software.html#org4eef4d... you could teach this class to them the lectures really teach you concepts like trees and graphs and how to reason about them.

[ghj → hackermailman]

That Foundations of Applied Mathematics series is such a gem! I skimmed it a bit and it covers a huge breadth of topics ranging from from numerical methods to traditional algorithms. All the while being practical about how you would actually apply it in the real world (for example it has code samples for using the networkx library instead of writing your own graph algorithms, and many more code snippets for using numpy/sympy/cvxpy/etc).

Thanks for linking it!

[rahimnathwani]

Maybe these:

Practical Object Oriented Design (Metz)

Refactoring (Feathers)

Test driven development with Python (Percival)

Missing Semester: https://missing.csail.mit.edu/

[semicolonandson]

If you're comfortable broadening the scope to _non-language-specific_ engineering skills, check out my Semicolon&Sons screencasts.

The drive behind this project is to demonstrate non-beginner issues facing professional developers. Instead of ephemeral syntax tricks or "how to use library x v5.3", the focus is on skills affecting codebases over decade-long timelines, such as:

- gaining confidence in code with (non-brittle) integration tests

- deep understanding of what the OS offers (and fluency in the command line)

- building excellent monitoring and instrumentation for rapidly diagnosing production issues

- vetting dependencies so they don't bite you in the ass

- architecting code for re-use and intuitive navigation for freshly onboarded team members

- the paramount importance of data integrity (and how to enforce it at the DB level)

etc.

https://www.semicolonandsons.com/

[probinso]

Introduce them to Rich IDEs and help them set it up with a consistent configuration so they can help each other. Put git hooks in to control for silly errors. Create a style guide that scales with the importance of the project. for understanding the language more deeply, "python cookbook" by David Beasley. ask them to make everything pip installable. introduce code walk throughs to your workflow, with the intention of group peer-review.

When you see an interesting block of code that you see could be refracted, hold a session showing your refactoring of that code. You'll have to do it twice, once to figure it out, second to present. It's very time-consuming but it's the best way for people to learn

[verdverm]

Hire a Python dev / eng to work with them. Pairing eng with domain is always a good choice

[_9iuc → verdverm]

I've never done pairing, but this seems like a good suggestion. I've also been looking for an excuse to try https://tuple.app

[verdverm → _9iuc]

I'm not saying pair programming, though that is an interesting addition. Just having them collaborate on the code, knowing where their roles lie. You see knowledge transfer happen, which will benefit both sides, because you also want to expose engineers to the theoretical. They'll come back with wild ideas that the domain people refine. It's really quiet beautiful to watch happen, they each make each other better together.

[njsubedi]

I'd suggest letting them dive into your code, and make minor changes. That will increase the chances of them structuring their software the way you do, which makes teaching easier. If they get stuck somewhere, getting help is much easier for them.

[op03]

Just hire better programmers and keep the scientists focused on science. It's just much more efficient. Its like wasting time getting your pitcher to improve his batting skills. Don't mix roles up.

[undergrowth54]

The Wikipedia article on Working Memory followed by at least the first four chapters of A Philosophy of Software Design. This is really good for providing an underlying “why” for any other technique.

[kazinator]

I would start with these:

1. Commit hooks that lint code (pylint3, flake8, ...). Code must be clean before it is merged.

2. Introduce testing: code must be covered by tests.

3. Require documentation: there must exist an independent specification of what the code does, at least from an external point of view: what are the interface contracts.

Once you have code that is specified and covered by tests, only then do you have a proper environment for transforming badly structured code that works into better structured code that still works.

Other than that, organization of programs is a big topic that requires study of theory. For instance, you could probably rediscover the idea that code should be organized into modules such that the content in a module belongs together, and isn't tightly bound to content in another module. You might not have a name for this but an intuitive feeling for it, which is very good. But if you study software engineering, you will learn about cohesion and coupling directly, so you don't have to reinvent or rediscover the wheel. The cumulative effects of learning material directly add up; you can shave years and years from the time it takes to become competent.

I think that the decline in the "Wirthian" languages like Modula 2 has been detrimental. The biggest initial boost that I got in the understanding of structured, modular programming was from programming in Turbo Pascal and Modula 2 in the late 1980's.

In Modula 2, you structure the program in modules which have a clear interface and implementation, declared in separate files. Modules declare their dependencies on other modules. From this, the compiler knows all the dependencies. There is no need for any Makefile or dependency generation; you tell the compiler to build the program, pointing it at the main module and that's it.

Each module has a block of global initialization statements. The compiler/linker, knowing the dependency among the modules, ensure that these statements are executed in module dependency order: if A depends on B, B's global initialization runs first, then A.

When I started coding in C, I realized immediately that this is missing: there is just a main function and after that you're on your own. Moreover, unless you declare your dependencies to the Makefile or IDE system or whatever you're using, or get it to figure them out externally, you will not get correct incremental builds.

So I simulated modules by carefully structuring header files, and giving each module a global initialization function, and then call these in some disciplined way, like from one big init function.

I later saw (thanks in a large part to open source) that other people's big C programs did things like this. I felt that programmers who had exposure to a language supporting modularity had an edge in understanding the issues and motivation for good organization patterns in C programs compared to those who were just getting randomly burned and improving their approach by trial and error.

[DrNuke]

Case studies from interesting and / or hyped domains maybe?