
Using a logbook to improve your programming - jmlr
https://routley.io/tech/2017/11/23/logbook.html
======
leviathan
This article seems lacking some examples. I have found that such ideas are
nice in theory, but when it comes to practice, they tend to fall short.

So the author says that it's a good idea to write logbooks, and that it will
help. But no mention on how it would help, and what would you actually do with
them after you write them up.

A few concrete examples would have been nice.

If the idea is to help you solve the problem more efficiently, why should it
be formalized? as opposed to scribbling notes on a notepad that you throw away
at the end of the day.

~~~
barrkel
Well, I use this logbook approach, in a text editor with persistent undo
(which allows a few tricks, like emulating folding by deletion and relying on
undo history to unfold). I approximately follow this algorithm:

\- Consider the high-level goal; can it be completed immediately? if yes, do
it, otherwise break it into smaller parts.

\- First part is usually some kind of research. If it's a bug, it'll be a
reproduction. If it's a well-specified feature or a refactoring, it'll be
locating all areas of code that need to be updated. If it's design, it'll be
an initial sketch.

\- Subsequent parts depend on the task at hand. For a bug, I generally try and
work through to a minimal set of reproduction steps. For a feature, it'll be a
generated list of tasks that came out of the research step.

\- When working on any given task, I add sub-entries in the form of a
dialectic. For a bug reduction, I come up with a hypothesis, then try and
prove or disprove it by testing. For design, I propose one approach, then
consider the implications; then another approach, and so on. For a feature,
tasks generally get added for any "todos" that crop up as I implement things,
to ensure I complete everything.

I essentially record salient details of my internal monologue to ensure that I
can pick up context again easily, and generally fit more into my working
memory by offloading it into text.

I reckon it made me perhaps twice as effective an engineer, particularly in
distraction-prone environments. It's also made me a lot more confident that
I'm not relying too much on memory as I get older - not that I think I forget
much now, but I believe I won't be very dependent on a large working set.

~~~
fao_
Do you have any ideas on how to adapt this to someone who doesn't have a
"present" internal monologue?

If I concentrate, I can sense something akin to the typical "internal
monologue", but it isn't present enough to be copied down, and even when it is
it tends to move so fast that I only really get the conclusion.

~~~
bramen
Interesting, when you think about something you don't ever hear yourself
talking it out inside your head?

~~~
fao_
I can hear small parts of it now and then, and I can definitely 'hear' myself
read and write text, but most _thought_ doesn't pass through my internal
monologue. I can't tell if it's because it's going too fast (Which is
certainly possible), or because of some other reason.

~~~
bramen
Could be you're a more visual thinker?

------
JAFTEM
Surprised no one has mentioned doing this in a physical notebook with a pen. I
started doing what the author describes a long time ago to document bugs to
come back to later. I soon realized it's just a good way to keep my thoughts
organized and be more productive. I tried doing something similar with a text
editor but grew tired switching screens or tabs. A notebook is also a bit more
accessible than a laptop when you leave your desk. During lunch I can read
through my notes.

~~~
swah
Most likely you love high quality notebooks and writing instruments, and you
probably have neat handwriting.

I rather have files that I can understand later, and easily grep as a bonus...

(I do admit that crossing done tasks on a piece of paper feels very good.)

~~~
50CNT
I like to think of blank pages as mini whiteboards. With a multifunction
copier i can print out lined, grid paper or specialty layouts and whip it
through the scanner once im done with it keeping a digital copy. 3USD buys me
500 pages, and with a laser printer [250USD], toner refill kit, etc. that
comes out to less than a cent a page, or 3 cents if I amortize the printer
over 10k pages (assuming it breaks once it hits its one month maximum duty
cycle, which is stupid conservative).

That plus a nice fountain pen [Lamy Safari 18.7USD], ink [J. Herbin Perle Noir
7.2USD/30ml], mechanical pencil [Lamy Safari 12USD], pencil leads [Uni Nano
Dia 0.9USD/20leads], technical pen [Rotring Isograph 16.3USD], and I've got a
nice little setup.

~~~
k_sze
“I like to think of blank pages as mini whiteboards.”

This. I never enjoyed drawing diagrams in Inkscape, though my boss swears by
it.

I’d much rather draw with pens on paper and then digitise by scanning.

~~~
50CNT
I could see your bosses workflow working as long as he has a library of common
elements. For graphs I tend to use yED, which is nice. You know how LaTeX has
the mantra that you input the data, and it worries about the layout for you?
yED is similar, you add nodes, connect them, and then you chose from one of
two dozen layouting algorithms, like "hierarchical", "swimlane", or even
"family tree".

------
lucisferre
Based on a some of the comments so far I think people are glossing over the
key elements of the logbook. This isn't simply an activity log.

The first step is to describe the nature of the problem as well your planned
solution before beginning. I think this can be a pretty powerful practice in
being mindful.

All to often I find myself jumping into a half-baked solution while at the
same time having only half-understood the problem.

~~~
dominotw
>All to often I find myself jumping into a half-baked solution while at the
same time having only half-understood the problem.

This is what kills me in tech interviews. Unfortunately its hard to calm down
and think quietly when you are being hounded by the interviewer to "think out
loud".

------
cgrusden
I did not go to college/university for engineering, but just hearing the tools
of 'real' engineers is great.

I've been opening up Jekyll in the morning as the start of my day. I write the
blog post for the work I'm about to do. I've noticed that I'm getting better
at explaining the problem clearly and the solution. When there is a paragraph
that feels out of place I yank it into a different file immediately to get it
out of the current flow of the post. The paragraph felt out of place because
it was probably a distraction from the problem Im trying to solve today. Also,
the great thing about this technique is that I have a blog post at the end of
the work day that I can look back on and say, "Damn, I shipped something" if I
didn't feel like I shipped anything else (common problem in our field of
software developers)

------
perseiden
Heh funny, I started writing a "labbook" like this ~6 months ago with markdown
writer in atom for every day. I have a background as a Lab technician where I
was used to keep a daily detailled labbook so I didn't have to develop that
habit..

I like to structure my programming lab book in a similar way, for every task/
bug/ project I try to jot down a description, hypothesis, expected outcome and
document my progress with code snippets / outputs / plots. I also keep a Todo
list at the top to visualize my current workload. It really helps if you have
to come back to a problem later on or just to get a good overview of your
current task. I feel lik this also works nicely in conjuncture with GTD to
increase my productivity.

------
partycoder
Keeping a journal is particularly useful to resume your work.

Before lunch, or a meeting, or end of day, or when someone interrupts me and
needs something, or any form of break, I "serialize" what I am working on in
the journal, and then I read it when I am coming back.

In this way I can keep my mind clear and focus in the next activity.

It is also useful for standup meetings, retrospective meetings, etc... when
you often need to describe what you did yesterday or this week.

------
neya
I believe many of us already do what the author suggests, maybe in a slightly
different way.

Here's how I use my own flavor:

1) I create a GOAL. The GOAL is my ultimate big picture, which will give me
immense happiness. So, the most recent example is, I wanted to build an
E-Commerce application.

2) I map out the PATHS to achieving my GOAL. So, what I consider PATHS is:

a) What programming language should I use? b) Which APIs should I use, Paypal
or Stripe to accept payments[1]? and so on...

3) I follow a flavor of DDD[2] to come up with the important modules I need to
have for a functioning E-Commerce project.

Examples: I will need a Order Management module, I will need a Marketing
module (for sending emails, running campaigns). I will need a CMS module to
host product pages.

4) For each of the modules, I will create an issue on Github and add a
checklist[3] of the stuff I need to get done.

5) Everytime I make a commit, I will reference this issue and the tasks along
with it. Eg. "Removed Paypal #12"

6) At the end of the week, I will review the commits and evaluate whether this
particular method I used was worth it. A good use case would be experimenting
with a library and comparing it with another - Eg. Bootstrap vs Semantic UI. I
will use the learnings from this workflow in my future projects.

7) I will also see if something can be automated. For example, for my current
E-Commerce project, I use Phoenix/Elixir. It has it's own set of scaffold
generators, but they're opinionated (they use Bootstrap). So, for every
project I got tired of replacing bootstrap and customizing the generated
scaffolds. So, I wrote a custom generator that drastically saves me a lot of
time for large projects.

Hope this helps :)

[1] Not sure why anyone would use PayPal in this day and age tho.

[2] [https://stackoverflow.com/questions/1222392/can-someone-
expl...](https://stackoverflow.com/questions/1222392/can-someone-explain-
domain-driven-design-ddd-in-plain-english-please/1222488#1222488)

[3] [https://github.com/blog/1375-task-lists-in-gfm-issues-
pulls-...](https://github.com/blog/1375-task-lists-in-gfm-issues-pulls-
comments)

~~~
chillax
Somehow got me thinking about impact maps again:
[https://www.impactmapping.org/drawing.html](https://www.impactmapping.org/drawing.html)

------
tigershark
Maybe I go against the common thought here, but for me I think it is
completely useless. Whenever I start working on a problem I obviously have an
approach in my mind, but very often I find out that is the wrong one. And I
can realise it only after I start working on the actual problem and solving
it. At a certain point I will find some obstacle that will make me rethink the
whole approach. For this reason I can't really see the added value in spending
more time before actually tackling a problem once you have a plan in your
head. The most important thing to do when solving problems is to be prepared
to change your point of view multiple times and to attack the problems from
several different directions. I'm sorry but I really can't see how writing in
stone your approach before you fully understand the problem can help you in
any way...

~~~
andreareina
The way I interpreted it was basically an iterated Feynman Algorithm[1]:

1\. Write down the problem

2\. Think real hard

3\. Write down the solution

4\. Write down how well it worked

5\. GOTO 2

I do something similar using org-mode, and it's useful when, after some time
has passed and I need to do the same thing again, I can go back to my notes
and not start from scratch again.

[1]
[http://wiki.c2.com/?FeynmanAlgorithm](http://wiki.c2.com/?FeynmanAlgorithm)

------
notinventedhear
I've found keeping a software-engineering logbook very useful at work.

Recommend the vim-wiki plugin for this
[https://github.com/vimwiki/vimwiki;](https://github.com/vimwiki/vimwiki;) in
any vim-window the logbook for today is accessible via [leader-W-leader-W].

~~~
mmrezaie
or maybe [http://orgmode.org/](http://orgmode.org/) if you are using emacs. I
do not know how awesome orgmode was, and I was missing it. plus you can export
it even to latex if you wanna track your research and write your thesis.

------
mattfrommars
I find this interesting. Mainly because, I've been trying to implement
similiar strategy on my daily works to fight off low productivity hours and
really figure out where all my time is going. One of the things I've been
doing is noting down the time and date when I set to do things. Seeing
something similar presented in the post with more formal and clean approach
made me happy - as in being on the right track.

But than again, there was TED talk of all this organization is ultimately
shifting the energy from actual. Which led me reducing number of
'productivity' application and sticking with a few.

Wunderlist Onenote Evernote - mainly for the phone.

------
HumanDrivenDev
Good idea, but as others noted it was a bit light on details.

Some more concrete suggestions from what appears to be the university of
Idaho's Electrical Engineering department can be found here:

[http://www.ee.uidaho.edu/ee/power/jlaw/COURSES/CAPSTONE/F05/...](http://www.ee.uidaho.edu/ee/power/jlaw/COURSES/CAPSTONE/F05/handouts/EngineeringLogbooks082205.pdf)

------
torbjorn
What the author says about aliases in the bashrc being evaluated only once
when the bashrc is run when the shell session is opened is wrong.

Go ahead and try this:

Put,

    
    
        alias lb='vim ~/logbook/$(date "+%s")'
    

in your bashrc.

Source the bashrc.

Run the lb command. It creates a new file with a different epoch timestamp
each time. This `lb` command doesn't need to be a bash function. It can be an
alias and work just fine.

Maybe how this works in zsh is different...

~~~
nerdponx
Zsh is different, although I'd be surprised if this wasn't the Bash behavior.

It doesn't make sense. If you do

    
    
        alias e=vim
    

it's not like Vim opens when your shell starts.

The only way to have a "precomputed" alias is to get the quoting wrong, and
leave $() in double quotes instead of single quotes, meaning it will be
evaluated before the alias is even assigned.

------
krupan
Emacs org-mode is great for this.

~~~
gavinpc
I've been using org-mode for about ten years to document moment-to moment
progress on tasks. I'll give a basic summary of the problem, copy in
significant commands and results as I run them, and ask the next questions to
drive the next steps.

Some features that make Org useful for this:

\- I'm clocked into the task so I can always jump to it easily, even as I hop
around other emacs buffers

\- supports code snippets with formatting in any language

\- it's easy to add timestamped notes or write in free form

\- can export the subtree to HTML or Markdown or Latex/PDF, to share with
others

\- GitLab will apply basic Org formatting (as with markdown), so I can share
them directly just by pushing

\- supports regex search across all agenda files, regardless of where they
are. This is nice if, say, I know I had a similar issue with e.g. some docker
command but don't remember which project it was on

~~~
meesterdude
i'd be interested in learning more about this and maybe seeing some real-
worldish examples.

~~~
abhirag
Not the OP but I use org-mode to write articles for my website and also to
manage my dotfiles, this--
[[http://abhirag.in/articles/org/i3_setup.html](http://abhirag.in/articles/org/i3_setup.html)]
is just an org-mode file exported to html, you can also see the literate
dotfiles on that page, they have been generated using org-babel. I am also
trying to setup a proper literate programming workflow using org-babel but
that is still a work in progress. For a glimpse of what org-mode supports you
can look at this this article of mine --
[[http://abhirag.in/articles/org/custom_org_theme.html](http://abhirag.in/articles/org/custom_org_theme.html)].
Hopefully these examples helped but I have only scratched the surface here,
have a look at-- [[http://orgmode.org/worg/org-
tutorials/](http://orgmode.org/worg/org-tutorials/)] and other stuff on the
official website to find more information :)

------
sent-hil
This is similar to what I've been doing on and off for nearly 5 years now:
[https://gist.github.com/sent-hil/3444793](https://gist.github.com/sent-
hil/3444793), except on paper.

[https://news.ycombinator.com/item?id=4448361](https://news.ycombinator.com/item?id=4448361)
has some good tips and tricks.

------
nicoster
I have similar ideas. At the beginning (20 years ago) I use windows
notepad.exe. It has a trick that if you put “.LOG” on the top of a txt file,
every time you open it with notepad, it appends current date to it. Later on I
switched to Evernote, then Apple Notes.app and settled. Recently I added a
service with Automator to allow me insert current date to a note. Quite
convenient.

~~~
j_s
F5 inserts the current date / time in Notepad.

------
kilon
I don’t keep a log but I use pen a paper to create diagrams which visualize my
problem and potential solutions. It has been very helpful and good excuse to
rest my eyes from the monitor. I don’t care about dates cause I like to work
in a chaotic kind of way and I don’t like planning ahead I prefer exploration
and critical thinking during execution.

------
rmchugh
Might as well auto generate a header for the file...

    
    
      function lb() {
        today=$(date '+%Y-%m-%d')
        fp="/home/you/logbook/${today}.md"
        if [ ! -f $fp ]; then
            echo "# ${today}" > $fp
        fi
        vim $fp
      }

------
samuell
My "bash function" adds just a few more lines, so as to git add/commit/push
after closing the editor, and adding a good title by default:

\----

journalfile=$(date --utc +%Y%m%d-%u.md);

if [[ ! -f $journalfile ]]; then

echo "# "$(date "+%A, %e %B, %Y") >> $journalfile;

git add $journalfile;

else

echo "File already exists: $journalfile";

fi;

vim $journalfile;

git commit -m "Changes in journal file $journalfile" $journalfile;

git push origin master;

\----

Source:
[https://github.com/samuell/mdnote/blob/master/editnewjournal...](https://github.com/samuell/mdnote/blob/master/editnewjournalfile.sh#L3-L18)

------
jungletime
I've been using Typora to make my todo lists recently. It's a markdown editor.
Super simple to type, and you can really got some professional looking layout,
with headings and titles quickly too. Recently they built in a file browser.
Open one file in a directory, and it will show you all the files in side
panel. Which in my case are just todo lists for day of the week. But it could
be any type of notes. Highly recommend it. For Example typing ###Title would
give a large title typing - [ ] will give you a checkbox, and so on.﻿

------
kayoone
I like the idea, i just think MD files might be too limited. I really came to
enjoy Onenote because it is so easy to put unstructured data, images, links,
videos and even draw on top of it all with a stylus but i didn't use it for
every task i do. Will try that in the future as i often have issues resuming
old tasks or remembering some details to a problem in the past.

------
nohope
This is a nice and simple! Usually I keep a logbook separated for each project
but it looks more like a diary where I record failed approaches and new tries
to solve a problem.

For more structured documentation, I keep separated text files for each
solution that I can easily find if I need it later.

EDIT: Also, scratching your plan with pencil and paper before implementing it
helps a lot.

------
keithnz
I keep a log of things I don't really know but should invest some time in to
learn it properly as I make use of it. That's quite useful, not so much the
log, but the forced awareness of making a mental note. I've also been thinking
of keeping a log (repo with notes) of the incremental experiments I do when
learning things.

------
henryslater
In between commit messages, bugs and design docs, I'm not sure what I'd put in
a log book.

~~~
iAMAGuest
\- The logic/design of parts of the code base which you might newly encounter

\- What work was done for the day, yes you might use something like Jira but a
journal can be also good at the same time

\- New things you learn.

etc etc etc

~~~
henryslater
Each to their own of course but it seems to me, there's better places for all
of that. If there's logic or design that's complicated and not properly
commented or documented, then that's worth doing and sharing with the team.

Work done for the day for me is typically captured in commits or bug updates -
that way my colleagues can easily take over if I win the lottery or are
suddenly taken ill.

And stuff I learn is learned, so I don't need to write that down :-)

Working in a team, shared information is critical and logbooks just aren't the
best way to achieve that.

~~~
iAMAGuest
> If there's logic or design that's complicated and not properly commented or
> documented, then that's worth doing and sharing with the team

It's the meta also, it is capturing the models mentally and ideas and notes.
This reduces cognitive load, and actually by making notes increases the
chances of you remember something.

> And stuff I learn is learned, so I don't need to write that down :-)

Hey if thats the case then fine, but generally memory deteriorates, recall
becomes more difficult and the details change.

> Working in a team, shared information is critical and logbooks just aren't
> the best way to achieve that.

Log books are not generally for the team, they are for you.

But as you say each to their own.

~~~
maximg68
This does not have to be "either-or". As we see an increased focus on teamwork
this could evolve into a shared (electronic) team logbook.

------
pwelch
I’ve been exploring keeping a logbook for awhile. Some apps I recommend:
Quiver, Boostnote, and jrnl.sh

All of them have open data formats so you’re not locked in and can save
everything in git which are requirements for me.

------
ioddly
I find writing documentation as I go to be a particularly good form of this. I
start before I fully understand how I'm going to solve something, and only
check the documentation in once it's correct.

------
jungletime
Typora is a great markdown editor for something like this.

------
amelius
What could help is if you take pictures of the relevant logbook pages with
your smartphone, and then commit them into your git repo, along with your
code.

------
wordpressdev
I keep a simple text file to log features and bugs. This approach of creating
date-based files seems cool. May try it out as well.

------
lldata
Free tip: commit often and write proper commit messages.

~~~
loving-g
If you change organisation and were working on a closed source project, you
lose everything. You need to keep your notes on a higher level though (i.e.
not describing confidential information). Plus I believe that a commit is
generally only covers the third point ("Describe the process of carrying out
the method").

------
tomtompl
I started doing this "naturally" :)

