Related to this, I wrote a CLI tool called Ledger Reconciler[0] to automate the process of logging into all my financial accounts and inputting those transactions into Ledger. It uses headless Chrome underneath (via the puppeteer API[1]) so it could theoretically work with any system without needing to rely on public/private banking APIs.
I also tried to make the plugin system as friendly as I could so people for people to add their own financial instutions. I'm optimistic someone will contribute a plugin one day :)
What do you think of Weboob, http://weboob.org for French and other banks?
"Weboob is a collection of applications able to interact with websites, without requiring the user to open them in a browser. It also provides well-defined APIs to talk to websites lacking one."
It would theoretically use whatever meachnism you would use to interactively log into your bank. For Canadian banks this is user, password, and security question/answers. The "plugin" system was designed to account for each banking system being unique so how it logs in really depends on the bank.
This runs locally and it has the ability to read gpg-encrypted credentials (using gpg-agent).
This of course gets complicated with TOTP tokens but I'll cross that bridge when needed :)
I think we all know the answer to that question :)
I would note that professional bank feed apis (e.g. yoodle) also use username password pairs and not only break when you change password but can result in a locked bank account due to too many retries.
That said, there is no universal solution in the US.
> I think we all know the answer to that question :)
May I ask-- for those not in the know --that what is it? :) From your comment I'd infer that username/password, but that hasn't been the case for many years, at least at the banks I use (in Europe).
If he's using headless Chrome then it's likely form filling. I.e. just automatically typing in the answers to the authentication prompts on the pages as you'd normally do when you log in. Most banks, even those with TOTP or alternate MFA methods, allow you to login with standard authentication details to get access to statements and not make any changes.
Does anyone have any advice on how to be successful with ledger? I've tried a few things. I entered a lot of things, but the issues I have are more structural nature and probably general lack of accounting experience:
What types do you use is there a useful list, that I can then translate to SMB or Personal tax filing? A somewhat standardized structure of assets, liabilities, accounts payable, accounts receivable etc.
How do you deal with Cash? (EDIT: most answers assume that you only spend little cash, but in some countries cash can account for a decent amount of your income)
How do you convert your balance sheet into a tax filing compatible format?
How do you track things like rent? Do you keep it as a recurring entry i.e. `~ Monthly` or do you only book it when you pay with buchhaltung?
Do you keep completely separate ledger files if you're sole proprietor or can you keep them semi separate and work with includes to create separate reports for private and sole proprietor tax filing?
How do you track small expenses on the go? There's a web app called fava from beancount which looks nice, but i'm not sure you want to expose it to the outside world and i'm not sure the files are fully compatible with hledger https://beancount.github.io/fava/
For personal finances, when I get cash (say from an ATM) I transfer the amount from asset Bank to asset Cash (i.e. CR Bank/DR Cash). I then record (important) expenses against either Cash or Bank (e.g. CR Cash/DR Groceries). At the end of the period, I check the actual amount of cash vs recorded, and charge the difference against a Misc expense account.
This is in the spirit of the standard accounting rules for handling petty cash in an organization.
> How do you deal with interest?
Consider interest an expense, e.g. CR Bank/DR Interest
> Advice?
Consider learning the following rules of double-entry accounting:-
Asset ...
Increase = DR, Decrease = CR
Liability ...
Increase = CR, Decrease = DR
Income ...
Increase = CR, Decrease = DR
Expense ...
Increase = DR, Decrease = CR
Equity ...
Increase = CR, Decrease = DR
Aside: yes an increase in an asset like Bank is a DR. Your actual bank statement is showing what is recorded in your bank's accounting systems, not in yours. If you have a positive balance at your bank it is recorded as a CR on your bank statement because it is a LIABILITY for them, i.e. they owe you the money.
Knowing these rules allows you to think how to deal with things. For example, a purchase on your credit card is what?
Some example entries:
Get Paid:
CR Income 100 (see table increase in Income) /
DR Bank 100 (see table increase in Asset)
Pay Groceries:
CR Cash on Hand 20 (see table decrease in Asset) /
DR Groceries 20 (see table increase in Expense)
Purchase on credit card:
DR Airline Tickets 1,250 (see table increase in Expense) /
CR Accounts Payable 1,250 (see table increase in Liability)
Pay credit card:
CR Bank 1,250 (see table decrease in Asset) /
DR Accounts Payable 1,250 (see table decrease in Liability)
Interest on bank statement:
CR Bank 35 (see table decrease in Asset) /
DR Interest 35 (see table increase in Expense)
edit: tried to deal with HN not allowing any sort of text formatting
To answer your question
Re Cash: Note I may have a different definition of cash. In my ledger, cash is anything that I can liquidate in <1 days. That includes saving/checking/CD. As for your question about the real cash in your wallet, I tend to not account for those.
Re Interest: Interest should be correctly recorded. That means the interest for your checking/saving/CD should be recorded, as well as clearly separate your loan payment into two legs (principle and interest).
> How do you convert your balance sheet into a tax filing compatible format?
I don't, sorry. But it's actually pretty easy to get the relevant data if you split your taxable incoming accordingly.
> How do you track things like rent? Do you keep it as a recurring entry i.e. `~ Monthly` or do you only book it when you pay with buchhaltung?
It's part of the bank statement, so yes it moves money from Assets:Checking to Expenses:Rent whenever the transaction happens.
> How do you track small expenses on the go?
I don't. After a while I found it's not worth the effort. I rarely make financial decisions based on how much I spent on beers and sandwiches, so most of these little things are just subsumed in Expenses:Cash.
> I don't. After a while I found it's not worth the effort. I rarely make financial decisions based on how much I spent on beers and sandwiches, so most of these little things are just subsumed in Expenses:Cash.
Agree completely - when keeping track of expenses, it is only worth tracking the numbers that matter to you. If it won't effect your decisions, don't bother tracking.
You can also move from Assets:Checking to Expenses:Cash, and then from Expenses:Cash to Expenses:Haircuts. This allows you to track the larger cash expenses if you like, while letting the small purchases accrue in Expenses:Cash.
I just found that workflow described in a blog post elsewhere too:
"Pro-tip: tracking cash withdrawals and purchases can be a pain. I typically just maintain an Expenses:Cash account for these sorts of things. If I can remember how I spent a bit of cash, great: I can just debit Expenses:Cash and credit the appropriate account for what I spent the cash on after the fact. Otherwise, I just leave the amount attributed to Expenses:Cash. It's often not worth the trouble of trying to keep finer track of cash purchases than that." -- http://matthewturland.com/2014/03/29/ledger-basics-and-habit...
Sounds smart, though at the scale of my personal finances, I get much more value out of tracking the petty transactions. I guess I'm pound-wise but penny-foolish.
Probably you should read up or watch some youtube videos on accounting. (H)Ledger is a great tool which allows you to find your solutions to your problems.
In particular section 3 answers some of your questions.
I tend to not keep things in order on the go, but I do organize things when I really need to understand my finances (e.g. when deciding if I should buy a house, take a new job, move countries)
> How do you deal with interest?
Interest paid are expenses, interest received is income.
> How do you deal with Cash?
For petty cash, just treat it as its own expense category (i.e. consider the money spent as you withdraw it). For larger amounts, just treat it as an account in assets.
Bank statements don't always tell the whole story. E.g. for my mortgage payments and my salary, I break these down further (e.g. bank statement only reflects net pay). They're usually the same, so often you can copy last month's record and just adjust the numbers.
> How do you convert your balance sheet into a tax filing compatible format?
You don't. Is the tax interested in your assets/liabilities? Generally I use ledger to verify my income statments (W-2 in the US) match what I actually received, and that interest-payment statements from my bank and lender do too. If I have costs to deduct, I keep those in separate expense accounts in ledger.
> How do you track small expenses on the go?
I don't bother if its cash (which I don't use much). For card transactions, I broadly categorize them according to the description that the bank gives me.
I use GnuCash [1] with non-compressed save files enabled so that I can store changes on Git. It works flawlessly for me: I can edit accounts across different computers, get the full force of a mature accounting package to do my finances, and can use/write scripts that plot some statistics using various bindings (e.g. Python) [2].
Have you found a sane way to handle massive initial imports? As far as I can tell, there's no way to bulk-categorize transactions. That means I'd have to go through each of my 6k+ transactions one-by-one for categorization.
Not sure - I moved to GnuCash with only a few hundred transactions to import, so I did it manually. It may be possible to use the search tool to find the transactions you want to give a particular category to, and then categorise all of those (I'd be surprised if there wasn't a way to bulk categorise a list of transactions).
This is very impressive! I have a SO pursuing accounting, and there are so many subpar proprietary systems in the accounting world it's crazy. I have used gnucash, but there are many issues with gnucash that text only systems like ledger address, but the learning curve for them is daunting to your average user. So I could definitely see great opportunity for a text based ledger system and all its benefits along with a GUI overlay.
All that, with a gplv3 license, this is the kind of project I can get behind!
I've been trying to replace the last dependency I have on Google products, a "finacial forecast" Spreadsheet integrated with Calendar. It lets me see approximately how much cash I'll have on hand at future dates, basically letting me live paycheck to paycheck without too much stress :) Can this do something like that? If not, does anyone know of something with similar features (local program, simple data files, etc.) that can?
Depending on how complicated your spreadsheet is yes. I enter my fixed expenses a month ahead of time. I can then see how much money I expect to have on a given date with something like
ledger -e 2017-12-15 bal checking
At CurrySoftware GmbH we use ledger + recutils + aqbanking + custom bash scripts to automate many business processes.
Some of these processes are required by german law and many companys outsource these to accountants or tax advisors or buy shitty software from DATEV.
The great advantage in staying fully plaintext is, that you do not need to automate everything for the whole system to work.
Incoming invoices are frequent -> write a bash script, Outgoing invoices are frequent -> write a bash script. Paying the state back some VAT happens once a month, use emacs to do it manually. I really love it.
It would be interesting if you could share your workflow/scripts.
Especially how did you implement SKR03 (guess you use that) into your hledger account hierarchy, and how do you generate e.g monthly VAT reports, or yearly VAT and trade tax reports.
The system is not yet ready to be published but drop me a mail and I will send you some bash scripts and will try to answer questions.
We use a custom "Kontenrahmen" based on the same ideas as SKR03, as using numbers is not necessary with ledger. That also means we can only communicate with tax advisors through paper. But as we communicate with the Finanzamt directly, thats ok.
I have been using this for almost two years now. I am not sure if there are any other users but I find it very convenient. (Disclaimer: I wrote the extension).
I've been using ledger for years. I really wanted to give beancount a try, but keeping documentation in google docs aggravates me. I understand that it might be easier for the writers to write and keep up to date. But it's an absolute pain to read/browse/skim/search, which is the whole point of documentation.
It is the transaction import script that has really worked well for me without quirks. A lot of people recommend "reckon"--which I've tried--but I found that while Bayesian scoring sounds good, a solid set of regex work better. icsv2ledger does a great job of allowing you to build up a pool of search terms to map inputs to categories. And since the regex set is an artifact that you can tweak and add to, you end up with a very stable and comprehensive import process.
I also like that it will (optionally) add the raw CSV that was imported as a comment in the transaction, and use that to avoid reimporting transactions. While this is a little visually noisy in the ledger file, I'm blind to it by now and it makes it rock solid to reimport CSV files without fear of duplicates. This also provides a neat way of dealing with each side of a transaction coming in separately. For example, both my checking and credit card list the transaction that corresponds to me paying my bill, but that is only one transaction in the ledger file. After I've done the import of everything, I simply delete one of the new entries but first copy its CSV comment into the remaining remaining transaction. So one ledger transaction lists both CSV comments, and subsequent imports from either account are duplicate free.
This looks like a very well done project. If you are more a traditional software kind of person, I recommend heavily KMyMoney[0]. I have my full family accounting since 2007 in it (every single income/expense) and it works flawlessly allowing me with the great reports to know exactly where our family money is spent and saved.
I've tried using ledger in the past. I ended up moving all the data to a database. Ultimately it is much easier/flexible to work with SQL, than arbitrary command line tools. Also it's easier to manipulate and process the data from anywhere (psql, cli app, web app, ...), without fear of corruption.
I can't share everything here, but basics are this. Three tables: accounts, transactions, entries. Entries have amount, unit, notes. Accounts have name, parent_account. Transactions have subject, notes, date. That's the data. Then I have a view that flattens accounts tree, so that each has name and fullname. Name being Fun, fullname a full path to that account, eg. Expenses:Fun.
With that base structure, you can start answering any questions by just summing entries groupped by accounts filtered by a regexp on fullname and fitler on date. What is my cashflow for any time period, what is my equity, what is my balance sheet, what I actually have, what changed recently, how any of this develops over time...
Corruption not in a sense of storage issues, but in a sense of a fragile text format made for human editing, but not entire pleasant for that.
I wanted more pleasant entry. It became tedious/error prone to search for similar entries with vim and copy pasting. I could have made a specialized tool with nice autocomplete on top of ledger, but then why not drop the entire thing and use a database, where you don't need to re-invent groupping, sorting, filtering, summing, date interval operations and all the mundane stuff.
I also wanted to import data from a bank, and keep all the info associated with the entries. Easy to do in a database, not so in ledger text format.
I also tried to make the plugin system as friendly as I could so people for people to add their own financial instutions. I'm optimistic someone will contribute a plugin one day :)
[0]: https://disjoint.ca/projects/ledger-reconciler
[1]: https://github.com/GoogleChrome/puppeteer