I was actively trading cryptocurrencies a few years ago. All the fun died at tax time. Thousands of transactions across several exchanges and currencies, manual calculation of capital gains was out of the question.
At the time, I recall finding one or two tools on GH, none of which seemed suitable. There was also a 3rd party service for calculating tax which integrated with exchanges, but it didn't sit well with me for privacy reasons.
Eventually I felt I had no choice but to build a tool to calculate capital gains from an input CSV of transactions. I wanted to keep it simple, so it puts the burden on the user to download transactions from exchanges and get them in the correct format, which it then processes.
I finally submitted some very late tax returns, and the tax office was kind enough to withdraw the late interest penalty after I explained the effort involved.
Urgh, yeah I ran into that problem. I had absolutely no idea. I was buying and selling small amounts of dozens of obscure cryptos just for shits and giggles. What a nightmare it turned into.
A few years ago I traded cryptocurrency extensively. I eventually ended up with a tax nightmare, needing to account for thousands of trades across several exchanges. After months of talking with my accountant and tax office, I eventually built https://github.com/dleber/capitalg
It was still a lot of work aggregating trade histories from various exchanges into a standardized schema, but I took some comfort in understanding the process. I also avoided the need to share exchange API keys and trading data with 3rd party accounting tools.
If you discover any bugs, please don't tell the tax authorities.
Was about to mention this. If I recall correctly, the 2d-sphere index rounds geospatial coordinates to 5 decimals. Very occasionally, I found it would distort polygon geometries just enough to cause them to become invalid (e.g. overlapping geometries), which causes the index build to fail.
In my recent experience working with collections containing million of documents, each containing a geoJSON-style polygon/multipolygon representing a property (i.e. a block of land), I found invalid geometries to occur for about 1 document in 1 million. For a while, I suspected the data-vendor was the cause, however it became more puzzling when other geospatial software confirmed they were valid. Eventually we traced the issue to the 2d-sphere index.
A very clever workaround was suggested by a colleague of mine, inspired by [1]. It preserved the original geometries. In each document, we added a new field containing the geometry's extent. A 2d-sphere index was then built on the extent field instead of the original geometry field. Invalid geometries were no longer an issue since we were dealing with much simpler geometries that were substantially larger than the max precision of the index.
When running geoIntersects queries on our collection of millions of documents, we did so in 2 steps (aggregation queries):
1. GeoIntersects on the extent field (uses the index).
2. On the result set from the last step, perform geoIntersects on the original geometry field (operates on a much smaller set of records compared to querying the collection directly)
The same things get invented over and over again and named different things depending on the field. Sometimes it's not immediately clear that they are the same things mathematically.
Consider upgrading to MongoDB 5, which finally offers usable schema validations. It may warrant a drink of lower proof, and maybe even avert a hangover.
Totally agree. I'll struggle to get in the zone if I anticipate distractions. For me at least, distractions are much fewer outside of regular business hours.
My experience has been the opposite, with simple python docker containers showing noticeably slower cold starts than python zips. I've also found the size of the zip package matters (smaller is faster). However things change over time, maybe there have been recent improvements?
Specifically regarding cold starts of lambda instances, I recall this excellent article from Jan 2021 which measures cold start latencies for a variety of AWS lambda runtimes:
Our benchmarking was only for a to a warmed lambda, I don't believe any language can do a 50ms cold start on lambda. The fastest example in your link is python with around 150ms cold start as the fastest, and that's an outlier from the average.
Questioning the end-user value of technical decisions (or more broadly, aversion to complacency). It's easy to give merit to past decisions: "this feature is obviously necessary otherwise it wouldn't be there, even though I can't understand how this is useful to anyone". I believe a stronger engineer won't let this slide, especially if the feature is proving painful to support.
I've seen others (and have myself) supported costly features of questionable value. Sometimes a few conversations with product managers and customers have resulted in us entirely dropping a feature.
I've been liberally trading cryptocurrencies for the past several years and ended up with a bit of tax nightmare, with thousands of trades across many exchanges. I found my self with little choice but to write my own capital gains calculator. At the time I needed it, I couldn't find a suitable open source solution, and I have privacy concerns about paid services. It's no longer just for "personal use", as I've recently published it on GH https://github.com/dleber/capitalg
I'm very curious to take a look at this. Built something similar a few years ago because none of the tax software sufficient accomplished what I needed. Mine takes a list of transactions + transfers, and allows me to audit the balance in each wallet (to spot check and make sure I didn't miss anything). It creates csv exports that I feed into bitcoin.tax and use for tax lot selection.
I'd like to update it to handle optimized tax lot selection, so I'm curious to see how you built that part. Longer term I'd like it to automatically import transactions for me, but haven't made it that far.
At the time, I recall finding one or two tools on GH, none of which seemed suitable. There was also a 3rd party service for calculating tax which integrated with exchanges, but it didn't sit well with me for privacy reasons.
Eventually I felt I had no choice but to build a tool to calculate capital gains from an input CSV of transactions. I wanted to keep it simple, so it puts the burden on the user to download transactions from exchanges and get them in the correct format, which it then processes.
I finally submitted some very late tax returns, and the tax office was kind enough to withdraw the late interest penalty after I explained the effort involved.
https://github.com/dleber/capitalg