> With more than 5 million downloads since its launch, the app has helped block more than 3.7 million stolen or lost mobile phones, while more than 30 million fraudulent connections have also been terminated.
I might be reading this wrong but these numbers seem very weird. Did more than half the people who downloaded the app block a stolen phone? And did each person who downloaded the app terminate 6 fraudulent connections?
It's not rare to have multiple phone numbers registered to a person's name fraudulently in India. Therefore, in this aspect the app will list out all the connections under the user's Aadhar (Indian Digital ID).
While having the same code for sync and async sounds nice, monkey patching code at runtime seems hacky. Any library that wants to use a lower level implementation of network calls would need to handle the monkey patching themselves i assume.
The idea would be that if this was accepted into the core as the way to go forward, it wouldn't be "monkeypatched" anymore, it would just be in core, officially supported.
The idea is to map N tasks to M threads. This is useful more than just when you needd more threads than the OS can spin up. As you scale up the number of threads you increase context switching and cpu scheduling overhead. Being able to schedule A large number of tasks with a small number of threads could reduce this overhead.
Having too many threads all running at the same time can also cause a performance hit, and I don't mean hitting the OS limit on threads. The more threads you have running in parallel(remember this is considering a GIL-less setup) the more you need to context switch between the. Having fewer threads all running in an event loop allows you to manage more events with only a few threads, for example setting the number of event loop threads to the number of cores on the cpu.
The use case I wrote it in mind with is FastAPI. In that case, there wouldn't be any change to the Python code. You'd just use a different ASGI server that would use this sort of multithreaded event loop. So instead of running it with uvicorn main:app, you'd run it with alternateASGI main:app.
I have an example of a very basic ASGI server that does just that towards the end of the blog
Hard agree. If you want resource efficiency and high performance you're probably better off looking to lower level languages most of the time. In my experience FastAPI usually gets used by teams that need a server done quickly and simply or are constrained by a lack of experience in low level languages. That being said, I do think its worthwhile trying to improve efficiency slightly even for these cases.
run_in_executor is pretty powerful for running sync code in async code, but my use case was more making async code utlize the cpu better. I think, just using run_in_executor would add a lot of complication and changes to how you use async await. But great point none the less!
I went back to look at some of the old code as my memory was hazy. It was tornado's implementation of the run_on_executor method that we used, which is used as a decorator.
> but my use case was more making async code utlize the cpu better
But run_in_executor achieves that as well! If you use no-GIL Python and a thread pool (or GIL Python with a Process pool), you will utilize more CPU cores.
run_in_executor that can be spelled as asyncio.to_thread() won't help utilizing cpu for pure Python code. It may help with blocking I/O, or cpu-heavy C extensions that release GIL. Otherwise, you have to utilize different processes to take full advantage of cpus (there are also subinterpreters (same process, separate GILs) but that exotic).
If I'm understanding your point correctly that wouldn't prevent them from offering higher ram specs for the lower storage eg. 512 gig macs. So it seems like it is just price gouging
The large(100k tokens) context window together with the fact that it can actually use the information in that context window. From personal experience other models including open ai fail to properly answer when provided large(more than 5k tokens) inputs as context even when the model officially accepts much larger contexts. But Claude 2 models are uncannily good at taking all that context into consideration.
I might be reading this wrong but these numbers seem very weird. Did more than half the people who downloaded the app block a stolen phone? And did each person who downloaded the app terminate 6 fraudulent connections?