PyTorch is developed by multiple companies / stake holders while jax is google only with internal tooling they don’t share with the world. This alone is a major reason not to use jax. Also I think it is more the other way around: with torch.compile the main advantage of jax is disappearing.
It's the old age question in programming: Do you use a highly constrained paradigm that allows easy automatic optimization or do you use a very flexible and more user intuitive paradigm that makes automatic optimization harder?
If the future is going to be better more intelligent compilers, then that settles the question in my opinion.
> with torch.compile the main advantage of jax is disappearing.
Interesting take - I agree here somewhat.
But also, wouldn't you think a framework that has been from the ground-up designed around a specific, mature compiler stack be better able to integrate compilers in a more stable fashion than just shoe-horning static compilers into a very dynamic framework? ;)
Depends. PyTorch on the other hand has a large user base and well defined and tested api. So should be doable; and is already progressing and rapid speed..
I think that PyTorch is part of the puzzle and it certainly helps that it is supported by AMD [0]. That said, there is code that needs to run closer to the metal too.
US has no bureaucrats? All the EU member states, no bureaucrats? How many projects in member states cost the tax payer billions? Btw the EU budget is tiny in comparison.
The UK would do much better had they not chosen Brexit. Basically everyone here admits that (bank of England etc) apart from some zealot politicians. And the health system has collapsed. The problems are not solely caused by Brexit but it exacerbates nearly all of them.
Exactly. Brexit long term net effect could be a few points off their GDP. That's not the end of the world, but it's certainly significant and it will make British people a poorer, which of course has an impact on public services. Whether this is worth it's up to British people to assess.
My personal suspiction is that UK citizens were mislead about the consequences of leaving but, hey, democracy is about taking decisions with incomplete information. People have spoken and their decision is sacred. I wish the UK the best luck and it's with sadness that I believe they will need a lot of it.
If your definition of Democracy requires no deception or falsehood in politics then there likely has never been a Democracy in human history. The voters never work with perfect knowledge, and the politicians always lie.
well technically yes, but since it's in Rust once you have your embeddings, with some tweakings you could just store them and query them offline via ffi or wasm in an app for example. Also I am not a huge fan of maintaining python projects they often don't run after a while if I didn't properly maintain a virtual env etc never had this issue with Rust projects.
Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers John Guibas, Morteza Mardani, Zongyi Li, Andrew Tao, Anima Anandkumar, Bryan Catanzaro
reply