I mean, when relying on third party code, things like `tf.enable_control_flow_v2() and tf.disable_control_flow_v2()` can and will go horribly wrong. It looks like some operations are changing behaviour depending on a global flag being set. And not just some operations, but control flow operations! That will lead to some very hard to figure out bugs.
In my opinion there are some architectural problems with TF, which have not been adressed in this update. There is still global state in TF2. There is still a difference in behaviour between eager and non-eager mode. There is still the control flow as a second class citizen.
If you need to transition from TF1 to TF2, consider doing the TF1 to pytorch transition instead.
In addition if you don't want to fiddle with the containers, there's also a conda channel that lines stuff up. I work on a peer team in machine vision, and use these for personal and professional projects.
It also works well for machine learning projects.
I think part of your problem is the last word there. Windows is a bad match for such an environment. On Ubuntu it is pretty much painless.
I vastly prefer creating hermetic environments with either venv or Docker. They're much cleaner and easier to work with. I wish data scientists would adopt these tools instead.
Sadly, many of the ML models I investigate on Github don't even have their package requirements frozen. It's an uphill battle...
I suspect you have a lot of time on your hands. But for me the 'batteries included' approach really nails it, why repeat the headache over and over again when a single entity can take care of that in such a way that incompatibilities are almost impossible to create? The hardest time I've had was to re-create an environment that ran some python code from a while ago, with Anaconda it was super easy.
I'm sure it has its limitations and just like every other tool there are situations where it is best to avoid it but for now it suits me very well.
conda install -c anaconda tensorflow-gpu
pip install with almost exactly the same set of packages: 3-5 minutes total.
Ah I guess it's on the way but not fully there yet
Or consider https://github.com/google/jax !
The thing is, TF has more than tensor ops. It has pre-defined NN layers, data loading/serialization, distributed training, metrics, and model serving.
It seems like a bit of a step backwards, that's all.
Edit: "matrix ops" -> "tensor ops"
Yes, with a compiler to make this fast.
> The thing is, TF has more than tensor ops. It has pre-defined NN layers, data loading/serialization, distributed training, metrics, and model serving.
Yes, it is a simpler and smaller API.
For things like data loading, you can use the tool of your choice -- TF, pytorch, whatever. For pre-defined NN layers, there are libraries that build this as a very thin wrapper around JAX's low-level API, see e.g. lax, which is include in JAX.
I think jleber means stax here, for pre-defined NN layers.
- that’s exactly what we did and we don’t regret that decision
If you are trying to apply one of these libraries to a production system that doesn’t get a lot of throughput you probably shouldn’t be using them (try a linear model first). If you have a high throughput application you probably want tensorflow and just deal with the shittyness.
layer = F.relu(Linear()(input))
layer = Dense(activation_fn='relu')(input)
If you switch to PyTorch, what are you going to use for prod deployment? Is there any way to use TPUs?
PyTorch has an optional XLA device, that let's you use TPUs: https://github.com/pytorch/xla
And that's before Sonnet, the Estimator API (?) and TFLearn (I'm probably forgetting a bunch).
But since the supported third-party platforms (CNTK and Theano) have stopped development due to TensorFlow, well...
If you miss Keras' way of defining NNs, you can use PyWarm: https://github.com/blue-season/pywarm which offers a fully functional NN building API for pytorch.
 - https://github.com/facebookresearch/visdom
Before my switch I tried out Keras for Tensorflow, and even got a lot of support from Google in my endeavours to resolve the issues I encountered (kudos to Google for that!). In the end I felt it was still not mature enough. Further, although I do believe TF and Keras are moving in the right direction, I still felt that in some cases the way the software was set up just didn't sit well with me.
Maybe it is worth to try again in a year or so, or by then I will tryout Swift for Tensorflow, which I think has a great future ahead.
That said, TF 2.0 changes a lot. Many repos might break, so expect to see lots of tensorflow==1.14 in requirement.txt files from now on.
I've been using the rc's for a while now and I must admit, it's a big step up for projects you are starting from scratch. Migrating... Probably not as clean as I would like to admit but it does the job. Overall tf 2.0 removes a lot of the boilerplate code, which is awesome.
I’ve got a very expensive Bitcoin mining rig paperweight at the moment with two Vega 64s (along with another Vega 64 in my main rig) — it’d be great to re-purpose them for something potentially useful.
However, AMD has a history of shipping poor OpenCL drivers, so everyone just went with what actually worked - NVIDIA.
Edit : directly linked to the ROCm deep learning page.
Case in point: people are still trying to figure out on their github how to apply global weight decay when training a model, and to get a "correct" resize for segmentation you have to fall back to the legacy 1.x API and specify align_corners=true there. These bugs existed for many years, and nobody gives a damn. That said if your choice is between 1.x and 2.0, 2.0 is much easier to work with, especially if you use something other than TF (e.g. PyTorch) for data pipeline and augmentation. You can hook that up pretty seamlessly if you train in eager mode.
- iOS: https://github.com/pytorch/pytorch/pulls?utf8=%E2%9C%93&q=is...
- Android: https://github.com/pytorch/pytorch/pulls?utf8=%E2%9C%93&q=is...
Or you get to use Tensorflow Lite and target both platforms, also with a smaller feature set than its big brother.
Still though, it's the only game in town if you don't want to have insane un-debuggable headaches everywhere you deploy to device. Plus it also supports embedded Linux boards, and pretty much all current TPU-like things available there.
Like there is a mostly finite set of typical solved things you would want to use ML for like image classification, object detection, etc.
I find myself spending my time copying an example verbatim and replacing the csv / images with my own.
If it's the first case, what you want to do is find the best GitHub repos for the task(s) you are trying to do. Make sure the GitHub repo has a model zoo and good support and start from there. In CV, if you are trying to do high-end work, the repos to check out are:
- https://github.com/facebookresearch/maskrcnn-benchmark (don't be misled by the name, it has support for lots of the high-end modern CV models)
- https://github.com/TuSimple/simpledet (haven't explored this one as much, but it looks very solid)
If you want easy-to-read code for non-trivial tasks, I would suggest taking a look at Gluon (GluonCV - https://gluon-cv.mxnet.io/ and GluonNLP - https://gluon-nlp.mxnet.io/). I haven't worked much with the fast.ai library, but that's probably also a good suggestion.
I've been learning Tensorflow recently for a side project, and the style transfer work I'm doing means I need to build my own Tensorflow graphs, so I haven't had much use for this kind of thing. But it sounds like it was made for you, not me.
Some may complain about big API changes but I think it is occasionally healthy to tag old stable versions and do massive code refactoring.
I assume that TensorFlow.js is also getting updated - I find it almost equally nice to prototype with (the bundled examples are first rate).
Aurelien's writing is clear and clean compared to most other books on ML.
It says CUDA 10.0 and cuDNN >= 7.4.1.
Was incredibly fussy when I tried it & tends to silently fail over to CPU if it doesn't like something (and worse builds of versions. Eg 7.4.1 comes in 10.0 Cuda flavour and 10.1)
I'd love to be proven wrong about that, though.