Worth pointing out for anyone else that it seems PlaidML is AGPL licensed - so maybe not worth getting too excited about if you have any commercial applications in mind.
Resnet50 : 100 inference/sec (0.009983 sec per each run)
Mobilenet: 450 inference/sec (0.002220 sec per each run)
Resnet50 : 107 inference/sec (0.009302 sec per each run)
Mobilenet: 473 inference/sec (0.002112 sec per each run)
My benchmark script for tvm is here
If I just pull the overall kernel runtime from our logs, I get ~525 inferences/sec.
plaidbench keras mobilenet
plaidbench keras resnet50
time_evaluator is what tvm/nnvm folks use for benchmark. See their benchmark script here https://github.com/dmlc/nnvm/blob/master/examples/benchmark/...