
Show HN: Deploy Scikit and Keras Models with a Simple Drag and Drop - theo31
https://inferrd.com
======
zara_arias
Is this able to support more than 50 requests per second? Are there any
benchmarks on performance overhead of the underlying web server/routing that
is handling the requests?

~~~
theo31
Theoretically we can support up to thousands of request per second under the
entreprise plan.

We stopped at 50 request/s in the pricing table because that seemed like a
reasonable number for most use cases.

------
bpg_92
It seems very interesting! What about support for pytorch models or .onnx? I
usually use the pytorch->onnx->tensorrt to deploy models.

~~~
theo31
Hi! We do not currently have PyTorch model but it's one of the next items in
our roadmap.

If you have a more custom pipeline, we have a custom environment where you can
deploy any custom code with specific package versions!

------
ishcheklein
Looks interesting! How about models that require dictionaries - e.g. tf-idf to
convert text into a feature vector? Does it allow for some preprocessing?

~~~
theo31
Hi! If don’t need any pre processing we have built in support for all the
major librairies.

In addition we support custom pre and post processor via custom environments!
Simply write your inference code into a predict.py and we take care of the
rest.

(Btw I am really into CML.dev, great idea)

------
ebalit
I see that you accept models up to 1 GB. It seems the inference time might be
high for models of this size on CPUs. Do you use GPUs to speed up inference
for deep learning models ?

~~~
theo31
We don’t offer GPU accelerated inference for now. However it’s on our roadmap,
sign up for inferrd to get updates!

------
rubatuga
You might want to change 100 Mb --> 100 MB

~~~
theo31
Thank you for spotting that, it's 100MB.

