> I want to have a Jetson for LLM purpose, but after checking the specs, I don't think it would be powerful enough though...
Don't waste your money, you can make a way more useful and powerful build for slightly more money. Unfortunately the Jetson Xavier (at least the Xavier, no idea about the Orin but seen the price I'd say you can build a mid tower for the same price) is unable to run any LLM in a decent way ( i got like 0.1 token / s on a 7b model lol)
> How do you like the ZimaBlade and NVDA Jetson? I actually want to get these two, and have been consider going with ZimaBlade or ZimaBoard...
I use the Jetson only because else i would have to throw it away, to be honest I prefer the ZimaBlade hands down even if it is the dual core version.
About that, I am waiting for the quad core version to arrive. I liked it so much I bought another unit that I will use just for Nextcloud in RAID.
I am amazed by the zimablade performance especially as it is a very low specs machine theoretically. Never tried the zimaboard but I have no reasons to do it.
> Do you mean you are moving these hosting services to a mid-tower server (English is not my first language, so to is easier for me to understand than on). Would you say that even if they are not more powerful from performance perspective, but they offer some server-specific functionalities that NUCs don't, such as better thermals?
I am not native too, sorry if something was unclear. Yes, we were moving these hosting services to a mid-tower "server" (actually a normal PC with linux). Thermals surely were better handled due to the size of the build but from my personal experience you can add some thermal solutions to NUCs and get the same / better results. Note that the mid-tower build I am talking about would be considered low spec nowadays (I don't remember exactly the specs but it was surely pre-2019).
In general, I'd suggest trying the ZimaBlade 2 cores which is pretty cheap and can give you an insight of SBCs used as servers.
In my opinion, while being way cheaper, the ZimaBlade obliterates the Jetson Xavier (which btw is forced to run at 20w vs 6w and must use Ubuntu 18.04 with a 5. something kernel)
> About that, I am waiting for the quad core version to arrive. I liked it so much I bought another unit that I will use just for Nextcloud in RAID.
>
> I am amazed by the zimablade performance especially as it is a very low specs machine theoretically. Never tried the zimaboard but I have no reasons to do it.
Thanks! Then I will consider buying one for myself!
> Yes, we were moving these hosting services to a mid-tower "server" (actually a normal PC with linux). Thermals surely were better handled due to the size of the build but from my personal experience you can add some thermal solutions to NUCs and get the same / better results. Note that the mid-tower build I am talking about would be considered low spec nowadays (I don't remember exactly the specs but it was surely pre-2019).
No problem, thank you for the clarifications!
> To give you a practical example, you can try
>
> https://view.tcsenpai.com/feed/popular (invidious instance hosted on the Jetson Xavier)
>
> https://swing.tunnelsenpai.win/#/ (hosted on the ZimaBlade)
>
> In my opinion, while being way cheaper, the ZimaBlade obliterates the Jetson Xavier (which btw is forced to run at 20w vs 6w and must use Ubuntu 18.04 with a 5. something kernel)
Don't waste your money, you can make a way more useful and powerful build for slightly more money. Unfortunately the Jetson Xavier (at least the Xavier, no idea about the Orin but seen the price I'd say you can build a mid tower for the same price) is unable to run any LLM in a decent way ( i got like 0.1 token / s on a 7b model lol)
> How do you like the ZimaBlade and NVDA Jetson? I actually want to get these two, and have been consider going with ZimaBlade or ZimaBoard...
I use the Jetson only because else i would have to throw it away, to be honest I prefer the ZimaBlade hands down even if it is the dual core version.
About that, I am waiting for the quad core version to arrive. I liked it so much I bought another unit that I will use just for Nextcloud in RAID.
I am amazed by the zimablade performance especially as it is a very low specs machine theoretically. Never tried the zimaboard but I have no reasons to do it.
> Do you mean you are moving these hosting services to a mid-tower server (English is not my first language, so to is easier for me to understand than on). Would you say that even if they are not more powerful from performance perspective, but they offer some server-specific functionalities that NUCs don't, such as better thermals?
I am not native too, sorry if something was unclear. Yes, we were moving these hosting services to a mid-tower "server" (actually a normal PC with linux). Thermals surely were better handled due to the size of the build but from my personal experience you can add some thermal solutions to NUCs and get the same / better results. Note that the mid-tower build I am talking about would be considered low spec nowadays (I don't remember exactly the specs but it was surely pre-2019).
In general, I'd suggest trying the ZimaBlade 2 cores which is pretty cheap and can give you an insight of SBCs used as servers.
To give you a practical example, you can try
https://view.tcsenpai.com/feed/popular (invidious instance hosted on the Jetson Xavier)
https://swing.tunnelsenpai.win/#/ (hosted on the ZimaBlade)
In my opinion, while being way cheaper, the ZimaBlade obliterates the Jetson Xavier (which btw is forced to run at 20w vs 6w and must use Ubuntu 18.04 with a 5. something kernel)