I'm definitely not expert and probably this is a dumb question , but why smart edge things like smart robot and not dumb edge with smart central brain ? Anyway data are useful aggregated central;ly why not incorporate the brain centrally too?
Well, first some clarification - "edge" means "on robot" versus something in the cloud. And the reason you do this is latency and connectivity.
I am designing a four wheel drive robot using the NVIDIA AGX Xavier [1] that will follow trails on its own or follow the operator on trails. You don't want your robot to lose cellular coverage and become useless. Even if you had coverage, there would be significant data usage as Rover uses four 4k cameras, which is about 30 megapixels (actually they max out at 13mp each or 52mp total). Constantly streaming that to the cloud would be very expensive on a metered internet connection. Even on a direct line the machine would saturate many broadband connections. Of course you can selectively stream but this makes things more complicated.
Latency is an issue. Imagine a self driving car that required a cloud connection. It's approaching an intersection and someone on a bicycle falls over near its path. Better send that sensor data to the cloud fast to determine how to act!
On my Rover robot it streams the cameras directly in to the GPU memory where it can be processed using ML without ever being copied through the CPU. It's super low latency and allows for robots that respond rapidly to their environment. Imagine trying to make a ping-pong playing robot with a cloud connection.
I am also designing a farming robot. [2] We don't expect any internet connection on farms!
This is a bit off topic, but I'm constantly looking at ways to efficiently stream 4K cameras live to local displays as well as remote displays at the highest framerate and resolution possible. How feasible would it be on the xavier to stream 2 4k cameras and display them on at least 2 4k screens? Extra points if you could do that and simultaneously upload to a streaming service, such as twitch.
Best bet is to take the raw video and run it through gstreamer. You should be able to setup a pipeline which displays the raw video locally while sending a compressed stream to the network. I'd bet that Nvidia has gstreamer modules which make use of their compression HW, so it might be possible. To be honest though, that's a lot of data, so I don't know how well dual 4k would work. You can always scale it down in gstreamer before you send it to the compression module.
You'll probably want to use the CSI-2 interfaces to connect the cameras, but that depends. CSI-2 was developed for cell phones and is hard to run over long distances. It's optimized for low-power and designed for very short interconnects. We had a ton of problems using it at the last company I worked for. I really wish there was a competing standard for embedded cameras.
Not on a Xavier but on a TX2 I was able to get a single reasonable 4K stream with NVENC and gstreamer. Quality wasn't super awesome since it's realtime encoding. We ended up using two TX2s (one for each stream). I'd probably go with two Xaviers for better quality or step down and do 2 higher quality 1080p streams.
You can certainly do this using machine vision cameras. Either USB3, Gig Ethernet, or CSI interface (16cm max run length I believe). I forget how best to attach two displays to the Xavier but that’s seems doable.
I got my cameras from e-consystems and they’ve got some USB3 cameras that could do it. At least I’m pretty sure. My USB3 cameras just showed up and I haven’t tried them yet.
Interesting. I know e-consystems provided their own kernel modules for the Jetson which help with some of the camera streaming, and they provide a special binary of I think guvcview to view live camera output. That is for Xavier though so I don't know if they have something for just a regular computer.
The other thing to look at is using hardware transcoding in an NVIDIA GPU, a raw 4k feed from a 4k camera is huge, but transcoded to h.264 or better h.265 the footage is much more playable from disk in my experience. It may help with live footage. Here are some notes I made when setting up GPU transcoding.
What is your monitor resolution? If you can't display the full resolution, I've found that using the nvidia encoder hardware to resize each 4k stream to 720p makes transcoding much faster. I've added my video conversion scripts to github so you can see how I've done that.
https://github.com/tlalexander/rover_video_scripts
You can also contact e-consystems, as they seem eager to provide application support. Finally feel free to email me to the email in my profile, or better yet create an account and ask the question on my website http://reboot.love so other people can see our conversation and benefit from what we learn.
EDIT: I JUST saw that you meant one local display and one remote display. Sorry busy day. In that case the other poster mentioning gstreamer is spot on, and I believe econsystems has some gstreamer plug ins at least for some cameras, or maybe nvidia does...
I think the magic camera interconnect is CSI/CSI2 and it's not really flexible enough. You either have really short copper interconnects, or unavailable fiber interconnects.
What would be cool is if csi to ethernet were a thing. either low latency put-it-on-the-wire or compressed. I don't know, maybe it is. But make it a standard like rca jacks.
But 'edge,' as used in context of AI, is also a wink-and-a-nod that the device is inference-only (no learning, no training). The term "inference only" doesn't sound very marketing-friendly.
AGX Xavier can do training on device just fine - and run every CUDA workload. It's just not the fastest device at that, you'd prefer a desktop GPU if you can for such a purpose.
I assume what fizixer means is, if you're making an Amazon-Alexa-type-thing, training 1 model on 1 million user's data will work better than 1 million models trained on 1 user's data each.
AFAIK the "Roomba learns the layout of your house" type of edge learning is generally done with SLAM rather than neural networks. There might be other applications for edge learning, of course.
Yes, but the GPU is needed for training so "we" do this on prem or in the cloud. But the actual prediction computations are fast. Do we really need an edge GPU for these?
Yes. An edge GPU is required to run inference on high resolution sensor data for many robotics applications. Running inference is faster than training but it’s not all “fast”. Inference on CPU can take many seconds with large image operations. Not useful for certain robotics applications.
Not a dumb question at all: data traffic is expensive.
If you have thousands of remote sensors collecting Gbs and GBs of real time data, for ~1000$ you can add a "streaming" supercomputer to your sensor to analyze the data in place and save on network and storage costs.
Notice however that the announcement is for an Nvidia AGX product, which is for autonomous machines. The Nvidia "edge" products for processing data on the sensors are called Nvidia EGX.
For autonomous machines, you often need to analyze the data in the machine anyways, e.g., you don't want a drone falling if it looses network connectivity.
For things like industrial robots or UAVs latency is biggest problem.
I've worked on fruit sorting machine and there was about 20ms to make decision if the object passed or not + there was continuous streams of 10000s of objects per second to classify. The computer vision/classifier had to be both fast and reliable about spitting the answers, which was actually more important than precission of classifier itself.
But we've also developed distributed abilities with quite boggling amounts of specialization.
If you chucked one random human on a desert island, they'd probably die. Chuck a dozen, they have a better chance of survival. Chuck a thousand, you might have a civilization.
Conversely if you say chucked 2 or 100 rabbits on an island - end result is probably going to be an island full of rabbits.
Cloud GPU/TPU resources are still somewhat expensive. Also bandwidth can be an issue when you would first need to feed video through potentially metered connections. Last but not least latency can be an issue for e.g robotics and automotive.
The network is very, very unreliable at the edge. Better to have each piece work independently and store up processed results to transmit eventually, opportunistically. If that processing involves real time video processing there's no way you're going to get that done over a reliably unreliable connection.
I mean it could be way more powerful like stadia on phone vs using phone gpu , latency is not too high for the described usage .. imo just automotive may require a dedicated brain on the edge, I'm totally wrong ?
I work at a startup that uses edge AI. There are many factors that edge is preferred over cloud. Security is one. Latency is important in many cases. If the internet connection is another dependency for a critical system, it can be a big headache. Once you start working on a real-world project you run into these issues. In return you give up monitoring the data and model that can be done with cloud deployment.
