
The Technology Behind A Low Latency Cloud Gaming Service - chrisd1100
https://blog.parsec.tv/description-of-parsec-technology-b2738dcc3842
======
pankajkumar229
We built the exact same technology at
Agawi([http://arstechnica.com/gaming/2012/09/report-cable-
companies...](http://arstechnica.com/gaming/2012/09/report-cable-companies-
planning-to-stream-games-to-subscribers/)). The only difference was that we
did not have the parker API. We worked very closely with Microsoft and NVidia
to make it work back then with full headless Windows GPU servers. H264
encoding both in GPU and CPU. We could have reduced latency by distributing
the servers but we did not get to the stage of distributing GPU cloud back
then. But the business never took off. I was not on the business side so I
cannot tell exactly why. Probably latency but there are 3D strategy type games
that you could stream. If you need, I can ask the business head of our team
and he can elaborate.

~~~
kabdib
One hard nut: the cost of provisioning datacenters with enough GPU capacity to
meet the demand curve of a "hot" title, and still make sense in terms of
inevitable idle time and depreciation.

Either you have insufficient capacity to meet day 1 load when everyone piles
onto a hot new title, or you over-provision, meet that demand and have much of
that hardware doing nothing during doldrum seasons (or when a title bombs).

Probably you need to figure out how to make GPU capacity useful when it's not
rendering games, and sell that as a service as well (GPU-based machine
learning?). It doesn't help that OSes have been prickly about letting
processes share GPU resources; I imagine there are a fair number of thorny
security problems, even with GPU MMUs.

This doesn't seem like something a cloud gaming start-up can really tackle;
lots of capitalization, with lots of competition from entrenched providers,
and no really compelling reason to put games in the cloud to begin with. The
bigger (console) players probably realized that having consumers buy their own
compute is not only cheaper and more resilient depreciation-wise, but also
causes a nice platform lock-in effect once the customers have purchased a few
titles.

------
vvanders
This stuff always seems to be _just_ around the corner, however there's some
fundamentals that are always going to keep it from working.

1\. Some games just aren't built for _any_ latency. I'm talking fighting
games(1 frame windows), majority of FPS' and anything that has a reaction
instead of a prediction game model. These can be made network compatible with
a time-rewind gamestate resolution(which many good ones do), however they
aren't drop-in.

2\. The market that's interested in this tends to be much more demanding than
your traditional consumer. Drop ~150ms of packets at a critical time? You've
got a really frustrated customer where a game designed for network degradation
will handle this gracefully.

Kudos to them for trying but you're not dealing with just technical issues,
there's fundamentals of design that need to be adjusted for that can't be done
without direct developer involvement.

~~~
EGreg
1) You can simply have vector clocks, and resolve differences like anything
else. Most of the time the game's prediction would be correct. Sometimes there
would be a small adjustment.

2) Yes, anything real-time is like this.

~~~
vvanders
Except you don't have access to game state in these cases so there's no
adjustment that can be made.

You can only move forward in the frames you display. You can't walk back in
time to evaluate a player's action in relation to their latency and then
reconcile game state with all other clients.

~~~
EGreg
Yes you can. When the server sends the "official" record of moves made, if the
client detects any discrepancy with expected results, it resets the game to a
slightly earlier time and replays the "real" moves to reach the current point.
All this is done instantaneously, and only at the end is the interface
refreshed.

~~~
vvanders
> resets the game to a slightly earlier time and replays the "real" moves to
> reach the current point.

That's exactly what properly engineered network games do. However we're
talking about video streaming here, in which you don't have access to _any_
game state. It's not possible to do without the developer going in and adding
specific feature support for your streaming service.

------
chrisd1100
Hey guys, I'm the author of the post. Let me know if you have any questions!

~~~
erichocean
I'm very interested is streaming 3D content creation UIs.

A major issue with virtual filmmaking is the datasets are huge, which makes it
very, very difficult to work with a remote team, because you can't effectively
get the dataset to people's machines even just to work on.

What I'd like to do is host our dataset AND the 3D content creation apps on
our own hardware at Switch (in Las Vegas), and then have our employees and
contractors throughout California access 3D content creation apps remotely,
via something like your streaming approach.

Thoughts?

~~~
kayoone
I just tested this service with a full blown 3D Development Environment like
Unity3d using an AWS Instance and it works amazingly well and is very
responsive. Through Amazon Gigabit connection everything installs in seconds
as well. I had a fairly low resolution (800p) set, but i can totally see this
being viable for CAD/3D Development etc.

~~~
erichocean
Thanks for the feedback!

------
jstanley
This is really interesting, I did not know it was possible yet.

A future advancement that could be made possible by this technology is the
complete elimination of "laggy hitbox" type effects: if the server that
handles physics is also handling rendering, it can make sure to show a
consistent picture of the environment to every player.

~~~
neckro23
That would be equivalent to disabling lag compensation, which isn't possible
anyways if you're just streaming video. Some older FPS games (Quake 1 original
netcode, Halo) don't do lag compensation, and it's not pretty.

The "laggy hitbox" effect only happens because the client predicts non-player
actions before the action data actually arrives. The alternative is to have
actions "officially" occur when the packet hits the server instead of when the
player actually acts, which means that players with high latency will be at a
disadvantage. With streaming video it seems this problem is unavoidable.

~~~
jstanley
All of the rendering would be server-side, so you won't even see the muzzle
flash until the server is aware of it.

With client-side rendering, the client draws a muzzle flash as soon as you
fire, and there could still be an enemy in the line of fire at the time the
muzzle flash goes off, and the shot still not hit if the enemy has already
jumped out of the way.

------
kukx
To sceptics: John Carmack believes that cloud gaming can be successful. Here's
his view on this (2013):
[https://www.youtube.com/watch?v=Uooh0Y9fC_M&t=26m24s](https://www.youtube.com/watch?v=Uooh0Y9fC_M&t=26m24s)

------
goda90
I've been wanting to do a project to let a group of friends play multiplayer
in a certain turned based 4X game by streaming from a remote server like this.
The latency wouldn't need to be super low since it's turned based, so a
browser based streaming client could work. The game has a hotseat multiplayer
mode, so one person could take a turn, and then the server could go into a
suspended mode while it waits for the next person to log in. The idea is to
save the time of having to load up the game to take one turn, and having to be
at a computer that has it installed already.

~~~
witty_username
RDP/VNC should work for this.

------
atemerev
Tried the AWS configuration, things went great (from Geneva to eu-east,
symmetrical 1Gbit at home). Indeed, major source of frustration were
videocodecs, not networking.

------
PureSin
I saw the post on using AWS to play games too. Used it to play the division
for a while. Glad to see a service being built for this.

Signed up and will be trying it out this weekend.

------
ddavidn
Interesting. I'll have to try it and see how it compares to my current gaming
AMI where I just use Steam or Remotr to stream things like Dirt and ARK.

------
falcolas
As someone who does not live near any major datacenters, this kind of service
just doesn't work for me. At all.

The absolute lowest ping I can reliably maintain from my house to a major
datacenter are those in CA, and it's around 60ms. So, I'm effectively
attempting to play a game at 15fps. Workable for some 4x games and a few
single player action games, but nowhere near pleasant.

Since my real world pings to game servers are much closer to 100/150ms (10fps)
with the occasional burst of packet loss (especially in the evenings), cloud
gaming is unrealistic (PS Now basically says "No, you can't use this
service").

I'm getting 60ms averages from work (which is on fibre) to AWS-East (where
parsec hosts their website from), so there's no way this would ever be viable
for me.

Seems like it would have limited appeal to non-casual gamers, if I'm honest.
You have to near a major datacenter with reliable broadband, and you could
never play competitively in a broad set of latency-sensitive games. On the
other hand, I could definitely see the appeal for those who want to play the
latest Peggle or Parking Dash games.

~~~
jerf
You can't convert latency into FPS like that. You can have seconds of latency
but stream at 60 fps, or 5fps streams with very low latency. The two numbers
are independent.

------
AstralStorm
ELI5 version:

\- use a video encoder variant with low latency and CPU/GPU usage

\- keep video output in video memory as much as possible

\- no solution to the problem of getting low latency high bandwidth internet
to the customers

------
ksec
Um.... could PlayStation 4 implement something like this and have the game
played via local network?

~~~
chadnickbok
They already do!

Playstation bought Gaikai and turned it into Playstation Now. They also use
the technology for PS4 Remote Play.

Its awesome and you should check it out :D

------
Paul_S
People laughed when onlive said they could do it. And then onlive went
bankrupt.

As you admit yourself you can't do anything about network lag and reliability.
Even if you get down to 0ms video enc-dec this will be not viable for any
games where the camera moves.

~~~
chrisd1100
I've played CS:GO from NY to Virginia with Parsec (to an AWS machine) and
topped the scoreboard consistently (in pugs).

Also did a 3 hour raid night on the new Legion content with it on WoW, lot of
dynamic camera movement there. Cleared Nythendra on Heroic as the 4th DPS.

May not be to everyone's liking, but worked for me.

~~~
ttul
Hmmm. I have 10Gbps fiber in my office that is 3ms away from us-west-2...

------
wmf
Don't NVIDIA, AMD, and Steam already provide this?

------
isaaaaah
I do not support the idea one bit. Sure, great stuff an all, if you do not
care having everything behind a bottleneck, in the hands of other people. I
do.

