Mea Culpa I'm in the industry. I knew we should have had backups. She is brilliant but not technical and while the backups would have been expensive and difficult due to size, I should have built multiple NAS and distributed them geographically and used Syncthing to keep them updated.
To be clear we were legacy G-suite users and have converted to paying customers, so it was a free service while this happened, and is a paid service (I guess is youtube part of that?) now.
I just assumed if there was a problem we could youtube-dl everything, it didn't occur to me that control of the account would be suspended arbitrarily with no recourse for literally no reason. Despite reading about similar things here for years.
I guess I just thought those were lightning strikes, and I was focusing my time outside of work on the new SPA rollout we were working on for the company. I understand excuses like armpits are something we all have and they all smell, so I'll just say I fucked up and own it.
I appreciate all the advice and I understand why you are all saying what you are saying.
This was spur of the moment post (I didn't think it through) and I hadn't updated my contact information on my profile. I've done so now.
Insist that your issue be escalated to an internal bug so that it is properly triaged.
Even if the videos were deleted as part of the suspension, it's highly likely that there are still multiple backups and cached versions in Google's systems. They should be able to help you restore them at the very least.
Assuming you mean the GDPR, it only obligates them to share the data they have. If their response is "we don't have that data any more", then the GDPR won't help you.
It opens them up for a massive claim if it later turns out that they really did have the data somewhere, though.
The "we don't have that data any more" should only apply to "I closed my account" not to the "we unilaterally chose to close you're account and delete all your data just because we can". And if they jumped the gun and can't, they must pay for the prejudice. Period.
They bet on people not suing and they're right most of the time customer don't demand. It does not mean, just because they actually get away with it by play dumb, that there are not legal basis.
It would certainly be interesting to see the limits of that law tested. Does a fragment in a cache count as them still having the data? What about a copy in an unallocated block on some disk? What about traces left after the data was overwritten (no storage medium is just bits at the physical layer) - are they obliged to extract every last bit that is technically possible?
Fulltime YouTuber here. Just as one datapoint, I was recently [very] pleasantly surprised to learn that YouTube was able to restore a video that I had accidentally deleted. It went back to being live as if nothing had happened—though my contacting support was admittedly right after the deletion happened.
You did the best you could with the information you were given. I don't think anyone should have to resort to building multiple NAS and distributing them geographically. It's 2022 and this an extremely esoteric thing to do.
Going forward though, Vimeo feels like a better fit for you guys. Caveat emptor, never rely on free services for critical parts of your operation.
> Caveat emptor, never rely on free services for critical parts of your operation.
Don't rely on a single service. Having the same data at two free services is probably more safe than relying on a single paid service. At least for data loss.
If data is sensitive don't rely on any service, just encrypt it with your own key.
>I don't think anyone should have to resort to building multiple NAS and distributing them geographically. It's 2022 and this an extremely esoteric thing to do.
Given how giant companies (especially Google) behave, I do not feel it is an esoteric thing to do. Anyone with even a modicum of technical skill can roll their DIY NAS. For the non-tech savvy, buying off the shelf is fine.
Should not be needed in real life but this is the world we live in where robots have way too powerful ban hammers and the only way to get any support is to make sound in public forums or have a large social following.
Buy commodity computer parts and build a PC. If it's strictly storage, then a cheap pentium/i3/Ryzen 3 would suffice. If you want to do more, then spec for a higher CPU. PCPartPicker should be able to give you the list of compatible parts.
Then depending on what you want out of the device, do one of the following -
1. If it's strictly storage and you want the niceties of ZFS, install TrueNAS
2. If you want expandable storage without ZFS + VMs + Docker with a nice GUI and are willing to pay for a license, install Unraid.
3. If you want ZFS-backed storage + VMs and no docker, and maybe some datacenter-style capabilities, install Proxmox
4. If you want all of the above and are willing to manually configure everything, then install Ubuntu Server + Cockpit for managing the machine headless + Portainer for managing a large number of docker containers.
If you can afford it, the easiest way to start might be to buy a popular NAS. I would recommend Synology. The setup and interface are fairly intuitive, especially if you're technical. Then as part of your workflow, just make sure the completed videos are stored on the NAS as they are uploaded to YouTube.
Takeout + store encrypted in AWS S3 or even Glacier? Preety cheap storage and supposedly high redundancy. I guess it's better to not use Gmail email address to create that AWS account.
The important thing for your business going forward is to ensure that you have good contracts. Your contract with YouTube was their terms of service. Those terms basically give you no guarantees yet give Google immense freedom.
I think Vimeo is a good choice for hosting new videos. For archival I would consider just using a single NAS of your own and then have that replicate to Amazon Glacier in at least 1 other region (unless you already have the rack space and someone to physically manage the NAS box in another region).
Hence why I didn't say to use Glacier as the only archival copy. In the setup I'm proposing, they would have a copy on Vimeo, at least 1 copy on Glacier, and a copy on their NAS. An offline backup mailed to his mom isn't a bad idea either though.
The point I'm making about contracts here is not solely about having the option to sue if the other party doesn't uphold their end of the contract. It's about even having the option to agree to conditions that aren't ridiculously one-sided in the first place.
Back up everything in G-Suite outside of Google. If your Youtube account is connected to your G-Suite services they are at risk as well.
Geo dispersed NAS is probably the most cost effective long term but for now the important thing is getting data off Google services before you lose it forever. So take a look at other cloud providers in the short term. The more you interact with Google and ask questions the more likely it is they shut down more access.
If you have your domains registered with them move it somewhere else, preferably a dedicated registrar so you can at least re-host email and website.
This cross-connection of services scares me as well. I once happily ordered a Pixel phone using my gmail account. The phone was stolen before delivery (the delivery guy handed me an empty box with a hole in it!).
I reported the situation to Google and they could confirm with the parcel company the delivery was not completed successfully (plus some additional background checks they did after requesting access to my account) and they refunded the order.
The thing is I was not brave enough to place a second order of a Pixel because of fear of risking closure of my gmail account if the phone in the second order was also stolen before delivery, so I ended up ordering a Nokia instead.
I second this. I can't fathom that on HN of all sites people recommend renting backup space from the same big tech companies that continuously fuck over people. Setting up a NAS is not magic. Setting up two isn't either. Put one in you parents basement and check on it once every year when you visit for Christmas. If you don't need 16TB go with an SSD (although with the current contamination situation not the best timing to buy flash).
Don't even need to be NAS, couple external HDDs work as well. Add the new videos every month or something. Keep one at work and one at home. Or in some other place. Anyone not Gen Z should be entirely familiar with process of moving files between disks.
Yup. I just buy bare drives, and have a SATA-to-USB adapter, and away we go. Although I did write my own rsync clone program to do the work (it only copies changed files).
Aren't you making it sound a bit too easy and straightforward than it actually is? It might be that simple for you but buying a NAS device like from synology can be pricey too, setting it all up so it is secure but network accessible, and then also handling possible drive failures down the line (and the chances of that are much higher than google fucking you over IMHO) is much more than an average non-tech-savvy user would be interested in.
People look at the cost of properly storing the data without thinking of the cost of not doing it.
A team I was helping with was in charge of making a lot of training videos which were then used in paid classes. Several employees working at least 4 days a week shooting and editing videos. They had been keeping all their master copies on the same memory cards they used in their cameras, "backups are too expensive!". Then one of their memory cards corrupted, and I asked them to calculate the amount of salary went in to making the data on that flash card.
That afternoon a NAS was delivered with authorization to backup that to separate cloud services. Suddenly when they bothered to have that perspective the cost of a NAS was dirt cheap despite being way too expensive days before.
Maybe not really easy but it is really just due-diligence and getting a solution that is worth the money you are making by providing the service.
If you weren't a mechanic, you wouldn't avoid maintaining a car and then get upset that it broke down eventually, you would pay a specialist to maintain it for you.
Youtube is cool because it's "free" but you also get almost zero support from Google and they are pretty the worst in my opinion. I have got decent support from AWS and Microsoft (never tried with FB).
Yes, I wouldn't recommend my mom do it. This was with the average HNer in mind, but even an average tech enthusiast could do it. I think the OP commented here too saying he is a techie.
But tbh I think even getting and external drive and backing up to it should at least be done in parallel to any cloud service you choose.
You're making this too complicated. Upload the video to youtube, then plug in your 16TB USB drive and copy the file there as well, then unplug the drive and put it back on the shelf. When you fill up one drive, you buy another, and the first one stays in the closet, forever. That's it.
If youtube deletes your account, you take the drives out of the closet and re-upload. If a drive fails, you use youtube-dl to repopulate it.
The only way this fails is if a drive dies / house fire on the same day that youtube deletes your account. If you are really that paranoid about it, keep a second copy of each drive at work / home. But you really, really don't need to.
Indeed you have become one of those "lightning strikes". What would have convinced you that it was a real possibility though?
Even reading this post, many readers are will also just assume the same as you did, that it was a lightning strike and will never happen to them... Not sure i have a solution myself.
This is really dependent on where you live, but as you are paid customers I would suggest to file a lawsuit in a small claims court. Seriously! Google is not playing its part on a legally binding contract, and money exchanging hands is enough to make it valid for the courts. The amount of work poured into these videos, compared with the equivalent compensation that you would have to pay for an employee to make the same work would also qualify for damages. This can also protect you against retaliation from google. But again, IANAL and check our local legislation carefully before.
Where I live it's a thing a person can do in an afternoon (usually takes less time than fighting big companies' support online), do not require lawyers, and the other part also can not bring a lawyer to conciliatory session (for big companies this is usually a sham: even if their representative does not have a lawyer present at court, be sure that s/he was heavily counseled by company legal staff before and everything presented was proofread by them too).
> multiple NAS and distributed them geographically and used Syncthing to keep them updated
I decided to look at using AWS S3 to archive my media library. If you use the `DEEP_ARCHIVE` storage class, I believe costs are ~$1 per month per TB. The downside of such a storage class is that it takes ~48 hours for files to be "defrosted" and ready to download.
Before I push any of my media (that I have legally purchased), I also encrypt it to hedge the risk that AWS suddenly decides to start doing automated scans/fingerprinting for copyrighted content. I assume that's overkill but w/e.
Probably not that expensive. Vimeo cheapest plan is $7/month (w/ limits) and $50 for unlimited plan and cloud storage (for backup) is cost competitive. Even having it on a local disk would probably be enough. But... it's not your fault. It's just that despite the propaganda these companies don't give a fuck about their users.
Free is perfect for content that is free to make. If your content has clear monetary value in workhourse, why not pay at least fraction of that for actual service?
> multiple NAS and distributed them geographically and used Syncthing to keep them updated
As some one working in storage, please do not get tons of NAS they are pain to manage eventually. Example: linus-tech-tips could not do it properly.
If you use youtube-dl how large is one video? Lets say 2 GB. I presume, every week you create have 2 videos? Then 4GB per week.
Google gives you $20 for 100 GB. Every year create a new account - like - company name.2022 dump everything there. Hire an clerk to make sure the credit card is payed every year.
That way you do not lose all videos at a time.
Heck you can even create free Google drive account every few months and segment them.
Linus tech tips are not exactly professionals though they have learnt a bit over the years, but it is why they mess up all the time. It makes it amusing for me, but their advice is often worst practice.
NAS is fine if you know what you are doing, cloud storage (S3, GCS, B2) is probably best though.
Recommending Google as a backup in case Google decides to delete your videos sounds a bit suboptimal. Go with hosted backup; sure, for this use case that makes sense. But do so with a different company.
Note that in 7 years OP had one problem now... I know people with local NAS one need to do maintenance and so on. Again, depends on your technical ability. Bit-rot, power failures etc.
Linus Tech Tips built and installed a custom ZFSonLinux server, which I'd also not suggest OP does.
The difficulty in managing a NAS is completely different if you buy two off-the-shelf Synology boxes and log in and point-and-click turn on snapshot sync between the two and create an Amazon Glacier backup task.
Perhaps you need to look at average human trying to setup a router/DSL modem in a house hold. Then you may note that all your second sentence is a large task for any CEO. (No offence they are good in their field but anything more than dropbox or email or drive is a no go!)
In my experience, paid GSuite "support" are just underpaid contractors in developing countries who don't have much power to do anything outside of a few common actions in response to customers' requests.
Yep, even if you pay Gsuite you get some guy in the middle of nowhere earth just feeding you macros for support. They don't care at all big gov contracts is their business
To be clear we were legacy G-suite users and have converted to paying customers, so it was a free service while this happened, and is a paid service (I guess is youtube part of that?) now.
I just assumed if there was a problem we could youtube-dl everything, it didn't occur to me that control of the account would be suspended arbitrarily with no recourse for literally no reason. Despite reading about similar things here for years.
I guess I just thought those were lightning strikes, and I was focusing my time outside of work on the new SPA rollout we were working on for the company. I understand excuses like armpits are something we all have and they all smell, so I'll just say I fucked up and own it.
I appreciate all the advice and I understand why you are all saying what you are saying.
This was spur of the moment post (I didn't think it through) and I hadn't updated my contact information on my profile. I've done so now.
Thanks.