How the fuck does Youtube manage to store everything when several hours of video are uploaded every minute...

How the fuck does Youtube manage to store everything when several hours of video are uploaded every minute? Are they just adding like a truckload of hard drives every day? How much storage does Google/Youtube have in total in 2017?

Other urls found in this thread:

youtube.com/watch?v=NTMkc0bLRlI
cirrusinsight.com/blog/much-data-google-store
en.wikipedia.org/wiki/JEDEC_memory_standards#Unit_prefixes_for_semiconductor_storage_capacity
youtube.com/watch?v=XZmGGAbHqa0
twitter.com/SFWRedditImages

Yes.

>How much storage does Google have

A lot

youtube.com/watch?v=NTMkc0bLRlI

Holographic storage.
They have a mass drive of random sequences. When something is uploaded they make a matching pattern algorithm that corresponds to the sequences.
It's faster than reducing the raw data to a mathematical sum.

Youtube is a money, manpower and workhours dump that is yet to yield money to Google.

I guess google uses it as test ground for their big data algorithms? I have no clue, but if google ever gets in trouble, it's the first thing to go.

except entropy is so fucked on youtube video that that makes _NO_ sense. what patterns are they matching? all I, P, and B frames are gonna be compressed completely differently. pattern matching in a single video is impossible, but for the entirety of youtube? you're joking.

You know that massive NSA data center that Ed Snowden talked about and was in Citizen Four? Yeah, well all your shit goes there.

>How much storage does Google/Youtube have in total in 2017?
More than 3GB

>Using this method, they determined that Google holds somewhere around 10-15 exabytes of data. If you are in the majority of the population that doesn’t know what an exabyte is, no worries. An exabyte equals 1 million terabytes, a figure that may be a bit easier to relate to. Using our estimate earlier of a personal computer holding around 500 GB, that would mean 1 exabyte would equal 2 million personal computers, and Google's 15 exabytes would be around 30 million personal computers!

cirrusinsight.com/blog/much-data-google-store

>How much storage does Google/Youtube have in total in 2017?
About tree fiddy

But 500GB has been the standard for hard drive sizes for prebuilt desktops for close to a decade now.

Google here, we use laser inscribing on crystals.

An exabyte is 1000 petabytes
A petabyte is 1000 terabytes

Yes?

hes saying 1000 terabytes != 1000000 terabytes

1000*1000 = 1M

What's your point

Him and you either lack reading comprehension or don't understand basic math. Please leave this board.

can we take the botnet down with kryptonite?

D'OH

they use winrar (but its registered)

>I-I was just pretending..

That's a funny way to type 1024.

where did you get that? i see my mistake now, that was a simpsons reference

KB vs. KiB

there's no real need for more.

most normies don't even "produce" 100gb.

KB has been 1024 bytes since forever.

Could some malicious group upload huge amounts of trash to stretch Google's storage capacity to its limits?

Only in contexts where the amounts are naturally powers of two, so mostly bus-related stuff like RAM.

A 4 TB hard drive is most likely very close to 4,000,000,000,000 bytes, and a 1 Gb/s Ethernet connection really does transfer 1,000,000,000 bits/s, because there's literally no reason to measure those quantities in powers of two. That's why it really doesn't make sense to use binary prefixes for file storage or network transfer speeds.

They can match copyrighted work to every video, can't they?

Nobody was talking about binary.

that's KiB.

It's 1024 when talking about storage, not just in binary context. For data transfer (i.e. network speeds), it's indeed 1000. Check JEDEC.

en.wikipedia.org/wiki/JEDEC_memory_standards#Unit_prefixes_for_semiconductor_storage_capacity

Decimal only for serial communication data rates.

This is because marketing a 3.6 TB drive as 4 TB is better for the manufacturer than selling an actual 4 TB drive, not because a TB is anything less than 2^40 bytes.

well, you're not wrong.

/thread

I thought the same thing and someone pointed something out. The thing that help them out loads is de-duplication. I expect it is very common that the same info is uploaded on a regular basis.

That's looking for things that fit a signature. It's a different concept. It's why false positives exist.

Yes, and it costs them way too much money. Youtube's popularity and belief that you can "store videos on the cloud" for free is going to ruin it. Youtube should (and hopefully will) start charging content creators for operating costs, and leave them to monetize their channels on their own.

serious answer: shingled hard drives

It's kind of funny to me how little of specifics we have about google's datacenters

Facebook transitioned from hard drives to bluray discs, Google probably have something similar.

It irks me how little we know. Does Google have like a few huge, half empty buildings that they just keep adding to? Or do they buy new buildings when they need to add more servers and fill them up all the way at the begining?

there's a little we know there. They have buildings with full copies of all their (probably only public) data everywhere, to prevent lag. From some YT explanation or something.

>_NO_
Get the fuck out of here, Redditor

>group upload huge amounts of trash
You mean youtube users? Because that's what youtube is now. A bunch of people uploading trash in huge amounts. Taking down google by sending them lots of data is basically taking down the internet. They seriously operate on another level.

without google people would still have a few other sites, and those sites would just have to put up hyperlink sections at the bottom again. Like the good old days.

I'd say bait but they're probably cheaper than tapes.

How do you know that's from Reddit if you aren't a redditor

Common boding code applicable to places not here.

It's not an issue of whether or not the internet can survive without google. It's an issue of size. To bring down google with sheer volume of data would require a similar amount of bandwidth to bring down the entire internet. Maybe you could do it if you gained control of Netflix's computers and started scripts to upload movies to Youtube.

On second thought someone get on this. The copyright merchants would have a heart attack.

But how reddit specifically?

So what the fuck did the answer turn out to be here?

So not underestimate our autism for spotting things that don't belong.
Only a newfag redditor would do such a thing even tho they've never seen bolded text on this board.
Now go back to lurking, faggot.

I've been here for 5 years. I do not understand how underscores are a "reddit" thing. Do you lurk reddit for hours a day to be able to recognize their memes? What do underscores have to do with bolded text?

Not hard drives. All of youtube is stored on mercury delay toroids that encircle the globe in orbit.

I've been here for 10 years, and admittedly reddit for 5 years, and I have no fucking idea what underscores has to do with reddit. Granted, I've also never seen that used anywhere to begin with.

They already have all the data via library of Babel type design, and when you upload a matching video they just publish the already generated data

They're constantly expanding all over the world, they've just begun building their largest datacenter yet in Quebec, Canada.

> Sees common markdown code
> Instantly assumes Reddit

Stack Overflow and Github and damn near everyone who uses markdown uses the same basic syntax.

The fact that you don't know that and assume Reddit makes me think you're a Redditor... and nothing else.

pic related. reddit likes this meme.

You don't need to say anything, just google it.

I agree with you.

We know when redditors post here. You give off a distinctive stench like old fish and rotten garbage. Get the fuck out, filthy subhuman.

No one would be here since 2007, then turn around and openly admit that they use reddit. You've outed yourself and you have to go back.

>Stack Overflow and Github
>not redditors
Pick one.

> Pick one.
Wait, what?

> No one would be here since 2007, then turn around and openly admit that they use reddit.

That p.much describes me. Maybe earlier. I'm getting very oldfig and memory's the second thing to go.
Reddit's fine though. If you just eat one flavor of autism all day, it's nice to try a different one.

I admire your devotion though.

>p.much
>I'm getting very oldfig
Are you 14 years old? Fucking kill yourself, you little shit stain. You sound like you've been here for a month.

Is he reddit, Sup Forums?

The people who talk like you are usually the ones who are new.

I made you a cute emoji so we can be friends. ( ꒳ )

The _underscore_ to emphasise has been around much longer than reddit. It was more common before you even found Sup Forums (which was like what, 2013?).
It's always been taboo here though since it's retarded garbage, but since you call it 'reddit', you're the one who needs to go back.
What even is reddit anymore? I have no fucking idea since I never go there.

The botnet IS cryptonight

The ones who try to deflect or reverse accusations are usually underage faggots with an inflated sense of self worth who need to drink bleach and die.

You have to go back.

youtube.com/watch?v=XZmGGAbHqa0

>defending cancer
Go back. Right now, nigger. Your kind is not welcome here.

Oh dear. It seems you did not like my emoji.

I think we can still be friends though. I went to reddit like you asked and brought you back something from the home page.

I hope you like it.

All of your posts scream underage b&. You need to leave.

Well if you're not even going to try then neither am I.

Get the fuck out of here, faggot kid.

compression.

>since it's retarded garbage
why would it be?

you caught me
pic related. it's me.

on that note, I think I'm done here. was hoping you'd be more fun. :/

Will some big IT company eventually start making data centers on the moon or in space

Maybe once people are living on the moon/in space. I can't imagine that you get good latency to the moon from earth.

Plus you'd have to put transmitters all over the moon so that it can transmit from any side that's facing us. Same goes for receivers all over the earth, depending on where the moon is. Then it'd be routed to its destination.

Fun idea to think about, but not very practical.

I wondered that, that's why older contents are way slower to load, I mean, picture you uploaded 5 years ago. Also all Facebook pictures are recompressed under 90KB and the video quality is always shit. I think that's why they avoided GIF for a long time and now they're basically looped h264. I understand what they do, Facebook is their primary source of income and they must keep the costs down.

What happens if one of their storage devices break with YouTube content on it? They probably have a pretty good back up system though.

They store it inside your mum because she's so big

>semiconductor storage
RAM, in other words. That's pretty much what I said. See picture for how hard drives work.

What I'm trying to say is that hard drives aren't naturally bound to powers of two, unlike RAM, so there's no reason to think of hard drive terabytes as binary terabytes rather than decimal terabytes.

The SI prefixes are decimal in every other venue of use. The only reason an exception is made for some things, such as RAM, is because RAM naturally comes in powers of two, not because "it's computers, so we do binary for everything". There's a reason the prefixes are normally decimal, and so if there's no particular reason to make an exception, then it's better to stay with decimal.

>Plus you'd have to put transmitters all over the moon so that it can transmit from any side that's facing us
user...

>en.wikipedia.org/wiki/JEDEC_memory_standards#Unit_prefixes_for_semiconductor_storage_capacity

Did you read your own link?

>The specification notes that these prefixes are included in the document only to reflect common usage. It refers to the IEEE/ASTM SI 10-1997 standard as stating, that "this practice frequently leads to confusion and is deprecated".

Not as long as there's a one second latency to the moon.

Yeah they're definitely not using any technology we know

Hopefully they're not running the whole thing on a single Seagate drive in that case.

They also have backups and backups of backups.

Also spread onto multiple datacentres to load faster

>How the fuck does Youtube manage to store everything
alien technology

underrated post

Damnit. :

I know I'm late at the party but I remember back in a ~2000,around when divx(later on xvid etc) became widespread,
I ran into argument with my friend, it was(at the time) total wonder that 2h long movie could fit on 701,59mb(we all used outer circle for another shoehorned mb).
Let's leave quality/compression ratio on the side, shit was watchable, almost DVD quality.
So, he said, in 5-10 years you will see 2h long movies on 50mb(this is still the era of 576p tvs,playstation2 ,NTSC/pal etc),
I told him that shit is just physically impossible(we went into argument, got into a fight and we still don't talk), lo and behold
circa ~2008!?(who will member all that shit) all known YIFY releases 720p at 300mb, some other groups that specialize in maximal quality/size/compromise managed to encode actually reasonable flix in under a 99mb@1080p/23fps.
Now, that was !?5-10 years ago... Now we have x265, it still isn't as good as x264 but it has space to grow.
But in future As far as video data goes, I see some master file that is really small and on demand it can be recoded up to a !?8k,
it sounds as space bologni now but so did 300mb 720p 2h long flix back then.
>in b4YIFY meme, I used it just as example everyone here is familiar (yes,you are..)

But how does it handle 3D Video?

No, it is not, its actual Sup Forums quality post from before, problem is expectations fell so low, this post feels like it won a 1000 interwebs

3d video is just stereo picture, so x2

most new games on steam now take like 50-100GB per game