>tfw you are facebook CTO and need special snowflake hardware because your architecture is pooj-tier
>51.2 TBps in a single 1u chassis to compensate for years of terrible technical choices
>Implying this isnt for hosting 3rd party snooping payloads directly within your social graph for low latency access to literally anything going on in realtime
Seriously what the fuck could this be practically used for. Who needs to aggregate this much traffic at one point aside from the NSA and CIA niggers?
Why does facebook need this much throughput in one box?
Jack Richardson
Other urls found in this thread:
servethehome.com
globenewswire.com
twitter.com
Tyler Gray
Oh here we are. Interesting fucking shit how the internal traffic utilization growth is scaling at a super linear rate and the actual user traffic growth is flat as glass. Makes you wonder what kind of use cases drive these traffic patterns...
Charles Carter
Im with you OP its fucking skynet but i will need a source
Hunter Ross
>unlabeled axes
it's ok i don't trust succurburg either
Sebastian Phillips
Data analyst / scientist here. All of Facebook's ad revenue depends on their real-time prediction algorithms. They most probably use dosens of different models ranging from regular statistics, through machine learning stuff to network analysis. None of these, however, function purely on transaction data (transaction data for Facebook is literally any action of a user - send a message, look at a post, wait 3 seconds before scrolling further, logging in). The transaction data needs to be aggregated and then sent for scoring.
Hardware is likely segergated by purpose - there would be transaction storage machines and media storage machines. The transaction storage machines likely do not do any of the aggregation, but rather transfer their data via an extraction, transformation and loading (ETL) process to dedicated analytics machines. The analytics machines score their models and feed the results either back into the transaction machines or into separate hardware that's responsible for all marketing data.
It's also highly likely that there is at least one data mart or data warehouse plugged in between the above processes. The warehouses usually serve up real-time interactive business intelligence to the company's decision makers.
tl;dr Facebook likely pours ever more data analysis and business intelligence on top of their data, which requires data to be transferred internally multiple times.
Parker Thomas
There's always use for more resources. It's only up to the imagination of their users.
Sebastian Baker
I'm interested in data analysis, what are some resources for learning about higher end stuff? I get the entry level analyses, I can wrap my head around them even if I have but mastered them, but when I try to look beyond finding basic trends and plotting pretty graphs in tableau I hit a wall, or rather a great void of anything to learn from or even point me in the right direction.
Chase Morgan
For data mining of course. 90% of people use facebook
Carter Ward
Alexander Bennett
The trick is to master the tools and concepts, because every data problem is basically its own special universe of rules and reasoning.
Charles Thomas
Well, there isn't really anything specific that an aspiring data analyst should do. Just get a solid foundation in statistics first - means, variances, distributions, linear regression, generalised regression, autocorrelation, all necessary laws such as the law of large numbers, etc.
Then you only need to get acquainted with the most frequent types of analyses as well with the (machine learning) algorithms associated with each problem. The basic types of problems are:
1. Classification - logistic regression, trees, etc.
2. Regression - linear regression, some trees as well, etc
3. Segmentation - k-means, hierarchical clustering, k-nearest-neighbour, etc
4. Association - apriori algorithm
5. Dimensionality reduction - principal component analysis (PCA), factor analysis, elastic net
Parallel to the above, you need to learn specific programming languages that let you wrangle data and conduct machine learning models. The open-source languages R and Python (+ Python's data analysis packages like numpy, pandas and scikit-learn) are most popular currently, although proprietary solutions like SPSS, SPSS Modeler, Matlab, SAS are also used. The former two are more widely-used, so you should focus on learning one or both of them.
Finally, it is highly useful to have a basic grasp of databases - you should have at least basic knowledge of the structured query language (SQL) and basics of database design - tables, normailsation, private and foreign keys, indexes. Keep in mind that different SQL databases are mostly compatible, but each has its specific language quirks. In the industry you'll generally find MySQL (or its libre fork - MariaDB), Microsoft SQL Server, Oracle SQL and others.
I'm attaching a cute machine learning flowchart. If you want more info, we could exchange contact info.
Parker Jenkins
Thanks, that's a good deal of keywords to work with. I have more background in SQL/Python than in statistics and other stuff that you mentioned, perhaps that's the missing ingredient that holds me back from progressing.
Adam Edwards
higher quality video and pictures probably
Justin Cook
I hate Facebook and I hate Zucc, but why do you care? It's his money, not yours. Are you salty because you can't afford ridiculously overkill hardware to tinker with?
Sebastian Thomas
>It's his money, not yours.
No, it's not his money. It's Facebook's money. And that means it's the investor's money who buy stocks. And if the company is doing something that will later cause a backlash among its customers, that means the stocks will suffer. And that's MY money.
How much are you getting paid for damage control on these boards, shill?
Nolan Myers
You're the one doing damage control, not me. If investors are giving Facebook money, they lose any right to tell them what to do with that money. Stay salty, retard
Jason Foster
buying stocks isn't the same thing as donating
Blake White
>If investors are giving Facebook money, they lose any right to tell them what to do with that money.
And if the company does something that is illegal, it matters to the investors, and they should sell their stock, and they take back that money (what ever it is they can get for their stock at the time). So no, they do not lose their rights to that money/investment. Nice try, shill.
Jeremiah Taylor
>lookoverherefaggot.pdf
Investors could arguably file lawsuits today over what has transpired already at FB. Being a publicly-traded company in America is actually a fairly large liability because you are beholden to the shareholder more than anything at that point. Not because its the right thing to do, but because the SEC will delist your fucking ass if you don't follow the regs. Serious individual investors and all institutional investors in America are not to be fucked with.
OH WAIT THEY ARE ALREADY DOING IT
>globenewswire.com
>getfucked.exe
Samuel Hughes
I find it funny that there are basically 2 groups of (((data analysts)))/(((data scientists))): those who have no clue what they're talking about like yourself, and those that do. Especially funny is that your ilk has somehow developed a mythos that is objectively opposite to reality. For example, in deep networks, you think that dropout is additive across layers, and you believe there is such a concept as a gold standard for a given domain and that such a concept can even possibly make sense.
In the end the only things that matter are clustering algorithms, deep learning, and reinforcement learning. Moreover, the only language that matters is python, period.
Thomas Cruz
Actually the exact opposite of the truth. When they give money to facebook, in exchange, facebook gives them weight relative to their contribution in decisions as to how facebook shall operate. That is pretty much exactly the point of giving them money: to GAIN the right to tell them what to do with that money.
Anthony Collins
Because data-mining requires exponentially more resources with the more data you have.
Adam Gutierrez
>aside from the NSA and CIA niggers?
Facebook unironically *are* NSA and CIA niggers
Isaiah Barnes
>2 groups of category x
>those that are right and those that are wrong
>clearly you're wrong
Just provide a real counter instead of needlessly stroking your ego. I'm not even that guy and this is infuriating to read. As someone interested in the topic, you've provided no insight other than:
>THAT'S how you that works?! Lmao @ ur life kiddo.
It's worthless discord.