Let's say I have the money to buy Sup Forums from Hiro, which in reality I'm buying the userbase rather than the tech of Sup Forums, which is terrible.
youtube.com
What servers would I need and what would be an optimized imageboard for less bandwith consumption?
Why not just own the servers themselves instead of buying server power?
I have many Sup Forums ideas in mind, but I'd like to talk with you first before I find out if they're feasible.
Other urls found in this thread:
From memory moot did co-lo for his servers.
If you want less bandwidth you will up cpu time compressing files. Hosting overseas might be cheaper, but you won't have physical access.
>4MB gif
You will never do anything substantial and you will forever remain a clueless dumb faggot.
Suck my poptart.
You would recode it in C. Then use bittorrent for file hosting.
>Create a natural filter culture for each board, like /tg/, where people simply feel discouraged and uncomfortable around people they don't belong with.
>Create a blue Sup Forums and depart red Sup Forums and other dumb boards to a different service provider so the CIA stops knocking on my door.
>Fix fast boards by splitting them automatically into smaller boards.
>Implement crossboarding catalogs. If you want to browse /out/ and /vr/ at the same time within 1 catalog.
>:rename the site luckychan, call it a ban belt, make sage actively bring down a thread faster to page 15. Bring back pages 11-15.
>different types of CSS/site layouts for people who enjoy the forum layout and are sick of imageboard layouts.
>put data mining to good use to ban retards without murdering the ISP. Implement trial mode posting and full registration to allow mods to keep a close eye on the new influx of users/cancer and make bans more potent. While still allowing anonymous posting between users.
>Finally get rid of the bot filter/captcha which doesn't work anyway.
>small chatroom boxes at the bottom of each threads, to allow for simple fast chatting without endless paragraphs of drivel.
>Your posts and threads storage and your favourite threads tab, in case they get deleted.
>bulkier looking buttons for scrolling, for clicking on boards and resizing your post box.
>Automatically detect mobile posters and subtly tell them to use a PC.
Suck my superior video format.
>Using lossy compression for something that can be perfectly saved in a small manner in a lossless format
>something that can be perfectly saved in a small manner
What the fuck does that even mean, you imbecile. Do you know what you're saying? I can't wait for the day when the retarded .gif format withers and dies. Webm is superior in every way.
>Automatically detect mobile posters and subtly tell them to use a PC.
Just make it as painful to use as possible. Some CSS stuff renders extremely poorly on mobile, overuse that, then make all the elements have random names so you can't write an app, also make mobile IP ranges have a 30% chance of getting a random error and speed capped to 50kbit.
Use different servers instead of kicking boards off entirely. For example, main server is in germany, but Sup Forums is in the US, /t/ is in Romania/China, and so on.
Also, allow .onion posting.
Why go through all that trouble? Just block certain combinations of screen resolution, platform, and browser. This will get rid of vast majority of phone posters.
>Just make it as painful to use as possible.
Pretty sure gookmoot already has this implemented. Ads up the wazoo, buggy captcha, also it's just slow in general. Doesn't matter if you have the latest from the smartphone market.
...
Hard blocking is easier to circumvent.
Phone poster here, been posting on here by phone for the last 9 years (this site is a pain in the ass with a BlackBerry track pad, just saying, much better on a touch screen). Why do you hate phone posting with such passion? It's no different than posting from my PC at home.
Buy a bunch of Mac minis
Because a lot of phoneposters are normie trash, and the good posters can just post on their computer instead. I've made 6000-7000 posts on my phone, but it should still be blocked for the greater good.
But my computer is at home. And I'm rarely at home. Work takes me far away from there, and much of it is "hurry up and wait" work, this place keeps me entertained while sipping on the data unlike every other website that chews through the data plan in no time.
Yes, there are some good phone posters, but a lot of them are pure trash. It's also a boogeyman, but a soft block should be sufficient.
It's already painful as hell to post from android or even a microsoft nokia phone.
The rational decision for "mobile" posting is to use a laptop. Large screen, a real keyboard, no more auto correction and it's optimized for internet browsing.
You have to be masochistic to post from your phone.
Phone posters are predisposed to make shitty low-effort posts because of the limitations behind using a phone.
Example: Average WPM using a phones keyboard for me is 40, whereas on a desktop I can reach 100.
Is there a way for the MODS / Admin to ban phones by SIM card number? Or is that not part of the ID data that Sup Forums servers would receive?
Delete gif boards.
Thats it, now it does not take 10k a week to keep the servers up.
Sup Forums gets ~250 page loads per second (Sup Forums.org/advertise), so you need to generate a page in
You should be banned, not the phoneposters, for asking such a retarded question.
>>Finally get rid of the bot filter/captcha which doesn't work anyway.
It may not be perfect, but I guarantee that the amount of spam would be unreal if you got rid of the captcha.
Private messaging/whispering in threads would be a very nice feature. That way anons can stealthily tell each other kill themselves without ruining the thread for others.
Especially for anons who just want to carry a private conversation and share intimate details and information without everyone else and their dog knowing.
why would you ever want to bring ownership of a place like this on yourself?
also chan's are an extremely simple format for a website, you really don't need much at all. I think most out there run on a LAMP stack.
It's all text and Images. I'd just store the text in NoSQL tables according to boards, Images in a clustered Object Store. Write some wsgi or go app to serve it behind nginx.
if I'm paying for hosting I'm host all of this out of s3. Probably don't need anything more than a few Lambdas manage it. you can do file uploads direct from browser to it so you don't even really need to think about your file-chunking methods etc.
2+2+2+2 chan has almost removed it, works fine, you can even post from a .onion url. You need to enter it once every 24 hours or 5 posts.
Yes, but it requires JS. And you would get spam issues. And it would be bad for posterity, you wouldn't be able to read old threads. Better to run a separate irc server where you have one channel for each thread (i.e. join #g61185765 as user) where you mask out the usernames unless requested by the client. Then integrate webchat in each thread.
Simple, but still easy to fuck up, just look at 2+2+2+2chan and the issues they have.
I've never actually really looked at 2chan. Worth going to? I already feel bad enough coming here I figured if they had a Sup Forums it'd just be a laser focused version of here which sounds terrible. Are they using a non-relational data tier?
That's actually pretty cool that they have tor capabilities that's huge. I think some form of the Sup Forums pass functionality with a really strict expiration policy on it would be a good way to get rid of captcha's
I follow a few git projects and the various nyaa's on rizon isn't there a Sup Forums irc already?
err sorry, i mean that other chan
>hachan
Autism: The Post
>Are they using a non-relational data tier?
What?
>isn't there a Sup Forums irc already?
There is, but this would basically just be a case of "use the existing goddamn protocol instead of implementing a new shitty copy of it", or in other words, using an existing IRC server for in-thread chat without having to completely fuck up the existing code.
8ćhan
But the issue which needs to be solved is the image/bandwidth one. It could be done via webtorrent, that's probably the best. Recompress image losslessly on upload, then make a single-file torrent so you can canonically map image -> infohash. If other sites like gelbooru uses the same system you can get significant savings.
Use ipfs for image hosting, and give users who serve a lot of files a captcha pass.
You could also use the h@h system, but that would require retooling.
Sup Forumsreddit the post, always fearing originality and obsessed with downvoting.
> pure SPA/ajax based website with a super sleek custom compressed protocol non-json API (protobuf?) that minimizes bandwith usage
> further reduce thumbnail fidelity
> reduce jpg quality of full size images
> optimize lossless .png compression
> speed up thread death
Do those other forum websites with the chat box at the bottom also require javascript?
IPFS is a very good idea. Are IPs revealed in a direct relation to the file like bittorrent?
What's the H@H system?
But then you'll need js up the wazoo. And all the compression stuff will take lots and lots of cpu.
Yes, or manually refreshing the page. The optimal solution is IRC.
make it accessible via ssh (or telnet, but that has no encryption)
make catchpas not shit
force https
>IPFS is a very good idea. Are IPs revealed in a direct relation to the file like bittorrent?
They aren't, if the machine they're coming from isn't a gateway. In such a case, the gateway would know.
Everyone knows the IP of the gateway, because it's a regular web server.
>What's the H@H system?
hentai@home. It's the hosting system that sadpanda uses.
The server itself is called Glados, it's written in java.
>webm looks like shit
>"but I saved 30KB!"
H@H is easy to log IPs and everyone doesn't participate. IPFS or webtorrent would be much smarter.
The low hanging fruit first: use Http2.
Then use flif for images, vp9 for video, convert gif to video.
Then if you want to save more bandwidth, you can remove thumbnails and load flif images partially as the thumbnail. For the whole image, just keep downloading.
>>webm looks like shit
??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
You guys are missing optimization opportunities, which I'm sure are already being utilized.
Let's assume we're not going to use cloud flare for simplicity sake.
>All pages served as static content.
>write changes to database but don't update pages yet
>regenerate all changed pages every 10 seconds (you can tell this is already happening)
IPs are revealed, just like BitTorrent. But all it means is they have a chunk of data with the same hash as the chunk of data you want. Not necessarily the whole file.
Two computers can have two files with similar chunks of data inside, for example image headers.
It would be even smaller if you served just one template once, cache it using a manifest, then serve only protobuff files (smaller than json) as the static content cached every 10 seconds. And let the client apply the contents to the template.
why are you bragging about have 50 bucks?
>Why not just own the servers themselves instead of buying server power?
Server power is cheaper and easier to scale.
You are stupid, have bad eyes, or a bad monitor if you don't see the difference between and
Applying a template is cheap if done correctly, only a few cycles. The problem is database access, if all threads are kept in ram it's very easy to code something where the kernel is the bottleneck.
And it's not like bittorrent where if you have file X you must have requested it, it can just "end up" there, no?
>webm looks like shit
I thought I'd given up on this board before, but I've somehow given up even more.
You just request requests hashes for chunks so all you can know about the other IP is that they have that chunk. Not necessarily as part of the same file.
Nice I like that idea
Well no shit, fucktard. I tried to compress it as much as possible. A gif can handle 256 colours, a webm can handle 4,294,967,296 colours.
>optimize lossless png compression
I've been thinking about this one. The problem is, optipng can't detect if an image is already optimized. A very large number of images on Sup Forums are reposted images from Sup Forums, so if you could mark an image as already optimized using metadata, you could save huge amounts of bandwidth at the cost of optimizing an image only once.
As I remember Sup Forums archives about 2TB of data so you only need fast servers, not much user db since most don't pay
this
>Let's say I have the money to buy Sup Forums from Hiro
You don't. And I honestly don't think anyone browsing this site has that kind of money. Or if they do they wouldn't want to spend it.
Pic related is prolly the only person who could afford to buy this site just for his own amusement.
You could send a hash of the image before attempting to upload it. If the same hash already exist, reuse the image.
Use reference counting to check which images are not in any thread and delete them. You can do this in a bloom filter for maximum space optimization.
Store he hash of both optimized and unoptimized images, that way you deal with users posting images they downloaded from the site as well as new unoptimized images.
>send a hash of the image before attempting to upload it
That seems kinda unsecure, as users could send fake hashes and cause all sorts of problems.
I think the best solution would be storing the hashes after optimization of all the images that have ever been optimized on the server in a table. It wouldn't take up that much space, and searching wouldn't take much time either. When a user uploads an image, and its hash (computed on the server, shouldn't take much time) matches one of the hashes in the database, the image won't be optimized, otherwise it will be optimized. Plus you could still do the thing you mentioned about saving space through only storing duplicate images once, although saving space isn't really important, as the images on Sup Forums take up roughly a constant amount of space.
I also looked into PNG metadata and read the optipng manual in the meantime, it seems possible to store the optimization level of optimized images in the utf-8 text field, and it looks like the development of optipng has stalled, as there is a list of features that are not yet implemented (such as detecting images with 1-bit colour).
Butthurt: the post
>rename the site luckychan
wew
>make sage actively bring down a thread faster
What, like a downvote?
>Implement trial mode posting and full registration
"no"
>Finally get rid of the bot filter/captcha which doesn't work anyway.
>he doesn't remember Sup Forumsharbor
Your whole post reeks of leddit.
Hash the raw image.
Here's how it should be done:
On image uploaded:
Do sanity checks (size etc)
Hash raw image
Check if imagesUploaded[rawImageHash] > 0
If so, put filename[rawImageHash] in URL field
If not, store image in database
Store image in database:
Strip all metadata
Optimize image (at the lowest level, just the one that gets rid of the most bloat)
imagesUploaded[rawImageHash]++
Generate a torrent where piece size = image size
Upload to ipfs
Store hashes of raw and optimized image in most common hash formats
Fuck, I really want to write a C imageboard engine now.
hook about 10 hard drives to a bananapi
4chinz is now free money, you only need enough ad bucks to pay electricity and replace failing drives
>no sound
lol
No sound enabled on Sup Forums.
So why use a video format with no sound.
Hello, my name is Hiroyuki Ni- err, Nichijou. I also wish to know how to optimize an imageboard for less bandwidth consumption. I will be monitoring this thread.
Billions of more colors, larger resolution, much longer clip duration, and all for a smaller file size. Gif is a highly inefficient "video" format.
I think this is already the case for iOS. I tried posting from my iPad and it's basically broken... the quick reply window doesn't close so you have to wait/keep updating to see when your post went through, then refresh to be able to post another.
Use amazon aws and you're good to go. If you keep the entire userbase, note that it would be expensive. About as much as your rent.
>nosql
Fuck me, why on earth would you want to do a thing like that?
Solution: Require a pass to post anywhere.
Commit suicide. The solution is to use external free hosting, including for example using ipfs/retroshare/gnushare/whatever.
tor, ipfs, and all the other anonymous network shit are 99% child porn and drug deals. Nobody's gonna install that shady shit just to shitpost.
This wasn't a video that was posted though, it was a simple image that looped for a couple of frames. gif is better for that.
At least google things before proving you have no clue what you're talking about.
>AWS
kekd
It's not very expensive to run, hiro is lying. You need 9gbit connection running at 100% 24/7 to serve all the images, that's not a whole lot. CPU is very cheap, could probably run Sup Forums off of hardware for $500-1000. Cloudflare is the expensive part.
Expensive part is bandwidth and redundancy, not hardware. Cloudflare should be avoided like the plague.
>This wasn't a video
Look at the file extension. Is it a video extension? Yes? Then it's a fucking video.
I know what they are and what they were made for, and I'm telling you 1) what they are used for in reality and 2) why nobody normal is interested in using them
It was a .gif though.
Look again. Take some time before you post stupid replies like this.
?
...Did you miss the post literally two posts below it? That's not mine.
No, but that's not the original which was what the discussion was about. It's the content that you put in your video. The original wasn't a video, so why put it in a video format? It doesn't benefit from any of the video format's benefits.
Because I was making the point that webm can produce a much smaller file size, which it did. I can make it look better for a bigger file size, but it would still be smaller than than the gif.
>Because I was making the point that webm can produce a much smaller file size, which it did.
It also looks worse.
>I can make it look better for a bigger file size, but it would still be smaller than than the gif.
I honestly doubt that, considering you'd always be using a lossy compression instead of just using lossless from gif.
>since they cant cope with
Jokes on you nigger, I have unlimited data and 90% of the time a connection greater than 50Mbps
>sent from my htc u11 via firefox mobile beta
How much can this garbage sewer site be worth? Without Sup Forums Sup Forums Sup Forums which supply most of the traffic.
This is at 1MB bitrate.
Also looks worse, looks about the same as the other webm.
At this point the quality difference is so minimal that any difference you do see is a trick of your mind.
You are doing some sort of 4D mental gymnastics there son.
Gotta calm down.