/dpt/ - Daily Programming Thread

Old thread: What are you working on, Sup Forums?

Other urls found in this thread:

thegamesdb.net/api/GetPlatform.php?id=3
developer.download.nvidia.com/NsightVisualStudio/2.2/Documentation/UserGuide/HTML/Content/Timeout_Detection_Recovery.htm
amazon.com/dp/0321486137
thegamesdb.net/api/GetPlatform.php?id=15432432
thegamesdb.net/api/GetPlatform.php?id=3)
docs.oracle.com/javase/8/docs/api/java/io/BufferedInputStream.html
learnxinyminutes.com/
thegamesdb.net/api/GetPlatform.php?id='
shop.oreilly.com/product/0636920028000.do
twitter.com/SFWRedditGifs

Trying to my shitty Discord Bot working on Google Compute

LINQ isnt C#, its a bolted on hack.

I'm going to be honest with you, guys. The most I've programmed is fizz buzz in Python and I shit up EVERY thread with pro-C, C++, Lisp and Haskell shitposts.

int[] arr;

What's special about LINQ anyways. Is it any better than Haskell list comprehensions/monad, or Racket for loops?

Working on an image orgranizer

Is anyone interested?

No

not really, but for the unenlightened masses raised on Java it's pretty refreshing

How is any Rx library (and LINQ, by extension), any more of a "hack" then a languages standard library? It's just another tool in the box

D?

mhm

pretty neat. i just have a folder of raw images and then i make symlinks. that way i can have an image in cats and dogs but it uses the same amount of space.

I have a script written in python that interacts with a MySQL database. It runs super fast at first, then completely slows to a crawl within 24 hours, maybe less. I can restart the script and have it continue from where it left off. When I do that it runs fast again. I close connections when appropriate, and I've given MySQL a lot of resources.

Why does this slowdown happen, and what can I do to make it consistently go fast?

It better than anything from Haskell/Lisps only because it can be used by normies/drones.

go is fun as hell to code in

>in during muh gc

How is the memory consumption?

what's the memory usage like after 24hrs?
is it just the script itself that is slow or does all mysql activity slow down until you close the script?
my guess is something is not being freed but i haven't used python in a long time

Have you read your SICP today, /dpt/?

> no pattern matching & abstract data types.
> no immutable variables.
> no generics.
> no exceptions, (uses error codes everywhere)
> error type is simply an interface to a function returning a string

I was slacking off for weeks, but yes.

Have you done the exercises too?

Yes, that's why, I'm afraid to move on without making sure I fully understand what's going on.

Good stuff user.

>muh jen 'n' erics

I FUCKING KNEW IT

>Jen & Eric

Is it worth doing both SICP and HtDP? Or will you learn enough from just doing one of them?

Memory consistently stays at about the same 25% throughout according to the server status page on MySQL workbench. Almost all MySQL activity is slower than usual if the script is running. However when the script is slow, running selects on either MySQL workbench or the mysql client command line tool is faster than when the script is fast.

According to the resource monitor the disk read/writes start out very fast (SSD speeds) but then significantly slow after 24 hours just like the script.

It is imperative that I remind the rest of the thread on a day-to-day basis that Go is just C for people who can't code in C.

Go is an insult to C.

Start with HtDP and do SICP after that. HtDP is beginner tier and SICP is CS tier.

>CS
Cocksucker?

computer science dumb nigger

What does that last word (beginning with 'n') stand for?

In gdb, is there a way to set a breakpoint that activates right when a certain register is changed to a specified value?

Since the dawn of history the Negro has owned the continent of Africa – rich beyond the dream of poet’s fancy, crunching acres of diamonds beneath his bare black feet and yet he never picked one up from the dust until a white man showed to him its glittering light.

His land swarmed with powerful and docile animals, yet he never dreamed a harness, cart, or sled.

A hunter by necessity, he never made an axe, spear, or arrowhead worth preserving beyond the moment of its use. He lived as an ox, content to graze for an hour.

In a land of stone and timber he never sawed a foot of lumber, carved a block, or built a house save of broken sticks and mud.

With league on league of ocean strand and miles of inland seas, for four thousand years he watched their surface ripple under the wind, heard the thunder of the surf on his beach, the howl of the storm over his head, gazed on the dim blue horizon calling him to worlds that lie beyond, and yet he never dreamed a sail.” — Charles Darwin

When did /dpt/ get infested with Sup Forumstards?

Time to get back to Sup Forums.

Yesterday I was bored so I started working on a game backlog tracker thingy.

At first I was going to manually add the games and the platforms but then I thought fuck it, I'll scrape thegamesdb.net for their DB.

They have this API that gives you an XML file with all the info of the things you request; e.g. thegamesdb.net/api/GetPlatform.php?id=3

So I started coding, but it runs PAINFULLY slow.

I don't know if it's because I fucked up somewhere or because I'm scrapping in a shitty way.

Here's the code:

public void updatePlatforms()
{
BufferedReader bufferedReader = null;
int currentElement = 1;
boolean finishedUpdating = false;
while (!finishedUpdating)
{
bufferedReader = getPlatformXML(bufferedReader, currentElement);
try
{
Platform platform = parsePlatformXML(bufferedReader);
insertPlatform(platform);
} catch (PlatformAlreadyExistsException ex)
{
} catch (EndOfPlatformsException ex)
{
finishedUpdating = true;
}
currentElement++;
}
}


private BufferedReader getPlatformXML(BufferedReader bufferedReader, int currentElement)
{
try
{
URL url;
URLConnection urlConnection;
url = new URL("thegamesdb.net/api/GetPlatform.php?id="
+ currentElement);
urlConnection = url.openConnection();
urlConnection.addRequestProperty("User-Agent", "Mozilla/4.0");
bufferedReader = new BufferedReader(
new InputStreamReader(urlConnection.getInputStream()));
} catch (Exception ex)
{
}
return bufferedReader;
}

Show me the parsePlatformXML method.

Cont.

private Platform parsePlatformXML(BufferedReader bufferedReader) throws EndOfPlatformsException
{
String currentLine = advanceLines(bufferedReader, 5);
if (currentLine.equals(""))
{
throw new EndOfPlatformsException();
}
currentLine = advanceLines(bufferedReader, 2);
String platformName = currentLine.substring(11, currentLine.length() - 11);
Platform platform = new Platform(platformName);
return platform;
}


private String advanceLines(BufferedReader bufferedReader, int lines)
{
String lineToReturn = null;
for (int i = 0; i < lines; i++)
{
try
{
lineToReturn = bufferedReader.readLine();
} catch (IOException ex)
{
}
}
return lineToReturn;
}


It took the "scrapper" 9 seconds to add those 18 rows.

I can't even imagine how long it would take it to add scrape the game list.

Any thoughts?

Beautiful.

The code is complete shit, but I don't see any obvious performance bottlenecks. Profile that shit!

Is it possible for Oracle to make Java programming language proprietary in the future if they wanted too?

I completely forgot how to handle generics in jaba
public class TestClass {

private T key;

public TestClass(int input) {
//There are three input options, and I need to assign one of three data types based on that
//how do
}

}

I feel like this is really obvious but I'm just retarded

Java is a specification, not an implementation. If you're talking about OpenJDK, Oracle doesn't own it.

>The code is complete shit
Why?

> Profile that shit!
From my light testing, I found that the slowdown occurs in the getPlatformXML method.

Java is a language specification. Anyone can make a proprietary JVM if they want.

So what does Oracle actually own?

>I need to assign one of three data types based on that
What do you mean by this?

happens to us all man

So if I read that correctly, you are making sequential requests to their api, waiting for the response before starting the next request. Measure how long the requests take, my guess is you're spending most of those 9 seconds just waiting for the response. You can improve this by sending out multiple requests in parallel, and/or checking to see if you can request larger data sets from the api using some parameter.

It means I'm retarded and forgot that I could just use Object

you can build a JVM that behaves in a very Java like way and be completely open source, but, if you want to call it a "Java JVM" you need to buy the certification suite under a non open source license.

Your mom...
Oracle is a core contributer of Java, but they don't own anything. If they did, the project would've already been forked like what happened to MySQL and OpenOffice.

i have 0 java experience but my guess would be because it's not downloading the page until you walk through using the reader. maybe it is downloading one line at a time per get request (which would be slow as fuck and depending on the site they might throttle you for too many requests. especially as an api)

maybe try rewriting it to download everything at once and see if that fixes it?

>Why?
exception thrown on success
unhandled exceptions
magic numbers
hard coded strings
>From my light testing, I found that the slowdown occurs in the getPlatformXML method.
I guess lineToReturn = bufferedReader.readLine(); is the bottleneck of your program.

Still trying to setup some sort of debugger for D. Now I have two compilers, DMD and LDC. Even though both can compile fine and can be executed with debug arguments, I still can't debug much. GDB refuses to work with either compiler outputs. Can barely use Windbg. It allows me to step through code but I have no visibility into what variables exist in the background. Not sure what to do.

I spent 2 days debugging my CUDA kernel and I just found out that the problem was this:
developer.download.nvidia.com/NsightVisualStudio/2.2/Documentation/UserGuide/HTML/Content/Timeout_Detection_Recovery.htm

I don't know what I feel right now.

does java not have an xml library? this seems like an incredibly low level way to interpret an xml file

If your type, function and variable names are longer than 8 characters, you are a codemonkey.

Give up user, D sucks.
Join Rust :^)

Yup, I think you're right.

I get what you say but I have no idea how to implement that.

What you're saying is part of multi threaded programming, right?

It downloads the whole "page" with each call.
You might have a point with the API throttling though.

Thanks for pointing them out, I'll try to fix them.

The one about the magic numbers I really don't know how to, though.

I got those from looking at the XML page and figuring out how many lines I had to advance and how many characters I had to trim from the string.

>I guess lineToReturn = bufferedReader.readLine(); is the bottleneck of your program.
According to the profiler, the slowdown occurs in this line:
bufferedReader = new BufferedReader(
new InputStreamReader(urlConnection.getInputStream()));


I believe it does, but I started yesterday with zero experience regarding APIs and XML, so I'm just trying to survive, I'll look into it later, when things work..

one last time:

Is it still slow if you read all lines from the input stream without doing anything with each read line?

This book is pretty good IMO. Used it in school.
amazon.com/dp/0321486137

But Rust doesn't interest me.

Yup.

Parsing the file is basically free.

when he's talking about magic numbers he's referring to within parsePlatformXML

Why advance 5 lines? why advance 2 lines? why cut the first 11 characters?
As outsiders reading your code we have no fucking idea what the significance of these numbers are. And in 6 months when you read it again neither will you.

The better way to do it is to declare variables that are set to 5,2 and 11 with names that indicate what the 5 lines, 2 lines or 11 characters you're trying to advance through are

Thanks, I'll check it out.

Tell me about your type system

Strongly normalizing

Oh, okay.

I get it.

Based on the file: thegamesdb.net/api/GetPlatform.php?id=15432432

I should make clear that I advance 5 lines to get to the "" to check if I'm done, then advance 2 more lines in case I'm in a valid file (e.g. thegamesdb.net/api/GetPlatform.php?id=3) to get to this line: "Nintendo 64"

The substring part takes care of removing the XML tags (start from the 11th character and stop at the length - 11 character).

It is anime-complete.

YOUR

WAIFU

A

SHITSUNE

hey guys, what was the url for that site where you just choose a language and it has one long text document that goes over all the syntax and pertinent use cases? I forgot what it was and I want to bookmark it on my new machine for reference

In my experience, constant reads to an external source outside a program can be a rather slow process, at least in Java. It depends on what you are doing and the size of the data on the outside but I'd just ingest all the data into an array and do manipulations from there.

With a BufferedInputStream object, you can check how many bytes are available in the steam when you first create it. As a result, you can create an array of the perfect size to hold the data. Read the data to said array and then close the BufferedInputStream when you are done.

docs.oracle.com/javase/8/docs/api/java/io/BufferedInputStream.html

Can we all agree that C++ arrays are too confusing?

Why are arrays pointers? Why do pointers even exist? I know smart people who've gone to university for three years and still don't understand pointers.

What's the >best typed lambda calculus and why do you like it so much

You'd understand if you learned C first.

That looks very nice. Is there a git?

I'm writing a list of things to work on.

First out is a blog written in racket.

The pointer points to the first member of the array. That way you can just add n to the pointer to get to the n:th element. Pretty simple, in my opinion.

Sounds like you're a brainlet desu senpai

learnxinyminutes.com/

Sorry I don't know java very well, but yeah you probably have to use multithreading, or async io if this is supported.

I whipped up a quick sample in python:

import requests, grequests, time

ses = requests.session()
url = 'thegamesdb.net/api/GetPlatform.php?id='

def scrapeSequential():
for i in range(1, 50):
res = ses.get(url + str(i))

def scrapeConcurrent():
urls = [url + str(i) for i in range(1, 50)]
requests = [grequests.get(u) for u in urls]
responses = grequests.map(requests)

start_time = time.time()
scrapeSequential()
print "sequential: %s seconds" % (time.time() - start_time)

start_time = time.time()
scrapeConcurrent()
print "concurrent: %s seconds" % (time.time() - start_time)

output:
sequential: 11.367000103 seconds
concurrent: 2.89800000191 seconds

>Can we all agree that C++ arrays are too confusing?
Sorry I don't have double digit IQ.

>The pointer points to the first member of the array.
The pointer is just a really long number, though. It's not the first member at all.

shop.oreilly.com/product/0636920028000.do
Enlighten yourself instead of bitching.

Not yet but I'll upload it when i fixed all the problems

Array is just a bunch of bytes addressable by numbers. Like, think of a street where each house has a number.

awesome, thank you

*memory is

why is there such an overlap of sheltered,
goofy southern christian men and notable programmers?

>Charles Darwin
nah, this was a KKK leader that said this iirc. not sure why it get mis-attributed so often

watch $eax == 0x0000ffaa

To break at a point:
break test.c:120 if $eax == 0x0000ffaa

>The pointer is just a really long number, though.
That's because it points to an address in memory. You have to dereference it to get the actual value it points at.

Just learn how to use pointers.

Thanks!