What the fuck is wrong with python?
Test file: 33752686 lines of 10 random characters, 354M
Test one:
with open('out') as F:
strs = F.readlines()
newlist = []
for i in strs:
newlist.append(i.upper())
real 0m29.758s
user 0m17.106s
sys 0m9.439s
Test two:
with open('out') as F:
strs = F.readlines()
upper = str.upper
newlist = []
append = newlist.append
for i in strs:
append(upper(i))
real 0m19.691s
user 0m16.086s
sys 0m3.348s
?????????
I am in halfway long project (6000+lines) using Python/Django and I am cursing this shit all day,
I wish I used Go or Elixir
In the second one:
>upper = str.upper
You didnt even define str. Or is str a standard object? I do not know python
>elixir
>read file
well.
what the fuck is the point of this benchmark
That type function lookups are slow due to the lack of a JIT.
hey, try this:
with open('out') as f:
newlist = list(map(str.upper, f.readlines()))
Yes senpai
I don't even know code
The python way.
There's pypy
real 0m15.967s
user 0m11.961s
sys 0m3.730s
Noice
Not OP, thanks for this, havent heard of this before.
>ruby3: let's make it 3 times faster!
>python3: let's make all the code broken!
It has to do lookups every time because it's just so damn dynamic. str.upper could be a @property that returns a new object every time. There's no way for it to know so it can't cache the lookup. Python is meant to be easy and quick to write glue code. If this is your bottleneck then you are doing it wrong.
Calm down mr Autism, that's exactly what I was trying to show in my example.
If you're doing the same task on every line anyway, then you can just do
with open('out') as F:
whatever = str.upper(F.read())
real 0m1.109s
user 0m0.483s
sys 0m0.604s
Would you have the file perhaps so I can benchmark too?
Files are iterators now:t
newlist = []
with open('out') as f:
for line in f:
newlist.append(f.read().upper())
Just generate a file, there is nothing specific
def junk():
abc = 'abcdefghijklmnopqrstuvwxyz01234567890'
with open('out', 'w') as F:
for i in range(8500000):
F.write("".join(sample(abc, 10)) + "\n")
And fuck user that's fast:
newlist = []
with open('out') as F:
newlist.append(F.read().upper())
real 0m1.108s
user 0m0.485s
sys 0m0.613s
>with as f:
What is that syntax for? loop?
"Open this file, refer to the file object as f and close it when done"
Temporal and spatial locality
poorman version of
File.open('out') do |f|
end
its actually a python class with an __open__ and __exit__ function. Open calls the __open__ function within and exit is called when you exit the loop.
The idea is that you can create your own classes so that exit() gracefully handles exceptions and cleans up your environment when done. In open's case it opens the file and gracefully closes it without needing F.close()
if you want fast code, don't write in python. end of story.
Weird, doesn't really seem to work for me (800K lines, atom shit cpu)
Function ran in: 9.590599298477173s
Function ran in: 10.747447967529297s
Function ran in: 10.441354751586914s
Similar to using in C# and try with ressources in java
or spend the time to actually optimize your code using cython