Subject:
Re: C++ vs. Python vs. Perl vs. PHP performance benchmark
From:
Isaac Gouy
Date:
20.6.2012 г. 20:59 ч.
To:
Ivan Zahariev

Hi Ivan 

> I'm testing "loops, arrays with numbers, basic math operations".

1) Please change your blog post to say what you've told me you want to test -- not "arrays with numbers" -- but "lists with numbers" or "resizable arrays with numbers" or "dynamic arrays with numbers".


2) You assume that you're mainly testing - loops, resizabe arrays with numbers, basic math operations - but do you know whether that actually is what you're testing for each of those programs?

When I profile your Java program, I find that 58% of the total time is spent on garbage collection -- not loops, not lists with numbers, not basic math operations.

$ java -Xprof PrimeNumbersBenchmarkApp

58% of the total time is spent on garbage collection and growing the heap, because the initial heap size has not been set appropriately.


> Regarding the special function -- you are right. ... How does this make Java slower?

3) Does it make your Java program relatively slower compared to your Python program?


> "... you can work around the problem by moving the body of main to a new 
> method and calling it once from main to give the compiler a chance to compile 
> the code".
> 
> That's exactly what I've done, so I don't think I'm violating 
> any Java recommendations here.

4) That's what you think you've done but -Xprof shows 18.5% of the calls were interpreted.



> Look, I'm not biased towards Java or any other language -- I don't even 
> program in Java *or* Python. ... I don't know what your idea here is ...

5) The idea is to help you understand that you aren't comparing what you say you're comparing.

If your intention is to compare time taken to grow a list, then shouldn't you start with lists that are the same size, and start with program heap that's the same size, and use the same max heap size?

user+sys for your Java program = 20.56s

But change 
        ArrayList<Integer> s = new ArrayList<Integer>();
to
        ArrayList<Integer> s = new ArrayList<Integer>(8192);
and run with

        java -Xms1G PrimeNumbersBenchmarkApp

user+sys  = 6.9s

I don't think you're comparing what you say your're comparing.



> P.S. You are not answering all my questions -- System.nanoTime() is returning 
> the wall-clock time? If yes, how are you sure that the compiler isn't 
> executing something on more than one core (for example running the loop on one 
> core, and doing garbage collection on the other).

I expect that the JVM is running GC on a different core if possible. Those wall-clock times were for comparison of the first run of the Java program with the second run of the Java program, with the third run of the Java program. Not for comparison of the Java program with the Python program.

best wishes, Isaac