This is a question I’m currently interested in knowing the answer to. I don’t know much about Smalltalk or Squeak unfortunately, but from what I do know, it seems like a good fit for the latest idea I have. The primary advantage over other environments I know, is that the image is available at all times. If an image receives a message it can’t handle because of a programming error, it can enter the debugger and wait for me to come along and debug it. I think that a decent Common Lisp implementation will offer me similar functionality.
Unfortunately, playing around with a new language for a couple of days gives a very limited idea of what it will be like to use it to develop a medium-sized application. For me, during the initial stage of investigation, it is very important to know whether it is worth taking the time with very quickly so that as little time is wasted as possible. Unfortunately, Squeak is a little harder to get to grips with than most languages I am familiar with as it is necessary to learn the IDE at the same time as the language. It is not possible (as far as I know) to simply enter a hello world program into a textfile and execute it.
The first script I entered into the Workspace (which is kind of a scratchpad for exploring new ideas) displayed the numbers from 1 to 200 on to the transcript, which is a kind of logging window. Here is the code to do that along with a timing method. If you highlight thecode and select “print it” from the menu, it tells you how long it took to run.
Time millisecondsToRun: [1 to: 200 do: [:i | Transcript show: i ; cr]]
This was the response: 10335
Ouch! 10 seconds. This is by far the worst performance I’ve ever seen in any language including BBC BASIC on a 2MHz machine. I was certain I must be doing something wrong so I asked on the Squeak beginners list. Unfortunately, it seems that it really is that slow. No-one has made any effort to tune the performance. Here is the comment from the Transcripter class:
"Transcripter is a dog-simple scrolling stream with display. It is intended to operate with no
support from MVC or color in a minimal, or headless version of Squeak. No attention has been
paid to appearance or performance."
Okay. Well that is a pity – it is one of the first parts of the Squeak people will see and I would have thought that it would put a few people off. Anyway, the “everything is an image” is still selling me so I try to see how long it takes to write 10,000 numbers to a file.
| myFile ios |
ios := ReadWriteStream on: ''.
Transcript show: 'Populate Buffer: ',
(Time millisecondsToRun: [
1 to: 10000 do: [
:i | ios nextPutAll: ((i asString) , String crlf)
]]) asString , ' millseconds' ; cr.
MessageTally spyOn: [
1 to: 10000 do: [
:i | ios nextPutAll: ((i asString) , String crlf)]].
Transcript show: 'Position: ', (ios position) asString ; cr.
myFile := StandardFileStream fileNamed: 'c:/test.txt'.
Transcript show: 'Output Buffer: ',
(Time millisecondsToRun: [
myFile nextPutAll: (ios contents)]) asString,
' milliseconds' ; cr.
myFile close.
The MessageTally spyOn: is a nice profiler to see where it is spending its time – it is not really part of the initial test.
This produced the following output on the Transcript:
Populate Buffer: 275 millseconds
Position: 58894
Output Buffer: 9 milliseconds
Hmmm… is 275 milliseconds good or bad? I wrote a quick perl script and ran it with Cygwin.
use strict;open (FH, "> t3.txt") || die "Error: Can't write to file";
for (my $i = 0; $i < 10000; ++$i) {
print FH "$irn";
}
close FH;
It ran in 0.061s – more than 4 times faster than the Squeak. Well, this is not very good at all. MzScheme and Chicken Scheme both produced results similar to the Perl. Asking on the mailing list again, it seems that the integer to string conversion is implemented in Squeak for flexibility whereas [it is suspected that] all the other languages use C libraries.
Now, I’m not the sort of person who is in favour of premature optimisation, but if all the primitives are 4 times slower than Perl for reasons of flexibility that seems like a huge performance hit. It seems like it is harder to drop into C from Squeak than it would be from other scripting languages and Perl speed is about as slow as I like to go normally anyway. Maybe further investigation would reveal that the Transcript and integer to string comparisons are unusually slow. However, I have to balance the time spent investigating new programming languages is time I’m not spending developing my application.
Hi Ramon,
You’re right – these tests are not very comprehensive but at my current level of Smalltalk, they are about as good as I can do. I can’t find benchmarks saying how fast Squeak is in comparison with other languages (although I get the impression that people think it is at least as fast as the popular scripting languages with the exception of I/O performance). Now, if I spend say, 3 months playing with Squeak and implementing a real-world problem and then I discover it isn’t fast enough for my needs then it seems to me that I could have used that 3 months more productively. This is always the worry with trying out a new system which is why I normally try a couple of trivial tasks before deciding whether to investigate further or move on to the next thing. How easy is it to implement a Squeak primitive in C – is there some documentation on the wiki that describes how to do that? The page I found at: http://users.ipa.net/~dwighth/squeak/diy_sq_prims.html makes it look like it isn’t very easy to do.
Ian
Measuring a programming languages speed based upon how fast it can do something trivial in a tight loop is never a good measure of anything. You should test it on a more real world problem, and keep in mind that Squeak gives you the tools to easily identify any bottleneck, and you can always implement a primitive in C to speed up that one piece.
Squeak is hardly the fastest Smalltalk, it’s more a research Smalltalk where being implemented in itself is more important than speed because it makes the system more hackable. Squeak is a place where people try out new ideas, a place where things like continuations, sections, futures, and currying can and have been implemented in a few lines of code.
If much of the system were implemented in C like most other scripting languages, these things wouldn’t be possible, the system would be too hard to change. There’s a reason things like Croquet and Seaside are coming out of the Squeak community, it’s the only one flexible enough to allow these things to flourish. Squeak is written in Squeak, for a good reason!
I am making these numbers up, so they might not be accurate:
Java: 1
Strongtalk: 10x Java
Squeak: 100x Java
Ruby: 300x Java
Jules wrote I am making these numbers up…
The puzzle is why are you making up numbers and posting them as though they mean something?
IanO wrote … from what I do know, it seems like a good fit for the latest idea I have
Are those 2 simple code snippets a good representation of what you need to be fast when you implement your “latest idea”?
IanO wrote I can’t find benchmarks saying how fast Squeak is in comparison with other languages…
Squeak compared to Perl
Thanks Isaac, that is link is good, cheers. As to whether those two snippets of code will need to be fast when I implement my app, well, certainly I will want to be logging a lot of information (but I guess I won’t be using the Transcript). I don’t know how much I will be converting integers to strings, but I generally think of this as a primitive operation which is really easy to make fast simply by calling the appropriate C function. Certainly the other 5 or so languages were within 10ms or so of eachother with Squeak the only outlier.
Ian
IanO wrote I don’t know how much I will be converting integers to strings, but I generally think of this as a primitive operation which is really easy to make fast simply by calling the appropriate C function.
If it’s not a bottleneck in your app/service it really doesn’t matter how fast or slow it is – identify what your app/service will need to do, and then bother about those things being fast enough.
If Squeak is too slow for your application, your effort still isn’t necessarily wasted, since there are commercial Smalltalks that are quite fast. Recently on reddit I found an independent benchmark of one that came out as fast as C++. And there are free versions available as well…Dolphin Smalltalk on windows is quite nice, and has a free version with most of what’s in the commercial version.
Unfortunately, without writing the app, I can’t really tell what is going to be a bottleneck. However, I can extrapolate from a small number of tests. My reasoning was that if one primitive is implemented in Squeak for flexibility then the majority will be. In this case, the primitive was more than 300% slower than in the other languages I tested. Extrapolating from that, is it reasonable to assume that Squeak is going to be more than 300% slower than Perl for example? Judging from the link to the shootout you posted, the answer is no.
Dennis, that is an interesting comment. Does that mean there is an easy way to port Squeak code to other Smalltalks?
IanO wrote Unfortunately, without writing the app, I can’t really tell what is going to be a bottleneck.
Write a prototype to figure out what your app is going to be about and learn what things seem important – then throw it away and rewrite your app.
(Haven’t you just learned that “extrapolating” from insufficient information gives very bad answers?)
“The puzzle is why are you making up numbers and posting them as though they mean something?”
Because I think they do. They are based on things I’ve read on the internet (example: http://smallthought.com/avi/?p=17) and on my own benchmarks. I posted them because I think it’s good to know that there are much faster Smalltalks available and that Squeak isn’t really slow compared to Ruby.
I’m no expert…I’ve been playing around with both though. It looks to me like the standard Smalltalk stuff is pretty similar; the language is the same, and they both have the libraries described in the Smalltalk-80 books.
The big differences would be the experimental stuff in Squeak (like traits and Morphic) and the win32-specific stuff in Dolphin.
I haven’t actually *tried* porting between them, but I have seen opensource smalltalk projects with versions for both. One major example is the web framework Seaside, which has downloads for Squeak and VisualWorks, and an initial port to Dolphin.
Also of interest is Exupery, a compiler in development for Squeak that speeds it up to about half the speed of C.
At the moment I’m doing more with Dolphin…it comes with a pretty extensive tutorial, doesn’t look so weird, compiles normal Windows apps, and actually uses Cleartype….little dots make me crazy. Down the road I’ll probably do more with Squeak.
Well no, I haven’t just learned it – I always knew that. But sometimes if you haven’t got enough data, it is the best you can do. I’m suspect that most people would say the data from the shootout is also insufficient. I’m still not convinced that Squeak is fast enough for my purposes or that it will be easy to speed up the slow parts if they become a problem. If I stayed with Python for example, I would be confident that I could rewrite bits as necessary in C.
It’s a while since I did any Smalltalk seriuosly and I’ve never done any Perl, but it looks to me like the two versions aren’t doing the same thing. I’m not sure how much difference it would make though – maybe very little.
The Smalltalk version seems to be building a huge string so there’s going to be a lot of string buffer re-allocation and copying going on. This could get quite time consuming as the string gets really big. This big string is then dumped into a file.
The Perl version however is just writing the numbers sequentially into the file so there is no memory overhead.
Did I misread the code? Of course the amount of memory isn’t really all that big, but might still be skewing the results.
Am I reading that right? You’re using a profiler in Smalltalk, and just timing the perl? Apples and oranges. Profilers have an inherant performance penalty in any language.
My initial version wrote directly to the file and took almost the same amount of time. On the Squeak beginner list it was suggested that this was because the I/O is buffered in Perl and unbuffered in Squeak and the recommendation was to write to a stream in memory first and then dump to a file.
The timings reported come from this line without the profiler running:
Transcript show: 'Populate Buffer: ',
(Time millisecondsToRun: [
1 to: 10000 do: [
:i | ios nextPutAll: ((i asString) , String crlf)
]]) asString , ' millseconds' ; cr.
The profiler shows that 80% of the time is spent converting i from a SmallInteger to a String.
I think if you look at the use some people have put Squeak through (heavy multimedia capabilites), you would see that Squeak isn’t such a loser proposition. Take a look at the Scratch programming langauge for kids, or Croquet.
[…] When I retried the same in Squeak it took around 5700ms although I’m certain it had earlier taken more than 10 seconds. This is a different computer and a different image but maybe the Squeak Transcripter isn’t […]
I have started learning smalltalk just a couple of months ago so i decided to try out your code, i tried your code in 3 different smalltalk implementations (squeak, Smalltalk/X and GNU Smalltalk) i had to make some minor changes (#printString instead of #asString and different classes for file streams). The results were as follows:
Squeak: ~170ms
Smalltalk/X: ~10ms
Gnu Smalltalk: ~80ms
I know Smalltalk/X has some optimizations through primitives but i’m not sure about Gnu Smalltalk.
After, i decided to try a different approach outputting the 10,000 numbers directly to a file:
|ios|
ios := StandardFileStream fileNamed: 'c:\test.txt'.
Transcript
cr;
show: 'Write to file stream: ';
show:
(Time millisecondsToRun:
[1 to: 10000 do: [:each| each printOn: ios. ios nextPut: Character lf]]) printString;
show: ' ms';
cr.
This last code performed with the following results:
squeak: ~210ms
Smalltalk/X: ~10ms
Gnu Smalltalk: ~62ms
So if i see it correctly the Smalltalk/X primitive optimizations result in a constant time for both approaches. Gnu Smalltalk slightly benefits from this approach (FileStream possibly performs buffered I/O ??) while squeak comes last with even worse results. I’d believe commercial smalltalk implementations should yield even better results (Cincom Visual Works, Instantiations VA Smalltalk, Dolphin).
So although i find smalltalk very acceptable in terms of performance for most applications, i must agree that at least when comparing to other implementations and almost certainly with other languages, squeak is somewhat slow.
Hi!
I am a long time Squeaker/Smalltalker and looked into this for fun.
If we look at your original code there are a few things we can do:
1. Use #printString instead of asString. Of course gives almost no improvements but it might be the “correct” message to use.
2. Avoid concatenation since it has to create new buffer and copy etc.
3. Not call “String crlf” every time. We can assign “crlf := String crlf” in a local var and use that in the loop.
4. And finally of course, we could create an optimized #printString for SmallIntegers.
So let’s see, here is SmallInteger>>printString:
printString
"Highly optimized version for base 10, and utilizing the fact
that we know SmallInteger maxVal and not using a WriteStream."
| integer next result i sz |
self = 0 ifTrue: [^'0'].
self 0] whileTrue: [
next := integer // 10.
result byteAt: i put: 48 + (integer - (next * 10)).
i := i - 1.
integer := next].
sz := 10 - i.
^(String new: sz) replaceFrom: 1 to: sz with: result startingAt: i+1
And here is your snippet with some modifications:
| myFile ios |
ios := ReadWriteStream on: ''.
crlf := String crlf.
Transcript show: 'Populate Buffer: ',
(Time millisecondsToRun: [
1 to: 10000 do: [
:i | ios nextPutAll: i printString; nextPutAll: crlf
]]) asString , ' millseconds' ; cr.
MessageTally spyOn: [
1 to: 10000 do: [
:i | ios nextPutAll: i printString; nextPutAll: crlf]].
Transcript show: 'Position: ', (ios position) asString ; cr.
myFile := StandardFileStream fileNamed: 'test.txt'.
Transcript show: 'Output Buffer: ',
(Time millisecondsToRun: [
myFile nextPutAll: (ios contents); close]) asString,
' milliseconds' ; cr.
The end result? On my box this makes your test run more than 4x faster – from 141 ms to 34 ms for the population. I would presume this matches both Perl and GNU St given your numbers.
Now, note that I did this just for fun – the comparison in itself doesn’t say anything in general IMHO. I was just curious how much faster we can make it using “regular” optimizations.
regards, Göran
Hmmm, ok, the code-tag didn’t like < and > obviously, one more try:
printString
"Highly optimized version for base 10, and utilizing the fact
that we know SmallInteger maxVal and not using a WriteStream."
| integer next result i sz |
self = 0 ifTrue: [^'0'].
self < 0 ifTrue: [^'-', self negated printString].
result := String new: 10.
i := 10.
integer := self.
[integer > 0] whileTrue: [
next := integer // 10.
result byteAt: i put: 48 + (integer - (next * 10)).
i := i - 1.
integer := next].
sz := 10 - i.
^(String new: sz) replaceFrom: 1 to: sz with: result startingAt: i+1
Hehe, ok, after further trixing I am down to 30 ms with cleaner code.
I am posting these changes to squeak-dev.
Hi Goran,
That is really cool: it really nicely demonstrates the benefits of turtles all the way down I guess. Cheers for the update.
Ian
Lots of answers to your question already, to summarize: depends on the implementation on the methods you’re using in Smalltalk, and on the implementation of the virtual machine.
Well, and then did I mention that it depends on the implementation of the virtual machine? It turns out that fast VMs for dynamic languages are kind of like black magic in terms of the number of people who understand them well, but you can get one for Squeak now!
http://www.mirandabanda.org/cogblog/downloads/
It has a JIT, polymorphic inline thingummies, and gets the job done right fast. Combined with improvements made to the Smalltalk code recently in Squeak, the system has gotten quite fast.
Wandered past this post for the second time and thought I’d set the record straight now that the game has changed in the Squeak world:)