-1- Venkat Subramaniam on polyglot programming, the JVM, and ...

Java technology zone technical podcast series: Season 1 Venkat Subramaniam on polyglot programming, the JVM, and multicore processing Episode date: 12-28-2010

LANINGHAM:

This is a special series of the developerWorks

podcasts with Andy Glover, a developer, author, speaker, and entrepreneur. Andy is managing partner at Beacon50, a regular contributor to several technical publications, including developerWorks; he writes the Java development 2.0 column. He's also creator of the award-winning easyb, a behavior-driven development framework. And here he talks to a variety of experts in the Java and related technologies community on topics ranging from Clojure to open source business intelligence, and even .NET. Here's Andy.

GLOVER:

Venkat Subramaniam is a world-renowned speaker

and author and obviously excellent developer. Back to what he's written, though, he's written four books, one on Groovy, one on Scala, one on .NET, and then one on the practices of being an agile developer.

But specifically, the three books that I first mentioned were all different languages or even different platforms: .NET, Groovy, and Scala. So, Venkat, why are you interested in all these different languages and platforms?

-1-

SUBRAMANIAM: Languages certainly are very important, because I think they greatly influence our ability to design software.

What's interesting is there is a bit of a controversial theory called Sapir-Whorf theory or hypothesis, and what these guys are saying is that the languages that we know -and their research is on natural languages, not programming languages -- they're saying that the languages we know influence our thoughts. Now, this is quite contrary or counterintuitive, if you will. A lot of times we think that we express our thoughts in languages. What these guys are saying is that the languages that we know actually influence what we think.

And I'm certainly not a natural-language expert, even though I do speak a couple of different natural languages, but I can certainly vouch for that when it comes to programming languages.

There was a time when I used to know one language, and I would design software, but the time that I got exposed to multiple different languages, I can certainly see that the way that I would approach and design software is quite different. So if you want to be a better software designer, I think it's important that you learn more than one language.

-2-

GLOVER:

Interesting you mention that, because your

Practices of an Agile Developer -- you co-wrote that with Andy Hunt, right?

SUBRAMANIAM: Yes.

GLOVER:

And obviously Andy Hunt wrote for the Pragmatic

Programmers the actual book The Pragmatic Programmer. And in that, didn't they kind of espouse this whole notion of learning a new language every year?

SUBRAMANIAM: Yes. In fact, if you know, let's say, C++ and then you learn Java or C#, it doesn't count as learning a new language. You really have to learn a language that's substantially different from the language that you are familiar with.

And there's a huge benefit. And Pragmatic Programmers recommend that we learn a new language at least once a year. I would argue we should be learning a language almost all the time.

And the reason is -- I was teaching a course for a client a couple of months ago, and in the audience I had several people of different age, and thinking about two people of almost the same age in the room, one person had a lot of

-3-

difficulty understanding the concepts, another person with about the same experience, same age, found it much easier.

And I was kind of analyzing this along the way, and what I realized was the person who found it the most easy was the person who was programming in multiple different languages.

What he was doing along the way is he was not learning a new language; what he was doing was he was performing the deltas. He knew quite a bit, and every step along the way he was comparing the delta stuff -- how this language is different and how it is similar to what he already knows.

GLOVER:

I see.

SUBRAMANIAM: The guys who had the most difficulty were the ones who knew one language fairly well, but it was a whole new learning in all the concepts for them in this new language.

So I would argue the time it would take for somebody to learn a new language is inversely proportional to the number of languages they have learned over the past 10 years. If they have learned and used only one language, I would say that probably is going to take them the longest time to learn a new language.

-4-

If they have picked up, let's, say four or five different languages in 10 years, they're going to probably pick up a new language in a few weeks, at the best.

GLOVER:

Speaking of, then, different languages,

obviously this is a Java-focused podcast, but we've touched on Clojure, we've touched on Groovy in other podcasts. What kind of languages do you, Venkat, recommend that a Java developer start to seek out and learn?

SUBRAMANIAM: This is the beauty of the world we are in right now. We are not limited. As a programmer, we have the power in our hands. If you are interested in an object-oriented style of programming, you have languages like Java and Scala, where Scala is more object-oriented than Java itself, more statically typed than Java itself, if you will, in a lot of different ways.

And but if you're interested in more of a functional style of programming, you could be looking at Clojure or Scala for that matter. If you're interested in dynamic languages, you can pick up JRuby or Groovy, for example.

So literally, you could just sit there and reach out and pick up the languages that you like and still exercise those languages on the Java Virtual Machine. And what's even better is you can interact and also pretty much use any

-5-

Java library and access those from any of these languages. I can't think of having any better power on our hand today than we did let's say four or five years ago. This is luxury for us, I think.

GLOVER:

I agree with you. So let's go back to Scala.

How will Scala change the way I develop or design applications? What's the benefit of Scala?

SUBRAMANIAM: Scala is a very interesting language. It is a language that's a bit intimidating for beginners. However, I see real beauty in the language if you have the patience to work with the language itself.

But what's really exciting about Scala for me is Scala is a statically typed language, for one. You may say, hey, they should all be statically typed. Well, it turns out Java is not as statically typed as Scala is.

If a language is more statically typed, you would actually end up typing less using your fingers. Java, on the other hand, is less statically typed. You actually have to specify a lot more types. Scala interprets typing quite a bit. So Scala is more statically typed.

Scala is concise and very highly expressive. If you look at some of the code that you write in Scala, it's just simply

-6-

amazingly beautiful and highly expressive. XML is a first-class citizen in Scala. Any well-formed XML syntax is a valid syntax in Scala.

If you want to go out to a web service and pull some XML data and process it, you're looking at about two, three lines of code. The XPath queries are built in as a language syntax. It's just purely amazing.

The pattern-matching capability is just awesome in Scala. And then of course Scala supports the actor-based concurrency model, so you're able to write fairly powerful, high-performing code that's not as prone to error as in Java, and you can deal with concurrency quite effectively.

So in several ways I think Scala is a very interesting language. But one thing that Scala brings to the table -- as Martin Odersky, the guy who created Scala, took it as a challenge to say -- we can have a language that is a marriage of both functional style of programming and object-oriented style of programming.

Now, this is a blessing and a curse, and the reason is if you want pure functional style, Scala doesn't give you, but if you want to move into a functional style of programming, then you have the benefit of either coding object-oriented or coding functional or mixing them in Scala.

-7-

So it brings the best of both worlds to you so you can take advantage of imperative style of coding or you can turn the dial all the way to the right, and then you can do a functional style of coding as well in Scala. So it's pretty interesting to mix the paradigms, if you will, in one language.

GLOVER:

Let's go back to different platforms, different

languages, and you even mentioned different paradigms. You were talking about object-oriented programming and functional programming.

Could you describe for us maybe the differences between the two? I'm sure there are listeners that probably are morewell versed in object-oriented programming than they are in functional. So what is functional programming? When would I want to use functional over object oriented?

SUBRAMANIAM: This is a very interesting and important question, I think. I would say languages over the past 20 years have worked really hard to dumb us down. And while object-oriented programming is certainly interesting and it brings value, don't get me wrong, not everything in the world needs to be objects.

If you think about object-oriented principles itself, we

-8-

have abstraction. But abstraction is nothing new in object-oriented programming. We have done abstraction ever since we have been thinking.

What is really important is not inheritance. Inheritance actually is an abused concept. So if you leave those two alone, what's really important in object-oriented programming is encapsulation and polymorphism. And it turns out that to enjoy encapsulation and polymorphism, we really don't need objects.

Objects can have encapsulation and can provide polymorphism, but so can procedures. And so you could write procedures that are encapsulated fairly well, and we could inject polymorphism in these languages as well.

When you decompose an application, it doesn't have to be decomposed into objects. It could be certainly, but we could also decompose applications into functions. So the essence in functional programming is that functions are first-class citizens that we can create and pass around functions.

If you're used to object-oriented programming, you know that you can create objects within functions. Well, you can create functions within functions in a functional language. Again, back to object-oriented programming, we can pass objects to functions, and you can return objects to

-9-

functions. In functional languages, you can return functions from functions; you can pass functions to functions as well. So functions are first-class citizens.

That gives you granularity of reuse, if you will. But going beyond that, functions really are higher-order functions, so you can pass functions to functions. But functions also in functional programming don't have any side effects.

So this really is an interesting concept to think about, because when you have been programming in Java -- if you are a programmer who has used Java extensively, as both you and I have, Andy -- we are used to mutable state. We create objects and then we call functions on it, we continue to modify the object.

Now, if you really think about it, mutating objects is not a bad thing. If you are only looking at mutating the object, that itself is not a bad thing. If you want to share objects, sharing objects is not a bad thing. Sharing is actually a good thing. Mom always said sharing is good, right, go share.

GLOVER:

That's right.

SUBRAMANIAM: So sharing is good and mutability is good, it's not bad. But shared mutability is pure evil. And the minute

-10-

you take a mutable object or data and you share it, you've got trouble with concurrency. So what functional programming says is we're going to promote immutability. You cannot modify something once you create it.

And it takes immutability to the next level. If you have a function and you pass a parameter to the function, the function will not modify the parameter it takes. And the function will not modify any object outside its reach, and the function will simply create new data or new objects and then return those to you.

In other words, if you want to think about it, you can think about a function as a cylinder, and you pass some data. As long as you keep passing in the same data, the output this function is going to give is going to be exactly the same, no matter how many times you call it, because this function has no side effects. And because the function has no side effects, it is not affecting anything outside, and it is not affected by anything outside.

So you say, okay, what's the big deal about having functions that have no side effects? Well, for one, it is easier to prove mathematically the correctness and the behavior of these functions. Maybe most of our listeners don't care about that part, so let's ignore that. But let's say we have expressions.

-11-

And after all, functions are, you know, composition of expressions. So if I take an expression ... I remember back in time I was working for a company, we were doing scientific computing, and we were porting applications from I believe it was from a UNIX platform to a Windows platform back in time.

And I had a programmer who spent literally several days at work. I mean, I could tell you that because we couldn't go near him. He was beginning to smell. He hadn't gone home. And he literally spent several days, and he was trying to find out why the response of this program was not the same on both platforms.

And what he eventually found was one expression in which there was a postincrement operation on a variable being done twice. He was furious when he found this. And he brought back Bjorn Stoustroup's book and said, right here on this page he clearly tells you not to do this, and somebody had written the code to do exactly that.

But this is not just a lack of compliance to what C++ gave in those days. It's a much deeper problem. And the problem is that this is an example of an expression that has side effects. It's leaking all over.

-12-

And when you have an expression like that, it becomes very unpredictable. But when you have a function or an expression with no side effects -- so imagine you have an expression like this, and if this doesn't have any side effects, I can break it, and I can run them, the first one and the second one, or I can switch them over and I can run them as a second one and the first one, or I can even run them concurrently now.

We call this referential transparency. It turns out functional programming, because of its nature -- meaning functions are higher-order, functions don't have side effects, and it promotes immutability -- it is very suitable for concurrent programming.

And this has been around for, what, 40, 50 years now. The time is right for us to make use of it. And I say this has been around for 40, 50 years, because we don't want to pick up something that's newfangled and not tested. This has been around for a while. This has been used fairly effectively. But now we have seen a real good purpose for really utilizing it.

So I think the timing is really right for functional languages, because it really solves the problem in a very effective way, and we're being driven to the situation where we really need that solution as well.

-13-

GLOVER:

With Scala, when you're programming in a

functional style and let's say you develop a function, you mentioned that functional programming kind of doesn't allow mutability, is this something that Scala guarantees? Can you write a functional program in Scala that by its nature isn't functional? Or is that the power of Scala in that it guarantees that if you write a function, that you can't screw it up, let's say, like perhaps you could in Java.

SUBRAMANIAM: That's a good question. Scala doesn't guarantee it. If somebody is determined to shoot them in their foot, hey, remember, we live in a free country. We can't stop people from doing that. No. Scala, unfortunately, doesn't guarantee it.

And that's by design, by the way. The intent of Scala was to really marry and have a language that bridges functional style and object-oriented style. It's a bridge between imperative style of coding and functional style of coding.

So in Scala you could write purely imperative code like in Java, or you can also write purely functional code like in Erlang, or you can mix them together.

But here's the good news, though. In Scala, if you want something to be mutable, you would use the var type. If you

-14-

want something to be immutable, you would use a val type. And it's easy to correct for where there's a var in the code.

If you can ensure that whatever you treat as a var is completely encapsulated and never exposed between multiple different actors, then it's easy to verify this.

And, generally speaking, when I write Scala code, I tend to use vals much more than vars. And if I do want to use a var, I would be thinking about it several times and I would ensure that this is thoroughly encapsulated and never exposed and doesn't go across multiple actors or threads at all. So while the language doesn't guarantee, there are ways for you to verify that fairly decently, I think.

GLOVER:

You wrote a book on Scala. You also wrote one

on Groovy. And I'm guessing you'll be able to wax very poetically on the differences of the two. We covered, if you want a more functional, more rigidly or statically type language, Scala might be the way to go. Why, then, would someone want to leverage Groovy?

SUBRAMANIAM: The real purpose, I would say, is the fundamental difference between the two certainly to begin with is Scala is statically type language. Groovy is a dynamic language. But more than the typing itself, the clear

-15-

benefit in my mind for a language like Groovy is its metaprogramming capability.

If you want to do method synthesis, now, metaprogramming is where you write code that can synthesize code at runtime. And this has a huge benefit. Frameworks like Rails and Grails depend heavily on metaprogramming capabilities. Rails uses the metaprogramming capability of Ruby and, similarly, Grails uses the metaprogramming capability of Groovy. And to me that is one of the most significant benefits.

Ola Bini talks about this nicely in what he calls a language pyramid. He looks at a pyramid, where the bottom part of the pyramid is infrastructure code. This could be written very effectively in a very statically typed language like Scala or Java where you want good type verification, you want very good performance, you want to be able to enforce a certain typing through interfaces, maybe you want to solve the problem of concurrency, and so on.

And then on top of that he talks about a layer which is more of a dynamic layer. This is where you would enjoy metaprogramming capabilities. And, seriously, what metaprogramming does for you is it reduces the amount of code you would have to write. And you have direct experience with this, Andy, is the amount of code size reduction you can see, where you can write something extremely flexible in

-16-

code. What it may take you maybe thousands of lines of code in a language like Java, you could finish within a hundred lines of code or even less in a language like Groovy, because you can do quite a bit of method synthesis.

And then on top of that, the tip of the pyramid is more of a domain-specific language layer, or a DSL layer. And, yes, surely you can write DSLs in a language like Scala, but a lot of times, when I look at DSLs, I'm really interested in a lot more dynamic behavior than a predetermined set of keywords, if you will.

What if I want to do something that is command-driven or like a command pattern, where language learns and evolves based on a certain situation or condition or a state of an application? If you want to go with really that dynamic, it is incredibly difficult to do in a statically typed language unless you use external tools like AspectJ and so on, which are pretty heavyweight and cumbersome to use.

Whereas languages like Groovy, they have this built in. It's almost embarrassingly trivial to do these things in languages like Groovy. So in a typical enterprise application, I clearly see a need for mixing these languages. If we want heavyweight lifting, very heavyweight XML processing, pattern matching, concurrency solutions, a functional style of programming, I would probably drop in

-17-

and use something like Scala.

If, on the other hand, I want to do writing DSLs or I want to write the web tier in my application, I would much rather use some language like Groovy, which provides me greater flexibility and metaprogramming capabilities.

So it's not one or the other, it's not mutually exclusive. I think we have a choice. I look at these languages as vehicles. I may ride a bicycle to my park, I drive a car around town, I take an airplane when I have to fly longer distances. In a given day I may have used a couple of different vehicles.

And the language is a vehicle to navigate your applicationdevelopment landscape, and I think it's only wise for us to be able to pick and choose the one that solves the problem the best.

GLOVER:

I agree a hundred percent. I love the analogy

of traveling. And the beauty of this whole paradigm is that the JVM itself now supports all these different paradigms: like you said, Groovy, JRuby, Scala, you name it.

What I think is also interesting, and I'd love to pick your brain on this, is that this is by no means a new paradigm. This has been happening for more than a few years now. How

-18-

is that now affecting the JVM itself? We're waiting for Java 7. Are there influences based on what's going on in the JRuby, the Groovy, the Scala world that are now reflecting themselves back into the core JVM itself?

SUBRAMANIAM: Oh, actually, that's a very good question. While we have these multiple different languages, Java, when it came out, wouldn't quite distinguish, you know, in the day one as much as programmers between the Java language and the Java platform. You know, it was kind of bundled together to begin with.

GLOVER:

Yes.

SUBRAMANIAM: But as time went on we realized that the real power on hand is the Java Virtual Machine. You have an incredibly powerful platform. The garbage-collection capabilities of the platform, the performance and the stability of the Java Virtual Machine. The Java Virtual Machine has only become stronger and stronger over the past 15 years.

The Java libraries that we have only become stronger over the past 15 years. The Java language, on the other hand, has become somewhat the weakest link of these three. The real correct statement was: Java was simpler than C++, but Java itself has become complex over time. And if you look at the

-19-

capabilities that got added into the language, they have been making the language a bit more complex and not as productive as you would expect.

But here's the power in our hands. Rather than complaining about the Java language, we could really celebrate the Java platform, and then we can reach out and pick and choose. If you really want static typing and a very highly expressive language, you could program in a language like Scala and still program on the Java Virtual Machine and benefit from the existing libraries, because you can interoperate with them extremely well.

On the other hand, if you want to make use of more of the dynamic-language capabilities. you could use something like JRuby or Groovy, so this really gives us the power on our hand. But then the question is, all right, that's cool, I'm decided, I want to program in a language like Groovy or JRuby, what's the gotcha?

The problem is that if you look at the Java code or the Scala code that you're writing and the minute you hit the compile button -- it's a use javac or scalac to compile your Java or Scala code to the bytecode -- and when you examine the bytecode, just do a javap -c on it and take a look at the bytecode produced from any of these languages, and what you'll notice in the bytecode is calls like invokevirtual

-20-

and invokespecial, invokeinterface, and so on.

So there are pretty much four invoke methods that are supported at the bytecode level. And it turns out that the four invoke methods at the bytecode level inherently are statically typed. In other words, the Java Virtual Machine and the JIT compiler clearly expect you to specify the type of the object at the time that the bytecode is going to be compiled down into lower-level code.

So in a language like JRuby or Groovy, where you defer the type declaration, you don't even care what the type is, these languages certainly have a disadvantage. And the problem is there are two levels of optimization that happen in the Java code. The first optimization is the optimization that the Java compiler does itself, or any compiler does for you.

But the second level of optimization is the optimization done by the JIT compiler, the Just-in-time compiler. But because the Just-in-time compiler is not dealing with direct calls but a greater amount of calls, you get less optimization for dynamic-language code than you do for statically typed code.

So the dynamic languages start out with a greater disadvantage than static languages as of this moment, if you

-21-

will. And this is what Java 7 is trying to change and provide an equal playing field for dynamic languages as static languages do.

GLOVER:

And this is the invokedynamic ...

SUBRAMANIAM: That's correct. The fourth keyword that the bytecode level introduced, if you will, is the invokedynamic. So you have invokevirtual, invokespecial, and so on, now you have the invokedynamic.

And invokedynamic simply says, hey, let's go ahead and accept a call at compile time. Don't do a full type checking on it, accept it for what it is -- that it's an object whose type will be resolved later on -- and then during execution time let's do a method dispatching and then verify the type information at runtime and then dispatch the call appropriately.

So this makes the code really dynamic, but it also paves the way for optimization at the JIT compiler level. Because now these dynamic languages don't have to go through an intermediate level of bloated bytecode. They could directly call into an invokedynamic, and then they can enjoy the optimization that the JIT compiler can provide under the hood. >> This developerWorks interview continues in Part 2.

-22-

[END OF PART 1]

LANINGHAM:

This is Part 2 of Andy Glover's interview with

Venkat Subramaniam.

GLOVER:

Every time these two things come up, you have

one side that says dynamic typing is the way to go, and the other side says static typing is the way to go, dynamic is too dangerous.

What are your viewpoints on the two? Where do we stand these days? And, like I said, it was an argument we always had in the last five, six, seven years. Are we still having to answer this question? Are we still having this argument? Are there still two sides diametrically opposed?

SUBRAMANIAM: Absolutely. And I think we will continue to have that conversation.

When you ask that question, you reminded me of -- what was that story? -- Gulliver's Travels. If you remember, the Lilliputs were these small human-like beings and they were fighting constantly, and their battle was over whether to break the egg on the bigger end or the smaller end. So this is the programmer version, I think: whether you should be using the static languages or dynamic languages.

-23-

There was a time when I was only programming in dynamic languages, and I was adamant that dynamic languages were bad, they were evil, and it's harmful to program in those languages. Over the years I have completely reversed my opinion.

What I've realized over time is that the days that I hated are -- I should even use the word, I was fearful to program in dynamic languages -- are the days when there were several things that were missing that we have today.

If you go back about, oh, 10 years, maybe, maybe a little bit more, this was before the Internet. Some of the dynamic languages were fairly closed. They were not accessible and available to the greater public or the greater audience of programmers.

But today with the Internet and with the open source community, any person out there, it doesn't matter who they are, has the ability, a) to download, experiment, and play with almost any language that's out there, and, not only that, to go beyond that, b) to start participating and contributing to active development and extension of these languages.

And this is I think is incredible. If you really asked me 10 years ago, I couldn't have said this. But today it's a very

-24-

different world we live in where any programmer -- doesn't matter where they are and what their capabilities are, as long as they're willing and able -- they can participate in these language extensions and growth.

The second thing that's happened, I think, over the past about 10, 15 years is the power of computers. We have had only better, faster hardware, and so where we were fearful that dynamic languages were slower, the machine speed has greatly compensated for that.

And a third thing, I would say, which is probably the most important thing, in my opinion, is a greater awareness -may not be a greater acceptance, but a greater awareness -of test-driven development and the need for unit-testing code.

And the last point I want to emphasize: When I program in dynamic languages -- I'm not saying this isn't important in static languages, static-type languages, but it's even more important in dynamic-type languages -- is to have a greater discipline to write unit tests and ensure that the code doesn't do what you typed but the code actually does what you intended.

This, I think, is extremely important. But there's a greater awareness, on one hand, and I think the people who are

-25-

seriously using and benefiting and being successful in dynamic languages have embraced test-driven development.

So to answer your question, if I'm going to be a cowboy coder and I don't have discipline and I would only write code, my first answer is you should quit writing code, but the second answer is if you can't quit writing code, maybe a static language is better for you.

But if you are person who is passionate and you're a craftsman who takes responsibility for what you do and you have the discipline to write unit-tested code, then I think you have a fair chance of succeeding, both with static-type languages and dynamic-type languages.

And then the question is: then why should I use the dynamic type language? Well, a compiler certainly is useful. I don't want to discredit a compiler. It's got benefits. But I think a compiler's benefit is overstated and overrated.

While a compiler is certainly useful, if a particular problem I have on hand requires metaprogramming, then I would benefit a great deal by using a dynamic language. And by using a proper unit-testing tool and discipline, I actually write less code, and I achieve a lot more with dynamic-type languages than static-type languages.

-26-

Over time, what I've been convinced is that it is not an either/or. I think there are times when I do want to use a static-type language, and there are times when I do want to use a dynamic-type language, and I really feel good that I am not convincing myself to stick to one, and I have the power and the capability and the wisdom to pick and choose between the two of them even on the same project. So I'm not really convinced at this point that we should only use static languages or that we should only use dynamic languages.

GLOVER:

I hear you. Yes. In fact, maybe it was you that

said this, or someone else at a talk or a conference that I had the luxury of attending that, you were talking about the compiler and dynamic languages, and I think someone mentioned -- maybe it was even Stu Halloway -- said something to the effect that in a dynamic language your tests become the compiler, if you will, or give you that check.

SUBRAMANIAM: Right. In fact, Stu was saying that in about five years we will view the compiler as the weakest form of unit testing. And we're close to that five-year mark, I think, so we'll see how it goes.

GLOVER:

Certainly. You were talking about, hey, you

know, it's not an either/or, it's when do I use this and

-27-

when do I use that, and there are use cases for both. We're coming back to where we started this conversation with the whole notion of polyglot programming, and we've touched on Scala, we've mentioned JRuby, we've mentioned Groovy.

Those are pretty much the three we've talked about. Obviously there's Java in there. We've been talking about Scala and Groovy and JRuby now for at least five-plus years.

What else is out there? Are there other languages that people could start looking at? Are there other languages that are gaining mindshare, perhaps at the expense of something like Groovy or Scala? What do you see on the horizon, given that the JVM is now starting to adapt itself to be more friendly to these alternate languages? What do you see on the horizon?

SUBRAMANIAM: Over 200 languages on the JVM, and certainly some of the languages are a bit more esoteric than others. It's not as important, I would say, which language we pick. What is more important, I think, is which paradigm we pick.

And in terms of the paradigm itself, I think clearly we could say we have the object-oriented paradigm, we have functional programming, we've got the dynamic programming capabilities -- dynamic language capabilities, rather, not dynamic programming -- and then of course the elephant in

-28-

the room that we haven't talked about is logic programming - doing things like Prolog. I'm not aware of anything major happening in that area, but I would say it would be really interesting to see some of that. Maybe that's the next wave once we get through this, and then once we reach a certain amount of stability.

Certainly you remember this, Andy, back in, oh, I would say late '80s and early '90s we had the war going on if object-oriented programming was the right thing to do. It's so funny to even think about this back in time now, but group of people saying objective programming would suck, it would really not perform at all, performance would be so poor. And today, in 2010, we don't even question that. We use object-oriented programming languages even without thinking.

Maybe in the next about five years or so we would be using functional style more so and then -- or a mixture of functional and object-oriented style -- and then maybe we'll be ready to start learning and applying some of the logicprogramming capabilities and paradigms and benefit from that.

That's what is interesting to me is to start thinking about how maybe there are more things that we can benefit from that we haven't really taken the time to learn as an

-29-

industry and as being really available to a small community of people in the field.

So that's one thing I would say. Of course, you know, there's Clojure, a dynamic language which has a Lisp-like syntax. Clojure is receiving a lot of attention these days. Certainly that's a good language to look at. There are versions of Haskell available on the JVM. I'm also looking at variations of Erlang. I would certainly be interested in that when Erlang can run on the JVM. That would be really be excited to me.

I've been using Erlang for a few years, and I think that would be nice to have a dynamic language with as pure functional capabilities as Erlang be available on the JVM as well.

But I think more than languages -- languages often bring us the syntax, but what excites me more is not the language or the syntax but really the programming paradigms and the idioms they bring. That's what really catches my attention. What kind of idioms are out there and how can I learn them and how can I benefit and how can my applications benefit from those idioms? That's what I would be looking at.

GLOVER:

Certainly. We kind of glossed over this earlier

in the conversation when you mentioned idioms and the

-30-

different platforms and languages, how they approach concurrency, right? In a previous conversation with Alex Miller, who, again, is a friend of ours, he started talking about the different models for abstracting concurrency, whether it be actors, agents, threads, et cetera, et cetera.

To kind of continue that conversation, to pick your brain, where are we going with concurrency? Is it something that we as Java developers need to be further aware of, and how does it then relate back to languages like Groovy and Scala, et cetera, et cetera?

SUBRAMANIAM: The problem is this. The problem in Java is not that we cannot create threads. The problem in Java is that Java fundamentally promotes the paradigm of dealing with mutable state. It is going to take a lot more effort for us to promote immutability in Java.

When you look at the Effective Java book, one of the tips in their recommendation is that we should promote immutability as much as we can. But this is counter to the way we are used to in Java, for one, and also the language itself is really making things easier for us to deal with mutability than to deal with immutability.

I got this funny email. And I think this is funny, but I'm sure the guy who sent this email doesn't think so. His email

-31-

said he's got an application which is running fine, and they deployed the application on a multicore processor and then all hell broke loose.

And his email went on to say, don't tell me to do the right thing, because I already know what the right thing is and I cannot do it right now. So tell me what's the simplest way to make this application run fine on this multicore processor.

And I replied to him saying, well, the easiest thing to do is to go in and turn off one of the cores. Why is it a program behaves correctly on a single core but ends up misbehaving on a multicore? Well, the first thing is the chances are the program was not behaving correctly in the first place on a single core; it's just that it pretended to be behaving correctly on a single core.

GLOVER:

Right, right.

SUBRAMANIAM: You know, it's like driving on a one-way street in the opposite direction at 3:00 in the morning and then claiming that's a good thing to do, right?

So what happens on a single-core processor is the multiple threads generally share the same cache. There are two places where the problem creeps in. One is the JIT compiler

-32-

optimization, where the JIT compiler says, hey, I really don't need to be putting this value back into the memory, I can simply keep this in the cache. I don't need to push this into the memory at all.

And so this is the issue of viability. A great book to read is Brian Goetz's Java Concurrency in Practice book. And he talks about where the visibility can become a huge concern.

The second area of concern comes in even beyond the JIT compiler optimization, is where the caches ... the locality of the data at any given time. When multiple threads are running in a core in a single processor, they may end up using the same cache, so even though the application is broken, it may not really reveal the problem to you on a single-core processor.

But when you're a multicore processor, imagine this for a minute, that we have a huge conference room, and imagine for a minute that you and I are sitting on one table and I am looking at some data that's on the wall, so I walk up to the wall, copy the data on a Post-it note, and I bring it and put it on my table, and I'm changing it on my table, but I'm too lazy to go back and put it on the wall now.

But because you're sitting at the same table as I am, you could be editing and changing and viewing the Post-it note I

-33-

have on my table, so you and I are sharing the same data. Imagine you and I are two different threads in this case.

But in the meantime somebody is sitting across the room at another table is going to be looking at a different copy of the same data, and the changes they are making are not visible to us, and the changes we are making are not visible to them.

This is kind of what happens on a multicore processor. When you're dealing with a multicore processor, there are different caches that different cores end up using, and so depending on the locality of the threads, they may actually be looking at different caches in the memory. And as a result, the probability of your program misbehaving is a great deal if it is not written fairly well.

That is why this problem is even more dire in a multicore processor than in single-core processor. All of a sudden the code that was supposedly working properly for years explodes and doesn't work anymore when deployed in a multicore processor.

So the fundamental problem here to address is not to take the data and synchronize it. Here is the problem with synchronization, right?

-34-

If you take an object and synchronize it -- and I'm sure you have done this before -- you write code and you use synchronization, then you start driving home, and then you realize on the way home maybe you didn't synchronize this properly, and it messes up the rest of your evening, because you realize maybe you have not protected your code properly and you are really worried what's going to happen to the state of the object when multiple threads access it. Then you rush back to work and you take a look at this code and you realize this is broken.

The problem in Java is not that we don't have the ability to synchronize. The problem in Java is synchronization works only if you have synchronized the code properly in every single place where you use it. For the program to be wrong, you just need to mess up in one place. But for the program to be correct, every place where you access the data has to be protected properly.

Over time I've realized two things. One is to get the code correct for multithreading in Java requires divine capability; a mortal human being like me cannot just do it, I think. And what's even worse is when you make a mistake, Java doesn't tell you that you made a mistake.

It doesn't call Andy and say, Andy, your code sucks. Or it doesn't call Venkat and say, Venkat, your code sucks. It

-35-

probably calls other JVMs and tells that look at the code that Venkat wrote, right? So I've realized that getting multithreaded code working in Java is like working with a mother-in-law. It's just waiting for you to fail. And we just can't afford that.

Now, there are two distinct ways to solve this problem, I think. One way is to promote immutability and say, you know what, I'm not going to change anything, period. And every data that you deal with is immutable, and because it's immutable there is no need to synchronize it.

That solves the problem of contention, but then how do I communicate between threads and exchange the data, and then that's where we could use the actor-based model that Erlang and Scala promote. We could have actors and we communicate between these actors by passing these data back and forth, but the data being immutable, it is quite safe. That's one approach.

The other approach --which you can also do, by the way, in language like Scala and Clojure -- but a different approach is ... just entertain this thought for a minute. Imagine for a minute that in Java you synchronize and access the object, but then you forget to synchronize or you fail to synchronize. And rather than quietly misbehaving, Java calls out and says, hey, you, you forgot to synchronize this code,

-36-

so I'm angry at you, and throws an exception at you. Just imagine that for a minute.

GLOVER:

It would be helpful.

SUBRAMANIAM: It would be really helpful. Well, that exactly is the STM, the software transactional memory model. So what the software transactional memory model does is it binds your access to the data within a transaction boundary, just like how your databases provide you ACID capability -atomicity, consistency, isolation, and durability. It provides you the capability while you can't have durability of data in memory, so it takes care of the atomicity, the consistency, and the isolation.

And so you mark your access within a transaction boundary. If you try to touch a data for change outside of this transaction boundary, immediately you get an exception. But if you started a transaction for this operation and then when you finish up your transaction, it's kind of like your optimistic locking. At the end of this, if the data that you're trying to modify had been changed, then it repeats your transaction.

Now, it may be a bit scary to know it repeats your transaction, but it turns out this actually works extremely well when you have a single writer or an infrequent writer

-37-

and a large number of readers.

In applications where you have infrequent writes and extremely frequent reads, this model works fairly well. And that's why the software transactional memory in Clojure can be a pretty effective model.

Rather than focusing on these as language capabilities, let's take a minute and look at these as paradigms. You could solve concurrency in one of three ways. The poorest way to solve concurrency is shared mutability, which is the synchronize model.

A better way to do it is an actor-based model to communicate immutable data across threads. And a third option, when you have infrequent writes and frequent reads, is to have a transactional software memory.

So what I'm thinking is maybe in the next two to three years, maybe four years, we won't be worried about languages as much when it comes to concurrency, but we will be more interested in the paradigm of how to solve concurrency. And then we can reach out into libraries and then make use of these in almost any language on the JVM.

It doesn't matter whether you're programming in Java, Groovy, JRuby, Clojure, or Scala or name your language of

-38-

choice, but you're going to say here are three different models and I want to pick and choose what makes the most sense, and I'm hoping you would lean more towards the actor-based model for the most part, and then in special cases you would lean towards the software transactional memory model as that makes a lot more sense in the case of frequent reads and infrequent writes, I think.

GLOVER:

I think that's just a beautiful way of putting

it in terms of when it comes to the concurrency concerns: don't so much focus on a language but focus on, like you said, a paradigm or a library or a type of how you do it, and then, like you also mentioned earlier, because everything on the JVM is "interoperable," if you can find an STM library that was written in Java, there's no stopping you, then, from using that library in Groovy should you decide that you need some aspect of Groovy in coordination with STM or, like you said, actors. Obviously it's built into Scala, but there are actor libraries for Java that you could then leverage in JRuby should you want to.

And I think that again underscores the beauty of the JVM as it stands now. As you mentioned, 10 years ago when you and I were hacking away at code on the JVM, we ended up writing a lot of our own stuff. I still remember writing my own logging framework, and I want to shoot myself: why'd I waste so much time doing that, right? I still remember writing my

-39-

own test frameworks back then.

And now, all those are done, right? There are smarter people -- way smarter people than me, tremendous amount of people smarter than me -- that have written excellent libraries for A, B, C, and D, and now I can use them all and ultimately I can build solutions tremendously quickly by assembling all these pieces.

So I think you've underscored that just beautifully in terms of now we have the JVM and we can pick whether we want to fly or drive, and then when we decide we want to fly, we also can pick on what type of airplane we want to fly and where we want to sit. I think this is a very exciting time for the Java platform. I'm sure you would agree with that.

SUBRAMANIAM: Oh, absolutely. This is the power to the programmers. That's been the strength of the Java platform all along in terms of picking the frameworks and the libraries. Now that flexibility and power is available in picking a greater programming paradigm as well.

So I think this is definitely exciting. It gives me a lot more excitement, strength, and energy to go solve more problems on the virtual machine. I think this is an excellent time for programmers to be on this platform.

-40-

GLOVER:

Well, Venkat, I know I speak for everyone

listening when we say thanks for spending this time with us and elaborating on all these different ideas and opinions and languages.

Let me ask you real quick. Let me summarize. We've covered Scala, we've covered Groovy, we covered Clojure when we covered JRuby. Ted Neward, a friend of both of ours, has written a number of articles on Scala for developerWorks. We covered Groovy, another joint friend of ours or common friend, Scott Davis, has written quite a few articles on Groovy for developerWorks. I also wrote a few myself.

Clojure, there are certainly some Clojure articles on developerWorks and there was a podcast that I had the opportunity to speak with Stu Halloway, another friend of ours. Not to be left out, I also had a great conversation with Alex Miller, and he talked about some concurrency models and whatnot, and he also wrote an article for developerWorks.

You've written four books but two of them we covered, hit the nail on the head in terms of you wrote Programming Scala for the Pragmatic Programmers, and you also wrote Programming Groovy for the Pragmatic Programmers.

Where else do you think developers can go to learn more

-41-

information? What would you recommend? We've covered a whole lot of things, and I'm sure a lot of developers are sitting here saying: This is awesome; where do I start? Do you have any further recommendations?

SUBRAMANIAM: Reading books is certainly a good idea. Being an author I would say that. One of the things I do is on any particular topic I don't pick and read one book. I probably pick and read six or seven books or as many as are available.

And then the reason simply is it's not that this one author that I've read is not good enough. It is getting a different perspective. It's getting different views on a topic. Also, when I start reading a book on a topic that I may be somewhat familiar with already, it gives me an opportunity to think about certain things a little differently. So certainly I would recommend reading quite a few books.

The other thing of course, as you mentioned, is a number of wonderful articles. Searching on the web and finding these articles is a great way. This kind of goes back to my other book, Practices of an Agile Developer. One of the things that we recommended in that book is to leverage the teams that you work with.

And in terms of that, I want to -- this nothing new, but a

-42-

lot of times we tend to ignore it -- is, first of all, within a company your best learning opportunity is among your own colleagues.

So as an enterprise programmer, if you're working in a company, find two, three, maybe four people. Don't make this into a 50, 60 people, but into a very small group. Once a week, pick Wednesday afternoon, for example, lunchtime, a quick brown-bag session or a lunch learning session.

This has been extremely valuable even when I used to work for companies, is to have these brown-bag and lunch learning sessions. You could pick a particular book. I've had e-mails from people from companies saying, hey, we picked up your book this week or this month or this quarter, and we're going to go through one chapter at a time every week.

And that's a great opportunity. Somebody comes to the table and they write some examples, show what they have learned along the way. That's a great way to learn from each other.

And the next thing I cannot overemphasize is going to your local Java user group. Java user groups are an invaluable source. And I have enjoyed visiting user groups whenever I can. With my travel schedule these days, it's hard for me to go to a user group, but that's been something that's been valuable to me over the years.

-43-

I would, once again, remind people, if you have not had a chance to go to your user groups, please do take the time. It's an investment. And user groups are great because you don't have to pay a dime to attend and they give you free pizza to eat. What else could be made easier, right? So it's a great opportunity, I think.

So, again, reading books, reading articles is a good thing, but I tend to learn ... One of the reasons I go to conferences, Andy, is I get to hang out with smart guys like you and Stuart Halloway and Scott Davis and Neal Ford, and we mentioned a lot of these people.

And that's the selfish motive is I learn by listening to these guys a lot. And even in the things I think I may know, but I sit there and we start a conversation and things come to your mind. And I have tried and tested a lot of my thoughts with the groups of people that I interact with. I'll throw some idea at people and see if they knock it down or they appreciate it.

The community value is extremely important. If you're a programmer in a city, wherever you are, you should be able to know who are the smart programmers in your community. And a great way to do that is to attend local user groups.

-44-

And then local conferences that you may have are great resources. And here at a No Fluff Just Stuff conference, and certainly it's a great conference to attend. And there are other conferences as well, but find a conference that has no marketing and high-technical content where you can spend your time wisely.

I can't emphasize again how much I learn from the community, from other people. The friends that I have are the friends from which I learn the most. And I think, again, different people have different learning styles. That's my personal learning style is learning from other people.

GLOVER:

I couldn't agree more, Venkat. And as a quick

plug to kind of give a success story from that learning from other people: easyb, a framework that I'm intimately familiar with and connected to, was born out of hanging out with you, Venkat, and Jeff Brown, who is a committer and developer on the Groovy platform, specifically Grails.

But both you and Jeff were instrumental in guiding and actually producing in a very, very early version of easyb, which I selfishly took it and ran with it. But it was opportunities like that that I'll never forget, and I'm sure there are thousands of programmers that can tell similar stories of having sat down with someone like you and all of a sudden learned something new and took it and ran with it.

-45-

So thank you very, very much for giving us all this advice and talking extensively about Scala and Groovy and where we can learn more. And obviously the whole concurrency conversation is fascinating; functional programming.

I've certainly enjoyed this conversation, Venkat, so I say once again, thank you for myself and all our listeners for your time today.

SUBRAMANIAM: Hey, thank you for having me, Andy.

LANINGHAM:

That was Andy Glover, and this is the

developerWorks podcast. Follow us on iTunes and at ibm.com/developerworks. And specifically keep track of Andy's interview series at ibm.com/developerworks/Java. Thanks for listening.

[END OF SEGMENT]

-46-