Saturday, April 2, 2011

Anonymous Functions and Type Erasure in Scala

Before I get into anything, I would like to make it clear that I'm using Scala 2.7.7.final, not the newest (and significantly different) 2.8.1.final.  I'm using for my thesis, and I learned the hard way long ago that you should try to avoid changing your toolset once your underway.

Recently, I've started to really get into the use of anonymous functions.  I'm writing my own higher-order functions, and I like the simplicity of everything.  They can be used as alternatives to a number of design patterns, including Template Method, and tend to have a lot less superfluous syntax associated with them.

However, today I encountered a bit of an issue.  I wrote two higher order functions in the same class whose type signatures differed only in the types used for the anonymous functions.  I was greeted with a double definition error message.  What?

So I delved a bit into the implementation of anonymous functions.  The Scala library defines a series of traits called Function0 through Function22.  These appear to refer to anonymous functions, where the number refers to the number of parameters the function takes.

First thing's first: let's see what happens when you have more than 22 parameters, because that sounds like it'd be entertaining.  So...
scala> ( ( a: Int, b: Int, c: Int, d: Int, e: Int, f: Int, g: Int, h: Int, i: Int, j: Int, k: Int, l: Int, m: Int, n: Int, o: Int, p: Int, q: Int, r: Int, s: Int, t: Int, u: Int, v: Int, w: Int ) => "hahaha" )
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 23
    at scala.tools.nsc.typechecker.Typers$Typer.decompose$2(Typers.scala:1502)
    at scala.tools.nsc.typechecker.Typers$Typer.typedFunction(Typers.scala:1504)
    at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:3153)
    at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:3358)
    at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:3406)
    at scala.tools.nsc.typechecker.Typers$Typer.computeType(Typers.scala:3457)
    at scala.tools.nsc.typechecker.Namers$Namer.typeSig(Namers.scala:859)
    at scala.tools.nsc.typechecker.Namers$Namer$$anonfun$typeCompleter$1.apply(Namers.scala:415)
    at scala.tools.nsc.typechecker.Namers$Namer$$anonfun$typeCompleter$1.apply(Namers.scala:413)
    at scala.tools.nsc.typechecker.Namers$$anon$1.complete(Namers.scala:982)
    at scala.tools.nsc.symtab.Symbols$Symbol.info(Symbols.scala:555)
    at scala.tools.nsc.symtab.Symbols$Symbol.initialize(Symbols.scala:669)
    at scala.tools.nsc.typechecker.Typers$Typer.addGetterSetter(Typers.scala:1139)
    at scala.tools.nsc.typechecker.Typers$Typer$$anonfun$10.apply(Typers.scala:1219)
    at scala.tools.nsc.typechecker.Typers$Typer$$anonfun$10.apply(Typers.scala:1219)
    at scala.List.flatMap(List.scala:1132)
    at scala.tools.nsc.typechecker.Typers$Typer.typedTemplate(Typers.scala:1219)
    at scala.tools.nsc.typechecker.Typers$Typer.typedModuleDef(Typers.scala:1114)
    at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:3091)
    at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:3358)
    at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:3395)
    at scala.tools.nsc.typechecker.Typers$Typer.typedStat$1(Typers.scala:1598)
    at scala.tools.nsc.typechecker.Typers$Typer$$anonfun$19.apply(Typers.scala:1643)
    at scala.tools.nsc.typechecker.Typers$Typer$$anonfun$19.apply(Typers.scala:1643)
    at scala.List$.loop$1(List.scala:300)
    at scala.List$.mapConserve(List.scala:317)
    at scala.tools.nsc.typechecker.Typers$Typer.typedStats(Typers.scala:1643)
    at scala.tools.nsc.typechecker.Typers$Typer.typedTemplate(Typers.scala:1221)
    at scala.tools.nsc.typechecker.Typers$Typer.typedModuleDef(Typers.scala:1114)
    at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:3091)
    at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:3358)
    at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:3395)
    at scala.tools.nsc.typechecker.Typers$Typer.typedStat$1(Typers.scala:1598)
    at scala.tools.nsc.typechecker.Typers$Typer$$anonfun$19.apply(Typers.scala:1643)
    at scala.tools.nsc.typechecker.Typers$Typer$$anonfun$19.apply(Typers.scala:1643)
    at scala.List$.loop$1(List.scala:300)
    at scala.List$.mapConserve(List.scala:317)
    at scala.tools.nsc.typechecker.Typers$Typer.typedStats(Typers.scala:1643)
    at scala.tools.nsc.typechecker.Typers$Typer.typedTemplate(Typers.scala:1221)
    at scala.tools.nsc.typechecker.Typers$Typer.typedModuleDef(Typers.scala:1114)
    at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:3091)
    at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:3358)
    at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:3395)
    at scala.tools.nsc.typechecker.Typers$Typer.typedStat$1(Typers.scala:1598)
    at scala.tools.nsc.typechecker.Typers$Typer$$anonfun$19.apply(Typers.scala:1643)
    at scala.tools.nsc.typechecker.Typers$Typer$$anonfun$19.apply(Typers.scala:1643)
    at scala.List$.loop$1(List.scala:300)
    at scala.List$.mapConserve(List.scala:317)
    at scala.tools.nsc.typechecker.Typers$Typer.typedStats(Typers.scala:1643)
    at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:3084)
    at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:3358)
    at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:3395)
    at scala.tools.nsc.typechecker.Analyzer$typerFactory$$anon$2.apply(Analyzer.scala:41)
    at scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:267)
    at scala.tools.nsc.Global$GlobalPhase$$anonfun$run$1.apply(Global.scala:246)
    at scala.tools.nsc.Global$GlobalPhase$$anonfun$run$1.apply(Global.scala:246)
    at scala.Iterator$class.foreach(Iterator.scala:414)
    at scala.collection.mutable.ListBuffer$$anon$1.foreach(ListBuffer.scala:266)
    at scala.tools.nsc.Global$GlobalPhase.run(Global.scala:246)
    at scala.tools.nsc.Global$Run.compileSources(Global.scala:574)
    at scala.tools.nsc.Interpreter$Request.compile(Interpreter.scala:820)
    at scala.tools.nsc.Interpreter.interpret(Interpreter.scala:505)
    at scala.tools.nsc.Interpreter.interpret(Interpreter.scala:494)
    at scala.tools.nsc.InterpreterLoop.interpretStartingWith(InterpreterLoop.scala:242)
    at scala.tools.nsc.InterpreterLoop.command(InterpreterLoop.scala:230)
    at scala.tools.nsc.InterpreterLoop.repl(InterpreterLoop.scala:142)
    at scala.tools.nsc.InterpreterLoop.main(InterpreterLoop.scala:298)
    at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:141)
    at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)
 


This crashed the REPL, sending me back to the command line.  Yup, entertaining.  (If you need more than 22 parameters, or any number of parameters even close to 22 for that matter, you shouldn't be allowed within 500 feet of a computer.)


Back to actual work.  I constructed the simplest example that showed the same error:
class MyClass {
  def method( fun: Int => Int ) {}
  def method( fun: Double => Double ) {}
}


...which fails at the REPL with...
<console>:6: error: double definition:
method method:((Double) => Double)Unit and
method method:((Int) => Int)Unit at line 5
have same type after erasure: (Function1)Unit
       def method( fun: Double => Double ) {}
           ^


Type erasure.  My old nemesis.  These traits do their magic with generic types.  The syntax Int => Int is really just syntactic sugar for Function1[Int,Int], which explains the type erasure error.  Ho hum.

But wait.  Here's something that does work:
class MyClass {
  def method( fun: Int => Int ) = fun( 0 )
  def method( fun: Double => Double ) = fun( 1.0 )
}

...what?  The only change from the nonfunctional version is that the return types have changed from Unit to Int and Double, respectively.  Although the parameters are specified with generics, the return types (based on those parameters) are not.  However, this still shouldn't matter, as the return type doesn't get factored into the signature.  After all, one can simply not have anything grab the returned value, in which case there is no way to extract return type information.


I probed into this a little further.  Calling getDeclaredMethods on MyClass's class gets two relevant methods with the following signatures:
public double MyClass.method(scala.Function1)
public int MyClass.method(scala.Function1)


Ok, now I'm really confused.  It doesn't appear that there is any additional magic that Scala's performing behind the scene.  So the next move was to get the Scala compiler out of the picture, and do this in Java with:
 import scala.*;

public class Test {
    public int method( Function1< Integer, Integer > fun ) {
        return 0;
    }
    public double method( Function1< Double, Double > fun ) {
        return 1.0;
    }
}



This compiles.  What?  What?  So the next move was to get Scala out entirely.  Two classes were needed:

public class Pair< T, U > {
    public T first;
    public U second;
}

public class Test2 {
    public int method( Pair< Integer, Integer > pair ) {
        return 0;
    }
    public double method( Pair< Double, Double > pair ) {
        return 1.0;
    }
}
This compiles.  However, if you change the return types to be the same, it doesn't.  I'm actually at a loss at this point.  I went back and read a number of basic materials from Oracle, including materials on type erasure and method overloading (note that the method overloading information includes type signature information).  Based on the documentation, this shouldn't work.  If anyone has any insights, that would be great.  I posted the reason why in my next post here.  Tricky, tricky!

No comments:

Post a Comment