True, reflection is slower than regular access - though JOS does a fair bit of caching to avoid some of the cost (for when you serialize many instances of the same class). I think your experience with rewriting as Externalization shows that JOS is one of those performance vs. ease trade-offs.
BTW: Are you sure that no-arg constructors and un-finaling fields makes a significant performance difference? The only instance I've come across for this showed the (surprising) result that JOS's setting final fields is actually faster than reflective setting of non-final fields - but I haven't profiled that explicitly. I would expect deserialization with a no-arg to be slower, because then it has to call the constructor in addition to actually creating the object (allocating memory etc). But I haven't profiled this either.
You can customize the initialization of a class by creating a readObject() method for it, which is a sort of extra-linguistic constructor. It is confusingly named the same as the method that you call to start deserialization - but here it is a callback method that you write, that JOS itself will call:
private void readObject(ObjectInputStream ois)
throws IOException, ClassNotFoundException {
ois.defaultReadObject(); // "default" deserialization
// ... your initialization here ...
}
You can also read data from the stream explicitly, and set the class fields yourself - if so, you need to write a corresponding writeObject() method that writes the data to the stream. By avoiding reflection, this should be faster than the "default" (but again I haven't profiled this). If you don't provide a readObject, it will call defaultReadObject() for you by, you know, default. Same for writeObject().
There's also hooks for validating the object graph after all of it has been deserialized; and for replacing (resolving) one object with another.
I know something of JOS, but not much of its performance, which seems to be your crucial requirement. So I might not be much help, but let me know if you'd like more info or if I've misunderstood something (I might not reply til tomorrow - it's 3am here).
Ahh, I see where the confusion comes from. I meant initializing in the Ruby way...which is analogous to construction. There's no way for you to specify how to construct the object being deserialized from the stream, and so you need a no-arg version of the constructor (or can allow JOS to generate one for you) and have to do the logic that would be in-constructor in a separate piece of code like readObject. And in my latest experiments with Externalizable, the JOS code being unable to construct objects the way I want it to has become the latest bottleneck.
It also interferes with us serializing objects for which we do want to have final fields initialized on construction. If we want to avoid the reflective construction, we need a no-arg constructor. To have a no-arg constructor, we can't make important fields final. And if we want a particular value to be passed into the constructor, we always need to do it in readObject, without any context provided as to where and when serialization is being called. That's so cumbersome that we simply can't do it.
To check I understand: you need to deserialize objects, and you also want to pass in some arguments to some of those objects, so that their initialization is based on both the serialized data and the arguments? Is that to do with currying (is currying in Ruby?) In a complex object graph, how do you know which arguments should go to which objects?
You could use a global object to hold these arguments (or, subclass Thread, and associate the data with that, and yield while the deserialization runs in that thread). Ugly, yes.
BTW: it is possible to set final fields outside a constructor, using some black hacking (by accessing the same hooks that JOS uses internally). A few serialization tools use this, and I recall a project that consisted entirely of providing nicer access to these hooks (I searched for it, but couldn't find it).
BTW: There's a super-fast approach to serialization involving pointer twiddling: you save the raw memory, and when you load it back, just adjust the pointers for its new location (heard from Andy Hunt).
This doesn't help you if you want to serialize in order to inspect the serialized form; and it throws away Java's lispy symbolic bindings; and it would have to be implemented within the JVM - but it is fast.
That's exactly the sort of thing I was hoping to see out of the JVM's default serialization. As it stands I think I'm going to basically have to define a custom marshaling format and hand-serialize everything for the domain I'm interested in (which, btw, is serializing pre-parsed Ruby code to disk so it can be loaded later without parse time...done to avoid the slowness of our parser during the cold first seconds of the JVM!).
Writing a de/serializer is pretty straightforward, the only slightly tricky bit is handling graphs (you record each object, associated with a label; when you encounter it again, you serialize a ref symbol to its label). It sounds like a whole project, but it's only for your specific use, it wouldn't be that much work, and definitely be worth it.
But hand-serializing everything is a serious grind. Would you use a code generator (or use BCEL or similar) for the default case? I bet the vast majority of cases are automatable, and for the other cases it's a starting point.
For cold start, I personally like the idea of reusing a JVM. It would be nice if it just had a reset or clear button. :-)
BTW: Even though serialization is just a tiny part of your project, you are right on top of it - I'm a little awed.
Thanks for this discussion, you helped inspire me to write my first Ruby program tonight (a regular expression engine) - it felt much lighter-weight than Java (but it's a bit difficult to judge, because it's a problem I'd previously worked out how to implement in Java).
BTW: Are you sure that no-arg constructors and un-finaling fields makes a significant performance difference? The only instance I've come across for this showed the (surprising) result that JOS's setting final fields is actually faster than reflective setting of non-final fields - but I haven't profiled that explicitly. I would expect deserialization with a no-arg to be slower, because then it has to call the constructor in addition to actually creating the object (allocating memory etc). But I haven't profiled this either.
You can customize the initialization of a class by creating a readObject() method for it, which is a sort of extra-linguistic constructor. It is confusingly named the same as the method that you call to start deserialization - but here it is a callback method that you write, that JOS itself will call:
You can also read data from the stream explicitly, and set the class fields yourself - if so, you need to write a corresponding writeObject() method that writes the data to the stream. By avoiding reflection, this should be faster than the "default" (but again I haven't profiled this). If you don't provide a readObject, it will call defaultReadObject() for you by, you know, default. Same for writeObject().Googling for the specific methods turns up many tutorials. The following docs are for serialization in Java 1.3, but these "basics" haven't changed: for readObject() http://java.sun.com/j2se/1.3/docs/guide/serialization/spec/i... (section 3.1 has the API, with defaultReadObject() in it). Here's the overall contents page: http://java.sun.com/j2se/1.3/docs/guide/serialization/spec/s...
There's also hooks for validating the object graph after all of it has been deserialized; and for replacing (resolving) one object with another.
I know something of JOS, but not much of its performance, which seems to be your crucial requirement. So I might not be much help, but let me know if you'd like more info or if I've misunderstood something (I might not reply til tomorrow - it's 3am here).