Somewhat similarly, as a stopgap in '07 I built a Django (well really, Python and duct tape) Sarbanes-Oxley SOD system over a month or so.
I had to deal with well over a million records, but fundamentally it was a pretty simple ask, laying a 2d matrix of rules over user and permission records from 30 different systems. I even hooked it up to SAP and various AS400s through MQ, print queues and various other things.
It was a total stopgap move - so a whole bunch of vendors bid on the "real" system, one was selected, and now almost 3 years later and a few million dollars spent, and the "real" system still isn't working.
"Web 2.0" frameworks have a lot of applications in business, the ORM + presentation layer tools give you most of the groundwork you need, but as the article says, the key is in the polish and the business oriented design.
If they have been using your system for 3 years, you might want to bid ~500k for a ~6 months contract to get it to the next level. It might not be fun but you can make crazy money building custom enterprise applications once you have some credibility.
I would absolutely consider doing that if I had time. I'm kinda all-in on my own startup at the moment and I'm doing my level best to not be distracted by contracting.
And oh how right you are about it not being especially fun! You make a good point about the sort of money that I should chase to do the job properly, too.
I have recently begun a similar project: I work for a private bank and am rebuilding our client website. The main system (which is used internally for booking transactions and managing portfolios etc.) runs on an Oracle DBMS, so it isn't really legacy, but it does have a goulash data schema that is convoluted and slow. I'm using Rails for the site, which of course assumes certain conventions, but I've actually found it easy to loosely couple the site to the old system (by using views to extract the relevant data plus a little extra Ruby code). Most people in my dept haven't even heard of RoR but I've already had a positive reaction to the proof of concept which blows the old site out of the water and was built in less than a week.
The UI advice in the article hits the nail on the head. When dealing with legacy systems, smart use of aggressive caching and AJAX (loading icons etc.) can help improve user experience without any changes to the underlying system.
Although I don't work with COBOL, I do work at a financial services company and we have a lot of old software that we are not going to replace with Ruby, PHP, or any other dynamic language because the precision and accuracy of decimals and floats in dynamic languages is inadequate. I'm really surprised that the precision issue gets ignored at the native numeric level within dynamic languages.
It isn't ignored. While I don't use Ruby or PHP, Lua and Python both have libraries for high-precision numeric computing and rationals, and the Scheme standard requires the full numeric tower. Erlang also uses them by default. OCaml has arbitrary-precision numbers in the core language, though (somewhat characteristically) doesn't use them by default for performance reasons.
Ruby bundles Bignum and BigDecimal libraries which provide arbitrary-precision math primitives in the standard library. Python similarly provides the 'decimal' module. Most other "dynamic" languages provided at minimum bindings to the GMP library.
Basically, you seem to be considering only the fast, C-based math primitives most languages offer by default, and ignoring the more powerful options available to those who need them.
And, in fact, Rails translates NUMERIC fields in the database to BigDecimal values in Ruby. (You still need to be careful, though that you don't inadvertently coerce a BigDecimal to a Float in the course of working with it.)
I haven't gotten very far with this issue in Ruby, however in PHP there is also a way to get around the lack of precision by treating numbers as strings and then using additional libraries to perform the necessary calculations. The ability to do this in a clear and concise manner and be absolutely certain that everything is adding up properly is not easy.
From my perspective, the fact that these calculations do not perform correctly right out of the box using dynamic languages is the problem, and adding additional libraries is not the solution.
The last project we did with money, we just treated all amounts as integers storing the number of cents. No accuracy problems w/ whole integers, so 1000 because $10.00. Easy to do math with, and keeps it's accuracy.
Might not work for hard-core financial work, since the number of decimals might be too deep for that to be efficient. (although there's nothing stopping you from saying the last 5 digits of every number is the decimal part).
A better thing to do is to not use raw integers to represent money at all, create an immutable money object, use integers to represent units and a currency string (cents/USD) inside the class. Use the money as the primitive it is, giving it the appropriate numerical operations added/subtracted/multiplied/allocated. Money is not an integer, or a decimal, or a double, or any other number, Money is Money and should be represented by an appropriate abstraction of its own.
As a bonus, this strategy also makes it (relatively) simple to handle multiple currencies. By using raw integers, you never have currency information attached so it's a lot tougher (and messier) to keep track of the currency.
> A better thing to do is to not use raw integers to represent money at all...Money is Money and should be represented by an appropriate abstraction of its own.
Good point. This makes me wonder though whether it is ever acceptable to use primitive math without some kind of units, except maybe in pure math. You are always counting something, be it dollars, or people, or widgets sold, or hard drive seeks.
I would depend on whether adding different units together was an error. 1 dollar + 1 euro should be an error. So raw numbers for pure counting is OK, but raw numbers for counting units of a specific measure are not OK, an abstraction should be introduced. If you just willy nilly add feet and meters you might end up missing your desired orbit and losing a billion dollar satellite.
I agree, and while the project we were on didn't have strong typing of money, what I described was the database storage of the numbers. We did very few calculations, and the system was explicitly USD, so going deep into "proper" handling of money was just unnecessary.
I really do like using the type system to keep track of units (type system either in the object oriented sense, or the haskell type system). This is similar to Joel's old article about using a naming scheme for dirty vs. sanitized strings for the web. It's all a type system, just his was in a language that didn't support language-level types.
You could work with Common Lisp. It can handle rationals, arbitrary precision, it has macros, useful debugging tools, solid environments, it compiles code on a function-by-function basis to highly performant bytecode (or machine code or C, depending on which implementation you're using).
I don't know, all the cool shit you can do with Ruby, PHP, etc. can be done with Common Lisp and you won't suffer from shitty performance or inadequate meta-programming facilities.
I get your point, but that is essentially what my company does.
We primarily use CL for bank software (not just the website, but for payment processing/fraud detection, trading algorithms, etc.) We've also used it for occupational medicine software, PEO management systems, pension administration systems, etc. [I'm largely using the royal "we" - essentially none of my woefully bad code ever makes it into a shipping product. But signing the checks gives me some sense of involvement.]
CL is a fiddly mess to get started with, and there is not as much open-source code available, but once you get going it is a very powerful tool. If you're going to make the ecosystem argument, why did you stop at Rails? Don't you develop everything in Java? It has more of an ecosystem, so surely you've abandoned those wheel-reinventing Railheads for Javaland? Or PHPistan? Or C++ville?
I don't necessarily disagree with what you are saying. My day job doesn't either, which is why they do indeed develop almost everything in Java.
The thing is, for my business, Ruby (via Rails and friends) was ready for production without me having to play system architect, a role that I am manifestly unsuited for. It may not be ready for different projects but, well, I have little worry about my capability to implement projects which I do not actually have to implement.
There was no fiddly messing -- two years ago when I launched my site it was just "code code, launch", the framework has gotten better (without my intervention!) over the past two years, and when I upgraded it recently from "a mostly static site with a half-baked CMS up its sleeves" to a full-blown, users actually banging on it as we speak webapp, it required fairly little pain and delivered something which exceeded my expectations. Time spent on programming in Rails reliably delivers features my users can use.
I can write Common Lisp, in a very very theoretical sense (much like my ability to write Ruby two years ago). Its just, well, I think if I started to play "find that library" I'd discover a lot of pain points. Ruby had ActiveRecord, which is a nice safe choice for an ORM that I can rely on meeting my very modest needs going forward. Common Lisp has a wealth of choices but no clear winners, and I am disturbingly certain that if I bring that up I'll be told "Yeah, but writing your own not-quite-a-database is pretty fast".
(I mean no slight against the author of a certain site which is locally popular but since I am neither a particularly good programmer nor independently wealthy any project which includes "write your own in-memory database!" fills me with existential dread. I would rather spend my limited budget of time delivering features that users will pay for. Writing databases is worth great amounts of geek cred but merely being able to persist data does not convince my users to pay me money -- it is a prerequisite, yes, but it isn't the value add or anything close to it.)
I also understand where you're coming from...we actually use Django (or other Python frameworks) for most of the simpler sites we have to do. CL can be both painful and overkill when tackling certain types of projects. We haven't built a blog engine or standard CMS in CL, for example, because there are plenty of other polished contenders. But we don't do many simple websites/projects...and CL really comes into its own when you have a more complex project to tackle. Quite a bit of what we do would be characterized as "enterprise" software with a web interface. The 'weird' CL approach (DSLs, bottom-up, context-oriented, etc.), when fully embraced, CAN provide the fabled competitive advantage. But we try to use CL only where it is wise to do so, and we don't reinvent wheels needlessly. We haven't had to write our own database, for example...we usually use Postgres. We've used other databases and 'not-quite-a-databases', but we've never had to write our own. Most of the guts of our software is written in CL, but we have called out to Java, Python, C/C++, and Erlang libraries, as well. CL is very flexible, and when there is not a decent existing library for something it is pretty easy to Borg another codebase. But you'd be surprised how much decent code in CL is already out there.
Oddly enough, I got into CL because it was basically the only language that clicked for me. I'm the last guy on earth that would be a Lisp snob, or who is going for geek cred or technical wizardry. I think you should use what works for you. I largely use Lisp because I'm self-taught and I suck as a programmer. No joke. When I was "language shopping," Lisp seemed like a loser because of the lack of libraries. But I'm a horrible programmer, and I struggled with learning most languages beyond a certain level. I struggled with syntax. I struggled with the seemingly arbitrary rules. I struggled with understanding the approach other programmers took when solving problems. I even struggled with "easy to learn" Python beyond a certain level. But once I finally tried CL, it simply made it easier for me to build abstractions I understood and to get things done. I can totally understand how weird the "NIH" culture seems from the outside, but it often IS easier to write custom-fit code for your project in CL than it is to learn someone else's general solution well enough to make it work. Sometimes things are counterintuitive...or maybe some people are just weird, I dunno? Fortunately, there are plenty of language/framework options, so most people can find something that fits.
I had to deal with well over a million records, but fundamentally it was a pretty simple ask, laying a 2d matrix of rules over user and permission records from 30 different systems. I even hooked it up to SAP and various AS400s through MQ, print queues and various other things.
It was a total stopgap move - so a whole bunch of vendors bid on the "real" system, one was selected, and now almost 3 years later and a few million dollars spent, and the "real" system still isn't working.
"Web 2.0" frameworks have a lot of applications in business, the ORM + presentation layer tools give you most of the groundwork you need, but as the article says, the key is in the polish and the business oriented design.