Dreaming in Code

I’ve just recently purchased a new book, Dreaming in Code, by Scott Rosenberg.

It’s a good read, so far, but I’m only on page 100 or so, so I got a ways to go. Still, I’ve got to chime in right now because I’ve heard and/or read this before and I think it’s bullshit:

Here, once more, was the archetypal dilemma of software reuse. Build or borrow? Virtually every software project sooner or later arrives at this fork in the road. (The full phrase traditionally is “Build, buy, or borrow?” From a technical perspective, though, “buy” and “borrow” are similar, the commercial and open source sides of the same coin.)

Look, I don’t give a damn if you’re the most productive developer or architect in the universe, if you have a solution that fits your needs but perhaps only to 80% and you ain’t buying or borrowing, then you really shouldn’t be in charge of the project. Do yourself and your employer (or client, as the case may be) and focus on technology and let somebody do project management because you suck at it.

Bottom line is: write an interface. Then have the implementation of this interface be a simple adapter to some borrowed or purchased code. If you have time to make it better in the future, then plug in a new implementation. If you rewrite because you ain’t getting that last 20%, then you’re a fool. If you’re at this fork in the road, realize that you currently have 0%, and that you can probably crank out an adapter within a couple of days. 80% of the desired functionality within a few man/days. You have to be smoking something if you don’t follow that path!

Yes, I’m fairly extreme in my views on this, but I’ve found that I’m always right when it comes to this issue, too.

Note to Self: Use session token as argument across all your APIs.

If you’re gonna be running code on any kind of multithreaded environment, a service layer of any kind, whatever… In other words, unless you’re actually running on the GUI and you’re actually dealing with event-driven development… please please please add a session token as an argument to every method call in your API.

You probably will think: “Why? I don’t need it.” But if and when you do, it might be too late… because you might be live at several clients, and you may not be able to change the API at that point in time, or you have to be backward compatible, right? Well, if you ever plan on having two-phase commit, or horizontal access permissions, whatever… you better have some kind of session token that allows you to retrieve your “context.” If you don’t have that, you’ll hack it…. oooohhhh, you’ll hack it real good, because you won’t have a choice. But you know what? If you ever want a clean fix: you’re shit out of luck!

Think Outside the Box

I came upon an interesting problem late last week.

I’ve recently implemented an OTR Bond interface to Calypso so that, when new TIBCO messages alerts us of a new “On-the-run” Bond issue, we can insert the appropriate data into Calypso. Of course, if it’s a new Bond, there’s also a good chance that we don’t yet have the Bond definition in Calypso and, for that, we use the Bloomberg Connect API to retrieve the Bond and link it to the Calypso BondBenchmark… Sorry. That was just an overview but I can already see that most of y’all have your eyes all glazed over. Didn’t mean to bore you to death.

Anyway, I’m rather familiar with Calypso’s Bloomberg Connect API and how it works. Puh-lease… give me a hard one!!! After all, I pretty much wrote the whole thing single-handedly a couple of years ago. Let’s just say that a request file is generated, then FTP’ed over to Bloomberg. Then, it’s your responsibility to poll the remote directory for the equivalent response file. Once it is available, you download it and parse it locally with the data you requested. Pretty straightforward, right?

Well, when I implemented the Bloomberg interface, I thought of the “requested” scenario and never bothered (foolish me!) to think outside the box. Let’s say you request a Bond with CUSIP 123456, then I’d simply generate a request file called cy_123456.req (cy being short for Calypso. Y’see? To top it all off Bloomberg states that there’s a maximum number of characters in the file name. Some puny amount like 12 or 15, I think.) So anyway, cy_123456.req would get processed and FTP’ed to the remote directory. Then, I’d poll every once in a while for a file called cy_123456.out. When the file would be present, I’d FTP it back to my local directory and remove the file remotely, thus cleaning up after myself.

This ain’t rocket science, right? That should work just fine, no?

If you’ve got one instance of Bloomberg Connect running, it runs fine and dandy, yes… but here at Countrywide, we got a bunch of environments all running in parallel. We’ve got a couple of development boxes, UAT, and of course, the production environment. The configuration for these boxes are identical and the OTR Bond adapter task kicks off from 4 AM until 5:30 AM. So now you have these 5 or 6 processes all running on different boxes each listening to TIBCO. Let’s say at 4:05 TIBCO broadcasts: The 3 month OTR Bond has CUSIP 2223123. Well what’s been happening is this: these processes would check Calypso and determine it that Gosh-darn-it… we ain’t got that Bond definition. Let’s go fetch it!

5 processes on 5 different machines each generating a request file, cy_2223123.req. Some time passes and the Bloomberg Engine running on each box processes the file and uploads it to the remote (COMMON!) Bloomberg directory. Some time later, these 5 processes would each look for some file, cy_2223123.out, download it back locally and proceed to delete the remote file.

Hmmm…. can I get semaphores implemented on top of FTP? On second thought, don’t answer that…

I’m surprised that the process failed so infrequently, but occasionally, you’d have some poor process who’d never get its .out file. No wonder, some selfish sibling would have wiped it out before he got a chance to retrieve it. The funny thing is that, had I even pondered the question back at Calypso, it would have easily been apparent. I just never bothered to think: “Hey, wait… what if they run this process 5 times instead of just once, which seems logical.”

The quick fix? We’ve decided, until Calypso issues an official fix, to stagger the execution time for OTR Processing. Typically TIBCO will broadcast OTR information sometime between 3 and 4 AM and, of course, that information remains cached for new subscribers. So as long as we don’t connect before 4 AM and we have each box start processing OTR Bond messages on TIBCO at 10 minute interval, each instance of Bloomberg Engine should, hopefully, get to do its thing by itself. It’s not the most elegant solution, but hopefully it’ll do the trick!

On-the-run Bonds

For the last few weeks, I’ve worked on implementing OTR Bonds in Calypso. An OTR Treasury security is the most recently auctioned Treasury bill, note or bond of a stated maturity, with OTR an abbreviation for “On-the-Run.” Today, 09/29/2006, for instance, the 3 Month OTR Bond has CUSIP 912795YL9 and was issued yesterday with a maturity of 12/28/2006. In 15 days, however, it will roll to a different Bond issued sometime in mid-October. So, in essence, the 3 month OTR Bond is a reference to an actual bond. Calypso’s modeled this quite nicely, in fact, with the concept of a Bond Benchmark which, itself, is an instrument that can be priced or traded, but actually references a “benchmark” Bond. Simply put, OTR securities provide a convenient way to model market data curves for Fixed Income Trading.

The thing that’s been rather tricky, actually, is to interface Calypso with a Treasury TIBCO feed that supplies us with the latest OTR Bond aliases. Obviously enough, the first step is to retrieve the Bond Benchmark to make sure that OTR Bond and the actual benchmark match the data provided by TIBCO. If they don’t match, then the Bond Benchmark needs to be updated as well. Then, however, we also need to update Calypso’s feed config so it knows how to retrieve the latest quote from TIBCO in the future.

It’s actually not all that complicated, except that the persistence layer for Calypo’s Bond Benchmark keeps barfing up NullPointerExceptions. Apparently it tries to do some caching, but it’s really not all that good at it, and considering that you’re dealing with… hell, let’s say one thousand bond benchmarks, Calypso’d be better off just fetching the data from the DB instead of spewing exceptions at me. That’s simply not polite.

Unfortunately, Calypso doesn’t give you the ability to trade a Bond Benchmark. You can trade the underlying Bond, but apparently trading a reference was too hairy to design. It’s true that it adds a level of complexity since you would need to take a “snapshot” of the benchmark at the point in time when the trade is created. If that trade settles a month down the line, the 3 month OTR Bond will have changed, but obviously the Bond which was traded will not! Then again, seeing it from that perspective, it might be just a little too confusing to keep track of. A nifty feature, though, would be to at least be able to enter a trade with the 3 month OTR Bond and having Calypso automatically fetch and populate the trade entry screen with the appropriate bond.

I’m kind of curious why not create a generic ProductPointer that can be traded and have the BondBenchmark either extend the base class or implement the interface. My gut says I’d go with an abstract base class since you’d want to bring down as much functionality as possible into the superclass and just give hooks to customize the behavior.

Have a good weekend,

Ch-ch-ch-ch-changes

So I haven’t blogged in a while, but then again I’ve been rather busy lately. I got a new job with Countrywide Securities Corporation, moved to LA, and got married just over a month ago. We just finally got rid of our last box last weekend and we plan to hang up our framed pictures in our new digs either tonight or tomorrow.

Change is good, right? Well, yeah, change is good, but change is hard as well. I really enjoy the new job and I’ve already been exposed to quite a few new technologies and concepts over the last couple of months. The team I work with is really nice and they’re all very bright. We did quite a bit of apartment searching before settling on the Summit in Woodland Hills and it’s actually quite a nice place. Still, I miss the Bay Area, I miss our loft in Vallejo and I really miss our backyard!

Thankfully, we’re not selling. We’re currently looking to rent it out so if you know anyone who’s interested in renting a kickass loft in Vallejo, 25 minutes to the Napa Valley and 1 hour Ferry ride to SF, e-mail me at david (at) ottow (dot) net.

Peace.

Disappointed…

So my favorite Soccer team in the whole word, Holland, lost today in what is probably the most insane game I’ve seen in the last decade. The Russian referee, Ivanov, a small and bitter man with clearly a huge inferiority complex, had in him the desire to send off both teams with red cards… He managed in ejecting 4 players from the game for the lamest possible offenses… I seriously hope FIFA sends Ivanov a couple of thugs with some pliers and a blowtorch so they can get medieval on his ass.

I’ve been waiting for Holland to win the World Cup since they were beaten in the 1978 finals against Argentina. Sadly, the Dutch never cease to disappoint me…

Livin’ on the Edge

That’s the title of a good Aerosmith song; not the best, but still quite good… I think that the edge is also where search needs to go. Google is trying to do it in one centralized location using hundreds of thousands of PCs, but search will only scale if you do it on the edge.

Actually, that’s not quite true. We couldn’t be where we’re at today without Yahoo!, Google, Microsoft, and before them. But we’re now in the era of Web 2.0, and while a whole bunch of companies are sprouting up touting AJAX-y features that they claim will disrupt life as we know it, I think what Web 2.0’s huge disruptions are still to come. We have yet to see the Netscape IPO of Web 2.0. I think it’s time to migrate the search knowledge we’ve acquired so far to the edge network.

In fact, I think that search now belongs on the client as an application that sits there, observes, and learns from your behavior. Let’s be honest: Search 1.0 is trying to divine your hopes, fears, wants, and needs by using a lousy text box as the interface between you and it. If it gets lucky, you might throw a few more words at it, but that is usually due to the fact that it was utterly incompetent in finding what you wanted to begin with. Worse, if you get exasperated and move on to a different search service, it knows not whether it has succeeded or failed.

Now a search application on the client knows a lot more about you since it has full and undisputed access to 100% of your clickstream. It knows that you were listening to Concrete Blonde when you launched your browser, googled the band, read up on them at Wikipedia, and noticed that Johnette Napolitano released an album called ‘Sketchbook 2′. You pointed your browser to Amazon, where you searched for Johnette Napolitano and came up with absolutely nothing. You went back to Google, queried ‘Johnette Sketchbook 2′, and found what you were looking for right here, and promptly made the purchase.

Search 2.0 looks something like this: Clickstream logging and aggregation occurs on the client. I can also bookmark and tag items, either stored locally or on the network and I can share this information with others (or not). Others may be more nuanced than public or private. Perhaps my privacy policy can tap into my social network so I’ll only allow friends and friends of friends to view my favorites. Whatever… No matter what the key is this: I have the ability to store 100% of this information right here on my own machine: I *own* my data, and I’m sure of it because the code I’m running is open-source so I don’t worry about Spyware. If I choose to share the data, I can do so freely since the client application supports common P2P protocols.

Now maybe I want the convenience of storing my data on the network so I can use and retrieve it wherever I am. The choice is mine. Perhaps I pay a monthly fee to have a trusted 3rd party host that data for me or, perhaps, I willingly give some of my data to pay hosting costs, or, if I really have nothing to hide, I might even try and make a buck by selling every piece of data I produce. I repeat: the choice is mine.br /br /I’d think that as far as business model, running these “edge boxes” where you either host data for users for a fee or you buy their data, aggregate it, slice and dice, and sell it to corporations after stripping off the identity or, for those few brave souls, along with it.

It’s time to move search to the edge. When a new programming language surfaces, the first compiler is written using an older established language. The language comes to life when you can finally write a compiler for that language using the language itself. Extending the metaphor, Search 1.0 has provided us with a “new programming language.” It’s time to write the compiler.

C’est ça la France!

I’m really saddened by what’s happening in France right now… Although I haven’t blogged about it before, my mother is French and I lived in France for about 17 years on-and-off throughout my life. I’m terribly saddened by what’s happening in the country right now. Although I’ve never felt a strong French identity, it’s innate..

Let me first preface this discussion with a disclaimer: I’m going to facilitate my task by remaining abstract and don’t wish to address specific religious or personal issues. Suffice it to say, that today brought the first reported casualty of this debacle, and that has certainly disturbed me a great deal.

Speaking on the abstract geopolitical level, this is probably a good thing to happen to France at this point in time. The tension between France and the United States has been quite high since George W. decided to impose our might over the Middle East and secure a source of oil for the next decade: granted, if it had been put in that light, I probably would have been less against the War than I am right now. I think we’re all smart and wise enough to accept the consequences of our actions. What I find fundamentally wrong with this endeavor in Iraq is that it was veiled in secrecy and that we were, instead, fed a daily dose of fear, uncertainty, and doubt, crudely attempting to associate Al Qaeda with Saddam. I digress…

The situation in France right now will hopefully be humbling to the country. France needs to step off its pedestal and accept that it’s a Western nuclear power not so different and certainly not superior to the United States. France was the first to bitch and moan about the inequities of the United States and the abhorrent treatment of minorities in Louisiana during the recent ravages of Katrina in New Orleans. Yet is France so bloody superior?

France’s state ensures universal health care and education for its people. That’s certainly admirable and the United States could learn a thing or two about providing a decent security blanket for its citizens. That said, a lot of first, second, and even third generation Muslim immigrants from Morocco, Algeria, and Tunisia have not been able to properly integrate with French culture. France, feeling a deep suppressed guilt dating back from the early 1900s where they utterly failed to “build nations” (Yes, Dubya… You’re not the first to think up of the concept of Nation building… Why don’t you talk to Tony and Jacques about that when you find the time to leave your Texan ranch?) The war of Algeria was France’s last attempt to retain a colony, and that didn’t work out so well either… So what next?

What’s next is France developed utterly lax immigration policies and pretty much anyone who wanted to come to France was welcome to do so…. Great, right? Well, yeah, not so fast… France provides wonderful health care, education, and minimum living standards ensuring that any citizen can get adequate living: enough money for necessities and a roof over their heads. So now you have millions of ‘newly minted’ French citizens with basic necessities… So what’s the problem?

Aaaahhhh, so here’s the magic ingredient. Whereas in the US, you’re pretty much left to fend for yourself, you can, should you so desire and if you have enough ambition, move up the ladder and become rich, or at least “well off”. The State doesn’t need to lug around such an astronomical social machinery that, although it tries to minimize inequities, makes it virtually impossible to ‘get ahead’. So here you now have 5 million disenfranchised French citizens with no job, no hope, and no future. Yes, these people are “provided for” in that they get health care, a good education (just as good as any education you’ll find at any UC or any public university in the US), and basic living expenses. But then what? These people are discriminated against in the job market, with over 25% of those 25 and under without a job… So what? I studied my ass off, am just as qualified as anybody else, but because of my first or last name, I suffer discrimination and therefore can’t even find a job, though I can very easily debate existential philosophy vs. the more austere views of Kierkegaard…. Big fucking deal! Where’s my job, bitch? Where’s my ability to move up the social ladder and make something of myself? Or am I, because my first name is Ahmed, resigned to living in the ghettos the rest of my life?

I don’t have the answer… I don’t even have any semblance of an answer. It certainly does sadden me, however. Has France failed in its political vision, Liberty, Equality, and Fraternity? Well, it may just be that they’re finally realizing that though it might be easy to pick a slogan, it’s another thing to get your people to truly “feel” fraternity and embrace its diversity. And the worst part of it is… Jean-Marie Le Pen rejoices as he sees the cars burning in the ghettos…

Peace,

David

For sale: $35 1-year PUT option on Spider-Man #25

I just came across this picture and figured I had to blog it… After Incompetent Miers, Bush has now bent over to lick the shoes of creepy conservative christian fundamentalists by nominating Alito. The Supreme Court is on the brink of overthrowing Roe v. Wade and making it illegal to be anything but a rich white American male with several Uzis… The presidential elections happened just a year ago today. One down, three to go.

I’m blogging using now. It offers really sweet web 2.0 service integration and I hope that the ease with which I can now blog will boost my throughput, since I certainly haven’t been writing copious entries lately.

I’ve started to feel the tech buzz over the last few months… All kinds of delicious new technologies have emerged over the last couple of years, most of them revolving around concepts like tagging and folksonomy, peer production, and better GUIs using Ajax. We’re already into iteration 2.1, apparently, with tools like flock surfacing, which allow me to do all of the above using one well integrated client application.

I had this weird thought the other day and it seems like there’s a potential for it, but I haven’t completely refined the idea: Why hasn’t anyone come up with an eBay derivatives market? Obviously such a thing wouldn’t make much sense for magazines and books, but I think there could be an opportunity when dealing with big ticket items, appliances, cars, etc. Standard amortization practices of accounting would lead one to believe that this makes no sense at all…. Yet, that’s not quite true. I think that some principles of the long tail actually apply here. I was looking for an old vintage Donkey Kong game console a while back (I don’t mean the Xbox or Nintendo game… I want the standalone machine). Well, you’ll be surprised to find out how much they’re fetching… Clearly most of these collectibles are niche markets. It seems logical that one could spawn a derivative market on top of that… “I don’t have the money right now but I’d like to buy a 1979 VW Beetle convertible. Instead, I’ll buy an $4000 6-month call option on one for, say, $250.00.” I don’t know… maybe it makes no sense at all, but I’m dealing day-in and day-out with currency options, credit derivatives, and there are now even compound options which are options on Currency options… With all these abstractions, you’d think that a derivative market for niche markets could make sense, no? I don’t fully grok it myself… Seems like a fun idea to explore though. 😉

Peace,

David

Why is eBay buying Skype?

I’m trying to figure out the strategy behind eBay’s purchase of Skype for $4.1 billion, but then again… with Google and Microsoft’s recent VoIP acquisitions, the news is not totally out of left field.

There are some suggesting that the online auction market has matured, and that eBay’s merely trying to enter different playing fields… I don’t buy that. If that’s their play, then their CEO must have had some musings while hanging out at the corner liquor store: Hey, I’m in a liquor store and these guys sell prepaid phone cards… eBay is an auction broker, so why don’t we start selling phone calls too?

I think it’s actually a smart acquisition. What eBay really doesn’t want is to lose market share, but it seems like buying Skype will help them streamline the auction process and ensure that they retain their lead. Skype will bring a new payment option to the mix. Mastercard, Visa, Amex, PayPal, or Skype? A buyer can now close her transaction and, if she’s feeling reluctant to give her credit card information online (what with all the identity theft going around these days), she’ll be able to get in touch with the seller directly, free of charge, and talk to the seller to give the requisite payment information. Providing a free way to link buyer and seller could bring in a whole new set of potential customers, late adopters by definition, who are very hesitant to distribute too many personal bits online.