Comments (0)

By dottow, March 16, 2011 2:08 pm

I attended a breakout session on jQuery right after lunch, but I didn’t have my laptop up and, honestly, it was a pretty high-level presentation of jQuery in any case.   The technology certainly looks promising and I’ll have to find a way to play with it one of these days.   Then again, I’m still trying to play with Scala and Mule, technologies I heard and read about after last year’s TSSJS.   Aaaah…. where does the time go?

Agile based TDD sounds like a good session.   As much as I keep wanting to move to TDD, I have a really difficult time doing so.   Hopefully the more exposed I am to it, the easier it’ll become to actually use the methodology.

What is TDD?
Test-Driven Development is a software development process.

The TDD Process:
1.  Write a failing test for the target behavior
2.  Implement the behavior as simple as possible to get the test to pass
3.  Refactor the implementation

TDD is a Design Activity

TDD treats Tests as VIP Artifacts:  You need to treat your tests as first-class citizens

TDD is an “Agile” Practice:
1.  Working software is the primary measure of progress
2.  Continuous Attention to technical excellence and good design enhances agility
3.  Welcome changing requirements, even late in development

Types of Tests:
1.  Unit Testing: Verification of Design
2.  Acceptance Testing:  Verification of Features

What is a Unit Test?
Exercises a small testable piece of software
Tests a specific object behavior in isolation
Assumes that external dependencies work according to their contracts
Defines the assumptions that can be made of the target unitHow is unit test independent of its dependencies?  Mock Objects

Tools and Frameworks

JUnit
Openpojo:  a framework for validating that POJOs are adhering to a standard contract
Mockito:  Mocking Framework – Declare dependencies with @org.mockito.Mock

The Benefits of TDD

Test-Driven Development improves Design

SOLID Design
* Single responsibility principle: Can do one thing really well, loosely coupled
* Open/close Principle: Encapsulation – What, not how?
* Liskov substitution principle: Contract  first Design.  Avoid inheritance unless “is a” …
* Interface Segregation Principle: Interface clients should only depend on methods they need
* Dependency Inversion Principle: Depend on Abstractions instead of mocking

TDD Follows Law of Demeter

TDD avoids Static Helper Classes:  Procedural programming, not OO programming.

TDD Measures Progress, provides Documentation, and provides code coverage

How to Make it Work

* Flip Your Perspective – become a user, not a developer
* Always See Your Test Fail – make sure your tests have clear messages
* Listen to Your Tests – Urge to share test fixture state, setup, … => need to refactor
* Use an Interface Driven Approach
* Learn it well – TDD by Kent Beck

KATA

TDD Kata – Exercise in coding, refactoring and test-first, that you should apply daily for at least 15 minutes.  -Roy Osherove

http://www.osherove.com/tdd-kata-1/

Comments (0)

By dottow, March 16, 2011 10:32 am

I’m back at The Server Side Java Symposium this year and this is the first breakout session I’m attending.   I think I’ve got good knowledge of ClassLoaders and Java Reflection API, but we’ll see if Jevgeni Kabanov, the speaker, can enlighten me with some new nuggets of knowledge.   He’s Founder & CTO of ZeroTurnaround, a company I know next to nothing about, although I have heard of JRebel, one of their products.   Anyway, enough background info… Let’s jump into the meat of the presentation.

Basics

public class A {
    public void do() {
        B b = new B();
        b.do():
    }
}

Instantiating Object B causes the following:  A.class.getClassLoader().loadClass("B");

Problems & Solutions

1. Class Not Found
You build some code and get a NoClassDefFound exception when deploying and trying to run the servlet.   How do we find out what the problem is?

Some solutions:
* IDE Class Lookup
* find *.jar -exec jar -tf ‘{}’ \; | grep MyClass
* URLClassLoader.getURLs()

2. Wrong Class Found
What if you get a NoSuchMethodError?  What’s happening there?  Well, the Class is found, but there’s something wrong with the class.

Solution:
* Get ClassLoader().getResource(Util2.class.replace(‘.’,'/’) + “.class” to find out where the ClassLoader is getting the class.  Then you can use javap -private Util2 to inspect the contents of the file.

3. More than One Class Found
That error will typically occur if 2 different ClassLoaders are used and find different classes for the same name.   They are not the same.

Helpful:  -verbose:class, as well as ClassLoader.getResource() to identify the different classes.

Leaking ClassLoaders

If a Class has a reference to its ClassLoader, that object has a reference to all of its classes.    To leak a ClassLoader, it’s enough to leak just one object loaded by that ClassLoader.

Hierarchy is not enough?

* Isolation: Difference versions of the same library
* Performance: Class lookup is very slow
* Restricted: Why can’t siblings see each other’s classes?
* OSGI, JBoss, NetBeans and others implement a different system

Follow the speaker on Twitter: @ekabanov

Comments (0)

By dottow, February 24, 2011 12:32 pm

I’ve been spending the last 6 weeks configuring the Forward Ladder in Calypso 11 in order to get a real-time view into FX risk from Opics.  While I actually did some work on the Forward Ladder analysis when I was back at Calypso, they did a complete rewrite for version 11 and I must say that, while I did encounter a few issues, it is well thought out and quite extensible.  As such, I thought I’d share how easy it is to plug in a new objective in the forward ladder.

The request was to display the discount factor (df, for short) which is unfortunately not available in the new ForwardLadder analysis.   While it was a column in one of the earlier incantations of the analysis (i.e., FwdLadder), it seems to have disappeared in this latest version.   We could have of course asked for a P6 enhancement request, but that probably means we’d have had to roll this out next year as opposed to this year, so that was a no go.   Digging into the code, I came across the Forward Ladder Registry, a class that allows you to plug in and/or modify the objectives and flow generation extremely easily.

The ForwardLadderRegistry class actually reads the configuration from an XML file included in the resources.jar called com/calypso/tk/risk/ForwardLadderGeneratorMappings.xml.  Here’s a snippet from that file:

<registry>
    <global>
        <objectiveGenerators>
            <generator>
                <objectiveClass>com.calypso.tk.risk.forwardladder.objective.ForwardLadderObjectiveCore</objectiveClass>
                <generatorClass>com.calypso.tk.risk.forwardladder.generator.DefaultCoreGenerator</generatorClass>
            </generator>
            <generator>
                <objectiveClass>com.calypso.tk.risk.forwardladder.objective.ForwardLadderObjectiveCash</objectiveClass>
                <generatorClass>com.calypso.tk.risk.forwardladder.generator.DefaultCashFlowAmountCashGenerator</generatorClass>
            </generator>
            ...
        </objectiveGenerators>
    </global>
    ...
</registry>

I decided to create a new objective called MarketData in order to bring in the df, as requested by the users.   Thus, I added the following lines to the xml file:

            ...
            <generator>
                <objectiveClass>com.myco.tk.risk.forwardladder.objective.ForwardLadderObjectiveMarketData</objectiveClass>
                <generatorClass>com.myco.tk.risk.forwardladder.generator.DefaultMarketDataGenerator</generatorClass>
            </generator>
            ...

That’s easy enough, right?   The code for the ForwardLadderObjectiveMarketData is trivial enough and you can easily implement it based on one of the other objectives supplied by Calypso.   The meat & bones, however, resides in the DefaultMarketDataGenerator which actually implements the ForwardLadderObjectiveGeneratorI interface.  (Incidentally, I’m not quite sure why Calypso forces you to implement 2 interfaces here.   I think one interface could easily have described the contract for both the objective metadata and the actual implementation!  Go figure…)

Here’s what the code looks like to display the discount factor:

public class DefaultMarketDataGenerator implements ForwardLadderObjectiveGeneratorI {
    ...
    public void populateObjectiveData(ForwardLadderObjectiveI objective,
			              FlowGenerationContextI context,
                                      List<ForwardLadderFlowItem> flows) {

        Trade trade = context.getTrade();
	PricingEnvBasedFlowGenerationContext peContext = (PricingEnvBasedFlowGenerationContext)context;

	for (ForwardLadderFlowItem flow : flows) {
            for (ColumnMetaData metaData : objective.getObjectiveColumnMetaData(context.getParams())) {
                String dataName = metaData.getName();
                if (ForwardLadderObjectiveMarketData.CURRENCY_DF.equals(dataName)) {
               	    CashFlow cf = flow.getCashFlow();
                    String ccy = cf.getCurrency();
            	    CurveZero curve = peContext.getPricingEnv().getPricerConfig().getDiscountZeroCurve(ccy);
            	    double df = 1.;
            	    try {
            	    	df = curve.getDF(trade.getSettleDate(), QuoteSet.MID);
                        Amount dfDV = new Amount(df, ForwardLadderAnalysisUtil.DEFAULT_CONVERSION_RATE_ROUNDING);
            		ForwardLadderObjectiveData data = new ForwardLadderObjectiveData(metaData, dfDV);
            		flow.setObjectiveData(dataName, data);
            	    }
            	    catch (Throwable t) {
            		Log.error(LOG_CATEGORY, t);
            	    }
                }
            }
        }
    }
}

That just about does it, believe it or not!   The only thing is you’ll have to tweak one more resource bundle file or the ParamViewer will choke when trying to add the new objective.   Look for a file called com/calypso/bundles/apps/forwardladder/ForwardLadderObjectives.properties in the resources.jar and add the following properties at the end of that file:

objective.displayname.MarketData=Market Data
objective.help.MarketData=

As long as your classes and modified resource files come before Calypso’s in your classpath, you’ll be able to add this new objective to your Forward Ladder.

Comments (0)

By dottow, April 28, 2010 2:09 pm

I spent most of the day yesterday listening to the Goldman Sachs testimony.  Since I missed a few hours, I decided to revisit those parts I missed earlier today at the C-Span archives.   I found it quite interesting how the Goldman Sachs traders spun their tales to deceive the Subcommittee.   In particular, they often mentioned that their goal throughout 2006 and 2007 was to reduce risk.  Yeah, fair enough.  I’ve got to imagine with all the volatility in the market at the time, the middle and back office would have pushed pretty hard to reduce the risk.

One interesting thing I thought I’d dig into deeper is the difference between selling a long position and covering a short position.  The witnesses often repeated the mantra that not only did they reduce their risk by selling their long position but they often also reduced their risk by covering their short position.

Well, yes.   But not all risk is created equal, now, is it?

Between 2006 and 2008, to cover your short positions in the market basically meant you were “locking in” your profits.  That’s because the mortgage backed security market was basically in freefall.   By locking in your profits, you may indeed be reducing risk; risk that your counterparty (cough… Bear Stearns… cough) will not be around to pay you tomorrow.   Either way, when you cover your shorts, you realize some amount of profit and reduce the company’s overall counterparty risk.

Selling your long positions between 2006 and 2008, however, was a whole different ballgame.   In fact, I’d venture to guess that Goldman Sachs did very little of that because by the time they wanted to sell, there were probably no more buyers and therefore no market liquidity.   They had no easy way to sell their longs since the buyer might have only been willing to pay pennies on the dollar.   Instead, Goldman Sachs most probably offset these losses by taking opposite short positions.   If they indeed made such spectacular profits in 2007, they must have indeed done the Big Short by several orders of magnitude more than their long positions.   In any case, your aim in selling your longs is to minimize your market risk.

It’s interesting that the Goldman Sachs witnesses would combine market risk and counterparty risk and just call it risk in order to say their aim was to “minimize” risk and not go directional.   Nice rhetoric guys.

Comments (0)

By dottow, April 27, 2010 1:04 pm

I’m listening to the Goldman Sachs testimony and as they float around words like CDO’s and ABX, I thought I’d check on what the ABX prices currently are as per Markit.   If you recall, I actually wrote the code for CDS on ABX back when I was at Countrywide in 2006 and 2007.

Wow.

I realize this is somewhat cryptic, but take a look, say, at the price for ABX.HE.AAA.07-2. This was the AAA tranche of an ABS Index made up of Home Equity (HE) issued sometime around July of 2007. Do you see the price? 44.85. The price started at 100.0 at issue so for this particular Senior investment grade tranche, you’d have lost more than 50% of your money if you’d invested back in 2007.

Wow.

If you decided to take a little risk back then, you might have bought the Mezzanine branch, as the good ol’ boys at Goldman Sachs call it.    If you’d bought $100 worth, guess how much you’d have right about now?   $3.58.

If you think we’ve come even close to hitting housing bottom, I’ve got a bridge to sell you!

Comments (2)

By dottow, April 6, 2010 5:52 pm

After my very interesting and enlightening attendance at The Server Side Java Symposium last month, I’ve decided to start exploring new technologies and how better do this than by creating some Open Source tools and libraries.  Since I’ve been working almost exclusively with the Calypso API for the last decade, it’s somewhat logical that I would start there.  I decided that my first project is to integrate Calypso with GridGain as Dispatcher to distribute Risk Analysis execution on the Grid.   Calypso has its own Dispatcher implementation, but they themselves acknowledge it’s not ready for prime-time.   They do sell an adapter to plug into DataSynapse as well, but I figured an Open-Source alternative would be a worthwhile tool for Calypso implementors.  Besides, once I get it all up and running, I want to play with Scala as it seems to integrate very easily with GridGain.

So last week I launched Eclipse and created a new project.   As I began creating a new package, com.steepi, I paused.   In order to plug my package into Calypso, I would need to implement CustomGetPackages in the calypsox.tk.util package.  But if I do that, how is a Calypso implementor going to use it?   After all, they most certainly already have their own instance of CustomGetPackages!   Now granted, they certainly could make the modification to their class to add my packages.   Still.  What if I truly wanted to provide a library that would attach my packages simply by adding a jar to the CLASSPATH?   This problem merited further investigation…

After a few hours of research, I found a solution by locating an undocumented feature of Calypso.   During startup, the AppStarter class will at some point try to instantiate calypsox.apps.main.UserStartup.   If an instance of this class is found by InstantiateUtil, the method start() will be called via reflection.   Just the hook I needed!   I could place my custom code there to attach my custom packages.

Wait, though…

How would I go about doing so?   Calypso’s API does not provide a method to add packages to InstantiateUtil.   It’s all done within a static block when the class is loaded.   Thankfully, I’ve encountered limitations with the Calypso API before and the way to circumvent this problem is to use Java reflection to render methods and fields accessible.   Here, then, is the code that does exactly what I wanted!  If you compile the following code and add it to your CLASSPATH, you’ll be able to attach custom packages to existing Calypso implementation projects that have their own CustomGetPackages implementation. It’s a nifty way to provide a third-party library on top of Calypso, don’t you think?

1:  package calypsox.apps.main;
2:
3:  import java.lang.reflect.Field;
4:  import java.util.Collections;
5:  import java.util.List;
6:
7:  import com.calypso.tk.core.Log;
8:  import com.calypso.tk.util.InstantiateUtil;
9:
10: public class UserStartup  {
11:     public void start() {
12:         // In order to attach our packages, we operate on
13:         // InstantiateUtil class using reflection so as to
14:         // reach through its encapsulation.
15:         Class clazz = InstantiateUtil.class;
16:         Field field;
17:         List packages = null;
18:         try {
19:             // Retrieve the _packages field in InstantiateUtil
20:             field = clazz.getDeclaredField("_packages");
21:             // Since this is a private field, we need to set it
22:             // to accessible so we can access it
23:             field.setAccessible(true);
24:             // Get the value of the field via reflection.
25:             // This is the actual List object
26:             packages = (List)field.get(clazz);
27:             // Add Steepi packages so they are available when
28:             // instantiating through reflection
29:             packages.add(0,"com.steepi");
30:         }
31:         catch (Throwable t) {
32:             Log.error("Error", "Unable to locate InstantiateUtil._packages via reflection.", t);
33:             return;
34:         }
35:         // TODO: Apply same logic to update _invertPackages field as well
36:     }
37: }

As noted inline, you’ll want to do the same to update the inverted packages field. I’ve omitted the code for brevity.

This solution should work with any version of Calypso prior to 11.1.   Who knows whether or not they will eventually remove this logic since it is, after all, undocumented.   Still, for the time being, it should be easier to deploy my packages in existing Calypso implementations without needing any code change.   Sweet!

Comments (0)

By dottow, March 19, 2010 8:27 am

The slides for this presentation can be found here

JSR-223: Scripting & what it means to you.

You want to use scripting languages to implement code faster through continous prototyping.

Scripting is built right into Java 6.   Spring has limited support, though.  But it is available on Mule ESB, ServiceMix, and other Spring containers.

Take a look at the javax.script package

Reasons for Scripting

• Prototyping
• Better tools for problem domain

Jawk – Processing a lot of text
Jython – System Management tools
XSLT – XML Manipulation

If the only reason you’re using scripting language is for dynamic code deployment, though, you should instead look at OSGi.

There’s also a tool called JRebel that’s worth taking a look at.

The Java Language is Awesome as long as you don’t break its abstractions!

Mule is great for pulling services together across multiple protocols.

Comments (0)

By dottow, March 19, 2010 8:16 am

Bob McWhirter by far gave the best keynote session I’ve seen yet.   His delivery is humorous, he’s got great stage presence, and the slides were very creative.  Furthermore, he was best able to give me an understanding on what the hell the cloud is anyway.

The cloud is the next logical step in application delivery.

First, we jammed some servers in some closets.
Then we did some colo.
After that, we leased some managed servers.

With the cloud, we lease what appears to be servers.

The Cloud Computing Stack

Software as a Service (SaaS) – Abstracts software
Platform as a Service (PaaS) – Abstracts Services
Infrastructure as a Service (IaaS) – Abstracts Hardware

Tenets of the Cloud

1. Illusion of Infinite Resources
2. On Demand Acquisition and Provisioning
3. Someone Else’s Responsibility

Virtualization makes it affordable to make mistakes.  Repeatedly.

The network isn’t all that slow, compared to spinning disks.

Messaging (REST-MQ)
Scheduling
Security & Identity (OASIS)
Computation (Query and otherwise)
Transaction (REST-TX)
BPM
Telephony

It’s a stack of services, instead of a stack of software.

More info at http://cloudpress.org

Comments (0)

By dottow, March 19, 2010 8:02 am

This session provides an introduction to the GWT for designing web clients.

With GWT 2.0, the Hosted mode browser plugin allows you to run in the various browser directly.

It’s probably due to limitations from my hangover, but apparently we’re running Javascript on the client but it looks like we’re writing Java.  Is GWT somehow converting my Java into Javascript?  I think that Geary alluded to that fact, but I’m not quite sure.  The code for ClickHandler certainly looks like Java though.   As a matter of fact, I later got confirmation that this is indeed the case.   With GWT, you write Java and GWT takes care of generating the Javascript!  So sweet!!

Hosted mode browser plugin – running in the various browsers directly
Code splitting is a useful pattern to use.  Because GWT applications are downloaded to the client as Javascript, they can get pretty big pretty fast.  Code splitting lets you split out your code and lazy download as needed.

Layout panels provide an easy way to create layouts without too much fuss a la Swing.

Client bundles lets you pull down a bundle of resources in one http request

WebAppCreator application is used to generate the web application:

webAppCreator com.clarity.SimpleApp

GWT now creates a bunch of stuff for us.

ant hosted

Import project into Eclipse.   Nice to know it integrates seamlessly with Eclipse, by the way. 

All GWT Applications are modules.  From the code, it looks like the server implements a Remote interface.   Also it would seem that GWT allows you to plug into Hibernate.  It isn’t just a Web framework, then?

Event Handlers

Event Handlers are very similar to Swing/AWT/SWT listeners.   They’re typically implemented in anonymous inner classes.

Swing came with adapter classes for event handling with no-ops.   GWT first came out with the same paradigm.   Not all GWT widgets are notified about all events.   You can sync events on a widget to be notified of events you typically wouldn’t receive.

History

GWT has a history mechanism as well.  It’s apparently very slick and it handles something of a state, but I’m not quite understaning why it’s so slick.   Then again, I was at the Blackjack table until 2:30 AM drinking free Gin and Tonics, so my mental capacity is somewhat muted.

Go, Speed Tracer, Go!

Speed Tracer monitors any web application and gives all kinds of data as to how many requests are being made, how long it takes, etc.  In short, it’s a GWT supplied performance analysis app and it’s good to use to improve the performance of your web app.

Code Splitting

Code Splitting is achieved by surrounding the code you don’t need right away inside a GWT.runAsync() block.   That allows for lazy-loading of code that’s not needed right away.

Best Practices

• Use History from outset
• Design for UIBinder from the outset
• Consider MVP and on event bus
• Use Speed Tracer and code splitting for performance

GWT, in many respects, is like Swing.  It allows you to build your web page GUI using a very Swing-like API.  No Javascript, No Ajax.  Just pure 100% Java goodness!

Comments (0)

By dottow, March 17, 2010 1:20 pm

Not a good start.   The speaker, Eugene Ciurana, is nowhere to be found.   Apparently he’s on his way.  Hopefully his mission critical apps are more responsive than he is.     He finally arrives out of breath as he was apparently working out at the gym.   After he catches his breath, we begin.

The presentation is about how to design, implement and roll-out cloud/enterprise hybrid applications.   There are a lot of different cloud architectures:  PaaS, SaaS, and IaaS (infrastructure as a service).   Technologies include: ESB, clods, mini-clouds, Java, App Engine, Chef!/Puppet, etc.    Cloud technologies lower your cost:

* Which applications are best suited for cloud deployment?
* Identify the advantages of PaaS or SaaS resources?
* What are the caveats of cross-platform and cross-language integration?
* High-performance alternatives to XML serialization for data exchange?  (i.e., JSON)

What is the Cloud Anyway?
* Platform as a Service (Amazon, Google, Rackspace, etc.)
* Software as a Service (Salesforce.com, Amazon)
* Infrastructure as a Service (IBM, HP)
* Pure-infrastructure (Data centers)

Cloud Services Features
* Quick deployment of prepackaged components
* Uses commodity, virtualized hardware and network resources
* “Pay as you consume”
* Horizontal scalability is achieved by adding or removing resources as needed
* May host full applications or only services
* They could replace the data centre
* Basic administration moves to the application owner
* For the bean counters… it’s an operational expense!
* Assuming sensible SLAs, the ROI is better than for co-located or company-owned data centres (but won’t achieve 4 9s)

Uptime != Availability.   Many factors affect availability: network, storage, process

Hybrid Cloud Architecture

* Many mission-critical systems will live behind the corporate firewall
* The cloud is used for high-load applications and services
* The cloud applications work independentlyof the data center applications, and vice-versa.

Case Study: Video game company with  hybrid cloud application

Objectives
* Stable architecture
* Low cost
* Build scalability whenever possible
* Optimal data transfer rate for all properties

Initially, there were some 8 machines that were running everything: QA, PROD, debugging, etc.   The performance was atrocious!

Phase 1: Scalability

* Introduces a CDN for asset delivery (media, images)
- Amazon S3 for asset delivery
- Reduces load on company servers and bandwidth costs
* Introduces database replication for production environments
* Establishes a continuous integration environment
- Improved build/release process
* Fail-over with traditional database replication techniques

Phase 2: Cloud Deployment
* Web applications move to a uniform technology (.Net)
* The database and stored procedures normalized and optimized
* Applications use common resources via Mule ESB and services
- No more direct calls from apps to database
- Business logic is implemented as stateless POJOs
* Software stacks was best of breed
* Web and other RPC services must coexist
- Different partners use different protocols
* Bandwidth can be expen$ive!
* Data exchange protocols
- clients: custom, XML, JSON
- images: HTTPD, S3
- Cloud-to-HQ: custom, XML/SOAP, protocol buffers
- HQ data center: XML/SOAP, protocol buffers
* Replication strategy: data centre
- The cloud isn’t trustworthy yet
* Deployment involves using an Amazon Machine Image (AMI)
* AMIs need a post-configuration step in a load-balanced environment
* Elastic Load Balancer and Elastic IP limitations

And so it ends.   This session clearly pitched to C-levels (in particular CIOs.)  There was very little meat and substance for Java geeks to get their hands on.   It’s a shame.   Clearly Eugene has a lot of knowledge, but this was just way too high in the clouds (pun intended)