Liberating Legacy Data

Liberating Legacy Data

The latest buzz word in the Industry is Cloud computing and there is a lot of efforts directed towards porting legacy data which maybe residing in the IBM mainframes (banks, financial institutions, automobile companies, insurance agencies, hospital records and even federal government data). Some of us, who have been in the legacy world forever find it difficult to understand why the IBM mainframe, which stood for security, scalable infrastructure and volume computing, is in threat of being made a relic by the new kid on the block-the Cloud. What really happened? The high MIPS cost associated with legacy platforms has always been a pain point for organizations using them. The ever-increasing volume of data requires these investments to multiply with each passing year. Cost effectiveness is the key driver behind the efforts to move data to a new age, more flexible storage solution which can grow as required without burning a hole in the IT budget.

Larger firms have begun experimenting their way out of green screens, some have started moving to cloud, new tech companies are already using cloud in the IaaS, PasS or SaaS models. Few are using cloud together with on-premise to create a hybrid solution. Large financial institutions are venturing to the arena, for e.g. JP Morgan has created a public cloud called Gia. There are still some who prefer UDB or are trying to get data out of IBM mainframe.

As wiki puts it “Cloud computing is the on-demand availability of computer system resources, especially data storage and computing power, without direct active management by the user.” This is where Mainframe gets a blow – with the firms who really want to pay only per usage or increase computing power on high volume days automatically. Mainframe has always been a cash cow – with MIPS usage being billed as per contract and peek time MIPS made even costlier, high volume days MIPS are charged even higher and all firms are bound by a contract – even if a firm manages to cut down mainframe usage we are bound by a contract before its renegotiated. Fast forward to cloud and there is a distinct advantage to the firms – since you pay per usage only and that too at a price significantly lower than Mainframes.

Cloud is not without its short comings. There are major concerns around latency and data privacy which revolve around the following:

• Most of the cloud service providers are US companies which means that has most of their data centers residing in US and the companies themselves are governed by the US laws. 
• With the datacenters remotely available in various parts of the world, there is a bit of latency in accessing data from cloud. Most cloud customers aim for data centers to be located locally to reduce this latency.
• Vulnerability to attacks – since data is transmitted via internet there is a probability that this is subject to information theft. Though advanced security measures are deployed, storing and moving confidential data in cloud has the smallest bit of risk associated with it.
• Labour and training – Its been said that "Technology is just means to the business”. What the industry will have to do is to step forward to train its current staff on cloud – since business knowledge becomes the core competencies for larger firms.

For a large mainframe of more than 11,000 MIPS, the average annual cost per installed MIPS is about $1,600. Hardware and software accounts for 65 percent of this, or approximately $1,040. Consequently, the determined annual infrastructure cost for a 15,200 MIPS mainframe is approximately $16 million. On the AWS side, using the AWS Simple Monthly Calculator to configure a similar infrastructure to the performance test, we estimated the annual cost to be around $350,000 ($29,000 monthly). This AWS cost could be further optimized with Amazon EC2 Reserved Instances. (Source: aws.amazon.com blogs).

Just kicking my grey cells – is Amazon going to be next IBM since they are already the world leader in cloud computing and management. What IBM did many years ago of selling “Infrastructure-as-a-service” through the release of IBM mainframe is being replayed by cloud service providers like Amazon, Microsoft, Alibaba and Google. With IBM taking over RedHat, is IBM getting back in the game of cloud computing? 

Whatever the outcome - I plan to start writing on a series of tools and methods which can help in the migration of data out of mainframe to cloud or UDB  or Linux. It is time we take this bull by the horn.

Following Leadership Legacy

Roosevelt used to say “The best executive is the one who has sense enough to pick good men to do what he wants done, and self-restraint to keep from meddling with them while they do it.” I had the honor of working under very good managers and it seemed they used to follow what Roosevelt underlined above in the message. Well, you reciprocate the same as you grow up the ladder making it a better work place for you, your manager and even your subordinates. And that’s where I think people say “I look up to my manager and is a role model for me”.

The comic strip Dilbert's workplace was started to satirize technology, workplace, and company issues. Dilbert portrays corporate culture as a world of bureaucracy issues for its own sake and office politics that stand in the way of productivity, where employees' skills and efforts are not rewarded, and busy work is praised. Much of the humor emerges as the audience sees the characters making obviously ridiculous decisions that are natural reactions to mismanagement. Scott Adams puts across the mismanagement issues in a concise manner in the comic strips portraying what should not be done - but on the bright side people just laugh it off as a good joke not really prodding their own views or behavior.

Jokes apart, wherever the macro trends are headed, the ability to engage and retain talented employees is a critical skill for managers and the firm. Most of us aspire to leave our mark in the world in some way, to be remembered for something. Involuntarily you tend to sow the seeds of the legacy through one of these:

• Improving skills and capabilities of your peers and subordinates by your actions (coaching, developing or creation of roles)
• Mentoring people in the breadth of the organization. I wouldn’t term on the quantity but the quality of the people mentored across the breadth of the organization.
• A better giving and receiving feedback framework.

DB2 Replication – Near Real-time

In a world where everything has become real-time, mainframe is fast catching up to the term near real-time. It was not long ago, that customers were asking for real-time information. This did have an impact on mainframe community too, Db2 or Adabas, the word was out to see if a replication mechanism can be developed and if such a replication is a viable option. Software AG (developers of xml & Adabas/natural) was able to size up the opportunity quickly by creating the new tool called Adabas replicator driven on PLOG. Db2 wasn’t far behind either.

The only cliché was that Adabas Replicator was yet another tool (well let us say you have to get a licensing cost to it) to be marketed by the software giant, while db2 provided a solution called event publishing for converting committed source changes into messages in an XML format and publishing those messages to applications such as message brokers by leveraging the system files within db2. Before I proceed further, I just want to pause and say this – in both the instances the replication is only near real-time and it really depends on the target systems, on how fast they can process the messages from the queue.

A simple configuration of event publishing looks like this:

DB2 MQ setup table : SYSIBM.MQSERVICE_TABLE

DB2 ID used by db2 to access MQ = DB2MSTR

Sample DDL Set up as follows:

CREATE TRIGGER “REF_DATA_TRIGGER”

AFTER INSERT ON “REF_DATA”

REFERENCING NEW AS REF_NEW

FOR EACH ROW MODE DB2SQL

BEGIN ATOMIC

SELECT DB2MQ.MQSEND

(

'MYSERVICE',

CAST(‘<REFDATA>’||XML2CLOB(

XMLCONCAT(

XMLELEMENT(NAME “FIELD1”,REF_NEW.FIELD_1),

XMLELEMENT(NAME “FIELD2”, REF_NEW.FIELD_2),

XMLELEMENT(NAME “FIELD3”, REF_NEW.FIELD_3),

XMLELEMENT(NAME “FIELD4”, REF_NEW.FIELD_4),

XMLELEMENT(NAME “FIELD5”, REF_NEW.FIELD_5),

XMLELEMENT(NAME “FIELD6”, REF_NEW.FIELD_6)

))||’<REFDATA>’ AS VARCHAR(2000)))

)  

FROM

SYSIBM.SYSDUMMY1

Special thanks to my colleague Jeegar Doshi who helped me compile the information on DB2 replication.

Adabas Reorder on the most used Super

One of the biggest saving on cpu time, and run time of batch jobs can be achieved by just doing a reorder on the most used Super descriptor. But the catch is that the reorder has to be done on the most used super and the super descriptor has to be under normal compression and all the fields in the super should be always filled. Yes, unfortunately this comes with the “disclaimer”.

The theory of such a change is that since the records are loaded back in the sequence in which the super descriptor exist, the records are fetched more faster and thereby using less cpu. Please note: the fields that you identify for the super descriptor which will be used for the re-order has to be always filled otherwise whatever fields are having null won’t be loaded back into the environment once the reorder completes. Confirm with your DBA before this is implemented in live production environment.

How to Lead a High Performing Team of Superstars

Most of you might have seen the movie “The Avengers” and even might have thought what an amazing team of super heroes- mighty, incredible, and invincible and with king-sized egos and insecurities. They can handle themselves solo pretty well – but when it comes to working together they screwed up initially. Nick Fury had a difficult role making a team out of the super heroes.  I have had the privilege of leading few such teams over the years. My team mates were stars before they came to work for me, but a few principles have gone a long way in team management and accomplishing tasks like true super heroes. J - And every member of the team eventually figures out how to be an outstanding team member.

Let them be individuals

If you have a team of high performers, ask a simple question to yourself – Are you letting them run things and operate the way they’re most comfortable doing it or the way you’re most comfortable doing it. If it’s the latter, give up some control.  Let your people be themselves. They’ll give you a lot more than if you try to get them to have some arbitrary standards you are comfortable with – let the chaos and unpredictability rule. A sense of thought leadership helps the individuals accomplish any goal if they are given the reigns. But make sure you understand the strengths and weaknesses of your team members.

Unite them under one Goal

In the Avengers movie, Nick never mentions to the avengers “Loki is bad, go and beat him hell and blue”, but they are united by a common agenda when things became personal. So set goals for the team. Rally not their minds but their hearts around that measure of success. Nick deliberately brought together a volatile group of individuals with incredible abilities and unleashed them on a colossal problem.  He didn’t give them directions or plans.  He didn’t give them rules of engagement.  He simply knew what they were capable of, what their intentions were, and the strength of character and values underlying their powers. 

Do you trust your people?  Do they trust you?  Are you confident in your team members’ abilities?  If not, understand the source of your discomfort and get it resolved fast if you want to get the best out of them because it require ultimate trust in their abilities and intentions.

Expect Conflict

With high performing teams come bigger conflicts. Everyone will have a different view of the problem, the solution, and how to work together. Do you welcome conflict on your team or do you try to eliminate it?  Are team members free to air their opinions or do you try to manage the conversation? Let them hash out their own differences.

Cover their backs

Fury says “I recognize the council has made a decision, but given that it’s a stupid-assed decision, I’ve elected to ignore it. “  He was backing the Avengers and their ability to win the day.  He stood up for his team and protected them from undue interference. If your team knows you’ve got their back and are giving them the freedom to operate, they’ll run through brick walls for you

Give them Challenges

The biggest challenge of having a team of high performers is to keep them occupied with challenges because they get bored pretty easily. So you either need a think tank who provides good ideas or you need to challenge them in new projects.

In the end, leading a high-performing team full of superheroes is an incredibly rewarding, challenging, and frustrating role to play.  They’ll amaze you with their abilities.  They’ll test your patience and intestinal fortitude.  They’ll sometimes put your entire career at risk.  That said, if you lead them well, they just might save the world.

Innovation or Idea Generation

You don’t need to be a Nano-technologist to start innovating. You don't have to be Einstein either to disrupt standards. Well, actually, you do — Einstein himself said that his greatest asset was his imagination, not his knowledge. Well the buzz word in all organizations is all about Innovations. Not just within the organization but also within the whole wide world. Everyone is talking about Innovation and idea generation. My manager had forwarded an amazing article on Innovation, the one that was catalogued by HBR (Harvard Business Review). It was just an article on how innovations evolve or what can be done to foster the culture of innovation.

Not every organization is a Google or an Apple to develop and foster the culture of innovations or innovate something new every other day. The opportunity as it stands now is not in the hands of the beholder but like Harvard puts it “Give your best employees, the most monotonous or stultifying issues – they will come up with approaches and solutions to annihilate the inefficiencies and enable new opportunities for value added enhancement and growth”. Come to think of it who best can get that idea or who best can think through the situation to accomplish the final goal? If you want something to get done, give it to a busy person because they have the energy, incentive and capability to do it. 

The other day, I was reading about Thomas Edison. He was the most innovative man of his times and if not for him we would be in Stone Age forever or would have been using a computer under the candle lightJ. His company Edison labs was all about innovations and researches and failures of course. He had innumerable ideas and had researchers to try and test if it could be done. Well did anyone know that when he invented quadruplex telegraph and wanted to sell it. He didn’t expect more than $4000 and was surprised to hear an offer of $10,000?

DreamWorks (yes the movie making unit) seems to be the best Innovation and Idea generation firm - where even accountants & lawyers are trained to provide ideas – no wonder they are able to bring out lot of good animation box office movies like Shrek, Madagascar, etc. So what makes it different from other companies? The company it seems takes pain to keep all employees up to speed on new projects and trains all workers to pitch their ideas effectively “that’s what engages people. To feel integrated and part of the company, no matter what your job is”.

Some critical success factors that I feel which provide a platform for innovations are as follows:

-          Willingness to take risk and see value in Absurdity – Albert Einstein had said “Any intelligent fool can make things bigger and more complex. It takes a touch of genius - and a lot of courage to move in the opposite direction.” Give people the chance to take risks. Otherwise you are just creating the obvious ones.

-          Visible Senior Management Involvement - It motivates employees to have their ideas and innovations get that limelight that it deserves – getting noticed and above all a visibility of their achievements. Senior management involvement also catapults the idea to a bigger arena of major players. And the pursuit of game changing innovation cannot happen unless the person who can provide the cover and yes to big changes.

-          Foster teamwork in support for passionate champions – A passionate champion can make decisions and engage the team to support the decisions. Otherwise consensus sinks between ideas to its lowest possible value.

-          Ability to synthesize facts – A person who can identify issues and has the ability to synthesize these facts quickly to a workable solution gives the right balance in the innovative team.

P.S. The biggest enemy of Innovation is “Brainstorming” – Killing an idea before it’s born.

Code review or Technology Knowhow – Getting it right

Coding is considered the most simplest of the tasks in a software development life cycle. Most people I have seen want to get the code right the first time they write, a bang for buck for the time spend. If only it works that wayJ. But the interesting part is that good code is hard to find. It’s not quite too often that I hear the words – “We should corner the guy who wrote this and beat the hell out of him”. So what makes people write bad code? Just a code review that went horribly wrong or the programmers coding concept which is wrong or a cheat sheet of best practices in coding which is way too wrong? The blame as it stands invariably shifts among all the three and comes a full circle back to the developer.

In my earlier posts I have mentioned instances where we were able to optimize code to such an extent that, it cut down cpu and run time of a process that the result send shockwaves through the developer community in the firm – A process which was running for 1.5 hrs printing 6 records in the output and consuming a good load of cpu a year began running in less than a minute and using cpu that would have last another 30 years if it ran every day as compared to its usage for 1 day now.

Well that puts us in a spot. And that brings to question, what’s a good coding practice? Or is there definite guidelines that one need to follow? Or better still, how good do you know your technology?

A few thumb rules go a long way:

·         Know your data

·         Make sure you have a design (either document it or have it in your mind)

·         Run the design through your team or peers (however small it maybe)

·         If you are planning to use any jazzy feature see how good it performs in the system or better still check with someone who has used it

·         And if possible get a code review with an expert in technology