Category Archives: DBALife

SQL Server specialization: the journey

I am a self-proclaimed In-Memory OLTP Evangelist, dispelling myths and misconceptions as I preach.

Some folks might be interested in how I got here . . .

I’ve been a production SQL Server DBA for more than two decades, but throughout almost all of that time, I have not concentrated on a specific area of SQL Server – I was a “generalist”.

Around 2011, I became aware of SQL MCM Brent Ozar (b | t) , and was inspired by his dedication to the SQL Server community. One piece of advice he has stressed repeatedly, is that you must be known as a person that can relieve a specific type of pain – you must specialize. However, even with the release of SQL 2012, I didn’t latch onto anything specific.

When SQL 2014 arrived, I became fascinated with In-Memory OLTP. I studied its inner workings, probed and poked it, but became disillusioned with its extremely narrow set of use cases, so I put it down.

When I received the early CTPs of SQL 2016, I could see that things were changing for In-Memory OLTP. Most of the issues with the 2014 release were addressed, and it was integrated with new features. I became fully committed to learning everything about it – I read white papers, blogs, books, forum posts, and every page of the documentation on In-Memory OLTP for SQL 2016.

After a while, I felt ready to present to the local user group about In-Memory OLTP.

Presentation itself is an art form, and again I must mention Brent, as it’s an area of keen interest for him (many posts on his personal blog are about presentation).

In May of 2016 I was fortunate enough to connect with SQL MCM Edwin Sarmiento (b | t), who has profound SQL Server and presentation skills. Edwin pointed me to one of his online slide decks that listed presentation resources (you can see the slide deck here, and he gives excellent advice about mastering a new SQL Server feature here).

I knew my presentation skills could be improved, so I studied and practiced presenting, and when I felt I was ready, I started applying to SQL Saturdays.

The advice I received from my mentors has proven to be true – if you really want to learn something, you must teach it. Even the act of preparing educational materials makes you think like an attendee, and helps you distill the essence of what you are trying to communicate.

Timing is everything

Towards the end of 2016, I had not seen wide adoption of In-Memory OLTP. Then on November 16th, Microsoft issued a blockbuster announcement: as of Service Pack 1 for SQL 2016, In-Memory (and other formerly Enterprise-only features) would be available in non-Enterprise editions. As a result, there seems to be a lot more interest in this feature, and I’m really excited about that.

To the blogosphere

After working with the SQL 2016 CTP releases for a few months, I started blogging about In-Memory, but I didn’t want to repeat what others had already written about it. I had read dozens and dozens of blog posts, and 99% of them fell into one of two categories: internals and speed comparison. Instead, my focus was about the various ways in which deploying In-Memory can affect the database environment, from restoring databases to database architecture, and how it works with other features, such as Availability Groups and Replication, and how to troubleshoot it. I also blogged about the inevitable “gotchas” that come with adoption of any new feature.

If a question about In-Memory pops up on Stack Exchange, MSDN forums, or #sqlhelp on Twitter, I try to help. I remember how difficult it was for me in the beginning, mostly because the documentation hadn’t yet been updated for SQL 2016.

“Jaw on floor…”

I was recently contacted by SQL MCM Robert Davis (b | t), who asked if I might be interested in joining his team (I was honored to be considered). I’ve been following Robert a long time, and have learned a lot from reading his blog and attending his presentations.

My reply began with: “jaw on floor…”

I have no idea if I’ll end up joining Robert’s team, but that’s not the point.

The real point is – my efforts to specialize have paid multiple dividends, and one of them is being recognized as a technologist having deep expertise within the In-Memory OLTP space. For the last eighteen months, I’ve had a feeling that I’m on a path that’s right for me.

My journey was set in motion by a variety of factors:

  • inspiration from members of the SQL Server community
  • a genuine desire to help others
  • having an open mind
  • extreme personal motivation
  • following my intuition

I’m writing this post so that others might draw inspiration from it – just one more way to give back.

Ned Otter

New York City, June 2017

The Road to Technology

A Tale of Perseverance

Initial resistance

During the mid-1980s, as personal computer technology started to gain acceptance in the work place, I was steadfastly against learning anything about it. I had various types of jobs, including croupier, piano tuner, trash man in my apartment building and foot messenger.

By 1988, however, I had somewhat relented. Based on my newly discovered interest in genealogy, my birthday present that year was a DOS software package called “Roots III” that arrived on 5-1/4 inch floppy disks (seriously dating myself, I know). As I struggled to learn the difference between a path and a folder, technology began to win me over. Computers were awesomely cool, and my inner-gadget-guy came alive.

In April of 1988 the phone rang (yes, they used to have bells and literally “ring” when you received a call) with an offer to go on the road with Dizzy Gillespie. Despite my mother being very, very ill at the time, I agreed to hit the road for a tour of the USA and Europe, for a total of three weeks. We played Carnegie Hall, which was a real thrill, and all the major jazz festivals of Europe. Dizzy was about 72 at the time, and other than in 1987, had not worked with a full big band in many, many years.


in Europe with Dizzy Gillespie, July 1988

I stayed in Europe after the tour with Diz ended, and returned to NYC in late October of 1988. Having always been too stubborn to play any music I didn’t feel passionate about, I considered learning word processing to fill in the gaps. I had a friend at the time who did this type of work, and agreed to let me spend time on his IBM “clone”.

In order to get a temp job doing word processing, you had to type at least 50 words a minute, with very few mistakes. I already owned an electric typewriter, and so I bought a typing practice book. After a while my typing improved to the point where I thought I was ready to look for work.

Without fail, each and every temp agency that I applied to had a typing test, and I flunked them all. But then I found one agency which had only a computer test for Word Perfect (does anyone even use that any more?). The guy at the front desk asked me if I had ever been there before, and I replied no. I took the test, and just missed a passing grade. So I went back home, researched the parts of the test that I thought I had difficulty with, and I returned to the same agency a week later. When the guy asked me if I had ever been then before, I said no. The test was exactly the same and you will not be shocked to learn that I passed.

I was assigned to the Asia Bureau of the United Nations Development Program, a few blocks north of the famous Secretariat building, and my rate was $14.50 per hour. While there I met James Oliver, a desktop database contractor (dBase, FoxPro) who was making the staggering sum of $45 per hour. We became friends, and I started to become more curious about what James did. I began to wonder if I could ever wrap my brain around the type of work that he was involved in.

In late 1989 I came into enough money to take an extended break from the work world, and concentrate full-time on becoming a computer programmer. I left the UNDP job, and purchased a 286 Toshiba laptop for the whopping sum of $2,500.

My goal was to become a desktop database programmer, and I blocked out 18 months to get it done. There was just one small problem:

I had no idea how to go about doing it.

The internet did not yet exist for public consumption, and there was only a single book on the specific technology that I was interested in. But there were dial-up services like Compuserve, which had many bulletin boards with specific topics. One was about FoxPro, a desktop database (pre-Microsoft purchase). It was a fantastic alternative to dBase, which was owned by then software giant Ashton-Tate.

Long is the road, and hard is the way

I am truthful when I say that I spent so many hours per day programming in FoxPro, that towards the end of each day, I could no longer sit down. I took a stack of LPs (vinyl records for you young folks) from my shelves, set my laptop on the stack, and continued to program into the wee hours of the morning while standing up. Every day. Every night. Every month. I wrote programs for my sister’s real estate office, my dentist, non-profits, for anyone that would let me, and I didn’t get paid a cent (except from the dentist). I locked myself in my 400 square foot apartment in Greenwich Village, and vowed not to emerge until I was a good programmer. During the approximately two years I studied, I would guess-timate that I put in 10 to 15 hours per day, and got about 5 years experience.


One of my 5-1/4 FoxPro floppies

I had started to look for programming work a little on the late side, and by the end of 1991 my money ran out. I was four months behind on my rent, and had received shut-off notices for both my electrical and telephone service. My credit card debt exceeded $18,000 (and those were 1990s dollars).

Light at the end of the tunnel

In February of 1992 I had an interview at Chemical Bank (later devoured by Chase), and the interview went well. I worked there for a year as a FoxPro programmer.

While at Chemical I got word that FoxPro programmers were in high demand at a high profile Wall Street bank. I interviewed there and was accepted. But after a year of working without a break at Chemical, I wanted to have two weeks off before starting on Wall Street. But I had to give two weeks notice at Chemical. The rep at the agency that I was working through thought that a month was too long to wait before starting, but I insisted. On my last work day at Chemical Bank I received a phone call from the agency. They had heard from the new bank that they “no longer required the services of Ned Otter.”

But I was done with Chemical, and moved on to freelance work.

A short while later, I had another interview at the same high-profile Wall Street bank, but in a different department. While in the building, I ran into the manager that interviewed me for the first position (she wished they had hired me). She asked what I was doing there, and I told her that I had another interview. She looked me dead in the eye and said: “After what they did to you, I would never set foot in this building again”. But I was determined to gain entry to the forbidden inner sanctum of Wall Street banking.

A lot of the early desktop database systems that were implemented at this bank were actually coded by traders, not programmers. They had deep analytical knowledge of their business, but their code was unreadable, uncommented, and unmaintainable. I knew I was in trouble when just such a person ushered me into a room where I took a written test, and I would not be considered for a position unless I passed this phase. This quagmire of formulae and symbols was somehow expected to be interpreted by those with perhaps vast programming experience, but zero business knowledge.

Needless to say, the entire experience was a disaster. Afterwards, they told the agency to send their best candidate. The agency said that they had already shot down their best candidate (me).

A few weeks later, I was told that another department in the same bank needed someone with my qualifications (they knew of the prior debacles, but agreed to have me interview). The staff that interviewed me weren’t immediately convinced to hire me (the agency rep half-jokingly offered them a set of free steak knives if they would give me a chance). We all finally came to an agreement that I would work there for one week, and if they didn’t like me, they didn’t have to compensate me (an outrageous proposition, but I was sure they would keep me if I could just get my foot in the door). Things went well the first week, and they decided to keep me on. The manager later asked me why I kept coming back for interviews. I told him: “Because you mofos kept saying no”.

I ain’t no accidental DBA

FoxPro was a derivative of dBase, both products using non-standard ways to access, retrieve and manipulate data. FoxPro had started to incorporate enough of the standard query language for me to consider making a shift to corporate database platforms that were based on SQL (Structured Query Language). One of those database platforms was Unmentionable-DB, and at the Wall Street bank, there were many Database Administrators (DBAs) of Unmentionable-DB on staff. I asked one of them what it was like to be a DBA.

“If hours and hours of sheer boredom, followed by moments of absolute terror sounds good to you, you’ll enjoy being a DBA.”

That intrigued me, but there were two other motivational factors:

1. I could see the end of desktop databases on the horizon

2. While faxing a time sheet to my agency, by chance I saw an incoming fax from Unmentionable-DB to the bank. It was an invoice for one of their consultants who was on site at the bank, and my jaw dropped when I saw that the daily rate was in excess of $1,200. I was stunned. That was four times what James Oliver was making at UNDP.

So I set my sights on becoming a DBA of the Unmentionable-DB platform.

In 1994 I took the Unmentionable-DB certification exams and passed, never having actually touched the product (hence the universal distrust of most certifications). I wanted to get my hands on the software to get some real-world experience, and was overjoyed to find out that Unmentionable-DB had a developer version of their database that was priced (outrageously) at $1,000. There was only one catch:

You had to pay an additional $4,000 for an annual support contract.

That’s a lot of money today, in 2013; it was a small fortune in 1994. I argued with them that I wouldn’t need support, as they had just certified me on their database platform. But they would not yield. I paid the outrageous sum and got my hands on it, but the entire episode put such a bad taste in my mouth, that I vowed to not use or touch Unmentionable-DB ever again (and have maintained that vow to this day).

Then a new player in the database market made its mark.

While at Chemical Bank in 1992, one of the guys I worked with got a hold of a new database platform called Microsoft SQL Server, and it ran on the IBM OS/2 operating system. This was a time when all software was delivered on 5-1/4 inch floppy disks, or 3-1/2 inch not-so-floppy disks (OS/2 had to be installed from approximately 20 not-so-floppies). It took forever to install, and then on the last disk, it failed. Ultimately I got it loaded, but on my puny 286 computer it ran so slowly, I lost interest completely.

Fast forward to 1995 – Microsoft had introduced its own operating system, Windows NT. I committed to learning their database platform, and have never looked back.

Ned Otter

New York City, 2013