I implemented a service called SafeChange that proxies changes, allowing them to proceed only if the change is safe. For example, a change to a resource is 'safe' if the resource is not serving or supporting customer traffic, and the resource is not configure to serve or support customer traffic.
Migrated quite a few unsecured secrets into the organization's secret store.
Flux was an interesting but struggling startup. I worked there five months.
I scaled the overburdened Jenkins infrastructure to multiple servers.
I used Docker to run a full Linux desktop on demand, inside the container, in order to facilitate full and automated Selenium testing.
Ariba provides a wide variety of business to business automation, and is one of the most important of SAP's cloud aquisitions.
I took the lead role in researching, architecting, designing and implementing the next generation of operational tooling and automation.
I also assisted, tactically and strategically, the other teams in Hosted Operations, across a wide variety of areas and topics.
LiveOps provides very sophisticated telephony services with world-class reliability and scalability. While our server count was not enormous, the work those servers did was very sophisticated. The code base was large and extremely diverse.
My primary job was to lend technical support to any other areas in Operations that needs it, including Network Engineering, Linux System Administration, Release, DBA and others. This support came in two primary forms: tactical, hands on expertise in any of those areas, and intense, strategic development of custom code that provides automation, fault management, performance management and correlation.
My administration experience at Wal-Mart was notable, but it was not the focal point of my work. At LiveOps, we lived, breathed and thought Linux. Also, my MySQL experience at Wal-Mart was good and useful, but the MySQL infrastructure at LiveOps contained many billions of rows of data, spread across scores of really large Linux systems. I have learned much about practical MySQL and Linux since April 2009.
When there weren't pressing operational issues at hand, my primary focus was monitoring and automation. The existing Nagios infrastructure was aging and very out of date. Over 12 months, I constructed a custom agent based monitoring system built with many standard technologies that had excellent and flexible configuration, correlation and escalation capabilities. This system had a framework that also provided many other important and interesting system management applications, including:
I technically lead the Network Management team at Wal-Mart Stores, Inc. Our team's job was to enable the 70+ Network Engineers and our Network Operations Center to manage 150,000+ routers, switches, APs and various other equipment that make up Wal-Mart's global network. These devices are running in more than 15 countries all over the world.
In essence, we wrote code to automate the process of managing this vast array of devices. No commercial Network Management tool had ever worked well for us, so almost everything was custom written. My team had six members.
For most of my time at Wal-Mart, the company did not allow Linux. We had, over the years, installed a number of 'pirate' Linux boxes. Once Wal-Mart officially supported Linux, we moved most of our development to a Linux-based infrastructure. These servers provided a huge boost of performance over the existing array of HPUX and AIX systems. Until that point, we had been responsible for all management, hardware and OS, of these Linux boxes.
We also used MySQL for our database back-end since 1997. In 2009 we had 200-300 million records in our database, replicated in real-time among 9 systems running on three different OS' in all of our data centers. This made our databases more redundant than anything else inside of Wal-Mart.
We interfaced with many different teams in Information Systems because of the unique data that we collected. We were forced to write a custom layer-2 discovery engine because nothing commercial suited our needs. The data this engine collected was widely distributed among various areas in the division, because it was accurate and fairly timely.
As of 2009 I was serving as project manager, senior architect and lead developer of various sub-groups of my team in a variety of development efforts. These were some of the things I did on a day to day basis:
I was a Computer Programmer in the 81st Medical Group based at Keesler Medical Center, Keesler Air Force Base, Biloxi, Mississippi. I was there from 1993 until 1997.
My title notwithstanding, I was an extreme jack of all trades. I was the prime administrator of our Unix and AOS/VS systems at the hospital, as well as the primary Network Administrator, as well as the Computer Security Manager. Day to day, I would:
We used HP Open View and some other commercial programs to do the basic network management. I wrote a lot of other programs to automate everything from installing new Windows 95 PCs to watching our Squid cache log for unauthorized surfing.