The Peril of Shadow IT
Doing Shadow IT better with Python Skill Sprints
Shadow IT
We essentially have three choices for creating business processes today. ‘Formal’ Software development, Managed Cloud Services or we create ‘Shadow IT’ - complex processes and logic developed in Microsoft Excel spreadsheets.
Though constantly improving, formal software development is relatively slow, expensive and usually inappropriate for new and experimental stuff. Managed cloud services are great until you need a feature developed or want to retain ownership of your data.
This leaves most of us with Microsoft Excel. Whilst it works wonderfully for certain use cases - working out the coffee rota for example - when it becomes the basis for making financial decisions, the shit can really hit the fan.
In this post we will be exploring a fourth option - teaching ‘The Business’ how to code basic programs in Python using Skill Sprints. We can maintain our independence by learning how to create lightweight, powerful and safe programs instead of using Excel or other desktop tools.
Spreadsheet fails
Thomas Piketty’s arguments in his best selling economics book, ‘Capital in the Twenty-First Century’, were eviscerated by the Financial Times after they found numerous errors in the spreadsheets that he published with the book.
Nicknamed, ‘The London Whale Trade’, an error in an Excel spreadsheet used to model risk at JP Morgan seriously underestimated the risk of a credit portfolio. As a consequence the bank declared $6 billion in losses with regulators seeking $600 million in fines. The echos of this fiasco still reverberate throughout the finance industry 8 years later.
In their seminal paper, ‘Growth in a Time of Debt’, Reinhart and Rogoff argued that countries with debt ratios above 90% of GDP suffered from slow economic growth. This paper went on to influence the economic policy of many countries before a serious error was found in their spreadsheet calculations.
Software Development
Software development teams have been effectively handling these problems for years. Dev teams have checks and safeguards built into their processes which greatly limits the types of risks that spreadsheets expose us to.
Functions, the primary elements of software, have inputs and outputs. Software developers write tests to ensure that these functions are operating as expected. This is especially important if we are reworking a function. Our work is always validated by an automated test.
Many developers use pair programming when working with higher risk parts of the program. With one person controlling the computer and the other acting as ‘Navigator’ - complex and risky problems can be safely tackled. Two heads are better than one. With both participants learning faster it has been found that the product of pairing is of much higher quality.
Code is stored in Git repositories where changes can be properly peer reviewed. Git platforms such at Github give us transparent workflows for controlling change. A full history of changes is available - you can see what changes were made by whom, when.
Some software development teams feel so comfortable with the safety of their environment that they commit changes to production systems in real time - these teams understand that effective automated testing is far more accurate and safe than the most diligent human quality assurance.
End-User Developed Applications
Businesses would not be able to operate without Microsoft Excel. It is an indispensable tool - allowing us to sort and filter data, run complex calculations and myriad other tasks. These ‘End-User Developed Applications’ are often essential for the operation of the business. However, as organisations become more data driven - it’s understandable that spreadsheets are being stretched to their very limit and beyond.
It is essential that less technical ‘Business people’ have access to data and are given tools to efficiently and safely process this information. The liabilities that Excel exposes us to, are only getting worse as our jobs become more and more data orientated - the volume and complexity of data increasing exponentially.
Coding is Hard - but not that hard
Learning to write code is a rather daunting task - and, for the majority, trying to learn it from an online course is nearly impossible. Most of us are not wired to learn complex things by ourselves - especially when the entire context is changed. Online courses and trainings try and teach coding with abstract examples that have little or no meaning to the learner. Nothing is familiar.
With Skill Sprints however you learn new skills, ‘on the job’, in your familiar environment - with data and processes that you are familiar with - closely guided by an expert. The only thing that changes is the tool that you use to express those processes and logic. We teach you and your learning partner elementary software development practice - using git to track and control changes to the code.
Skill Sprints are one or two week micro-transformation projects that help two participants get to grips with a new technology - such as Python. Skill Sprints are led by a seasoned expert who will keep you off the rocks and out of the weeds - you will receive face to face instruction and leadership as you deliver your first programs in Python.