The value of software is no longer just about the logic of its algorithms, but also about the data flowing through that logic. Think of popular services like Google search, Bing search, Yelp, Facebook, and Instagram. These services are useful because of a combination of user-generated data (restaurant reviews, status posts) and the algorithms that surface that data (rankings and recommendations based on machine learning). Even software that is valued for its logic, like games, often have data-driven features, like matching up players. Behind the scenes, software companies are also using data to make engineering and business decisions. For instance, some product teams monitor real-time metrics to troubleshoot problems and to decide when to scale out to more servers. Other teams analyze usage data to triage bugs and to brainstorm new features. Whether inside the product or behind the scenes, today, software = logic + data.

Sadly, programming tools have not kept up and are still designed for authoring logic. Indeed, the popular languages and tools for analyzing data, like R, MATLAB, and the IPython Notebook, are a separate world from development environments, like Visual Studio, XCode and Eclipse. This separation creates awkward and inefficient workflows. For example, a data scientist might use one language and tool (say, R) to explore new recommendation algorithms; then, to deploy the final algorithm into the service, a programmer will entirely re-implement it, using a different language and tool (say, C# and Visual Studio). This problem gets worse as companies switch their focus from stored data to real-time streaming data, like live service telemetry and sensor data from wearable devices and the Internet of Things. In this talk, I’ll describe how data is changing the nature of professional software development and demonstrate new programming tools that make it easier for the user to work with both data and logic together.

Rob DeLine, a Principal Researcher at Microsoft Research, has spent the last twenty-five years designing programming environments for a variety of audiences: end users making 3D environments (Alice); software architects composing systems (Unicon); professional programmers exploring unfamiliar code (Code Thumbnails, Code Canvas, Debugger Canvas); and, most recently, data scientists analyzing streaming data (Tempe). He is a strong advocate of user-centered design and founded a research group applying that approach to software development tools. This approach aims for a virtuous cycle: conducting empirical studies to understand software development practices; inventing technologies that aim to improve those practices; and then deploying these technologies to test whether they actually do.