Recommending Software Configurations

by Carson Reynolds

In order to build a new variety of interfaces for recommender systems a new approach and perspective on adaptive systems and privacy will be employed. An open prototyping approach will provide constructed candidate interfaces and raw materials; participants will critique these candidates and construct bricolage-interfaces of their own made from the raw materials. These interfaces will provide metaphors and mechanisms to describe homeostatic preferences for a recommender system for Linux software configuration. The most favorable prototypes will be evaluated in terms of traditional measures like time on task, questionnaire-based subjective evaluation, and a new protocol to evaluate privacy/functionality trade-offs.

The design and evaluation of an interface to a recommender system that adaptively manages software configurations in an Open Source development environment.

When people first begin using Open Source operating systems, applications, development environments, etc. they make personal decisions about how they want their systems to behave. These decisions are usually related to a wide variety of factors, including: the system’s purpose, the system’s stability, the system’s performance, the system’s design, the system administrator’s preferences and experience, and the system administrator’s position in and knowledge of Open Source communities. While this “set-up” phase is an important part of any system’s design, the system administrator inevitably finds herself in a less well understood phase once the system needs to be upgraded and iterated upon.

With proprietary software, the upgrade process is opaque (e.g. Microsoft Windows Update) but with Open Source systems, decisions must be made. The system administrator must weigh the costs and benefits associated with each change to the system. Will my system become more or less stable after the upgrade? What is the likelihood of this change affecting some other features of my system, necessitating further changes? What will be the effect on my system’s performance and how will end users become acquainted with the new system? How well do I understand what change is being made and am I and my Open Source community experienced enough to fix new bugs? In a sense, every system is in a constant state of flux with every Open Source user, developer, system administrator and Linux hobbyist assuming some stance on how involved they choose to be in this dynamic process.

The usual way to manage this dynamic process in the Open Source community is through a cumbersome and confusing set of interfaces in which the effects and dependencies of software upgrades are poorly communicated. Moreover even an extremely communicative interface does not simplify the task of managing the software configuration. An understudied approach for managing this upgrade process is the use of adaptive systems that recommends alterations to software configuration for users. The user teaches the adaptive processes their notion of suitable preferences and the system automates the processes of altering system configuration by suggesting the most relevant alterations. These recommendations are based upon usage data collected from users of our package configuration management interface.

Guiding Research Questions

  • How can users communicate their preferences to an adaptive process?
  • Under what circumstances will people barter privacy for increased functionality?
  • Can interfaces constructed using an open prototyping methodology perform better than the most widely used examples of system configuration management interfaces?

Open Prototyping
In recent decades we’ve seen the emergence of open source methodologies for software development. These methodologies have yielded many compelling pieces of software (viz. the Linux Kernel, the world’s most widely used webserver–Apache, the Mozilla web browser, and many system configuration / package managers like RPM, apt-get, and Portage). However many open source projects are ridiculed for producing interfaces that are very difficult to use.

However the usability community could certainly benefit from many of the approaches used by open-source developers. Likewise the open source community could certainly benefit from improving the quality of interaction with their software.

Open prototyping is a methodology in which candidate interface prototypes are publicly presented to a community of users. Along with these interfaces the raw-materials used to construct these prototypes are also presented. The open prototyper facilitates the construction of new candidates that represent the community’s expectations about interaction. The open protoyper also solicits criticism of the original seed-candidates and the variations constructed by the community. The domain is bound by the raw materials presented to the prototyping community. The prototyper can constrain the space of possible interfaces by selecting materials that afford certain sets of possibilities. Conversely, the space can be unconstrained by the selection of more primitive materials that allow for a larger variety of expression.

For the domain being considered for this project participants will be presented with several examples of ways of interacting with an adaptive package managers. Additionally the users will be presented with the basic components used to construct these examples. These components consist of interface elements, usage information sources, different control strategies, and usage scenarios. From these, with the aid of the seed prototypes to help them, the community is expected not only to criticize the original prototypes but to construct their own variations. Depending on the experience and time-commitment of the participant, different approaches can be employed to construct these variations. These approaches range from construction of bricolages (through simple rearrangement, inclusion, and exclusion of interface features) to the construction of altogether new materials (through the use of writing and programming).

Privacy Evaluation
In order to compare the merits of the different interfaces the sort of information they require of users will be linked with the sort of functionality they provide. Using traditional experimental design methods we will collect data about not just user performance and subjective ranking but how users choose interfaces that impact their privacy.

Unresolved Issues
De Morgan once noted that “Great fleas have little fleas upon their backs to bite them, and little fleas have lesser fleas, and so ad infinitum.” One problem with our approach is that critics might argue the following. When asked why we want to adapt software configurations, we would reply that this would allow us to better tailor to the individual needs of real users. This in turn suggests that an adaptive interface should be used to manage the adaptive process which alters our software configuration, which raises the question: how does one communicate and interact with this adaptive interface to our adaptive process? Ideally adaptively, but in reality at some point interaction must be stable and concrete because otherwise we build an adaptive house of cards in which each interface is adaptive and the system as a whole is unpredictable and hard to control. We instead will approach the problem of interacting with adaptation by allowing groups of users to design an interface for interacting with a system to adapt their software configuration. Thusly we sidestep the reductio ad absurdum implied by adapting into recursive infinity, but do not resolve it.

Research Process

  1. Develop concrete prototypes which allow users to communicate with and design adaptive processes to manage their software configuration. Examples would include:
    • a metaphorical legal system that allows people judge their software and convict it when it performs poorly
    • a direct manipulation interface in which users explicitly set thresholds which a homeostatic processes uses to filter out upgrade candidates
    • a system which automatically selects variations and allows users to see a mirror image of their current operating environment.
  2. Publish the common widgets and information used in the development of the candidate prototypes as raw materials. Develop easy to use interfaces so that participants may construct their own candidates from these raw materials. These materials will be usage information, GUI widgets, and task analyses. The usage information will be comprised of data and statistics that can be computed or gathered from users.
  3. Engage Open Source community users, developers and system administrators in the Media Labs, Dublin and around the world in different design activities in which they construct GUIs to adaptive package manger systems. Carry out on-going discussions with participants about their design decisions, the materials and the Open Source community in which they are situated. Participants will be motivated by the possibility of reducing system administration time and the common altruistic ethic ingrained in the Open Source development communities.
  4. Describe this design process and interface results, paying particular attention on people’s understandings of how they chose to interact with adaptive processes and the kinds of GUI design decisions they made to support and reflect these understandings.
  5. With refinements, take 1 or 2 of these interfaces and compare them to existing package managers (e.g. Windows Update, Debian apt-get). Use time-on-task, questionnaires, and privacy analysis for comparison.
  6. Conclude with a discussion of how this process and the resulting interfaces compare with current work in the field.