Sustainable HCI Must Account for Uncertainty in Sensors and Materials

Kristin Williams frames sustainable human-computer interaction around a conflict: computing can help people respond to environmental problems while also contributing to them. The growth of electronic waste, continued greenhouse-gas emissions and rising demand for energy and water make sustainability an immediate design concern, she argues. Data centers and AI add pressure to resources that communities already have to manage.

The conflict appears in the low-carbon innovation paradox. A technology intended to reduce environmental impact may increase it through its own manufacture, operation and supporting infrastructure. Williams points to the Internet of Things as a case in point. Sensors placed at the point of energy use could help people understand consumption and make better decisions. But adding connected devices also increases the number of products that must be manufactured, networked and supported, as well as the data that must be collected, stored and analyzed.

Williams contrasts the roughly five to ten devices per person associated with the shift from desktop computers to phones, tablets and wearables with the possibility of 50 to 100 devices per person as computing becomes embedded in buildings. The tools used to measure and manage emissions, in other words, have environmental costs of their own.

That tension does not lead Williams to reject computing as a sustainability project. Instead, she argues that researchers need to work with uncertainty as a design problem. In sustainable-computing research, uncertainty is often framed as a question of whether near-term efficiency gains will be offset by greater use over the long term—the rebound effect. Williams considers that question important but insufficient. Her focus is another uncertainty: whether the data used to make decisions reliably describes what happened in the first place.

One route to more sustainable computing is to build on things people already own rather than replace them with connected equivalents. Williams became interested in this possibility while exploring lightweight ways to add Internet of Things capabilities to everyday objects. The question was not simply whether an object could be modified, but whether the modification would leave its owner better off.

Upcycling describes a transformation that gives an artifact greater value; downcycling leaves it with less value or use than before. In the smart home, Williams argues, the drive to add computing can displace objects that already work. A household may be asked to replace ordinary bulbs, sound systems or other functioning objects with smart versions. Yet families in her research often felt less attached to digital devices than to their analog counterparts, and had trouble giving those devices the same place in daily life.

To understand what existing objects do for households, Williams and her colleagues studied ten families in the Pittsburgh area. They spent roughly seven to ten hours with each family over a week, including home tours in which residents identified objects they wanted to augment and others they wanted to keep unchanged. The researchers found that families sorted objects partly according to whether they worked reliably or demanded ongoing attention. Objects that “do what they’re supposed to” were valued for supporting daily routines and fading into the background. Adding IoT could threaten that reliability by introducing fragility and dependence on companies whose service or priorities might not fit a household’s needs.

The role of an object, moreover, cannot be reduced to its function. One participant, Lisa, preferred a physical cookbook to using recipes on an iPad: she could mark the pages, see stains from recipes she had made often, and leave the book to her children. The book carried evidence of use and a family history that the digital alternative did not provide. Other objects supported people’s interests and creative practices. Lisa adapted a rolling pin to help with baking, and her family gave her discretion over a kitchen she had remodeled around cooking.

Household arrangement mattered too. Families used the placement of objects to keep track of projects, breakdowns and repairs, and to hand tasks off to one another. What may look like clutter can organize activity and externalize a person’s understanding of what needs doing. For Williams, upcycling offers a way to bring existing objects into a computing ecosystem without erasing those histories, relationships and routines.

The household findings also complicate the idea that a new connected feature is automatically a benefit. Sheila, a retired systems administrator, had built personal systems for managing medication, bills and social invitations. She nevertheless described feeling as if she was always working. A folder left on the counter marked an unresolved task: she had asked a car company for an owner’s manual, called again when it did not arrive, and left the folder visible until the company fulfilled its promise. The physical environment helped her track what remained unfinished.

Her household was also reluctant to upgrade its computers’ operating systems. They feared an update might make Sheila’s custom systems obsolete, and preferred software they knew worked. In Williams’s account, reliance on services and upgrades can make a home more fragile, even when the technology is introduced as convenience.

Williams says physical objects can also help families negotiate whether and how to introduce IoT. In follow-up discussion, she described how an object can make a proposal concrete: people can point to the thing they might modify rather than debate an abstract idea. Some families needed time to live with an idea before they could judge it. Living with a prototype for a day, then stepping back to reflect, helped them consider both its practical consequences and how it might change household routines. Not every family reached agreement; sometimes one person drove the idea while others went along.

Upcycling creates its own design challenges. Conventional prototyping and fabrication often assume materials with consistent, known properties. Reclaimed materials carry traces of earlier use: old wood may contain nails or other embedded metal, toxic coatings, cracks or structural inconsistencies. These conditions affect what can safely and successfully be made from the material.

Williams and her colleagues studied four experienced woodworkers, each with more than ten years of upcycling experience. In workshop visits, they asked participants to recall recent work with reclaimed wood, use a generative toolkit to make tacit inspection practices visible, and discuss how material quirks shaped design choices. Williams did not present the empirical findings from this work, which she said was under review, but described four design principles that guided a system the team developed.

First, people need to recognize risks early. Less experienced woodworkers may overlook a condition, misread it or decide prematurely how to use a piece. The consequences could include unsafe work, wasted material or an infeasible design. Second, tools should help people decide what to do with a defect: feature it, repair it or discard the affected region. Third, design should happen on the material itself, because each piece has its own dimensions, shape and distribution of imperfections. Fourth, tools can make expert practices available during ordinary work, helping learners develop skills for handling defects rather than avoiding reclaimed material altogether.

The resulting system, UBench, combines sensing tools with direct interaction on a wooden workpiece. An overhead camera detects the wood’s outline and visible features such as cracks and knots. Instrumented pens detect hidden conditions: an inductive sensor can locate metal, while a capacitive sensor measures moisture. The camera tracks the pens, and a projector places annotations directly on the wood, keeping a detected defect tied to its location on the actual piece.

A tracked pen can be used to select a defect and open an explanation of possible safety or structural concerns. The user can assign a strategy—feature, repair or discard—and draw a design directly on the material. If a planned cut intersects a defect, UBench can project a contextual warning. Williams gave the example of a cut line passing through areas containing metal, which could pose problems if the woodworker used a circular saw. The aim is not to replace expert judgment with a sensor reading. It is to make risk visible at the moment of design and support learning about how to manage it.

This approach also shows why sensing uncertainty reaches beyond data visualization. A system has to translate a measurement into something a person can interpret and act on. The sensor may indicate a condition; the interface must help a woodworker understand what that condition means for a particular design.

Williams defines sensing uncertainty as the possibility that an event did not occur exactly as a sensor reports it. The underlying value may lie within a range rather than at the single value shown. Uncertainty can enter during measurement, processing, analysis or synthesis. In air-quality sensing, for example, sensors can drift as contaminants accumulate, environmental conditions change or equipment is mishandled. If those problems are not addressed, readings may confuse decision-makers.

To examine how interface design affects interpretation, Williams’s group developed nine visualizations of air-quality data by varying two dimensions: context and interactivity. Each visualization used one of three context types and one of three interaction types.

The visualizations converted particulate-matter readings to the US Environmental Protection Agency’s Air Quality Index, using its guidance and color scheme. For spatial context, the team used kriging to estimate values between sensors. Temporal views showed seasonal variation in monthly or seasonal groupings; sensor-only views used a gauge. Static visualizations used color and visual marks to encode uncertainty. Animation showed possible values across frames, and the interactive version let users step through those frames with a slider.

The team evaluated the designs in a crowdsourced, within-subjects study. Of 150 recruited participants, 103 passed the study’s data-quality checks. The researchers measured three distinct outcomes: whether participants interpreted the AQI category correctly, whether they could compare the reliability of two sensor streams, and how confident they were in those judgments.

The results did not point to one visualization style as best for every task. Static visualizations supported more accurate interpretation of AQI categories. Williams said participants were more likely to give inaccurate than accurate interpretations with the animated and interactive versions. But interactive visualizations better supported comparisons of reliability between sensor streams: participants were 1.7 times as likely to make the correct comparison as with animated visualizations and 2.1 times as likely as with static ones.

Spatial context, contrary to the researchers’ expectation, produced less accurate AQI judgments than temporal or sensor-only context. Williams offered a possible explanation: kriging can visually obscure a location’s reading through interpolation from neighboring sensors, so the color shown at a point may not reflect the actual particulate-matter concentration there. In the study, a location that might have appeared green instead appeared yellow.

The researchers also found that only four percent of responses indicated a lack of confidence. Williams reported little to no correlation between correctness during the training task and confidence in participants’ answers. She raised another open question about the gradient colors used to communicate uncertainty. The EPA’s categories use distinct color bins, but the study used gradients to show intermediate values. That choice may have made uncertainty more visible while making categorical judgments harder.

A question from the audience clarified what “accurate” meant in the study. Participants were not asked to recover an exact sensor value or select a particular color. They had to choose a semantic category, such as “moderate” or “bad” air quality, based on the AQI bins. When a range crossed categories, they still had to choose one. That measure captures categorical interpretation, not whether a participant’s choice fell somewhere inside the full uncertainty range. Comparing reliability between two sensor streams was a separate task, not another measure of AQI category accuracy.

Williams’s interpretation was therefore tentative: people may have been making judgments that reflected the range of possible values but did not match the category experts would assign. The results suggest a tradeoff. Clear, static bins may support more accurate categorical readings, while interactive displays may help users compare uncertainty. Hiding uncertainty can make a map easier to read, but Williams worries that it can also undermine trust when people later learn that the data behind a seemingly definitive display was uncertain.

The central issue, for Williams, is not simply how to make a visualization accurate or how to display a sensor’s error. Different interfaces invite different forms of reasoning. A static map can make a category legible; an interactive display can expose variation, but may also let someone fixate on a preferred interpretation. Animation may help viewers attend to a range of possibilities, though unfamiliar presentation styles can make interpretation harder.

These tensions extend to systems that rely on emerging, low-power techniques. Williams pointed to wireless and paper-based computing approaches that could reduce the need for batteries and the toxic residue associated with them. But sensors in such systems may themselves be uncertain. Users may need a role in interpreting noisy readings, while the system must still present information in a form that supports tangible interaction. Her group is also using participatory methods to understand how interface affordances can align people’s mental models with what the underlying data can actually show.

That agenda returns to the low-carbon innovation paradox. Reusing objects or components does not automatically make a system sustainable. Williams raised concerns about occupational hazards in reclaiming electronics from e-waste: working with components can expose people to toxic materials, and safe handling may require specialized knowledge and protective measures. Woodworking offered a more tractable setting for studying material risk, while still addressing how contaminants and imperfections shape design.

In discussion, Williams described modularity and easy-to-detach components as promising approaches for repair and reuse, particularly where parts such as batteries fail while others remain viable. But she distinguished that kind of replacement from a more radical second life: reappropriating a device or its materials for a purpose unlike the original. If scarce resources are being reclaimed, she asked, should they necessarily return to the same product category or industry?

Williams argues that sustainability research should not expect everyone to become a sustainability expert. It should instead develop forms of interaction that help people work with uncertainty in everyday settings: choosing how to use imperfect materials, interpreting data that may be noisy, and understanding the consequences of adding computing to familiar objects.

That requires more than presenting uncertainty as a number or disclaimer. Systems need to make uncertainty perceptible and available at the point where people act, while accounting for the limits of human attention and perception. They also need to leave room for people to interpret data and material conditions without implying that a sensor or interface has settled the matter.

Williams’s closing challenge is to consider what can be substituted, replaced or imagined otherwise. Treating uncertainty as a central research question, she argues, can make sustainable design more responsive to the actual materials, communities and decisions involved.