The Future of Context

For instance, you should be able to call up an explanation of why a post appears in your feed — e.g. because it’s related to one of your projects, it’s on a topic you’re interested in, and it’s been liked by someone you trust. There and then, you’d be able to dismiss a criterion (e.g. I’m not interested in seeing things about this topic right now) or, conversely, focus on it (swiping right on the explanation to ask for more on this topic). The feed would then alter its content based on your feedback and your user model (your currently active contexts) would be updated.

The user should have access to the list of his existing contexts and be able to explicitly select which ones should be active right now. For instance, if you’re spending a few minutes at work buying a gift for your sister, you should be able to easily select the contexts related to your sister and temporarily deactivate the ones related to your work, so that your browsing and searches use the correct information without it being polluted by non-relevant criteria.

Finding the appropriate contexts in the list for an interaction should be as easy as finding the appropriate data when you are in the right context. Contexts should be organised like your data is: hierarchically depending on their granularity (e.g. People I care about -> Family -> Sister), semantically (e.g. Gifting <-> People I care about), and they should even have their own meta-contexts (e.g. I buy gift for people I care about for their birthday). Contexts are the properties of your data that make sense to you.

Human-readable interactive contexts make up the language that allows users to communicate about their purpose.

Note: the above mock-up is just there to illustrate this concept. A proposal for a universal UI for navigating and interacting with data and contexts will be presented in another post.

Contexts as Shareable Objects

So, in our ideal system, contexts are objects that exist in the OS and that provide purpose-related information to the user’s services and apps on their device. But they don’t have to stay confined to the device. They can be shared with services and other users. Contexts can become the building blocks of conversations about purpose.

In the above example where you are shopping for a birthday gift for your sister, how convenient would it be if you could share the relevant contextual information with shopping sites or apps? You could inform them with a single action of which gifts you’ve given your sister in the past and what she thought of them, of her favorite hobbies and recent topics of interest, of her wishlists, of her favorite tastemakers, etc. The service, in response, could help you refine your search using the feedback mechanism described above: showing among suggestions which ones match which contexts and how well, even suggesting additional contexts as appropriate if it can identify patterns in the selection (e.g. price ranges, people who bought X, popular items in category Y, positive reviews from trusted users, etc.).

It would be like implicitly building a database query by conversing with the service, although not with data-related terms but with intention-related terms. All the cognitive work that we do on our own during such task would be instead supported and enhanced by an AI partner.

Of course, on the user’s side, this would require AI-augmented privacy right management tools (reacting to the active contexts), so that users could easily share only the data they wanted to make public to the service or anonymize parts of it.

Previous post: A Purpose-Driven User Experience
Apologies to my readers. This post is a bit tougher to consume than the previous one but, if you want to revolutionize UX, you have to grapple with some complex ideas.

Living in the age of context feels like we’re constantly under the gaze of benevolent overseers: Google Now tells us when to leave for the airport in order to catch your flight and account for traffic; our phone launches our “Exercise” playlist when we plug in your headset while going for a run; Facebook shows us the stories that it thinks we are the most likely to find engaging… But context is about much more than filtering and predicting, and we’ve only seen the very tip of the iceberg.

Context as we understand it today is software-controlled and opaque. In order to achieve a purpose-driven user experience, contexts have to become human-readable, user-controllable, and shareable objects.

In the common acceptation of the term, context is how the unwieldy amount of data a user is potentially exposed to each day is filtered and how services are surfaced to them when an AI predicts they need them. Ideally, when context filtering and prediction work, the user only sees things that are important or interesting to them, and they can easily access the next software features that are useful to them. In order to build models of users that allows such filtering to occur, devices and services draw from many heterogeneous data sources: calendars, app usage, contacts, message contents, searching and browsing habits, favorites and bookmarks, time, location history, various device signals, public and private APIs, all kinds of metadata, etc.

However, when all this data is analysed and patterns are identified, they remain out of reach of the users. Services make models of the users but the users have no visibility and no control over them. If a prediction is unexpected, they can’t get an explanation of why it was presented. Worse, if a prediction is wrong, they can’t provide explicit feedback to alter the models — like explaining that you’re vegan so that you won’t be presented with burger joints every time you’re looking for a restaurant. Or the model can be right but a useful context can be unavailable because the service is judging that it’s not appropriate at the moment — like wanting to switch to “work mode” at 3 a.m. on the week-end.

Contexts should have a human-readable interface that lets user browse, examine and edit them. When filtering or predictions occur, the user should be able to see which contexts are involved in selecting the current data and provide on-the-fly feedback to alter them, if need be.

On-the-fly human-readable contextual explanation and user feedback:
swiping a filtering criterion left to dismiss it.

Now imagine what such an intention-driven conversation could be like if it instead took place between several users.

Today, we share very little of our contextual information with other users. That’s why a service like x.ai exists, in order to facilitate meeting scheduling: it solves the problem of coordinating schedules by iteratively propagating constraints to the users and asking each of them to solve them locally (i.e. do these dates and time work for you? Can you suggest alternate times?), the process possibly involving many back and forth messages. It’s a testament to how poorly supported by our devices such tasks are that one of the best solutions out there can be so cumbersome. But it can be easily explained: x.ai doesn’t know enough about its users to get better results.

But our devices do. They have all this contextual information and their disposal and users can select which ones are relevant to the task at hand (just like in the shopping example). In one action, when scheduling a meeting with shareable contexts, they could communicate information about availability and preferences, future locations, preferred venue types or conferencing services, meeting agenda, links to relevant documents and conversations on the meeting topics, as well as any additional constraints they’d deem important. Again, this would be mostly automated by the context-managing AI and the users would be able to easily add or remove contextual criteria or change their importance.

From there, the recipients of the contextual information would be able to validate constraints — automatically if their devices could do it, like matching proposed times to availabilities in the calendar. They could also invalidate others or add their own. This would launch a real-time multipartite negotiation facilitated by each user’s AI, each participant editing the shared context for the meeting and seeing instantly the effects of their changes. The AIs would then ask the users to agree on one of the possible solutions, ranking them by global desirability (i.e. putting first the ones that satisfy the most important constraints for the most users). If a solution could not be found, the AIs would inform their users and suggest which constraints to relax first (because they would be aware of how important each constraint is for each user).

This technique could be applied to any collaborative endeavor: picking a movie among friends, prioritizing a project’s features, brainstorming, reviewing something, etc. And since the shared contextual objects would be persistent, they could even handle long complex tasks with many sub-tasks, like planning a wedding over several months, each stakeholder seeing and being able to affect only the tasks and constraints that the privacy-managing AIs would let them (e.g. a flower delivery service would have no say about the date constraints and a guest would not see the tasks related to flower delivery).

Shareable contexts open up a whole new world of interactivity and AI-enhanced collaboration.

We’ve had general constraint solvers since Prolog in the 70’s. This design does not require inventing a new AI technology. But what it needs is the correct data to apply known AI techniques to: human-readable interactive contexts.

First published on Sep 7, 2016

Next: On-Demand Serendipity