Digitizing Food by Digitizing Scent

Digitizing Food by Digitizing Scent Digitizing Food by Digitizing Scent A child experiencing the first digital scent book, the oBook, exhibited at the Museum of the Moving Image in 2015. The oBook combined a digital version of the Goldilocks story with fruit and vegetable aromas, experienced on the oPhone, a digital scent device invented by David Edwards and Rachel Field, with the intention to flavor educate children about the eating of fruits and vegetables. The ability for humans to craft fragrance (not to say flavor, which until recently was best delivered by the act of eating) is truly old technology — dating back to 4000 years ago with the creation of incense by the Mesopotamians. A first technological advancement was the creation of liquid perfume attributed to the Greeks. By the 17th century, perfume was prolific, masking much of the less pleasant aromas of urban living. In the 19th century, chemists began to isolate individual aroma molecules to make the production of perfume much more affordable and creative, produce combinations of olfactory molecules not possible simply from raw materials. In the 20th century, multiple technologies debuted to deliver scents on demand and in sequence and to incorporate scent with audiovisual technology. Why does this matter to food? As other contributions to this issue make clear, the experience of food is very significantly olfactory, and digital food, the delivery of an apple (or an emotion) by an email, might be part of the future of hospitality. Perhaps the most famous example of audiovisual scent was Smell-O-Vision — essentially comparable to a slide projector for smell. But why, as audio and visual delivery has improved so massively in the past decades, has scent technology lagged behind? And how can these challenges be addressed? There are a variety of factors involved that are specific to scent among the sensorial experiences, most prominently: Scent is messy! A scent consists of a combination of olfactory or aroma molecules. Different than a sound or light wave, a molecule has a certain staying power, hence the obvious correspondence between the amount of perfume someone uses (or the amount of food one eats) and the intensity of the perceived scent (or flavor). When someone douses themselves in perfume, there are quite literally more molecules of aroma present, and thus statistically those molecules will take longer to fully dissipate. This means that if one is trying to time the arrival of scent with, for instance, a movie or audio — or a meal — one needs to account for the amount of time from the production of the scent until it reaches an individual’s olfactory receptors. And, to release another scent in sequence, one needs to know the exact amount of aroma molecules to use to have the desired clearance rate. On top of this, the sensation of scent changes as you are exposed to it — our nervous system adapts to the aroma rather quickly, so that the intensity, even of a constant concentration of scent, diminishes with time, typically on the order of seconds to minutes. Hence the phenomenon of an individual who adds more and more perfume until it may knock you right off your feet, but to them, it is barely detectable. Scent is personal! A 2013 study showed that olfactory stimulation triggers more brain activity than visual stimulation (Arshamian et. al., 2013). We experience this regularly in the real world: the smell of a chocolate chip cookie is more compelling than the picture of a cookie. This is likely because when we come into contact with an aroma molecule, the olfactory receptor cells in your nose send a signal to the olfactory bulb in the brain which directly connects to your hippocampus, a region linked to memory and cognition, and amygdala, the area of the brain responsible for processing emotion. Scent is effectively a direct signal from the world to the brain with stops at the memory and emotion centers. These neural connections explain the phenomenon of “olfactory time travel” — the powerful ability of an aroma to trigger a very specific and robust memory of a person, place or event with which that odor was associated. In 2012, the oPhone, as in “olfactory Phone,” was developed: a device that connected to your smartphone and could play 32 “base” scent notes. Much like a musical instrument, the oPhone could layer these scent notes to create a more complex olfactory experience or could be programmed to play scents in sequence, a “scent song” of sorts. We brought the technology into a range of environments, from cultural spaces to workplaces. The creativity of how people programmed the device was amazing and revealing of how people engaged with scent. For instance, in a workplace installation people programmed scents to match songs played while they worked: for a 3 PM pick-me-up, they chose “productive” songs and “calming” scents. Perhaps our most interesting experiment was to make an app that allowed people to tag photos with scent, which would then be “played” from the oPhone. Would people use it as perhaps an enhanced food menu or the ultimate Yelp experience? Some people, yes. But many more took pictures of non-foods and associated aromas that were figurative — chocolate for the little girl, jalapeño for the little boy, and meaty for the father’s big stomach on which the little boy’s head lies. These interpretations, linking the visual and the aromatic in a figurative way, demonstrate the emotional associative properties of scent. Scent is complex! As Linda Buck and Richard Axel published in 1991, there are approximately 1000 genes for scent receptors, which encode an equivalent number of olfactory receptor types. In humans, only about 1/3 of these genes are functional, while in some species most all of them are used! These receptors are distributed across approximately 6 million specialized sensory nerve cells that reside in the upper part of the nose. Compare this to three known types of photoreceptors in the eye! In addition, many of these receptors vary in ways that make them more or less sensitive to the preferred odor, and people may have a unique portfolio of receptors rendering their experience of aroma as individual as their fingerprints! On top of this genetic individuality, our sensory apparatus is exquisitely adaptable to our environment and our experience. Regular exposure to an odor changes our sensitivity to it. Of course, it is important to distinguish between an odor and the perception of an odor. For instance, the average coffee scent contains 727 aromas — although an individual typically only needs to perceive 23 aromas to distinguish coffee. So, when approaching the design of the oPhone or another method for digitized scent relay, it’s crucial to consider what extent of scent complexity is necessary or even understandable. So, what are the next steps for digitized scent? The discovery by Isaac Newton that white light was composed of individual colors, and that you could recreate white by combining red, green, and blue light laid the foundation for creating the color wheel and mapping the perception of color to a fundamental property of light: wavelength. Being able to map wavelength to color enables the digitization of color images. Similarly, frequency can be mapped to pitch to enable the digitization of sound. Because the olfactory system is much more complex than vision, it has been a challenge to identify a set of “primary” odors and their mixtures that would enable creating an aroma wheel or map. Without this map, we cannot reliably recreate a scent based on digitally transmitted information — if I simply said, “This smells fishy,” — that would cover an “odor space” including 10s to 100s of possible individual odors and combinations. Our descriptors are simply inadequate. In addition, simple chemical structural information such as whether a chemical is an acid or an alcohol, a long- vs a short-chain fatty acid, has not been easily translated to predicting odor quality. A mathematical code — like what underlies the color wheel — that describes the full perceptual experience is required. In olfaction, we don’t yet have a color wheel, but we’re working on it. The first leap was discovering that the digital solution to the problem may not be as complex as the biological solution. We may not need to replicate the precise activity patterns of 100s of receptors to deduce the code that translates from chemical information to sensory experience. The foundational data that we need is being built from very large datasets comprised of odor chemical and human sensory information. We have learned that the type of sensory data needed can best be obtained from trained human subjects describing their perceptual experience of 100s of distinct odor chemicals in a very standardized way. This information is combined with information about the chemical and physical properties of the odor molecules. Then, artificial intelligence systems are applied to deduce the “rules” used to translate chemical information to sensory experience. These rules (the “code”) can be used to predict the aroma quality of a novel chemical — or even whether that chemical has an odor! Most recently, a leap forward was made by a team led by Joel Mainland of the Monell Chemical Senses Center and Alex Wiltschko at Google Brain. Using advanced machine learning techniques, these researchers successfully created the beginnings of a map that links a chemical’s structure with the aroma perception it may elicit. This map performed as well as trained humans for a set of individual odors, including chemicals that had never been smelled before. The map was also able to predict how similar two chemicals would smell. However, much work remains. Chemicals that exhibit the same physical properties but are mirror images (“enantiomers”) can smell different, yet the map failed to predict this. More importantly, most of the scents we experience are mixtures, and these mixtures can evoke even more complex patterns of receptor activity which interact in ways that we cannot yet predict. But this work has demonstrated the immense power of artificial intelligence techniques to solve these complex problems and lays the foundation for scaling up to larger and more complex aromas. Indeed, the die is cast for the challenge of digital scent to be overcome within the foreseeable future. Given the emotional power of scent and the increasing remoteness of our society, as accelerated by the last few years, it seems all the more important to integrate this sense into the digital world. What does all of this mean for the future of food? When the oPhone debuted it did so by way of an app called oSnap. The app gave people the chance to take a picture of any food, associate it with an aroma, and to send this picture to anyone in the world with the aroma included! If people had the oPhone they could receive the picture with the smell of the food. We had a limited vocabulary of transmissible aromas at the time. Actually, we found that aroma vocabularies were so personal, so complex, that we needed to focus aroma cartridges around themes. We launched the app as a small startup in Paris. Tens of thousands of oSnaps later what did we find? Smells like chocolate, meaty, and woody found surprising interpretation — the photo of a boyfriend got attached to chocolate, of a father with a big stomach on which a little boy had fallen asleep snapped to meaty, and a sad-looking hamburger was assigned dirt. The emotions people associated with food aromas mattered more to people than the literal association of the aroma to the food itself. From a practical perspective for the future food world, digitizing smell will enable the creation of new palates of olfactory experience that we cannot even imagine today — by putting tools in the hands of flavorists, food creators, and chefs that allow them to explore a myriad of new combinations and formulations in a fraction of the time and cost of traditional methods — to focus on more sustainable, health-promoting options with specific, desired nutritional and sensory characteristics. In hospitality, imagine going into a hotel room and immediately feeling truly relaxed and at home — because you sampled the ambient aroma in your own bedroom and entered into their personalized environmental choice system when you made your reservation! For me, that might have notes of cat, rose-geranium, and toast — and as morning approaches, the toast gets stronger, rose-geranium fades toward citrus, and coffee begins to build! I’m ready for the day! There is an opportunity in the future of food to enter the MetaVerse. With digitization science, flavor, and food experiences might enter multi-dimensional fully immersive online life, bringing restaurants into MetaVerse land, not only as visual and cognitive experiences but as metabolically and sensorially relevant ones, accessible to anyone, anywhere.