Interactive Telecommunications (ITPG-GT)

What can computation add to human communication? Creating computer applications, instead of just using them, will give you a deeper understanding of the essential possibilities of computation. Conversely excitement about your computational project ideas will best propel your acquisition of skills necessary to realize those ideas. This six week course is divided in two parts. The first portion starts with the expressive capabilities of the human body & how we move through the world. The Physical Computing skills will allow you to go past the limitations of the mouse, keyboard & monitor interface & at locations other than the home or the office. The platform for the class is a microcontroller (Arduino brand), a very small inexpensive single-chip computer that can be embedded anywhere & sense & actuate in the physical world. The core technical concepts include digital, analog & serial input & output. The second portion of the course focuses on fundamentals of computer programming (variables, conditionals, iteration, functions & objects) as well as more advanced techniques such as data parsing, image processing, networking, computer vision & 3D graphics. The Java-based ‘Processing’ programming environment is the primary vehicle. Processing is more oriented towards visual displays on desktops, laptops, tablets or smartphones but can also connect back to the physical sensor & actuators from the first part of the class. The course is designed for computer programming novices but the project centered pedagogy will allow more experienced programmers the opportunity to play further with their project ideas and make lots of friends by helping the other students. Note: This course is for non-ITP students.

"This introductory class contains the seeds of your ITP experience! This course is designed for you to… -See how the things you learn at ITP can be applied across many different fields), and to begin to develop your own nuanced point of view on those applications -Think through the lens of designing engaging experiences, through a process of observation and articulation as a starting point for an iterative design practice -Practice collaboration in a way that emphasizes generosity, curiosity, communication, and an openness to take the road you don’t usually travel. -Build a shared language with your cohort - about ITP, about your interests, about emerging technologies and ideas. Get to know each other! In Applications you’ll get a glimpse the foundational character of the program, which emphasizes: experimentation, play, thoughtfulness, emergence, interdisciplinarity, creativity, iteration, and collaboration Each week, we will be visited by a guest from a field in which skills and ideas you might encounter at ITP are applied. In response, two groups per week will create an experience for the whole class, inspired by the prior week’s talk. By way of this process, all first-year students, for the first and only time in their ITP experience, are together in one room at one time, and as a community, encounter, and respond to, the challenges posed by the invited guests. There will also be room for discussion and other small responses assigned as the need for them emerges - the class itself acts as a model for iterative design. In Red Burns’ words (the founder of ITP and the original instructor for Applications): <Applications> is a reminder that without a problem to solve or a message to convey, a skill is void of meaning and direction. Technology does not make a community. ITP is about enhancing the human spirit, using the power of technology to respond to the human need for communication and expression."

This course explores the fundamentals of storytelling through animation. Students will create short animated pieces over the course of seven weeks. The first part of the course is devoted to the stop motion using Dragon Stop Motion. The second part of the course is devoted to digital collage animation using After Effects. The third portion of the course will experiment with 3d characters and game engines. Drawing skills are not necessary for this class. Basic video and sound skills are required and students should take Comm. Lab: Video and Sound as a requisite for this course

Storytelling is a powerful method of communicating messages and engaging an audience. Whether you are building an expressive narrative world, documenting existing work, or pitching a new project, a carefully crafted story is a powerful way to connect with others. How do new technologies change the way we tell stories? Communications Lab: Hypercinema will examine new forms of communication and storytelling through an investigation of time-based media. This course will cover principles of interpersonal communication, media theory and collaborative storytelling through a series of weekly assignments and labs. Over the course of the semester students will become comfortable with the basics of time-based media, and examine the contexts in which this media exists in society, both historically and in current practice. Topics include: sound art, animation, immersion and interactivity, and real time 3d game engine environments.

Until recently 3D sound was a novelty reserved for special uses and reaching a limited audience, no medium in popular culture has been as inherently dependent upon spatial audio as virtual reality. The widespread and standardized implementation of surround sound in film brought cinema to a new level of immersion, but is limited to theatrical exhibition and home theater systems. Today a considerable amount of content is consumed on mobile devices and laptops which excludes the cinematic experience of spatial sound. With the current rise of cinematic VR and the blurring line between gaming and experiential VR, spatial audio is no longer just an added bonus, but rather a necessity in designing immersive VR experiences. In this course we will explore the emerging field of 3D sound design and for both 360 video and game engine-built VR using a digital audio workstation, Unity, and 3D audio plugins.

The goal of this course is to teach students about product design, development, and commercialization through the specific lens of food production. Today, most of the world would be unfed were it not for a few technology breakthroughs from the mid-20th Century. But global agricultural output is peaking. The one remaining unused resource is land: at this point, growing more food is mostly a matter of destroying more forestland, irrigating the resulting arid wasteland with water sourced from rapidly depleting aquifers, and contaminating both the land and water with chemical fertilizers and pesticides that destroy ecosystems near and far. Meanwhile, the world will add another 2 billion eaters in the next 20 years. How do we feed the world without destroying it? The agricultural innovations of the 20th Century were primarily chemical and industrial. As a result, agricultural technology is huge and expensive. Meanwhile, the information revolution of the past 25 years has gone largely unnoticed by the agriculture sector. It’s a domain ripe for disruption by things that are small and cheap: Precision hardware that applies chemicals only where they’re needed. Monitoring tools that alert distributors to rot. Sensor networks that tell farmers when to irrigate, and when not to. Also, drones.

Reengineering current technologies to solve new or different problems is a design process that humans have adapted for centuries. This course explores socially-driven alternative uses of existing technologies, and carves out the space for engaging discussions about the future of responsible engineering and critical design. “We cannot solve the problems we have today by thinking in the same way that we did when we created them.” -Albert Einstein

Data is ubiquitous. Yet, it's often invisible. In this course, we will explore ways to create physical data visualizations using contemporary design and digital fabrication tools. Students will learn how to collect data, find interesting patterns, design creative digital models and build tangible pieces using laser cutters, 3D printers and woodworking tools. We will visualize everything from street performers in Washington Square to Instagram influencer trends. Topics related to creative coding, Arduino, artificial intelligence, projection mapping and traditional art-making techniques will also be discussed.

Knitting, a relatively modern (only eleven to twelve hundred years old) technique for making stretchy fabrics, got its beginnings in the Middle East, and spread via trade routes into Europe, and then on into the rest of the world. Knitted items were initially a rare luxury, and then sailors and peasants learned to knit and make things for themselves, and the art took a turn into the realm of a common handicraft. Later, in the 1500s, machines were created that could knit, but it was still cheaper for people to make their own clothes. With the introduction of synthetic fibers, along widespread colonizations and the globalization of the garment industries, manufactured knitted clothing became so inexpensive as to be disposable. Today, hand-knit (and even hand-drawn knitting machine made) items straddle the line between handicrafts and luxury items (and the process sometimes becomes a performative art). With the use of new/non-traditional materials, knitted objects are also being used in healthcare and industry.

Want to make an interface that can be squished, stretched, stroked, or smooshed? This course will introduce the use of electronic textiles as sensors. Focus will be placed on physical interaction design - working with the affordances of these materials to create interfaces designed to invite or demand diverse types of physical interaction. This course does not require knowledge or love of sewing - a variety of construction methods will be introduced. It will rely on a physical computing approach, with Arduino being used to read sensor values. Working with a breadth of conductive and resistive materials, students will learn to design and create bespoke alternative interfaces that can live in our clothing, furniture, and built environments.

"Research is formalized curiosity. It is poking and prying with a purpose.” - Zora Neale Hurston “Necessity is the mother of invention, and research is her midwife.” - Anonymous Research is both fun and serious. It involves playful, structured exploration of the unknown, as well as critical appraisal of existing thoughts, beliefs, and systems. It is essential for the conception and development of new ideas, new connections, new processes, and new products, as well as informed decision-making and active engagement in civic life. Doing research is more difficult than a Google search, but it’s vastly more rewarding. In this class, we will do research involving text, images, sound, and multimedia. We will investigate where different kinds of information come from; how they are organized; and thus, how to find and evaluate them. We will examine the social and economic forces that shape knowledge production/distribution, and various issues of ethics and social justice as they relate to access and use of information. Upon completion of this course, students will have in-depth experience and familiarity with locating print/analog and digital information sources. They will be able to document these sources using Zotero (an open source reference/research management tool). They will also know how to identify/hone topics of interest, how to keep a research diary, and how to write a literature review

Time-based art, performance - and theater most specifically - should be perfect manipulators of experience. Many creators of time-based art look for the “universal” in content and overlook what we all have in common in form: brains. What happens in the minds of all truly happens (what happens in the lobby also truly happens). How can we use art to make our brains experience the same things? What behind-the-scenes work can we employ to manipulate experience. Film scoring works on us in ways we don’t perceive in the moment. Can we pay closer attention to this when making work? Sometimes what we have for lunch effects our experience of art more than the hours of labor that went into the most minuscule of decisions when making that art. Is empathy always ethical? Show don’t tell. How does the art take place in the room in front of us? How can public performance not be awkward? How do we experience a performance rather than watch it.

In an age of pressing and complex problems like climate change, extreme inequality, and surveillance capitalism, “problem solving” is a central feature of innovation, design, and planning. But can these wicked problems actually be “solved”? And why does the cutting edge of problem solving look so limited? Machine learning. Predictive analytics. Algorithmic decision-making...Is planning for the future being outsourced to machines? In this class, we’ll take back control of the future by learning how it has historically been predicted, planned, and produced in board meetings, think tanks, writers’ rooms, and policy circles, and how those methods are being impacted by new technologies. During a series of discussions and hands-on workshops, we will learn specific, tangible, and collaborative practices for prediction and planning that can augment and transcend computational capabilities, making for marketable future-proof skills that can help redefine the future for humanity.

Designers are at the forefront of shaping space and have the power to reinforce or destabilize inequitable power relationships with space. With that context, how can we design for human equity? What does it look like to co-create spaces/projects under the pressure of gentrifying forces? How might technology and design be anchors for safe and equitable spaces/projects of the future? This course will explore these questions and the multiple facets that complicate and enrich design processes within communities. We will learn about models of community design center practices, civic vs. community participatory engagements, and the systems, tools and actors needed to produce legible works within a place to design for an inclusive future.

This open studio class is for students wishing to explore the design and construction of living systems: both terrestrial and aquatic. The labs and lectures are designed to lend clarity to the individual’s pursuit, as they create their chosen ‘World in a Box.’ For those endeavoring to create ‘smart’ systems; instructor developed infrastructure will be made available to support and accelerate project velocity. Our toolkit for exploration and expression will include Raspberry Pi, building and deploying applications on Google Cloud Platform (authored in python), designing systems in Fusion 360, remote system monitoring, and motor control.

Today we no longer solely connect to the digital world through computers. The result of this push to connect the digital and the analog world is the increase necessity for low cost, low power, and self-contained electronics. This course is an applications-driven intro to electronics for inventors. Through a hands-on approach students will learn basic concepts about analog circuits, boolean logic, digital devices interfaces, and low-cost code-free electronics. Topics will include basic principles of electricity, as well as understanding of electronics components such as resistors, capacitors, diodes, transistors, audio amplifiers, and timers.

Is augmented reality technology about to enter the mainstream? AR platforms have finally become widely accessible to artists, designers, and technologists thanks to recent advances in mobile performance and a new collection of powerful computer vision techniques. As such, the medium offers rich possibilities for experimentation and a chance to rethink how we experience the intersection of the physical and digital. In this course, students will acquire an understanding of basic concepts and techniques necessary to prototype and build simple AR experiences - with a consideration of not just visual but also aural AR. We’ll supplement practical exercises with an overview of the history of AR, and discuss the ethical, legal, and societal considerations cropping up around this topic. Our tool of choice will be Unity, but we will go over prototyping techniques outside of the platform to speed up the design process. If there is interest, we will cover how to get started building projects in openFrameworks, mobile, or web AR - and discuss why or when you might want to work within other platforms. Even though code samples will be provided, students are highly encouraged to have a basic understanding of Unity or at least have taken an introductory programming course. A working knowledge of Unity can be gained through Unity tutorials (https://unity3d.com/learn/tutorials) or Lynda (https://www.nyu.edu/lynda).

This class presents the diverse trends of light art. This includes film and animation systems, light systems, and visual information systems; their context, meaning, and manipulation. We will look at the historical relationship of technological discoveries on artistic methodologies and ways communicating information and ideas. The core of this course lies in ways of seeing; breaking down the physics of light and human perception, to the cultural, conceptual, political, and art historical context of these visual systems. The production work will include light manipulating materials and systems, the technologies influencing how we see, and methods on how to control and manipulate these systems.

A programming course where we'll explore various techniques and solutions for tracking and sensing people or objects in space. Students will get familiar with the terminology and algorithms behind many sensing topics such as computer vision, depth cameras, positional tracking, and coordinate mapping. As these subjects are explored, we will also dig into communication, and how this information can be transmitted from one tool to another, for example using OSC, Spout/Syphon, MIDI, DMX/ArtNet. The goal being to use the right tool for the job and not limit ourselves to a particular piece of software.

Time is at once fundamental and mysterious. From the 2000-year-old Antikythera Mechanism to modern cesium-fountain clocks, humans have long sought to understand temporal patterns in nature, and build mechanisms to measure, reflect and predict those patterns. We're at a unique moment, one in which we've developed the ability to perceive relativistic effects on time at the smallest scales, while struggling to think and plan across generations. In this course, we'll reflect on the deep mysteries of time while also gaining hands-on skills applicable to temporal media and technologies. Topics will range from historical clock and orrery design through modern computer architecture ("A computer is a clock with benefits" writes Paul Ford in Bloomberg's issue dedicated to code). Practically, we'll build mechanical and software clocks; experiment with time-series data and time protocols; and survey techniques for digital signal processing and real-time operating systems. Students will execute several short assignments and a final project.

Course description (optional): We are currently living in a society that operates under the principle that one body equals one agent, one vantage point, one identity. But emerging technologies may create a future in which the notion of a single personal identity becomes outdated. That future includes: machine learning techniques that make emulating the style and behavior of other people fast and easy; widely available AR/VR headsets that get people to identify with however many faces and bodies they choose, instead of just those they were born with; cryptocurrencies enabling the use of pseudonymous economic identities to transact across the planet in a permissionless manner. This is a course where we will get to explore and anticipate the utopian and dystopian aspects of this weird future of identity.

Course description (optional): What does it mean to feel joy while playing a game? How is it distinct from fun, and what can it bring to the table--both to our design practices, and the world at large? In this class, we’ll be exploring how to create mechanics and interactions that invoke a visceral, rather than purely intellectual, sense of delight. From Roger Callois’ definition of ilinx to Bernie DeKoven’s transcendent collaboration and beyond, we’ll dive deep into finding new and weird ways to make games, toys, and interactives that spark joy and facilitate connection--among both individuals and groups. We’ll also pull in readings from moderns arts practitioners and activists thinking about the politics of joy and pleasure, and what joyful games can bring to the world. The course will involve weekly design exercises, using a variety of paper prototyping and digital production techniques. We'll define games broadly, welcoming a variety of playful, interactive approaches and outcomes. Assignments will include a blend of reading, writing, making, and playing (low-cost/accessible titles), to help students sharpen their prototyping skills as well as their ability to hone in on meaningful interaction mechanics. The primary coding platform for lessons and examples will be 2D and 3D games in Unity, but there will be options to explore platforms like Twine and Tracery, or delve into experimental media like augmented reality. For their final project, students will develop a game/interactive in small groups.

This course will provide students with an introduction to the area of machine listening. Machine listening is the general field studying algorithms and systems for audio understanding by machine. It deals exclusively with general audio as opposed to speech recognition. The most basic goal of all machine listening systems is to reliably recognize and react to very specific sounds. Over the course of the semester, we will create our own unique machine listening systems that provide us with new and interesting ways to interact with our projects. We will use live coding and real-time data visualization to demystify some of the more daunting underlying topics like digital signal analysis, music information retrieval, and machine learning.

Masks have been used around the world since antiquity for ceremonial and practical purposes, as devices for protection, disguise, entertainment and bodily transformation, made to be worn or displayed. Sociologist Erving Goffman wrote about the everyday life as a masked theatrical performance. The performative aspect of our lives today is ever so present in our use of social media, where we present a curated version ourselves for the immediate visual consumption of others. In our `Selfies`, we can assume a multitude of identities and characters. Recent tools and platforms have evolved social media portraiture to an art form and have created new opportunities for artists to create and distribute interactive augmentations, forming new relationships between artists and viewers. This class explores the developing language of social media portraiture enhanced by Augmented Reality. Students will: - review masks in art history, leading up to today - ideate, design and develop an interactive mask (AKA effects/lenses/filters) - learn to use the Meta Spark software to create AR effects. This course requires CL: Hypercinema or equivalent experience.

This class focuses on the exciting creative possibilities of emerging ambisonic, photogrammetric, volumetric, and depth capture technologies including their respective applications within XR. Instead of creating scenes, objects, and characters on a computer, more compelling and resonant opportunities for experiential storytelling can now be produced by recording real environments, things, and people. Students will be asked to stretch their imaginations, embrace the distinct technical and aesthetic affordances of each capture technology, and then weave them together into experiential montages. The format of the class will change week-to-week but combine lectures on experiential storytelling, group critiques, and hands-on workshops diving into ambisonic audio recording, monoscopic and stereoscopic 360 video, DSLR photogrammetry, and volumetric video capture. All of these tools are becoming part of a greater developmental ecosystem for evolving immersive and social media platforms. Students will then stylize and blend captured assets within Unity to create a demo or project based on an original idea. While various exercises will help familiarize students with capture technologies presented in class, as well as how to utilize captured assets in Unity, experiential design will still be paramount. A major component of the curriculum will be the discussion and critique of student ideas as well as a presentation of a final project. This combined conceptual and technical focus will challenge students to experiment with an expanded sense of possibility, while also preparing them to pitch ideas for labs, residencies, and festivals.

This course is intended for students planning to conduct qualitative research in a variety of different operational settings. Its topics include- case studies, data, documentary evidence, participant observation, surveys, and supportive technologies. The primary goal of this course is to assist students in preparing their thesis proposals/ projects. Description: A survey of creative and qualitative research methods applicable to the design, media and visual arts in practice. Purpose: To assist current graduate students in comprehensive research processes and practices and, additionally, to initiate students' preparation for executing scholarly activities and descriptive analysis. A third objective is for aspirants in the field to learn to critically unpack both quantitative and qualitative research. Methodologies: Research requires persistence, creativity, ingenuity, and refinement. There are several different types of design-based research directions that include experimental, analytical, investigative, and etc. Students will be introduced to a full range of possibilities and world-views. The course supports advanced students at ITP to evaluate and develop novel methods and methodologies specific and appropriate to their personal research projects at this level and in the context of a practice-led research environment. A key objective is to promote innovative, experimental and ambitious research that relates to academic study and creative practice at the forefront of the field of contemporary art/ media/ design and related disciplines.

In the 1960’s and 70’s, club culture was a social phenomenon that introduced new forms of technologies, experiences and designs. In this course, students will study a variety of different cultures (from social, to visual to sound) from pre 1600s to present day. These learnings will realize historical patterns and psychologies displayed throughout the decades. These social investigations are generally practiced through a United States lens, but students will also be invited to explore how similarly communities were developed around the world due to a variety of parallel conditions. Relationships between counterculture movements and visual and sonic creations will be examined. Students will then experiment with audio-visuality and study how light, sound and color and other senses are used to design an experience within different spaces, particularly spaces for entertainment. Students will be invited to launch 370 Jay St's performance garage in development of their own temporary club environments. Technologies introduced in this course would then include Cinema4D, MaxMsp (and potentially Touch Designer), Ableton, and Photoshop.

The 17th century philosopher Spinoza described "wonder" as a state of suspension in the mind, a paralysis resulting from a confrontation with something wholly new, disconnected from past experience such that judgements of whether it is good or bad are not possible. At this moment in time, we are caught in such a state of suspension with digital technologies. Creating computer applications instead of simply using them will provide you with a deeper understanding for the essential possibilities, limitations and unknowns of computation. The first half of Introduction to Computational Media focuses on the fundamentals of programming the computer (variables, conditionals, iteration, functions, and objects) and includes a basic introduction to HTML5/DOM. The JavaScript-based 'p5.js' programming framework is the primary vehicle for the class. All sections assume no programming experience at all. The second half focuses on applying fundamental programming concepts to generate and manipulate various media including imagery, sound and text and data. This course assumes a working knowledge of JavaScript and the 'p5.js' programming framework. Students who already have experience with programming in JavaScript and p5.js may join the course in the second half. Pre-requisite: ICM: Code or equivalent programming experience.

Introduction to Critical Mapping and Open Source Geospatial Web Analysis and Visualization is an introduction to critical perspectives in cartography and geospatial information systems and web technologies. This course will introduce students to the foundations of geographic data analysis and visualization, grounding practical studio based exercises and projects with critical readings and theory. Students taking this course will gain an appreciation for geographic thinking, learn to ask geographic questions, and apply basic methodologies to “make sense” of geographic data. The course will be conducted with open source geographic information systems and web technologies. Students are not expected to enter the class with knowledge of these tools, but will be expected to learn and apply them through studio exercises and project briefings.

With Machine Learning models are getting smaller, and microcontrollers are getting more computing power, Machine Learning is moving towards edge devices. This class explores the idea of how machine learning algorithms can be used on microcontrollers along with sensor data to build Physical Computing projects. In this class, we will learn about TensorFlow Lite, a library that allows you to run machine learning algorithms on microcontrollers. We will talk about common machine learning algorithms and techniques and apply them to build hands-on interactive projects that enrich our daily lives. Students will learn to use pre-trained models, and re-train the models with sensor data. We are going to talk about Image Classification, Transfer Learning, Gesture and Speech Detection. For each topic, we will first discuss its history, theory, datasets, and applications, and then build simple experiments based on the topic. Prospective students are expected to have taken Introduction to Physical Computing and Introduction to Computational Media course, or have equivalent programming experience with Arduino and JavaScript.

Language is more than just words and meanings: it's paper and ink, pixels and screens, fingertips on keyboards, voices speaking out loud. Language is, in a word, material. In this course, students will gain an understanding of how the material of language is represented digitally, and learn computational techniques for manipulating this material in order to create speculative technologies that challenge conventional reading and writing practices. Topics include asemic writing, concrete poetry, markup languages, keyboard layouts, interactive and generative typography, printing technologies and bots (alongside other forms of radical publishing). Students will complete a series of weekly readings and production-oriented assignments leading up to a final project. In addition to critique, sessions will feature lectures, class discussions and technical tutorials. Prerequisites: Introduction to Computational Media or equivalent programming experience.

Inspired by the Japanese art of Chindōgu, this class will introduce a playful and whimsical approach to learn industrial design. In this 14-week studio format class, students will develop gadgets, inventions, and electronic devices that present absurd solutions to problems, while learning concepts and techniques of design ideation, prototyping, model making, CMF (color, material, and finishes), and manufacturing. This is a production heavy four-credit course, where students will learn about industrial design and tangible interactions.

In this course, we'll investigate two closely related ideas: public data and data publics. We'll learn how to access and represent data from an assortment of existing public data sources, how to liberate currently obfuscated data sets, and how to create our own useful/whimsical/critical APIs from scratch. We'll also investigate the act of putting data into public space - through sculpture, projections, performance, and participatory interventions. Particular attention will be paid to methods which bring data back to communities from which it was collected, and to tactics which build grassroots literacies, respect local ways of knowing and reinforce capabilities for stewardship and sovereignty.

Generative machine learning models open new possibilities for creating images, videos, and text. This class explores the idea of how artists, designers and creators can use machine learning in their own design process. The goal of this class is to learn and understand some common machine learning techniques and use them to generate creative outputs. Students will learn to use pre-trained models, and train their own models in the cloud using Runway. For each week, we will discuss the history, theory, datasets, application of the machine learning models, and build experiments based on the model. In addition to Runway, we will be using JavaScript libraries like the p5.js, ml5.js, and TensorFlow.js, and software like Photoshop, Unity and Figma. Students are expected to have taken ICM (Introduction to Computational Media), or have equivalent programming experience with Python or JavaScript. A list of ML models we will be covering: Image generation: StylanGAN: https://github.com/NVlabs/stylegan BigGAN: https://github.com/ajbrock/BigGAN-PyTorch Style Transfer Fast-style-transfer: https://github.com/lengstrom/fast-style-transfer Arbitrary-Image-Stylization: https://github.com/tensorflow/magenta/tree/master/magenta/models/arbitrary_image_stylization Semantic Image Segmentation/Synthesis Deeplab: https://github.com/tensorflow/models/tree/master/research/deeplab Sapde-coco: https://github.com/NVlabs/SPADE Image-to-Image Translation: pix2pix: https://phillipi.github.io/pix2pix/ pix2pixHD: https://github.com/NVIDIA/pix2pixHD Text Generation LSTM gpt-2: https://github.com/openai/gpt-2

Touchless devices and systems have become an essential part of our built environment—for example, automatic doors, automatic toilets, faucets, hand sanitizer dispensers, thermometers, and even paper towel dispensers have sensors that allow them to detect when they are needed. You might have experienced interactive artwork or exhibits that are activated responding to your presence and body movement—for example, interfaces where people can type with different body postures, musical instruments that you can play by waving your arms in the air, or mechanical systems that respond to your breath. Intangible interactions are those that we engage in without involving direct physical contact. Intangible interfaces don’t have a tangible form that explicitly instructs us how to interact with them, and these interactions utilize other forms of feedback than those we feel through touch. While technologies used for intangible interaction such as sensors and computer vision are now more available and accessible, philosophy and knowledge around the design and implementation of effective intangible interactions is a much less documented subject.

The ways in which we communicate has changed radically in the last 100 years. As the communication systems we use have increased in complexity, so has the effort it takes to understand how they work. Most of us use protocols like LTE, HTTP, TCP/IP, and BLE every day. We take them for granted, almost like we do the laws of nature. But there are more than the laws of physics, more than techniques of engineering, embedded in the design and implementation of our protocols of communication. To understand their role in our lives, we need to look into the societal and economic contexts in which they came to be. In this class, we will examine communication protocols using Raspberry Pi’s, Arduinos, Software Defined Radios, and other connected devices. We will look closer at organizations like iSOC, ICANN and IEEE to better understand how protocol designs are implemented and standardized. Through readings, research and hands-on work we will build an understanding of how these protocols work, how their designs incorporate the physical, technical, cultural, corporate and political assumptions of the actors behind them. In the first half of the class readings and assignments will help familiarize students with some of the different protocols we rely on every day. In the second half, students will work in groups to investigate a communications protocol and consider its impact from a technical, societal and environmental perspective. Final projects will communicate their findings in whatever form students deem appropriate - explanatory blog posts, physical or digital installations, or even videos and podcasts.

This class will introduce the basic skills and resources required to pursue a career as an artist. Students will learn the day to day tasks of working artists, such as writing critically about their own work, drafting grant proposals, and planning the business administration of their studios. They will also learn how to balance commercial and experimental projects, collaboration and community work, and teaching and studio practice. They will engage in the critical text about ethical dilemmas of working with art institutions, corporations and academia. By the end of class, students will write a personal statement, update an art portfolio, and apply for an artist residency. While the class will focus on contemporary art and artists, students who are interested in other disciplines may benefit from learning about entrepreneurship, authorship and cultural industry.

In this class students will learn math tools to boost their digital practice, fix common problems, and understand the math behind our human perception of the physical world. This course spans different branches of math including geometry, linear algebra, logarithmic thinking, and statistics as they relate to a programmer making digital art with our contemporary media ecosystem. The aim of this course isn't to become calculators, rather strengthen our intuition through historical and ethnomathematics perspectives and foster a new relationship to math. The prerequisites to this class are basic arithmetic skills and an introduction to programming. We will create applications using free and open-source software, including Python and p5.js.

Microinteractions are everywhere, you interact with them on a regular basis. Animated emojis, heart buttons that burst with love when you tap them or password fields that shake when you type in the wrong password - these are the tiny animations that produce delightful moments while providing users with valuable feedback. They can intuitively guide your users without having to explicitly write a set of descriptive rules. They are the soul and character of your interface, and if crafted well they can turn every interaction to a joyful moment while improving the user experience. Using Adobe After Effects and Lottie as our primary tool, we’ll learn how to design, animate and export playful SVG animations as interactive components. We’ll examine the basic principles of UI motion, cover how to create loopable animated states with fluid transitions between them.

There is a substantial gap between what AR/VR is capable of today and the future that enthusiasts envision. The hardware is progressing, yet there are lack of design tools and methodologies. Effective augmented and virtual reality games and experience require good storytelling, animation, production and solid graphics. Students will learn a myriad of processes including spatial interface design, volumetric capture, working with spatial audio, porting animations and game programming. This course presumes no prior knowledge and is intended to jump start a career in AR/VR development and interaction design. The goal is to have relevant portfolio projects for entering industry. Students will create content in Unity 3D to agnostically deploy on headsets, mobile phones or tablets. Students will have access to a wide range of available hardware. In the beginning of the course, students will learn to implement the most important AR/VR interaction concepts – concepts such as selection, manipulation, travel, wayfinding, menus and inputting text in Unity3D. In the latter part of the course, students will implement prototypes that will be periodically critiqued by experts at Oculus, Magic Leap, Google, and other top companies. *You do not need to be a programmer. This is intended for students interested in 3D art, interaction design, sound design and game development. All are important for immersive development*

Tangible interfaces are interfaces that you touch. You control them with your hands, feet, and other body parts. Their shape, feel, and arrangement provide feedback. This is where interaction design meets industrial design. In this class, you’ll design, program, and build devices with tangible controls in order to better understand how humans understand and control technical systems through our sense of touch. We’ll discuss physical interaction concepts such as expressive interfaces and utilitarian ones, real-time control vs. delayed control, and implicit vs. explicit interactions. You'll learn programming and electronic techniques to sense state change, thresholds, peaks, and other signs of user action. You'll also learn how to design, shop for, and construct housings for the devices you build. On the electronics side, the primary tools will be the microcontroller and common tangible controls: pushbuttons, switches, rotary encoders, rotary and slide potentiometers, force sensors and touch sensors. The class will also cover on-device feedback through LEDs, speakers, and force-feedback actuators. On the fabrication side, you'll work with the tools of the shop and XXX CAD program. You'll design and build four projects in the course of the semester. Projects will be designed (and parts specified). Projects will build on the skills learned in Intro to Physical Computing and Intro to Fabrication. Prerequisites: Intro to Physical Computing and Intro to Computational Media, or a working knowledge of microcontroller programming in Arduino; Intro to Fabrication or basic knowledge of laser cutter.

Are you energized by the possibility of creating equitable products, services and content? Are you interested in being a more intentional designer? Are you committed to racial equity? As designers, you are trained to solve problems, but the pull of quickly doing/solving/redesigning can actually cause more harm. There are countless examples of racially biased practices that have resulted in biased algorithms in tech, so how might we interrogate existing design practices while revisioning what's possible around technology and what equitable tech could look like? In this multimedia and interactive Intro to Design for Diversity (D4D), students will be provided with critical thinking skills to begin to explore the complexities of designing for equity. This foundational course will support you in reframing how you are approaching these complex equity dilemmas by examining "who we are" as designers, interrogating root causes of inequity and shifting how we are relating to – and centering – the communities most impacted by structural racism. Through a combo of lectures, storytelling, exercises, group work, case studies and weekly assignments, you'll engage with tools and practices to interrogate your thinking and design practices and processes at the personal, team, institutional and systemic levels, and explore possibilities to design for equity using key relational and strategic tools from our equity framework, D4D.

With platforms like Steam and Itch.io making independent games more accessible to the public, we're starting to see a movement toward physical installations of indie games as well. The New Arcade pays tribute to arcade cabinet designs of the 80's and 90's, but infuses them with new interfaces and digitally fabricated components. In this class, students will learn how to use the Unity game engine to design a simple arcade game. They'll learn about aspects that separate an arcade game from other types of games, and interface their game with different kinds of hardware using microcontrollers. In the second half of the class, students will use Fusion360 to construct a new arcade experience using digital fabrication tools like laser cutters, and CNC machines. The class will culminate in a physical installation that showcases their game in a public gallery. Prerequisites: Physical Computing About Mark Kleback: https://wonderville.nyc

What is the relationship between American musician and poet, Gill Scott-Heron and cybernetics? Heron's "The Revolution Will Not be Televised" was created in hopes to wake-up 1970's America from complicity in societal oppression to then realize the revolution begins in your mind and something created through active participation, not passive media consumption. While, iterated forms of cybernetics also challenge the notion that "technology [or society] is self-correcting", through fostering design thinking and systems theory through a meta-scale analysis of computational practices, essentially promoting being an observer of systems through conversation with process and feedback loops. Both ideologies center intention and holistic design within innovation — concepts vital to the equitable well-being of society in a technocratic era. In this course, students will explore the origins and history of the internet and HCI technologies as stemming from hippie communes, counterculture and warfare; survey historical and current social relations in the U.S.; study how these social relations have been replicated and embedded into everyday technologies; and consider potential futures in tandem with rapid technological advancements. We will study the works of pioneering designers and thought-leaders from Norbert Wiener, Stewart Brand, and Victor Papanek to Alondra Nelson and Ruha Benjamin. Phenomena discussed throughout the course range from the power of Black Twitter, the social media revolution of Egypt's Tahrir Square, to cryptocurrency being used in bail reform. After in-depth historical analysis of societal structures, geo-political dissent, and algorithmic patterns, students will engage with futurism and speculative practices, techno-culture manifestos, and design and systems theory to develop a variety of research-driven projects. We will conclude with using both second-wave cybernetics as a design framework, and an anthropological lens to imagine, innovate and design more equitable technocratic futures. Topics heavily examined throughout the course include: surveillance, race, socioeconomics, carceral capitalism and technoscience, algorithmic bias and oppression, and post-work societal structures.

This advanced experiential production course will introduce students to the latest techniques in 360 video recording and manual stitching, camera-paired Depthkit Cinema volumetric capture, DSLR photogrammetry/retopology, and a quick look at emerging virtual production systems using Blackmagic cameras in Unreal. Alongside an intense technical focus, the course will also deconstruct other experiential works that utilize similar experimental production designs to introduce the expectations demanded by professional productions. The course ultimately hopes to show viable paths for students to engage with immersive media capture pipelines, at an expert level, with an expanded sense of possibility and inspiration.

Auto Fictions is a studio class focusing on the creation of immersive, multi-path and interactive experiences based on personal narrative. Documentary art has included the art of installation for decades, but new technologies have given artists affordable tools that allow them to rapidly prototype and then refine immersive media experiences. Auto Fictions is an interdepartmental course that may include students from Film, ITP and Theater disciplines. Students will create production teams. Each team will make a project and each student will help the other create their work through intensive collaboration. The intention here is to practice collaboration across disciplines, methods, values, and artistic cultures to create a work of immersive fiction based on materials gathered from the past, captured or created in the present and/or imagined for the future. We will look at several possible approaches to the creation of immersive media works. Including multiplexing software programs. Students with the requisite knowledge in these tools or gaming engines with similar capabilities may use them for the creation of their works. But it is not required. There may be an opportunity for building intuitive interactivity into some of the experiences. In response to the Covid-19 pandemic we will be exploring how to adapt our working methods and our work itself to these conditions as they develop. The goal of the course will be to physically realize the works but to do that we will go through a detailed comprehensive design, planning and budgeting process so that if conditions permit all aspects of the work will be fully created or planned and ready to build. If conditions permit, we will meet one day at the beginning of the semester to install and introduce the platform, create teams, and review the assignment. Otherwise we will meet online and begin work there. Students will be provided with access to online tutorials for relevant software and systems.

What makes a work of art an "experience"? What do we mean when we say something is "immersive"? By working with a variety of technologies such as 360 video, VR, and virtual paintings, as well as reading and discussing terdisciplinary texts on experience design, game design, interactive art, students will gain an understanding of how to create immersive experiences with a focus on virtual reality headsets. The class will also touch on related technologies, methods, and fields including augmented reality and interactive installation. The primary software tool in this course will be Unity3d, with no previous experience required. Some basic media production skills and programming fundamentals are suggested.

"The smartphone is not only the primary site for digital communication and consumption, it also hosts emerging forms of media production. Let's investigate the potential of the mobile touchscreen as a creative instrument! This is a project based course, and we will explore by creating and testing a series of functioning web-based toys - including drawing apps, character creators, and writing tools. You can expect to sharpen your skills in javascript and design. "

Today’s internet, made up of mostly text documents and two-dimensional images and videos, is the result of historical limitations in bandwidth, graphics processing and input devices. These limitations have made the internet a place where the mind goes, but the body cannot follow. Recent advances in motion capture devices, graphics processing, machine learning, bandwidth and browsers, however, are paving the way for the body to find its place online. This course will explore embodied interactions in the browser and across networks. Specifically, we’ll explore TensorFlow.js models like PoseNet and BodyPix, and Microsoft Kinect in p5.js and Three.js. Assignments will consider designing engaging embodied experiences for individual and social interactions online. Experience with Node, HTML and JavaScript is helpful but not required. ICM level programming experience is required. The course will have weekly assignments that explore embodied interaction online. Assignments will begin with exploring single points of interaction (i.e. one mouse or one joint), and progress to considering full bodies and multiple bodies in one browser. Students will have a 2-3-week final project with which they will delve more deeply into the subject matter in one piece of work. Students will have readings/watchings focused on embodied and networked user experience. Some influential works that will likely be assigned/discussed are Laurie Anderson’s “Habeas Corpus,” Todd Rose’s “The End of Average,” and Myron Krueger’s “Artificial Reality.” The course examples will be taught in Javascript using web technologies/frameworks. However, students are welcome to work in their preferred medium.

This course is designed to provide an overview of 3-D visual storytelling in the newsroom from theory to practice. We will explore a variety of narrative formats and design principles, learn about reporting techniques for visual stories, touch on the best practices and ethics of journalism and learn how to use computer graphics and animation techniques in order to implement a three.js interactive.

Juxtaposed to traditional comics, Experiential Comics combines emergent tech, unconventional comic book art/structure, and game engines to offer users a more immersive, continuous storyworld experience. Challenging the status quo of classic and contemporary digital comics, students will explore new technologies/world-building techniques better suited to craft innovative comic book narratives and formats --worthy of the Fourth Industrial Revolution. Students will ingest a brief history of classic and digital comics formats, collaborate with comic book artists to design engrossing characters, engage in world-building sessions, play with Unity/Unreal engines to generate avatars/ virtual environments, work with actors in motion capture/volumetric capture studios, learn the latest iteration of the Experiential Comics format, and share their unique expressions of Experiential Comics in a final presentation. Throughout a 7-week period, the course will be divided into 7 themes 1) The Disconnection of Digital Comics 2) Classic and Unconventional Comics Continuity 3) Marvel vs DC vs Insert Your Universe Here 4) Fourth Industrial Revolution Technologies 5) Capture & Creation 6) Infinite Engagement and Unlocking Immersive Format 7) Experiential Comics Presentations. Each weekly class will be divided into two halves 1) Exploration of Theme/Discussion 2) Process, Practices, & Play. This course requires CL: Hypercinema or equivalent experience.

In this class students learn how to use data. If you can learn to live with uncertainty, you can make something beautiful and true. Students will learn about data as another form of evidence. We will collect our own datasets to learn about challenges and opportunities. We will explore sources of uncertainty, and how imagination and empathy can help uncover ways that data can lead to insight or alternatively, lead one astray. We cover basic stats principles to show how even properly collected data may lead you astray; we cover design principles, we introduce technical tools for visualizing data. Students collect their own data set, share reactions to assigned readings, and find their own examples of data usage that are positive or misguided.

Cybernetics is the study of how we shape and are shaped by systems. What can it teach us about the ideological and sexual reproduction of gender and sexism? How does sex become gender and what are the politics surrounding who gets reproduced? This class is a hybrid coding, critical theory, and art production class. We will explore how social regulatory systems are encoded into technological platforms and disentangle how they produce social pressure and govern behavior through coding exercises, somatic exercises, discussion, and project making. In this class, we will not shy away from difficult conversations and work closely together to cultivate a space of openness and mutual support. Discussion and project-making are core to this class. Together we will express ideas through code, computational consent, and read the work of scholars such as Donna Haraway, Ruha Benjamin, Paul Preciado, Silvia Federici, & Audre Lorde. Students will be encouraged to develop their own research interests and explore their personal histories. When projects are discussed, we will practice communicating ideas through presentation as a medium and will co-create a culture of constructive feedback.

When technology advances, teaching styles regress. Every new wave of technology touted as a boon to education -- radio, TV, DVDs, the internet -- has tended to revive the idea that the ideal class structure is the lecture, where faculty broadcast information to disconnected student recipients. Yet we've known for decades that lectures are poor ways to create learning experiences, and that actively involving students--with the class, the teacher, each other--is far more effective. The thesis of Imagination and Distributed Learning is that the range of possible experiences available online is larger, more varied, and more interesting than most of what is on offer from colleges and universities’ online courses. Students will read teaching and learning research, and do field work where they set out to learn something online and reflect on the experience, and then set out to teach something online, and reflect on that experience. The goal of the course will be for students to build up alternatives to existing (and often quite dull) online educational practices today. The final project will be a proposal, plus designs or partial implementation, for online tools or experiences that will create experiences users can learn from. These proposals will be presented to a panel of people managing existing online programs at NYU and elsewhere.

This course is about how to tell stories with your projects. Like a classic linear story, we’ll start at the beginning with the question of how good stories are told. We’ll learn about classical storytelling techniques and conventions from a variety of cultures, genres, and media. Next, we'll explore what various tech and media can and can't do in the context of story. We’ll end the semester throwing linearity out the window to create narrative work that engages with the tropes and conventions of non-linear storytelling. Chaos may ensue, as the defining feature of non-linear storytelling is that the author cedes some control of the narrative to the audience. There is no creative writing involved. Students will not be asked to invent new stories for this course—non-writers are welcome! The work of the class will involve reading, reading responses, active class discussion, and group work/play with projects riffing on assigned existing stories and narratives. For example, we might ask students to use a specific canonical story, poem, myth, parable, or film plot as a narrative jumping off point for assignments. The course is co-taught by Kio Stark, a researcher/writer of both fiction and nonfiction and Mia Rovegno, a theater writer/director who focuses on site-specific and immersive work. The semester is divided into three units. Unit 1: Narrative structures. In this unit, we dig into what it means to tell a story. Some of our major themes include: • What are the most commonly used story structures in media such as the novel, graphic novel, film, TV, and theater—and what expectations do they set up for the audience? • What are the techniques, tropes, and conventions of both western and nonwestern storytelling traditions? • What are some approaches that are in dialogue with or rebel against these traditions? • How do we understand and manipulate the audience’s narrative expectations? • What makes a story ‘work’ / how do we define a good or successful story? Unit 2: Using tech to tell a story. In this unit, we will do something that in other contexts is a terrible idea—we will start with the technology. We’ll explore briefly what counts as technology in our conversation, and then play with what specific technologies make possible, complicate, and make impossible when it comes to exploring narrative work. • What can and can’t we do with sensors, motion, projection, AR/VR, paper, film, light, voice etc in the context of the storytelling knowledge we’ve gained in Unit 1? • What unique storytelling conventions might be available to us as makers? • What kinds of objects, interfaces, situations, and places can be experienced as narrative? Unit 3: Non-linear storytelling. In this unit, we will play with situations in which the creators do not have total control over how the narrative is experienced and in what order it unfolds. • How do we use the viewer’s relationship/familiarity with conventions of linear storytelling to engage them in a non-linear narrative? • What are the storytelling conventions we see used in media beyond the page, big screen, and stage, where non-linearity is a common feature? For example, AR/VR, video games, social media, site specific performance, and museum design. • How do we make a story that works in more than one direction? • How do we play with the audience’s expectations? • How can we experiment with the temporal to establish duration and clear beginnings and endings for audience entry and exit? • How can we creatively engage both facility and innovation in a user journey? • How can we explore engagement of the audience’s “role” when the work demands a virtual, immersive or interactive experience? • How can we prime audiences for fluency in our storytelling modalities, without depending on cumbersome directions, real time interceptions or demonstrations of technology? In other words: how do we hide the man behind the curtain?!

In this course, students will be exposed to different qualitative research methodologies and disciplinary approaches to those methods (in terms of technique, ethical standards, approaches to citation, etc.). Students will practice using research methods to study subjects of their choosing and produce reflections, sketches, or prototypes based on weekly research findings. By practicing the act of "doing research" students will think about ways to incorporating research into their practice and better understand and articulate how research is already part of their practice.

This course combines two powerful areas of technology that will enable you to leap from being just a user of technology to becoming a creator with it: Physical Computing and Programming. The course begins with Physical Computing, which allows you to break free from both the limitations of mouse, keyboard & monitor interfaces and stationary locations at home or the office. We begin by exploring the expressive capabilities of the human body and how we experience our physical environment. The platform for the class is a microcontroller (Arduino brand), a very small inexpensive single-chip computer that can be embedded anywhere and sense and make things happen in the physical world. The core technical concepts include digital, analog and serial input and output. The second portion of the course focuses on fundamentals of computer programming (variables, conditionals, iteration, functions & objects) as well as more advanced techniques such as data parsing, image processing, networking, computer vision. The Javascript 'p5' programming environment is the primary vehicle. P5 is more oriented towards visual displays on desktops, laptops, tablets or smartphones but can also connect back to the physical sensor & actuators from the first part of the class. The course is designed for computer programming novices but the project-centered pedagogy will allow more experienced programmers the opportunity to go further with their project ideas and collaborate with other students. What can computation add to human communication? You will gain a deeper understanding of the possibilities of computation–– possibilities that will augment and enhance the perspectives, abilities and knowledge you bring from your field of study (e.g. art, design, humanities, sciences, engineering). At first it may feel foreign, as foreign as learning a new language or way of thinking. But soon, once you get some basic skills under your belt, you'll be able to make projects that reflect your own interests and passions.

The class will teach how to architect and lead a virtual production by creating a dialogue between the Producer, Director, and Cinematographer in filmmaking with the Technical Producer and Director in creative technology. The class will cover an overview of all of the technical skills required to produce a remote virtual production through the lens of a project manager making administrative and creative decisions. This class will culminate in a real-time 3D project exploring motion capture and virtual production that will adapt a pre-existing cinematic work with the class themes in mind.

Zoom has become one of the primary modes of gathering, communicating, and interacting with many of the people in our lives. This simple but robust tool has replaced the complexity found in typical day to day modes of engagement with tiny portals locked in a grid. Everyone is burnt out, everyone is tired of it, and everyone dreads the next Zoom meeting in their calendar. This class is all about bringing a sense of joy, fun, and experimentation to a platform that has lost much of its novelty. Over 7 weeks we will engage in a series of prompts and experiments to reimagine what kind of experiences we can have on Zoom, culminating in a collection of group performances and events open to invited guests. We will use tools such as OBS, ZoomOSC, Virtual Cameras, Virtual Audio, and more to learn how we can control and manipulate the core functions of Zoom. While these tools will go a long way in crafting these experiences, the class is also about sharing ideas and finessing our existing tools and skill sets to collectively experiment with what is possible and what is engaging.

In this experimental seven week seminar/studio, we’ll explore media meant to be experienced from a first person point of view (ie, training simulations, found footage horror, first person shooters, Go Pro videos, VR). This class will include a gentle introduction / brush-up of creative applications of Unity3d (culminating in the ability to create a simple 3d FPS style experience, subverting the “first person shooter”), as well as an introduction to a variety of techniques for crafting first person experiences. Ultimately, the final project you develop will use the medium/technology of your choosing, on the topic of your choosing, leveraging the affordances of first person POV. Class activities will include: discussion of readings and artworks, research and presentations on self-determined sub-topics of interest, technical demos, critique. If there is shared interest, we will do a live-streamed performance or other collective digital presentation of final projects. We will: Co-construct a shared critical language around first person POV media Read and discuss media and texts about: violence in gaming, empathy, embodiment, subjectivity, etc. Review or reinforce basic Unity skills Deeply engage with each other’s ideas, work, and points of view!

This online asynchronous course introduces the fundamentals of building "full stack" web applications. It will focus on modern, client- and server- side web technologies and provide practical methods for approaching web development for creative and functional applications. The core technologies used in this course are HTML5, JavaScript, Node.js with the Express framework, and MongoDB database. Students will learn to design, develop, and deploy web applications and gain the necessary skills to extend and explore web development independently. The course will consist of a series of online videos, other online resources such as a class website with notes, feedback, forums/chat (discourse, slack, discord), as well as live office hours and/or help sessions with the professor and GAs. There will not be regularly scheduled class meetings.

On October 8th, 2015, a team in Ecuador identified 431 species of birds - the world record for number counted in a single day. Earlier that year in Myanmar, a scientist counted one Jerdon's babbler, the first in nearly eight decades. In December of 2019, eBird announced that its database held over 737 million bird observations. This morning, in Brooklyn Bridge park, I counted 38 house sparrows, 4 black-and-white warblers and an ovenbird. This course will consider birding as a practice, and will dive deep into the processes by which observations become data. As a collective, we will investigate how crowd-sourced data is transforming ornithology, and will explore ways to tell stories about the natural world through visualization and more radical forms of data representation.

The goal of this class is to gain an understanding and proficiency with Computer Aided Design (CAD). We will become familiar with CAD software, mechanical design, and simulation. The class will cover common CAD modeling techniques. We will use our designs to get physical parts made as well as use them in virtual projects. We will create parts both real and impossible.

Society has always dreamed about humans coexisting with automatons, robots and talking machines that fit into every facet of daily life. As a consequence of computation and the internet leaving the flat screen, alternative forms of human-machine relationships are increasingly becoming more ubiquitous. Designing for these new machines brings novel challenges and requires a different approach. From HAL 9000 to early automatons, this class presents an overview of the history, methods, technologies, and design challenges involved in building and living with Robots and Social Devices. This 4.0-point class is conceived as a hybrid format (Lectures + Workshops + Discussions). Discussions and explorations will emphasize the cultural, political, and technical. Over the course of the semester, students will interrogate and deconstruct examples of Robots (in sci-fi, popular culture, art installations, assistive tech, connected devices), and design their own systems by appropriating existing technologies. In this class, students will utilize Arduino Nano BLE Sense, RaspberryPis, OpenMV, Google AIY Voice and Vision. Physical computing and ICM are prerequisites.

This course will explore the history and meaning of the ubiquitous concept of “cursed” media, and provide students with a survey of digital art tools for the creation of their own cursed animation, video, photography, music, and web art. Many people were first introduced to the concept of cursed media when it exploded into mainstream internet discourse in 2016 with the @cursedimages Twitter account, which posted found photos bound by their unsettling effect on the viewer. Cursed media predates this account, however, stretching back to medieval notions of cursed objects. We will demonstrate how throughout time, cursed media has functioned like a slip of the tongue that provides a window into the cultural unconscious, where we encounter uncensored thoughts and feelings about race, gender, class, and what it means to be human. From Amazon Muzak generators to Artbreeder’s GAN tools for image creation, from machine learning text generators to robots who work at Walmart, cursed media and tools for its creation bring into view the ways that that culture reacts to tension between the increasingly precarious position of human beings in the capitalist 21st century and the threat of human erasure by the powerful forces of nature. Students will be introduced to digital art tools for creating music, manipulating photography and video, working with 3d animation, and building web art. Students will attempt to create their own cursed media, and in the process will gain a deeper understanding of the unconscious biases and ethical implications of contemporary digital creation tools.

Bluetooth Low Energy (BLE) provides low power wireless connections between devices, including but not limited to mobile phones, smart home devices, and wearables. Following the standardized protocols, it’s easy and efficient to develop low-energy wireless interactions using Bluetooth LE. This 7-week class introduces fundamental information about Bluetooth LE and provides hands-on practices for students to learn and understand BLE. This class is a good fit for students who are interested in designing and developing IoT devices, wearables, or who would like to learn more about physical computing. What will students do? The topics we will discuss in class will include wireless communications, hardware protocols, power management, etc. Students will get hands-on experience of designing and prototyping BLE projects individually or in groups. How will the course be structured? Each class will be a hybrid of lectures, demos, discussions, and hands-on workshops. For each class, the instructor will bring up different topics and share existing projects and research for the students to have an overall understanding of the topics. The topics will be followed by in-class practices where students will practice the techniques they’ve learned. In a remote class format, students will be divided into small groups to get their questions answered.

For most of its brief history, the domain of “smart cities” has belonged to large corporate vendors who promise and offer ubiquitous, citywide intelligence that utilizes their proprietary systems. More recently, an increasing number of startups have developed solutions which can make the technology somewhat more accessible. Unfortunately for municipalities, working with product vendors can be a heavy lift and the commitments involve lengthy procurement and contracting processes. In the past five years or so, the access to connected technology has increased and the hobbyist or “maker” movement has seen an uptick in offerings related to IoT. From the Arduino IoT Cloud and Adafruit.io software, to the Raspberry Pi and Particle hardware, building connected IoT devices has become easier than ever before. Through platforms like Google Coral and Nvidia Jetson, even edge computing and AI has become available to those with coding skills and a modest budget. What this course aims to explore is what happens when low-cost, readily available electronics platforms address the data needs of municipal governments. Rather than thinking of smart cities as large scale, big data projects that provide intelligence across a city, we will look at targeted applications that would be too small or costly to pursue as a conventional IoT procurement. What intelligence can be gathered in a short period of time with a small budget? During this course, we will examine successful and troubled smart cities projects, discuss the ethics of public technology projects, and review the prevailing best practices and guidelines relating to the Internet of Things in government use. Thinking in terms of “rapid IoT” and “little big data,” students will partner with NYC agency representatives* to uncover insights about a particular issue related to their work. Students will undertake a semester-long project, working to understand the agency’s data needs and develop an IoT solution to gather data for analysis that could inform the agency on planning, policy, or operational issues. The course will cover current connected microcontroller platforms and connectivity options like WiFi, Cellular, and LoRaWAN as well as the software tools needed to store and present data in user-friendly dashboards.

A course where we will follow the computer rendering pipeline, and understand how shapes and images end up on our screens. The course will cover different steps of the process, but will focus primarily on shaders and how they can be used to program the graphics card and render out visuals. Topics will include meshing techniques and topology, the ins and outs of the different shading steps, and how graphics cards can be used for non-graphics computing. The course will primarily use web technologies (either p5.js or three.js) making it easy to view and share each other's work.

While the automation of religion may sound like science fiction, the foundation of this course is examples of physical religious robots already built and spanning most major religions. Their existence raises complex questions about spiritual assistive devices, religious jobs as work, technology as “miracles,” idolatry, transhumanism, sacrilegious robots vs secular robots, the role of rites for robots, and whether a robot can be a member of a religion. Religious robots confront us with the boundaries of personhood, and can provide insights of what that means for technology, spirituality, and society. The course will combine reading, writing, discussion, research and speculative design.

Tangible interaction is a difficult topic in 2020. In the early months of the year, there were various calls for a move to a “touchless future” in an effort to counter the fear of infection from touching shared things. Even though more recent research suggests that transmission via inanimate objects (fomite transmission) is a low risk, the perceived fear and its effect still persists. Given this environment, what is the future for tangible interaction? This class will investigate that question. In this studio, students will survey the current state of practice in tangible interaction design, investigate different sensors and haptic controllers, and generate information that can be used by other practitioners. Class work will include: - literature review and summary, i.e. getting to know what's already in the field, collecting and summarizing examples of best practice on a public website - practical research and tutorial documentation, i.e. figuring out how to make a given control or interface, then documenting the process for doing it, and/or verifying someone else's tutorial through duplicating their work. For those students who are in New York and feel safe using the shop, there will be projects which will require shop work. For those who are remote, or who do not feel safe using the shop, there will be research, design, documentation, and verification work. Class will be structured as a series of multi-week group projects including research, experimentation, and documentation. Students will work in small teams of 2-3, and teams will meet with the instructor weekly. The full class will meet 4-5 times to introduce the field and the project, and to review the whole project. In between, students will meet with the instructor individually or in groups, during the scheduled class time or office hours. Examples of related work: * The UX of Lego Interface Panels, George Cave * Comparing Force-Image Schema dials (video) (paper), Jörn Hurtienne, Diana Löffler, Patty Gadegast, Steffi Hußlein * Enclosures, (tutorial) Ben Light * Distance Sensors, The Basics (tutorial and comparison), Noah Pivnick * Collision Course: an essay on the interface design of the USS John McCain's navigation system and how it led to the collision with the oil tanker Alnic, T. Christian Miller, Megan Rose, Robert Faturechi and Agnes Chang for ProPublica Students will gain skills in review and analysis of existing practice, in written and multimedia formats; design research and specification; and electronics and programming techniques related to this field.

WEB ART AS SITE addresses the history and practice of art made for and inseparable from the web, while teaching basic coding for the web. We explore key examples of web art from the early days of the internet through today, asking questions about this idiosyncratic artistic medium like: How do different forms of interaction characterize the viewer and/or the artist? What happens to our reading practice when text is animated or animates? How is an internet-native work encountered, and how does the path we take to reach it affect our reading? Who is able to see a work of web art, and what does access/privilege look like in this landscape? How are differently-abled people considered in a web artwork? What feels difficult or aggressive in web art, and when is that useful? How do artists obscure or reveal the duration of a work, and how does that affect our reading? What are the many different forms of instruction or guidance online? As we ask these questions, we exploit the internet pedagogically, collaborating online, playing with anonymity, and breaking the internet spaces we know. Students learn web coding through specialized online tutorials; most of class time is reserved for discussion (of web art and supplementary readings) and critique. Throughout the semester, students will produce two major works of web art. Students need only a standard laptop, and will not be expected to purchase any software or text (cost of materials: $0).

"I want to be famous!" For a decade, this has been the most common response to the question "What do you want to be when you grow up?" Influencers, TV stars, D-listers - the media has exploded with celebrities who could not have existed even 20 years ago. But who actually gets to be famous? It's certainly not always the most attractive, charismatic, or talented. Why shouldn't it be us? This production-centric class will create celebrities. We will study the underlying principles, tactics, and technologies that go into generating the phenomenon of fame, and the deliberate strategies of PR and cross-platform narrative that sustain it. We'll examine the evolution, neuroscience, and cultural positioning of celebrity in modern society. Then we will try it out on each other. And also maybe on some cats, puppies, and ferrets too. Let's demystify these questions of self-presentation, media attention, fan behavior, and of course, commercial profit. Celebrity isn't nearly as random as it appears!

This studio class will explore the in-between space of video, painting, installation, and new technologies. What unconventional methods can be employed to create a 3D model? What innovative strategies can be used to integrate 3D animated video and installation in a single work? How can an artist resolve the fundamental differences and build the connections between 2D moving images and 3D objects and space? Over the course of the semester we will introduce the basic 3D skills in Maya, and apply these skills to build a cohesive body of artistic work with 3 different outputs: Animated video, installation and augmented reality using Spark AR.

This course provides critical and curatorial insight into global art practices and interactive technologies from a post colonial perspective. Designed to provide a critique of imperialism the course is underpinned by ideas pertaining to the rise of the Global South, decoupling, indigenous knowledge and ancient and contemporary innovation through contemporary art, emergent technologies, new media and exhibition practices. Students will also investigate the role of shifting digital landscapes and conservation of new media coupled with museum collecting practices, from both a deconstructive and ethical lens, providing regular opportunities to reflect upon their own respective practices. Presented as a combination of presentations/ critiques, seminars, readings, virtual field trips as well as special guest visits with noted experts, the course presents a compact and timely overview of globalization, and the effects of rapid interactive and technological innovative, in lieu with ideating towards a more equitable and diverse art and technological ecosystem.

We are experimenting with a change to Thesis with this pilot class. The idea is to extend thesis over both semesters, each with a 2 point class that meets every other week. Part 1 focuses on research and concept development. Students will have the time to explore deeply into their areas of interest, narrow their focus and, finally, develop a concept and plan for their final thesis project. They will be structured assignments throughout on research methodologies and concept development techniques. They will end the semester with a paper and a plan for design and production.

Internship can fulfill a Tier 2 (elective) requirement. An internship is done with an outside agency that provides a student with opportunities to work on projects that enable the student to develop and demonstrate his or her practical abilities, and which involve both new interactive technologies and their users. Internship requires a minimum of three hours per week per credit. The standard ITP lab fee is attached to this course.

This course is designed to help students define and execute their final thesis project in a setting that is both collegial and critical. It is structured as a series of critique and presentation sessions in which various aspects of individual projects are discussed: the project concept, the elaboration, the presentation, the process and time-table, the resources needed to accomplish it, and the documentation. Critique sessions are e a combination of internal sessions (i.e., the class only) and reviews by external guest critics. Students are expected to complete a fully articulated thesis project description and related documentation. Final project prototypes are displayed both on the web and in a public showcase either in May or the following semester.

"Blockchain is the new Internet" - something bigger is going on here, than just another form of digital payment like Bitcoin. The blockchain enthusiasts promise applications from smart contracts, to autonomous organizations, to anarchistic systems of government. This course introduces fundamental concepts and functionalities of the blockchain and its applications, and offers a way to playfully explore its multiple dimensions. The goal of the course is not only to improve skills in this utopian however very real technology, but also to creatively apply it, to come up with design fiction and push the concept to the edge. Students will learn the basics of blockchain technology, cryptography, and the functions specific to the blockchain like crypto-currencies, smart contracts, and autonomous organizations. Students will create their own designs and applications (real or fictional) of blockchain, and their speculative designs what the technology might become. The course includes reading and practical work as homework and in class work.

According to anthropologists Filip de Boeck and René Devisch, divination "constitutes a space in which cognitive structures are transformed and new relations are generated in and between the human body, the social body and the cosmos." In this class, students will learn the history of divination, engage in the practice of divination, and speculate on what forms divination might take in a world where the human body, the social body, and even the cosmos(!) are digitally mediated. Starting with an understanding of ritual and folk culture, we will track the history of fortune-telling from the casting of lots to computer-generated randomness to the contemporary revival of Tarot; from reading entrails to astrology to data science; from glossolalia to surrealist writing practices to the "ghost in the machine" of artificial intelligence. Weekly readings and assignments culminate in a final project.

Magic windows that allow us to peek into different realities without leaving our physical space, lenses that reveal hidden layers of objects or navigating new universes within the same room. More than ever, mobile devices are getting a human-scale understanding of space and motion allowing us to create more intimate interactions with our surrounding spaces, leveraging them as a canvas to experience other realities. We now have the potential to give life to inanimate objects, tell stories through space, customizing private views of public spaces and recognize places we've never been. We'll question what it means and how can we blend reality exploring themes such as: augmented space and new paradigms in social interaction, public space and privacy; storytelling and navigating the physical space like turning pages in a book; tangible interfaces, mixed objects and animism; Magic windows, x-ray vision, time-machines and impossible universes; Far away so close: telepresence and remote collaboration. The course will survey the past, current and up and coming technologies and experiences in Mixed Reality including environmental augmented reality and interactive projection mapping, handheld devices while fostering a strong user experience perspective on the affordances and constraints of each. We'll research and discuss the design principles and guidelines for creating mixed reality experiences focusing on the links between real and virtual objects, interaction space and asymmetries between physical and digital worlds, environmental semantics and multimodal and tangible interaction. Technologies explored will be focusing on mobile platforms (phones, tablets) including Vuforia, SLAM, image and object recognition, depth sensing, projection mapping. Unity3D will be the development platform: students must have previous working knowledge of Unity3D and feel comfortable with independently developing using this platform. A working knowledge of Unity3D may be gained by going through the Unity 5 3D Essential Training Lynda Course prior to the course (log in to Lynda for free via https://www.nyu.edu/lynda).

Because the full potential of the emerging field of soft systems is unrealized, there are countless opportunities for curious innovators to discover or develop novel soft systems. Soft robotic skills and techniques also open up a world of possibilities for large scale or surprising artwork. This course teaches hands-on fabrication techniques for constructing simple pneumatic actuators from cast silicone and heat-sealed mylar, and challenges participants to design and build their own. Lectures and discussion center on concepts from soft innovation history, the current state-of-the-art, and sister disciplines of bio-inspired and hybrid (soft/hard) robotics. Consideration of both brand new soft materials, from a class visit to Material ConneXion library, and everyday overlooked soft mechanisms, found in average retail stores, will require participants to look at softness through a new lens. Final projects will be the development of an original soft/flexible/hybrid research or artistic concept presented with context, material swatches with justifications for choices, and physical or modeled proof-of-concept. About Kari Love: http://www.karimakes.com

Tangible interfaces are interfaces that you touch. You control them with your hands, feet, and other body parts. Their shape, feel, and arrangement provide feedback. In this seven-week class, you'll build devices with tangible controls in order to better understand how we learn about and manipulate the world through our sense of touch. We'll discuss physical interaction concepts such as expressive interfaces and utilitarian ones, real-time control vs. delayed control, and implicit vs. explicit interactions. We'll discuss programming and electronic techniques to sense state change, thresholds, peaks, and other signs of user action. The primary tools will be the microcontroller and common tangible controls: pushbuttons, switches, rotary encoders, rotary and slide potentiometers, force sensors, touch sensors and others. The class will also cover on-device feedback through LEDs, speakers, and force-feedback actuators. Weekly projects will be designed (and parts specified) in pairs in in-class design sessions, and executed as homework. Projects will be mostly microcontroller-driven, and will build on the programming and sensor-interfacing skills learned in Intro to Physical Computing. Prerequisites: Intro to Physical Computing and Intro to Computational Media, or a working knowledge of microcontroller programming in Arduino; Intro to Fabrication or basic knowledge of laser cutter.

We've been tinkering with the living systems that generate our foodstuffs for millennia. But climate change is radically and rapidly shifting these food landscapes, and the impacts include the extinction of many of the foods we love: chocolate, wine, beer, coffee and more importantly starvation for those in the world who are already food insecure. In this class, we'll explore biotechnologies and bioengineering along with microbes and mushrooms to design and create pathways for the restoration of some of the damage we've wrought on our food system. We'll also use art and design and systems thinking to build speculative and actionable projects that will focus not just on the future of food but the future of our planet and all of its inhabitants. This class is part of the Biodesign Challenge.

We use light in all aspects of our lives, yet we seldom notice it. Most of the time, that’s no accident. Lighting in everyday life, well-designed, doesn’t call attention to itself. Instead it draws focus to the subjects and activities which it supports. In this class, you’ll learn how lighting is used for utilitarian, expressive, and informational purposes. We’ll consider the intersection of lighting design and interaction design, paying attention to how people interact with light. We’ll practice both analyzing lighting and describing its effects, in order to use it more effectively. On the technical side, you’ll learn the basics of the physics of light, its transmission and perception. We’ll talk about sources of light, both current and historical. We’ll work with computerized control systems for lighting and modern light sources, and we’ll create a number of lighting designs for different purposes. You’ll get practice building AC and DC electronic circuits, programming microcontrollers for physical interaction, and learning digital communications protocols such as DMX512 and HTTP and REST. Projects in this class will range from indicator lighting on devices to task and wayfinding lighting in everyday environments to stage and environmental lighting. We won’t spend time on projection or light used for purely expressive purposes, but will look at how to put light to work instead. We’ll focus our attention on lighting the subject at hand, whether that subject is a person, a living environment, or a workspace. This class will be production-intensive throughout the course of the spring semester. Second-year students will not be able to combine the assignments in this class with their thesis projects, though some of the skills may be complementary.

Do you want to teach or run workshops? This class explores pedagogy, syllabus and curriculum design for artists and creative technologists. A good teacher is also a great student themselves. They transform their curiosity into knowledge and share their learning process with others. One can learn to become a better teacher by staying fearless about ‘not knowing' and unlearning to embrace radically open ideas and connecting various expertise and knowledge. Teaching can be a form of artistic and creative practice in collaboration with a diverse community. Teachers can invent new forms of learning spaces, new kinds of collaboration and new sense of community. In this class, students will learn about creative and logistical aspects of teaching. Students will read about the history of artists in and out of academic institutions, Black Mountain College as well as more recent experiments. Students are expected to engage in a critical discussion about the topic. Class will include a few field trips and meetings with museum educators, teaching artists and community organizers. Students will create their own workshops based on a class guideline.

“Why would the government care about me? — do I really have to worry about surveillance? Facebook and Google already know everything about me….I can’t really do anything, right?” We’ll answer these questions and more in this course. Students will walk away with some fundamental digital security skills, and the ability to learn new skills and think through security risks. This class is essential knowledge for every student at ITP, since your work inherently requires use of technology, including cutting-edge technology that could present new safety, security, and privacy risks. It’s especially useful for any student who could be working with or belong to marginalized communities, or who is doing any work that could be deemed political. We’ll use hypothetical profiles to learn how to conduct a risk assessment. Students will learn the basics of how to assess digital security risks for themselves and for their projects, a process known as threat modeling or risk assessment. They will also learn some digital security basics, such as how to install and use encrypted messaging on their mobile devices, how to avoid phishing/malware attacks, and how to create and store secure passwords. This 1-unit course will take place over three evenings. The first meeting will be an introduction to the concepts of risk assessment and digital harm reduction. The students will be assigned a risk assessment for themselves for the next class. In the second class, we’ll discuss some basic security measures. In the final class meeting, we’ll review the assignments and discuss specific security measures based on how the students assessed their own risks. Prior to the first class meeting, students should please read: Eva Galperin and Jillian C. York, Yes, Online Privacy Really Is Possible, http://www.slate.com/blogs/future_tense/2014/02/14/threat_modeling_and_finding_the_right_level_of_online_privacy_for_you.html, Slate (Feb. 14, 2014) Dia Kayyali, Getting Started with Digital Security, https://blog.witness.org/2016/11/getting-started-digital-security/, WITNESS (Nov. 2016) Please take a look at the://ssd.eff.org/ Electronic Frontier Foundation’s “Surveillance Self-defense” site: https

Transformative technologies (a.k.a. Transtech) are the wave of the future, yet many challenges remain before their use can become as effective and widespread as that of personal computers and cell phones today. This course will introduce students to this exciting field, starting with the examination of the potential for optimizing experience through mindfulness and meditation, the understanding of basic issues in obtaining and interpreting physiological signals, toward the aim of generating ideas for wearable transtech projects. Students will examine the ideas behind efforts to optimize human experience; practice different meditation techniques to experience the variety of cognitive and affective strategies they use and the varied effects they generate; explore the basic issues in obtaining and interpreting physiological data, and the use of brain stimulation methods such as TMS, dTCS, etc., and use this info to come up with ideas for wearable transtech devices. The class will be mix of lectures, exercises and demos.

In this experimental video class students will learn to use Python and command line tools to explore the possibilities of automating the film-making process. We will cover techniques for capturing, analyzing, editing and manipulating video with code. We’ll treat video as a textual as well as visual medium, repurposing found footage to generate new compositions and narratives, and experiment with home-made camera rigs that can be controlled remotely and algorithmically.

“Lies. Hoaxes. Conspiracies. Rumors. Propaganda. Fake news is an age-old phenomenon—but the internet is making targeted misinformation cheap and scalable. That is affecting politics, public opinion, and the everyday experience of the internet. In this 6-week class, we will explore the cutting edge of “fake news” by engaging in ethical research and fabrication. Participants will manufacture and observe a controlled “fake news” event. We will experiment with command-line tools for doctoring video, neural nets and deepfakes to fabricate reality, Twitter bots, behavioral psychology, and the dark underbelly of the ad economy.”

Whether it’s through photo realistic scans found in current-gen video games or the cartoonish and low-fi aesthetic of Bitmoji there is no limit to ways in which the body and the self are represented in digital spaces. This class will look at how avatars have been historically used in the realm of art, commerce, and entertainment and utilize existing avatar creation tools to develop projects that examine identity, body politics, and contemporary performance. In class, we will cover the basics of Unreal Engine, photogrammetry, 3D scanning, and model rigging although students will be encouraged to use existing skill sets and creative thinking to complete some of the smaller week-by-week assignments. The class will culminate with a short performance, small installation or single/multi-channel video piece using one or more of the techniques covered in class. This can be a solo project or a group project. In this class students will: – Explore how avatars can be utilized in your creative practice – Gain an introductory understanding of Unreal Engine, photogrammetry, model rigging, and 3D scanning. – Learn how to recontextualize digital spaces for the purposes of art, installation, and performance. – Broaden your thinking of what performance can be, both in a physical setting and digital setting. – Think critically about how physical bodies inhabit digital spaces and how the hardware and software we use reinforces the acceptance and value of certain kinds of bodies.

Digital tools of all kinds are deeply embedded in how our society operates. Innovations in basic communication, data processing, image manipulation, and even financial systems have transformed our social worlds and our artistic practice. This became even clearer and more present during the current global pandemic, where, during times of social isolation, digital and networked tools almost fully replaced in-person social life. This course will examine the ethical and esthetic implications of digital and networked world through the lens of socially engaged art and explore how digital tools are and can be used in socially engaged art practice, where art and creative work intersect directly with people and civic life, including a concentration on how art and social practice transform when virtual presence takes the front seat. This includes discussion of how digital and networked tools both increase and complicate physical, economic, and cultural accessibility, and the ethical and social implications of the newest digital and networked technologies, including Web3 and quantum computing. We will look at artists like Stephanie Dinkins, Kyle McDonald, Ari Melenciano, and the group Forensic Architecture. Students will be asked to propose several projects as thought experiments, and fully realize one online/digital socially engaged project. We will review and discuss the different definitions of “socially engaged art”, including discussions about “best practices” to use for working with different communities, and the politics of how we interact socially and how we approach the physical and digital social spaces around us. We will work on how digital tools have been used in socially engaged art and how they could be used further, and experiment with how online life functions as a public space, guided by the understanding that working digitally with socially engaged concepts means both using digital tools within projects AND interrogating the inner workings of how digital practice operates socially and culturally. Depending on circumstances, we will have some meetings and activities in public spaces, “field trips” to organizations such as Eyebeam, and practical applications of methodology, as well as two or three guest lecturers.

Successfully communicating is a critical skill not only for a graduate thesis, but also in the career that will follow. It boils down to this question: Are you persuading, influencing, or communicating your thoughts and ideas effectively, to any audience be it three people or three hundred? In this class we will systematically work our way through a four-step method to improve your ability to connect with your audience. We will explore the science that explains why stories work. We will tackle a basic framework for what a story is, using a process and foundation to develop any talk or presentation. Now that we have what you are going to say, we will also focus on how you say it, along with strategies to give you confidence to be your best self when speaking in front of a crowd. This is a particularly good class to take in preparation for your thesis in the Spring. This seminar examines and deconstructs verbal storytelling as a discipline in its own right. It is an exploration of speaking and storytelling as a fundamental building block of human evolution and innovation. We will look at the learnings from ancient times through modern scientific research—looking at theories attempting to explain what happens physiologically and psychologically when we are moved by a spoken narrative. This is a contextual approach that will focus on both the theory and the application in the marketplace of developing and delivering narrative as it relates to presenting oneself, a product or a service. As such, we seek to understand what drives current trends toward narrative education and storytelling as a competitive advantage in learning, communicating, persuading and influencing. Students will also contribute to designing a collaborative verbal communication template for the class and for the Final Project: a presentation that applies some of the concepts learned to themselves or their projects, products, ventures and/or service concepts.

What do we do when the Internet is killed or dies? Or when the network is taken down by natural circumstances? How can we re-establish standalone or minimally dependent communications? How can we detect and differentiate between natural interference and designed interference, or can we? This class will investigate the design of the Internet itself, current communications protocols, and emerging techniques to supplement the Internet or establish separate networks. Besides architectural basics, we will look at case studies on how communities, activists, and organizations have reacted to network outage. Guest speakers will describe what they learned about the consequences of dependency on cloud and grid based communications. Topics covered will include meshnets, online-offline synch, re-utilization of non-Internet networks, layers of the current family of Internet protocols, alternate analog techniques, and case studies from Standing Rock, humanitarian disasters, Red Hook Brooklyn, and others. The goal of the class is to challenge students to conceptualize at least one other workable alternative to the Internet in the case of interruption of service. Students with or without network background are both highly encouraged to enroll.

Meaningful growth involves discomfort. For individuals, in relationships, for communities it can be a key aspect in the process of reaching a desired outcome, in what Ida Benedetto calls "patterns of transformation". Our unique insights as designers, artists, and creative technologists can lead to innovative applications of this unorthodox tool. The Uses of Discomfort is an experience design course where we will spend six weeks delving into how this response functions, why it's of interest to us, and what we might be able to do with it. We'll look at four broad categories (visceral, intimacy-related, control-related, and cultural) through assigned readings, in-class lectures, and creative challenges. These challenges will ask you to manage safe risks for yourself and others, putting your ideas to the test as you critically explore the course content. Conceptual in nature, you will be expected to explore how you can apply your favored skills, technical approaches and design tools to the questions and opportunities presented. At the end of the session, you will be empowered to incorporate the course learnings into your ongoing work as designers, artists, and creative technologists. You will benefit from some prior familiarity with one or more of the following: speculative design, art-practice, concept development, or user-experience design.

As game and interaction designers we create systems and choices that can either prey upon our psychological foibles or help us avoid decision pitfalls. It is our responsibility to understand how we decide, to consider the ethics of the systems we create and to practice designing systems in a purposeful manner. Game Design & The Psychology of Choice will provide interaction and game designers with an understanding of the factors that influence behavior and decision-making by looking at the intertwining of cognitive psychology and economics through the development of behavioral economics. These disciplines study behavior on the individual and group level, often revealing some of the why behind the rules of thumb and folk wisdom that game designers come to intuitively. But understanding the why—why we fall into decision traps; why certain tradeoffs tax our brain more than others; why we are overconfident about our abilities; why certain decisions make us uncomfortable—allows us to more purposefully apply our design craft, both in and out of games. Finally, as a class, we will take what we learn about how we think and create series of game experiences based around key cognitive science concepts. Assignments may include: •Mod a cognitive science experiment into a game or experience •Analyze and present a game through the lens of cognitive science and behavioral economics •Create game or experience based around a particular insight from cognitive science or behavioral economics

In New York City, every storefront window has the possibility to tell a story, spark a conversation or inspire an interaction. This workshop will focus on creating innovative interactive pop up installations designed for public window displays. A successful window is one that clearly delivers a message directly to the public. How do we create interactive displays that engage the public with a distinctive voice or style? Over seven weeks, students will concept, prototype and build an interactive experience meant to be installed in a storefront or commercial display. This course will explore lighting, design, and budgeting of durable interactive window installations. Previous fabrication or programming experience is encouraged.

This is an advanced seminar exploring the themes, production methodologies, and dynamic definition of sculpture in the 21st century. Much of the class will look with a critical eye through the lens of technology – What is technology’s role in the motivation, production, and proliferation of sculpture? Students in this course will: Gain understanding of major topics/themes in contemporary sculpture; Learn about the various digital and historical processes used in sculpture / object making; Apply critical thinking and discourse to weekly readings and discussions; Visit studios / museums / facilities dedicated to the production and support of sculpture in NYC. The class will consist of weekly discussions, based on readings, lectures, guest artist visits and a class trip to a fabrication facility. The decision to split the class into 3 sections: Production, Themes of power / the politics of objectification today, and A multi-sensory, multimedia definition of sculpture, was inspired by the Sculpture Center’s publication series Inquiries Into Contemporary Sculpture.

"Memories are motionless, and the more securely they are fixed in space, the sounder they are." (Gaston Bachelard, The Poetics of Space). This course is about exploring the unique affordances of virtual space in order to create VR experiences that transcend the (still awkward) headset. In class we will: Read and discuss relevant interdisciplinary writings from architecture, philosophy, neuroscience, art history, and poetry; Have in-class demos for core concepts of Unity3d for VR; Group critique of each other's work; Analyze the successes and failures of available VR apps; Have visits from guest VR creators Students will walk away with: An introduction to 3d concepts and Unity for VR (for multiple headsets) A deep dive into what makes a meaningful VR experience A light introduction to other methods of developing Virtual Reality experiences (360 video, Oculus Medium, AFrame) Command of current practices in VR design Several weekly explorations (both low-tech and virtual) A final interactive Virtual Reality project (which can be executed in Unity or on another platform)

This course re-conceives interactive media as a form of choreographic intervention. Instead of asking how moving bodies can control media, we will ask how interactive systems can influence movement. How do you make someone feel soft inside? How do you shake an entire room? How do you orchestrate duets between strangers? To accomplish this, the class facilitates a semester-long collaboration between ITP students and dancers from the Barnard/Columbia Dance Department. Choreographers will learn to apply computational thinking to choreography and creative coders will learn to apply choreographic thinking to computation. To whatever extent possible, we will attempt to embody code. Using computer vision and visual media, we will look at directing both how people move (quality of movement) as well as where they move (pathways and spatial relationships). We will evaluate the strengths and weaknesses of the various sensing technologies available to us today. How wide is the gulf between what we can see and feel (strength, hardness, contortion) and what a computer can see and interpret (locations, contours, velocity, acceleration)? Class time will be split between movement exercises, playing with examples and deconstructing code. The class will culminate in a final showing of student work. All classes will take place at NYU with a separate weekly technical lab for ITP students.

Rules of play shape competitive games from checkers to football. But how do rules of interaction shape everyday life? And what happens when there are no established conventions and the rules are being made up as we go along as they have been for the past 6 months of living almost exclusively online? In this course, we will explore, code and test design strategies for playful, serious and bizarre group interactions while at the same time interrogating both what it means to play and how individual identities and group behaviors emerge. (The interactions we create in this class will be designed specifically for synchronous, in-person, embodied experiences.) Some of the questions we will ask and attempt to answer: What motivates participation? What hinders it? When does participation become oppressive? What's the difference between self-consciousness and self-awareness? Who has power? Who doesn't? Are leaders necessary? What’s the difference between taking turns and engaging in conversation? What happens when the slowest person sets the pace? And most importantly, what happens when there are no rules? Interaction inputs we will play with will include: mouse, keyboard, mobile device sensors, and microphone. Outputs can include visuals, text and sound. We will use p5, websockets and node.js for real-time interaction. Class time will be split between group improvisation exercises, playing with and critiquing examples and translating design strategies into code and logic.

For better or worse humanity is heading down the virtual rabbit hole. We’re trading an increasingly hostile natural environment for a socially networked and commercially driven artificial one. Whether it's the bedrooms of YouTube streaming stars, the augmented Pokestops of Pokemon Go, the breakout rooms of a Zoom meeting, or even the "airspace" of Airbnb; we are witnessing a dramatic transformation of what occupying space means. The socially distanced measures as a result of the COVID-19 pandemic have only accelerated this societal embrace of the virtual. So where are these dramatic spatial paradigm shifts occurring? Who owns and occupies these spaces? Who are the architects and what historical and ethical foundations are they working from? What world do they want to build for humanity and where does the creative individual fit into it? Will it be a walled garden, a role-playing adventure or a tool for creating more worlds? The course will ask students to embrace the role of virtual architect, not in the traditional brick-and-mortar sense of constructing shelter, but in terms of the engagement with the raw concept of space. However this virtual space must be considered and evaluated as a “site,” that is activated and occupied by real people and all the limitations of physical space that they bring with them from the real world. This is the foundation of synthetic architecture; simulated space met with biological perception. This conceptual architecture is free from the confines of physics but host to a whole new set of questions: How do we embrace the human factors of a dimensionless environment? How do we make or encourage meaningful interactions within the limits of current technology? New models of interaction must inform and shape the architecture of virtual space - what does that look like? How can architecture and aesthetics inform the creation of virtual environments and immersive narratives? How do we acutely consider the psychological and social impacts of the worlds we design and what is the metaphorical ground plane to make sense of this virtual world, unbound by physics? About Jonathan Turner: http://www.jonathanwilliamturner.com/about/

One of the most transformative consumer products in history, the iPhone remains the standard bearer for great design and user experience. With the latest version of iOS and the introduction of the iPhone X, Apple puts depth sensing and augmented reality in our pockets. How do we take advantage of this incredible platform to produce our own compelling experiences? This course will be a hands-on workshop where we explore the world beyond generic apps and push the boundaries of what’s possible on iOS hardware. We will cover both the design and technical elements that pertain to end-to-end mobile app development. Topics will include augmented reality, the Internet of Things, new interfaces for musical expression, voice interaction, wearables, and others... While the class will involve iOS programming instruction, the instructors will help facilitate coding using an established code base to ensure technical skills are not a bottleneck to implementation. Students should expect to spend additional time outside of class learning Swift and other related programming concepts. Full-time access to an iOS device and a Mac laptop running the latest operating systems are required. As part of the design process, we’ll host workshops and guest critiques with designers from top studios around New York City. At the end of the class, students will have a working app to add to their portfolios and a strong basis for future mobile application work.

Across the City’s universities, the Technology, Media and Democracy program will bring together journalism, design, and technical disciplines to understand the various threats to journalism and media, and attempt to address these challenges using technical and computational methods and techniques. The free press, journalism and the media are some of the most critical elements of our democracy, but have been increasingly under attack by political and market forces. These challenges include: dwindling resources and support for deep investigative journalism; smear, law and technical and even physical assaults of media organizations and journalists; challenges to credibility and reliability including fake news and discrediting campaigns; and shifting business models and income sources that threaten both local and national news organizations and coverage. This course will include various elements that will help frame the problem and build/prototype solutions.

There are more “free” applications and services than ever before that help us to quantify and track what we do, when, how, and with whom. The quantified self holds the promise of improving our lives, but there is an ambivalence to how these technologies are affecting our lives. This course will examine, question, and critique the perspectives of personal data and “the quantified self” from multiple perspectives. We will explore these perspectives by working with the tools and methodologies for collecting personal data and generate visuals and other tangible output from these data. We will introduce students to guest speakers, review and critique readings, projects, and software around the quantified self, and experiment through lab-based exercises that encourage a fluency with digital and analog visualization and data tracking tools. In this 12 week course, students will explore the topic of “the quantified self”. Together we will learn how to create narratives from our own personal data by collecting, retrieving, and analyzing patterns in our data, sketching and designing visualizations (e.g. charts, maps, etc), and developing programmatic methods to generate output from these data.

The class will focus on the many overlooked aspects of paper, and how it can be used as a three-dimensional material. We will learn the disciplines of making Pop-Ups, Origami, Paper Crafting, and Visual Design. Using these methods as a starting point, students will build prototypes to explore new ways to tell stories, inform, interact, play with, engage, and challenge a younger audience. Most classes are hands – on. The rest, dedicated to criticism (including from children), analysis, and refinement, technical and conceptual. We will discuss how they could be mass produced and distributed. Students will build three prototypes, during the semester. From these, each student will select a favorite to fully develop as the final.

What do WhatsApp and Nelson Mandela have in common? How about Mr. Robot and the UN Council for Human Rights? When most people think “digital security”, they rarely think of these connections; but the connections are there. Digital security is much more than an industry buzzword— it encompasses techno-social idealism, open source development, and symbiotic coordination between sectors in tech, the humanities, and civic society. Certainly, we’re going to talk about Signal, Tor, VPNs, and OTR. But let’s dig even deeper. In this course, students will learn the principals of digital security; from end-to-end encryption, to circumvention technology, resilient communications, and beyond. Part lab exploration, part oral history, and part prototyping workshop, students will come away with the tools they need to undertake development projects with end-user security in mind; and foster an appreciation for digital security’s integral, timely, and often life-saving role in human rights struggles across the globe.

With emerging research and development with soft circuit technologies and its integration into textile and clothing design, the garment as a reactive interface opens up new possibilities in engendering self-expressions, sensory experiences and more. This 14-week class is to introduce students to this realm by creating connections between hardware engineering and textile crafting. The class is for students with basic physical computing knowledge to explore the possibility of wearables, and arouse discussion about the potential in re-imagining our relationship with personal devices, textiles and garment design as an interactive media.

Sculpture is defined as a three-dimensional form of artistic expression concerned with space: occupying it, relating to it, and influencing the perception of it. In this class we will look at new ways of implementing video mapping, interactive time based media and augmented reality as a medium for creating engaging interactive physical and virtual sculptures. How do we create video sculptures that move, emote and react to our presence? The course will focus on taking video off the screen and into three-dimensional space in the form of site-specific and or physical installation. Through a series of weekly experiments and assignments, students will work with projection, video mapping, mixed reality and physical sensors to hack video into meaningful works of art. Class will be divided between lectures, guest speakers and critical discussion/presentation of work.

This course explores alternative forms of permaculture while carving out a space for designers to develop technologies, products, methods or systems to facilitate urban gardening. Students of all creative backgrounds and skills sets, both tech and non tech are invited to apply to this course. Students will will learn how to use their creativity and existing skills to design and produce products and systems that investigate and help transform urban environments into green spaces. The course is both a seminar class and a production studio. Final outputs and projects for this course could include the following examples: Designing software or systems to map potential guerrilla gardening sites in NYC; working with community gardens to create tools that help cultivate and harvest food in urban centres; developing interactive technologies to monitor and improve plant health in urban spaces. This course will invite participants to interact with specific communities and sub-cultures in order to create user-centred designs that consider environmental factors, socio-economic conditions and cultural implications. What tools, technologies and methods can be designed in order to facilitate urban gardening? Over the course of 3 weeks, students will meet once per week for 3 hours to explore the role technology and design can play in transforming urban environments. Students will touch on the history of urban gardening and how it has impacted the way we see and experience our cities. While working closely with local groups, students will develop and field test the designs they have created. Students will also have the opportunity to consult with experts who can advise on suitable plants and sowing methods. The second part of this course will allow students to apply their research to design tools and systems that can assist gardeners in creating green spaces. These designs should be functional and provide gardeners with products that can be integrated into their everyday routines. Through an iterative process, students will prototype, test and eventually produce their designs. By the end of this course, students will have had an introduction to the history of urban permaculture as well as alternative forms of gardening, connected with specific gardening groups, explored existing gardening technologies and designed solutions that can help better urban gardening.

Is it a plaything? Sculpture? Nostalgia? A Product? Art toys exist at the center of a unique Venn diagram. Each student in this class will develop an original limited edition art toy. We will cover toy fabrication, character design, material selection, packaging design, and art toy culture. The class will be fabrication heavy, there will be weekly assignments, and a final project.

Beginning with the release of Crowther and Woods’ “Colossal Cave Adventure” in 1977, the potential and unique affordances of computation as a means of storytelling have become more and more apparent. Combining approaches from literary theory, anthropology, computational creativity and game design, this class considers how narrative structure can be represented as data and enacted through computation, and invites students to implement practical prototypes of their own interactive and procedurally-generated narratives using a variety of technologies. Topics include (but are not limited to) hypertext fiction, “choose your own adventure”-style branching narratives, text adventures, visual novels, story generation from grammars and agent-based simulations. Students will complete a series of bite-size weekly assignments to present for in-class critique. Each session will also feature lectures, class discussion, and technical tutorials.

This course considers aspects of the materiality of typography and type design in the context of electronic media. Students will gain an understanding of how letterforms, typefaces and the layout of text have been represented as data throughout the history of electronic media, and experiment with different ways to author, manipulate and misuse that data through computation. Our eclectic and opinionated historical cross-section of topics includes (but is not limited to) typewriter art, minimalist and asemic poetry, 8-bit home computer text modes, interactive/kinetic text, parametric and generative fonts, and emoji. Students will complete a series of weekly assignments for presentation and critique in each session. In addition to critique, each session will feature lectures, class discussion, and technical tutorials. Prerequisites: Introduction to Computational Media or equivalent programming experience.

The course focus is on the design and creation of digital musical instruments. Music in performance is the primary subject of this class. We approach questions such as "What is performance?" "What makes a musical interface intuitive and emotionally immediate?" and "How do we create meaningful correlations between performance gestures and their musical consequences?" Over the semester, we look at many examples of current work by creators of musical interfaces, and discuss a wide range of issues facing technology-enabled performance - such as novice versus virtuoso performers, discrete versus continuous data control, the importance of haptic responsiveness as well as the relationship between musical performance and visual display. Extensive readings and case studies provide background for class discussions on the theory and practice of designing gestural controllers for musical performance. Students design and prototype a musical instrument - a complete system encompassing musical controller, algorithm for mapping input to sound, and the sound output itself. A technical framework for prototyping performance controllers is made available. Students focus on musical composition and improvisation techniques as they prepare their prototypes for live performance. The class culminates in a musical performance where students (or invited musicians) will demonstrate their instruments. Prerequisites: H79.2233 (Introduction to Computational Media) and H79.2301 (Physical Computing).

The 17th century philosopher Spinoza described "wonder" as a state of suspension in the mind, a paralysis resulting from a confrontation with something wholly new, disconnected from past experience such that judgements of whether it is good or bad are not possible. At this moment in time, we are caught in such a state of suspension with digital technologies. Creating computer applications instead of simply using them will provide you with a deeper understanding for the essential possibilities, limitations and unknowns of computation. The first half of Introduction to Computational Media focuses on the fundamentals of programming the computer (variables, conditionals, iteration, functions, and objects) and includes a basic introduction to HTML5/DOM. The JavaScript-based 'p5.js' programming framework is the primary vehicle for the class. All sections assume no programming experience at all. The second half focuses on applying fundamental programming concepts to generate and manipulate various media including imagery, sound and text and data. This course assumes a working knowledge of JavaScript and the 'p5.js' programming framework. Students who already have experience with programming in JavaScript and p5.js may join the course in the second half. Pre-requisite: ICM: Code or equivalent programming experience.

This class focuses on the art of computer graphics and image processing. We explore the concepts of pixilation, image representation and granularity and the tension between reality and image. Students are introduced to the tools and techniques of creating dynamic and interactive computer images from scratch, manipulating and processing existing images and videos, compositing and transitioning multiple images, tracking and masking live video, compositing and manipulating live video as well as manipulating depth information from Kinect. The class uses Processing.

This course covers the next several years of evolution in technology, culture, and other trends. It uses scenario planning, a technique for considering complex interrelationships that can’t be predicted, distinguishing predetermined elements from critical uncertainties, and exploring the underlying patterns that influence events. Students will conduct original research on significant trends, use those trends to develop compelling, sophisticated, plausible stories about possible futures, and present the futures – and the strategies they suggest – to a public audience. The course will take place at a pivotal moment of historical uncertainty: recovering from a global pandemic, with AI and other digital technologies crossing a threshold, and dramatic political and economic tensions. All of these, and more, affect media development - and are deeply affected by them. The goal of the course is to enable you to make more robust decisions now in the face of uncertainty — applicable to planning for technological change, starting a business, plotting a career or making major life decisions. This class has developed a longstanding following at ITP because it helps us make sense of complex issues without oversimplifying them. In a climate of candid, respectful discussion and debate, the class explores theories about system dynamics, long-wave organizational and societal change, and economic and technological development.

This course expands the students' palette for physical interaction design with computational media. We look away from the limitations of the mouse, keyboard and monitor interface of today's computers, and start instead with the expressive capabilities of the human body. We consider uses of the computer for more than just information retrieval and processing, and at locations other than the home or the office. The platform for the class is a microcontroller, a single-chip computer that can fit in your hand. The core technical concepts include digital, analog and serial input and output. Core interaction design concepts include user observation, affordances, and converting physical action into digital information. Students have weekly lab exercises to build skills with the microcontroller and related tools, and longer assignments in which they apply the principles from weekly labs in creative applications. Both individual work and group work is required.

In this course, students will be introduced to 3D rendering and modeling techniques within the context of international architectural practices. Students will be exposed to scholarship and research surrounding form, structure, aesthetics, and philosophies to engage in a design practice rooted in both societal context and experimentalism. Throughout the course, students will engage in theories around psychogeography and other design-oriented psychologies while studying international design practices through readings, guest lectures, and class-wide conversations. By the end of the course, students will develop their skills in comfortability engaging in an electromedia practice that moves from ideation and research into digitally rendered 3D environments. The course will culminate with students creating WebVR environments with their 3D renderings. This course will be taught in Cinema4D with exposure to Rhino3D. Students are also invited to explore other technologies like Blender.

How could we reimagine and expand our interactions and relationships with voice-enabled devices? Artificially synthesized voice introduced personal devices beyond push buttons, twisting and swiping. Nonetheless, while mediating notions of service, facilitated access to knowledge, and digital companionship, currently voice assistants are often reduced to trivial, task-oriented power plays: order me this, turn on that, play these, entertain. The course will aim to reconsider voice assistants as subjects rather than objects and attempt to create more holistic relationships with them through speculative and critical design methods in unison with some programming and machine-learning.

In this course, students will explore and engage with Afrocentric and countercultural design practices and theories when considering pedagogy, technology, community building, and social engagement. Through guided research, design exercises, conversations around readings and other media, along with guest presenters, students will be exposed to an omni-specialized and comprehensive design practice that cultivates radically imaginative futures. This course will introduce students to the philosophies and forms of Afrotectopia, a pioneering social institution developed out of ITP, and equip students with the skills to build thoughtful communities and spaces for creativity in and outside of the classroom. The course will culminate with students materializing their research into an immersive experience (whether a workshop, theatrical event, zine, short film, etc.) that realizes a new future and healthy societal order.

Over the past 3 years, we have seen many aspects of our lives thrust online. Increasingly, we are working, learning, socializing with family and friends, attending live performances and more through 2D grids of video feeds on platforms such as Zoom and Google Meet. These communication tools have become essential for remote communities to connect, yet fail to replicate many of the most engaging, messy and human aspects of our in-person experience. What happens when we break out of this grid and explore new forms of real-time social interactions online using webcam video and audio? Recent explorations in this realm have shown the promise of spatial metaphors in creating engaging real-time social interactions online. In this course, students will create their own series of experimental social spaces that explore these questions: how does the shape and nature of our environment affect the way we communicate? What unique forms of real-time expression and sharing might be possible online (and only online)? How might we design experiences for the unique social dynamics we want to support? Students will be exposed to principles of spatial design as well as a series of open source Javascript tools for arranging live webcam video and audio in 2D and 3D space in the browser. They will use WebGL (through the three.js library) to build 2D and 3D environments, and will be exposed to WebRTC (Web Real-Time Communications) and Node.js to add interactivity to those environments.

Listening is not something we do through hearing alone. Engaging with multiple perspectives, from Deaf studies and critiques of ableist hearing ideologies, to the possibilities and pitfalls presented through machine listening and imaginative sonic speculation, we will playfully deconstruct and question what it means to listen at all. Each week, readings will be assigned and students are asked to respond to prompts in the form of light-weight exercises that will orient most of our in-class discussion. Time will be spent discussing readings, presenting, and providing critique for each other’s projects in order to help draw connections between theory and practice. Through in-class presentations, we will encounter works by Christine Sun Kim, George Lewis, Pauline Oliveros and read from Jennifer Lynn Stoever, Tina Campt, and François Bonnet. The class has no technical prerequisites, and students are welcome to respond with works using tools and techniques from other classes (audio/video presentation, programming and physical computing, installation and fabrication, etc.). Through this work, we will unpack how the way in which we attend to the physical world and its inhabitants through our listening has real consequences. In much the same way our interpretation of the world is informed and influenced by factors outside of ourselves, our capacity to listen is conditioned through societal, political, economic, historical, and racial dimensions. We will disentangle listening from hearing and consider listening as a practice that begins not with how we hear sounds but how it allows us to (or prevents us from) interfacing and relating to our exteriors (and interiors). By the end of the class we will attempt to converge and coalesce our own ideas and perspective of what it means to listen.

This course will offer techniques for producing performances and multi-media broadcasts via video conferencing platforms. We will consider the politician’s use and abuse of performing arts and media strategies. Assignments, screenings and readings will trace the lineage of political performance and its mediation from François Delsarte’s 19th century system for oratorical expression to current live-feed montaging used in American political campaigns. Taking into account the use of visual mediation in both process and propagandistic product, we will look at how scientific management techniques such as stop motion filmmaking are used to maximize the effect of the politician’s every gesture and word. As live performances have shifted to online platforms in our quarantine era, how do we incorporate changing notions of liveness and social cohesion into artistic and political practice? We will examine the influence of the improvisational acting style promoted by reality TV to produce bombastic, and hence entertaining performances. In our public discourse authenticity is now associated with improv acting as opposed to traditional methods of rehearsing lines to be performed, now regarded as phony. Exploring the increasing use of Zoom, Instagram live and Tiktok by theater, dance and visual artists, we will analyze the impact of pre-recorded versus live stream video, as well as the appropriation of surveillance techniques inherent in these new media.

Wireless technologies have become a very critical part of our lives. However, even for some electrical engineers, radio frequency (RF) related topics are considered to be somewhat governed by “black magic”. This course aims to provide an intuitive understanding of how RF circuits and systems work, and to help you utilize wireless technologies in a more predictable manner. Topics such as wave propagation, modulation, RF transceiver topology, and basic antenna design will be covered. Lab sessions are tailored to provide hands-on exploration of various RF circuits/modules and their real- world applications, such as wireless Arduino communications, antenna fabrication and testing, and oscilloscope / spectrum analyzer measurements. Basic circuit design and prototyping skills are preferred, but a review class is provided at the beginning of the course. Schedule (7 weeks) 2.5-hr lecture, plus 1.5-hr office-hour per week Class 1 Recap of fundamentals of circuit design, prototyping, and Arduino topics. Class 2 Basic wireless communication concepts – wave propagation and modulation (w/ in-class demonstrations) Class 3 Lab 1 – Signal measurements (oscilloscope and spectrum analyzer/software defined radio) Class 4 Basic wireless communication concepts – transceiver architecture and antenna (w/ in-class demonstrations) Class 5 Lab 2 – Simple transceiver (Arduino communication through ASK module) Class 6 Project – Antenna design and fabrication Class 7 Project – Antenna measurement and integration

This 7-week class focuses on design as a vehicle for change. As designers, we have the opportunity to create products and experiences that can change mindsets, break patterns and introduce new behaviors in ways that can have meaningful impact at both an individual and societal scale. This course combines the practical skills of UX design with the conceptual skills of behavioral psychology to provide a framework for designing products focused on initiating meaningful change. This class is for students with passion for an issue and enthusiasm to apply their creative and technical skills to solve real problems in meaningful ways. The course is both a seminar class and a production studio.

This class looks at "creative signal processing" in the same way we look at "creative coding" - celebrating our ability to manipulate signals - functions and blocks of information that travel across time, frequency, and space - in the wider context of human perception, communication, and culture. In this course, we will consider standard building blocks of signal processing - audio and image synthesis, time- and frequency-domain processing of audio signals, chromatic and spatial processing of images, and feature extraction for signal analysis - in the same context as human languages, writing systems, and cultures of generative and procedural design in music, performance, and visual art. The project scope for the course is designed to be flexible and open-ended, allowing for students to develop projects ranging from guitar effects pedals to audio-visual plug-ins to preprocessing workflows for machine learning to toolkits relevant to integrating real-time data into interactive performance systems. Students will be assigned a series of short exercises around each general theme, and will be expected to develop a semester-length project. The class will use the Gen architecture of Max/MSP/Jitter to prototype algorithms for C++ and GLSL, with explorations into other coding environments, hardware circuits, and physical explorations of signal processing along the way.

2020 has amplified foundational flaws in our ecological, food, racial and environmental systems creating a persistent state of whiplash from crises, climate related catastrophes, racial injustice and industry meltdowns. While these areas may seem unrelated they actually exist within a Venn diagram of interdependency and influence where pulling a lever in one area directly impacts another area and the ability to shapeshift and adapt in the face of these rapid changes has become required for survival. This class will uncover the connections within these systems and use science, design and systems thinking to explore ways to create actionable interventions – either analog or digital – to address both the core problems and the collateral impact. Speakers from these industries: farming, restaurants, BLM, climate and ecological sciences - will present specific problems within these fields to the class in order to inspire the creation of small projects to tackle these issues. For example, interventions could be a particle sensor for a restaurant HVAC system, algorithms for precision agriculture, AI to mitigate racial profiling, data visualizations and infographics to illustrate pathways to better climate resilience.

People think in stories. They're how we make sense of the world. Professionally, it isn’t just artists who tell stories: activists, advertisers, and politicians all know that a compelling story can capture an audience in a way that arguments and statistics can’t. This course will introduce students to the fundamentals of narrative structure across a variety of media: written stories, movies, games, comics and graphic novels. Across these media, students will learn to identify the key elements that grab people's hearts and minds. Outside of class, students will focus on developing one interactive or transmedia project inspired by a work we study in class. Students will walk away with an understanding of, and practice working with, the principles of storytelling.

Fast Fun: Physical Controllers for Unity will guide students in the creation of game controllers for Unity applications. Leveraging simple circuits and easy communications protocols, students will create physical controllers to control their Unity creations. Strategies will be oriented towards quickly creating multiple, effective prototypes. An emphasis will be put on reusable, ecofriendly materials and methods that work in your bedroom as well as the shop or lab. This class will be part code workshop, Physical Computing review, and HCI/UX analysis. Discussions of readings will frame and give context to technical exercises.

Generative Adversarial Networks (GANs) are the current state of the art machine learning technology to synthesize imagery. In Making Visual Art with GANs students will use popular deep learning models to create images, videos, and interactive experiences. The focus will be on hands-on experimentation with style transfer, CycleGAN/Pix2Pix, and StyleGAN2-ADA in RunwayML and Google Colab. The course will cover the theory behind deep learning and GANs, using pre-trained models, dataset creation, and training custom models. We'll look at ways to generate images and videos using Python, p5.js, as well as reacting to sound and motion inputs. Coding in Python is not required for this course, but a general knowledge of programming basics is encouraged.

50 Days of Making is a 1.0 unit online course that offers students the opportunity to pursue a creative passion and develop or refine a skill over a 50-day period. Students choose a topic of interest and produce an expression of that topic every day for 50 days. For examples of past projects from the 100-days version of the class see here: https://itp.nyu.edu/classes/100days/. This course will meet four times on a bi-weekly basis over the course of the 1st 7-weeks of the term (every other week). Class time is spent discussing student progress and reflecting on students' creative journey. Note that this class is a heavy lift for 1.0 unit, so only committed students should consider registering for it. Failure to complete the 50-day challenge may result in an incomplete grade for the course.

This course will use the open source COVID-19 Impact Dashboard as a basis to explore ways to humanize the unfolding data on the coronavirus pandemic. Students are invited to collaborate on the COVID-19 Impact Project. Students will discover how data flows from public github repositories and tools needed to visualize the data. We will review other data-centric open source projects related to COVID-19 and discuss the questions they are trying to answer or problems they are trying to solve. We will examine historical and contemporary data visualizations. Using data visualization as a scaffold, we will explore ways to support community driven mourning and memorialization. Students can choose to participate as javascript coders, p5js explorers, UI/UX designers, citizen journalists, or data science explorers.

A new world is emerging at the intersections of machine learning and physical computation that will offer wide-scale access to bringing intelligence to everyday devices and spaces at extremely low costs. In this course, students are offered the opportunity to become pioneers in a new field of hardware machine learning as they are introduced to the most used machine learning platform in the world (TensorFlow) that has been embedded into an incredibly small microcontroller, called TinyML. Students will learn about building with machine learning, the ethics and societal impacts of ML, and how to start realizing creative computation through ML-based physical computing.

Narrative holds a place in discourses of health, illness, caregiving, and disability, carrying and conveying the densely detailed, nuanced, and complex threads of personal emotion, social experience, and cultural meaning that accompany all instances of these subjects. Narrative also plays a growing role in clinical practice, research, and health education, as increasingly registered in the burgeoning field of Medical Humanities. This course introduces students to texts, practices and major works in the emergent fields of Graphic Medicine and Narrative Medicine, using traditional humanities methods of critical reading and analysis as well as experimental and creative methods including field observation and art-making in a variety of media. Building upon a series of practice-based assignments throughout the semester, students will complete a final project that exemplifies some of the ways narrative and graphic design foster understanding and knowledge in contexts of illness.

So much of life is circumstance, being in the right place at the right moment in time. Can we shape a life or at least a few brief moments of one by designing a space for it to inhabit? In this course, we will treat space as a time-based medium and ask how experiential spaces can generate narratives that are lived rather than told. We will look to a broad range of storytelling traditions to interrogate four so-called “space-narrative” forms: Wandering The Desert, Processions, Circles and Territories. Through play, discussion and technical and conceptual workshops, we will ask and attempt to answer some of the following questions: What constitutes a space? How do we experience a space over time? How does space shape our experience of time? How can space shape both a personal and collective narrative? We will build spaces with lighting, projection, sound and physical objects. Class time will be split between group improvisation exercises, playing-testing and critiquing projects. The class will culminate in a showing of work at the end of the semester.

This course will focus on the prototyping of wearable electronics projects for a single user: you. In this class we will wear what we make, following an iterative cycle of research-design-make-wear. Lectures, readings, and discussions will serve to provide historical and contemporary framing for our work. Wearable technology prototyping strategies and techniques will be shared and tested. Special focus will be placed on circuit building and fabrication approaches that are compatible with a home studio environment. Previous experience with electronics or physical computing is strongly encouraged. Students will emerge from the course with a deeper knowledge of what it takes to develop and refine a robust, bespoke wearable electronics project.

Live streaming is so seamlessly embedded into our online experience. We lay in bed, on our phones watching hearts flicker across the screen as the person we’re watching greets all of the competing messages in the chat, asking for birthday shout outs and follow-backs. While the ability to live stream feels more accessible than ever, it feels very tied to corporate structures, branding and self promotion. How can we push the concept of a live stream in a new direction and rethink what a live stream can be?

What can an object tell us about the world? What can the world tell us about an object? As students and practitioners in creative media, investigating the world around us is a core part of an art/design/tech practice. This course focuses on a specific research approach for doing so: the Implosion Method (developed by Joseph Dumit based on Donna Haraway’s work). Over the course of this two-weekend workshop, we will individually and collectively critically examine, break down, rearrange and communicate as Haraway says, the “sticky economic, technical, political, organic, historical, mythic, and textual threads that make up [an object’s] tissues.” Students will choose an object in the first class and over the course of a week will follow the Implosion Method process, culminating in a presentation of findings the following weekend. The implosion methodology will be supplemented by lectures, group conversation, hands-on activities and skill building of complementary research methods. The final presentation will be to communicate findings and learnings through a multimedia or art/design/tech-based form. By working through the implosion method, students will build skill sets in methods and tools for 1) conducting research in the media studies/creative tech/art fields and 2) structuring and communicating the information they collect by creating a media-based representation. This class is for students interested in fostering, developing, starting, or deepening a research-based art practice with topical interests in how technology relates to society, social issues, and ethics.

This course traces along the scholarship of Anna Tsing and Donna Haraway, and Animist and Perspectivist cosmologies to study and consider our positions within multispecies relations via material exploration and interspecies storytelling. This is a hands-on course engaging in communal growing and fabrication using biomaterials such as bacterial cellulose, mycelium, lactobacilli, yeast, and more. During the course students will participate in guided somatic exercises and writing as a generative pathway to create their own multispecies allegories and exploratory projects. We will look over and talk about the work by artists such as: Natalie Jeremijenko, CAConrad, Alexis Pauline Gumbs, Anicka Yi, Bo Zheng, Una Chaudhuri, Agnieszka Kurant and Ernst Karel.

In this course, we will explore how to create narratives that leverage our lesser used senses like touch, taste and smell as well as lesser-known ones like space, time, balance and scale. We will dig into the history of experiential storytelling, starting from immersive theater and Smell-O-vision to cutting-edge haptics and mind-bending illusions of proprioception. To help center this back in practical applications, we will also explore how this evolving art is commonly used in exhibition design, experiential marketing and brick and mortar retail. The class will be a healthy mixture of game theory as well as experienced based learning (meaning there will be a couple field trips and multisensory VR projects to explore). A basic knowledge of game engines is ideal but not mandatory because we will be using predesigned templates in Unreal engine to be experienced and manipulated in real-time through virtual reality hardware.

This course aims to provide students with the analytical skills to interpret current trends, policies and problems into futuristic (5+ years) product proposals and the thought leadership and communication skills to clearly articulate and pitch those ideas. Projects can range from how prolonged quarantine will impact home exercise to the future of entertainment in self driving cars. Students will research and explore an opportunity space of their choosing where they will infer future problems from current trends then create a speculative solution. They will package their product thinking into a pitch deck and present back to the class. The class format will include lecture, in-class and out-of-class design exercises that apply the concepts covered in the class lecture and a final presentation and critique.

If social change begins in the imagination, how then can creators better envision and render the more just and beautiful worlds we want to make? Storytelling has the power to be an alchemical force for revolutionary change. Together, we seek to interrogate and apply interactive storytelling as a technology we can deploy in service of our collective liberation. In this course, we pair a study of story as liberatory praxis with a hands-on grounding in emerging tech tools that allow viewers/players to take an active role. Interactive storytelling technology in video, audio, and text powerfully situates viewers inside constructed narrative worlds. Creators in these emerging media gain the capacity to design choices and respond accordingly, propelling imagination toward agency and enhancing empathic connections between viewers/players and characters. What will it mean to use these tools to tell deeper stories that ask urgent questions about how we want to live in the world?

Course description (optional): Not all innovation starts in the West and gets exported to other parts of the world. In many places with less developed capitalist economies and infrastructures, technology is rapidly developed and adapted for hyper-local use. We'll gain inspiration from a broad spectrum of creative uses of technology in the developing world(s) -- from art and design, hacktivism, and community-oriented work that increase social good, and then conceive of and prototype our own projects. Special attention will be paid to speculative 3D fabrication tools and processes, web-based platforms, circuit bending and making them all work together! Students will experiment and research at the intersections of art and technology to develop the beginnings of work grounded in post-colonial reality and the late capitalist near-future.

Buildings produce a large percentage of the carbon emissions threatening the planet and multi-family residential buildings make up a significant portion of it. With Covid-19 changing the way we live and work, and the increasing amount of data available from buildings, a key to fighting the climate crisis will be turning this data into action. In this course, learn how to analyze interval data and explore visualizing data to motivate tenants and building operators to change their behavior to reduce energy usage at the optimal times. This will involve analyzing the data streams coming from installed sensors and building equipment, understanding how usage varies over time, and transforming raw data into visual interfaces that mobilize us all in the fight to save the planet. This course will teach basics of how time series data can be stored, how to query time series data, and how to understand energy usage from a data set. With these new skills students will design a project using time series data and their JavaScript skills to visualize this data.

Project-based development studio incorporating dramaturgy techniques, user/audience planning, and social/contextual awareness. You bring in a project. We explore how to make it more engaging through paying close attention to medium, context, and details. Students bring existing project ideas and we investigate various methods and ways to make and direct experience within the infinite combinations of contexts of the present moment together. This particular studio is just as appropriate for projects in the areas of interactive art, programming, physical computing, XR as it is in the areas of performance, sculpture, and sound walks (everything). Everything you make is time-based the moment a user interacts with it (even a painting). All time-based work can be thought of in terms of how a user is led (or not led) through it. I call this directing. We will apply various techniques of story-telling and world-building equally to seemingly “non-narrative” projects as we do to traditional-narratively structured projects. No matter what you are working on, I believe that you are a maker / director of experience. You are making / directing with intent (whether you know it or not). Your user (audience, tester, public, patron) brings with them the entirety of their life’s experience. Your intent cannot possibly meet every user's lived experience. It is your job as the maker / director to draw a circle that encompasses both. This is the studio’s lens. Students will be directed to make using placeholders instead of waiting for perfection to manifest. Step one will tell you what step two is. We will pay rigorous attention to detail, while holding close to the notion that art-making thrives in adaptability. We will critique using various, structured, co-facilitated methods. Let’s pay close attention. Let’s learn through failure. Let’s listen to what we are making. And let’s listen to each other. Here we go.

Throughout history, musicians have channeled their creativity into outrageous fashion statements and invented personas: think MF DOOM, Sun Ra, Ghostface Killah, Daft Punk, Leikeli47 and Rammellzee. By embracing their alter egos in extreme and outlandish ways, artists have found their authentic creative voices. This course will introduce participants to the art of masquerade using their resourcefulness to create costumes from found materials, and performance as an exploration in creative expression using new media and technology. Students will be introduced to ideas surrounding abstract storytelling, experimental audio + video production, and A/V performance using a combination of technical and hands-on approaches.

To be a VR creator, it's not enough to learn the hard skills—it's also our responsibility to prime ourselves for the human impact of our work. As a means to design VR that is both enjoyable and accountable, this class proposes we borrow design principles from Hedonomics, a branch of ergonomic science that facilitates pleasurable human-technology interaction. Through the Hedonomic Pyramid, we’re able to section our thinking off into regions (Safety, Function, Usability, Pleasure and Individuation) and map out industry-tested VR design guidance for each. The result is a hierarchical checklist of proven principles, specifications and practices—that promote a culture of inclusive and holistic design—built to serve as a quickstart guide to designing accountable VR interfaces and systems. This class, divided into units that represent each level of the Hedonomic pyramid, will unpack both technical and conceptual strategies for creating VR, from visual interface fidelity to avoiding locomotion cybersickness to designing safer social VR spaces.

Artists and creative practitioners are widely seen as responsible for destabilizing or critiquing dominant cultural norms and social systems; for example, in her book "Race After Technology", Ruha Benjamin argues that artists "...can better understand and expose the many forms of discrimination embedded in and enabled by technology". But when we talk about resistance or refusal enacted through creative means, what exactly do we mean? How does artistic work serve to resist, protest and subvert—and where does it sit in relation to the ideas it aims to critique? This class engages with the notion of "creative resistance", unpacks the meanings and ethical stances associated with the term, and evaluates how it has been applied in both artistic and academic contexts. Students will explore theories of resistance, refusal and solidarity, and experiment with applying them to (or discussing them through) creative work. In the first half of the class, through reading, class discussion and student presentations, we will engage with scholarly and activist literature on resistance, protest and subversion, and look at examples of creative technological works that purport to achieve these goals. We will discuss commonly used strategies such as dark sousveillance (or "looking back" at the machine), speculative design, and distributed or guerrilla artmaking; examine their mechanisms of action; and debate their effectiveness in achieving their professed goals. In the second half, students will apply this thinking to their own project ideas. Students can choose to produce either a final project responding to the themes of the course, accompanied by a short written artist statement, or a more academic written piece that engages relevant literature from the class readings and beyond in service of an argument about the role of resistance in creative practice.

With recent advances in hardware and software, millions of us now carry unprecedented spatial computing and world-sensing technologies in our pockets. Notably, the latest technologies extend beyond simple recognition into areas of comprehension and semantic understanding. And yet despite these advancements, most augmented reality experiences still closely resemble those from previous decades – 3D content overlaying the camera feed. Few experiences explore the full possibility space enabled by the latest technologies. In this hands-on workshop, we’ll explore a wide range of technologies and techniques for augmenting reality on iOS. Emphasis will be placed on experimentation – exploring the boundaries of what’s possible, embracing unpredictability, and hacking things together in unexpected ways. Topics will include spatial audio, computer vision, scene understanding and semantics, speech recognition and synthesis, music recognition (ShazamKit), haptics, body tracking, and more. Should Apple release its AR headset prior to the course or during the semester, we’ll find ways to incorporate the device into the class. For design and development, we'll use Apple technologies – ARKit, RealityKit and RealityComposer. 3D modeling skills are helpful but NOT required. While we'll cover the basics, students should expect to spend additional time outside of class learning Swift and other related programming concepts. Full-time access to an iOS device (LiDAR-enabled is ideal but not required) and a Mac laptop running the latest operating systems are required.

This course will serve as an incubator to imagine a speculative product advertisement in the year 2030. In films like Blade Runner, or Her adverts fill the world and become an important aspect of exposition for the film. And in the real world, works such as Alisha Wormlsey, Alexandra Bell, and Hank Willis Thomas begin to re-imagine advertisements as an art practice in society today. Our work will begin to speculate on near-future objects in which topics such as communication, energy storage, transportation can begin to be re-imagined in the next industrial revolution. Using 3D tools, students will gain experience in speculative design thinking, industrial design modeling, product lighting, and custom post-production methods. The final project will be a product advert that will be designed to promote a speculative design entirely made from 100% Biodegradable plastics. The course will look at the ready-made objects all around us as a launching pad. We will be starting with modeling an object in detail. Using Moi 3D, Maya, Render Engine TBD, After Effects, and premiere over the course of the semester. I will go through some of the latest tools within the VFX industry and support this course with a series of artists who have re-imaged the role of cultural production. The final will be an advertisement poster and animation.

Where does healing reside in media-making? How do we approach creating artistic processes and tools that move towards minimizing harm, supporting collective care, and understanding what healing means for ourselves and with one another? This course examines artistic processes and frameworks that reconstruct, reclaim, and decolonize ‘healing.’ Together, we will 1) gather embodied data from our daily lives, 2) critically engage with disability, race, queerness, and diasporic experience through reading and discussion, and, 3) develop our own healing processes through experimentative tool-making, performance work, and/ or space-making.

Investing in Futures is a two day speculative worlding workshop in which you and your classmates will co-imagine worlds (some absurd, impossible, likely, or improbable) and inhabit them through conversation, craft, and play. Day 1: You’ll use a constraint based design process to co-imagine a world you want to live in. You’ll be introduced to concepts of world-building, speculative design, and systems thinking to develop this world. Depending on class’ interest, there may also be a brief introduction to Unity3d as a tool for depicting your shared worlds - though note the workshop is tool-agnostic and you may use a medium or technology of your choosing. Then, over the course of a week you’ll create artifacts and a story around your co-imagined world (physical, digital, or performance-based). Day 2: you’ll use the artifacts you created to share back the story of your world. As a class, we’ll discuss the themes and ideas your world embodied! This class can be especially fruitful for students looking for a creative way to see their thesis research through a totally new lens! You might also consider taking this class if they are interested in: Designing interactive pedagogical tools and workshops, Having lively conversations with your peers about systems and our roles within them, Constraint-based design as a way of generative creativity, How imagination and play can produce change, A break from our world.

This1-pt course examines the evolving paradigm of "Blockchain Aesthetics" using an interdisciplinary Art History and Science and Technology Studies (STS) perspective. Students will learn and discuss the dominant methods, concepts, and strategies of applied blockchain technology in contemporary digital art, and together attempt to analyze and theorize the potential affordances and harms of this new artistic medium as it intervenes in established visual art institutions and communities (e.g., from museums and galleries to auction houses, social media and the decentralized web). The class will also look backwards in history to situate "Blockchain Aesthetics," and the wide artistic realm of "Crypto Art," in a broad cultural and technological history beyond NFTs, highlighting how ideas around cryptography, contracts, reproducibility, decentralization, scarcity and financialization developed in modern and contemporary art. Approaching the blockchain as a novel artistic tool and a technology of power, this course considers recent aesthetic evaluations of Crypto Art against its sociopolitical, infrastructural, and ecological impact, centering its contested potential for—and negation of—equity.

As the scale of human impact on global climate and ecosystems deepens, we see the need to alter our trajectory, to be more inclusive of other species in our imagining of the future. This class sets out to investigate the relationships we humans have with nature and non-human animals, to dive deep into the meaning and utility of being in relationship, and ultimately to translate these ideas into tangible, multimedia experiences that expose a larger audience to a multi-species worldview. This class sits at the intersection of art, science, and technology. It combines studio practice and research with example case studies and critical texts. Together, we will meet artists, designers and scientists who build multispecies futures through urban ecology, biology, and public art. This class is for students who are eager to develop XD (experience design) and storytelling skills. The course follows a research-driven process that results in a design proposal and proof-of-concept that can be pitched to a public arts org. Keywords: bioArt, interactive installations, experience design, research, eco-activism

As you know, projection mapping and Light Art are becoming popular again because of large-scale pop-up installations worldwide: ARTECHOUSE, SuperReal, Meow Wolf, and TeamLab. Technology has advanced over the years, but how people enjoy light art have not changed so much. How do your ideas and artwork fit into these site-specific installations? This class is for anyone interested in creating a site-specific installation using mapping technologies to create new experiences for the public audience. This class guides students through conceptual and technical processes of project and artist development. It consists of three parts: Project & Artist Development, Projection Mapping, and LED Mapping. We will research and discuss the history of visual artwork, public engagement, and technical exercises using real international contests and festival sites. The student will learn the latest Projection and LED Mapping techniques using Madmapper. And we will also focus on advanced techniques like multi-projector projection, projector calculation, Interactive Mapping, and software & hardware to culminate in a final project. The class will also invite guest speakers to discuss the nuts and bolts of their art and business. About Chika Iijima: www.mappathon.com, www.imagima.com

Everyone wants to be a YouTuber, but building a business as a digital creator is about more than just being an online celebrity. In this course, students will learn how to build a YouTube channel, from titles and thumbnails to video production to sponsor relationships to analytics and collaborations. We will examine what it takes to build a sustainable business around online video, learning from real-world examples and applying them to the students' own YouTube channels.

Course description (optional): The future of work is ________. Starting with this fill in the blank question, this class will examine the opportunities and challenges in the imminent future of work affected by the rise of automation, the advent of artificial intelligence, and the global pandemic. Students will identify, research, analyze problems related to the changes in the way we work, and propose project ideas that can help workers survive, prepare for, adapt to, or resist the changes. How can we make art and design projects—practical or speculative—to raise awareness about existing issues in the workplace such as intensified automation and algorithmic hiring bias? How do we empower and uplift workers in an ever so close future of ubiquitous surveillance and the gamification of everything? What are some creative ways to bring the team together in the age of remote work? These are among the questions that the class will explore. Activities in class will include topic-based workshops, discussions, project ideation, and student presentations. At the end of the class, students will have gained more understanding about the implications of the future of work, improved their research and presentation skills, learnt a few project ideation methodologies, and completed a research-based creative project proposal and prototype related to the topic of the class.

When we see the shape of an uppercase serif letterform, we may subconsciously be reminded of the Roman Empire. What we may not consciously realize is that this association has its roots in the technology used to make these letters, thousands of years ago. Serifs are a wedge-shaped artifact that occurs when a chisel hits stone—the tool used by the Roman Empire to carve their letterforms into monuments called capitals (now a word synonymous with “uppercase” due to this same history.) Though some debate exists among historians, it is widely believed that “capital” letters get their geometric shape from the constraints of the tool of the chisel itself. To understand how the wide stylistic variety of letterforms arrived in our font library (and to understand where our own hazy associations with letterforms originate), one must look to the technology which produced them. From the exigencies of the sign painter’s brush to the psychedelic warping of 1960s Phototype to the 8-bit pixel-based typefaces found in 80s video games, letterforms contain the technological history of the world in microcosm. The subtle choices in each typeface’s form bear the imprint of their moment’s philosophical, technological, and visual conditions, capturing an era’s zeitgeist with a miraculous economy of expression. The letters that we use today are more than 2,000 years old—persisting longer than any other artifacts in common use—but have undergone dramatic fluctuations alongside tech’s major physical transitions from stone to paper to metal to celluloid to digital information. Parallel to this technological history, letters shifted context from cuneiform to letterpress to Linotype to phototype to digital screens in a continual reinterpretation of the the fundamental question “what is a letter?” In the 1970s, technologists and computer scientists found themselves grappling with this same fundamental question as they carried letterforms over into the digital realm: What are letters? Are they fixed visual information? Or are they an idea—a set of executable, gestural instructions? Are letters best understood as reconfigurations of a set of modular parts— building-block components rather than the choreographed gestures of calligraphy? Are they the organic product of the human hand or the output of a system? Early digital technologies wagered “is this what computers are for?” with typefaces in tow—choosing which aspects of the old analog world to reconstruct—in deciding what attributes to port-over. The world we live in today has been impacted by how technologists answered these questions. Questions which, just as easily, could have been answered differently. This course will begin from a place of reflection on our own lived associations with typographic morphology. We will then explore the possible technological origins of those associations while reflecting upon how [what seemed like] tiny digitization decisions delivered us the typographic reality we inhabit today. Students will be asked to look to history for “reasons” for typographic form (which is fun!) But we will also practice looking to history for alternate futures—to examine the “dead ends” that might have otherwise been and daydream about where these paths lead. Typographic technological history offers a manageable jumping-off point for such a thought experiment. This thought experiment scales up to larger problem-solving (and conceptualization) skills related to understanding the implications and effects of tech.

The central focus of this fabrication class is 'replasticing.' Replasticing: the act of remaking/reforming single use plastic into new objects. In addition to learning about plastic's properties, various forms and history, students will also learn how to fabricate and 3D Print PLA Plastic, DIY recycle and use extruders and injection molds to recast "waste" plastic in their class projects. Students will then take a close look at the waste stream in NYC and Brooklyn, and research the end-of-life cycle for plastics. The class will culminate in a collaborative project contributing to and creating new solutions for the Tandon Makerspace in managing their excess of PLA 3D print waste. Solutions can be anything from designing recycled plastic objects and tools, to systems for community engagement and efficient processing of the PLA scraps in the Makerspace. By creating opportunities for communities to have access to DIY recycling, we will re-imagine waste; re-configure design practices; and re-value plastic's potential in a circular economy.

Cooking programs with an image based language is a fundamental skill in the production and design of modern digital processes. Visual programming is not only an alternative way to code, but a solution to approach generative and interactive Real-Time media. This class reviews the past, present and future of visual programming languages used to procedurally generate and manipulate media such as Max/MSP(Nato.0+55+3d), Isadora, Touch designer, Cables.gl, and Unreal Engine among others. The core of this course is the study of Unreal Engine’s Blueprint Visual Scripting as a way to produce prototypes that can be used in performance; however, the use of other visual programing environments such as Touch-designer, Max/MSP and Isadora will be encouraged as tools for prototyping and producing media in the Performance Arts. Prerequisites: Knowledge of a visual Programing Language such as Max/MSP, PureData, Blueprints and Touch-designer. Also knowing how to program in any Language is a desired skill.

In this course, students will explore the fundamentals of augmented reality by dissecting the interaction between camera, computer, and user. Each week we will focus on a different AR modality: image, face, body, environment, and object, and consider their real-world applications. Through weekly explorations, we will examine the existing affordances of AR as well as their impact. This course will culminate in a final project, and our tool of choice will be Lens Studio. Course syllabus: https://docs.google.com/document/d/1wMWVnDdzgz2bbsCNp7jRAiCe1job4glq61o67sWAP00/edit?usp=sharing

Computational simulation and videogame engines offer thinkers and makers a new way to reflect on the question “If I can make a world, what would it be like?” In Worldmaking with Unity, students will be exposed to various theories and approaches of worldmaking, and realize their own creative visions by constructing an original, conceptual, and playfully simulated world with indie game development engine Unity. This seven-week seminar/studio course will include a gentle introduction to computer programming, 3D modeling, character and scene design, and rendering techniques with Unity, as well as related production tools such as Blender. Projects may address system, agency, narrative, generative design, critical computing, and more. Student projects created in Unity can be compatible with augmented reality (AR) and virtual reality (VR). This course is designed as an intro-level game design and development course. There is no prerequisite. More advanced production techniques such as shader language and AI might be introduced as optional topics only. More information at: https://docs.google.com/document/d/1UppRjSyFlMpGPNLMowqKmP7wwq29s5Oexnpvrj4Gubs/edit?usp=sharing

You can’t fight for justice if you can’t see the injustice. In today’s digital world, injustice lurks in the invisible choices made by algorithms. In order to understand who these algorithms might discriminate against, as well as who they might favor, it is necessary to precisely measure their impacts in the real world, not just in simulations. In this class, you will learn about some fundamental investigative data journalism techniques, by looking into case studies and doing weekly exercises. You will learn the basics about research and reporting strategies that can help you gather evidence and collect data. You will also learn about how to clean dirty data, visualize it, and use it for analysis. These skills are essential to data journalism. We will discuss a range of stories that have relied on them investigating apps that power the gig economy, data brokers that track our online habits and AI-enabled products such as facial recognition software or tenant screening software; to name just a few subjects. Familiarity with research and reporting techniques, and/or experience with scripting languages such as Python or JavaScript will be useful, but are not a requirement.

One of the most transformative consumer products in history, the iPhone remains the standard bearer for great design and user experience. With the latest versions of iOS and iPhone, Apple puts depth sensing and augmented reality in our pockets. How do we take advantage of this incredible platform to produce our own compelling experiences? This course will be a hands-on workshop where we explore the world beyond generic apps and push the boundaries of what’s possible on iOS hardware. Each week, you’ll be asked to complete a programming exercise meant to foster your understanding of iOS application development. We'll leverage existing open source libraries to quickly build out your app with features such as real time communication and cloud storage. We aim to create distributed instruments for computed expression. Full-time access to an iOS device and a Mac laptop computer running the latest operating system and development tools are required. Prereq: Some programming experience (such as ICM) and willingness to learn Apple's Swift programming language.

Microbial ethics in an anticolonial space faring future is a hybrid theory and hands-on course that examines microbial informed ethics as a guide for our travels beyond earth. This class takes the biopolitical and the geopoetical as our frame of reference: \\\The microbial\\\ We are ancient bacteria, in the event of an injury where large amounts of mitochondrial cells are spilled into the bloodstream, the human body still recognizes it as foreign bacteria and triggers a systemic inflammatory response. We shouldn't forget that symbiosis just means "together" + "living", and there are many modes of living together. \\\The geological\\\ Our bodies are starmass, but our thingness isn't just materially constituted, it is also historically coded. The thingification process of colonization transforms subjectivity into objecthood, ranked through animacy hierarchies to justify various forms of extraction and subjugation. In space, can we rewrite the possibilities of being and non-being?

In this precarious and critical time, it is important to expand and evolve our ecosystem of thought away from the current hierarchies that prioritize the established frameworks of the Anthropocene. There is vast potential in combining emerging technologies and artistic expression to re-envision possible futures and ways of knowing/learning. This 1 point class will introduce concepts of speculative design, design fiction, tactical media, and adversarial design through a post-dualistic non-eurocentric approach to designing futures. Format: This class will take shape as hands-on co-design workshops where students will learn about resilience as futurism and ideate, design, and develop artifacts for/from the future.

Our users have senses that they use to perceive information in different ways. Some perceive best through sight, some through hearing, others through touch. Designers often prioritize visual information, excluding participants who would benefit from auditory and tactile feedback. In this class, we'll take a multi-sensory approach to design that makes interfaces more inclusive and accessible to everyone. Students will learn how to design for the senses (think tactile controls combined with atmospheric sounds and olfactory or taste experiences), while gaining an understanding of the assumptions we make about our users' sensory preferences. Students should come with prior experience with physical computing and fabrication techniques and can expect to learn the processes for the user research, usability testing, and iterative design of physical multi-sensory interfaces. Over the course of 7 weeks, students will design interfaces for the 5 senses (sight, hearing, touch, taste, smell), culminating in one final project that includes at least 3 sensory modalities.

This course will cover a variety of academic topics within the field of Interactive Telecommunications. For further details, please refer to the individual class section notes and topic subtitle.

This course will cover a variety of academic topics within the field of Interactive Telecommunications. For further details, please refer to the individual class section notes and topic subtitle.

This course will cover a variety of academic topics within the field of Interactive Telecommunications. For further details, please refer to the individual class section notes and topic subtitle.

This course teaches the ins and outs of using image processing software with an aim towards some type of real-time use (e.g. a performance or installation). The class looks at ways to manipulate different visual media (time-based, still, vector, and rendered) in real-time to allow students to develop interesting real-time performance systems. While the focus of this class is on using Max for visual work (through a software package called Jitter), it also looks at how to integrate interactive elements (sound, physical interfaces, etc.) into the work. Class time is spent on interface design and software development issues as well. The class explores some interesting capabilities of the software in terms of real-time camera input and tracking, generative graphics systems, and media transcoding. Throughout the class students develop and share ideas on live performance as a medium for visual expression, and learn the software tools necessary to put these ideas into practice in the form of idiosyncratic performance systems. A final presentation in the form of a group performance will be arranged. Students should have some working knowledge of Max/MSP before taking this class.

Assistive or Adaptive Technology commonly refers to "products, devices or equipment, whether acquired commercially, modified or customized, that are used to maintain, increase or improve the functional capabilities of individuals with disabilities." This multi-disciplinary course allows students from a variety of backgrounds to work together to develop assistive technology. Partnering with outside organizations students work in teams to identify a clinical need relevant to a certain clinical site or client population, and learn the process of developing an idea and following that through to the development of a prototype product. Teams are comprised of ITP students as well as graduate rehabilitation, physical and occupational therapy students. Prerequisites (for ITP students): H79.2233 Introduction to Computational Media and H79.2301 Introduction to Physical Computing. This course has a lab fee of $201.

When working with electronics we most often think about sensors as parts or components with clearly defined size, shape, and tactile properties. But what happens when we as designers start to consider sensors as materials rather than objects? How do we design with sensors that can be squishy, sticky, or slippery? And how do our designs change when we can sprinkle a sensor like fairly dust, spread it like paint, or knit it like yarn? This course will explore the softness of sensors both in physical form as well as in concept. Leveraging practices and research from the world of electronic textiles and material science, students in this course will learn to identify and utilize the conductive and resistive properties of materials that can be used for sensing.

From the crass rattle of early pager motors to the sophisticated clicks and purrs of the iPhone Taptic engine, the ability to buzz has increasingly worked its way into our devices. This course focuses on physical prototyping and interaction design for non-visual feedback. Specifically, it will explore how haptic feedback can be utilized and integrated into handhelds, wearables, objects, and environments – anything that we touch or that touches us. Traditional tools such as eccentric rotating mass (ERM) motors, linear resonance actuators (LRAs), and haptic motor drivers will be introduced as well as less conventional methods such as gentle poking, prodding, warming, cooling, squeezing, and tickling. Through hands-on experimentation and a review of research to date, students will emerge from this course well-positioned to incorporate haptic feedback into their future projects. Note: This course is designed for students who have previous experience with physical computing and Arduino.

he virtual expansion of screens began during the 1960’s with the exploration of head-mounted displays. Since the 60’s, virtual reality has been explored in a multi-disciplinary context including philosophy, design, arts, behavioral therapy. Baudrillard, with his publication of Simulacra and Simulation (1981), declared that human experience is being replaced by a simulation of reality (HyperReality). His theories brought the dystopian narrative of the virtual to mainstream pop-culture, as seen in films such as The Lawnmower Man and The Matrix . Contrary to Baudrillard, Canadian VR Pioneer Char Davies brings a more positive perspective to Virtual Reality, “facilitating a temporary release from our haitial perceptions and culturally biased assumptions about being in the world, to enable us, however momentarily, to perceive ourselves and the world us freshly.” Throughout the class, the friction between Baudrillard and Davies will create the foundation of our exploration of Virtual Reality, where we will use room scale headsets and game engines to create meaningful “temporal experiences” exploring themes from behavioral sciences to narrative storytelling. We will be exploring ● existing VR projects, popular culture references and theory. ● concepts such as sense of embodiment (SoE), social VR design, and interactive storytelling techniques. ● methods for designing, modeling and rigging avatars for VR. ● live and pre-recorded animation. ● spatial audio techniques such as ambisonic sounds engines. ● packaging and distributing applications for social VR. This is a production class, along with a theoretical foundation, in which we will prototype projects with networking, inverse kinematics, raycasting and face tracking technologies to explore questions such as “how does the viewer become part of the experience?” and “how does the real space relate to the virtual worlds we design?” In the second half of the class, students will work in groups to build a final social VR project based on their exploration of the above framework.

Libraries like TensorFlow.js and ml5.js unlocked new opportunities for interactive machine learning projects in the browser. The goal of this class is to learn and understand common machine learning techniques and apply them to generate creative outputs in the browser. This class will start with running models in the browser using high-level APIs from ml5.js, as well as explore the Layer APIs from TensorFlow.js to train models using custom data. This class will also cover preparing the dataset for training models. At the completion of this course, students will have a better understanding of a few machine learning models, how do they work, how to train these models, and their use case to creative projects. Students will also be able to create interactive ML web applications with pre-trained models or their own models. Prospective students are expected to have taken an ICM (Introduction to Computational Media) course, or have an equivalent programming experience with JavaScript, HTML, CSS.

From the most ephemeral thought to the rise and fall of civilizations, every aspect of your life, and indeed the universe, involves energy. Energy has been called the "universal currency"by prolific science author Vaclav Smil, but also "a very subtle concept… very, very difficult to get right" by Noble physicist Richard Feynman. It is precisely this combination of importance and subtlety that motivates the Energy class at ITP. Maybe you fear the existential threat of anthropogenic climate change, or maybe you just want your pcomp projects to work better. Either way, the class will help you understand energy quantitatively and intuitively, and incorporate that knowledge in your projects (and perhaps your life). How? Building on skills introduced in Physical Computing, we will generate and measure electricity in order to see and feel energy in its various forms. We will turn kinetic and solar energy into electrical energy, store that in batteries and capacitors, and use it to power projects. We will develop knowledge useful in a variety of areas, from citizen-science to art installations, and address topics such as climate change and infrastructure access through the lens of energy. Students will build a final project using skills learned in the class. To accommodate pandemic restrictions, students will have the option to do more or less hands-on work or research as their situation allows, and support material such as an annotated supply list will help students tailor material expenses to their interests. Prior Physical Computing or equivalent experience required.

Forget the screen. People want to be part of the action. They don’t want to watch detectives and control superhero avatars. They want to solve the mystery and be the hero. They want to experience it. We see this craving for playful experience in everything from immersive theater to escape rooms to the Tough Mudder to gamified vacation packages. Designing live experiences for large audiences that demand agency offers a distinct set of challenges, from how much choice you give each participant to how many people you can through the experience. We’ll look at examples from pervasive games to amusement parks to immersive theater, examining both the design choices and technology that make the experiences possible. Along the way we’ll create large, playful experiences that put the participant at the center of the action.

he course title, "Veillance" is a reference to the root of "surveillance" and "sousveillance", watching from above and watching from below respectively. As digital media becomes a greater part of our everyday lives, it is important to understand the new forms of surveillance that it enables as well as to harness these capabilities and perhaps to create systems of sousveillance. Through the course, we'll critically examine technologies that have become integral part of our lives; the technologies that drive the internet, the capabilities of web browsers, mobile phones, and the emerging class of networked devices such as IP cameras and Amazon's Alexa and Google Home. Through the course of this examination, we'll look at how this technology works, the policies that govern their use, what their benefits are, and finally look at what we are giving up by embracing them. This will be a hands-on endeavor; we'll develop software to illustrate their use and misuse as well as looking for ways that we can harness their "veillance" capabilities for creative, educational, and possibly subversive purposes. Of course, in this process, we'll pay particular attention to ethical and moral concerns. The course will have weekly reading and technical assignments, encompasses group work, and will culminate final projects.

If you were inventing a museum today, what would it look like? Who would be there? What would its main purpose be? What would the audience experience? The first museums were called Cabinets of Wonder. Usually, a viewer with a guide, often the collector, would open doors and drawers to see what was inside--amazing things from different parts of the world, different times. They were windows on the world to places the visitors would probably never be able to go; to see things they would never otherwise be able to see. And now there’s television, movies, the internet, and travel. Why do people go to museums now? Will they in the future? Today, most museums seek to educate and to include more and more diverse visitors than they used to. How do people learn in public spaces? How do we know that they do? How can they make use of the new interactive technologies to support the experience and not lose what’s special about them? The class is an exploration, observation, theory, and design class for you to imagine the future of museums and exhibits. Museum and exhibit visits are your primary assignments for the first half of the course—usually accompanied by a reading. You will also make some record of your visit (including a sketchbook, a diorama, reviews). There will be guest speakers from Museums and exhibit design firms, and several field trips. In the second half of the course, you begin to imagine how you might reinvent a museum and develop a full-scale presentation of your own Cabinet of Wonder. About Emily Conrad: https://www.linkedin.com/in/emily/

How do you process data from connected devices? This class examines how to build systems to collect, process, store, and visualize data from connected devices. The class will review and discuss real world Internet of Things (IoT) systems using case studies and actual projects. We will build systems using Arduino hardware and open source software. We will discuss how IoT systems are built on commercial cloud infrastructure. Students will learn about IoT devices and the data pipelines for processing data. They will build an Arduino based device to send and receive data over WiFi using standard protocols like MQTT. Students will write code to move data from MQTT into a database. Students will learn how to query the database and present data as tabular data and graphs. To gain an understanding of an entire IoT system from device to application, we will start at a high level and then drill into each of the pieces -- we will: - Discuss sensor hardware and wireless options (WiFi, Cellular, LoRaWAN, LTE-M, etc) for moving data to the server - Discuss transport options MQTT, CoAP, AMQP, HTTPS, etc. - Examine SQL, NoSQL, and Time Series Databases - Look at tools and techniques for querying and visualizing data. - Collect and store sensor data using Amazon Web Services (AWS) Prerequisite: - Introduction to Physical Computing - Introduction to Computational Media (suggested) The class will be a mix of lecture, discussion, and building IoT systems. Real world examples and case studies will be used to demonstrate how IoT can be built.

This class is a project development studio for interactive music projects —that is, pieces of music that are not linear, but rather offer multiple dimensions for listeners to explore. Applications include generative music installations, novel instruments, participative performances, museum exhibitions, games, and tools for producing and teaching music. Students will take a project from concept to execution over several iterations, applying interaction design, creative coding, and music production tools and techniques. The project development process will include gathering aural and visual references, composing graphic/tactile notations, and creating focused studies to explore interaction with specific musical elements. This work will inform the design and implementation of a functional prototype which students will test, evaluate and refine to produce their final project. The first half of the semester, the class format will be a combination of lectures, design and coding exercises, and reading discussions. To support different kinds of projects, in-class examples will include desktop (Max/MSP, Ableton Live, Max for Live), web (Tone.js + p5.js), and physical computing frameworks (Arduino compatible micro-controllers). During the second half of the semester we will shift to a more self-directed approach, as students work on their final projects. Professional practitioners will come in to share their work in the field and give students feedback on their projects. Coming in with a specific project to develop is welcome; conceiving a project during the class is encouraged, too. Some experience in making or producing music will be useful, but is not required. ICM and Physical Computing or equivalent experience are required.

Our perceptions of the future have been conjured via media, cultural references, and select readings as well as fact. In this we receive particular cultural and gendered perspectives, even unconsciously. Prototyping the Margins asks us to re-envision more diverse and inclusive futures, to ensure that they will exist. Through the reading and reference of alternate texts, using nontraditional perspective and high-lighting bias, this class will deconstruct previously held notions of the future through reexamining the past and rethinking the present via storytelling and prototyping. In this course, students would engage in literary exploration, design thinking, creative research, storytelling, and rapid prototyping. They will read and investigate “alternative culture" resources, remix their own narratives based on their findings, then prototype speculative expressive artifacts or environments that speak to the diversity of futures found. Throughout the semester, students will create 3 projects in the categories of Environment, Body, and Object that exist in their narrative world. Their final presentation would be in the form of an exhibition, in which students will have an opportunity to discuss their narrative, methodology, and process with external expert critics. The course functions as a space for the discussion and recontextualization of notions of futurity and history. Additionally, the course is a workshop for designing and prototyping speculative artifacts and environments of revised histories and alternate futures. Interested students should have some experience with programing or new media related production. Selected course material includes, but is not limited to, works by Drexciya, Louise Erdrich, Octavia Butler, and W. E. B. Du Bois amongst others.

In this course, students will learn the process of developing products that address user needs. Students will go through the process of identifying a user need, developing a product prototype, evaluating the product with the target user, and outlining the next development steps. Topics will include such elements as need finding, archetype development, user journey maps, ideation, prototyping, user evaluation and validation.

Can we capture the unpredictable evolutionary and emergent properties of nature in software? Can understanding the mathematical principles behind our physical world world help us to create digital worlds? This class focuses on the programming strategies and techniques behind computer simulations of natural systems. We explore topics ranging from basic mathematics and physics concepts to more advanced simulations of complex systems. Subjects covered include physics simulation, trigonometry, self-organization, genetic algorithms, and neural networks. Examples are demonstrated in JavaScript using p5.js (https://p5js.org). Much of the class time will be dedicated to in-class exercises and self-study as the course is available online through a video series (https://www.youtube.com/user/shiffman/playlists) and textbook (http://natureofcode.com/book).

Light Emitting Diodes or LEDs are used creatively all around us. They have the ability to emit light at different colors and intensities instantly and from very tiny points. How can we make creative visual works out of these amazing devices? What construction methods can we use to make those works reliable? Big LEDs will cover the process of designing large LED systems. We will cover LED array hardware and how to map pixels from computer generated media onto them. We will go through every major part of the hardware - different styles of LED arrays, drivers and gateways, cables, data protocols, and how to safely power all of them. We will learn to use the pixel mapping softwares Enttec ELM and Madmapper. We will also cover the paperwork needed to furnish a professional LED installation, including drafting riser diagrams, plan, section, and elevation views, creating a bill of materials, and writing instructions for users and installers. This year's final project will be a site-specific LED sculpture installed in a public space at 375 Jay St. The installations will be able to display student chosen media that can be viewed for one minute. Students will work either in groups or alone and can choose from one of four installation options to present on: - A prepared square section of 2.0mm pitch LED video tiles (approx 256px x 256px, 2’-6” x 2’-6”) - A prepared low-resolution sculpture with diffused linear elements (approx 500px, 2’-6” x 5’-0” overall) - A student conceptualized LED video tile project - A student conceptualized low-resolution project Because of this year's pandemic, unprecedented changes have come to the professional world of LED installations. As a result, we will be using remote tools such as networked-based cameras, remote desktop applications, and virtual private network connections to watch and operate the final projects. We will spend class time setting these tools up together. The two prepared options for the final project will be installed and maintained by the instructor.

This class will apply a product and service design lens to the rapidly evolving role of technology in politics, government services, and human rights, with a dual focus on the use of technology to advance the public good, and the threats that technology can pose to various aspects of civil society. On the one hand, there has been an explosion of technology tools that aim to promote healthy and peaceful democracies, provide more efficient government services, and promote human rights. This course will survey those efforts and the product design methodologies that guide them. On the other hand, 2018 has been a reckoning year for technology all over the world. The post-mortem of the 2016 US Presidential election shined a light on technology’s role in a promulgating a deeply polarized electorate, enabling election-meddling from foreign actors, and profiligating fake news, while also raising deep concerns about data privacy and security for everyday citizens. Meanwhile, new technologies like AI and Blockchain are poised to have deeply transformative effects on multiple aspects of society - from policing to education to financial services. This course will also explore the risks various technologies can pose to privacy, safety, liberty, and well-being. The course will feature guest speakers from leading practitioners in the space. Potentially including the ACLU, Higher Ground Labs, Elucd (YC17), NYU Better Policing Initiative, Microsoft, and more.

Stereo (2-speaker) sound is the default way we produce and distribute most audio. This class challenges the stereophonic-centricity of digital sound and instead focuses on the context of listening, interfacing and interacting with audio beyond 2 speakers. We will take a novel approach to spatialization by interfacing web technologies (Javascript, Web Audio, WebRTC) with multichannel audio to create room-scale interactive music and sonic spaces, and then make our findings publicly available through musical artifacts, open source tools, and documentation. We will explore conventional and unconventional loudspeaker arrangements ranging from 3 to 40 channels and how distribution and context affects music. Together, we will participate in a semester-long project examining the affordances of spatial and multichannel sound, designing and build open source tools to interface Tone.js, Higher-Order Ambisonics (HOA) and other multichannel techniques. Students will perform/install their final on the 40-channel speaker array at Dave&Gabe’s studio in Bushwick. Topics include 3D sound, open source, space and listening. Prerequisites: willingness to listen critically, some programming experience. No formal music training required.

The Library of Congress holds more than 160 million physical items, alongside countless more digital resources. The collection spans vast swaths of subject areas, geographical places, historical periods, and political eras. In this course we’ll learn about the unique properties of these holdings, about the ways that these objects are encoded in data, and how we can access the archive both remotely and in person. Most importantly, we’ll dream up ways that artists might interact with and interrogate the collections, to produce work in a variety of media from software to sculpture to performance.

The purpose of this course is to push beyond quotidian “problem solving” methods to equip students with the analytical capacity to tackle insoluble (“wicked”) problems and strategic uncertainty, with a particular focus on forecasting methodologies and “long-term thinking.” Taking a critical lens on analytical capacity building, the course will quickly progress from classical methods (i.e., critical thinking / design thinking / systems thinking) to more specific practices (i.e., forecasting, scenario planning, prediction markets). Readings from diverse disciplines, and exploration of timely and important issues such as information disorder, Surveillance Capitalism, biodiversity loss, persistent colonialism, climate change, algorithmic intermediation, and more will form the setting for collective inquiry. Specific design methodologies including human-centered design, design fiction theory, biomimicry, speculative design, agile, etc. will be both problematized and used to locate and expand on the “designer’s role” within critical forecasting projects. Throughout, participants will interrogate the cultural narratives, power dynamics, epistemological dichotomies, myths, and assumptions that uphold dominant paradigms of strategy, design, and planning.

How do we design for immersive, cooperative, and playful experiences? Students in this course will look at immersive and experiential design through the lens of one type of experience: the Escape Room. We will explore different experiences, narrative structures, group dynamics, and game mechanics. Over 14 weeks students will explore how to design immersive and participatory experiences through cooperation, play, and problem solving. Though weekly assignments and exercises students will use a variety of methods to design cooperative narratives and experiences. We will explore Zoom as a tool for collaboration and performance. We will also explore other web based protocols to encourage other forms of digital and physical interaction. During the last few weeks of the semester students will collaborate to design, build, and operate an escape room experience. This class will run remotely with the expectation of synchronous meetings over Zoom. If possible, there may be some potential offerings for optional in person activities. Prerequisites: Physical Computing and Introduction to Computational Media.

Artists working with digital audiovisual mediums deal on a daily basis with compression, sampling, digitalization, scaling, modeling, and quantizing. They also tackle challenges such as modeling natural events, noise filtering, time stretching, and parameterizing. In this class students will learn math tools for boosting their digital practice and fixing common problems, and also understand the math behind our human perception of the physical world. Each class we will learn different concepts and techniques, including probabilities, Fourier transform, and quantizing, and then will explore their influence in our human perception and audiovisual applications, including synthesis, manipulation, conversion, and rendering. This class does not need any programming or mathematical prerequisites, besides arithmetics. We will program audiovisual applications using free libre open-source software, including Python, p5.js, Pure Data, and Audacity.

This 2 credit class will focus on producing precision 2.5D parts, perfect for mechanisms and assemblies. We will cover 3 axis CNCs, 2D CAD, CAM, and machine setups. The class will be hands on and fabrication heavy, paying close attention to accuracy and craftsmanship. There will be weekly fabrication exercises, assignments, and a final project. This class will not cover the lathes, the 4 axis CNC, or 3D CAD, that is only covered in Subtraction: Turning.

This 2 credit class will focus on creating 3 dimensional sculptural parts from raw materials. We will cover the wood lathe, the metal lathe, 3D CAD, and the 4 axis CNC. The class will be hands on and fabrication heavy, paying close attention craftsmanship. There will be weekly fabrication exercises, assignments, and a final project. This class will not cover 3 axis CNCs, 2D CAD, or CAM, that is only covered in Subtraction: Cutting.

Much of our daily life is quietly being reshaped by AI while an entire generation of children is growing up with this technology in their homes. The best way to understand the algorithms that drive AI applications is to make your own -- to write and train them through playful and interactive activities. The Hacking Smart Toys for AI Learning course consists of a series of hands-on activities focused on designing and testing several smart toys, construction kits and play experiences that can support youth to better learn and play with AI. Both beginners and more advanced students are welcome. In this track participants will design with and hack existing smart toys and AI devices to support youth, families and educators to customize and appropriate these technologies in playful ways. The goal of the course is to imagine the future of smart toys and AI devices for youth and to explore the social and ethical conditions of children growing up with AI. The course will introduce students to different aspects of machine learning through play while engaging the local community of toy designers, artists, AI pioneers. The things we will do include writing applications for current smart toys and AI devices like Cozmo and Vector robots, Amazon's Alexa, Lego Wedo bricks. Students will also design their own AI toys or construction kits which they will test in local communities (schools, libraries, museum, community centers). Final projects will be displayed in a play exhibition where all communities involved throughout the course will be invited to participate.

Art, design and experimental electronics can be great tools for inciting discussions of complex issues such as privacy, sexism, racism, economic inequality and climate change. This course aims to provoke thoughtful discussions of pressing issues through the combination of Art, Industrial Design and Embedded Electronics (sensors, actuators, wifi enabled microcontrollers - ESP32, raspberry pis). Topics will include technological disobedience, adversarial design and critical engineering. In this 14 week class, students will combine technology, design, and critical theory to build Art Objects / Interactive Sculptures that are aesthetically intriguing while socially relevant. This is a production heavy four-credit course, where students will learn about new-media critical theory, design and electronics. Prerequisites include an open mind, the drive to make, and physical computing.

This course is a survey of some of the more exploratory new directions AI is heading towards in the coming years and the creative applications these new developments may enable. More specifically, the class will explore the following three subtopics: 1) Realistic language models: Turing-test passing text generators like GPT-3 are writing whole paragraphs with human-level coherence. We will explore techniques for generating creative fiction and chatbots using both APIs and our own home-brewed NLP models. 2) Generative music: Since WaveNet in 2016, generative models of audio have gradually evolved to capture music and natural sound. How and for whom will top-10 hits in the future be made, and how will they be listened to? We will speculate about such questions and experiment with speech and audio synthesizers and new kinds of digital musical instruments. 3) Decentralized AI: What happens when we can do machine learning at scale without putting all the data in one computer? This seemingly trivial difference opens up all kinds of new possibilities, including AI co-ops and autonomous artificial artists. Each subtopic is accompanied by practical resources to use or implement these techniques for yourself, as well as an overview of the available and relevant tools associated with it, and will finish with a short project to apply them to a topic you're interested in.

Exhibitions speak to all our senses. Most creative professionals specialize in one or another particular field, be it space design, or media production, interactive ideation or storytelling. But when we merge all these components into one congruent whole, magic happens – they mutually amplify each other and create a powerful experience that speaks to all our senses synchronously. During a two afternoon course, students will systematically explore the opportunities of "New-Media-Space" and team up to combine a broad variety of media to design conceptual experiences that involve all our senses.

As students look towards entering the working world as emerging directors, writers, and producers, they will benefit greatly by gaining a deeper understanding of the storytelling and creative opportunities that exist within the area of emerging media. The class will focus on the artistic possibilities of cinematic VR and other new media. Students will be asked stretch their imaginations and embrace the distinct affordances of each offering. We will focus on how to create impactful experiences through learning about new modalities, exposure to the latest works, and experimenting to understand the viewer’s experience. Several exercises will be assigned to familiarize students with 360 storytelling. Students will brainstorm ideas for various modalities, from spherical storytelling to room-scale VR and augmented reality, designing story experiences as narrative writers and visual directors. They will shoot short 360 films using Ricoh Theta S cameras and edit on Adobe Premiere. A final, well-developed plan for a narrative short VR film, a potential Thesis Project, will be pitched during the last class. Along with the exercises, a strong conceptual focus will challenge the students to think outside of current technological limitations, and enable them to hit this ever-changing ground running. The aim is for the class to create entryways for students to engage with emerging media, with an expanded sense of possibility and inspiration.

For centuries, great works of music, theater, and dance, have combined art and science to make integrated performances that move audiences. Today, we are seeing exciting changes as artists experiment with video and real-time interactivity to draw audiences even deeper into the performance, and enhance the shared experience of the moment. This class explores conceptual approaches to design, industry-standard software, prototyping frameworks, and data flow programming to provide student designers with the cutting-edge tools necessary to confidently collaborate with writers, directors, and performers. Structured as a studio course, students will make designs for contemporary performance pieces, and collaborate with working artists to design original projects.

This course focuses on programming strategies and techniques behind procedural analysis and generation of text-based data. We'll explore topics ranging from evaluating text according to its statistical properties to the automated production of text with probabilistic methods to text visualization. Students will learn server-side and client-side JavaScript programming and develop projects that can be shared and interacted with online. This fall the course will also explore topics in machine learning as related to text. There will be weekly homework assignments as well as a final project.

If we consider drawing as one of the most ancient forms of interaction, it has the power to engage users like no other technology. With the right setup and call for action – I’m always taken aback by how a cup full of crayons and a pile of paper can bring the child out of a serious businessman. But how do you pass the strong barriers of users who are afraid to draw? In this weekend workshop we’ll examine various techniques that can spark an interaction which is all focused on the action of drawing. Workshop topics include an exploring into collaborative drawing platforms, interactive drawing installations, drawing machines and drawing as a form of interactive storytelling. We’ll discuss the differences between digital and analog drawing and how to marry the benefits of both mediums. We’ll cover the work of important artist and researchers who are creating inspiring work in the field of drawing and interaction such as Shantell Martin, Zach Lieberman, Tobias Gutmann, David Ha and more There will be various collaborative drawing exercises in class. This workshop is meant for students who wish to focus on drawing as the driving force of their interactions and possibly expand it into thesis. Each student will design and prototype an interaction which is based on the subject of drawing. Alon Chitayat graduated from ITP in 2015, He works at Google as a Sr. UX and motion designer and on a collaborative digital whiteboard called Jamboard.

Digital cameras and printers are making photography more ubiquitous and more useful than ever. This course is a workshop that looks at changing the rules for capturing and printing digital imagery. By gaining a better understanding of the engineering fundamentals and limitations of digital photography, students can produce breathtaking images with all the benefits of digital media but with an image quality that rivals film. Students experiment using low cost, hands-on tips and tricks in software and hardware to capture high dynamic range, expanded color, night color, 3D, time lapse, and stop motion images using a digital camera and printer. While using mostly off-the-shelf tools, these experiments require students to dig down to see the nitty-gritty of today's and tomorrow's technologies for digitally sensing, encoding, compressing, transmitting and displaying images. This course has a lab fee of $201.

This is an environment for students to work on their existing project ideas that may fall outside the topic areas of existing classes. It is basically like an independent study with more structure and the opportunity for peer learning. This particular studio is appropriate for projects in the area of interactive art, programing, physical computing and digital fabrication. There are required weekly meetings to share project development and exchange critique. Students must devise and then complete their own weekly assignments updating the class wiki regularly. They also must present to the class every few weeks. When topics of general interest emerge, a member of the class or the instructor takes class time to cover them in depth. The rest of the meeting time is spent in breakout sessions with students working individually or in groups of students working on related projects.

The World Wide Web no longer stops at the edge of your screen. When it comes to products, if it powers up, it talks to another device. This class provides an overview of methods for connecting the physical world to web-based applications. We’ll consider what the emerging interaction patterns are, if any, and we’ll develop some of our own as needed. This class can be seen as a narrower and more interaction design-based complement to Understanding Networks. The latter class provides a broader overview of the dynamics of communications networks, while this class focuses specifically on the challenges of connecting embedded devices to web-based services. Neither class is a prerequisite for the other, however. This class will introduce network connection techniques for devices using microcontrollers like the Nano 33 IoT and MKR series or ESP8266 and processors running an embedded operating system like the Raspberry Pi. Prerequisites: Intro to Physical Computing and Intro to Computational Media, or equivalent experience with the topics covered in those classes. Learning Objectives: Students will gain an understanding of the basics of network programming for devices with limited computing power. They will learn about current protocols for communication between devices and networked servers, and about the rudiments of security for that communication. Reading: There will be an article or two to read each week, to foster discussion about the design of connected things. Assignments: There will be several one-week software and hardware assignments to get familiar with different technologies and communications protocols, and one hardware and software final application project.

This class explores the possibilities of subtle interaction with computers. Conventional computer interface tends to accommodate conscious, explicit, intentional communication. Many unconscious cues and actions that are valued in ordinary human expression are ignored or filtered by computer-mediated interactions. Relinquishing a conscious gatekeeper can be associated with such uncomfortable subjects as subliminal manipulation, subconscious repression, even a loss of free will and the insanity defense! On the other hand going past conscious control can be associated with achieving virtuosity in the arts and athletics, acquiring insight into your personality, and engendering trust in conversation. In this course students build on software and hardware tool kits to create hands-on experiments tapping less conscious parts of your experience. The prototyping exercises include using cell phone as personal sensor logger and then visualizing the results; sensing autonomic nervous responses such as heart rate; and trapping and analyzing language use on your computer. Group work is encouraged. The last part of the semester we concentrate on final projects. ICM and Physical Computing are prerequisites to this course.

This studio and seminar course explores the basic principles of food biochemistry, enzymology and food processing and how they relate to memory, the senses and the processing of information. Students will also learn basic principles of molecular gastronomy and modernist cuisine as framing devices for understanding how food also functions in the context of bodily health, environmental health as well as cultural and political narratives. Our food system consists of more than food production and consumption and this class will address how science and food science plays a more integral role in this system and how this knowledge can be mined for work that creatively and functionally contributes to this emerging field. Assignments for the class will be based on the incorporation of food science into design and technology projects that uses food as a substrate to explore and illuminate information within the food system. Workshops involve using liquid nitrogen + hydrocolloids as well as creating performative food objects and a Futurist meal.

Fascinating and terrifying things are happening at the intersection of data and culture. Our lives are being constantly measured, and information about us is being surveilled, stolen, and commodified. Dialogue around this data revolution has been dominated by corporations, governments, and industry – but what about the arts? In this class, we’ll investigate the means by which artists can engage (and are engaging) in the collection, processing, and representation of data. Using a research-focused, prototype-based approach, we’ll build a series of collective and individual projects to interrogate the ‘new data reality’. Students will use p5.js, along with a variety of analog media or open-source data tools (such as D3.js, Miso, OpenRefine, MapBox & Leaflet).

This 7-week, 2-point course will provide a framework for learning how to develop and program web applications. It will focus on server side development using JavaScript, Node.js with the Express framework, and persistent databases on cloud based infrastructure. Additional topics will include login and session management, web services and APIs, and will lightly touch on front-end web development. The course will be a mixture of lecture and in-class collaborative coding, with weekly programming and reading homework.

As a response to developing technologies, artists working in areas of new/digital media are continually inventing new concepts for self-expression - interactivity, the passage of time and resolution, just to name a few. Yet these concepts are new only in the sense that they are being adapted to new media. For example, the notion of interactivity, frequently observed as original and specific to the user-interaction component of computer-mediated works, was equally, if differently, specific to Gianlorenzo Bernini's 17th-century Baroque sculpture and architecture. Indeed the very concept of new media, and the concomitant implication of critically significant artistic development, applies throughout history. Oil revolutionized painting in the Renaissance, as did house-paint (on canvas) in the 1950s; in the 1910s, the found object indelibly altered definitions of art, the importance of the object being subsumed by that of the concept in the 1960s. This course examines how artists working before the boom of digital technology utilized other media, techniques and approaches to effect formal, conceptual and experiential dynamics comparable to those being investigated by new media artists today. The objective of the course is to provide students with not only knowledge of the immensely rich history of artistic creativity, but also a platform through which that knowledge might be utilized to reconsider new media strategies of artistic expression. It is the goal that through observation, discussion, reading and projects (both written and hands-on), students acquire mental tools to approach their own work with an expanded understanding of artistic possibility. Organized thematically, each class focuses on a different concept derived from the field of new media production and examined with regard to artistic precedents. The course focus primarily, though not exclusively, is on 20th/21st-century art. It is conducted as a combination lecture/discussion class. Critical theory is incorporated into the readings and discussions, but this is not strictly a theory course. The course has been conceptualized and designed to enhance understanding through a variety of means, from basic observation, to exploratory conversations, to more rigorous thinking informed by lectures, readings and focused discussions.

We live among the vast and relatively unknown Kingdom of Fungi. Mycelial networks have been likened to social and communications networks. What do we have in common with mushrooms? What can we learn from them? Fungi communicate, remediate, and decompose. They are used as food, medicine, spiritual guides, and material building blocks. Some are crucial to the soil food web; others will kill you. Fungi are closer to Animalia than to Plantae, and only 5% of the Fungi have been classified. Students will explore fungi through reading, research, writing and interacting with fungi, and making one case study and one project that explores the physical and conceptual material covered in class. Readings that span the biological, theoretical, social and creative will include Paul Stamets, Dale Pendell, Anna Tsing, and Gilles Deleuze. Students are asked to utilize "technologies" in the creation of projects, which might include digital media, cooking, mycotecture, or working with existent fungal communities.

In NYC we take our cellphone and Wi-Fi signals for granted. It’s always on and we are always connected. This not the case for 2 billion people who lack affordable communication and 700 Million people who have no coverage at all. New technologies are democratizing communications infrastructure; Software defined radios, lowered hardware pricing and open source solutions have made it possible to install low cost infrastructure that can be controlled by communities instead of multinational corporations. In this class you learn how to create your own communications networks. We will cover the software, and hardware used today in community based cellular networks in Mexico and Nicaragua. We will also delve into the business of international development through guest speakers working in the development or telecommunications fields.

This course explores music through the lenses of computation and interactivity. The first part of the semester consists of a structured exploration of rhythm, melody, timbre, and harmony, from the perspectives of code, design, and music theory. For each musical element, we will hold listening sessions, represent and manipulate the element in code, and create an interactive study around it. During the second half of the semester we will cover algorithmic composition techniques such as Markov Chains, Neural Networks and L-systems. As students work toward their final projects, assignments will take a more self-directed approach. Professional practitioners will come in to share their work in the field and give students feedback on their projects. In-class coding and assignments will be done in P5.js + Tone.js, but students will be free to use other languages and frameworks for their final projects. ICM or equivalent programming experience is required. This class is a good fit for students who are interested in: - Creating interactive music pieces and digital instruments - Deepening their understanding of how music works - Continuing to develop coding skills acquired in ICM Prerequisites: Introduction to Computational Media (ICM) or equivalent programming experience is required. About Luisa Hors: https://www.luisapereira.net/

This class is dedicated to experimenting with interactivity on large-scale screens. Students develop one project over the course of the semester, culminating with a showing at InterActive Corps' 120 X 12-foot video wall at their corporate headquarters on 18th St. and the West Side Highway. A mock-up of the system is available at ITP for testing. Class time is divided between independent project development, critique, technical demonstrations, and field trips to IAC. Students should be comfortable programming in Java and Processing.

This two-point workshop is centered on the examination and creation of collective storytelling environments. We will examine a wide-range of storytelling spaces including participatory and user-generated environments, site-specific works, community based arts practices, and transmedia storytelling. Weekly assignments, field trips, and student presentations.

Subtractive fabrication is a common manufacturing process that produces durable and functional objects. This class will cover multiple techniques on machining and milling raw material into custom parts. We will focus on both traditional and digital fabrication tools: lathe, CNC router, 4 axis mill, etc. We will cover CAD, CAM, and machine setups as well as research affordable desktop milling solutions for personal shops. The class will be hands on and fabrication heavy, paying close attention to precision, accuracy, and craftsmanship. There will be weekly fabrication exercises, a midterm, and a final project. It’s mill-er time.

Today's mostly digital world also requires a basic knowledge of analog circuits. In this course students learn about the basic principles of analog circuits design and operation. Students learn about discrete components such as resistors, capacitors, diodes and transistors as well as integrated components such as operational amplifiers. In addition, students become familiar with the operation of basic electronic test equipment such as digital multimeters, oscilloscopes, function generators. The instructor lectures on, and demonstrates, basic analog concepts so that students can form a basic rule of thumb understanding of analog circuits, concepts and components. In the lab, students can integrate analog solutions into their project work.

The World Wide Web has grown up to be a great platform for asynchronous communication such as email and message boards. More recently this has extended into media posting and sharing. With the rise of broadband, more powerful computers and the prevalence networked media devices, synchronous communications have become more viable. Streaming media, audio and video conference rooms and text based chat give us the ability to create content and services tailored to a live audience. During this course, we focus on the types of content and interaction that can be supported through these technologies as well as explore new concepts around participation with a live distributed audience. In this course, we look at new and existing platforms for live communication on the web. We leverage existing services and use Flash, PHP, AJAX and possibly Processing/Java to develop our own solutions. Experience with ActionScript/Flash, PHP/MySQL and HTML/ JavaScript are helpful but not required.

Hardware is not hard, and rapidly prototyping circuit boards is easier than ever with new tools available at ITP. Students will learn how to grow from a breadboard to a custom surface mount board, all without leaving the floor. This class is about artists and designers taking control of their hardware, and exploring the potential of embedding their projects into the world around them. Students will learn the multitude of tools and processes required to make a DIY circuit board. These include Eagle CAD, micro-milling machines, drawing schematics, ordering parts, surface-mount components, acid etching, solder paste and stenciling, reflow, pick-and-place, and others. In-class demonstrations will be done for each of the above, and students will complete assignments using online reference notes and videos. Two smaller projects and one final project will be assigned (3 total), each a circuit of the student's’ choosing. These three assignments will be designed to work off the most recently taught subjects, and to get the students to fail early. It’s a lot of new material to be learning across multiple domains, so the students will need the entire semester to iterate.

Course Description: This course provides an introduction to the concepts of motion capture and the motion capture production pipeline to perform and record 3D animations for film and video games as well as stream for live performances. Students will learn all of the tools for tracking props and performers using MAGNET’s cutting edge motion capture studio. Students will also develop concepts around the technology and integrate their data into 3D computer graphics along with keyframe and procedural animation and custom 3D assets to build final projects using the Unreal game engine. Program Learning Objectives: To design a workflow of MoCap and ‘how to’ guides for diverse type of projects using motion capture and the basics of world and character building in the game engine Unreal. Reading assignments introduce students to constructed historical perspective of MoCap and investigates principles particular to animation and performance. The objective of the course is to allow students to understand the importance of pre-production and planning which includes notions of storytelling, 3D modelling and texturing, best practices to capture the sessions, world building methods in video art and game development, and live show running. Course Objectives ● To create storyboards and define recording shots before capturing data. ● To understand and use the correct motion capture pipeline ● To learn the skills to direct an effective motion capture session (best calibration) ● To learn the best practices for capturing and cleaning data ● To be able to apply the data to models in Maya, Unreal Engine, and Max/Jitter ● To be able to perform real-time retargeting for Live Performances Course Structure The class consists of a weekly lecture and a weekly 2-hour lab session. Students will work in groups and will be required to attend a self-selected lab session on weekends. Examples of motion capture projects will be presented in lectures and discussed in the classroom. Students must demonstrate satisfactory achievement of course objectives through fulfillment of one final project at the end of the semester and a mandatory submission for the programs showcase. Final pr ation or live performance. Collaboration with students and faculty from other programs is encouraged.ojects will require students to use a wide variety of software and studio time at MAGNET’s black box mocap studio. Students will be able to process their motion capture data for use in an anim

This course introduces the Python programming language as a tool for reading and writing digital text. This course is specifically geared to serve as a general-purpose introduction to programming in Python, but will be of special interest to students interested in language and computer-generated text. Among the topics we'll discuss are: the history and aesthetics of computer-generated writing in literature and the arts; computational linguistics; ethics and authorship in the context of computer-mediated language; poetic structure and sound symbolism; performance and publishing. Programming topics covered include: data structures (lists, sets, dictionaries); strategies for making code reusable (functions and modules); natural language processing; grammar-based text generation; predictive models of text (Markov chains and neural networks); and working with structured data and text corpora. Weekly programming exercises and readings culminate in a final project. Prerequisites: Introduction to Computational Media or equivalent programming experience. Prerequisite: ICM

The television watching experience has radically transformed in the last several years. First technologies like TiVO and DVR allowed consumers to timeshift their viewing behavior. Then the Slingbox and services like BitTorrent allowed consumers to easily placeshift their viewing environments. Next, a wave of on-line services like YouTube, Joost, Hulu, Boxee, and Netflix introduced an unlimited supply of on-demand content ranging from short-form user generated content to weekly TV shows to long-form movies. As these technologies proliferated, the epicenter of the TV watching experience quickly shifted from in front of your HDTV in your living room to anywhere, everywhere, and anytime ? on your TV, PC, or mobile device. However, the continued disruption of television has only just begun as we now move into the era of Social TV. What had started as a two-screen experience (Facebooking on laptop waching TV) has now become a one-screen experience. (see Barack Obama?s inauguration viewing on CNN.com). Additionally, services like FiOS TV and Yahoo! are introducing widget platforms that allow designers and developers to integrate Twitter and Facebook into the TV watching experience. As a result, designers and developers are able to explore a myriad of social TV possibilities. In this class, students are challenged to design the future of television as they follow a rigorous design methodology that teaches them how to go from idea to functional prototype. Students conduct user research, create user personas, create conceptual user journeys, and wireframe their concepts before beginning prototype work. Students work in small project teams of 2-3 and weekly classes cover design methodology and various topics relevant to the future of television such as next generation user interface design, social TV, and TV on mobile devices. Each week, students are expected to present their project updates with open class discussion in the form of critique sessions. The students are expected to prototype a final project so prior experience with web programming, and prototyping software (Adobe Flash) is helpful, but not required. The final project requires a functional prototype with supporting design documentation. Executives from the advertising, media, and consumer electronics industries are invited to class to provide guest critiques and to speak about future trends within television.

Iteration and its impact on your creative process is the theme of this class. The format of the course turns its head on the traditional class structure and instead of focusing on syllabus that builds to a final project, the course is focused on a daily, iterative practice. Students will identify a theme, idea or topic they would like to explore over the course of 100 days and must commit to making or producing a variation on that idea and posting social evidence of their work every day for 100 days. Projects can focus on building, writing, drawing, programming, photographing, designing, composing or any creative expression. In parallel to the making, in-class lectures will examine the work of artists whose work has been defined by iteration and discuss the role of discipline and routine in the creative process.

This class will focus on the skills and frameworks for putting the user at the center of the design process and ensuring the products and experiences we create meet user needs and expectations. The course will cover the full design process including strategies for conducting design research, methods for creating journey and experience maps, wire-framing, ideating, prototyping and user testing. Students will be active participants in the class and should come to every class with a computer and sketchbook. The class format will include lecture, in-class design exercises and a final design project.

Interactive technologies seldom stand alone. They exist in networks, and they facilitate networked connections between people. Designing technologies for communications requires an understanding of networks. This course is a foundation in how networks work. Through weekly readings and class discussions and a series of short hands-on projects, students gain an understanding of network topologies, how the elements of a network are connected and addressed, what protocols hold them together, and what dynamics arise in networked environments. This class is intended to supplement the many network-centric classes at ITP. It is broad survey, both of contemporary thinking about networks, and of current technologies and methods used in creating them. Prerequisites: Students should have an understanding of basic programming (Intro to Computational Media or equivalent). Familiarity with physical computing (Intro to Physical Computing or equivalent) is helpful, but not essential. Some, though not all, production work in the class requires programming and possibly physical and electronic construction. There is a significant reading component to this class as well. Possible topics include: * topologies: how to think about them (nodes and links), how few workable ones there are, and how there's no topology so stupid it isn't in use some place. * addressing and routing: what a namespace is, three ways to generate a name (nesting, serial uniqueness, random pseudo- uniqueness), the difference between smart and dumb networks, why the phone network and the internet differ even though they use the same wires * protocols: envelopes and contents, the stack and the reference lie, end-to-end principles, reliability vs. speed tradeoffs * scale: more is different, scale breaks otherwise workable systems, makes redundancy and degeneracy critical, tends to push systems * a discussion of security and its effects Possible exercises include: * Basic socket communication, both software and embedded hardware versions * Client-server programming * A group protocol/messaging exercise * An HTTP/RESTian model exercise

Web browsers were originally used only for displaying simple HTML pages, but over the years they have become supercharged all-powerful web execution machines. In this class we’ll explore experimental new features and HTML5 APIs that allow browsers to communicate with the OS and their environment. APIs that will be covered may include: Battery Status, Geolocation, notifications, accelerometer usage, video access, speech recognition, and text-to-speech. We'll cover the mechanics of bookmarklets and Chrome extensions, with a sustained multi-week focus on building extensions and exploring Chrome's extensions APIs. Class workshops will include projects such as building an ad blocker, programmatically replacing text and images on a website, and making sites that respond to external events. Students will give weekly in-class presentations on web capabilities, complete small weekly assignments, and present a final project. This class leans heavily on web technologies, and experience with HTML, CSS and modern JavaScript (ICM with p5.js or Commlab Web/Networked Media) is required.

Throughout history, as new storytelling mediums have emerged, content has adapted to fit the developing form. From oral narratives to theater, cinema, and television, storytelling will always evolve to fit the possibilities enabled by the platform. Given its interactive nature, digital storytelling is gradually adapting to the medium. So - how is non-linear video shaping the future of digital storytelling? This workshop will combine filmmaking and classic storytelling with narrative gaming structures. The class will introduce the depths of non-linear video and allow students to create their own interactive experience. The focus is on what makes a good story in an interactive narrative environment. Students will have access to the Eko platform - the industry leader in interactive video. In addition, they will be given "backdoor" access to further customize the software for their own projects. During the course of the semester, they will work in teams of 2-3 students to produce a short interactive video experience. Weekly lessons will mimic their project creation process - providing tools and knowledge for creative ideation, scriptwriting, film production, and product integration.

How do we make something from nothing, and nothing from something? The idea of nothing, and optical illusions have been linked since the western discovery of zero lead to the beginning of linear perspective. In this course we will explore an array of optical illusions, ranging from traditional approaches to new technologies. Structured as primarily a studio course, we will work directly with Pepper's Ghost, disappearing acts, making solid objects appear transparent, invisibility, false sense of depth, and approaches to designing negative space.

The most difficult part of prototyping is not the building process, but the process of deciding how to build. If we choose proper technology for prototypes, we can improve their robustness and simplicity. This course will cover available and affordable technologies for ITP students to build prototypes. The course will start with soldering, wiring and LED basics. Then students will design an Arduino compatible board in Eagle, get it fabricated, assembled. And then using the debugger to dig deeper to understand how a microcontroller works. The class will also cover multitasking, signal processing, communication, document writing and advanced skills beyond the Intro to Physical Computing class. Each session will have lectures followed by in-class practices with guidance. The 14-week long assignment is called Do It Once – Do It Again. Bringing an idea or ongoing projects is highly encouraged. This course requires Physical Computing or equivalent experience.

What does it mean to become a “temporary expert?” How does one develop one’s own creative research-based practice? The Temporary Expert identifies problems, challenges and questions as a basis for research and imaginative art/design opportunities. In this course, students will adopt a wide variety of tools and strategies in order to lay the foundations of a research-based art practice that considers materials, media, context, and audience, as well as one’s personal strengths and desires. Students will develop art/design projects that interface with a multiplicity of other disciplines, and engage in idea exchange with experts in the field. A research-based art practice brings together an eccentric mixture of skills, including traditional forms of research (library and interview techniques, informal ethnographies) and experimental hands-on research (experimentation, systems thinking, prototyping, daily practice and user-testing). Through hands-on practice, case studies, and readings on systems thinking, communication, and the idea of “a public,” we will explore method, documentation and presentation of your research, as well as the merits of both success and failure. Weekly work consists of readings, interviews, writing, daily artistic practice and systems thinking exercises

The medium of motion pictures will be will be transformed by virtual reality technologies. But the emerging hybrid form will likely have less to do with the iconic VR headset and immersion, than in the newly possible flow of expression in the other direction, out of the participant. This class looks at the true potential of virtual reality as its mutability, to put ordinary users in the role of director of visual media as they already are in their dreams and fantasies. Democratizing media by breaking it down into discrete more easily remixable parts has historically, from DNA to alphabets to movable type led to an explosion of expression and knowledge creation. Motion pictures, perhaps our most convincing medium, is now undergoing such a transformation thanks to virtual reality technologies. Specifically, we will look at how tools like depth cameras, motion capture, and machine learning can treat a scene as a collection of elements instead of a collection of pixels. We will work with the real-time 3D rendering capabilities available, even in your phone, for the instant contrivance of visual reality using compositing, the transformation of images and models as well as virtual camera moves. The class will also consider how the more conceptual dissection of film by cinema scholars and psychologist into things like plots and character types might be templated into reusable formulas for non-experts to create an emotionally satisfying dramatic arcs.

The ability to digitally fabricate parts and whole pieces directly from our computers or design files used to be an exotic and expensive option not really suitable for student or designer projects, but changes in this field in the past 5 years have brought these capabilities much closer to our means, especially as ITP students. ITP and NYU now offer us access to laser cutting, CNC routing, and 3D stereolithography. In this class we will learn how to design for and operate these machines. Emphasis will be put on designing functional parts that can fit into a larger project or support other components as well as being successful on a conceptual and aesthetic level. In this class we will discover methods to design projects on CAD applications for total control of the result, and we will develop algorithmic ways to create designs from software (Processing) to take advantage of the ability to make parts and projects that are unique, customizable, dependent on external data or random. The class will include 3 assignments to create projects using the three machines (laser, router, 3D) and the opportunity to work on a final project.

Code without content gets boring fast. This seven week course will show you how to create stories around which you can weave the technology learned in other classes. When content comes first, interesting problems arise to solve. Participants will get solid grounding in how to tell a visual story using words and images in a traditional format, so then they can take that format and reimagine it in entirely new and unique ways. The first few classes are devoted to getting basic comic skills. The remaining classes will hone and expand these abilities while posing the question: what can be done differently, and how can technology add to what we have created? At the end of the semester you will have a something that sets you apart; – original content AND technological know how. Students will combine words and images, look at each other's work, look at examples of published works. Reimagine how these stories can be told in new and unique ways. This a demanding course. There is a lot of work involved, they will end up with a lot of original content. During the first half we look at and make traditional comics. Second half we experiment with comic format WHILE honing storytelling skills. Relevant speakers will come in to discuss what they do and how they work etc.

The objective of this course is to explore analog and digital drawing not only as a static exercise, but also as a tool for performance installation and collaboration. The course will explore different methods for expression and capturing output. Examples include drawing under camcorders, digital projection, digital drawing software, and simple code platforms. Students will gain the skill and confidence to draw in real time using a variety of different mediums, improve their improvising skills, and learn to perform without delay.

Why do we care so much about our pop culture obsessions? Why do Doctor Who, Anime, and PBR beer inspire such fanatical devotion? Over the last two decades, the internet has transformed geekiness from an embarrassing mark of stigma into an important focus for creators, marketers, everyday nerds, and a million internet celebs. Fandom is the study of the communities that form around pieces of popular culture, whether based on a shared love of Star Wars, the New York Yankees, Harry Potter, or a niche Java library. Good fans are adoring, evangelical, and useful. Bad fans can be toxic, or even dangerous. From Pokemon to Air Jordans, this class explores the influences and motivations that have led to the current renaissance in fandom. We chart the evolution of fan culture as a social and economic force, from early 15th century religious manias to its present rebirth in the age of digital connectivity. And we discuss issues of tech-assisted fan creation, management, commercialization and the neurological implications of lolcats. This is a class for everyone who wants to inspire millions of fanatical followers to do their bidding... or simply wonders why they still hum the theme to Super Mario Brothers in the shower.

Exhibition design is the art of marrying experience and information. The best do so seamlessly; the very best surprise and delight you along the way. In this class you will explore the craft of interactive exhibition design through practice. Working in small groups, you will select an NYU researcher whose work is of interest to you and create an interactive experience that presents this research to a broader, public audience. In the process, you will learn to interrogate content and form, audience and environment, medium and message to create a meaningful and playful exhibit experience.

You live with illusions. The nature of these illusions has long been described in mystical practices but is now increasing corroborated by modern research such as neuroscience, behavioral economics, social psychology, embodied cognition, and evolutionary psychology. What does this have to do with computational media? With technology, we have the ability to revisit some of these vestigial illusions that made sense in ancient environments but that might limit our personal happiness or the overall functioning of modern society. Will the computer’s ability to run more objective statistical analysis on data gathered tirelessly over time, across individuals and locations allow us to more accurately see ourselves and the world as it is. Can we build computer interfaces that give a fuller expression of our experience when we are not limited by an illusory view of ourselves? The insights into how into how to reach people more fully comes with a responsibility to then ask what should say to them. As the computers are able to understand us better than we understand ourselves will we relinquish control to them? At a practical level, the class looks at interfaces for digitizing signals from the less consciously controlled parts of your body using things like biosensors and cameras. It then moves on to opportunities for finding insights in from patterns in the mounds of already digitized expression you have produced every day for years, for instance in your email. Then we look at the possibility of bypassing consciousness with techniques like subliminal priming to affect behavior. Finally, we will look at how we can use things like browser extensions to manipulate shared media in an attempt to counter some filter bubble effects that emerge at a group level based on the illusions discussed in this class. This class will use skills from Physical Computing and ICM.

Computers are able to understand human speech better than ever before, but voice technology is still mostly used for practical (and boring!) purposes, like playing music, smart home control, or customer service phone trees. What else can we experience in the very weird, yet intuitive act of talking out loud to machines? The goal of this course is to give students the technical ability to imagine and build more creative uses of voice technology. Students will be encouraged to examine and play with the ways in which this emerging field is still broken and strange. We will develop interactions, performances, artworks or apps exploring the unique experience of human and computer conversation. Students will learn how to use text-to-speech and speech-to-text technologies, voice assistant devices, generative text techniques, open speech APIs, Node.js, and conversational UI design. There will be weekly assignments leading up to a final project. ICM or comparable programming experience required.

This class seeks to use the internet and web browsers in new and disruptive ways. Rather than the traditional use of websites as static means of one-to-many communication, we will use websites as stages to perform and intervene in front of a live audience. Students will learn HTML/JavaScript as a means of making interactive websites/instruments to be played for an audience and chrome extensions that will allow us to modify the content of existing websites to political or dramatic ends. We will draw on the art historical traditions of detournement and culture jamming to study what it means to make art out of other material with received authority. The class will also include readings and discussions on digital performing arts, and the implications of digital tools on the aesthetics of theater and performance art. Students will learn HTML and Javascript with a focus on interaction design with themselves or another skilled performer as the intended user, a departure from traditional user experience design. They will read and discuss critical theory and artistic examples of digital performing arts, culture jamming, mashup culture and performance art. Students will create digital instruments and performances and thoughtfully workshop the work of their classmates. Students will perform their work in front of strangers and learn to do so confidently. The first-third of the course will be focused on making single-page instruments, first in p5js and then in HTML/jQuery. The next third of the course will focus on chrome extensions, both applying the same interactive principles onto existing web pages and making site-specific interventions for dramatic effect. In the final third of the class students will create and rehearse a larger more polished performance using the techniques practiced thus far in the course and culminating in a public performance night for the larger ITP community.

This course will focus on the use of voice in live performance. We will discuss the voice as it relates to identity and semantics, study relevant precedents in performance and sound art, explore a range of aesthetic and compositional strategies, and become familiar with microphones and audio editing software. We will also consider the voice conceptually in light of recent technological advances in dictation software and synthesis. Students will complete small weekly assignments which will culminate in a final project. The course will focus primarily on speech, not singing, but we will do some singing and breath control exercises. Trained vocalists and musicians are welcome. No vocal or performance experience is required.

Weather Worlds is a studio course that explores the role that design plays in public engagement on climate disruption. In this outcome-focused course the students will design a popup Weather Worlds studio platform for the 24th Session of the Conference of the Parties to the United Nations Framework Convention on Climate Change (UNFCCC COP24) that will take place in December 2018 in Katowice, Poland. Students will design the format of the engagement in consultation with Red Cross Climate Centre experts, and produce it during the semester. The Weather Worlds studio will be a traveling platform that further can take place in different countries and occasions, serving local participation, and adaptable to local settings and contexts. At the COP24 conference in December 2018 the engagement will be facilitated by the Red Cross facilitators and volunteers, and by the course instructors. The aim for the format of the engagement is to create a blueprint that can be adapted and replayed subsequently in different to create social interventions in various settings. An example of such a flexible format can be a game design session through which participants model the local dynamics surrounding a local context-specific issue. The partnership with Red Cross will exponentially broaden and amplify the impact of this project, facilitating and developing engagements worldwide. Today we rely increasingly on effective engagement of multiple stakeholders in order to impact public opinion, government policy and private sector’s commitments and struggles aimed at climate change mitigation and adaptation. This course produces a platform that will explore the roles of design and art in influencing climate action, and investigate forms of experimental pedagogy, engagement, and participation in engaging issues linked to climate disruption.

This course will go over the field of generative music, from its inception to the current state. The course would focus on both the history and technical implementation of generative music systems. Some topics of discussion will include mechanical and analog processes of algorithmic composition, genetic algorithms and other biological systems, generative grammars and other overlaps with natural language (such as audio Twitter bots), and data sonification. We will work up to what has been done at the intersection of music and machine learning, examining both older practices (such as Markov chains) and current practices (such as clustering algorithms like T-SNE, and deep learning with the work of Google’s Magenta project). This course will include some basic music theory as well as an overview of the various implementation tools. Students will come out of the course with several small generative music projects of their own.

Making words and images public used to be difficult, complex, and expensive. Now it's not. That change, simple but fundamental, is transforming the media landscape. A publisher used to be required if you wanted to put material out into the public sphere; now anyone with a keyboard or a camera can circulate their material globally. This change in the economics of communication has opened the floodgates to a massive increase in the number and variety of participants creating and circulating media. This change, enormous and permanent, is driving several profound effects in the media landscape today. This course covers the transition from a world populated by professional media makers and a silent public to one where anyone who has a phone or a computer can be both producer and consumer. This change, brought about by the technological and economic characteristics of digital data and networks, is upending old industries -- newspapers, music publishing, moviemaking -- faster than new systems can be put in place. The result is chaos and experimentation as new ways of participating in the previously sparse media landscape are appearing everywhere. This course covers the history and economics of the previous media landscape, the design of digital networks that upend those historical systems, and new modes of participation from weblogs and wikis and Twitter to fan fiction and lolcats. The course centers on readings and field observation, with three papers due during the course of the term.

Genspace is collaborating on this course with ITP so that students can learn science literacy through several specialized workshops that will take place Genspace – topics include Biohacking (with an introduction to CRISPR) + Biomaterials. Students will create projects throughout the semester utilizing both Genspace and ITP resources. Additionally, students will learn the basics of biodesign and bioinformatics to help them frame and conceptualize their research and their projects and how best to use these skills ethically and responsibly in aesthetic and scientific ways. Since 2009 Genspace has operated a community biology laboratory in Brooklyn stemming from the hacking, biohacking, and DIYbio movements. It currently supports citizen science and public access to biology, biotechnology, synthetic biology, genetic engineering, citizen science, open source software, open source hardware.

Open Source Studio is a class about maintaining an inclusive, healthy open source project. The first half of the semester will focus on the technical aspects of project management and open source software development (Git, Github, Unit Testing, Continuous Integration). Students will be complete weekly short exercises around contributing to open source projects. The second half of the semester will transition to a “studio” style course. Students will work together and propose a contribution to an open source project or develop their own project. We’ll use a broad definition for “open source” project with an emphasis on documentation and collaboration. Guest speakers will visit the class in person or over skype to talk about their experience maintaining an open source project.

This course will focus on a range of human-centered design research and innovation workshop methodologies including Design Thinking, LEGO Serious Play, Lean UX, Google Ventures Sprints, Gamestorming, Futurecasting, and Service Design. Students will look for design opportunities within the unprecedented challenges that we are currently facing as global citizens. Students will define a problem space based on the drivers that they're most interested in exploring and will have the option to work alone or form small design research teams. They will learn how to conduct primary and secondary research, creating deliverables such as personas, journey maps, concept canvasses, and prototypes. Students will be required to apply design research approaches and workshop methodologies, develop and test a rapid prototype and then share their work in a final presentation.

Small, affordable single board computers enable you to blend the principles of Physical Computing with media playback and capture. This course uses the Raspberry Pi computer as a platform for creating portable devices that have the capability to display graphics, play video, play audio, take photographs, and capture video. As a foundation for the course, students will learn the basic workflow of using the Raspberry Pi computer for physical projects. This foundation includes an gaining an understanding of the Linux software, Python, and digital input and output. Students will work independently or collaboratively to build on this foundation to create their own media playback and capture devices.

Focusing on motion capture (ak. MoCap), this class introduces basic performance skills alongside 3d graphic manipulation to create real-time virtual experiences. In this class we will have the opportunity to build sets, produce interactive props, and design unique characters to tell stories or engage with audiences. Utilizing Optitrack Motion Capture system, Blender, Motionbuilder, and Unreal Gaming Engine; we will create, rig, animate, and perform as avatars.

This course would focus on the questions of 1) what makes people healthy? and 2) how can we design tools and environments that support healthy lifestyles? Key topics to be covered include public health concepts like the multiple determinants of health and the social-ecological framework, plus a little evolutionary biology; the role of behavior in health, key tenets of behavioral economics and behavior change strategies; and systems thinking concepts from Donella Meadows and others. Students will come away with a much more sophisticated understanding of the complex system of factors and forces that affect people's health; understanding of key systems concepts and some techniques for understanding systems; and experience designing for behavior at scale. A potential final project could be to reimagine/redesign a popular commercial service so that it would have a more health-producing impact -- or, alternatively, to focus on designing changes to the ITP environment that would promote better health for students, faculty and staff. About Steve Downs: www.stevedowns.net

This introductory course will focus on giving students a practical understanding of how to make ‘serious’ video games. That is, games that challenge the player to think and learn. This is a class where we will be ideating and producing playable games each week over 7 weeks using the open-source Godot game engine. By the end of the course, you’ll have a small portfolio of ‘serious’ games to show and build upon. Throughout the course, students will become familiar with concepts such as iterative design, play testing, object-oriented programming, user-interface design, animation and basic art for 2D games. They will take a design-based approach to content creation, developing a proficiency in the Godot game engine and the overall indie-game development pipeline.

We experience screens daily in many forms: in our hands, on our desktops, on walls and public installations as we travel. This course will explore the creative possibilities of real-time interactive and reactive art on screens in various forms. Using the recently developed p5VideoKit we will create standalone installations. p5VideoKit is a new library of live video effects - building on p5js - presented as a dashboard for mixing video in the browser. This library allows the user to apply visual effects to live video from connected cameras and sensors or streaming from devices on the internet. p5VideoKit is open source and can be extended with the user’s p5js code for a plethora of visual effects and interactivity. One possible application of p5Videokit would be a public facing installation allowing anonymous people on the street to use their hand held devices to interact with large street facing screens, thereby collaborating on real time creation of “digital graffiti”. Building on ICM, students will learn how to adapt simple sketches into components of p5VideoKit so that algorithms can be quickly composited and orchestrated into more complex works. Students will also learn how to edit and share code beyond the p5js editor, use nodejs/javascript to automate deployment of installations, and remotely configure dedicated computers with long running installations. Several dedicated computers and screens will be available to preview installations on the floor and street facing areas of the 370 Jay Street campus. Prerequisites: ICM or equivalent coding experience. About John Henry Thompson: http://johnhenrythompson.com

Quantum technologies are the next frontier of electronics and computing. Quantum Computers, Quantum Sensors, Quantum Materials are just beginning to emerge from laboratories and enter the realm of practical application. The course introduces the underlying principles of quantum physics, and reviews emerging capabilities of quantum computers and related technologies. Coursework will include programming quantum algorithms on a simulator, and running programs on actual cloud-based quantum computing hardware. The topics will be highly technical and will require the study of scientific principles and experimental apparatus. We will cover some basic mathematical operations in linear algebra, and we’ll be encountering some equations from physics and computer science.

This course focuses on designing, developing and delivering multimedia live performances via a virtual platform. The class will have an emphasis on experimenting with different possibilities of virtual performances, pushing the boundaries of the performative medium, and using emerging technologies to create experiences that allow for the unfolding of engaging narratives, and/or generate compelling visuals in real time. We will look at various examples of both online and offline performances, explore how we can apply the technologies we have learned to design performative systems, and discuss methods we can use to make our performances more engaging. Students will practice quickly coming up with ideas and performing in class. A few weeks into the course, students will propose final project ideas and then develop the performances in the following weeks with support from the instructor. The class will culminate in a virtual event featuring solo and/or group performances by the students.

In an era of remote everything, how can we create artwork that brings us back together IRL? This course explores our connection to physical objects within the context of community. How can sculpture, installation, immersive, and public art nurture our neighborhoods via collaboration, play, ritual, self-expression, and awe? Students will work collaboratively to radically imagine bold, sculptural, immersive works using innovative and lo-if techniques integrated with technology. Hands-on workshops include experiments creating found sculptures, AR prototypes, projection mapping, real-time interactive multimedia content, and more. We'll reference ancient monuments, sacred objects, NYC relics, street art and contemporary works to envision new artifacts that create awareness by reflecting the needs of our communities. Prerequisite: Comm Lab: Hypercinema About Ali Santana: http://www.alisantana.com/bio

The use of digital technology in mental health treatment, recovery, support, and prevention is rapidly gaining acceptance. For instance: The FDA recently approved the VR therapeutic EaseVRx to treat pain. Researchers recently found that exposure to natural environments in VR can provide emotional well-being benefits for people who cannot access the outdoors. Strobing lights can be tuned to stimulate temporary harmonic brain wave patterns usually only found in people who have been meditating for decades. Apps which help you track your mood could facilitate gaining knowledge and awareness of one's mood patterns and thus help maintain emotional well-being. ASMR videos are reported to be effective in inducing sleep for those susceptible to insomnia, and assuaging a range of symptoms, including those associated with depression, anxiety and panic attacks. This class will focus on the use of technology to activate any and all of our senses to aid in mindfulness and meditation, distraction therapy, body awareness and acceptance, and more, via the use of tools and techniques shown to have a direct impact on our physiology as well as supportive and accessible user experience design with broad applications in other areas. Prerequisite: Basic coding and physical computing About Brian Lobser: http://light.clinic

This course provides students an opportunity to sharpen their coding skills in several ways: by reviewing fundamental programming concepts, acquiring techniques to systematically develop code-driven projects, and then implementing those to develop an independent project with the structure and support of a classroom learning community. The first part of the semester consists of weekly exercises to practice strategies for learning new algorithms, writing pseudocode, pair programming, debugging, refactoring, version control, and more. Screen-based code examples for the activities and assignments draw inspiration from the history of creative coding. The second part of the semester shifts to a project development studio format for students to apply these strategies to a self-directed project. This could be an existing idea or one devised during the course. Ultimately this course aims to empower students to reflect on their process and teach themselves how to program with greater efficiency and independence. It is a direct follow-up to Introduction to Computational Media (ICM) or for anyone interested in advancing their coding practice. Examples and exercises will be provided in JavaScript using the p5.js library. However, students are welcome to consult the instructor about working with another programming library, framework, or language with which they have interest or prior experience. Prerequisite: ICM or equivalent experience

By exploring and dissecting the field of STEM education, we will research how STEM education currently exists with clear biases and gatekeeping. Through that we intend to create a framework to challenge the biases and design more inclusive and accessible pathways. As a class we will engage in discussions around spaces (community/public spaces and private spaces), STEM as an inclusive element, and definitions of accessibility. The hope is to yield an experience where students can observe, inspire (or get inspired) by mundane things around their day to day lives and connect them to STEM experiences that might seem rather oblivious. Students will create assignments in dialogue with “making with everyday objects”, STEM pedagogy practice, social/emotional learning in spaces, and human-centered design. Students will be exposed to STEM literacy pedagogy, will curate a pop-up space, practice comprehensive user-testing, and reconstruct the framework around accessible and universal design. Students will engage in critical thinking, critiques, visiting artist lectures, field trips and class discussions. About Sharon De La Cruz: https://www.sharonleedelacruz.com/about-me, https://khushbukshirsagar.weebly.com/about.html

In this special format studio class, students will investigate techniques and frameworks to challenge the socioeconomics of planned obsolescence. We will research, design, and develop projects that rethink our strained relationship with smartphones and re-imagine the future of "old" devices. This is a production-heavy, four-credit course, where students will contribute to original research, and develop projects that combine HCI, design, and critical theory. Prerequisites include an open mind, the drive to make, and graduate-level Physical Computing.

Since the introduction of the iPhone in 2007, the touchscreen has become the dominant manner for navigating Mobile devices. UX pattern best practices are enshrined in documents such as Apple’s ‘HIG’ (Human Interface Guidelines) or Google’s more recent Material Design. ‘No Screens Allowed’ is a class that challenges this ‘Screen First’ approach. In this 14 week class students will prototype solutions in response to Instructor assigned Design Challenges. The various assignments are structured to interrogate mobile device capabilities such as: Networking, Voice Recognition, USB interfaces, Machine Learning, and built in sensors. Students will be provided with identical hardware: Pixel 3 phones running Android, the chosen development platform for the class. Students should have successfully completed Introduction to Computational Media and Introduction to Physical Computing prior to signing up. What questions are we asking? Mobile devices have continued to evolve and expand their sensing and interface capabilities. These features remain for the most part remain ‘hidden’ from most users due to the screen dominant ux patterns. What types of experiences can be designed that make use of these lesser known technologies? How can we expand the student’s prototyping vocabulary? I feel this is an important topic as we move further into the realm of Ubiquitous Computing and away from screen based experiences. The Mobile phone provides an ‘all-in-one’ platform for driving these experiences, serving to jump start the ideation process. Additionally the devices used for the class are older, serving to demonstrate their continued utility. What’s old can be given a new life. The projects that came out of the Spring 2022 semester would be a good example of the types of projects this class would encourage. The goal here is to provide students with a “Protoyper’s toolkit”. A set of code modules that abstract away as many of implementation complexities as possible. These building blocks would be introduced each week, providing students with working examples that address demonstrate the instructor’s answer to the assigned challenge. Students can the code for use in their own projects. Prerequisites: Grad Students: Must have Completed Introduction to Physical Computing and ICM Undergrad 3/4 year students About Brian Jones: https://privatepractice.studio

Lo-fi/High-impact is a hands-on workshop for learning rapid response prototyping and distribution techniques. We’ll look at examples of zine makers and artist and activist collectives, like the Guerrilla Girls, Dysturb, and the Illuminator, who create campaigns and interventions with purposeful urgency, and then flex our own creative muscles for getting ideas out of our heads and into our hands. Leveraging materials within reach, we'll explore three paper-based methods of production: foldable books, enclosures, and wheatpasting. Using the cameo table-top cutting machines, the color and B&W printers on the floor, and materials sourced from our recycling bins, students will have the opportunity to try various techniques and have guided studio time to develop their own creative application/s for what they’ve learned. We will move quickly and get a little messy, and the outcomes will be reflective of the process!

Today’s internet, made up of mostly text documents and two-dimensional images and videos, is the result of historical limitations in bandwidth, graphics processing and input devices. These limitations have made the internet a place where the mind goes, but the body cannot follow. Recent advances in motion capture devices, graphics processing, machine learning, bandwidth and browsers, however, are paving the way for the body to find its place online. Experiments on the Embodied Web will explore the new realm of embodied interactions in the browser across networks. The course will include discussion of influential works in the development of online embodied interaction, including the works of Kit Galloway and Sherrie Rabinowitz, Susan Kozel, and Laurie Anderson. Together we’ll explore pose detection across webRTC peer connections in p5.js and Three.js. Experience with Node, HTML and JavaScript is helpful but not required. ICM level programming experience is required.

As software projects become more complex, it becomes increasingly important to keep the code organized and manageable; otherwise, it becomes extremely difficult to implement new ideas, and the project is much more likely to be prone to mysterious and frustrating bugs. This course will demonstrate several approaches to organizing code for larger-scale projects, including how to write and name functions and classes, DRY (Don't Repeat Yourself), pure functions, unit testing and Test-Driven Development (TDD), and why to avoid "magic numbers" and global variables. The focus of the course will be on JavaScript, using P5.js, but the principles will apply to most languages. We will be doing an ongoing, step-by-step, in-class refactor* of a complex sketch. We will also be using version control to track our changes every class. Students will be expected to complete weekly readings and assignments, and to refactor one of their previous projects, using the principles learned in this course. * Refactoring means rewriting the code, without any changes to how the program behaves. Students should have some programming experience prior to taking this course, and would ideally have an existing software project they would like to develop.

"This course will use the open source The COVID-19 Impact Project as an entry point to explore humanizing data on systemic inequity and injustice on a global and local scale. In this course we will: ● Explore and invent creative uses of data for advocacy and change. ● Discover how data flows from public github repositories and tools needed to visualize the data. ● Review other data-centric open source projects for the public good and discuss the questions they are trying to answer or problems they are trying to solve. ● Examine and draw inspiration from historical and contemporary data visualizations developed by advocates for social justice and the public good. ● Use data visualization as a scaffold to explore ways to support community driven mourning and memorialization after mass death events. Students can choose to participate as creatives, artists, javascript coders, p5js explorers, UI/UX designers, citizen journalists, data science explorers or social justice advocates. Course Outline ● Open Source Projects for the Public Good ● Data: Sourcing, Humanizing and Creating Visual Narratives from Data ● Storytelling with and from Data ● Data storytelling as a scaffold to support grief, ritual and memorialization after mass death events ** Students wishing to pursue their final projects beyond the class will be provided with information about resources at NYU for supporting student projects that amplify underrepresented narratives. ** Students wishing to continue their participation in The COVID-19 Impact Project after the course ends should notify us as we are seeking grant funding to implement viable concepts."

Your web browser is a digital canvas for 21st-century artists. While being one of the most common mediums today, web space has infinite possibilities for new aesthetics. This course covers Three.js fundamentals, providing students with the skills and insights to create arts in web 3D. This course requires ICM or equivalent coding experience.

"This 2-pt course aims to provide students with the critical thinking and practical skills for creating effective and compelling interfaces. We will dissect what a compelling user experience is, apply proven research techniques for approaching and defining UX problems and apply design frameworks including mapping and testing techniques. The class format will include lectures, discussion, in-class design exercises and a final project. Week 1: what is UX Week 2: inclusive research methods Week 3: frameworks for defining a problem Week 4: understanding behavior and motivation Week 5: mapping flow and visual strategies, final project intro Week 6: testing methods and future UX Week 7: final projects"

"In even the tiniest fragment of digital sound (especially music) there lies a multiplicity of information hidden within. Using audio analysis techniques, this data can be distilled into a vast array of characteristics that describe various different features of the sound. These include things like the loudness, pitch, or the spectrum of frequencies being detected. Through additional analysis, these data points can be used to detect higher level musical features representing things like tempo, rhythm, or melody. Furthermore, the sound and music information can be used to train deep learning models that can then make accurate predictions (eg. what a sound is, what genre a song is, what mood a song evokes). Or, we can use machine learning for generative purposes using the data to guide the creation of new sounds, synthesizers, or even entire songs. The preceding are activities that fall under the areas of digital signal processing, music information retrieval, and machine learning, a trifecta that form the technological foundation for the research area known as machine listening. With a focus on ambient sound and music, this class will explore how tools and techniques from the field of machine listening can become a powerful aspect, or even strategy, in the realm of creative applications. This course will not cover, nor will it assume knowledge of, the underlying technical aspects of machine listening, or music theory. Resources for further pursuance of each week’s topics will be provided but will not be required for class. Instead, our aim will be on understanding what these techniques are doing, when and where to apply them, and how to access and apply them effectively through powerful software libraries. This high level approach will allow us to keep our efforts directed towards creative experimentation without becoming bogged down. Ultimately, students will synthesize the semester’s work into their own creative application involving sound. Here are some examples of the types of projects this class could support: An app that visualizes audio through graphics or DMX/LED lighting to create synesthesia-like effects An automatic system for transcribing music based off of a recording or real-time input A music remixing system where tracks are automatically selected, spliced, processed, and rearranged A musical instrument that adapts to its player based on real-time analysis of the played sound A synthesizer that uses machine learning to optimize and tune its parameters A music education software that visualizes rhythm and melody for the purpose of instruction A rhythm game that derives its gameplay from music information (Guitar Hero, Rock Band, DDR) A tool that analyzes the health of a machine based on its sound through a contact microphone The course will be taught in JavaScript with ICM-level programming experience recommended. No formal training in sound or music is expected or required. This course will be a great fit for any student that is interested in sound and wants to explore it more deeply. Please feel free to reach out to me via email with any questions about the class."

"What is the medium of memory? In this 14-week studio class, we will dig into this question through creative storytelling. Starting from a lens-based practice, this class will introduce traditional and bleeding-edge documentary methods to inform our own varied approaches to activating archival material. Through weekly “readings” (articles, podcasts, films), written reflections, and creative assignments, we’ll explore: • how technology has impacted our relationship to memory; • how visual interventions can can surface alternative narratives; • how to make under- and unrecorded histories visible, and call into question the power dynamics embedded in “official” records; and • how we might recast objects and sites of memory-keeping, like heirlooms, journals, and memorials, as a mode of engaged preservation. Mid-way through the course, students will identify either personal or collective histories to open up to their own individual creative reexamination, memorialization, or transformation––each producing a final project with the technology and approaches of their choosing that serves to answer the question we started with––what is the medium of memory?"

Over the past year, the unprecedented advancement in text-to-image artificial intelligence models has sparked widespread attention, discussion, and mainstream adoption of these innovative co-creative interfaces, which has resulted in novelty, excitement, and curiosity, as well as concern, anger, and insult. Alongside this, the booming open-sourced text-to-image model development contributes to expanding access to working with AI tools beyond experts, tech giants, and professional technologists. In this 7-week course, we will go over the landscape of text-to-image AIs and dive deep into some of the most well known ones (such as Stable Diffusion and its variants), to see what potential they have in exploring new modes of content creation and helping us re-examine our language pattern. This will be a technique-driven course – we’ll explore different image synthesis techniques related to text-to-image AIs, use Python to train our own models to create customized visuals, and create animations from text. We’ll also discuss how such tools could intervene in the workflows of artists and technologists, what they can provide for researchers, and what are the caveats and things we should look out for when we’re creating with these AIs.

"How can motion capture (MoCap) be used to archive, preserve, and share intangible heritage forms, such as performing arts, rituals, and other social practices and traditions? This course approaches motion capture through the lens of ethnography -- drawing on techniques of observation, participation, and qualitative design research. This class will offer an overview of different motion capture technologies, such as 2D-3D pose estimation and depth mapping, with a practical focus on learning the OptiTrack system at ITP. We will start by covering the basics of OptiTrack and build up to other workflows and techniques used across animation, game design, and virtual production (e.g. OptiTrack to Unreal Engine or Unity)."

Live streaming is so seamlessly embedded into our online experience. We lay in bed, on our phones watching hearts flicker across the screen as the person we’re watching greets all of the competing messages in the chat, asking for birthday shout outs and follow-backs. While the ability to live stream feels more accessible than ever, it feels very tied to corporate structures, branding and self promotion. How can we push the concept of a live stream in a new direction and rethink what a live stream can be?

On Becoming is a two-part professional development course. Finding Your Artist Voice (part one) filters your fears and apprehensions so you can declare your creative process and practice courageously. The seven-week system will help you proclaim your artistic identity, theoretical underpinnings, and trajectory with clarity, precision, and commanding written language. Students will build personalized masterplans and workflows to facilitate measurable professional growth while learning to catalog and archive their work. Students will develop a working artist biography, artist statement, and fully documented work samples. For the final project, students will be supported in selecting and submitting a post-graduate fellowship, residency, grant, or open call!

Throughout history, musicians have channeled their creativity into outrageous fashion statements and invented personas: think MF DOOM, Sun Ra, Ghostface Killah, Daft Punk, Leikeli47 and Rammellzee. By embracing their alter egos in extreme and outlandish ways, artists have found their authentic creative voices. This course will introduce participants to the art of masquerade using their resourcefulness to create costumes from found materials, and performance as an exploration in creative expression using new media and technology. Students will be introduced to ideas surrounding abstract storytelling, experimental audio + video production, and A/V performance using a combination of technical and hands-on approaches. This course requires CL: Hypercinema or equivalent experience.

To be a VR creator, it's not enough to learn the hard skills—it's also our responsibility to prime ourselves for the human impact of our work. As a means to design VR that is both enjoyable and accountable, this class proposes we borrow design principles from Hedonomics, a branch of ergonomic science that facilitates pleasurable human-technology interaction. Through the Hedonomic Pyramid, we’re able to section our thinking off into regions (Safety, Function, Usability, Pleasure and Individuation) and map out industry-tested VR design guidance for each. The result is a hierarchical checklist of proven principles, specifications and practices—that promote a culture of inclusive and holistic design—built to serve as a quickstart guide to designing accountable VR interfaces and systems. This class, divided into units that represent each level of the Hedonomic pyramid, will unpack both technical and conceptual strategies for creating VR, from visual interface fidelity to avoiding locomotion cybersickness to designing safer social VR spaces.

"In this course, we will explore how to create narratives that leverage our lesser used senses like touch, taste and smell as well as lesser-known ones like space, time, balance and scale. We will dig into the history of experiential storytelling, starting from immersive theater and Smell-O-vision to cutting-edge haptics and mind-bending illusions of proprioception. To help center this back in practical applications, we will also explore how this evolving art is commonly used in exhibition design, experiential marketing and brick and mortar retail. The class will be a healthy mixture of game theory as well as experienced based learning (meaning there will be a couple field trips and multisensory VR projects to explore). A basic knowledge of game engines is ideal but not mandatory because we will be using predesigned templates in Unreal engine to be experienced and manipulated in real-time through virtual reality hardware."

Our users have senses that they use to perceive information in different ways. Some perceive best through sight, some through hearing, others through touch. Designers often prioritize visual information, excluding those who benefit from other sensory modalities. In this class, we’ll take a multisensory approach to design that makes interfaces more accessible to disabled and nondisabled users. Students will learn how to design for the senses (think tactile controls combined with atmospheric sounds and olfactory or taste experiences), while gaining an understanding of the assumptions we make about our users’ sensory preferences. Students should come with prior experience with physical computing and fabrication techniques and can expect to learn technical processes for the user research, usability testing, and iterative design of multisensory interfaces. Over the course of 14 weeks, students will design an interface for the 5 senses (sight, hearing, touch, taste, smell), culminating in one final project that includes at least 3 sensory modalities.

"As game and interaction designers we create systems and choices that can either prey upon our psychological foibles or help us avoid decision pitfalls. It is our responsibility to understand how we decide, to consider the ethics of the systems we create and to practice designing systems in a purposeful manner. Game Design & The Psychology of Choice will provide interaction and game designers with an understanding of the factors that influence behavior and decision-making by looking at the intertwining of cognitive psychology and economics through the development of behavioral economics. These disciplines study behavior on the individual and group level, often revealing some of the why behind the rules of thumb and folk wisdom that game designers come to intuitively. But understanding the why—why we fall into decision traps; why certain tradeoffs tax our brain more than others; why we are overconfident about our abilities; why certain decisions make us uncomfortable—allows us to more purposefully apply our design craft, both in and out of games. Finally, as a class, we will take what we learn about how we think and create series of game experiences based around key cognitive science concepts. Assignments may include: •Mod a cognitive science experiment into a game or experience •Analyze and present a game through the lens of cognitive science and behavioral economics •Create game or experience based around a particular insight from cognitive science or behavioral economics"

This course introduces students to modalities for creating site-specific and immersive art and performance. Assignments will examine the work of artists who challenge the limitations of the physical, psychological and transactional spaces that have come to define conventional production models. Students will regularly receive prompts from which collaborative work will be workshopped, generated and presented. The sites and practices explored will de-center script/text as spine, institutional space as gathering place, linear storytelling as narrative, and separation between audience and artist as social contract. Through group performance projects and presentations, students will investigate how Site evokes Narrative and Event differently in brick & mortar, virtual, historic, liminal, dead, found, contested, democratized and community spaces. Our work will unpack the challenges and opportunities presented when we relinquish creative control of such unfixed elements as serendipity, impermanence, improvisation, audience agency, public space, weather, and pandemic.

For centuries, food production practices such as permaculture fostered ecosystems intended to be sustainable and self-sufficient, while producing nutrient-dense food. Modern farming has introduced harmful monoculture practices proven to cause collateral destruction of biodiversity and seasonal harvesting, distancing us from our food ecosystems. The future of food can be regenerative or continue to contribute to massive health and environmental issues. How can we challenge ourselves to regain connection to our food system? How might we use innovation, personal prowess, design, and biotechnology to reimagine healthier ecosystems? This course examines the historical context of the food ecosystems and encourages students to identify with these systems that we (in urban settings) are disconnected with. Students will build a project around exploring innovative approaches to the future of food and our relationships with it. These projects will incorporate design, technology, science, and research elements.

A programming course where we'll explore various techniques and solutions for tracking and sensing people or objects in space. Students will get familiar with the terminology and algorithms behind many sensing topics such as computer vision, depth cameras, positional tracking, and coordinate mapping. As these subjects are explored, we will also dig into communication, and how this information can be transmitted from one tool to another, for example using OSC, Spout/Syphon, MIDI, DMX/ArtNet. The goal being to use the right tool for the job and not limit ourselves to a particular piece of software.

Decentralization has become a buzzword in the technology space, and there is much more to decentralized technology than NFTs and cryptocurrency. In this course, we will examine the fundamental concepts of the existing internet infrastructure, work to define what decentralization means, learn about the “why” of decentralization, survey the landscape of decentralized, distributed, and p2p protocols, and develop decentralized applications. We learn about will examine the implementation of decentralized technology and throughout the course, we will look at different use cases of decentralization such as evading censorship, protecting privacy, and creating resilient applications. We will also consider ethical questions about the decentralization movement—how will it grow, who benefits from decentralization, and whether a decentralized internet is even a good solution at all. We will examine the underlying technologies that enable decentralization, as well as looking at the current implementations of decentralized protocols and apps built on top of decentralized protocols. Finally, we will touch on adjacent topics such as local networks, mesh networking, and p2p networks. While this course will cover a breadth of decentralized and self-hosted applications, we will steer away from decentralized financing and NFTs and instead focus on decentralized information sharing. The goal of the class is to challenge students to think critically about the future of the decentralized web and develop applications that leverage these technologies. Students with or without a background in networking are both highly encouraged to enroll.

"What capabilities does computational media have for depicting and conveying the experience of our minds? In this course we will start out using 3D graphics to depict the conventional physical reality that appears before us. Then we will turn inward to reflect the multidimensional reality of our minds, using artificial neural networks. Finally we return to embodied interfaces connected with cloud networking and databases to share with other people. The class will operate at a conceptual level, inviting students’ empirical psychological and philosophical investigations of the nature of their experience and how to convey it with art and story. It will ask students to look critically at existing computational media’s tendencies to bore, divide or inflame its users. But this is also very much a coding class where students will prototype their own ideas for new media first with 3D graphics using the threejs library, and then with machine learning models like Stable Diffusion using Huggingface APIs or Colab notebooks and finally with networking and databases using Firebase or P5 Live Media. Students can substitute other coding tools but game engines will not work for this class. The coding is in javascript, with a possible touch of python, and is a natural sequel to Introduction to Computational Media."

How do we make things move, produce sounds, or maybe even emit light without batteries? Through this course, each student will design their own purely mechanical automaton. We will learn how to use simple materials and tools to hand prototype mechanisms in their early stages. CAD software will be used to refine the designs and then a series of traditional and digital fabrication tools (various wood shop tools, laser cutter, CNC, 3D printers, etc.) will be used to produce the final pieces. We will learn how to work iteratively in the shop through weekly exercises, and a midterm and final project.

Course Description: In this 14-week course, students will explore sound design fundamentals through modular synthesizers, leveraging the capabilities of microcontrollers. Modular synthesizers are a type of electronic musical instrument used to generate, manipulate, and shape sound through the interconnection of individual modules, or components. This course is designed to equip students with the skills and creative prowess required to craft their own unique devices that adhere to the Eurorack design format; a popular modular synthesizer standard. The curriculum blends the art of sound design with the technical aspects of hardware synthesizer architecture, building skills so that by the end of this course students will have the competence to bring their sonic visions to life in physical form through thoughtful interaction. By harnessing the modular nature of these components, students will work independently, taking into consideration the designs of their peers to ensure seamless compatibility between their devices, resulting in a distinct ‘voice’; a term used to describe a collection of components that define the signal path of a synthesizer. The first half of the course will focus on sound design coding techniques utilizing the Teensy microcontroller, with the second half dedicated to developing tangible hardware design skills. Prerequisites: Intro to Physical Computing No sound design/musical experience is required.