Michelle Zimmerman explores how artificial intelligence (AI) can facilitate communication and storytelling, and, thus, learning
LEILA: Why isn’t it talking?
CUB: Is it a good robot or a bad robot?
The children inched toward the robot and proceeded to inspect its wires, screws, arms, the wall plug, and the wheels that functioned as its feet. They wanted interaction. They wanted to know if it was friendly. Would it scare them, or would it shake hands with them? (Zimmerman, 2018)
Communicating with Technology
This excerpt with Cub and Leila, at ages 5 and 6, is from the new book, Teaching AI: Exploring New Frontiers for Learning (http://www.iste.org/TeachAI). Both children have practice communicating together and with other people around them. They hoped a robot would respond in the same way. Instead, when they tried to communicate, the robot stared blankly back at them. That kind of response didn’t seem very friendly to them, calling into question the moral compass of the robot. They knew both good and bad robots existed from stories with WALL-E and Eva, C-3PO and R2-D2.
Whether they realized it or not, asking each other those basic questions demonstrated a social process of learning and social uses for language. It demonstrated a command of language more sophisticated than a machine is currently capable of. Jared Zimmerman, design lead at Google, described the current level of artificial intelligence (AI) as a machine that is capable to doing “tasks a child would do, but quicker and in parallel. Basically, it gives you tens of thousands of six-year-olds doing something over and over again. Very specialized six-year-olds. It’s as if you can only train each six-year-old to do one thing” (Zimmerman, 2018, p. 22).
Cub and Leila’s questions demonstrated more than one thing. They showed transfer of learning, emotion, an understanding of context, and prior knowledge from storytelling. Trying to read the emotional state of the robot to determine if it was good or bad was part of attempting to understand communication. AI has still not been able to reproduce the social, emotional, historical, contextual, and symbolic understandings that humans have as they communicate. Machines learn from very large amounts of data, most often in a very different way than humans approach learning. However, Language Magazine recently published about a model that hopes to have a robot use perception to learn language similar to the way children learn by interacting with the environment. AI can learn from itself or other external sources. The difference in the way a machine learns typically leads to different outcomes and challenges in creating AI that interacts with humans in a way that feels natural to the humans that interact with it.
This excerpt with Cub and Leila, at ages 5 and 6, is from the new book, Teaching AI: Exploring New Frontiers for Learning (http://www.iste.org/TeachAI). Both children have practice communicating together and with other people around them. They hoped a robot would respond in the same way. Instead, when they tried to communicate, the robot stared blankly back at them. That kind of response didn’t seem very friendly to them, calling into question the moral compass of the robot. They knew both good and bad robots existed from stories with WALL-E and Eva, C-3PO and R2-D2.
Whether they realized it or not, asking each other those basic questions demonstrated a social process of learning and social uses for language. It demonstrated a command of language more sophisticated than a machine is currently capable of. Jared Zimmerman, design lead at Google, described the current level of artificial intelligence (AI) as a machine that is capable to doing “tasks a child would do, but quicker and in parallel. Basically, it gives you tens of thousands of six-year-olds doing something over and over again. Very specialized six-year-olds. It’s as if you can only train each six-year-old to do one thing” (Zimmerman, 2018, p. 22).
Cub and Leila’s questions demonstrated more than one thing. They showed transfer of learning, emotion, an understanding of context, and prior knowledge from storytelling. Trying to read the emotional state of the robot to determine if it was good or bad was part of attempting to understand communication. AI has still not been able to reproduce the social, emotional, historical, contextual, and symbolic understandings that humans have as they communicate. Machines learn from very large amounts of data, most often in a very different way than humans approach learning. However, Language Magazine recently published about a model that hopes to have a robot use perception to learn language similar to the way children learn by interacting with the environment. AI can learn from itself or other external sources. The difference in the way a machine learns typically leads to different outcomes and challenges in creating AI that interacts with humans in a way that feels natural to the humans that interact with it.
Human-Computer Interaction
Human-Computer interaction is on the rise. While storytelling has portrayed machines and humans interacting in science fiction for a while, these images and stories may lead adults and children to assume robots are at a level of sophistication with artificial intelligence where they should be able to interact and communicate with us at a nearly human level. We are not there yet, although there are daily advances.
The rapid advancement has inspired educators to begin teaching coding fundamentals and basics of coding languages to children as young as four and five to prepare them for the future. The hope is to help young people better transition to fields where they can help develop AI. In addition, another way of thinking is to build learning experiences that are more personalized and adaptive where machines support human learning with AI built into the tools. There are new uses of artificial intelligence tools like natural language processing, computer vision, and machine learning in tools that support reading comprehension, improve accessibility for English Language Learners (ELLs) or students with dyslexia, and assist students using Sign Language or learning world languages.
Ways of Thinking of Language Learning with AI
When thinking about how AI can support language and language learning, there are different approaches to consider. Teaching children to memorize processes with basic coding would be parallel to learning a vocabulary.
That is one step to take in preparing children for a future where they design AI experiences for themselves or others. It is not the same as learning to communicate with others in that language where vocabulary is only part of the communication. Computational thinking goes beyond being able to replicate a process for basic code. Computational thinking helps prepare students for creating and designing solutions and solving unknowns.
Another approach is to teach with tools that are supported by AI already. This can help support educators in creating a more adaptive or personalized learning experience for students. Changing education paradigms to prepare a mindset and way of thinking is another important way to help prepare young people for a world where AI is becoming more prevalent. Language will continue to be relevant in the development and use of AI. Here are some examples:
How to teach to prepare students to design AI solutions in the future
Teaching coding languages
Identifying problems and supporting students to practice designing solutions
How to teach with tools that use AI to support adaptive and personalized learning
IBM Watson and Sesame Workshop (http://www.sesameworkshop.org/) using AI to support early literacy and vocabulary development for English Language Learners (ELL) and students just learning to read (Harris, 2017).
Mia Learning’s voice chatbot (http://www.mialearning.com/) built with machine learning to support elementary and middle school students in their independent reading.
Microsoft Learning Tools and Immersive Reader have a wide variety of use cases, from learning to read for the first time to supporting struggling readers, English Language Learners and students with Dyslexia.
Microsoft “Accessibility Tools: Meeting the Needs of Diverse Learners” explores features that are already built into Office 365 tools, which use natural language processing and machine learning to support accessibility for people with dyslexia and vision impairment, as well as young students who need reading assistance.
Microsoft Translator (https://translator.microsoft.com/) uses natural language processing, part of artificial intelligence, to break language barriers.
Motion Savvy has UNI, a two-way communication software, in pilot phases now. UNI uses motion capture to convert American Sign Language into speech and text for students who are deaf or hard of hearing.
Read Robert Szczerba’s “Four Game-Changing Technologies for the Deaf and Hard of Hearing.”
How to teach in a way that supports knowing how to use AI
effectively now, and in the future
Train students to excel in skills machines cannot
Teach students to recognize AI in action and use cases
If machines can also be trained to learn, memorize, and repeat, then students need to excel in areas machines cannot. When students learn to teach, they are not only reinforcing language and communication skills, they are also building social and emotional capacity. Young people that learn to teach other humans first will also be better positioned to understand how learning is a social process. Knowing how to teach humans to learn will help them to suggest more authentic ways for machines to learn. This may help lead to a more natural human-computer interaction in the future.
The Human Side of Technology and Communication
Human Learning as a Social and Cultural Process
Language is more than memorizing words and turning them into sentences. Our brains have the capacity to develop empathy and relationships, transfer learning across multiple settings, remain flexible as we adapt to unexpected responses, and use storytelling to help support learning and remembering. At Renton Prep (https://www.rentonprep.org/), we are fortunate to have a Computer Science TEALS teacher (https://www.tealsk12.org/) with a background in linguistics and computer science who draws on multiple domains and varied experiences in the way he approaches teaching computer science. Chris Nearman spoke with me on the intersection of computer science, linguistics, artificial intelligence, and education:
“As we automate existing processes, humans need to adjust what they do to be able to react in ways that didn’t exist previously. It is an inevitability that the problems of tomorrow haven’t been even thought of yet, and building machines that can perform this kind of ‘thinking’ are ways we can continue to build on the education and knowledge of people previous to us.”
From observing young children like Cub and Leila who want a robot to interact with them as a human would, we need to creatively identify ways that allow for increased automation and different types of interaction with technology, while helping support what makes us human. An example of this kind of creativity using technology that supports humanity, language, and communication is seen in a 9th grade Ethnography project by Afomeya Hailu. Afomeya’s parents are from Ethiopia.
Language is one way to capture history, tradition, culture, emotion, and present a rich heritage to others. Because Afomeya speaks English, she is able to tell the story to an audience outside of Ethiopia. Among the range of technological tools she used, she compiled the story in Sway, a free tool built with machine learning, a component of AI. The foundations of AI in Sway support the design layout and research within the tool, so the user can focus on creating content. In this way, Afomeya could spend more time on the language of developing the story and communicating generational first-person perspective to share with her peers across cultures from her ancestry though Ethiopia. Preparing students for a future with AI will mean highly collaborative teams and a greater understanding of global perspectives. AI can assist with communication, language translation with Natural Language Processing, and facilitating the complexity of the process by doing the heavy lifting of designing a presentation to look polished. This next example demonstrates a focus on humanity though culture, art, and language, but supported by AI powered technologies, Sway and Microsoft Translator. In cases like these, we want the AI to fade into the background and let the communication and language become the primary focus.
Communicating though Art and Language
Last fall, lawmakers from Japan (http://www.rentonreporter.com/news/delegates-from-japan-shown-nontraditional-learning-at-renton-prep/) visited our school, Renton Prep, to experience the intersection of humanity and creativity, and the technology that supported such learning. They were interested in bringing back ideas for teaching creative processes in Japanese schools. We presented gifts of artwork that our students created as a way of communicating what our students learned about Japanese culture though research at the Seattle Art Museum and online. We wanted to balance technology and in-person experiences for learning about culture.
Students documented their creative process and used AI tools to support their construction from documentation to communication, then used natural language processing (NLP) to help make their thinking visible to others in Japan. NLP is still developing and is not perfect; native Japanese speakers suggested the translation is about 70% accurate at this point. Sway, a digital storytelling app which is part of Microsoft Office and designed for creating presentations, is supported by machine learning for searching creative commons licensed resources. It also allows students to focus on content, while machine learning suggests a layout design that looks polished at any point. Using Microsoft Translator, students included another level of machine learning as a way to communicate inside of Sway. See an 8th grade student example by Charisse Vales (http://www.sway.office.com/uox9YPUJ0mQx6Vwj).
As I continued talking with Chris Nearman about language and culture, he pointed out that language is a social tool:
“In and of itself, it’s a contract we’ve defined between ourselves and others as to which sounds carry what meanings. But of course, as cultural artifact it carries much more than that. The language you speak and how you speak it can often be a signifier of group identity. The stories that are communicated in your language help build your relationship to the group. Language doesn’t exist in a vacuum, it’s a living, breathing set of regulations that we’ve imposed on ourselves, but since we’re often masters of our own, we also know how to break out of them.”
Personalization with AI
Unlike more traditional approaches to education, personalized learning focuses on the human element of learning—the unique needs and differences of students as they explore the depths of their own curiosity and how they communicate with others. As we work to develop the capabilities in today’s young people that will distinguish them from ever-smarter machines, personalized learning offers a pathway for allowing vital skills like creativity and critical thinking to flourish though language and other modes of communication. AI tools help expand the teacher’s capacity to allow students to explore different avenues of inquiry, while gleaning valuable insights about how each child learns.
Students Learning to Teach with Technology
Unless people decide to enter a teaching or training profession, rarely do they have the opportunity to experience what it is like to shift from the role of a learner to the role of teaching someone else in a formal learning environment. Most often, students have the experience of being the one trained—the learner who memorizes, understands, and repeats. If machines can also be trained to learn, memorize, and repeat, then students need to expand their capacity beyond a machine’s capacity by learning to teach as well as to learn. This will be a necessary skill in communication for fields that need humans who know how to communicate with and teach machines. To help machines respond more like humans and interact with them in a more human way, humans need to first understand how to teach other humans. The next example demonstrates how middle school students learned to teach younger students. Camille Mercado used technology to support her as she taught an English Language Learner. This removed the learning from a setting where students complete an assignment for the benefit of the teacher, to having an authentic reason to practice communication and language skills—to help someone learn while developing relationships, empathy, and transferring learning across multiple settings. This project created the foundation for further projects that utilized AI in the process of creating story.
A Student Mentor: Camille’s Story
Camille used video captured on her laptop, audio, and digital ink with her little buddy for conversational English, pre-literacy, and practice in written communication. Tracy, at age three, was still working on mastering her first language at home, Vietnamese, and her second language in school, English. Without the footage Camille recorded on her computer, Tracy’s teacher would not have known how much skill Tracy gained through the mentorship.
Using technology to support older students mentoring ELL students benefits both students in the partnership. Camille, now 20 and planning to study Informatics at the University of Washington, contributed to the book, Teaching AI: Exploring New Frontiers for Learning and reflected on her experience as a middle school student: At the time, I never really understood the importance of how my class would spend a year being mentors to preschoolers. I just thought of it as a way that preschoolers can have the ability to become friendlier with older kids or adults; but it became way more than that.
Devoting my time that summer to review the information I gathered with my preschooler, Tracy, made me understand the importance of technology alongside its ability to improve social interactions and learning with children (Zimmerman, 2018, p. 91). The full research is published in Springer’s Human-Computer Interaction Series (Zimmerman, 2016), and told in context of how technology can support humanity and language learning in Teaching AI: Exploring New Frontiers for Learning (Zimmerman, 2018).
This became the foundation for future literacy projects including The Little Book Project, led by educator, Kayla Vrudny. Visit https://sway.office.com/dTo7zMhK6w1Br2Cb for an example of student work where humanity was at the center of learning and literacy, but the language and communication were supported by AI.
We know that communication is a complex process and technology is becoming more sophisticated in its ability to support language learning, acquisition, and communication. From observing young children like Cub and Leila who want a robot to interact with them as a human to a high school student like Afomeya who harnesses technology to support storytelling and culture, technology can help us highlight the beauty of language and facilitate communication.
As educators, we know how powerful storytelling can be in the learning process. Storytelling and language can help us remember. It can become part of our culture (Nasir, Rosebery, Warren, & Lee, 2006) and tradition, as various cultures express wisdom through oral and written stories (Choi, 2015; Faggella, 2015; Tillman, 2006). To tell stories is human. AI can support us in telling, reading, creating, and understanding these stories.
References
Choi, A. S. (2015, March 17). How Stories Are Told Around the World. We Humans. Retrieved May 5, 2018, from http:// https://ideas.ted.com/how-stories-are-told-around-the-world
Faggella, D. (2015, September 8). Finding Artificial Intelligence Through Storytelling—An Interview with Dr. Roger Schank. Lifeboat Foundation Safeguarding Humanity. Retrieved May 5, 2018, from https://lifeboat.com/blog/2015/09/finding-artificial-intelligence-through-storytelling-an-interview-with-dr-roger-schank
Harris, R. (2017, June 6). IBM Watson and Sesame Workshop Launches AI Vocabulary Learning App. Retrieved May 6, 2018, from https://appdevelopermagazine.com/5263/2017/6/6/ ibm-watson-and-sesame-workshop-launches-ai-vocabulary-learning-app-
Nasir, N. S., Rosebery, A., Warren, B., & Lee, C. D. (2006). Learning as a Cultural Process. In K. R. Sawyer (Ed.). The Cambridge Handbook of the Learning Sciences, 489–504.
Tillman, L. C. (2006). Researching and Writing from an African-American Perspective: Reflective Notes on Three Research Studies. International Journal of Qualitative Studies in Education, 19(3), 265–287.
Zimmerman, M. (2016). An Aqua Squiggle and Giggles: Pre-teens as Researchers Influencing Little Lives Through Inking and Touch Devices. Hammond, T. Valentine, S. Adler, A. (Eds.). Revolutionizing Education with Digital Ink: The Impact of Pen and Touch Technology on Education. New York, NY: Springer International Publishing. DOI 10.1007/978-3-319-31193-7_17
Zimmerman, M. (2018). Teaching AI: exploring new frontiers for learning. Portland, Oregon: International Society for Technology in Education.
Michelle Zimmerman, author of Teaching AI: Exploring New Frontiers for Learning, has worked in multiple areas of education, from apprenticeship and co-teaching to research, leadership, and educator training, and she has taught all ages from 3 through 16. She received her Ph.D. in Learning Sciences and Human Development from the University of Washington College of Education in Seattle, and her research on learning design has been recognized with multiple awards. At Renton Prep Christian School, she has put her research into practice with blended, authentic learning environments, in which students learn to become mentor teachers and researchers. Under her leadership, Renton Prep has been selected by Microsoft to be a Showcase School since 2015, and in September 2018, was selected as one of 17 schools around the world to be part of the Microsoft Flagship Schools program and the only K-12 school in the United States to be selected. Since 2007 she has presented around the world, including at the American Education Research Association, New York Academy of Sciences and New York Academy of Medicine, NYU Polytechnic School of engineering, UCLA CRESST, CARNET in Croatia, DigiPen, ISTE, BETT London, and Global Educator Exchange Leadership Summit in Singapore, as well as educational technology conferences, and multiple universities. Her students have been presenting and co-authoring with her since 2011. She has been invited to add perspective to global leadership events at the headquarters of Microsoft, Google, and T-Mobile. Michelle wrote this article to address the guiding principles outlined in: https://www.languagemagazine.com/2018/11/06/edtech-for-english-learners/
