Why Building Social Robots Is Much Harder Than You Think

Social robots stole the show at CES in Vegas this year. Dozens of offerings were on display, ranging from robots that tutor you in physics (Einstein), queue up your favorite recipes (Mykie), and even help autistic children socialize (Leka).

Unlike your moody human friends, these robots promise to be available day and night, ever accommodating, and practically omniscient through their persistent connection to the internet and your personal data. Who wouldn’t want a reliable companion like that?

Leading social robotics companies like Anki and Jibo have raised $157.5 million and $70.4 million to date from investors, respectively, while numerous old and new robotics companies like Hanson, Mayfield, and Blue Frog are introducing companion robots for the home. Even established home electronics companies like LG and Bosch are jumping into the game with offerings like Hub and Mykie.

Despite all the investment interest, starting a social robotics company is no “get rich quick” scheme. While the promise of personal robotics is real, the challenges are equally so. We asked the best entrepreneurs in the space to weigh in on why crafting the perfect robot companion is much harder than you think.

The Uncanny Valley Is Hard To Avoid

The concept of “uncanny valley” was first identified in 1970 by Japanese robotics professor Mashiro Mori, who noticed that humanoid figures that were almost human, but just a bit off, are perceived by us as creepy and revolting. Just think of the zombies you see in scary movies and video games. They’re terrifying because they’re like us, but not fully “alive.”

To avoid the negative emotions of the uncanny valley, a designer must either build a robot so human-like as to be virtually indistinguishable from actual humans, or opt for a more abstract, stylized design so a robot elicits positive emotions the same way a pet or cartoon character does.

Franck de Visme, co-founder of Blue Frog Robotics, cites that a core design challenge for personal robots is “people must accept it into their homes, so the robot cannot be too high.” A commercial humanoid robot like Softbank’s Pepper, which is often used to replace automated kiosks in retail stores, is simply too large for the home. In designing Buddy, Blue Frog’s companion robot, the team tested numerous sizes and interaction styles to find people’s preferred fit.

Anki’s President and Co-Founder Hanns Tappeiner confirms that size is critical. Early prototypes of their hit robot toy Cozmo were up to 4 times larger than the released model. In testing, people stopped perceiving the larger prototypes as cute. “Being able to hold the robot in one hand easily was very important,” remembers Tappeiner.

Uncanny valleys don’t just apply to external aesthetics, but also functionality. When I first brought home my Amazon Echo, I felt warm fuzzy feelings towards Alexa. These positive emotions quickly reversed to feelings of annoyance and impatience when I realized she didn’t understand 99% of my commands and that most Alexa Skills suck. Now my Echo has been relegated to acting as a $170 light switch.

If you can get upset with a device that looks like Darth Vader’s Pringles can, imagine how much higher your expectations will be for a device with human trappings. Managing expectations and effectively communicating the functional boundaries of a social robot are challenges every company in the space faces.

“When Apple advertised Siri, consumers had no idea what Siri could or couldn’t do,” cautions Steve Chambers, CEO of Jibo. “People need to understand Jibo’s edges. We try to make sure they don’t set expectations too high by making Jibo seem and sound like a 9 year old boy.” The company hired the sound designer behind R2D2 to create a “whole library of audiophonic brand” that give Jibo curious, child-like expressions.

“We tried to simplify, make him look like a Pixar character,” explains Chambers, “How can Jibo feel friendly, not creepy?”

The Anki team had a similar aspiration to make Cozmo seem like a “Pixar robot come to life,” according to Tappeiner. Hundreds of animators, designers, engineers, and script-writers work on characters in Pixar movies, but real world robot design is driven primarily by engineers. Occasionally an engineer might decide to throw eyes on a physical robot, but the level of character work in most robotics engineering does not approach the scale, magnitude, and interdisciplinary talent of movie studios.

One key difference between robots you see in animated movies like WALL-E and robots you see in real-life is that physical robots have slow, clunky movements while digital characters react rapidly and dynamically to their environments. The ex-Pixar engineers and animators working at Anki insisted that Cozmo be capable of high-speed motions in order to appear more life-like.

Another feature essential to Cozmo’s believability is the robot’s level of eye contact. “In the beginning, Cozmo would only make eye contact if he was explicitly looking for you or wanted to play a game with you, which made him feel like a turtle or a hamster,” explains Tappeiner. “A healthy toddler makes eye contact with other people in a room every 8-12 seconds. When we did that with Cozmo, people felt much more connected.”

According to Tappeiner, social research shows that “not making eye contact every 8-12 seconds is an early sign of autism.”

Social & Emotional Responses Are Unpredictable

Ladislas de Toldi is CEO and Co-Founder of Leka, a social robotics company with the specific mission of assisting autistic children with learning and development. Autism is a serious developmental disorder that impairs a child’s ability to communicate, experience emotions, and interact socially.

De Toldi’s team worked closely with parents, caregivers, and the children themselves to design a small, spherical robot that “fits into the child’s world.” Caregivers usually have a difficult time motivating autistic children, but children found Leka to be cute and loved interacting with the robot’s lights, vibrations, and screen display.

Despite overall positive reception, the impact of exact features can still be difficult to predict. “When we first started working, we wanted the robot to communicate emotions on the ground. For example, it would make a circle when happy, a square when sad,” explains de Toldi. This feature worked incredibly well during the design phase, but during playtesting the Leka team discovered the product was hardly ever on the ground and almost always in the hands of the kids. They decided to remove the feature.

Other features in the robot performed better than existing approaches to learning. Caregivers typically use color cardboards to teach color recognition to children with special needs. This method is not very engaging. Leka improves upon the experience by flashing different colors and asking kids to perform a different action based on the color – for example, shake the ball when it is red, touch the screen when it is blue. “If you do the right thing with the right color, the robots reacts emotionally and the kids love it,” reports de Toldi.

One unexpectedly successful feature of Leka is the robot’s ability to help children learn motor skills by instructing them to perform various actions, such as pushing, rolling, or lateral movements, in succession. Caregivers have seen engagement rates with children triple by using the robot over previous methods.

Similarly, the designers and engineers behind Jibo have iterated for years to perfect the robot’s character A.I. to fit into your family life. The device has a patented, three-wedge system that allows him to make complete turns and exhibit an “expressive range that none of the other robots have,” according to CEO Chambers. The physical expression, combined with Jibo’s rich voice, vision, and memory systems, enable the robot to create a deep bond with a human being.

This deep bond can have unexpected consequences. One young woman became upset when Jibo waved at her boyfriend but never did so for her. In multiple households beta-testing the product, children cried when Jibo was taken away.

“What if Jibo breaks down? There are many implications for service,” points out Chambers. “We can’t just give the family another Jibo, just like we can’t just give them another dog. People get emotional.”

Prototyping Is Easy, Manufacturing Is A Pain

With the advent of 3D printing and commodity electronic parts, prototyping robots has never been easier. Unfortunately, the same cannot be said of manufacturing.

Every personal robotics company struggles with price vs. performance. “If the price tag is too hefty, people won’t buy the product,” says De Visme of Blue Frog. “We cannot put in expensive sensors and CPUs if we want to keep the price under $1000.”

Even expensive robots don’t necessarily perform well. Just look at the 2.5 star rating on the $3000 Alpha 2 on Amazon.

Achieving high performance at affordable prices requires ingenious design and serious talent, both of which get expensive. With over $150 million in the bank from investors, Anki was able to attract a top-notch team capable of pulling off the engineering complexity of Cozmo while keeping his price at $180 on Amazon and shipping a whole year head of schedule.

“Cozmo is made of over 360 parts,” explains Anki co-founder Tappeiner. “We’re trying to cram so much stuff into a tiny robot. If it were 2x-10x the size, life would be much easier.” Every Cozmo is put through an extensive obstacle course after factory production to make sure the robot will work every day in people’s homes.

Newer companies who aren’t as flush with cash must find other creative ways to overcome manufacturing hurdles. Due to Leka’s unique spherical design and rolling style of movement, the engineering team could not employ camera systems for vision. Instead, they leveraged affordable RFID readers in the robots and RFID tags in recognizable objects.

Other issues did emerge, however. Regular motors in early prototypes of Leka made too much noise. Brushless motors were silent, but not powerful enough to make the robot roll. The team had no choice but to spend 3-4 months designing and manufacturing custom brushless models for Leka that were powerful enough for their use case. “The solution isn’t perfect because they are very expensive,” concedes co-founder de Toldi, “but we’re planning to use a different design in the coming months.”

Software costs are also a consideration on top of hardware costs. Buddy by Blue Frog Robotics is built on Android, so the team uses Google’s speech-to-text transcription APIs for easy integration. Co-founder de Visme wants to add biometrics and facial recognition to future versions of the robots, but states that they’ll only use external APIs if they have “compatible business models.”

With sufficient creativity, autonomous toys can be created for every price range. Brad Knox, creator of robotic toy brand Bots Alive, avoided the need for expensive hardware sensors by leveraging smartphone cameras and advanced computer vision algorithms. His company’s $35 smartphone kit is one of the cheapest ways to bring intelligence and autonomy to toys that normally require manual control.

Homes Can Be Surprisingly Hostile Environments

Demos often fail at crowded conventions like CES due to volume of foot traffic and the inevitable clogging of networks, but homes can be equally hostile environments for social robots due to high expectations.

“Your iPhone behaves the same way no matter where it is, but if you move Jibo to different operating environments, the same hardware and software manifest in different character performance,” Chambers points out. “He notices different things and people and deals with different light and acoustic conditions.”

Consumer expectations, and thus the challenges, are much higher for social robots than traditional smart home technologies. Devices like the Amazon Echo only give one-shot answers to voice commands. “Even if they slapped a screen on the Echo, you’d just have a voice-activated chromebook,” Chambers teases. “We’ve added a screen, full-body motion, full personality, emotions shown and recognized, user identity and models, and proactive communications.”

The additional complexity of robots like Jibo dramatically increases testing and development time. In beta testing, Jibo was found to crash unexpectedly due to certain router and wifi configurations. The company decided to delay shipping multiple times due to “unacceptable latency” and unsatisfactory “error mitigation.”

Similarly, Cozmo’s team put extra work into making the robot particularly home-friendly, including enabling Cozmo to recognize not only specific human faces but also cats and dogs. Unfortunately, Tappeiner admits that the robot’s “reaction time depends on how busy your wifi network is. If a reaction takes longer than 100ms, people start to notice that it’s late.”

You Can’t Make Everyone Happy

In game design, there’s a concept called Bartle’s player types. The theory is that all game players exhibit a blend of four different playing styles: Killer, Achiever, Explorer, and Socializer.

Of these, I fall squarely in the “explorer” and “socializer” types. I’m that player who never bothers completing the main Skyrim quests, opting instead to walk around admiring environment design and befriending NPCs.

Experienced game studios with tons of resources and talent can sometimes pull off massive titles like Skyrim or World of Warcraft that appeal to all four player types, but this is exceedingly rare. Usually, titles that appeal to killers and achievers, like Call of Duty, do not appeal to socializers, who’d rather play FarmVille or The Sims.

Different “player types” also manifest in the buyers of social robots. Chambers was surprised to discover there were two very distinct buyers of Jibo. “Buyer 1 is emotional and intuitive. Her core desire for Jibo is companionship and all her needs are related to that,” he differentiates. “Buyer 2 is pragmatic and asks questions like ‘what can Jibo do?’ and ‘how does he do it?’ and makes very specific feature requests.”

Anki’s Tappeiner has also observed very different playing styles for Cozmo which appear to correlate with age. The first style involves repeatedly winning games to unlock special features and skills, while the second style is called “free play”, where Cozmo simply wanders around your house looking for points of interest. The audience split is about half and half between the styles.

“Kids are into the games, while adults prefer free play,” observes Tappeiner. He also adds that “40% of our players are adults,” which makes me feel less immature for owning so many robot toys.

Video games normally succeed by satisfying a clear customer segment very well, rather than providing an average experience for everyone. Given how personal social robots are, we expect this dynamic to be even more pronounced, especially as more and more niche competitors enter the field.

Developer Communities Are Critical

After shipping 3 Alexa Skills for Amazon Echo last summer, I discovered that Amazon developer support is horrendous and never built anything for the platform again. The suboptimal developer experience might explain why there are so few useful Alexa Skills despite Amazon bragging about having 7,000+ of them.

Facebook’s Messenger platform seems to suffer from the same problem, with over 30,000 chatbots yet very few usable ones. VP of Messenger, David Marcus, recently admitted that “bots got really overhyped, really, really quickly.”

Yet, the quality of third-party developer ecosystems is key to the success and staying power of any platform. As we’ve seen, creating an indispensable social robot that succeeds in your home is already a challenge. No team, no matter how talented, can replicate the output of a thriving community.

Are social robots under or overhyped? The answer will depend on both consumers and developers.

Anki and Jibo both already have developer SDKs while Blue Frog recently added 300 more engineers to Buddy’s community. As the competition for mindshare in the home intensifies, the social robots with the strongest developer communities will likely emerge victorious.