Omotenashi & AI

, Chatani Masayuki, AI

Omotenashi (Hospitality) & AI

This blog is based on my speech at IBM Watson Summit on April 28th, 2017 following to joint press release of Rakuten AI Platform by Rakuten and IBM. (

The Rakuten AI Platform has intended to facilitate various AI applications such as automated chat bot for customer services to be utilized in wide spectrum of Rakuten services including E-commerce, FinTech, Digital content and etc.

I am Masa Chatani, Executive Officer and General Manager, AI Promotion Department and am responsible for accelerating commercial deployment of AI solutions developed by external & internal parties to Rakuten services. Also I am responsible for development and operations of Rakuten AI Platform mentioned above.

From Computer Centric to Human Centric

AI will realize a paradigm shift from “Computer Centric” to “Human Centric.” Today, we, as a user of computer, need to provide info which computer is able to understand. When we try to find something on the internet, it is quite difficult to search the item you cannot recall the name or the keywords to search. It is our ordinary experiences, on the net, to move closer or move away from the item you are looking for through keywords you input. I think AI has huge potentials to solve such troublesome experiences all of us have today.

Human will be responsible for more complex and creative tasks

I think human will be responsible for more complex and creative tasks. Recently we often hear that AI will steal your job in somewhat sensational context. For the domain of tasks computer can do better, we should rely on computer. Human will handle the domain which computers are not good at.

Followings are some examples of experiments at Rakuten.

Three phases of AI Penetration at Rakuten

At Rakuten, our department defined AI penetration phases into following three steps; (Phase 1) Routine task is AI-nized, (Phase 2) Decision making is AI-nized and (Phase 3) Collaboration is AI-nized. Phase 1 is AI-nizing so-called routine tasks and repeated tasks. In this phase, human will take non-routine tasks.

In second phase, AI will take decision making role. Human will take more strategic role in this phase.

In third phase, I envision that once many AI instances are in active, a group of AI will collaboratively & autonomously be interacted and operated. Human takes creative role in this phase.

AI understands your vague intention

AI will understand your intention even if it is a bit vague. Then AI will realize the world that computer approaches human instead that human approaches computer. Undoubtedly this is paradigm shift from “Computer Centric” world of today to “Human Centric” of tomorrow. We have envisioned to realize such paradigm shift and made several experimental prototypes.

Shopping AI

This is a prototype which we call “Shopping AI”. Input here by human is “Organize a birthday party for my 5 years old son in Shibuya next Friday.” This inquiry seems be made by some parent who are looking for his or her son of being 5 years old.

AI will understand and interpret the user’s request of demand level. Then associatively recalling from “birthday party”, AI offers some choices for “restaurant reservation”, “birthday gift for 5 years old boy” and “birthday cake”.

Respective choice of product is realized by connecting to APIs of Rakuten services such as Rakuten Ichiba (E-commerce) and Rakuten Dining based on the result of AI understanding from user’s inquiry.

Finding “More Cute” items

Finding “More Cute” items is challenging task for today’s computer. This looks easy thing but not so easy, isn’t it ? When you are looking for some product on the internet, there are some products which are not easy to describe in text.

Following prototype had applied AI to the products that color is important factor like fashion item.

Fashion Concierge

This is a prototype we call “Fashion Concierge”. After color information is extracted color information from the photos of product information on ladies fashion items, millions of items are placed in color space based on the extracted color information.

When user asked “Indigo” as a name of popular color in jeans & denim shirt, AI will find out the items with color closer to “Indigo” even if the color description is “blue” or “Ao (blue in Japanese)”.

As you see in the video (sorry demo are executed in Japanese), it enabled to find out by the word like “Motto Kawaii (more cute or cuter)” with keeping previous context of what user looked at.

Interacting with customers no matter when and where

One of “Omotenashi” (Hospitality) that automation by AI enables will be interacting with customers no matter when and where.

At Rakuten web services, we offer Q&A for frequently asked questions however it requires to find out answer by customers by themselves.

Following example is based the Rakuten AI Platform we announced in April 26, 2017 as a result of collaborative achievement with IBM.

FAQ Chat bot

This video shows our FAQ Chat bot. As mobile application, communication app Viber is used. Viber is Rakuten’s group company that has 800 million unique ID worldwide.

This FAQ chat bot automatically answers to the question from customer by the interactive & conversational function with IBM Watson from Rakuten AI Platform conntected.

By using AI, Rakuten will guide our customer to the final solution no matter when and where through this kind of interactive & conversational response.

In the post mobile era, IoT & AI will be directly connected

In the post mobile era, I envision the world that IoT or Internet of Things and AI are directly connected. Different from desktop PC and mobile phone, IoT does not have rich user interfaces.

Therefore, IoT will be useless unless cloud side of AI becomes smarter.

Following prototype is one of examples as an interaction with AI in the post mobile era.

AI Janken

This AI Janken (Rock, Paper & Scissors) offers Janken game plays with Cassis, a popular character of Viber, a Rakuten group company.

The input interface is only camera with depth detection. Machine-learned model is created from many images of hand shape of Janken and then AI recognizes the user’s hand and detects win-lose.

Not only ultra-fast recognition of hand gesture, but also almost 100% AI win.

This prototype becomes very popular exhibits at event of Rakuten Travel, was also placed at the entrance hall of Rakuten HQ in Tokyo and was broadcast on the TV program. It made a lot of audience fun.

More things enabled by AI

Internet enabled to shorten time by eliminating our efforts to visit shops and stores located far remotely.

AI will execute various tasks on your behalf concurrently. This is really an impact as if your time in your life is amplified. Amplified your time can be used for more necessary actions and matters. Therefore I believe it will be used for the best “Omotenashi” (Hospitality).

Looking forward to realizing such future, we continue to evolve our Rakuten AI Platform with the improvement of its function and performance. There are still undisclosed on-going projects therefore we are expecting to share it with you in near future.

End of “Omotenashi (Hospitality) and AI”