Remote Assistance for Elderly to Find Hidden Objects in a Kitchen

Remote assistive technologies are one of the most promising solutions for an aging society in the future. This paper describes a design of a remote assistive system to guide elderly to find hidden objects in a kitchen through ubiquitous technologies utilizing sensing and light projection. These technologies can play a vital role in taking care of an elderly by the caregiver’s lives or work far away. The main goal of this research is to provide visual guidance for the ease of elderly to find and identify kitchen objects that are in the closed places like cabinets or drawers. In a standard kitchen where objects can be placed in open and closed spaces, it is difficult for caregivers to find and locate objects that are invisible and sometimes hidden behind other objects. In any situation the caregiver must know where the object resides. In order to help caregivers RFID technology can locate the required object easily and projector can display the image of the object at the required location. In this study, we propose a prototype system developed by combining the RFID system and the projector-camera technology. Additionally, we conducted a user study with twelve elderly people and initial results shows that this system can reduce the burden and increase the efficiency of caregivers.


Introduction
The world population is aging and people aged over 65 are increasing faster in number than any other age group [1]. In general, people of this age group prefer to live at home independently as long as possible and able to age well [2]. In fact, more than 31% people older than 65 or above, live independently within the European Union (EU) [3]. Elderly spent most of the time doing different activities of daily living (ADLs) in the kitchen such as cooking or food preparation [4]. However, aging brings various kinds of motor, cognitive and memory disabilities. Consequently, it affects the sense of autonomy of elderly to do activities in the kitchen independently and increased the burden of the caregivers at the same time. Moreover, a kitchen task comprises of a set of step-by-step instructions and to complete each step requires a different kind of objects. An object can be found in an open and a closed place in a kitchen environment. Ikeda et al [5] and Uranishi et al [6] have developed a kitchen assistive system to support elderly by indicating with light projection visually. These systems support them to find and work with various cooking objects located in an open place such as a tabletop. In this work, we improve the efficiency of the elderly and caregivers to find and locate objects from not only an open place, but also a hidden place such as cabinets or drawers in a kitchen. In general, it takes time to express positions and appearances of objects located in a hidden place by verbal communication. In the case when a remote caregiver is guiding an elderly in an unfamiliar environment, explanations with only verbal communication take caregiver's time and increased burden.
We designed a remote assistive system that can help elderly to find objects from hidden places required during a kitchen task. The system displays visual prompt at the exact location of the kitchen cabinet. To achieve this goal, we utilized Radio frequency identification technology (RFID) and projection technology. Using our system an elderly doesn't have to search all the cabinets and shelves for a required object that can make the kitchen task more efficient and simple. Consequently, the remote caregiver can also easily provide instructions to an elderly at a local site. The remote caregiver doesn't need to remember the location and appearance of the objects while using our system.

Related Work
Recent developments in information and communication technologies (ICTs) can be used to develop smart spaces to assist elderly in their daily activities in known and unknown environments. Also the availability of small, powerful and bright video projectors enables augmentation of real objects with non-invasive displays [7]. Currently, radio frequency identification (RFID) has been identified as emerging wireless technology for object tracking, especially in indoor environments [8]. Moreover, RFID can integrate easily with other indoor positing wireless techniques such as ZigBee, Bluetooth and Wi-Fi provide more reliable and accurate indoor positioning [8]. By combining the RFID and a projector, our remote assistive system can dynamically detect and track objects from a hidden place and the visual image of the object can help to recognize the appearance of an object.
There are several research studies developed for elderly to guide them in various activities such as taking medication, finding objects, keep the inventory of objects using RFID system [9,10,11,12,13]. Concurrently, different research projects used projection system to guide elderly in kitchen environments. Suzuki et al [14] has demonstrated that visual presentation of cooking instruction was more effective than text based instruction using a projector camera system. Ikeda et al [5] has used visual prompts that can display prompts on plane surfaces such as counter tops and door of cabinets using a projector and a camera. Uranishi et al [6] has proposed a grid-pattern indication system for caregivers to directly indicate the target objects on open places using the projector. CounterIntelligence [15] the 'augmented reality kitchen', which illuminates handles of the drawer's to guide users the location of the utensils. In this paper, we combined the RFID system and the light projection system to achieve two problems elderly faced during cooking [16]; one is finding and locating an object and the second is the appearance of the object in order to solve sequencing problems.

Remote Assistive System
We designed a remote assistive system using the projection technology that augments visual prompts in a kitchen environment. The design of the remote assistive system consists of two sites: Local and remote site. An overview of the whole system is illustrated in Figure 1. The system starts working when an elderly in the kitchen environment is looking for an object needed in a kitchen task and the object is hidden in the cabinet. The caregiver at the remote site using an application simply selects the required object from a list of objects. The application retrieves stored objects from the database. At the local site projection system displays the image of the selected object at the exact location. Location of the object is obtained automatically. The elderly picks up the required object easily and he/she doesn't need to search out all the shelves of the cabinet.
The local site consists of a projector, camera, and an RFID system. All these tools are installed at the local site. The camera and projector attach to ceiling and pointing towards the kitchen cabinet. The projector displays the visual information on the cabinet surface and the camera provides the live view to the remote caregiver. RFID system is used to monitor the location of the objects because it's hard to recognize the object's location using a camera. Moreover, the RFID system consists of a reader, an antenna, and tags. Each object in the kitchen is equipped with a passive RFID tag. The RFID tag information helps the caregiver to locate the objects remotely. The remote site contains an interactive graphical interface that shows all required objects to the caregivers that assist them in any kitchen task. This application is connected via the internet to the remote site application. During a kitchen task, remote caregiver provides step-by-step instruction to the elderly using the audio connection. Remote caregiver selects the required object from the remote application and the projection system displays the image of the required object on the cabinet wall. Using this process, the elderly can easily find and locate the required object.

Implementation
Our prototype system consists of a camera (Logitech C210 640 x480 pixels), a projector (Epson H431B LCD 1280 x 768 pixels) and an RFID system (Takaya TR3-LN003FW4-16). We setup the proposed system in a laboratory environment. Figure 2 (a) displays the camera and projector on a pole, while the RFID system is placed inside the cabinet. Figure 2 (b) shows the scene from the local site with the projected image at the exact location of the cabinet. The cabinet with four shelves is shown in Figure 2 (c). The geometry between the camera, the projector, and the door surface must be calibrated in advance. The projector provides the image of the required object at the exact location and RFID reader updates the location of each object. Moreover, the camera was also intended to capture the live scene of the cabinet to facilitate remote caregiver.
The graphical user interface for remote caregiver was developed using OpenCV and MySQL database. The database is used to store the tag information attached to every kitchen object. This tag information will update the location of an object in the database automatically when an object leaves the cabinet and returns to the cabinet. The image of each object has been stored in the database along with an RFID tag. The graphical user interface lists all the stored objects used during a kitchen task. A remote caregiver using this interface selects the required object from the list and gives instruction via audio connection to the local user to pick an object from the required location. Figure 2 (d) shows the interface used by the remote caregiver. After this instruction the system will show the required object at the exact location that makes the object retrieval process straight forward. Figure 2 (e) shows the real scene where a person at the local site is picking an item from the cabinet. We conducted a pilot study with twelve elderly participants to assess the feasibility of the implemented system. All the participants performed a simple kitchen task in a laboratory environment. The task was to find and locate objects from a kitchen cabinet. They received step-by-step instructions from the remote caregiver. Both the participants were not familiar with the kitchen environment. All the elderly participants completed the task easily and efficiently.

Discussion
The system is intended to help elderly living independently to locate and find objects from hidden places in a kitchen. The initial results show that by combining RFID technology along with the projection technology can increase the efficiency and decrease the burden of caregivers taking care of elderly people with different disabilities via internet. As a result of our observation, some limitations were observed such as the registration of a new object when it arrives in the kitchen environment. Currently, we attached an RFID tag to each object and save it to the database before starting the actual system. To make the system more robust in the future, the system can scan all the tags and register the new tag automatically. The camera can take pictures of new objects and the picture along with the new tag can be saved to the database. The projector system can also be used to take the picture of the new object and saved it to the database. In the future, we would like to compare the efficiency of our proposed system with central projection mode and with searchlight [9] system.

Conclusion
In this paper, we have proposed a remote assistive system that aims to guide elderly people to find objects in a hidden place during a kitchen task. In the system, we aimed for a robust and simple design. The proposed method utilizes an RFID system to locate the objects from a hidden place and a projector to display the image of the required object. A prototype of the proposed method has been implemented. Additionally, a user study has been conducted with twelve elderly participants in a laboratory environment to confirm feasibility of the proposed system.