xBerry Case studies Maddie Robot

Maddie Robot

futuristic and independent housekeeper

Watch the video

At xBerry, we keep a close eye on how the world around us is evolving rapidly. Our answer to those changes is a constant quest for future-proof ideas to improve our reality. It was this approach that helped us succeed in the Maddie project – together with an international team, we created a prototype for a conceptual robot providing autonomous assistance for household duties.

Challenge

While working on Maddie’s prototype, we were part of a larger international team with excellent robotics know-how.

Our job was to create advanced software for Maddie to perform mosaic tasks smoothly. We knew that the robot should be tailored to different needs and meet the challenges of various service industries – from elderly care to security and transportation.

Maddie had to be independent of human operators, sensible and sophisticated enough to plan and logically organize her tasks. To make everything happen, our robot needed trailblazing programming for her to recognize objects, map space, remember user choices and preferences, and develop a sense of task organisation.

Goals

To meet all the defined challenges, we set our goals to create the right programming for Maddie Robot.

First of all, we needed to help her navigate based on an analysis of the environment. To provide this, we created expansion algorithms to help our robot create a map of the layout while moving through a given space several times and then analyze it to develop the optimal path for regular scanning. We needed to verify that Maddie can calculate the distance to a specific object and accurately plan her movement.

We also needed to develop and program vacuum cleaning algorithms specially adapted to Maddie’s attributes. Next, we introduced some advanced software that allows the robot to logically plan and prioritize tasks.

We also took care of data processing – for example, to make sure that unnecessary noise would be recognized and removed from the audio recordings made by the robot. We also needed to provide processing point cloud registration and ensure that Maddie enables communication over the cellular network and management from the mobile application.

Solution

We used Docker as a development environment, efficient in testing and monitoring possible errors, and finally a product for docker service.
By using OpenCV/Image processing in the entire project, we ensured fast segmentation of the apartment layout into rooms.
We joined the team after the start of the project, and some of the technologies had been agreed upon in advance. C++ was one of them – nonetheless, it was the best choice to offer maximum performance given the aim of the project.
We relied on ROS (Robot Operating System) to provide services for device and hardware control, core functionality implementation, and package management.

How Maddie Works?

Thanks to software that supports the recognition of objects with 2D and 3D cameras, Maddie logically plans, prioritizes, and executes tasks based primarily on visual information.
We used 3D mashing and mapping to determine the location of a given object in 3D.
We used CNN (Convolutional Neural Networks) for object classification, 2D image processing, and 3D mapping.

This, for example, helped Maddie determine the location of a specific object, like a mobile phone.
We chose SLAM (indoor positioning system) to construct a map of an unknown environment and simultaneously track the robot’s location.

Results

Even though fully functional household robots are still a long way off in the future, Maddie’s prototype success shows us how solutions we come up with today may influence the process of further inventions.

Enabling Maddie to move and perform in a realspace filled with actual objects required the application of several advanced techniques, from complex image and data processing to expansion algorithms that would allow her to analyze the mapped space and come up with optimal paths for movement.

Thanks to software that supports the recognition of objects with 2D and 3D cameras, by moving around the room and performing certain domestic activities, Maddie can plan and execute tasks based primarily on visual information.

The prototype is also compatible with an external mobile app that can schedule duties manually. Maddie’s prototype was presented at the General Electronics fair in the United States.

Real look and name of the robot protected by NDA.