Mitchell Hall, Room 395 69 Union Street West Queen's University Kingston, ON K7L 3N6
+1 613-533-2995

What Lies Beneath: Material Classification for Autonomous Excavators using Proprioceptive Force Sensing and Machine Learning

Autonomous Excavator

Brief description of research:

The aim of this research is to develop a novel methodology for classifying excavation materials using only proprioceptive sensor data (e.g., forces, motions, vibrations, etc.) that are acquired from robotic excavators during digging and loading. In this preliminary work, we present a proof-of-concept of this methodology for binary classification of rock and gravel materials using only force data. The force signals were acquired from 86 full-scale autonomous excavation trials with a 14-tonne capacity robotic load-haul-dump machine. Various features were extracted from each force signal using basic signal processing techniques. The classification accuracy of this feature set was then evaluated using two supervised and one unsupervised learning algorithm. An average classification accuracy of 90 % was achieved across all three algorithms, which proves the sufficiency of using proprioceptive sensor data for material classification.

Application(s) of this research:

Meaningful knowledge about excavation material classes can be used to improve the autonomous functionality of robotic excavators (e.g., through controller adaptation), as well as provide useful information to downstream processing operations in construction and mining. Some material properties, such as rock sizes, can be determined visually, but these methods cannot see what lies beneath. Lighting and calibration requirements also make vision-based methods impractical for use in many excavation scenarios (e.g., underground).


Excavation materials can be classified into numerous categories (e.g., based on type, mechanics, rock size or fragmentation). In this work, only a binary classification of rock and gravel material types is investigated to test the sufficiency of the proposed material classification methodology. Further development requires realistic data from digging and loading different types of materials, which is always difficult to obtain. We were fortunate to partner with Epiroc, a mining equipment manufacturer located in Örebro, Sweden, to conduct the full-scale autonomous loading experiments for this preliminary work. We hope to continue this partnership for future work in this project. In the meantime, we have prepared a 1-tonne robotic loader at Queen’s University to continue field experiments locally in Kingston.

Related Publications