1. INTRODUCTION

Digitilisation of the manufacturing industries is growing rapidly in what is often referred to as the 4^th Industrial Revolution or Industry 4.0, and in this respect the mining industry is no exception. Massive quantities of data drive these developments. The knowledge derived from these data is obtained by means of machine learning, which has been evolving in the broader field of artificial intelligence, since the mid-20^th century.

Deep learning, a subset of machine learning, has achieved significant breakthroughs in a range of applications in recent years (Krizhevsky et al., 2012; Silver et al., 2017). Unlike many other machine learning methods, deep learning naturally takes advantage of automatically discovering features and patterns from data combined with modelling structures capable of capturing highly complex behaviour.

Broadly speaking, these deep learning architectures can be catergorised as unsupervised, supervised and hybrid methods, as indicated in Fig. 1. Unsupervised methods are typically used for feature extraction and can be combined with supervised methods designed for regression or classification problems to yield hybrid methods. Some examples of these hybrid approaches include pretraining of convolutional neural networks with deep autoencoders (Wiehan et al., 2016) or deep multilayer perceptrons with deep belief networks (Lee et al., 2018).

Convolutional neural networks (CNNs), recurrent neural networks (RNNs) and deep belief networks (DBNs) are used most commonly in the resource industries. CNNs are primarily used the computer-vision related tasks, such as image classification, object detection, semantic segmentation and instance segmentation. RNNs, including long short term memory networks (LSTMa) are sequence modelling

techniques, where the network retains past information and combine them with new input data to make predictions.

Fig.1. Deep learning architectures (DBN – deep belief network, DAE – deep autoencoder, RBM – restricted Boltzmann machine, DNN – deep neural network, CNN – convolutional neural network)

One major drawback of deep learning is that satisfactory performance of the network ideally requires the availability of massive amounts of training data. However, this problem can be partly circumvented by use of transfer learning techniques. Pretrained networks by themselves have been shown to work exceptionally well on data sets from new domains (e.g. Fu and Aldrich, 2018a, Bewley and Upcroft, 2016) and may not require large amounts of data to be optimised (e.g. Fu and Aldrich, 2019).

The purpose of this paper is to provide a review of emerging applications of deep learning in mining and metallurgical operations.

Copyright lies with the authors                                                       12101

 

Preprints of the 21st IFAC World Congress (Virtual)

Berlin, Germany, July 12-17, 2020

2. OPERATION AND PRODUCTION

The sequence of operations required to extract valuable materials in mining operations typically comprise drilling, blasting, hauling, processing and transport of the beneficiated products or metals.

2.1 Drilling

Recent developments have focused on exploiting data obtained from operational drilling to get improved estimates of the characteristics of the ore body. For example, He et al. (2019) have used a CNN to estimate the unconfined compressive strength, cohesion and internal friction angle of rock.

Several studies have been conducted in petroleum engineering on the use of machine learning in the analysis of ‘measurement-while-drilling’ (MWD) data, e.g. Klyuchnikov et al. (2019) and Flegner et al. (2019).

Bai et al. (2019) have shown highly significant improvement over competitive methods by using a convolutional long short term memory fully connected deep neural network to process raw signals directly for the detection of external events in optical fibres.

2.2. Blasting

Blasting is the major rock breakage method for most mines, involving the release of massive explosive energy. A poor blasting design can cause many adverse effects such as ground and air vibration, back-break, and flyrocks (Guo et al., 2019). These effects are related to many controllable variables (e.g. burden, powder factor, amount of charged explosive, etc.) and uncontrollable variables, such as the nature of the rock mass being blasted.

Mining operations rely heavily on the experience and empirical judgement of site operators and engineers, but the development of sensor technology and big data analytics is enabling engineers to make more efficient, cost-effective or safer plans that are supported by data.

For example, Guo et al. (2019) have proposed a deep belief network (Hinton et al., 2006) to obtain an accurate estimate of the distance of flyrocks. In another study, Nguyen et al. (2018) have conducted a comparative study of three types of neural networks on the prediction of blast-induced air-blast overpressure in an open pit mine.

In underground mines, poorly-controlled blasting can cause severe mine disasters, such as rock bursts and roof caving. In order to monitor these microseismic events, Huang et al. (2018) have developed a CNN model to predict the time delay of arrival and to identify the source location of microseismic events based on the recorded seismic waves.

Pu et al. (2019) have concluded that deep learning could overcome some of the deficiencies in traditional data driven approaches, as more data become available in the near future.

2.3. Haulage

Autonomous haulage vehicles used in mining operations have been developed by several manufacturers over the last decade (Pradip et al., 2019). Regulations pertaining to mining

vehicles, and the traffic conditions are simpler than for private vehicles operated on public roads.

As a consequence, the operation of self-driving haulage trucks, loaders, dozers and excavators (Siau et al., 2018) has become well-established over the last few years. In the Pilbara region in Western Australia, large fleets of autonomous haulage vehicles are steadily replacing manually driven vehicles. Likewise, the use of unmanned aerial vehicles (UAV) or drones have also become routine.

However, there is still a strong need for the improvement of technologies, such as localization and navigation, vision-based sensing, proximity detection and collision avoidance. Dadhich et al. (2016) have highlighted the key challenges in the automation of earth-moving machines and discussed the potential use of reinforcement learning for intelligent automatic control.

The most popular vision-based localization method is called the simultaneous localization and mapping (SLAM) algorithm (Mur-Artal et al., 2015). However, this algorithm does not function well in underground environments, where there is dust, varying light conditions, and other ambiguous circumstances (Jacobson et al., 2018).

Sünderhauf et al. (2015) have developed a CNN-based robotic recognition system that allows for real-time place recognition by applying specialized binary hashing on the CNN features in a surface mining environment. Zeng et al. (2017; 2019) have further designed and improved this CNN-based localization system for vehicles in the underground environment.

Despite the wide use of autonomous vehicles on mine sites, a human presence is still required in many applications for safety reasons (Bewley and Upcroft, 2016). For example, small rock fragments may trigger false alarms for autonomous trucks and force the trucks to stop. Alternatively, unreliable or late detection of human and other vehicles can lead to severe collision.

Bewley and Upcroft (2016) proposed a CNN based object recognition system to detect light vehicles and personnel in an active open-pit mine site environment, and the model has significantly reduced the false positive rate. Moreover, a similar network architecture was used by Somua-Gyimah (2018) for terrain recognition and object detection in an open-pit mining environment and further for the purpose of collision avoidance.

2.4. Mineral Processing

Important areas of application include ore and rock characterization, milling circuits, and froth flotation. The application of deep learning in these areas have started to receive increasing attention in the wider literature, as discussed in more detail below.

2.4.1. Ore feed characterisation and ore sorting

Automatic identification of rock and ore properties is an important component for many mineral processing tasks, such as ore sorting and classification, mineral content recognition, and ore property estimation. This is therefore an

12102

 

Preprints of the 21st IFAC World Congress (Virtual)

Berlin, Germany, July 12-17, 2020

area likely seeing rapid growth in mineral processing operations.

The distribution of particle sizes has a major impact on downstream processing. Conventional methods such as watershed segmentation are usually contrast sensitive, as separation requires a high contrast between different phases. It therefore tends to fail when segmenting particles with presence of minerals with similar colours in images (Karimpouli and Tahmasebi, 2019b).

Moreover, traditional approaches are generally time-consuming and require considerable effort in manual adjustment of parameters for contrast adjustment and noise reduction (Shu et al., 2018).

To deal with the difficulties of these conventional methods, SegNet (Badrinarayanan et al., 2017), a convolutional autoencoder network, was used to segment digital rock images. A data augmentation technique, namely hybrid pattern and pixel-based simulation (HYPPS), was used to generate sufficient images for training the network (Karimpouli and Tahmasebi, 2019a). In this approach, the CNN performs pixel-to-pixel labelling (segmentation), i.e. every pixel in the original image is classified as either ‘particle’ or ‘background’.

However, the potential application of CNNs extends beyond binary classification tasks (Liang et al., 2019), such as ore sorting (Karimpouli and Tahmasebi, 2019b), as some network architectures, such as SegNet (Badrinarayanan et al., 2017), UNet (Ronneberger et al., 2015) and LinkNet (Chaurasia and Culurciello, 2017), are designed for semantic segmentation.

With this novel approach, the networks can segment images with multi-categorical objects. For example, in the context of ore and rock characterization, an image may consist of particles of different metal grades (e.g. waste and ore), as well as different mineral grains. An effective way to identify and count these particles can be beneficial to many aspects in mining operations (Hong et al., 2017) and civil industry (Iglesias et al., 2019).

Characterisation of the complex flow of granular solids from hoppers, bins and silos is still an open research issue and Aldrich & Olivier (2019) have recently made use of a convolutional neural network to extract features from mass flow measurements to enable better identification of avalanching phenomena in the flow.

2.4.2. Comminution

Grinding and comminution circuits are critical components in processing plants, as they produce the desirable size of ore particles, and an inefficient circuit design results in large energy consumption. These circuits are notoriously difficult to control and deep learning may lead to more advanced systems.

Since deep learning is superior in capturing the highly non-linear relationship from complex data, several studies have appeared in the literature, e.g., missing data imputation using variational autoencoders (VAEs) (McCoy et al., 2018), and estimation of mill load levels using soft sensor data

modelling with CNN (Wei et al., 2015). Baek and Choi (2019) have trained a five-hidden layer perceptron model to predict ore production and crusher utilization.

The operating states of grinding circuits are monitored from multivariate time-series signals (van Duijvenbode and Buxton, 2019) as they are or transformed to 2D images using distance matrices method (Bardinas et al., 2018), followed by feature learning using CNNs.

One of the emerging trends further driving the application of deep learning models in grinding circuits is the deployment of novel sensors generating large amounts of data. These include MillSlicer^TM (https://processiq.com.au/products/

milling-flotation-instrumentation/millslicer/) and SensoMag^TM (Clermont & de Haas, 2010).

2.4.3. Flotation

Deep learning has focused mostly on the development of more reliable sensor systems for online grade and reagent estimation in flotation, specifically based on froth image analysis. Traditionally, froth image analysis has focused on three related problems, namely the (i) recognition of changes in operation conditions from the appearance of the froth, often in combination with other variables, the (ii) estimation of bubble size distributions, as well as (iii) online estimators of grade or other chemical species being floated. Any of a number of methods can be used, often highly effectively to deal with (i), but (ii) and (iii) are more challenging.

As far as (i) is concerned, Li et al. (2019) have made use of a pretrained CNN to extract features from an antimony froth that could then be used as inputs to a classifier to identify aberrant froth conditions, but the specific advantages of this approach is not clear.

Little has been done as far as (ii) is concerned, although Stone Three in South Africa has patented an approach to estimate bubble sizes from froth images, based on the use of CNNs. This is an area where there is considerable scope for improvement over traditional methods and more work in this area is currently underway based on architectures and variants thereof similar to U-net (Ronneberger et al., 2015).

Regarding (iii), in comparatively analyses with other multivariate image methods, Horn et al. (2017), Fu and Aldrich, (2018a; 2018b; 2019) and Zhang et al. (2020) have shown remarkable advantages in accuracy to be gained from using CNNs in froth image analysis. This is a major step forward towards the online implementation of image sensors in advanced online control systems in flotation.

3. DISCUSSION

While a variety of deep learning architectures are currently being used in the mining and metallurgical industries, convolutional neural networks have seen most use by a large margin to date, as indicated in Fig. 2.

This is related to the current focus on sensor data analytics, particularly image-based sensors used in the characterisation of particulate feeds, drone inspection systems, and also the processing of hyperspectral images and multivariate time series data, as indicated in section 2.

12103

 

Preprints of the 21st IFAC World Congress (Virtual)

Berlin, Germany, July 12-17, 2020

Fig. 2. Application of deep learning architectures in mining, namely convolutional neural networks (CNN), long short term memory recurrent networks (LSTM), deep belief networks (DBN) and deep reinforcement learning (DRL).

Not all studies of deep learning in the mining industry are reported in the academic literature. Local and global competitions associated with the solution of real mining industrial challenges by use of data analytics, such as Kaggle competition (https://www.kaggle.com), and Unearthed competition (https://unearthed.solutions), are receiving increasing attention from developers and data scientists worldwide.

Mining and other related companies release datasets with some specific targets to achieve, and good solutions require considerable effort. It is noteworthy that the deep learning methods have taken the top prize in many of these practical challenges. This includes the detection of rock bridges at crushers⁽¹⁾, the detection and tracking of missing bucket teeth in large excavators⁽²⁾, and ore sorting using digital images⁽³⁾.

⁽¹⁾https://unearthed.solutions/u/challenge/reduce-impact-rock-bridges-our-telfer-crusher

⁽²⁾https://unearthed.solutions/u/challenge/ground-engaging-tools-get-detect-and-track-missing-bucket-teeth

⁽³⁾https://unearthed.solutions/u/submissions/cnn-transfer-learning-applied-rock-sorting-0

The availability of large amounts of data is key to the development and deployment of deep learning systems in the industry. Although rapid development in sensor systems have led to the collection and storage of such data, these raw data are not necessarily useful for model development.

For example, in flotation systems, while easy to collect prodigious numbers of images, these images also need to be labelled, which could be a major bottleneck in the development of the sensors.

Finally, most of the deep learning technology being applied in the mining and metallurgical industries are still being developed outside the mining industry. For example, while watershed algorithms are used in a large variety of contexts, including the mining industry, more advanced versions of these algorithms, or deep watershed algorithms (Couprie et al., 2014; Bai & Urtasun, 2016), are currently being developed. These algorithms is one example of algorithms

that would likely also have direct benefits in mining industry applications.

4. FUTURE DEVELOPMENTS

While digitilisation of the mining industry is advancing in many ways, the industry tends to adopt technology, rather than leading in its development. Insight into future development can therefore be gained from recent developments in manufacturing. In this area, some novel machine learning methodologies closely tied to deep learning are emerging. This includes deep reinforcement learning and adversarial learning.

Deep reinforcement learning facilitates complex decision making with obvious applications in advanced control, but also beyond this to potentially higher levels of intelligence that could benefit haulage vehicle fleet management or plant-wide control in mineral processing (Shipman and Coetzee, 2019. A prerequisite for this is the collection of sufficient data in real-world settings beyond simulated environments, which remains a significant challenge (Aggarwal, 2018).

Unlike deep reinforcement learning, which can be seen as an extension of an existing approach, adversarial learning (Cheng and Yu, 2019) improves learning efficiency through construction of a generator and a discriminator. By so doing, it is not necessary to specify a reward or loss function of the system. This enables learning tasks that were not previously possible, and has seen applications in image synthesis, monitoring and pattern recognition that would directly impact mining and automation.

5. CONCLUSIONS

Deep learning is rapidly making inroads in mining and mineral operations, where it is leading to more reliable sensor systems. The online characterization of particulate feeds in particular, is set to make a broad impact on operations.

Convolutional neural networks are the most popular architecture in use by a large margin, owing to their versatility and ability to deal with image data.

The development of digital twins of process systems and operations, as well as the deployment of novel big data sensors continue to drive the development and application of to deep learning.