MSc Robotics Final Project by Johanna Amunyela

  using the example instruction found on the  https://towardsdatascience.com/audio-deep-learning-made-simple-sound-classification-step-by-step-cebc936bbe5  website.  We will start with sound files, convert them into spectrograms, input them into a CNN plus Linear Classifier model, and produce predictions about the class to which the sound belongs.

Investigating using the EKF  The  Extended Kalman Filter  (EKF) has received abundant attention with the growing demands for robotic localization. The EKF algorithm is more realistic in non-linear systems, which has an autonomous white noise in both the system and the estimation model. Also, in the field of engineering, most systems are non-linear.  Reference video description :   This video composes three synchronized views at the beginning and also shows the 3D reconstruction of the structure at the end. The top left corner shows the real operation of the robot moving around the structure gathering the 3D scans. The bottom left corner shows the generated map with each scan painted with a different color. There it can be seen how the point clouds are added to the map with the estimation of the robot position (i.e. with the pose predicted by the EKF) producing inconsistencies and that, after the registration results are incorporated to the EKF the map consiste...

The new drilling process will contain three Machine Learning Models. These models work together to make the mining decisions required to mine accurately and efficiently. Improvements  The plan is to replace the current Sonar with a solid-state lidar sensor that produces point cloud data. The Current R2 Sonic 2024 Sonar produces real-time results but has too much acoustic fluctuation and might benefit from a stabilising approach with Graph Convolutional Neural Network   references  The finding from this Underwater mobile mapping video is very inciteful. comparing the traditional acoustic sonar approach, with the point cloud lidar approach.

Observations I notice the model is working well but not performing as intended so I will look at the data and hyperparameters to see what is going on. I will also remove the winch data. This is because the tag/sensor data for the vessel witches are not as consistent and it mostly flagged 0 which means no movement on the winches and this is not helpful. In the near future, when the data is more accurate the winch tag/sensor data will help clearly train for pre-move. Pre-move  is when the vessel moves while the drill is down to save time moving to the next hole.

After waiting some time, I received the 3D data for the panel that was mined on the days I selected to train the model against. I intend to train two models, one with geological 3d data and the other with drill bit data. The plan is that they will work together to achieve the desired outcome or to find an algorithm that will consider both data types and produce the binary classification with a distance in meters of how much to pick up the drill bit by. This will ultimately save time because you will not need to pull up the bit to the default 8 meters knowing the geology or lithology of the sea floor. Research into GCN (Graph Convolutional Networks ) I have also found this video that speaks about GCN. The content encourages using dynamic graph CNN for learning on point clouds, in our case the .xyz file. it also advised considering the points in a point cloud data as nodes in a directed graph. these models help learn from non- euclidean data like graphs and 3d objects using a different ...

I manage to install the drill visualisation software and received a bit of training on how to navigate it. The ultimate objective of the drill visualisation is to use the prediction data from the flask API to visualise the drill up and down composition in replay mode. this is purely experimental and might not yield the expected result since we haven't used 3d data this far when building the model.

 Looking at the data the previous model was not looking at the correct fields. after some investigation, I found 56 rows of data in the 460085 rows of data that did not have the binary value, causing errors and incorrect values.  below are the Sklearn model. I made a comparison between 3 algorithms  Decision Tree Classifier 99.5 % Logistic Regression  96.5% KNeighbors Classifier 98.2% The Model that performed better was the Decision tree classifier see image below:

With the model built, I started to look into tools and ways to perform inference regarding the model working on unseen data and hopefully making comparisons of the performance. Firstly I save the model on the PC and install Flask. with the code from https://www.datacamp.com/tutorial/machine-learning-models-api-python , i created the api.py. I am currently busy installing the Postman Agent so i use it to call the API and perform inference.   

Improvements  From the last model, I noticed that I was focusing on the wrong field. the binary classification was being implemented on the last field, so I move the composition tag data to the last field and the model gave me the training results below: I am happy with the 100% Accuracy but I'm scared it might be close to overfitting, so I will perform a few more tests to see the results. Additionally, I would like to do a bit more investigation with the regression algorithm.  but with the regression model, I need geological data to see if the lithology has outcrops or is flat. Then the model can know by how many meters to pull up the drill tool bit.

To improve the current mining rate I chose to train the model with data from the best performing days. I choose to looked at data from three vessels and see how the binary classification result look. below is the raw data from the on of the vessel. Here I am doing a comparison of the budget vs the target. Vessel 1  Vessel 2 Vessel 3 The objective of this exercise is to look at what the budget vs actual values are and see if we can get the model to perform similar or better and hopefully stablelize the performance of the mining rate.  meeting one of our initial goals which are : Overall objectives:   • Maximising throughput  • Process stability  • Increased mining rate stability  • Decreasing equipment wear • Improving energy efficiency • Increase delivery on targets   

  The Gantt Chart can be found on the link below  Update Project Plan- Gantt Chart

Observation while building the model  I noticed that the data I am working with is labeled data, with features and a label outcome. This means I have to look at it from a prediction perspective. prediction can be a classification or a regression problem. In my case, the outcome has only two class labels. 0 or 1 meaning the drill bit is down for mining or up to move to the next mining ground. Hence binary classification. Overall there are obvious relationships and some that are not. Regression will work best in advancing to the output label being a fluctuating value between an upper threshold and lower threshold value. For the data, I initially loaded the data and use scatter matrix and histogram functions from pandas and matplotlib libraries to try and see if there are more relations or something new. To teach the model the relationship between the feature and the outcome, I used several algorithms and compared them to see which one performed best in this scenario. see code below:...

I have drafted a flow chart of the old drilling process.  I am currently finalizing the proposed drilling process and state machine flow chat to illustrate how the model drill bit will mine below:

Random Probable simulation options i found interesting to watch 

Error Log  SQL errors encountered  Challenges  The data I need sits on a server that is on a vessel out at sea x using line of site and the vessel is about +-9000 km from me . The connection is lagging causing the response time to be slow. To over come this I started working to refine the sql query with picking a smaller range for the data  With a 1 hour  and the top 1000 records filter. Once I was satisfied with the query I remove the records filter and started adjusting the time filter.  For now I am going to try and work on a day worth of data and see if I can find new relationships and or anomaly.  The days worth of data posed another challenge I couldn't copy it of the server into the excel sheet as I did with the smaller data set , so I exported the file as a csv file and push the file of the server. Conversion failed when converting date and/or time from character string. solution : I removed the 00 to match the data format  The conversion ...

 

  Observable variables (inputs ) Name                                    Range                           Availability           potential importance  Airflow                      2000 - 2600,  preferred airflow is at 2200 CFM/KPA  air pressure            7 to 10 bars          water depth              shallow 70-90 meters   7- 8                                           optimum 90-130  9 compressors               ...